Describe what you want to a coding agent; get a validated, loaded, callable countinghouse module back — with the design approved before any file exists.
Authoring a module means making api.json, schema.json and a handlers/
tree agree with each other. Getting that wrong used to be quiet: the module
loaded, nothing appeared in tools/list, and the log said nothing useful.
Finding out took a running framework, which took Redis.
This toolchain removes both problems. Validation runs standalone in milliseconds, and the four MCP tools below let an agent drive the whole loop without a human editing JSON.
The tempting design is a tool that reads a natural-language requirement and returns a module. It is the wrong one: such a tool needs a model behind it, while the thing calling it is already a model — one holding the user’s codebase and the conversation. You would be putting a smaller, blinder model inside a tool call, where nobody can see its reasoning.
The agent thinks; countinghouse supplies ground truth the agent cannot derive. Decomposing a requirement into services and actions happens in the agent’s conversation with the user, where the user can correct it. What this toolchain contributes is knowledge (a skill) and checks (tools that answer precisely and cheaply).
npx countinghouse-validate ./my-module
Checks api.json, schema.json and the handler map against each other, and
prints every problem it finds — each naming the stage, what was found, and
the way out — rather than stopping at the first. No Redis, no server.
Exit codes are the contract: 0 clean, 1 problems found, 2 path unusable.
--json emits the result as one JSON document for machine consumers.
--verifyModule still exists for checking a module inside a running
framework; it needs Redis and a booted server, which is why the CLI leads.
Start the runtime with --authoringTools and call it with an admin key. Four
tools appear:
| Tool | Purpose |
|---|---|
countinghouse_validate_plan |
Check a proposed device/services/actions split before any file exists — names, duplicates, missing descriptions, collisions with tools already on this runtime, and an optional calls array (see Composing below). Returns the tool names the plan would produce. |
countinghouse_validate_module |
The CLI’s check over MCP. Returns {ok, module, problems}. |
countinghouse_load_module |
Load a module from a local path into the running runtime. Returns the tool names it made callable, plus discoveryComplete. |
countinghouse_call_tool |
Invoke a tool by name — so a just-loaded module can be called without waiting for the client to refresh its tool list. |
countinghouse_reload_auth |
Re-read the auth configuration without a restart, the same as POST /reload-auth. |
countinghouse_validate_plan.package.json, api.json, schema.json, handlers/.countinghouse_validate_module, until ok: true. It returns every problem
at once; do not fix them one per run.countinghouse_load_module. If discoveryComplete is false, toolNames
may be incomplete and the call is worth retrying.countinghouse_call_tool with one of those names, and real arguments.The skill that drives this ships in the repo at
.claude/skills/countinghouse-module/SKILL.md,
so pointing a coding agent at a clone is enough — nothing to install.
Its first rule is the one worth repeating here: decide what a tool is allowed to return before deciding what it does. The output schema is the guarantee. A tool whose schema has no field that can hold a file cannot leak a file, whatever its handler does or a model asks of it.
countinghouse.callsA module that needs to call another module by name declares what it may call
in its own package.json, not api.json — the device spec format did
not change for this at all:
{
"countinghouse": {
"calls": ["repo-scan/scanService.scan", "secret-detect/detectService.detect"]
}
}
Each entry is an address, <module>/<service>.<action> — the target’s
friendlyName, the raw last segment of its service URN, and its action
name. Not the target’s MCP tool name (repo_scan_scanservice_scan): that’s
deduped and load-order-dependent, so it cannot be hardcoded reliably. The
handler then calls it with ctx.call('repo-scan/scanService.scan', input) —
see module-development.md for
the full mechanism.
The identity is not yours to declare. A module says what it calls; it
never says who it calls as. Which auth identity a module’s inner hops are
authorized as is bound separately, after the module is written, by the
operator listing the module’s name (its package.json name, which the
server loads it under) in that identity’s runsModules entry in the auth
config, or with the sqlite CLI’s bind-module. That split is deliberate: the same
module can be deployed twice, under two different identities with two
different grants, without editing it.
Both validators know about calls, at different depths. They cannot
substitute for each other:
countinghouse_validate_plan accepts an optional calls array in the same
shape, checked against whatever targets are already loaded on this
runtime — useful for approving a chain’s shape before any file exists.countinghouse_validate_module (and the CLI it wraps) checks package.json’s
real countinghouse.calls for shape and duplicates, with no server needed
— but it cannot know whether a named module, service or action actually
exists, only that the address is well-formed.DeviceManager’s composition
verification, which runs once discovery finishes). A module can validate
clean and still fail at load time if no identity’s runsModules claims it,
or if the bound identity lacks a grant to a declared target. Load it (step
5 of the loop) and check the server log before declaring the chain done.A handler may put one of its action’s declared questions to the user
(ctx.ask, module development).
The handler does not wait: with no answer yet it ends, the client shows the
form, and the retry runs it again from the top. So the rule is one line,
ask first, act after, and it is stated plainly because it costs money
and can do damage:
countinghouse_validate_module does not analyse.
Review it.Over composition a part may ask too: its question goes to the user of the
call that reached it, under the listed scenario’s name (the part is never
named to the client). The scenario’s ctx.call then rejects with
ASK_PENDING; let it pass, and on the retry both run again. The rule holds
for the whole chain: a scenario that calls a part that asks should call it
before anything it cannot do twice.
The user reads a part’s question under the scenario’s name. So a scenario
vouches for its parts’ declared questions: review a part’s questions when
you review the scenario.
package.json countinghouse.role is "scenario" or "part". Without a role,
a module is listed as it always was.
--loadModule <scenario> also loads
every module named before the slash in its countinghouse.calls. Each one
is found from the scenario’s own directory, under its bare name
(ch-step-redact) or under any npm scope (@countinghouse/ch-step-redact),
and must be the module whose api.json has that friendlyName. Parts
that call parts load the same way, and each module loads once. A module
given directly with --loadModule is listed whatever the order. Parts are
expanded at startup only: a scenario loaded later at runtime brings no parts.tools/call or
invoke-action from outside answers MODULE_NOT_LISTED, while its
scenario’s ctx.call reaches it as before. A part loaded directly with
--loadModule is listed, so it can be exercised alone.ci_failure_triage). Every other module keeps <device>_<service>_<action>.PART_MISSING) or when a bare name is already a listed tool’s, a runtime
built-in’s or an earlier scenario’s (TOOL_NAME_TAKEN). The log says which. Other modules load
normally.PART_VERSION_CONFLICT).--store local without
--authConfigPath) lists only the scenarios’ tools. The balance check and
the authoring tools are not listed, and a call to them by name answers
MODULE_NOT_LISTED. Direct peer channels are off there. An explicit
--directPeerChannels is refused at start (exit 2, “choose one”), because
a peer hop carries no usage call id: its server calls would go unrecorded,
and the receipt would say less was read than was.Every tools/call on a listed module is counted at the runtime’s boundary.
inlineBytesOut),
and the bytes behind its resource links (linkedBytesOut).A scenario’s result also carries a receipt. _meta['countinghouse/receipt']
holds {calls, linesIn, bytesIn, linesOut, bytesOut, servers}, and a second
text block holds its sentence, for example read 48,210 lines (3.1 MB) in 14
calls to github; 35 lines (2.4 KB) reached the model.. structuredContent is
your output exactly as validated, except that an image or audio value is
replaced by its reference (see module-development), so a closed output schema
stays the guarantee. A scenario’s top-level output field named receipt is refused
RECEIPT_RESERVED, because there the name is the runtime’s. Any other module’s
receipt field (a payments receipt, say) passes through untouched: the
runtime adds no receipt to it.
A task-augmented call (tools/call with task) is counted the same way: the
run’s server calls under its own call id, and its result when the task
finishes. tasks/result carries the same receipt.
The call id that ties a call’s server calls to it belongs to the runtime. A
hop that states one its worker was not given is refused with
USAGE_CALL_FOREIGN, and nothing is forwarded or recorded. A hop made under a
call that has already ended (its run timed out) goes on without the id, so
nothing more is recorded under it. A part makes its ctx.call and remote
calls before it answers: a call made after the part answered is refused
USAGE_CALL_FOREIGN, because the call is no longer the part’s.
Two limits, stated rather than enforced:
countinghouse_usage_report, listed to every key, reports the same counts by
tool, server, pack or module. A key sees its own calls; an admin sees every
caller by opaque id. since and until set the window (since: "all": from
the first record; with no since, a plain key’s window is the last 31 days
and an admin’s is everything). The headline is the period’s totals, with the
tokens kept out of the model’s context: an upper-bound estimate, {estimate,
method, bound: "upper"}, per tool call max(0, bytes its server calls read -
bytes its result returned) / 4, rounded per tool call so that the day rows
add up to the total. It is an upper bound because it takes the model to have
read, without the tool, every byte the tool read. Below it come one row
per day with activity (at most 366) and a profile: call and server-call time,
errors by code, and server calls per call.
tools/list does not update by itselfThe server is already correct — tools/list is recomputed from live device
specs on every request, so a loaded module appears on the next call. What is
missing is the notifications/tools/list_changed push, and sending one would
require a server→client channel that stateless Streamable HTTP does not have.
Advertising the capability without being able to deliver it would be a lie in
capabilities.
So the loop routes around it: load_module returns the names it made
callable, and call_tool invokes them.
Two independent gates: --authoringTools (default off) AND an admin key.
Neither alone is sufficient, and the flag is checked before any identity work,
so admin-ness cannot substitute for it.
Default-off matters because load_module plus call_tool is arbitrary code
execution with a friendly name. It must not be one flag-flip away on a
deployment that merely happens to have an admin key configured.
With the flag off, these tools are byte-identical to tools that do not exist — same JSON-RPC envelope, same error code — so a caller cannot detect that the feature is present. That property is structural: the disabled response is produced by the same line of code as a genuinely unknown tool, not by a second call site kept in sync by hand.
countinghouse_validate_module runs the module under test in a spawned
child process, so a crash or a process.exit() in caller-supplied code
kills the child, not the gateway — and a re-validated module is never served
from a stale require cache.
This is a local development flow: an agent holding an admin key against a runtime the user started. It is not a multi-tenant feature.
load_module loads into the runtime and cannot be isolated. A module
whose entry calls process.exit() will take the server down when running
without --workerThread. Use --workerThread when authoring — the
worker dies instead, and the call returns a reported failure after a bounded
15s wait. Running validate_module first (step 4) catches this case anyway.discoveryComplete: false means the post-load discovery wait ran out;
toolNames may be incomplete. Retry rather than treating it as final.validate_plan is deliberately provisional. It catches a subset of what
validate_module catches, earlier. If agents skip it and go straight to
writing files, it should be deleted rather than defended.module-development.md — the module format itselfcomposite-tools.md — modules that call other modulescli-and-api-reference.md — flags and endpointssecurity-model.md — what worker isolation does and does not provide