harnessie.com / docs / Threat model
Threat-model comparison: Harnessie versus prevailing harness patterns
Harnessie's headline claim is that it is the safest first agentic harness for people. This document is the artifact that claim points at, and it is written to be falsifiable. Every row below names a failure mode common to prevailing agent harnesses, states what Harnessie does instead, and cites both the code that enforces the difference and the test that proves the code does what the row says. If a citation does not hold, the row is wrong, and the claim is weaker than stated. That is the intended way to read it: as a set of checkable assertions, not a marketing table.
Scope: this compares structural and mechanical properties, the things a role prompt cannot opt out of. Prompt-quality differences are out of scope because they are not falsifiable in code. The honest residual (what none of these layers catch) is stated at the end rather than hidden.
The four patterns the industry defaults to, and where they fail, are the spine: unsandboxed shell, prompt-level-only guardrails, self-verification, and silent dissent-merging. The remaining rows extend the same discipline to cost, secrets, approval, ownership, consent, and audit.
Comparison
| Prevailing failure mode | What prevailing harnesses do | What Harnessie does instead | Enforcing code | Proving test |
|---|---|---|---|---|
| Unsandboxed shell: an allowlisted interpreter writes anywhere or opens a socket by hiding the target inside the code string | Run python/bash with the parent's privileges and filesystem | Every child command is wrapped in an OS confinement: writes confined to the workspace subtree, network denied by default, fails closed on a host with no usable backend | harness/sandbox.py | tests/test_sandbox.py::test_worker_python_cannot_write_outside_workspace, ::test_worker_network_denied_by_default, ::test_run_shell_fails_closed_without_backend |
| Prompt-level-only guardrails: limits stated in the system prompt that a clever instruction can talk the model out of | Put all boundaries in the prompt and hope the model holds them | Boundaries are machine-owned text appended after the prompt file (the author cannot remove them) and role permissions are re-checked at dispatch, not just filtered from the schema | harness/roles.py (DEFAULT_BOUNDARIES, RoleDef.system_prompt), harness/tools/registry.py (dispatch role check) | tests/test_repo_configs.py::test_role_without_grant_is_denied, ::test_orchestrator_has_no_side_effecting_tools |
| Self-verification: the agent that did the work also grades it, so a plausible-but-wrong result passes | Ask the same model, in the same context, if its answer is good | A separate verifier runs in a fresh context, sees only artifacts and acceptance criteria (never the worker's transcript), and fails closed when evidence is missing | harness/verify.py (VerificationGate.run), harness/adversarial.py (position isolation) | tests/test_routing_verify.py::test_gate_deterministic_check_failure_blocks_verifier, tests/test_adversarial.py::test_position_agents_are_read_only |
| Silent dissent-merging: disagreeing agents are averaged into a false consensus with no human in the loop | Blend or majority-vote model outputs into one answer | A contested phase halts to needs_arbitration on any dissent; only a human may author the arbitration; agreement is treated as evidence, never authority | harness/adversarial.py | tests/test_adversarial.py::test_dissent_halts_needs_arbitration, ::test_any_oppose_blocks_convergence, ::test_human_arbitration_resumes_phase_to_passed |
| Prompt injection via ingested content: a scraped page or ticket carries text that hijacks the agent | Feed tool output straight into the model as if it were trusted | Quarantined reads are scanned at dispatch, hidden Unicode stripped, untrusted text fenced as data, and the loop re-asserts the boundary exactly where the flagged content landed | harness/quarantine.py, harness/loop.py | tests/test_quarantine.py::test_read_file_quarantine_at_registry, ::test_loop_tripwire_reasserts_boundary |
| Secret exfiltration: an injected command reads an API key and surfaces it into the transcript or deliverable | Inherit the parent environment and return raw output | Child processes run under a scrubbed environment (no inherited keys), shell output is redacted before return, and write_file refuses credential-shaped content | harness/tools/builtin.py (scrubbed_env, run_shell redaction, write_file guard), harness/quarantine.py (redact_secrets, find_secrets) | tests/test_quarantine.py::test_run_shell_env_is_scrubbed, ::test_run_shell_output_redacts_secrets, ::test_write_file_blocks_credential_shaped_content |
| Silent overspend: a runaway loop burns real money with no ceiling and no warning | Run until done, bill whatever it costs | Budgets are hard ceilings, and a live run refuses to start with no ceiling set, after a preview showing the worst-case dollar figure | harness/routing.py (Budget), harness/preflight.py | tests/test_routing_verify.py::test_budget_accounting, tests/test_preflight.py::test_live_run_without_ceiling_refuses |
| Approval bypass: a privileged or irreversible action runs silently when approval wiring is missing | Default to allow, or skip approval when unattended | Approval-gated tools fail closed under the default handler, so a misconfigured headless run refuses rather than mutating | harness/tools/registry.py (default approval_handler), harness/approval.py | tests/test_repo_configs.py::test_approval_gated_tool_fails_closed_under_default_handler, ::test_default_approval_handler_denies |
| Cross-agent file tampering: one agent overwrites another's work or an operator file | Shared mutable workspace, last writer wins | Ownership lanes: agents own the files they create, cross-agent writes are refused with a request_change remedy, and operator lanes are locked to all agents | harness/ownership.py, OWNERSHIP.yaml | tests/test_ownership.py::test_cross_agent_write_denied_at_dispatch, ::test_operator_lane_denies_all_agents |
| Consent bypass: a task packet is treated as a command and mutation happens before the work is accepted | Dispatch side effects as soon as the model asks | Consent lock: registry dispatch refuses write/execute tools until the worker calls accept_task; read tools stay available so consent is informed | harness/tools/registry.py (dispatch, side_effects_locked) | tests/test_consent.py::test_side_effects_locked_until_accept, ::test_read_tools_available_before_accept |
| Audit tampering: an agent hides what it did by editing the log after the fact | Append-only text log, trusted as written | Every event carries seq and prev (SHA-256 of the previous line); any edit or deletion breaks the chain and harnessie audit exits non-zero | harness/events.py, harness/audit.py | tests/test_audit.py::test_edited_line_breaks_chain, ::test_deleted_line_breaks_chain |
Honest residual
The tool-call and exfiltration paths are mechanically confined by the rows above. What remains is a well-crafted injection written as ordinary-looking advice, aimed at the deliverable rather than at tool calls: it passes every mechanical scan because it reads as plausible prose, and it can satisfy the acceptance criteria while still being wrong on purpose. That class is caught only by the independent verifier (a reasoning check, not a filter) and by human review of the diff before a run touches real credentials. This is a deliberate design boundary, not an oversight, and it is the reason layers 6 and 7 in SECURITY.md exist. A harness that claimed to filter this away mechanically would be overclaiming.
Two further limits stated plainly: the hash chain is tamper-evident, not tamper-proof (a whole-file rewrite is out of scope; anchoring the chain head externally is an operator decision), and an allowlisted interpreter can still write inside the workspace without a per-file ownership check (the sandbox confines the workspace as a whole; per-lane sandbox profiles are roadmap).
How to check this artifact
Run the suite and confirm the cited tests exist and pass:
python3 -m pytest -qEvery file::test_name above is a real node in that suite. A citation that no longer resolves is a defect in this document; fix the document or the code, never leave the claim standing without its test. This artifact is also a standing invitation to contest the claim: the disclosure path and the eval scenarios published as red-team targets are in SECURITY.md.