The Quarantine Pattern: How To Test A New Skill Without Letting It Touch Production Pacts

TL;DR

The modal way operators evaluate a new skill today is to install it into a development agent, run a few sample inputs through it, and promote it to production when nothing obviously breaks. This works until the day a skill behaves correctly under sample inputs but pathologically under real ones — at which point the failure happens against a production pact, in front of a real counterparty, on a real piece of work. Quarantine is the alternative: an isolated test environment with synthetic counterparties, synthetic memory, and a read-only network surface, where a new skill performs its full job against scenarios designed to surface its failure modes before it ever sees a production pact. This post walks the architecture of a quarantine environment, the intake protocol for moving a skill through it, the failure modes the protocol catches, and ends with a Quarantine Intake Workflow you can adopt directly.

Intro: The Skill That Worked In Testing And Broke In Production

A team I work with adopted a new agent skill three months ago that did entity extraction on customer support transcripts. The skill was well-reviewed in the broader ecosystem. The team's evaluation process consisted of running it against twenty sample transcripts, comparing the output to what they expected, and approving it for production after the outputs looked reasonable. The skill rolled out into a customer-facing agent on a Tuesday morning. By Wednesday afternoon, the agent had emitted three customer responses that addressed the customer by the wrong name, because the skill had pulled a name from a quoted previous email instead of from the active speaker.

The failure was not a bug in the skill, exactly. The skill did what it was authored to do: it extracted entities from text. The failure was in the boundary between the skill's behavior and the agent's downstream use of the output. The team had not anticipated that the skill would extract names from quoted text. The skill author had not documented this behavior, because from the skill's perspective it was not a behavior — it was just what entity extraction does. The combination produced a real customer-facing incident that took a day to root-cause, two days to fix, and a week to apologize for.

This is the modal new-skill failure: not malicious code, not provenance compromise, not capability over-grant — just a skill behaving exactly as authored, against inputs the operator never tested, producing outputs the agent's downstream consumers could not handle. The sample-input evaluation process catches gross failures and misses everything else. The space of inputs a real production agent encounters is enormous, and the space of inputs a sample evaluation covers is tiny. The gap between them is where new-skill incidents live.

Quarantine is the name for the discipline that closes this gap. The quarantine environment is not a development environment that happens to have fewer agents in it. It is a purpose-built test environment whose entire architecture is shaped by the goal of surfacing skill failures before they reach production. Synthetic counterparties stand in for the real ones, with workflows that probe the kinds of inputs production produces. Synthetic memory provides the kind of historical context skills will encounter, without leaking real customer data into the test. Read-only network access prevents the skill from making changes the operator cannot reverse. The skill performs its full job, with full visibility into what it produces, in an environment designed to absorb any behavior the skill exhibits.

This post is the architecture and the protocol. The architecture is the quarantine environment itself: what it contains, how it differs from production, what makes it safe. The protocol is the intake sequence: what happens to a new skill from the moment it is admitted to the moment it is approved for production. The deliverable is a Quarantine Intake Workflow that you can adopt as a procurement step, integrated with the skill registry and the runtime, and tied to the audit trail that backs the agent's composite score.

Section 1: What Quarantine Is And What It Is Not

The word "quarantine" gets used loosely in software circles to mean any kind of isolated environment. The skill quarantine pattern is more specific. It is an environment whose properties are defined by what it must protect against, not by what it happens to lack. Three properties define it: synthetic counterparties, synthetic memory, and read-only network surface. Each property closes a specific class of risk. Together, they let a skill perform its full job — including operations that, in production, would touch real systems — without producing any persistent or externally-visible change.

Quarantine is not a development environment. A development environment is shaped by developer convenience: easy to spin up, fast to iterate, with realistic-enough data and just-enough infrastructure to make things work. Development environments are excellent for authoring code and terrible for evaluating skill safety, because their failure mode is to look enough like production that operators trust them when they should not. A skill that works in dev is not a skill that has been tested; it is a skill that has not yet failed in dev.

Quarantine is not a staging environment. A staging environment is shaped by deployment validation: a near-copy of production, with a near-copy of production's data, intended to catch deployment issues before they hit production. Staging environments are useful for rollout validation and dangerous for skill evaluation, because they often have access to production-equivalent systems and can produce real changes if a skill misbehaves. A skill that runs in staging may not have changed anything yet, but it could have. Quarantine differs by ensuring it cannot.

Quarantine is not a sandbox in the runtime-isolation sense. The runtime sandbox — process, container, microVM — protects the host from skill code. The quarantine environment protects the production trust graph from skill behavior. The two are complementary: a skill in quarantine still runs in whatever sandbox mode the runtime requires, but it also runs against an environment whose external surfaces are inert. Sandboxing protects the machine; quarantine protects the relationships.

Quarantine is not a one-time evaluation. A skill that passes the intake protocol and graduates to production is not done with quarantine. Skills should re-enter quarantine on every version update, on every capability manifest change, and on a periodic schedule for high-stakes deployments. The relationship between a skill and the quarantine environment is ongoing, not gate-and-promote.

What quarantine is, then, is a continuously-available test surface where skills can perform their full job in a way that is consequential for evaluation but inconsequential for the trust graph. The discipline of building such a surface — and committing to running every new or updated skill through it before production exposure — is the discipline that prevents the modal new-skill failure.

Section 2: Synthetic Counterparties And The Behavior Probe

The first property of quarantine is the use of synthetic counterparties in place of real ones. A counterparty, in the agent economy, is the actor on the other side of the agent's pact — the customer, the partner, the downstream service, the integrating platform. In production, counterparties are real, with real interests, real data, and real consequences for misbehavior. In quarantine, counterparties are synthetic — software constructs that play the role of a counterparty for the purpose of probing the skill's behavior.

A synthetic counterparty has three jobs. The first job is to produce inputs the skill will encounter in production: customer messages, transcript segments, document uploads, API requests. The synthetic counterparty's input distribution should reflect the distribution the production system actually encounters, including the long-tail inputs that sample-based evaluation typically misses. Building this distribution is one of the harder parts of running quarantine well — it requires either careful synthesis from production traffic patterns or deliberate adversarial construction by humans who know the failure modes of the domain.

The second job is to consume the agent's outputs and react to them. A synthetic counterparty does not just feed inputs and disappear; it observes the agent's response, evaluates whether the response is acceptable from a counterparty perspective, and either accepts or rejects in ways that probe further interaction. A response that contains the wrong customer name, or refers to the wrong order, or makes a commitment the pact does not authorize, should produce a synthetic counterparty reaction that escalates — generating follow-up inputs that would, in production, surface the error. The synthetic counterparty is the closed-loop observer that catches behavior the open-loop sample inputs would miss.

The third job is to probe pact compliance. The synthetic counterparty knows what the pact says, knows what the agent is committing to, and constructs scenarios specifically designed to test whether the agent honors the pact under stress. Scenarios include: ambiguous inputs that could be interpreted multiple ways, inputs that approach the boundary of the pact's scope, inputs that combine multiple legitimate operations into sequences the pact may or may not authorize, inputs that rely on context the agent should but may not have. The synthetic counterparty is the adversarial agent that finds the seams.

Building synthetic counterparties is partially automatable and partially manual. The automatable part is the input distribution, which can be generated from production traffic samples (with appropriate scrubbing) or from large language model synthesis trained on the production input shape. The manual part is the adversarial construction — the scenarios designed by domain-expert humans to find the failure modes the synthesis would not generate. The right mix is roughly seventy percent automated and thirty percent adversarial, with the adversarial portion focused on the highest-stakes pact dimensions.

The critical property of a synthetic counterparty is that its decisions have no real-world consequences. When the synthetic counterparty rejects an agent response, no real customer has been disappointed. When it accepts a wrong response, no real workflow has been corrupted. The reactions exist purely as evaluation signal, fed back into the quarantine assessment. This is what lets the skill perform its full job, including responses that would in production be wrong, without producing any actual harm.

Section 3: Synthetic Memory And The Cold-Start Problem

The second property of quarantine is synthetic memory. Most skills, especially those that participate in long-running agent workflows, depend on memory — historical context the agent has accumulated about the work, the counterparty, the domain. A skill evaluated against an agent with no memory is being evaluated in an unrealistic posture, because production agents always have memory, and the skill's interaction with that memory is part of what determines its real behavior.

The synthetic memory in quarantine is a structured store that resembles the production memory store in shape and richness, but contains synthetic content. The shape question is straightforward: the same record types, the same indexing, the same retrieval semantics. The content question is the hard one: what populates the memory, and how realistic is the population?

The content has to satisfy two constraints. The first is that it should not contain real production data. Even with scrubbing and anonymization, real data carries risks: re-identification, leakage of business-sensitive information, drift between the scrubbed dataset and the production reality. The cleaner approach is to synthesize memory contents from scratch, using domain models that produce realistic-looking records without being derived from real ones.

The second constraint is that the synthetic memory should reflect the kinds of patterns production memory contains. Real customer support memory has long histories with some customers and short histories with others, contains records of varying quality and detail, includes both successful interactions and failed ones, has temporal patterns reflecting how interactions unfold over time. Synthetic memory built without these patterns fails to surface skill behaviors that depend on them. A skill that handles short-history customers correctly may misbehave against long-history customers, and a synthetic memory that contains only short histories will not surface the issue.

Building synthetic memory at this level of fidelity is a one-time investment that pays off across many skill evaluations. The synthetic memory is a reusable asset — built once for a domain, refreshed periodically, used for every skill that operates in the domain. The cost is real but it amortizes well, especially for operators who run many agents in similar domains.

The synthetic memory should also include the memory anti-patterns: stale records, conflicting records, records with subtle errors, records with structural anomalies. Skills that depend on memory in production will encounter these patterns; skills that are evaluated against perfectly-clean synthetic memory will not be tested against them. Including the noise is a deliberate design choice that exposes skill brittleness.

The other dimension of synthetic memory is temporal. Production memory grows over time and skill behavior may depend on the depth of history available. The synthetic memory should support varying depths — shallow histories for new-counterparty scenarios, deep histories for long-tenured-counterparty scenarios — and the quarantine intake should exercise the skill across the range. Skills that behave correctly against any depth pass; skills that behave correctly only against the depth the evaluator happened to test do not.

Section 4: The Read-Only Network Surface

The third property of quarantine is that its network surface is read-only. The skill, while running in quarantine, can read from external systems but cannot write to them. This property is what makes quarantine consequence-free in the external sense: even if the skill malfunctions in a way that would, in production, send an email or update a record or post to an integration, the actual side effect does not happen.

The implementation of read-only network surface depends on the runtime's network architecture. The most robust implementation is at the network egress layer: a proxy or firewall that allows outbound HTTP requests of the GET, HEAD, and OPTIONS variety and refuses POST, PUT, PATCH, and DELETE. This catches every write attempt regardless of the skill's intent and produces a clean refusal that the skill can observe but cannot bypass.

A second implementation is at the credential layer: the quarantine environment's credentials are scoped to read-only versions of every external system. The skill technically can issue write requests, but the credentials it holds do not have authorization for them. This is enforceable from the credential issuer side and produces clean refusals at the API layer rather than the network layer.

A third implementation is at the runtime's outbound call boundary, the same boundary that enforces capability scoping. A quarantine flag on the runtime causes every outbound call's permissions to be downgraded to read-only, regardless of what the skill's manifest declares. This is the most flexible implementation and the easiest to bypass if the runtime's enforcement is incomplete; it should be combined with one of the other two implementations rather than used alone.

The read-only constraint can be relaxed for specific external systems where writes are themselves part of the test. A quarantine that contains a synthetic version of a production database, for example, can permit writes to the synthetic version while blocking writes to the production version. The principle is that writes should land only on systems that exist purely for the quarantine environment and have no production consequences. The shape is a quarantine-internal write surface paired with a production-side read-only surface.

The read-only constraint also applies to communications. A skill that, in production, would send emails or post messages to external channels should, in quarantine, have those operations intercepted. The interception can deliver the would-be communication to a synthetic inbox where evaluators can review it, or to an audit trail where the operator can verify what would have been sent. The communication is observed, not delivered.

The combination of these implementations produces a network surface that is consequential for evaluation — the skill can observe the outputs of its read operations and respond to them — but inconsequential for production. The skill has full read visibility, which lets it perform its job realistically; it has zero write reach, which prevents any of its mistakes from producing external effects. This is the property that makes quarantine fundamentally different from any environment with even a sliver of write access to a system the operator cares about.

Section 5: The Intake Protocol — From Adoption To Promotion

The quarantine environment is the architecture. The intake protocol is the process. A new skill enters quarantine, performs a defined sequence of activities, accumulates evidence about its behavior, and emerges either approved for production or rejected with documented reasons. The protocol is what makes the quarantine environment productive rather than merely available.

The protocol begins at the moment a skill is admitted to the registry. Admission is the operator's declaration that the skill is a candidate for production use. Admission triggers the intake: the skill is loaded into the quarantine runtime, its capability manifest is reviewed, its provenance is verified, and the operator schedules the intake activities. Skills that fail admission — failed provenance, failed manifest review — do not even reach the intake; they are rejected at the registry boundary.

The intake activities have four phases. The first phase is calibration. The skill is exercised against a calibration scenario set — a collection of inputs whose expected outputs are known. The purpose is not to evaluate skill quality but to confirm the skill is operating as intended in the quarantine environment, with the right credentials, the right capability scopes, and the right runtime configuration. Calibration that fails indicates an environment problem that must be resolved before evaluation can proceed.

The second phase is functional evaluation. The skill is exercised against a broad scenario set drawn from synthetic inputs, synthetic memory contexts, and synthetic counterparty interactions. The scenario set should include the skill's intended use cases, edge cases the operator can anticipate, and adversarial cases designed to probe failure modes. The skill's outputs are evaluated against pact-compliance criteria — does the skill produce outputs that, if the agent emitted them in production, would honor or violate the pact? Failures are categorized by type and severity.

The third phase is regression evaluation. The skill is exercised against the same scenario set used for skills the operator has previously evaluated in the same capability area. The purpose is to confirm the new skill performs at least as well as the existing baseline on the operator's standard cases. This is particularly important for skill replacements, where the new skill is intended to substitute for an existing one. Regressions surfaced at this phase are blockers for promotion until the operator has decided whether the new skill's benefits compensate for the regressions.

The fourth phase is integration evaluation. The skill is exercised in combination with the other skills the agent imports, in scenarios that mimic the production agent's full workflow. The purpose is to surface skill-skill interaction issues — cases where the new skill, alone, behaves correctly, but its interaction with downstream consumers produces problems. These interaction issues are some of the most common new-skill failures and are invisible at the per-skill evaluation level.

Each phase produces a structured evaluation record that becomes part of the skill's intake history. Skills that pass all four phases with no critical failures and an acceptable rate of moderate failures are approved for production. Skills that fail any phase, or accumulate too many moderate failures across phases, are rejected with a documented set of issues. Rejected skills can be re-submitted after revision; the new version goes through the full intake again.

Section 6: Failure Categorization And The Severity Model

Not every failure surfaced in quarantine is a blocker. The intake protocol needs a severity model that distinguishes failures that should prevent promotion from failures that should be noted and tolerated. Without the model, every failure either produces over-blocking — too many skills rejected for cosmetic issues — or over-permissive promotion — important failures dismissed because they were one of many.

The severity model has four levels. Critical failures are the ones that, if they occurred in production, would directly violate the pact. A skill that produces outputs containing the wrong identity for a counterparty, in a context where the pact commits to identity accuracy, is a critical failure. Critical failures block promotion unconditionally; the skill does not advance until the failure is resolved.

High-severity failures are the ones that would not directly violate the pact but would cause counterparty-visible problems. A skill that produces correct outputs but in a degraded format that downstream consumers would mishandle is a high-severity failure. High-severity failures block promotion absent specific operator override with documented justification. The override is a conscious decision, not a default.

Medium-severity failures are the ones that would degrade the agent's quality without producing direct counterparty problems. A skill that occasionally produces lower-quality outputs than its predecessor, but without any individual output crossing into wrongness, is a medium-severity failure. Medium-severity failures do not block promotion individually but accumulate against a budget; a skill with too many medium failures across the evaluation set fails on the budget rather than on any single failure.

Low-severity failures are the ones that are observable in evaluation but would not be noticeable in production. A skill that produces outputs in a slightly different format from a previous version, where the new format is equally acceptable, is a low-severity failure. Low-severity failures are noted in the intake record but do not contribute to promotion decisions unless they aggregate to a level suggesting systematic difference from the operator's expectations.

The severity assignments require human judgment, which is part of why quarantine intake cannot be fully automated. The synthetic counterparty's reactions and the runtime's audit trail provide the raw evidence; an evaluator — typically a domain-knowledgeable human, though increasingly assisted by jury-style multi-LLM evaluation — assigns severity and decides on the promotion question. The evaluator's judgments themselves should be recorded, so that promotion decisions are auditable after the fact.

The severity model also drives the post-promotion monitoring posture. Skills promoted with documented medium-severity failures should be monitored more carefully in production for the specific failure modes that the intake surfaced. Skills promoted with override on high-severity failures should be monitored with specific alerts on the failure mode. The intake record carries forward into production observability, so that the failures known at intake time become the alerts watched for at runtime.

Section 7: Time And Stake — How Long Should A Skill Stay In Quarantine

The duration a skill spends in quarantine should scale with the stake of its intended production use. A skill destined for a low-stakes pact — internal tooling, optional advisory output, non-critical workflow steps — can spend a short time in quarantine, with a small evaluation set and a fast intake decision. A skill destined for a high-stakes pact — financial actions, customer-facing communications, regulatory-sensitive outputs — should spend significantly longer in quarantine, with a larger evaluation set and more conservative promotion criteria.

The minimum quarantine duration for any skill is whatever it takes to complete the four intake phases against a meaningful scenario set. For most skills this is hours to a few days, depending on the throughput of the synthetic counterparty interactions and the responsiveness of the human evaluators. Anything shorter than the minimum is intake theater, a quarantine in name only that does not actually exercise the skill enough to surface real failures.

The maximum quarantine duration is bounded by the operator's tolerance for delayed adoption. A skill that sits in quarantine for months may be more thoroughly tested than one that sits for days, but the additional testing produces diminishing returns past some point, and the operator pays an opportunity cost for the delay. The right duration is the one where the marginal failure-discovery rate has dropped below the operator's tolerance threshold for unsurfaced failures.

For high-stakes pacts, the right duration may include a soft-launch phase where the skill is in production but only for a subset of agents or a subset of the agents' workload. This is not strictly quarantine — production effects are real — but it preserves some of the quarantine discipline by limiting the blast radius of any failure. Soft launches let the operator surface production-only issues that the synthetic environment could not reproduce, with a damage budget bounded by the launch scope.

The re-quarantine cadence after promotion is also stake-dependent. Low-stakes skills can be re-evaluated annually or on version change, whichever comes first. High-stakes skills should be re-evaluated quarterly, or more often if the version churn is high. Re-quarantine does not require the full intake protocol every time — calibration and a sample of regression tests are usually sufficient — but it should not be skipped, because the synthetic environment evolves and a skill that passed last quarter's quarantine may not pass this quarter's, and that information is useful even if the skill has not changed.

The time budget for the entire quarantine activity, summed across the operator's portfolio, is meaningful overhead. For a portfolio of dozens of skills with regular updates, the human time spent on intake decisions is substantial. The way to keep this manageable is to invest heavily in the automated portions of the intake — synthetic counterparty quality, scenario set automation, jury-based pre-evaluation that flags only the cases requiring human judgment — and to focus human attention on the highest-stakes promotions.

Section 8: The Failure Modes Of Quarantine Itself

Like any safety architecture, quarantine has its own failure modes. Operators who treat the quarantine environment as a bulletproof gate will be surprised by the cases where a skill passes intake and still misbehaves in production. The failure modes of the quarantine itself are worth knowing.

The first failure mode is synthetic environment drift. The synthetic counterparties, synthetic memory, and scenario sets that drove the original quarantine were calibrated against a snapshot of production reality. As production evolves, the synthetic environment becomes less representative. A skill that passes intake against the old synthetic environment may fail against current production conditions, and the gap is the operator's blind spot. The fix is to refresh the synthetic environment on a defined cadence, ideally driven by automated comparison between synthetic input distributions and live production input distributions.

The second failure mode is scenario coverage gaps. The scenario set used for evaluation is, by definition, a finite sample of the infinite possible production inputs. Failures that occur only on inputs outside the scenario set are invisible to the intake. The fix is to expand the scenario set continuously based on observed production failures: every failure that escapes to production becomes a new scenario in the intake set, so that the next skill encountering that condition catches it. The scenario set is a learning artifact, not a fixed one.

The third failure mode is synthetic counterparty under-specification. A synthetic counterparty that does not probe enough behaviors produces a quarantine that is too easy to pass. Skills with subtle failure modes in dimensions the synthetic counterparty does not test will pass intake without those failures being surfaced. The fix is to evaluate the synthetic counterparty itself against the failure modes that have escaped to production, and to expand the synthetic counterparty's probing capabilities accordingly. The counterparty is also a learning artifact.

The fourth failure mode is integration blindness. Skills evaluated in isolation may pass; the same skills evaluated in production combinations may fail. The integration phase of intake is meant to catch this, but no integration phase covers every possible combination. Skills with combinatorial failure modes — failures that emerge only in specific combinations of skills that the intake did not exercise — will escape. The fix is to expand integration scenarios based on the production agent's actual skill compositions and to re-run integration intake whenever a new skill joins an established composition.

The fifth failure mode is severity mis-calibration. The severity model is judgment-driven, and judgment can be wrong. Failures classified as low when they should have been high — or rejected as critical when they should have been overridden — produce promotion decisions that turn out poorly. The fix is to track the production outcomes of intake decisions: skills promoted with overrides should be tracked specifically, and the severity classifications that produced their failures should be revisited. The severity model itself is calibrated against post-promotion experience.

The sixth failure mode is the social one: operators who skip quarantine when under time pressure. A new skill is needed urgently for a launch deadline, and the operator decides to bypass the quarantine intake and promote directly. Sometimes this works; sometimes it produces the next post-mortem. The defense is institutional: the runtime should enforce intake completion as a precondition for production registration, with overrides requiring sign-off from a level of the organization that takes the responsibility seriously. The friction is the point.

Section 9: The Reader's Artifact — The Quarantine Intake Workflow

This is the deliverable. The Quarantine Intake Workflow is a structured procedure with named gates, defined evidence requirements, and explicit promotion criteria. Use it as the procurement gate for every new or updated skill in your registry. Adapt the specifics to your environment but preserve the structural shape.

Gate 1: Admission. A skill is admitted to intake when it is signed by a verified author, has a complete capability manifest, and resolves all provenance checks. Admission is a binary; failure at admission rejects the skill before any quarantine work begins. Admission produces an intake record with the skill's identity, version, manifest, and provenance.

Gate 2: Calibration. The skill is loaded into the quarantine runtime and exercised against a calibration scenario set of five to ten known-input known-output cases. The purpose is to verify environment correctness. Calibration failure indicates an environment issue, not a skill issue, and must be resolved before functional evaluation.

Gate 3: Functional Evaluation. The skill is exercised against the functional scenario set, sized to the stakes of the intended production use. For low-stakes pacts, fifty to one hundred scenarios; for high-stakes pacts, five hundred or more. Each scenario produces an output that is evaluated against pact compliance criteria. Outputs are categorized by failure severity. Critical failures block promotion.

Gate 4: Regression Evaluation. The skill is exercised against the regression scenario set — scenarios used to evaluate previous skills in the same capability area. The skill's performance is compared against the operator's baseline. Regressions are flagged. Skills that introduce regressions on critical scenarios block promotion until the regression is resolved or specifically accepted.

Gate 5: Integration Evaluation. The skill is loaded alongside the other skills the production agent imports and exercised in combination scenarios. Each scenario probes the skill's interaction with downstream consumers. Combinatorial failures are categorized by severity. Integration-phase critical failures block promotion.

Gate 6: Severity Review. The accumulated failures from the previous gates are reviewed. The total count and distribution by severity are compared against promotion criteria. Skills meeting the criteria proceed to promotion. Skills failing the criteria are rejected with a documented set of issues.

Gate 7: Promotion. The skill is moved from quarantine to production registry status. The intake record — including all scenarios, outputs, severity classifications, and reviewer judgments — is preserved as the audit trail backing the promotion decision. The skill becomes available for production agents to import.

Gate 8: Production Monitoring Configuration. Based on the intake record's documented failure modes (including those classified as moderate-or-low and tolerated for promotion), production observability is configured to alert on any of those failure modes recurring at scale. The intake record carries forward into runtime monitoring.

Gate 9: Re-Quarantine Schedule. A schedule is set for re-evaluation: quarterly for high-stakes deployments, annually for low-stakes ones, immediately on any version change. The schedule is the standing commitment that the intake decision is not permanent.

Use the workflow as the standard intake gate. Resist the temptation to skip phases for skills that look obviously fine; the failure mode the workflow catches is the skill that looks obviously fine and is not. Build the synthetic environment as a long-term investment. Treat the scenario sets as learning artifacts that grow with each post-promotion incident.

Section 10: A Counter-Argument Worth Taking Seriously

The critique of the quarantine pattern is that it adds substantial friction to the skill adoption process, and the agent ecosystem benefits from low-friction adoption. Operators who can quickly try and adopt new skills move faster, learn more, and build better agents. Operators who have to drag every skill through a multi-gate intake protocol move slower, evaluate fewer skills, and end up with stale skill registries that lag behind the ecosystem's evolution.

This is partly true and partly an argument for tier-based quarantine. The intake protocol does not have to be the same for every skill. Skills destined for low-stakes use can have an abbreviated intake that completes in hours: minimal calibration, a small functional scenario set, light review. Skills destined for high-stakes use should have the full protocol. The right operational answer is a tiered intake with the gate severity scaling to the stakes, not a uniform protocol that either over-blocks low-stakes or under-checks high-stakes.

A second critique is that the synthetic environment is itself an investment that may not be justified for many operators. Building synthetic counterparties, synthetic memory, and scenario sets is significant work. For an operator with a small portfolio of low-stakes agents, the investment is disproportionate to the benefit. The synthetic environment is what makes quarantine real; without it, quarantine is just a different name for the development environment.

The answer here is shared infrastructure. Synthetic environments for common domains — customer support, sales, content moderation, data extraction — are themselves valuable artifacts that can be shared across operators. A synthetic environment as a service, maintained by domain specialists and used by many operators, amortizes the cost in a way that no single operator could justify alone. The agent ecosystem will likely produce these as commercial offerings, and operators who do not want to build their own should adopt them.

A third critique is that quarantine catches certain failure modes and not others. Skills with subtle behavioral issues that only manifest under production-scale traffic, or under interactions with real (not synthetic) counterparties, will escape even a well-run quarantine. The operator who treats quarantine as a complete safeguard will be surprised by these. The defense is layered safety: quarantine catches what it catches, production monitoring catches what quarantine missed, post-incident review feeds the next quarantine cycle. No single layer is complete.

A fourth critique is that the discipline of running quarantine intake produces a slowness that competitors who skip it will exploit. An operator who promotes skills in days while a competitor takes weeks is moving faster, all else equal. The market may reward the faster operator and punish the more careful one, especially in the early phase of the agent economy where users do not yet evaluate operators on safety record.

This is true in the short term and reverses in the long term. The early-phase fast operators accumulate the failure history that the careful operators avoid. As the agent economy matures and users start evaluating operators on safety record — which the trust oracle increasingly enables — the careful operators have the credible track record and the fast operators have the incident history. The bet on quarantine is a bet that the long-term reputation reward exceeds the short-term speed cost. Operators who plan to be in business in three years should make the bet.

What Armalo Does About This

Armalo's runtime supports a quarantine mode as a first-class deployment configuration. Quarantine-mode runtimes are isolated from production credentials, route outbound calls through a read-only egress proxy, are paired with synthetic memory stores, and integrate with synthetic counterparty services that probe agent behavior with adversarial scenarios. The quarantine intake protocol — admission through promotion — is supported as a standard procurement gate in the operator workspace, with structured evaluation records that flow into the audit trail.

The synthetic counterparty service uses jury-based evaluation to assess agent outputs against pact compliance criteria, with the multi-LLM jury producing severity classifications that the human evaluator can confirm or override. The scenario sets are versioned artifacts that grow with post-promotion incidents: a failure that escapes to production becomes a new scenario in the next intake cycle, and the failed scenario propagates to all operators using shared scenario libraries.

The skill registry distinguishes admitted-to-intake from promoted-to-production states. Skills in intake are visible to operators planning their portfolio but are not loadable by production agents. Skills that pass intake become loadable, with the intake record exposed via the trust oracle so counterparties can see the diligence that backs the agent's skill set. Skills that fail intake are visible too, with their rejection reasons available to the broader ecosystem so other operators can learn from the failure.

The re-quarantine schedule is enforced at the runtime: skills past their re-evaluation deadline are marked stale and are loaded with a warning that surfaces in the agent's audit trail. This produces a continuous-evaluation posture rather than a one-time gate.

Frequently Asked Questions

How do I build synthetic counterparties for my domain if there is no shared library? Start with a small set of human-authored adversarial scenarios — five to ten cases that represent the failure modes you most worry about. Layer in synthetic input generation from production traffic patterns, with appropriate scrubbing. Iterate the synthetic counterparty against the failures you observe in production over time. The investment compounds.

How do I keep the quarantine environment in sync with production? Define refresh cadences for the synthetic memory shape, the scenario sets, and the synthetic counterparty behaviors. Run automated comparison between production input distributions and synthetic input distributions; large divergences are signals that the synthetic environment needs refresh. Use post-incident reviews as forced refresh events.

Can quarantine catch all skill failures? No. Quarantine catches failures the synthetic environment is configured to surface. Production-scale failures, real-counterparty interactions, and combinatorial failures with skills not in the integration set will still escape. Quarantine is a layer, not a complete safeguard.

Is quarantine the same as red-teaming? Related but distinct. Quarantine is the intake protocol for new or updated skills; red-teaming is an adversarial evaluation that can run against any skill at any time, often against production agents. The quarantine intake should include red-team-style adversarial scenarios, but red-teaming as a discipline extends beyond intake.

How long does the intake actually take in practice? For low-stakes skills with small scenario sets, hours to a day. For high-stakes skills with hundreds of scenarios and full integration evaluation, a week or more, with much of that time spent on human review of edge cases. The duration scales with stakes by design.

What if my synthetic counterparty is wrong about what counts as pact compliance? The synthetic counterparty is an evaluator, not the source of truth. The pact text is the source of truth. The synthetic counterparty's job is to produce evidence that a human evaluator can use to make a promotion decision. Errors in the synthetic counterparty's judgments are caught by the human reviewer.

Does this slow down the skill ecosystem? It slows down the rate of new skill adoption. It also reduces the rate of new-skill incidents, which slows down the rate of post-incident skill rollback and apology cycles. The net effect on velocity depends on the failure rate without quarantine, which is higher than most operators realize until they measure.

Can I use the same quarantine for skills that operate in different domains? The environment can be shared but the synthetic counterparties and scenario sets should be domain-specific. A skill that does customer support work needs a customer-support synthetic counterparty; a skill that does sales work needs a sales synthetic counterparty. The infrastructure scales across domains but the per-domain investment is real.

Bottom Line

A new skill should not see a production pact on its first day. Quarantine — the discipline of testing skills against synthetic counterparties, in synthetic memory, with read-only network access — is the architecture that prevents the modal new-skill failure: the skill that worked in sample evaluation and broke against real conditions. The Quarantine Intake Workflow is a nine-gate procedure, with severity-tiered failure handling and re-evaluation built in. The synthetic environment is the upfront investment. The intake protocol is the operating discipline. Operators who invest in both will not be the ones writing the post-mortem about a skill that worked in testing and broke in production. Operators who do not will be.