Capability Diff Review: A Code Review Practice For Newly-Imported Agent Skills
Before importing a new skill, diff its declared capabilities against your existing skill set. What's new? Why? Required permissions? A reviewer template you can use today.
Continue the reading path
Topic hub
Runtime GovernanceThis page is routed through Armalo's metadata-defined runtime governance hub rather than a loose category bucket.
Turn this trust model into a scored agent.
Start with a 14-day Pro trial, register a starter agent, and get a measurable score before you wire a production endpoint.
TL;DR
Before importing any new agent skill, diff its declared capabilities against your existing skill set. What permissions does it ask for that no current skill needed? What network destinations are new? What action types? What identities can it speak as? Capability diff review is the agent supply chain equivalent of code review for dependency upgrades, and it should be a routine practice, not an emergency response. This post lays out the four diff dimensions, the reviewer checklist for each, the conversation patterns that make review productive rather than adversarial, and a Skill Import Diff Review Template that becomes the artifact you actually fill out per import. The framework introduced here is the New Capability Justification Test: every new capability requested must have a written justification that survives a 24-hour cooling-off period before the import is approved.
The Import That Should Have Been A Conversation
The failure mode this post addresses started, in our experience, the day MCP and similar protocols made it trivial to add a new skill to an agent. Before the protocol era, adding a new capability to an agent was an engineering project: someone wrote code, someone reviewed the code, someone deployed the code. The friction was real and the friction enforced a certain amount of consideration about what the new capability could actually do. After the protocol era, adding a new capability is a single line in a config file or a single click in a UI. The friction is gone. The consideration is also, in most teams, gone with it.
We see this play out in audits. A team imports a skill. The skill works. The skill is added to the agent's tool list. The agent uses the skill in production. Six weeks later, when we run our review, we ask the team why they imported this particular skill, what permissions it requested, and what the alternatives were. The answers are usually some combination of: "someone on the team thought it would be useful," "we did not look at the permissions specifically," and "we did not consider alternatives because this was the first one we found that worked." None of those answers are bad in isolation; together they describe a process where critical security and architectural decisions are being made by reflex rather than deliberation.
The specific incident that crystallized this for us was a team that imported a skill providing email-sending capability for an internal agent. The skill worked. The team did not notice that the skill also requested capabilities for OAuth token storage, calendar access, contact list access, and inbox reading, none of which were necessary for the email-sending function the team actually wanted. The skill was a perfectly functional email-sending tool that also happened to position itself for a much broader set of operations than the team had any reason to grant it. The skill author was not necessarily malicious; they had built a general-purpose Gmail tool that did many things, and the importer had taken the whole package without noticing the breadth. Two months later, the broader capabilities became part of an incident, when the agent's pattern of activity led to questions about why it had been reading inboxes that nobody had asked it to read.
The purpose of this post is to make the import decision a conversation, not a reflex. We will walk through the four diff dimensions that matter, the reviewer checklist for each, the conversation patterns that produce useful reviews, and the template that captures the review as an artifact. The artifact is the artifact. The conversation is the value. Teams that adopt the practice find that the review takes about thirty minutes per import, that the imports themselves are noticeably better-considered, and that the rate of follow-on incidents drops significantly. This is not exotic security work; it is the same kind of dependency review hygiene that mature engineering organizations have applied to library imports for two decades, finally extended to agent skills.
The Four Diff Dimensions
The framework that organizes capability diff review is the Four Dimensions: permissions, network destinations, action types, and identity claims. Every import should be evaluated along all four, because each catches a different category of misalignment between what the importer thinks they are getting and what they are actually getting. A skill that looks fine on one dimension can be problematic on another, and reviewers who only check one or two will miss the misalignments that the others would have caught.
The permissions dimension is the most familiar. Every skill declares the permissions it needs to function: filesystem access (read or write to specific paths), network access (allowed hosts and protocols), credential access (which credential classes the skill needs), runtime access (which agent operations the skill can perform). The diff is between the permissions the new skill requests and the permissions your existing skills request. New permissions that do not appear in any existing skill are the most important signal: they represent a new authority being introduced into your fleet, and the reviewer's job is to understand whether the new authority is justified. Familiar permissions in a new combination are the second most important signal: a skill that requests filesystem access is fine, a skill that requests filesystem access plus network access plus credential access is a different risk profile than the sum of its parts.
The network destinations dimension covers where the skill expects to make outbound calls. Every skill, if it does any external work, has a set of hostnames it expects to reach. The diff is between the new skill's destinations and the destinations any existing skill uses. New destinations are not automatically bad, but they need a justification: why this destination, why this protocol, why this specific endpoint. Reviewers should be especially attentive to destinations that look suspicious by domain (typosquats of legitimate services, freshly-registered domains, domains with no obvious business purpose) and to destinations that combine harmlessly-named tools with broad capability (a translation service that ends up routing your data through a third-party LLM API, for example).
The action types dimension covers what state-changing operations the skill can perform. Every skill that does anything beyond pure computation has some set of actions it takes: writing files, sending messages, updating records, making payments. The diff is between the actions the new skill can take and the actions your existing skills take. New action types deserve close scrutiny because they expand the agent's capability surface, and capability surface expansion is the leading indicator of incidents. The reviewer should ask, for every new action type, what the worst-case misuse looks like, and whether there are sufficient guardrails (capability mediation policies, rate limits, approval requirements) to bound the misuse.
The identity claims dimension covers what identities the skill can present in downstream systems. This is the most subtle of the four dimensions and the one most often missed. Every skill that interacts with downstream services does so under some identity: the agent's identity, a service identity, the skill's own identity, the human user's identity. The diff is between the identities the new skill claims and the identities your existing skills use. New identity claims raise structural questions: who is being represented in the downstream system, who is accountable for the actions taken, who shows up in audit logs. A skill that claims to act as the user, when previously all skills acted as the agent, is a meaningful structural change that should not be made silently.
The four dimensions are not independent. A skill that requests new permissions, makes calls to new destinations, performs new action types, and claims new identities is a much larger change than the sum of its individual diffs would suggest. The reviewer should look at all four together, build a coherent picture of what the skill actually does in your environment, and decide whether that picture is justified. This is the work that the review template is designed to make repeatable.
The Permissions Diff: What's New And Why
The permissions diff is where most reviewers start, because permissions are the most explicit declaration of authority that skills make. The diff process is mechanical: enumerate the permissions the new skill requests, enumerate the permissions every existing skill in your fleet requests, and identify the difference. The mechanics are simple; the analysis is where the work is.
For every new permission, the reviewer asks three questions. What does this permission allow? What does the skill actually need to do that requires this permission? Are there narrower permissions that would suffice? The first question is about understanding the permission as a primitive: not the marketing description of what the skill does with it, but the actual surface area of what the permission grants. A permission to read environment variables is not just about reading environment variables; it is about reading credentials, internal URLs, configuration secrets, and any other operational state that lives in the environment. A permission to write to the temp directory is not just about temporary files; it is about potentially staging payloads for other processes, persisting data across invocations, and interacting with whatever cleanup machinery operates on temp.
The second question is about understanding the necessity. Skills request permissions for many reasons, and not all reasons are good ones. Some permissions are necessary for the core function of the skill. Some are convenience permissions that make the skill author's life easier but are not strictly required. Some are speculative permissions that the skill author included "just in case" some future feature might want them. Some are intentional over-grants that the author hopes will go unnoticed. The reviewer's job is to distinguish these cases, which usually requires reading the skill's actual code or invoking it in a sandbox and observing its behavior. A skill author who cannot articulate, with specifics, why each permission is needed is making the import decision for the importer, and the importer should push back.
The third question is about looking for narrower alternatives. Even when a permission is genuinely necessary, the granted scope is often broader than the skill actually needs. A skill that needs to read one specific configuration file does not need general filesystem read access. A skill that needs to write to one specific output queue does not need general write access to the message bus. The narrowest workable scope is almost always smaller than what the skill author requests, and the reviewer's job is to negotiate for the narrow scope. Most skill authors will accept the negotiation if asked; some will refuse, at which point the import decision is whether to accept the broader scope or to find a different skill.
The specific patterns to watch for in the permissions diff are: permissions that span multiple categories (a skill that needs both network access and credential access has a different risk profile than one that needs either alone), permissions that do not match the skill's stated purpose (an email tool that wants calendar access has explaining to do), permissions for operations the skill performs rarely (a skill that needs admin permissions for a quarterly report can usually be redesigned to avoid persistent admin rights), and permissions for capabilities that did not exist when the skill was first written (a skill updated to ask for new permissions in version N+1 has a story to tell about why those permissions are now needed).
The Network Destinations Diff: Where Is It Calling?
The network destinations diff requires more work to produce than the permissions diff, because network destinations are not always declared honestly in skill manifests. A skill might declare that it makes calls to api.example.com when in fact it also makes calls to telemetry.collect.org and updates.notif.io and a third-party CDN that hosts its assets. The diff has to be produced from actual runtime observation, not from manifest declarations alone, and that means running the skill in a sandbox with network monitoring before trusting any single destination claim.
For every observed destination, the reviewer asks three questions. What is this hostname? What does the skill send to it and what does it receive back? What is the consequence if this destination is compromised? The first question is about due diligence: looking up the hostname's WHOIS record, the registrant, the hosting provider, the operational history. New domains, registrant proxy services, and hosting in jurisdictions with weak rule of law are all signals worth investigating further. Domains that look almost-but-not-quite like familiar services (with character substitutions, hyphens added, TLDs swapped) are particularly suspicious because they are the canonical pattern for typosquatting attacks.
The second question is about understanding the data flow. Reviewers should capture, in their analysis, what data flows from the skill to each destination and what data flows back. Skills that send minimal data and receive specific responses are different from skills that stream large quantities of data outward. Skills that receive data including executable code or configuration changes are particularly notable, because the destination has the ability to influence the skill's behavior in arbitrary ways. The shape of the data flow tells you what the skill could plausibly be doing on each call, and it lets you spot discrepancies between the stated purpose and the observed behavior.
The third question is about thinking adversarially. Every external destination is a dependency on someone else's security. The reviewer should ask, for each destination, what happens if that destination is compromised. Does the skill receive code or commands that it executes? Does the skill trust the destination's responses for security-critical decisions? Does the destination have access to data that would be sensitive in different hands? The answers should inform whether the destination is acceptable, and they should drive specific defensive measures: signature verification on responses, isolation of the destination's outputs from security-critical paths, monitoring for behavioral changes that would indicate destination compromise.
The diff process for network destinations should also catch the cases where a single skill connects to multiple destinations whose only relationship is that they are all owned by the same operator. A skill that connects to api.vendor.com is one trust dependency; a skill that connects to api.vendor.com plus analytics.vendor.com plus assets.vendor.com is functionally one trust dependency, but it is exposed across three different attack surfaces. The reviewer should be aware of these cases because the failure mode of one component (analytics service compromised) can affect the trust of the whole vendor, even when the failed component is not the one the skill primarily relies on.
The Action Types Diff: What State Can It Change?
The action types diff covers what state-changing operations the skill can perform in any system reachable from your environment. This is distinct from permissions (which describe what the skill is allowed to do at the runtime level) and from network destinations (which describe where the skill makes calls). Action types describe what the skill can change about the world: what files it writes, what messages it sends, what records it updates, what payments it makes.
For every action type the skill introduces, the reviewer asks four questions. What is the worst-case misuse of this action type? Who is impacted by misuse, and how badly? What guardrails exist to detect or prevent misuse? What is the recovery cost if misuse occurs? The questions are deliberately concrete; vague answers indicate that the team has not thought through the action type's implications carefully enough.
The worst-case misuse question is the most important, and it is where reviewers' imaginations have to be exercised. A file_write action's worst case is not just "writes a file the user did not want." It is "writes a file the user did not want, in a path the user did not expect, with permissions that allow other processes to read it, with content that includes secrets exfiltrated from elsewhere in the agent's context, in a way that persists across agent restarts." Each clause adds to the worst case. The reviewer's job is to think through the chain and identify whether the cumulative bad outcome is something the team would want to prevent.
The impact question forces the reviewer to think about who pays the cost of misuse. An action type that affects only the agent's own internal state is one impact profile. An action type that affects external customers is a different impact profile. An action type that affects financial state, regulatory state, or third-party systems is yet another. The action type's impact profile should determine how much scrutiny the action type gets in review and how much guardrail investment is justified.
The guardrails question is about defense in depth. Even when an action type is necessary and the worst case is bounded, there should be guardrails that catch misuse early. Rate limits prevent runaway invocations. Approval requirements bring humans into the loop for high-stakes operations. Anomaly detection catches patterns that deviate from normal use. Idempotency keys limit the damage of repeated invocations. The reviewer should map the guardrails that exist (or should exist) for each action type, and importing the skill should be conditional on the guardrails being in place before production use.
The recovery question is the often-missed one. When something goes wrong, how do you fix it? An action that writes to a temporary file is easy to recover from: delete the file. An action that sends an email is much harder to recover from: you cannot unsend. An action that makes a payment may be impossible to recover from: depending on the rails, the payment is final once made. The recovery cost should inform both the importer's risk tolerance and the design of the surrounding controls. High-recovery-cost actions deserve more conservative guardrails and slower default behavior.
The diff process for action types is the most important of the four dimensions, because action types are how the agent affects the world. A skill that does no actions, only computations, is much less consequential than one that does many actions. The diff makes the consequential additions visible and forces the team to confront them deliberately.
The Identity Claims Diff: Who Does It Speak As?
The identity claims diff is the dimension most reviewers miss. The question is: when the new skill interacts with downstream systems, what identity does it claim? In some systems, the skill acts as the agent. In others, the skill acts as the human user the agent is serving. In others, the skill acts as itself, with its own service identity. Mixing these identity claims across skills creates structural confusion that becomes hard to untangle when something goes wrong.
For every new identity the skill claims, the reviewer asks three questions. What identity is being claimed and where? Who is accountable in downstream audit logs? What permissions does that identity have, and are they appropriate?
The identity claim and location question forces the reviewer to be specific. "The skill acts as the user" is too vague; the reviewer needs to know that, for example, the skill makes Gmail API calls under the user's OAuth token, makes Slack API calls under the agent's bot token, and makes internal API calls under the skill's own service identity. Each of those identity choices has implications for who appears in audit logs, what permissions are exercised, and what the downstream systems see when something goes wrong. The reviewer's job is to make the identity choices visible.
The accountability question is about understanding the audit story. When the agent does something through the skill that ends up needing investigation, whose name is in the audit log of the downstream system? If the action was high-stakes, was it the user's name (which raises questions about whether the user actually authorized the action), the agent's name (which makes investigation easier but may not match what regulators or customers expect), or the skill's own name (which can leave the action looking orphaned, with no clear accountability path back to the human or system that initiated it). The right answer depends on context, but it should be a deliberate answer.
The permissions question is about whether the claimed identity has the right authorities for what the skill does. A skill that claims to act as the user but only needs to read public information is over-claiming identity in a way that creates audit noise. A skill that claims to act as a service identity but actually needs to operate on user-private data is under-claiming identity in a way that breaks the user's expected privacy. Both cases need correction before import, and both are common.
The diff process for identity claims should also catch the cases where the skill enables identity confusion attacks. A skill that lets the agent set the identity claim per call is dangerous because a malicious input could cause the agent to claim an unintended identity downstream. A skill that propagates user identity from one downstream system to another (without going through the proper identity propagation infrastructure described in the MCP trust boundary post) is dangerous because it creates ad-hoc identity bridges that bypass the controlled propagation path. Both patterns should be identified in review and either remediated or rejected.
The Conversation Pattern: Reviewer And Importer
Capability diff review is fundamentally a conversation between a reviewer and an importer. The diff produces facts; the conversation produces decisions. The pattern of the conversation matters because adversarial conversations make reviewers and importers into opponents, which is the wrong dynamic. The right dynamic is collaborative: both parties are trying to figure out whether the import is a good idea, and the diff is the structured input to a joint decision.
The productive opening is the importer presenting the case for the import. Why this skill, what alternatives were considered, what specific need is being addressed, how was the skill discovered. The importer's case forces them to articulate why they want this skill at all, which often surfaces that the importer has not considered alternatives, has not specified the need precisely, or has been led to the skill by something other than careful selection. The reviewer's first contribution is to listen, ask clarifying questions, and capture the importer's case as the foundation for the rest of the review.
The productive middle is walking through the four-dimension diff together. The reviewer presents what the diff shows. The importer explains what they understood the skill to do, where their understanding matches the diff, and where it does not. The mismatches are the most valuable parts of the conversation, because they reveal where the importer's mental model is wrong. Sometimes the mismatch is a misunderstanding the importer can correct. Sometimes the mismatch is a structural problem with the skill that should disqualify the import. The conversation distinguishes the two.
The productive close is a decision with documented rationale. Either the import proceeds (with any required guardrails or scope reductions agreed upon), the import is rejected (with the reasons captured), or the import is paused pending more information (with the open questions and the path to closing them documented). The decision is captured in the review template, which becomes the artifact for future reference. Three months later, when someone asks why this skill was approved or why a different version of the same skill is now being reconsidered, the artifact is the answer.
The anti-pattern to avoid is the rubber-stamp conversation, where the reviewer waves the import through without engaging the diff. Rubber-stamps happen when the reviewer is overworked, when the importer has a track record of careful imports, when the skill seems familiar, or when the time pressure is high. None of these are good reasons. Every import deserves the conversation, even when the diff looks small, because the small diffs are exactly where the surprises hide.
The 24-Hour Cooling Off Period
One specific operational pattern is worth lifting out as its own framework: the 24-hour cooling-off period for new capabilities. The pattern is that any new capability requested by an imported skill (a permission, a destination, an action type, an identity claim that does not appear in any existing approved skill) requires a written justification that survives a 24-hour cooling-off period before the import can be approved. The justification is written, reviewed, and then sat with overnight. If, the next day, the reviewer and importer both still believe the justification is sound, the import proceeds. If either has second thoughts, the import is paused for further consideration.
The 24-hour delay is not security theater. It is a deliberate friction calibrated to catch the cases where the import is being driven by enthusiasm or pressure rather than considered judgment. We have observed that approximately 15-20 percent of imports that look obvious in the moment look more questionable a day later, and the cases that look more questionable are usually the ones where the import would have caused regret. The friction is small (a one-day delay), the cost is bounded (most imports are not so urgent that 24 hours matters), and the benefit is the elimination of about a sixth of all bad imports that would have proceeded under a no-friction model.
The cooling-off period also has a second-order effect. Importers learn that new capabilities cause delays, and they start to prefer skills whose capabilities are already in the approved set. This creates a soft-gravity effect where the fleet's overall capability surface grows more slowly, because importers naturally prefer the familiar and have to spend specific energy to introduce the new. The bias toward consolidating the capability surface is exactly the bias you want in a security-conscious organization, and the cooling-off period creates it without anyone needing to articulate the policy.
The Skill Import Diff Review Template
The artifact this post promised is the Skill Import Diff Review Template, the structured document that captures the review for each import. The template has six sections, designed to be filled out in order, with the output being a single page that any future team member can read to understand what was approved and why.
Section one captures the import metadata: skill name, version, source repository, importer name, review date. The metadata sounds trivial but it is the basis for every later query: "who imported this and when" is a question we have asked many times during incidents, and the answer is much more useful when it is captured at import time rather than reconstructed from logs.
Section two captures the importer's case. The need being addressed, the alternatives considered, the reasons this skill was chosen over the alternatives. This section is written by the importer and reviewed by the reviewer; the conversation about it is the opening of the review.
Section three captures the four-dimension diff. New permissions, new network destinations, new action types, new identity claims. Each new item is listed with a one-line description. The diff is produced semi-automatically from the skill's manifest and runtime observation, with the reviewer responsible for verifying that the diff is complete and accurate.
Section four captures the analysis of new capabilities. For each new item from section three, the reviewer notes the worst-case misuse, the guardrails that exist or need to be added, and any scope reductions that have been negotiated with the skill author. The analysis is the substance of the review, and the conversation between reviewer and importer is what produces it.
Section five captures the decision. Approve, approve with conditions, reject, or pause for more information. Approval with conditions enumerates the conditions specifically: required guardrails, scope reductions, monitoring additions. Rejections enumerate the reasons. Pauses enumerate the open questions and the path to closing them.
Section six captures the cooling-off period commitment. The reviewer and importer both initial that they have understood the analysis and agree with the decision, with a timestamp 24 hours after the initial review. If either party withdraws or amends their position during the cooling-off period, the section captures the change.
The template fits on one page. It takes about thirty minutes to complete for a typical import. The artifact is stored alongside the skill registration in the registry, where it can be queried later. The discipline of completing the template per import is what turns capability diff review from a nice idea into an operational practice.
Counter-Argument: Doesn't This Slow Down Adoption?
The sharpest counter-argument is that capability diff review will slow down skill adoption to the point that the agent's value is undermined. The whole point of the modern skill ecosystem is that capabilities can be added quickly; if every addition requires a thirty-minute review and a 24-hour delay, the velocity advantage of the ecosystem is lost. Better to import freely, the argument goes, and accept the occasional incident as the cost of doing business.
The argument is structurally similar to the arguments against code review when code review was first introduced as a discipline, against dependency review when dependency review was first introduced, against pull request workflows when those were introduced. In each case, the friction was real, the velocity cost was real, and the eventual conclusion was that the friction was net positive: the bugs and vulnerabilities caught by review were worth more than the velocity given up. We expect capability diff review to follow the same arc. The teams that adopt the practice early will, at the cost of some velocity, build fleets that are noticeably more reliable and less incident-prone than the teams that import freely.
The second observation is that the velocity cost is bounded by the size of the diff. Skills whose capabilities are already familiar (no new permissions, no new destinations, no new action types, no new identities) have minimal review work, because there is nothing to discuss beyond the importer's case for the import. Most imports in mature fleets are like this, and they should pass through review quickly. The thirty-minute estimate is for imports that introduce new capabilities, which should be a minority of imports once the fleet has a meaningful capability surface.
The third observation is that the alternative is not no friction; the alternative is friction concentrated in incidents rather than spread across imports. The teams we work with that have the worst incident profiles are usually the teams that imported most freely. The teams with the best incident profiles are the teams that imported with discipline. The total time spent on agent reliability is roughly conserved; the question is whether you spend it preventatively in review or reactively in incident response. Preventative spending tends to be more pleasant.
What Armalo Does
Armalo's skill registry includes capability diff review as part of the certification process. Skills submitted for higher tier certification are diffed against the existing certified set, with the four-dimension diff produced automatically and presented to a reviewer alongside the skill author's stated rationale for any new capabilities. The 24-hour cooling-off period is built into the certification flow for skills that introduce new capabilities not present in any existing certified skill. The behavioral pacts that govern skill behavior reference the capabilities declared at certification, so any deviation from declared capabilities at runtime constitutes a pact violation. The composite score that drives certification tier includes capability discipline as a measured dimension: skills that consistently use only the capabilities they declare score higher, and skills that probe or attempt operations beyond their declared capabilities score lower. The Trust Oracle at /api/v1/trust/ exposes the capability set for each certified skill, so importers can do their own capability diff against any of the registry's certified options before deciding what to import.
FAQ
Should I review imports of first-party skills written by my own team? Yes. The capability diff is just as valuable for first-party skills as for third-party ones, because the question is not whether you trust the author but whether you understand what the skill actually does in your environment. First-party skills can introduce new capabilities just as third-party ones can, and the structural questions about appropriate scope are the same.
What happens if the importer pushes back on a rejection? The conversation continues. The reviewer's role is not to be an obstacle but to ensure the right decision is made. If the importer can address the reviewer's concerns (with additional information, with scope reductions, with new guardrails), the rejection can be revisited. If the importer cannot address the concerns, the rejection stands, and the importer should look for a different skill or accept that the need cannot be met within the current security posture.
Who should serve as reviewer? Someone with both security context and operational context. A pure security person tends to over-reject; a pure operator tends to under-reject. The right reviewer understands both the threat model and the cost of being wrong about the skill's necessity. In small teams the reviewer might be a senior engineer wearing a security hat; in larger teams it might be a dedicated security engineer who pairs with the importer.
How do I produce the four-dimension diff if the skill does not declare its destinations or actions clearly? Run it in a sandbox with monitoring. The runtime observation is the ground truth, regardless of what the skill declares. If you cannot run the skill in a monitored sandbox, you should not import it, because you have no way to verify what it actually does.
What about emergency imports where 24 hours of cooling-off is not feasible? The cooling-off period can be waived for genuine emergencies, with explicit waiver from a senior approver and a commitment to retroactive review within a defined window. Waivers should be rare and tracked, because frequent waivers indicate that the import-velocity expectations are misaligned with the security review capacity, which is its own problem to solve.
Can capability diff review be automated? Partially. The diff itself can be largely automated from manifests and runtime traces. The analysis of new capabilities, the conversation between reviewer and importer, and the decision-making cannot be automated meaningfully today, because they involve judgment about whether new authority is appropriate for your specific context. As LLM-based reviewers improve, more of the analysis may become automatable, but the human judgment loop will remain important for high-stakes imports for the foreseeable future.
What is the relationship between capability diff review and the Skill Capability Surface Map from the earlier post? The map is the artifact for any single skill: what does this skill do, in detail. The diff is the comparison: how does this new skill differ from what we already have. The two practices reinforce each other; teams that maintain capability surface maps for their skills find diff review much faster, because the maps make the comparison easy. Teams that do diff review thoroughly find that they end up producing capability surface maps as a side effect.
What if my fleet has no existing skills to diff against? Then every skill import is a baseline addition rather than a diff, and the review focuses on the appropriateness of every capability rather than the new ones. As your fleet grows, the diffs become more interesting because they show what is new. The early imports set the precedent for what "normal" capabilities look like, so they deserve particularly careful review.
Bottom Line
Capability diff review is the agent supply chain equivalent of code review, applied to the moment when a new skill enters your environment. The four dimensions (permissions, network destinations, action types, identity claims) catch the misalignments between what the importer thinks they are getting and what they are actually getting. The 24-hour cooling-off period catches the imports driven by enthusiasm rather than judgment. The Skill Import Diff Review Template makes the practice repeatable. The cost is thirty minutes per import; the benefit is a fleet whose capability surface grows deliberately rather than by accident, and whose incidents are concentrated in the cases where the review process itself failed rather than in the much larger set of cases where no review happened at all.
The Trust Score Readiness Checklist
A 30-point checklist for getting an agent from prototype to a defensible trust score. No fluff.
- 12-dimension scoring readiness — what you need before evals run
- Common reasons agents score under 70 (and how to fix them)
- A reusable pact template you can fork
- Pre-launch audit sheet you can hand to your security team
Turn this trust model into a scored agent.
Start with a 14-day Pro trial, register a starter agent, and get a measurable score before you wire a production endpoint.
Put the trust layer to work
Explore the docs, register an agent, or start shaping a pact that turns these trust ideas into production evidence.
Comments
Loading comments…