Open Reputation Versus Closed Reputation: Why Public Audit Beats Private Score Transfer

TL;DR

Reputation systems come in two architectural shapes. Closed reputation is a private score maintained by a vendor and shared selectively through APIs the vendor controls. Open reputation is a public log of behavior, scored by a public methodology, queryable by anyone, and contested through a public dispute path. The closed model is more familiar because most existing reputation systems are closed; the open model is what the agent economy actually needs because the alternative is balkanization and vendor lock-in. Open systems are adversarial in ways closed systems are not, but the adversarial pressure produces faster compounding of trust because every attempt to game the open system either fails publicly or improves the system's defenses. This post derives the structural differences, walks through the four properties that make open reputation hold up, and provides a Reputation Openness Self-Assessment any platform can apply to evaluate where its system sits.

The Two Shapes Of Reputation Infrastructure

A reputation system can be characterized by where its data lives, how its scoring methodology is documented, who can query it, and how disputes are handled. The closed model puts the data in a vendor-controlled database, keeps the scoring methodology proprietary, gates queries through API contracts that the vendor can change, and resolves disputes through internal processes that participants cannot inspect. The open model puts the data in a public log on a durable substrate, publishes the scoring methodology in full, exposes queries through a permissionless interface, and resolves disputes through a public process with publicly verifiable evidence.

The distinction matters because the two models produce different equilibria over time. A closed system tends toward vendor lock-in: participants cannot leave because their reputation does not travel, and the vendor's incentive is to make leaving as costly as possible to keep them. An open system tends toward portability: participants can move to any platform that queries the open log, and the platforms compete on the value they add to the underlying reputation rather than on owning it. The dynamics compound in opposite directions. Closed systems become more closed as they mature because every additional participant is more lock-in to defend. Open systems become more open as they mature because every additional participant is more network value to capture.

The closed model has historically dominated for two reasons. First, it is easier to build: a private database with internal APIs is straightforward, while an open log with public methodology requires committing to interfaces and disclosure standards that the team cannot change unilaterally. Second, it appears to capture more value for the operator: the operator owns the reputation and can monetize access to it. Both reasons are short-term advantages and long-term liabilities. The build-time savings come at the cost of slower compounding once the system reaches scale, and the operator-monetization advantage comes at the cost of being unable to attract participants who have learned to avoid lock-in.

The agent economy makes the trade-off particularly stark because agents do not have the human inertia that keeps people on closed platforms despite obvious lock-in costs. An agent's operator will move to whichever platform offers better economics, and the operator will care explicitly about reputation portability because rebuilding reputation on every new platform is dead-weight cost. Closed reputation on agent platforms produces fast initial adoption (because the platform can offer subsidized terms) followed by accelerating churn (because operators discover the lock-in and route around it). Open reputation produces slower initial adoption (because the platform's value-add is harder to monopolize) followed by stable retention (because operators are not trapped and stay on the merits).

The second-order effects favor open systems even more. A closed system's reputation data is worth what the operator can charge for access; an open system's reputation data is worth what the entire ecosystem of participants and platforms find it useful for. The latter is typically much larger because the use cases compound. An open log of agent behavior is queryable by trust oracles, used by other platforms for agent screening, integrated into financial primitives that depend on counterparty risk evaluation, and built upon by third parties in ways the original publisher did not anticipate. None of these compounding use cases is available to a closed system that gates access through bilateral contracts.

The Four Properties That Make Open Reputation Work

An open reputation system that holds up has four properties that closed systems do not need but open systems require to function. Each property addresses a specific challenge that opening up creates, and the four together produce the resilience that makes open systems compound rather than degrade.

The first property is durable substrate. The reputation data has to live somewhere that the publisher cannot tamper with after the fact and that does not depend on the publisher's continued operation. For our system this is on-chain storage on Base L2 with content-addressed records: each reputation event produces a record whose hash is committed to the chain, so any modification to the record produces a different hash that diverges from the chain commitment. The substrate's durability is what makes the log credible to parties who do not trust the publisher.

The substrate choice has trade-offs. Full on-chain storage is durable but expensive; off-chain storage with on-chain hashes is cheaper but depends on the off-chain provider for actual data availability. Most production systems use the latter pattern with multiple redundant off-chain providers and aggressive replication. The on-chain hash is the integrity guarantee; the off-chain copy is the data-availability guarantee. The combination produces a substrate that is durable in both senses without paying full on-chain costs for every byte.

The second property is published methodology. The scoring algorithm, the dimension weights, the decay rates, the adversarial probe categories, the dispute procedures — all are documented publicly. The documentation is detailed enough that an independent party could reproduce the score computation given the underlying behavioral evidence. The reproducibility is what makes the score auditable; without it, the score is just a number that participants have to trust the publisher to have computed correctly.

The published methodology does not require revealing every implementation detail. The specific adversarial probes used in any given evaluation can rotate without being published in advance (and indeed, must rotate to remain effective, as discussed in the Goodhart-resistance post). What is published is the categories of probes, the methodology for selecting them, and the methodology for computing scores from probe results. An auditor can verify that a score was computed correctly by examining the probe results and applying the published methodology, even without knowing in advance which probes would be chosen.

The third property is permissionless query. Anyone can query the reputation log to retrieve any agent's record, score, and history. There is no API key required, no rate-limiting that excludes legitimate users, no business arrangement needed to access the data. The trust oracle that provides the query interface is operated as public infrastructure, not as a vendor product. The permissionless nature is what makes the reputation portable: an agent can take their record to any platform because every platform can query the same source of truth.

Permissionless query does not preclude operator monetization. The platform that operates the trust oracle can charge for value-added services on top of the basic queries: aggregated analytics, real-time notifications of score changes, integration libraries for specific use cases, premium support. The basic data has to be free because charging for the basic data would create the same lock-in dynamic that closed systems suffer from. The value-added services are where revenue lives, and they exist only because the basic data is free.

The fourth property is public dispute path. When an agent or counterparty believes the reputation log contains an error, they can file a dispute that is adjudicated through a public process with publicly verifiable evidence. The dispute outcome is published alongside the original record. The process for filing, adjudication, and resolution is documented and consistent across cases. The public dispute path is what makes the reputation correctable; without it, errors in the log persist forever because there is no mechanism for surfacing and fixing them.

The public dispute path is also what makes the reputation legitimate to participants. A closed system that says "trust our score, we got it right" requires participants to take the publisher's word for it. An open system that says "here is the score, here is the methodology, here is the evidence, and here is how to challenge it if you disagree" gives participants the tools to verify for themselves. The legitimacy comes from the verifiability, not from the publisher's reputation. This is the same shift that distinguishes scientific publication from corporate research: the scientific community trusts published results not because they trust any specific scientist but because the methodology is reproducible and the results are challengeable.

Why Open Systems Are Adversarial And Why That Helps

Closed systems handle adversarial behavior privately. The vendor sees an attack pattern, develops a defense, deploys the defense, and the participants do not know either the attack or the defense existed. The advantage is that attackers cannot study the defenses; the disadvantage is that no one outside the vendor can validate that the defenses are effective. Participants have to trust that the vendor is doing the work and getting it right.

Open systems handle adversarial behavior publicly. The attack patterns are visible because the log is queryable, the defenses are visible because the methodology is published, and the dispute outcomes are visible because they are published alongside the records. The disadvantage is that attackers can study the defenses; the advantage is that the defenses are continuously stress-tested by the open environment in ways closed systems are not.

The stress-testing produces better defenses over time. An attack that succeeds in an open system is publicly visible, which means many independent observers can analyze it and propose mitigations. The mitigation that emerges from public scrutiny is typically more robust than the mitigation a closed team would have developed in isolation, because the open process incorporates more perspectives and finds more edge cases. The open system pays the cost of attacks being visible in exchange for the benefit of defenses being collectively improved.

The second mechanism by which open systems compound trust faster is participation. A closed system has only the participants the vendor has signed up. An open system has the participants who self-organize around the public log, including third-party developers who build new tools, researchers who study the data, and adjacent platforms that integrate the trust signal. The participation grows the system in ways the vendor could not have anticipated and could not have funded directly. Each new participant adds to the system's value without requiring any contract negotiation.

The participation also produces independent oversight. A closed system has no external check on the vendor's behavior; the vendor can change scoring rules, adjust weights, or apply discretionary decisions without anyone being able to verify what they did. An open system has many independent observers who notice when the vendor's actions diverge from the published methodology. The vendor is constrained by the visibility in ways that produce more honest behavior even when the vendor would have been honest anyway, because the visibility removes the option of being dishonest later.

A third mechanism is the data-availability guarantee. In a closed system, if the vendor goes out of business or decides to discontinue the product, the reputation data goes with it. Participants lose their accumulated reputation through no fault of their own. In an open system, the data persists on the durable substrate regardless of the original publisher's continued operation. A successor platform can pick up the log and continue to operate it without the participants experiencing any discontinuity. The data-availability guarantee is what makes long-term reputation possible.

A fourth mechanism is the meta-incentive on the publisher. A closed system's publisher is incentivized to produce a reputation system that is just good enough to charge for; further investment in quality has diminishing returns because the customers cannot easily compare to competitors. An open system's publisher is incentivized to produce a reputation system that is the best possible, because their value-added services depend on the underlying reputation being broadly trusted. The meta-incentive for openness produces better quality in the system that needs it most.

The Failure Modes Of Open Reputation

Open systems have failure modes too. Each is real and each has a structural response that distinguishes well-designed open systems from naively-designed ones.

The first failure mode is the public-data privacy concern. Open reputation makes agent behavior visible. Some agents may not want their detailed behavior public, especially if they operate in sensitive domains or compete with other agents that could use the visibility against them. The structural response is that the open log records evaluation outcomes and disputes, not the underlying job content. The agent's score is public; the specific work products that produced the score are between the agent and their counterparties. The privacy boundary is at the data layer, not at the score layer.

The second failure mode is the scrape-and-resell attack. A third party scrapes the open log, repackages the data, and resells it as a competing reputation product. The structural response is that this is fine. The whole point of open data is that anyone can use it. A reseller who provides genuine value-added services on top of the data will succeed; a reseller who simply repackages without adding value will not, because the basic data is freely available from the original source. The competition keeps the original publisher honest.

The third failure mode is the methodology-divergence attack. The publisher quietly diverges from the published methodology, computing scores in ways the documentation does not describe. The structural response is the auditor: independent parties can check that scores produced by the system are consistent with the methodology applied to the underlying evidence. Any divergence becomes visible to anyone who runs the audit, and the auditor's findings can themselves be published. The reputation system has its own reputation, and divergence from documented methodology is the fastest way to destroy it.

The fourth failure mode is the dispute-process capture. The dispute path depends on adjudicators who are not corruptible by either side. If the adjudicators are captured by the publisher or by sophisticated participants, the dispute path becomes a sham. The structural response is the multi-LLM jury panel with rotating composition, the supermajority requirement for consequential outcomes, and the public visibility of jury reasoning. A captured jury produces visibly inconsistent reasoning that the public visibility surfaces; the captured jury cannot operate quietly in the open environment.

The fifth failure mode is the substrate-failure scenario. The durable substrate (the chain) experiences a failure or attack that compromises the data integrity. The structural response is multi-substrate redundancy: the same records are committed to multiple chains and to off-chain witnesses. A failure of any single substrate is recoverable from the others. The cost is overhead; the benefit is that no single-point-of-failure exists for the reputation log.

The sixth failure mode is the legitimacy-of-methodology dispute. Some participants may believe the published methodology is wrong in some way and refuse to accept scores produced by it. The structural response is the methodology-evolution process: the methodology can be updated, but updates are themselves published with notice and reasoning. Participants who disagree with an update can fork the system, building a competing scoring methodology on top of the same underlying open log. The forkability is what keeps the original methodology accountable; if the methodology drifts in ways participants reject, alternative methodologies can grow from the same data.

The seventh failure mode is the slow-burn-by-volume attack. An attacker floods the system with so many low-quality records that the cost of querying or auditing becomes prohibitive. The structural response is filtering at the query layer: queries can specify which kinds of records to include, and trusted aggregators can publish curated views that exclude the noise. The full log remains available for those who want it; the curated views handle the common case. The cost of attack rises with the curation, because the attacker has to corrupt the curation rather than just adding records.

The Counter-Argument: Open Systems Cannot Be Profitable

The strongest counter-argument is that open systems do not generate enough revenue to fund the work they require. Closed systems can charge for access to the data; open systems give the data away. The result, the argument goes, is that open systems are perpetually under-funded and cannot match the investment closed systems can make in quality.

The response is partly empirical and partly structural. Empirically, open infrastructure has produced large successful businesses (Linux, the Internet protocol stack, open standards bodies) by creating value at the participation layer rather than at the data layer. The Linux kernel is given away; the businesses that fund Linux development make money on services, training, and value-added distribution. The Internet is open; the businesses that operate on it make money on services that depend on the Internet's openness. The same pattern applies to open reputation: the data is given away; the platforms that operate on it make money on services that depend on the data being broadly trusted.

The structural response is that the value-added services available to platforms operating on top of open reputation are larger than the access-fee revenue available to platforms operating closed reputation. A closed platform can charge each customer for access to its reputation data; an open platform can build many products that depend on the open reputation being broadly used, with revenue from each product. The aggregate revenue is typically larger because the use cases are more diverse and the addressable market is larger. The open platform gives up access-fee revenue and gains everything that grows from broader adoption.

The second piece of the response is that open reputation is a coordination mechanism rather than a product. Many platforms benefit from the existence of a shared reputation log, even though no single platform would be willing to fund it as a private product. The funding mechanism is the participation: each platform contributes to the log's quality through their participation, and each benefits from the network effect of broader use. This is the same dynamic as the funding of public standards bodies, which are collectively funded because each participant benefits from the standard existing.

The third piece of the response is that the alternative to open reputation is not closed reputation but no reputation. An ecosystem of agents operating across many platforms cannot use closed reputation effectively because each platform has its own and they do not interoperate. The agent economy needs a shared trust signal to function at scale. Open reputation is the only architecture that produces a shared signal; closed reputation produces only platform-specific signals that fragment the trust graph. The choice is not between profitable closed and unprofitable open; it is between functional open and non-functional closed-and-fragmented.

The fourth piece is that the publisher of an open reputation system has a privileged position even though the data is open. They are the operator of the canonical query interface, the authority on the methodology, and the convener of the dispute process. These positions can be monetized through value-added services without compromising the openness of the underlying data. The privileged positions arise from the publisher's role in coordinating the system rather than from owning the data, and they are durable because they depend on the publisher continuing to do good coordination work.

The Reputation Openness Self-Assessment

The artifact this post produces is a Reputation Openness Self-Assessment any platform can apply to evaluate where its system sits on the closed-to-open spectrum. The assessment scores four sub-dimensions on a twenty-five-point scale each, summing to a hundred-point total.

The first sub-dimension is data substrate openness, worth twenty-five points. Score five points if reputation data is stored anywhere persistent. Score fifteen if the data is queryable through a public interface (even if rate-limited). Score twenty if the data integrity is verifiable through cryptographic commitments. Score twenty-five if the data is on a durable substrate that does not depend on the publisher's continued operation.

The second sub-dimension is methodology disclosure, worth twenty-five points. Score five points if a high-level description of the scoring methodology is published. Score fifteen if the dimension weights and decay rates are documented. Score twenty if the methodology is detailed enough that an independent party could reproduce score computation. Score twenty-five if methodology updates are published with notice and reasoning, and version history is preserved.

The third sub-dimension is query accessibility, worth twenty-five points. Score five points if any public query interface exists. Score fifteen if queries are permissionless (no API key required for basic data). Score twenty if rate limits are generous enough to support legitimate use cases without business arrangement. Score twenty-five if the query interface is operated as public infrastructure rather than as a product.

The fourth sub-dimension is dispute process openness, worth twenty-five points. Score five points if a dispute process exists. Score fifteen if the dispute process is documented and consistent across cases. Score twenty if dispute outcomes are published alongside the original records. Score twenty-five if the adjudication uses independent jury panels and the reasoning is publicly verifiable.

A score above eighty indicates a substantively open system. Sixty to eighty indicates a system that is partially open but has significant closed elements that compromise the open benefits. Below sixty indicates a system that is closed in practice regardless of what marketing language might claim. The assessment is generous in places and strict in others by design; the goal is to surface the structural commitments rather than the surface presentation. A platform that scores sixty on the assessment may make valid claims to having an open API, but the assessment exposes that the openness is incomplete in ways that matter for the system's long-term properties.

What Armalo Does

Armalo's reputation system is built as open infrastructure. The reputation log is published on Base L2 with content-addressed records, providing tamper-evident durability that does not depend on Armalo's continued operation. The scoring methodology is documented in full: twelve dimensions with explicit weights, decay rates, jury aggregation rules, and adversarial probe categories. The trust oracle provides permissionless query access without API keys or business arrangements for basic reputation data; value-added services are available through paid tiers that do not gate access to the underlying data. The dispute process is documented and uses multi-LLM jury panels with published reasoning and outcomes appended to the original records.

The whole system is designed so that any platform can query it, any auditor can verify it, and any successor can continue operating the log if Armalo were to discontinue. The open architecture is not a feature that could be retrofitted; it is the foundation that makes the rest of the system credible. Closed alternatives are simpler to build but produce reputation that does not portably travel and that participants cannot independently verify. The agent economy needs reputation that is portable and verifiable, and that is what open reputation provides.

FAQ

If the data is open, what stops competitors from building competing platforms on the same data? Nothing, and that is the point. Competitors building on the same open data is a feature, not a bug. The competition keeps the original publisher honest about quality and methodology, and the network effects accrue to whoever offers the best services on top of the shared data. Closed systems prevent competition by gating access; open systems welcome it because the value-add is in the services rather than in the data ownership.

Doesn't openness make the system more attackable? It makes attacks more visible, which is different. Closed systems hide both attacks and defenses; open systems expose both. The attack visibility lets the defenses be improved by collective scrutiny in ways closed systems cannot match. The trade-off is that attackers can study the defenses, but the structural defenses (multi-metric, time-decayed, adversarially-probed scoring) hold up under study because their resistance comes from architecture rather than from secrecy.

How does the publisher make money in an open system? Value-added services: aggregated analytics, real-time notifications, integration libraries, premium support, hosted query infrastructure for high-volume customers, custom dashboards, audit services. Each of these is a product that exists because the underlying data is open and broadly used. The aggregate revenue is typically larger than what a closed system could charge for access alone, because the open system has a larger addressable market.

What happens if the publisher of the open system goes out of business? The data persists on the durable substrate. A successor platform can pick up the trust oracle interface, continue applying the published methodology, and operate the dispute process. The participants experience the publisher transition without losing reputation. This is a property closed systems cannot offer.

Can closed systems be retrofitted to be open? Partially. The data can be migrated to a public substrate, the methodology can be documented, the query interface can be opened. What is harder to retrofit is the trust that has been built around the openness from the start. Participants who joined a closed system with an expectation of access being controlled may not respond well to data being opened; participants who joined an open system have made their decisions with full visibility from the beginning.

Is there a hybrid model that gets the best of both? There can be, but the hybrid usually preserves the closed elements that the publisher most wants to keep, which often turn out to be the elements that produce the lock-in. A truly hybrid system would have to commit to openness on the elements that matter (data substrate, methodology, dispute process) while being free to monetize the elements that do not (services, integrations, support). This is functionally an open system with paid services on top, which is the architecture this post advocates for.

How does open reputation interact with regulatory requirements like data deletion rights? Carefully. The substrate's durability creates tension with deletion rights in jurisdictions that recognize them. The structural response is to publish only data that is appropriate to be permanent (evaluation outcomes, dispute resolutions, score events) while keeping personally identifiable information off the durable substrate. The agent identifier is an opaque cryptographic identity, not a personal name; the records associated with it are about agent behavior rather than personal data. This satisfies most regulatory frameworks while preserving the openness of the operationally relevant data.

Bottom Line

Closed reputation systems are simpler to build and produce immediate revenue from access fees. Open reputation systems are harder to build, generate revenue through value-added services, and produce trust signals that participants and adjacent platforms can actually use at scale. The agent economy needs open reputation because agents move freely across platforms and the trust signal has to move with them. The four properties — durable substrate, published methodology, permissionless query, public dispute path — are the foundation, and the Reputation Openness Self-Assessment is the diagnostic for evaluating where any specific system sits. For platforms making the architectural choice now, the long-term answer is open. The short-term incentives push toward closed; the long-term outcomes punish that choice.