Distillation Is Not an Open-Source Problem

Anthropic has identified a legitimate control problem. If competing AI laboratories created thousands of deceptive accounts to generate millions of targeted model interactions, the activity goes well beyond an ordinary customer learning from a model’s responses. The scale, coordination, and alleged efforts to evade controls warrant investigation.

However, characterizing this primarily as an open-source problem confuses the method of access with the eventual distribution of the resulting model.

The alleged extraction took place through Anthropic’s closed service. The actors did not obtain Claude’s weights or source code. They allegedly used controlled access to generate a large body of prompts and responses that could be used to improve another model. Open-weight availability was not the vulnerability that enabled the activity. The immediate control boundary was Anthropic’s identity, account, payment, access, and behavioral-monitoring layer.

Open distribution may affect the consequences. Once an illicitly distilled model is released as open weight, the transferred capabilities can be downloaded, modified, and deployed by other parties. The original model developer cannot revoke access or ensure that safeguards remain intact. That can increase the potential scale and persistence of harm.

But amplification is not causation. Conflating the two risks produces policy that restricts the downstream distribution model without correcting the upstream access failure. This is especially important because distillation is foundational to the AI market. Model developers use it to reduce inference costs, improve smaller models, transfer specialized behaviors, and make advanced capabilities practical for edge and enterprise environments. A policy that treats distillation itself as equivalent to theft could create uncertainty across a broad range of legitimate development practices.

The more difficult question is where legitimate learning ends and unauthorized capability extraction begins. Terms of service may prohibit using outputs to train a competing model, but contractual restrictions are not identical to intellectual property law. Model outputs can reflect learned capabilities without containing a direct copy of the teacher model’s code, weights, or training data. Policymakers and courts will need to determine which conduct constitutes contractual abuse, circumvention, misappropriation, infringement, or lawful competitive development.

That analysis requires evidence. Similar model behavior does not, by itself, prove illicit distillation. A strong enforcement framework should consider access records, coordinated account activity, payment and infrastructure linkages, prompt patterns, technical model evidence, and demonstrable attempts to conceal the identity or location of the user.

The industry should also be careful about creating an asymmetric standard. Frontier-model providers built their own systems using large volumes of publicly available, licensed, and disputed training data. Those companies have a valid interest in protecting access to their services, but they will face scrutiny if they attempt to define model outputs as protected intellectual property more broadly than the rights they recognized when acquiring their own training data.

The answer is not to dismiss Anthropic’s allegations. It is to define the misconduct precisely and design controls around it.

Moving the Control Boundary to the Access Layer

Large-scale model extraction is fundamentally an abuse-detection challenge. Traditional per-account controls are insufficient when one coordinated actor can distribute activity across thousands of accounts, providers, payment methods, proxies, and geographic locations.

Stronger identity verification can help, particularly before customers receive high-volume API access. However, identity verification alone will not stop sophisticated organizations able to use intermediaries, stolen identities, shell companies, or third-party resellers. Excessive verification requirements could also create privacy concerns and unnecessary barriers for legitimate developers.

The control strategy must combine identity with behavior.

• Customer and organizational verification: Providers should apply proportionate checks before granting high-volume access, elevated rate limits, or access to their most capable models.
• Linked-account detection: Account analysis should identify shared payment methods, infrastructure, devices, timing patterns, prompt structures, and other indicators of coordinated activity.
• Workload-level monitoring: Providers must evaluate behavior across clusters of accounts rather than assuming that each account represents an independent user.
• Capability-extraction detection: Repetitive prompts focused on reasoning traces, grading, coding, tool use, or other differentiated capabilities may indicate systematic training-data generation.
• Adaptive rate controls: Limits should respond to aggregate behavioral signals, not only the volume generated by a single account.
• Model and output provenance: Watermarking, fingerprinting, and other technical evidence may help establish whether another model learned from protected outputs, although these methods will not be conclusive in every case.
• Cross-provider intelligence: Model providers, cloud platforms, payment processors, and API intermediaries may need mechanisms to share indicators of coordinated abuse.
• Contractual and legal enforcement: When the evidence supports attribution, providers should pursue the organizations responsible rather than relying solely on technical blocking.

These controls will not eliminate model extraction. A sufficiently determined actor may still collect outputs through distributed channels or use legitimate user traffic obtained from third parties. The objective is to increase the cost, reduce the scale, improve attribution, and establish clearer evidence for enforcement.

This is an AI stack issue because the security boundary does not stop at the model. Providers need visibility across identity, API gateways, account relationships, payment systems, request behavior, infrastructure, and model telemetry. A conventional rate limit applied to one API key cannot detect an extraction campaign operating across thousands of apparently unrelated identities.

Protecting Open Models Without Excusing Misconduct

Defending open models should not require dismissing unauthorized extraction. Open-model developers should be able to demonstrate how their models were trained, identify major teacher models where applicable, and explain the legal and contractual basis for using generated outputs.

Better provenance would help distinguish developers engaging in legitimate distillation from organizations concealing industrial-scale extraction. It could also reduce the tendency to treat every capable open model as presumptively stolen simply because it improves quickly or competes with a proprietary system.

At the same time, provenance cannot become a requirement that only the largest companies can satisfy. Full disclosure of training recipes, datasets, and model-development techniques may expose legitimate trade secrets and create compliance costs that favor incumbents. Requirements should focus on material sources, risk-relevant information, and evidence necessary to investigate specific allegations.

Open-weight status should affect the risk assessment, not determine guilt. A model may be developed lawfully and released openly. Another may be illicitly distilled and kept proprietary. The licensing model does not establish whether the underlying development process was legitimate.

The same principle should apply to safety. Open release can limit the developer’s ability to revoke access or preserve safeguards, but proprietary access does not guarantee safe behavior. Both open and closed models require evaluation based on their capabilities, the environment in which they operate, and the potential scale of misuse.