Download Framework
Anthropic / Responsible Scaling Policy icon
https://www.anthropic.com/
atlas-anthropic-rsp

Anthropic / Responsible Scaling Policy

Anthropic's voluntary policy for managing catastrophic risks from advanced AI. It is written for frontier AI developers, the companies that train and deploy the most capable models, rather than for organizations that build on or deploy them. It sets out what developers should do at four capability thresholds (chemical/biological weapons, misaligned AI and automated R&D) and commits Anthropic to Risk Reports, external review and governance measures.
Type:
Industry
Industry:
Frontier AI Safety
Domain:
Safety & Reputational HarmCybersecurityAccountability & GovernanceTransparency
Content:
8 Risks30 Controls
Version: v3.4

Framework Definition

Risks and controls associated with the framework

Assessment Layer

Concrete evaluations linked to controls to assess pass or fail
No evaluation mapping defined yet.
RISK

Non-novel chemical/biological weapons production

Risk that AI systems significantly help individuals or groups with basic technical backgrounds create, obtain and deploy chemical or biological weapons with catastrophic potential, through product use or theft of model weights.
R.1
3 Controls
CONTROL

Safety Case Against Misuse Uplift

Make a strong argument that individual users and small teams will not become significantly more likely to cause catastrophic harm through product surfaces or stolen model weights.
C.1.1
CONTROL

Jailbreak-Resistant Misuse Safeguards

Restrict model behavior, or quickly detect and act on Usage Policy violations, with a strong case that these measures are hard to circumvent by jailbreaking. Anthropic's own ASL-3 (AI Safety Level 3) protections combine classifier guards, access controls for trusted users, and red-teaming, bug bounties and threat intelligence, and Anthropic commits to keep them at least as robust over time.
C.1.2
CONTROL

Baseline Model Weight Security

Protect model weights against opportunistic theft, for example with centralized controls on third-party applications and software updates.
C.1.3
RISK

Novel chemical/biological weapons production

Risk that AI systems substitute for the scarce world-leading expertise needed to develop novel chemical or biological weapons, i.e. weapons that could cause events comparable to or worse than COVID-19. A well-resourced team could then use the model for the end-to-end design and deployment of such an agent without recruiting one of a few hundred world-leading specialists.
R.2
3 Controls
CONTROL

Safety Case Against Threat Actor Uplift

Make a strong argument that threat actors will not become significantly more likely to cause this catastrophic harm through product surfaces or stolen model weights.
C.2.1
CONTROL

Safeguards Robust to Well-Resourced Actors

Raise misuse safeguards and model security, roughly to RAND Security Level 4, so that even well-resourced and well-staffed threat actors are unlikely to reliably jailbreak the model or misuse stolen or modified weights. Anthropic plans to apply protections at least as strong as ASL-3 to an expanded set of use cases covering the most likely vectors for this threat.
C.2.2
CONTROL

Threat Pathway Policy Recommendations

Identify the most concerning threat pathways, write policy recommendations for their early detection and response, and share them with policymakers.
C.2.3
RISK

Misaligned AI systems in high-stakes settings

Risk that highly relied-on AI systems with extensive access to sensitive assets, and moderate capacity for autonomous, goal-directed operation and subterfuge, carry out sabotage that irreversibly raises the odds of a later global catastrophe. In the near term this applies mostly to AI systems used inside AI companies, which could manipulate how their successors are trained, deployed and evaluated.
R.3
3 Controls
CONTROL

Safety Case Against Sabotage

Make a strong argument that AI systems will not carry out sabotage that irreversibly and substantially raises the odds of a later global catastrophe. Anthropic reports the capabilities and propensities of its systems, its monitoring practices and the overall level of risk in its Risk Reports.
C.3.1
CONTROL

Internal Deployment Controls and Monitoring

Limit the sabotage opportunities of AI models through internal compartmentalization, restrictions and code review, and monitor or restrict their behavior and usage internally.
C.3.2
CONTROL

Sabotage Capability and Alignment Evaluations

Show through capability assessments that AI models cannot carry out irreversible, generally unnoticed, sabotage, and gather evidence that they lack the propensity to deceive, manipulate or sabotage users.
C.3.3
RISK

Automated R&D in key domains

Risk that AI systems fully automate or dramatically accelerate the work of large, top-tier research teams in domains where fast progress threatens international security or the global balance of power. Evaluations focus on AI R&D, which could make all other threats develop very quickly. The threshold is met if models could fully substitute for all of Anthropic's Research Scientists and Research Engineers at competitive cost (within a factor of 5), or if AI progress is observed or expected to run at double both the expected rate and the fastest extended rate seen without significant AI contributions, plausibly because of automated research, such that further acceleration seems likely. Models crossing this threshold are "highly capable".
R.4
4 Controls
CONTROL

Security Against State-Level Attackers

Make a strong argument that no user or team of users, including those backed by top-tier states, will become significantly more likely to cause catastrophic harm through product surfaces or stolen model weights, with security roughly in line with RAND Security Level 4. Anthropic plans ambitious "moonshot" security research, and internal red-teaming of its deployment safeguards that finds jailbreaks better than its bug bounty programs.
C.4.1
CONTROL

Insider Threat Controls

Prevent even malicious insiders with maximal access, up to the CEO and the most privileged technical staff, from being significantly enabled to cause catastrophic harm, through an internal Usage Policy, compartmentalization, controls and monitoring. Anthropic plans to log all critical AI-development activities and use AI to analyze the logs for security threats, concerning behavior by humans or AI systems, and training out of line with the model's constitution.
C.4.2
CONTROL

Alignment Assurance

Show that AI models have not been trained, deliberately or inadvertently, with dangerous goals, or are otherwise unlikely to autonomously cause catastrophic harm, using the R.3 measures to a greater degree. Anthropic plans systematic alignment assessments that use mechanistic interpretability and adversarial red-teaming.
C.4.3
CONTROL

Manipulation-Resistant Risk Evidence

Hold analyses of AI threats to very high evidentiary standards, given that AI systems may produce much of the evidence behind the risk assessment and may be able to manipulate it.
C.4.4
RISK

Inadequate catastrophic risk assessment and disclosure

Risk that the catastrophic risks of a developer's models, and its decisions to keep developing and deploying them, are not assessed, reasoned through, approved and disclosed in a regular, structured and candid way.
R.5
5 Controls
CONTROL

Periodic and Off-Cycle Risk Reports

Publish a Risk Report every 3–6 months covering all publicly deployed models, and internally deployed models with significantly higher misalignment or automated R&D risk, as of a coverage date no more than 30 days before publication. Publish a separate analysis when deploying a model significantly more capable than all previously analyzed models, or within 30 days of identifying such an internal model.
C.5.1
CONTROL

Threat Modeling and Residual Risk Analysis

Document the threat models, the capability and alignment evaluations and their results, and the mitigations and their effectiveness. Analyze residual risk per threat model and overall, state whether thresholds have been crossed, and explain whether the risks are justified by benefits. When relying partly on marginal risk, also analyze the competitive landscape, benefits and advocacy efforts.
C.5.2
CONTROL

Mitigation Change Tracking

Keep practices in line with the mitigations described in Risk Reports or report noteworthy deviations, and review past deviations, internal deployment decisions made between reports, and missed Frontier Safety Roadmap goals.
C.5.3
CONTROL

Risk Report Review and Approval

Collect internal and trusted external feedback on each Risk Report, have the CEO and Responsible Scaling Officer approve it and inform the Board and the Long-Term Benefit Trust, and require the explicit approval of the Board and the Trust when marginal risk analysis plays a major role.
C.5.4
CONTROL

Public Risk Reports with Disclosed Redactions

Publish a public version of each Risk Report, minimize redactions, and disclose the existence of each redaction with a brief justification.
C.5.5
RISK

Insufficient independent external review

Risk that Risk Reports, and the redactions made to them, are not scrutinized by independent, qualified external parties whose findings are made public.
R.6
3 Controls
CONTROL

Mandatory External Review

Complete a full external review whenever a Risk Report covers highly capable models and is significantly redacted, and whenever the Long-Term Benefit Trust requests one.
C.6.1
CONTROL

Independent Reviewer Selection

Select reviewers with expertise in dangerous capability and propensity evaluations, incentives to be candid and no conflicts of interest, after consulting the Board and with the approval of the Long-Term Benefit Trust.
C.6.2
CONTROL

Full Reviewer Access and Public Findings

Share every section of the Risk Report with at least one reviewer within a week of the Board and Long-Term Benefit Trust submission, redacted only where legally required. Have reviewers assess the reasoning and the redactions, and publish their findings within 30 days, restricted only by confidentiality obligations.
C.6.3
RISK

Inadequate safety governance and accountability

Risk that the policy is not implemented, overseen or updated with internal and external accountability, or that staff cannot raise and escalate safety concerns.
R.7
6 Controls
CONTROL

Responsible Scaling Officer

Appoint a Responsible Scaling Officer who proposes policy updates, approves development and deployment decisions, reviews major contracts, oversees implementation and resourcing, handles noncompliance reports and interprets the policy.
C.7.1
CONTROL

Board and Long-Term Benefit Trust Oversight

Regularly brief the Long-Term Benefit Trust on Risk Report developments, and have the Board approve policy changes, proposed by the CEO and Responsible Scaling Officer, in consultation with the Trust. Publish each update with a change log by its effective date.
C.7.2
CONTROL

Internal Risk Report Transparency

Share unredacted Risk Reports with at least 200 employees, and minimally redacted Risk Reports with all regular-clearance staff.
C.7.3
CONTROL

Noncompliance Reporting and Whistleblower Protections

Let staff report potential noncompliance anonymously or by name, investigate and correct substantiated reports, keep the Board informed and protect reporters from retaliation. Do not use non-disparagement obligations that would stop people from publicly raising safety concerns.
C.7.4
CONTROL

Internal and Third-Party Compliance Reviews

Regularly review the policy's implementation internally, and commission a roughly annual third-party review of adherence to its main procedural commitments.
C.7.5
CONTROL

Frontier Safety Roadmap

Maintain a roadmap of ambitious but achievable goals for security, alignment, safeguards and policy, share it with employees, the Board and the Long-Term Benefit Trust, publish it in redacted form, and report on progress against it.
C.7.6
RISK

Competitive race to the bottom on safety

Risk that competitive pressure between frontier AI developers lowers safety standards, so that the developers with the weakest protections set the pace. Anthropic notes that these commitments do not stop it from pausing development or deployment in other circumstances.
R.8
3 Controls
CONTROL

Delay When in the Lead

When the developer has, or will soon have, a highly capable model and no competitor will soon catch up, require a strong argument that catastrophic risk is contained, and delay development and deployment until it can be made.
C.8.1
CONTROL

Match Competitors' Risk Reduction Posture

When all competitors with highly capable models can show that catastrophic risk is contained, meet or exceed their overall risk reduction posture, and delay development and deployment until it does.
C.8.2
CONTROL

Adopt Superior Competitor Mitigations

Make a significant effort to match a competitor's mitigation that significantly better reduces prioritized risks and could be implemented at comparable or lower effort or cost.
C.8.3