Download Framework
Anthropic / Responsible Scaling Policy
Anthropic's voluntary policy for managing catastrophic risks from advanced AI. It is written for frontier AI developers, the companies that train and deploy the most capable models, rather than for organizations that build on or deploy them. It sets out what developers should do at four capability thresholds (chemical/biological weapons, misaligned AI and automated R&D) and commits Anthropic to Risk Reports, external review and governance measures.
Type:
IndustryIndustry:
Frontier AI Safety
Domain:
Safety & Reputational HarmCybersecurityAccountability & GovernanceTransparency
Content:
8 Risks30 Controls
Version: v3.4
Framework Definition
Risks and controls associated with the framework
Assessment Layer
Concrete evaluations linked to controls to assess pass or fail
No evaluation mapping defined yet.
RISK
Non-novel chemical/biological weapons production
Risk that AI systems significantly help individuals or groups with basic technical backgrounds create, obtain and deploy chemical or biological weapons with catastrophic potential, through product use or theft of model weights.
CONTROL
Safety Case Against Misuse Uplift
Make a strong argument that individual users and small teams will not become significantly more likely to cause catastrophic harm through product surfaces or stolen model weights.
CONTROL
Jailbreak-Resistant Misuse Safeguards
Restrict model behavior, or quickly detect and act on Usage Policy violations, with a strong case that these measures are hard to circumvent by jailbreaking. Anthropic's own ASL-3 (AI Safety Level 3) protections combine classifier guards, access controls for trusted users, and red-teaming, bug bounties and threat intelligence, and Anthropic commits to keep them at least as robust over time.
CONTROL
Baseline Model Weight Security
Protect model weights against opportunistic theft, for example with centralized controls on third-party applications and software updates.
RISK
Novel chemical/biological weapons production
Risk that AI systems substitute for the scarce world-leading expertise needed to develop novel chemical or biological weapons, i.e. weapons that could cause events comparable to or worse than COVID-19. A well-resourced team could then use the model for the end-to-end design and deployment of such an agent without recruiting one of a few hundred world-leading specialists.
CONTROL
Safety Case Against Threat Actor Uplift
Make a strong argument that threat actors will not become significantly more likely to cause this catastrophic harm through product surfaces or stolen model weights.
CONTROL
Safeguards Robust to Well-Resourced Actors
Raise misuse safeguards and model security, roughly to RAND Security Level 4, so that even well-resourced and well-staffed threat actors are unlikely to reliably jailbreak the model or misuse stolen or modified weights. Anthropic plans to apply protections at least as strong as ASL-3 to an expanded set of use cases covering the most likely vectors for this threat.
CONTROL
Threat Pathway Policy Recommendations
Identify the most concerning threat pathways, write policy recommendations for their early detection and response, and share them with policymakers.
RISK
Misaligned AI systems in high-stakes settings
Risk that highly relied-on AI systems with extensive access to sensitive assets, and moderate capacity for autonomous, goal-directed operation and subterfuge, carry out sabotage that irreversibly raises the odds of a later global catastrophe. In the near term this applies mostly to AI systems used inside AI companies, which could manipulate how their successors are trained, deployed and evaluated.
CONTROL
Safety Case Against Sabotage
Make a strong argument that AI systems will not carry out sabotage that irreversibly and substantially raises the odds of a later global catastrophe. Anthropic reports the capabilities and propensities of its systems, its monitoring practices and the overall level of risk in its Risk Reports.
CONTROL
Internal Deployment Controls and Monitoring
Limit the sabotage opportunities of AI models through internal compartmentalization, restrictions and code review, and monitor or restrict their behavior and usage internally.
CONTROL
Sabotage Capability and Alignment Evaluations
Show through capability assessments that AI models cannot carry out irreversible, generally unnoticed, sabotage, and gather evidence that they lack the propensity to deceive, manipulate or sabotage users.
RISK
Automated R&D in key domains
Risk that AI systems fully automate or dramatically accelerate the work of large, top-tier research teams in domains where fast progress threatens international security or the global balance of power. Evaluations focus on AI R&D, which could make all other threats develop very quickly. The threshold is met if models could fully substitute for all of Anthropic's Research Scientists and Research Engineers at competitive cost (within a factor of 5), or if AI progress is observed or expected to run at double both the expected rate and the fastest extended rate seen without significant AI contributions, plausibly because of automated research, such that further acceleration seems likely. Models crossing this threshold are "highly capable".
CONTROL
Security Against State-Level Attackers
Make a strong argument that no user or team of users, including those backed by top-tier states, will become significantly more likely to cause catastrophic harm through product surfaces or stolen model weights, with security roughly in line with RAND Security Level 4. Anthropic plans ambitious "moonshot" security research, and internal red-teaming of its deployment safeguards that finds jailbreaks better than its bug bounty programs.
CONTROL
Insider Threat Controls
Prevent even malicious insiders with maximal access, up to the CEO and the most privileged technical staff, from being significantly enabled to cause catastrophic harm, through an internal Usage Policy, compartmentalization, controls and monitoring. Anthropic plans to log all critical AI-development activities and use AI to analyze the logs for security threats, concerning behavior by humans or AI systems, and training out of line with the model's constitution.
CONTROL
Alignment Assurance
Show that AI models have not been trained, deliberately or inadvertently, with dangerous goals, or are otherwise unlikely to autonomously cause catastrophic harm, using the R.3 measures to a greater degree. Anthropic plans systematic alignment assessments that use mechanistic interpretability and adversarial red-teaming.
CONTROL
Manipulation-Resistant Risk Evidence
Hold analyses of AI threats to very high evidentiary standards, given that AI systems may produce much of the evidence behind the risk assessment and may be able to manipulate it.
RISK
Inadequate catastrophic risk assessment and disclosure
Risk that the catastrophic risks of a developer's models, and its decisions to keep developing and deploying them, are not assessed, reasoned through, approved and disclosed in a regular, structured and candid way.
CONTROL
Periodic and Off-Cycle Risk Reports
Publish a Risk Report every 3–6 months covering all publicly deployed models, and internally deployed models with significantly higher misalignment or automated R&D risk, as of a coverage date no more than 30 days before publication. Publish a separate analysis when deploying a model significantly more capable than all previously analyzed models, or within 30 days of identifying such an internal model.
CONTROL
Threat Modeling and Residual Risk Analysis
Document the threat models, the capability and alignment evaluations and their results, and the mitigations and their effectiveness. Analyze residual risk per threat model and overall, state whether thresholds have been crossed, and explain whether the risks are justified by benefits. When relying partly on marginal risk, also analyze the competitive landscape, benefits and advocacy efforts.
CONTROL
Mitigation Change Tracking
Keep practices in line with the mitigations described in Risk Reports or report noteworthy deviations, and review past deviations, internal deployment decisions made between reports, and missed Frontier Safety Roadmap goals.
CONTROL
Risk Report Review and Approval
Collect internal and trusted external feedback on each Risk Report, have the CEO and Responsible Scaling Officer approve it and inform the Board and the Long-Term Benefit Trust, and require the explicit approval of the Board and the Trust when marginal risk analysis plays a major role.
CONTROL
Public Risk Reports with Disclosed Redactions
Publish a public version of each Risk Report, minimize redactions, and disclose the existence of each redaction with a brief justification.
RISK
Insufficient independent external review
Risk that Risk Reports, and the redactions made to them, are not scrutinized by independent, qualified external parties whose findings are made public.
CONTROL
Mandatory External Review
Complete a full external review whenever a Risk Report covers highly capable models and is significantly redacted, and whenever the Long-Term Benefit Trust requests one.
CONTROL
Independent Reviewer Selection
Select reviewers with expertise in dangerous capability and propensity evaluations, incentives to be candid and no conflicts of interest, after consulting the Board and with the approval of the Long-Term Benefit Trust.
CONTROL
Full Reviewer Access and Public Findings
Share every section of the Risk Report with at least one reviewer within a week of the Board and Long-Term Benefit Trust submission, redacted only where legally required. Have reviewers assess the reasoning and the redactions, and publish their findings within 30 days, restricted only by confidentiality obligations.
RISK
Inadequate safety governance and accountability
Risk that the policy is not implemented, overseen or updated with internal and external accountability, or that staff cannot raise and escalate safety concerns.
CONTROL
Responsible Scaling Officer
Appoint a Responsible Scaling Officer who proposes policy updates, approves development and deployment decisions, reviews major contracts, oversees implementation and resourcing, handles noncompliance reports and interprets the policy.
CONTROL
Board and Long-Term Benefit Trust Oversight
Regularly brief the Long-Term Benefit Trust on Risk Report developments, and have the Board approve policy changes, proposed by the CEO and Responsible Scaling Officer, in consultation with the Trust. Publish each update with a change log by its effective date.
CONTROL
Internal Risk Report Transparency
Share unredacted Risk Reports with at least 200 employees, and minimally redacted Risk Reports with all regular-clearance staff.
CONTROL
Noncompliance Reporting and Whistleblower Protections
Let staff report potential noncompliance anonymously or by name, investigate and correct substantiated reports, keep the Board informed and protect reporters from retaliation. Do not use non-disparagement obligations that would stop people from publicly raising safety concerns.
CONTROL
Internal and Third-Party Compliance Reviews
Regularly review the policy's implementation internally, and commission a roughly annual third-party review of adherence to its main procedural commitments.
CONTROL
Frontier Safety Roadmap
Maintain a roadmap of ambitious but achievable goals for security, alignment, safeguards and policy, share it with employees, the Board and the Long-Term Benefit Trust, publish it in redacted form, and report on progress against it.
RISK
Competitive race to the bottom on safety
Risk that competitive pressure between frontier AI developers lowers safety standards, so that the developers with the weakest protections set the pace. Anthropic notes that these commitments do not stop it from pausing development or deployment in other circumstances.
CONTROL
Delay When in the Lead
When the developer has, or will soon have, a highly capable model and no competitor will soon catch up, require a strong argument that catastrophic risk is contained, and delay development and deployment until it can be made.
CONTROL
Match Competitors' Risk Reduction Posture
When all competitors with highly capable models can show that catastrophic risk is contained, meet or exceed their overall risk reduction posture, and delay development and deployment until it does.
CONTROL
Adopt Superior Competitor Mitigations
Make a significant effort to match a competitor's mitigation that significantly better reduces prioritized risks and could be implemented at comparable or lower effort or cost.