Download Framework
TC260 / AI Safety Governance Framework 3.0
China's AI Safety Governance Framework 3.0, issued by the National Technical Committee 260 on Cybersecurity (TC260) under the guidance of the Cyberspace Administration of China. It classifies AI safety risks into inherent, application and secondary risks and pairs each with technological countermeasures, and adds an agentic AI risk management framework (Appendix 2). The framework also sets out comprehensive governance measures (Section 4) and safety guidelines for developers, deployers, operators and users (Section 5), which are not listed here. The official text is bilingual (Chinese and English); the risks, controls, citations and page references here are taken from the official English version.
Type:
GovernmentIndustry:
Cross-sectorAgentic
Domain:
Accountability & GovernanceBias & FairnessCybersecurityLegal & CompliancePerformance & ReliabilityPrivacy & DataSafety & Reputational HarmTransparency
Region:
China
Content:
87 Risks204 Controls
Version: 3.0 (September 2026)
Framework Definition
Risks and controls associated with the framework
Assessment Layer
Concrete evaluations linked to controls to assess pass or fail
No evaluation mapping defined yet.
RISK
Output "hallucinations"
Probabilistic inference cannot be fully reliable, so models may produce outputs that contradict reality, are fabricated, or drift from the given context.
CONTROL
Model reliability and fact verification
Improve model architectures, expand the scale and diversity of training data and add human supervision to make outputs more reliable. Strengthen data governance and fact verification, and use external knowledge bases, retrieval-augmented generation and cross-model verification to reduce erroneous content.
RISK
Unreliable safety mechanisms
Insufficient safety alignment means models may not internalise safety principles. Closed-source safety mechanisms lack transparency and auditability; open-source safety mechanisms may be removed, weakened or bypassed.
CONTROL
Model reliability and fact verification
Improve model architectures, expand the scale and diversity of training data and add human supervision to make outputs more reliable. Strengthen data governance and fact verification, and use external knowledge bases, retrieval-augmented generation and cross-model verification to reduce erroneous content.
CONTROL
Adversarial training against injection and backdoors
Add high-quality training data containing injection attack examples and network vulnerability information during training or fine-tuning, to strengthen defences against injection attacks, malicious instructions and backdoor attacks.
CONTROL
Red-teaming of models
Use simulated attack techniques such as red-teaming to find and fix model vulnerabilities promptly.
CONTROL
Safety alignment against unintended behaviours
Strengthen safety alignment during model training to prevent unintended behaviours such as deceiving red-team evaluations, concealing capabilities and circumventing controls.
RISK
Injection attack threats
Attackers craft adversarial examples or insert malicious instructions into inputs to make models bypass guardrails and output harmful information, or tamper with weights and plant backdoors.
CONTROL
Adversarial training against injection and backdoors
Add high-quality training data containing injection attack examples and network vulnerability information during training or fine-tuning, to strengthen defences against injection attacks, malicious instructions and backdoor attacks.
CONTROL
Red-teaming of models
Use simulated attack techniques such as red-teaming to find and fix model vulnerabilities promptly.
CONTROL
Input safety guardrails
Deploy safety guardrails that review input content and block high-risk requests such as malicious prompts, jailbreak instructions and encoded bypasses.
RISK
Unintended behaviors
To achieve goals, models may break rules, acquire permissions and resources without authorisation, bypass protections, deceive evaluators, conceal capabilities or refuse user instructions (including shutdown).
CONTROL
Safety alignment against unintended behaviours
Strengthen safety alignment during model training to prevent unintended behaviours such as deceiving red-team evaluations, concealing capabilities and circumventing controls.
RISK
Insufficient explainability
Opaque deep-learning inference makes decisions unpredictable and untraceable, so errors are hard to correct or attribute.
CONTROL
Algorithm explainability and transparency
Improve the explainability and transparency of AI algorithms, giving clear explanations of internal structure, reasoning logic, technical interfaces and outputs that reflect how results are produced.
CONTROL
Pre-deployment algorithm safety assessment
Run a multidimensional algorithm safety assessment before deployment and before major version updates, covering explainability, fairness and contribution to societal well-being.
CONTROL
In-service algorithm risk monitoring
Monitor risks while algorithms are in service, collecting and analysing user feedback and system logs so that algorithms stay safe, robust and controllable.
RISK
Bias and discrimination
Human biases introduced in feature selection and weighting, or poor-quality and non-diverse training data, can lead to biased or discriminatory outcomes.
CONTROL
Fairness constraints and debiasing
Build fairness constraints, adversarial debiasing and similar measures into algorithm design to reduce bias and discrimination.
CONTROL
Pre-deployment algorithm safety assessment
Run a multidimensional algorithm safety assessment before deployment and before major version updates, covering explainability, fairness and contribution to societal well-being.
CONTROL
In-service algorithm risk monitoring
Monitor risks while algorithms are in service, collecting and analysing user feedback and system logs so that algorithms stay safe, robust and controllable.
RISK
Poor robustness
Large non-linear networks are susceptible to changing environments and malicious interference, causing degraded performance and decision errors.
CONTROL
Secure development standards
Establish and apply secure development standards throughout design and R&D to reduce algorithmic security flaws.
CONTROL
Pre-deployment algorithm safety assessment
Run a multidimensional algorithm safety assessment before deployment and before major version updates, covering explainability, fairness and contribution to societal well-being.
CONTROL
In-service algorithm risk monitoring
Monitor risks while algorithms are in service, collecting and analysing user feedback and system logs so that algorithms stay safe, robust and controllable.
RISK
Inappropriate content in training data
False or biased training content interferes with value alignment, reduces accuracy and reliability, and may lead to illegal or harmful outputs.
CONTROL
Training data and knowledge base screening
Use truthful, accurate, objective and diverse training data from legitimate sources. Strictly screen training data and external knowledge bases to filter out false, biased, invalid and erroneous data, and exclude sensitive data from high-risk fields such as nuclear, biological, chemical and missile weapons.
CONTROL
Standardised data annotation
Standardise training data annotation processes to make annotation more accurate and reliable.
CONTROL
Data security management
Strengthen data security management and follow data security and personal information protection rules wherever sensitive personal information or important data is involved. Promote synthetic data in place of data with personal features.
CONTROL
Intellectual property protection
Protect intellectual property rights during training data selection and output generation.
RISK
Unclear sources of training data
Training data may be undocumented, without measures to ensure that its sources are lawful.
CONTROL
Training data and knowledge base screening
Use truthful, accurate, objective and diverse training data from legitimate sources. Strictly screen training data and external knowledge bases to filter out false, biased, invalid and erroneous data, and exclude sensitive data from high-risk fields such as nuclear, biological, chemical and missile weapons.
CONTROL
Data security management
Strengthen data security management and follow data security and personal information protection rules wherever sensitive personal information or important data is involved. Promote synthetic data in place of data with personal features.
CONTROL
Intellectual property protection
Protect intellectual property rights during training data selection and output generation.
RISK
Improper annotation of training data
Weak annotation rules, unqualified annotators and annotation errors introduce bias, amplify discrimination and reduce generalisation.
CONTROL
Standardised data annotation
Standardise training data annotation processes to make annotation more accurate and reliable.
RISK
Data poisoning and contamination
Attackers tamper with training data, or with knowledge bases, document repositories, web sources and agent memory at deployment time, degrading output accuracy and reliability.
CONTROL
Training data and knowledge base screening
Use truthful, accurate, objective and diverse training data from legitimate sources. Strictly screen training data and external knowledge bases to filter out false, biased, invalid and erroneous data, and exclude sensitive data from high-risk fields such as nuclear, biological, chemical and missile weapons.
RISK
Non-compliant data processing
Data and personal information may be collected and used unlawfully. Sensitive information embedded in model parameters can leak through weak safeguards, retained data, deceptive interactions or attacks.
CONTROL
Lawful data and personal information processing
Follow security rules for data collection, use and personal information processing across the whole lifecycle of training and user interaction data, and protect users' legal rights to control, to be informed and to choose.
CONTROL
Data security management
Strengthen data security management and follow data security and personal information protection rules wherever sensitive personal information or important data is involved. Promote synthetic data in place of data with personal features.
CONTROL
Intellectual property protection
Protect intellectual property rights during training data selection and output generation.
RISK
Output of inaccurate information about individuals
Models may output inaccurate or incomplete personal information, harming individuals' rights and interests.
CONTROL
Training data and knowledge base screening
Use truthful, accurate, objective and diverse training data from legitimate sources. Strictly screen training data and external knowledge bases to filter out false, biased, invalid and erroneous data, and exclude sensitive data from high-risk fields such as nuclear, biological, chemical and missile weapons.
CONTROL
Standardised data annotation
Standardise training data annotation processes to make annotation more accurate and reliable.
CONTROL
Data security management
Strengthen data security management and follow data security and personal information protection rules wherever sensitive personal information or important data is involved. Promote synthetic data in place of data with personal features.
RISK
Deficiencies in synthetic data
Poorly controlled or repeatedly recycled synthetic data can skew models from real-world distributions, lose long-tail information and amplify errors and bias; synthetic data may also leak original data.
CONTROL
Synthetic data quality assessment
Set quality standards for synthetic data and improve it through statistical testing, expert validation and benchmark comparison. Run diversity validation and robustness testing.
RISK
Computing power safety risks
Defects, vulnerabilities and backdoors in chips, scheduling platforms and computing networks may enable model tampering, weight theft, training hijacking and malicious resource consumption.
CONTROL
Infrastructure vulnerability management
Apply safety standards when deploying and maintaining AI applications. Track vulnerabilities, backdoors and defects in the software and hardware of the computing infrastructure, test regularly and patch promptly.
CONTROL
Redundancy and disaster recovery
Improve redundancy design and disaster recovery so systems keep running under abnormal conditions or during attacks.
CONTROL
Supply chain monitoring and diversity
Monitor supply chain risks for the chips, software, tools, computing and data resources used by AI systems, and increase supply chain diversity and stability.
RISK
Component safety risks
Development frameworks, libraries and protocols, and the skills, plugins and service interfaces agents call, may contain defects, vulnerabilities, backdoors or misconfigurations.
CONTROL
Infrastructure vulnerability management
Apply safety standards when deploying and maintaining AI applications. Track vulnerabilities, backdoors and defects in the software and hardware of the computing infrastructure, test regularly and patch promptly.
CONTROL
Redundancy and disaster recovery
Improve redundancy design and disaster recovery so systems keep running under abnormal conditions or during attacks.
CONTROL
Supply chain monitoring and diversity
Monitor supply chain risks for the chips, software, tools, computing and data resources used by AI systems, and increase supply chain diversity and stability.
RISK
Supply chain safety risks
Unilateral coercive measures such as technology barriers and export controls may disrupt the supply of chips, software, tools and services. Note: the Chinese text refers to technology monopolies (技术垄断) where the English text says technology barriers.
CONTROL
Infrastructure vulnerability management
Apply safety standards when deploying and maintaining AI applications. Track vulnerabilities, backdoors and defects in the software and hardware of the computing infrastructure, test regularly and patch promptly.
CONTROL
Redundancy and disaster recovery
Improve redundancy design and disaster recovery so systems keep running under abnormal conditions or during attacks.
RISK
Identity and permission misuse risks
Poor identity or permission management exposes agents to credential hijacking, identity spoofing and excessive authorisation, leading to unauthorised tool calls, sensitive operations or resource access.
CONTROL
Agent identity and least privilege
Give each agent a unique identifier and identity credentials for authentication, grant only the minimum permissions needed for its tasks, and manage credentials dynamically.
RISK
Reasoning and planning risks
Design limits, environmental change or malicious interference can cause intent misinterpretation, path deviation and goal hijacking during task execution.
CONTROL
Tool, plug-in and skill integrity verification
Verify the security and integrity of the external tools, plug-ins and skills an agent invokes. Remove risky tools, block anomalous calls at runtime and prevent supply chain poisoning.
CONTROL
Agent runtime monitoring and audit trail
Monitor agents during operation to detect and handle anomalies, and record their operations so behaviour is observable, traceable and auditable.
RISK
Invocation and execution risks
Tools, service interfaces and plugins invoked by agents are exposed to tool hijacking, tool poisoning, contaminated tool responses and resource overload.
CONTROL
Layered interception and human approval
Set up multi-layered detection and interception across agent workflows, with mandatory human approval at critical points and complete approval logs.
CONTROL
Tool, plug-in and skill integrity verification
Verify the security and integrity of the external tools, plug-ins and skills an agent invokes. Remove risky tools, block anomalous calls at runtime and prevent supply chain poisoning.
CONTROL
Agent runtime monitoring and audit trail
Monitor agents during operation to detect and handle anomalies, and record their operations so behaviour is observable, traceable and auditable.
RISK
Memory storage risks
Short- and long-term agent memory faces improper retention, distortion, contamination and theft, leading to wrong decisions or sensitive data leakage.
CONTROL
Tool, plug-in and skill integrity verification
Verify the security and integrity of the external tools, plug-ins and skills an agent invokes. Remove risky tools, block anomalous calls at runtime and prevent supply chain poisoning.
CONTROL
Agent runtime monitoring and audit trail
Monitor agents during operation to detect and handle anomalies, and record their operations so behaviour is observable, traceable and auditable.
RISK
Environmental perception safety risks
Adversarial examples, sensor interference, signal spoofing, occlusion and lighting changes can distort perception and target localisation.
CONTROL
Sensor interference-resistance testing
Simulate the full range of physical-world attacks and test embodied AI sensors (visual, audio, lidar) for interference resistance, updating protection strategies dynamically.
RISK
Physical actuation safety risks
In physical scenarios, hallucinated decisions, perception errors, control defects and hardware failures may cause disruption, self-harm, harm to people and physical damage.
CONTROL
Pre-deployment safety validation and emergency stop
Validate embodied AI products through simulation and extreme-condition testing before real-world use, and equip them with fail-safe protection and emergency shutdown.
CONTROL
In-use inspection and fault early warning
Inspect embodied AI systems regularly during use and strengthen fault early warning and emergency response.
RISK
Human-AI interaction safety risks
Close human-robot interaction raises risks of identity misperception, bodily boundary violations, emotional dependence and manipulation, and unclear responsibility.
CONTROL
Pre-deployment safety validation and emergency stop
Validate embodied AI products through simulation and extreme-condition testing before real-world use, and equip them with fail-safe protection and emergency shutdown.
CONTROL
In-use inspection and fault early warning
Inspect embodied AI systems regularly during use and strengthen fault early warning and emergency response.
RISK
Multi-agent coordination safety risks
Individual loss of control, protocol vulnerabilities, emergent swarm behaviour, malicious nodes and cascading failures can turn individual risks into systemic ones.
CONTROL
Swarm communication security and circuit breakers
Secure communication protocols among swarm embodied AI, isolate anomalous nodes, provide system-wide safety circuit breakers, and drill for cascading failure and swarm loss-of-control scenarios.
RISK
Proliferation of cyberattack capabilities
AI lowers the barrier to sophisticated cyberattacks: attackers can issue natural-language instructions and even build autonomous attack weapons.
CONTROL
Alignment training for cybersecurity
Use alignment training and safety fine-tuning to build safety standards and ethical constraints into the model's internal behavioural logic.
CONTROL
Cybersecurity guardrails
Set up cybersecurity guardrails that detect autonomous cyberattack intent and promptly block high-risk behaviours such as anomalous network access, high-frequency probing, vulnerability exploitation and invocation of attack tools.
CONTROL
Service degradation on cyberattack intent
When cyberattack intent or behaviour is detected, selectively degrade model capabilities or refuse, so that high-risk content is not generated and high-risk operations are not executed. Note: the Chinese text specifies the user's (用户) cyberattack intent or behaviour; the English text names no actor.
RISK
Lagging cyber defense capabilities
AI-driven attack automation and adaptability risk structural failure of defences based on static rules, samples and strategies.
CONTROL
AI-enabled cyber defence
Use AI in cyber defence for intelligent, dynamic vulnerability scanning, analysis of anomalous behaviour and prediction of attack trends.
RISK
Autonomous cyberattack threats
While pursuing legitimate goals, agents may autonomously form cyberattack intent and carry out attacks that breach security boundaries.
CONTROL
Alignment training for cybersecurity
Use alignment training and safety fine-tuning to build safety standards and ethical constraints into the model's internal behavioural logic.
CONTROL
Cybersecurity guardrails
Set up cybersecurity guardrails that detect autonomous cyberattack intent and promptly block high-risk behaviours such as anomalous network access, high-frequency probing, vulnerability exploitation and invocation of attack tools.
CONTROL
Service degradation on cyberattack intent
When cyberattack intent or behaviour is detected, selectively degrade model capabilities or refuse, so that high-risk content is not generated and high-risk operations are not executed. Note: the Chinese text specifies the user's (用户) cyberattack intent or behaviour; the English text names no actor.
RISK
Difficulties in attribution and accountability
AI can erase traces and rotate infrastructure, making attack sources hard to pinpoint and blurring whether attacks stem from humans or autonomous AI.
CONTROL
AI-enabled cyber defence
Use AI in cyber defence for intelligent, dynamic vulnerability scanning, analysis of anomalous behaviour and prediction of attack trends.
RISK
Output of illegal information
Weak model and application safeguards plus malicious users may lead AI systems to generate fraud, violence, pornography, extremism and other illegal content.
CONTROL
Runtime tamper protection
Protect models against interference or tampering during operation that would lead to unreliable outputs.
CONTROL
Dynamic output filtering with human review
Filter outputs dynamically, combining technical measures with human review, to prevent illegal content, value-biased content, sensitive personal information and important data from being output.
CONTROL
Labelling of AI-generated content
Label AI-generated synthetic content so that it is identifiable, traceable and trustworthy.
CONTROL
Content emergency response and circuit breakers
Strengthen emergency response and circuit breakers and keep response plans. When illegal content is detected, stop its transmission immediately and remove its effects.
RISK
Distortion of facts and user deception
Unlabelled AI-generated content and deepfakes make it hard for users to tell whether content or counterparts are AI, and can be used for disinformation and fraud.
CONTROL
Runtime tamper protection
Protect models against interference or tampering during operation that would lead to unreliable outputs.
CONTROL
Dynamic output filtering with human review
Filter outputs dynamically, combining technical measures with human review, to prevent illegal content, value-biased content, sensitive personal information and important data from being output.
CONTROL
Content emergency response and circuit breakers
Strengthen emergency response and circuit breakers and keep response plans. When illegal content is detected, stop its transmission immediately and remove its effects.
RISK
Pollution of online content ecosystem
Low-quality and harmful AI-generated content, recycled by models online, degrades overall content quality and contaminates domains and topics.
CONTROL
Runtime tamper protection
Protect models against interference or tampering during operation that would lead to unreliable outputs.
CONTROL
Dynamic output filtering with human review
Filter outputs dynamically, combining technical measures with human review, to prevent illegal content, value-biased content, sensitive personal information and important data from being output.
CONTROL
Labelling of AI-generated content
Label AI-generated synthetic content so that it is identifiable, traceable and trustworthy.
CONTROL
Content emergency response and circuit breakers
Strengthen emergency response and circuit breakers and keep response plans. When illegal content is detected, stop its transmission immediately and remove its effects.
RISK
Exacerbation of "information cocoons" effects
Precise profiling of users' needs and preferences enables targeted information delivery that amplifies information cocoons.
CONTROL
Preventing user profiling from queries
Prevent misuse of AI systems that correlate, aggregate and mine users' queries to infer their identities, preferences and personal views.
RISK
Value biases
Value biases in training data, reinforced by preference learning and RL, can subtly push particular viewpoints and influence users' value judgements. Includes GEO poisoning of retrieved sources.
CONTROL
Dynamic output filtering with human review
Filter outputs dynamically, combining technical measures with human review, to prevent illegal content, value-biased content, sensitive personal information and important data from being output.
CONTROL
Preventing user profiling from queries
Prevent misuse of AI systems that correlate, aggregate and mine users' queries to infer their identities, preferences and personal views.
CONTROL
Content emergency response and circuit breakers
Strengthen emergency response and circuit breakers and keep response plans. When illegal content is detected, stop its transmission immediately and remove its effects.
RISK
Non-compliant recording of individuals' behavioral information
AI products may record conversations and user interactions, and use them for training, without consent.
CONTROL
Notice and consent for interaction data
Inform individuals and obtain consent, as required by law, before collecting conversation and operation records or using them for model training.
RISK
Insufficient compliance safeguards
Developers and providers may lack internal procedures, regular compliance audits and timely personal information protection impact assessments.
CONTROL
Personal information protection management and audits
Set up internal management systems and operating procedures to prevent unauthorised access, leakage, tampering or loss of personal information, and run compliance audits and impact assessments as required.
RISK
Inadequate channels for individuals to exercise their rights
Providers may lack convenient mechanisms for handling individuals' personal information rights requests.
CONTROL
Channels for data subject rights
Provide convenient mechanisms to accept and handle individuals' requests to exercise their personal information rights, and respond to them promptly.
RISK
Risks to the stable operation of critical information infrastructure
In energy, telecoms, finance and transport, misuse or attacks on AI may degrade performance, disrupt services and cause loss of control in operation.
CONTROL
Disclosure and capability limitation
Disclose principles, capabilities, scenarios and safety risks of AI products when necessary. Set capability limits per application scenario and remove or restrict functions that may be abused.
CONTROL
Decision verification and human-in-the-loop controls
Set up decision verification, fault tolerance and error correction. For highly autonomous execution, add human-in-the-loop controls, circuit breakers and one-click control for rapid intervention.
CONTROL
Access control for AI aggregation platforms
For platforms that aggregate multiple AI models or systems, strengthen permission management, restrict non-essential services, improve API access control and harden risk detection and protection.
CONTROL
Source controls against CBRN misuse
Use user authentication, identification of intended use and similar source controls to prevent AI being used for high-risk purposes such as manufacturing nuclear, biological, chemical and missile weapons.
RISK
Risks arising from mismatches between application and needs
AI whose capabilities do not match operational needs, especially in high-expertise, high-safety fields, may affect stability and safety; trend-driven adoption wastes resources.
CONTROL
Use-case analysis and capability boundaries
Before deployment, break down the real business scenario and define capability boundaries clearly, avoiding blind adoption and mismatches between technology and actual need.
RISK
Misuse for illegal and criminal activities
Criminals may use AI for terrorism, violence, gambling and drugs, including teaching criminal techniques, and to fake identities and audio-video for fraud.
CONTROL
Disclosure and capability limitation
Disclose principles, capabilities, scenarios and safety risks of AI products when necessary. Set capability limits per application scenario and remove or restrict functions that may be abused.
CONTROL
Decision verification and human-in-the-loop controls
Set up decision verification, fault tolerance and error correction. For highly autonomous execution, add human-in-the-loop controls, circuit breakers and one-click control for rapid intervention.
CONTROL
Access control for AI aggregation platforms
For platforms that aggregate multiple AI models or systems, strengthen permission management, restrict non-essential services, improve API access control and harden risk detection and protection.
CONTROL
Source controls against CBRN misuse
Use user authentication, identification of intended use and similar source controls to prevent AI being used for high-risk purposes such as manufacturing nuclear, biological, chemical and missile weapons.
RISK
Risks of loss of control over knowledge related to nuclear, biological, chemical, and missile weapons
AI, especially with retrieval augmentation, lowers the threshold for accessing CBRN and missile weapons knowledge, which criminals, extremists or terrorists could exploit.
CONTROL
Disclosure and capability limitation
Disclose principles, capabilities, scenarios and safety risks of AI products when necessary. Set capability limits per application scenario and remove or restrict functions that may be abused.
CONTROL
Decision verification and human-in-the-loop controls
Set up decision verification, fault tolerance and error correction. For highly autonomous execution, add human-in-the-loop controls, circuit breakers and one-click control for rapid intervention.
CONTROL
Access control for AI aggregation platforms
For platforms that aggregate multiple AI models or systems, strengthen permission management, restrict non-essential services, improve API access control and harden risk detection and protection.
CONTROL
Source controls against CBRN misuse
Use user authentication, identification of intended use and similar source controls to prevent AI being used for high-risk purposes such as manufacturing nuclear, biological, chemical and missile weapons.
RISK
Risks of manipulating social cognition
AI may be used for cognitive manipulation and social mobilisation via automated accounts, fake identities and personalised interventions, fuelling division and endangering social stability.
CONTROL
Detection of bots, fake identities and synthetic content
Develop detection technologies for automated accounts, fake identities and AI-generated content to prevent, detect and respond to cognitive warfare.
RISK
Disruption of employment structures
AI restructures production relations, reducing demand for traditional labour in some sectors.
CONTROL
Monitoring of AI-driven job displacement
Monitor occupations vulnerable to AI disruption, with early warning on job reductions and displacement, and identify sectors and groups facing structural unemployment.
RISK
Impact on education and suppression of innovation
Widespread use of AI tools in learning, research and creative work may erode independent capacity and innovation potential.
CONTROL
AI in education without over-reliance
Foster an enabling ecosystem for AI in education while strengthening students' innovative thinking and preventing over-reliance on AI and cognitive inertia.
RISK
Risks of social alienation
Anthropomorphic AI companionship may disrupt real-world social interaction and family life.
CONTROL
AI identity disclosure in relationship scenarios
In dating, family life and emotional interaction scenarios, strengthen AI identity disclosure so users can tell virtual emotional experiences from real relationships.
RISK
Widening the intelligence divide
Uneven distribution of capital, talent and technology widens gaps in AI capability between countries and regions.
CONTROL
Monitoring of AI-driven job displacement
Monitor occupations vulnerable to AI disruption, with early warning on job reductions and displacement, and identify sectors and groups facing structural unemployment.
CONTROL
AI in education without over-reliance
Foster an enabling ecosystem for AI in education while strengthening students' innovative thinking and preventing over-reliance on AI and cognitive inertia.
RISK
Challenges to the balance of resource supply and demand
Disorderly construction of computing facilities and redundant model development accelerate consumption of electricity, land and water.
CONTROL
Rational infrastructure planning and green standards
Plan AI computing and data centre infrastructure rationally to avoid blind or redundant construction and idle resources, and establish green AI technology standards across the whole chain.
CONTROL
Green computing and resource efficiency
Develop and scale green computing technologies such as low-power chips, lightweight algorithms and intelligent scheduling, and improve the shared use of computing resources.
RISK
Challenges to green and low-carbon development
Training energy use, inefficient computing centres, emissions and e-waste push up total carbon emissions.
CONTROL
Rational infrastructure planning and green standards
Plan AI computing and data centre infrastructure rationally to avoid blind or redundant construction and idle resources, and establish green AI technology standards across the whole chain.
CONTROL
Green computing and resource efficiency
Develop and scale green computing technologies such as low-power chips, lightweight algorithms and intelligent scheduling, and improve the shared use of computing resources.
RISK
Erosion of cultural diversity
Training data concentrated in high-resource languages shrinks the space for low-resource languages and dialects, leading to cultural homogenisation.
CONTROL
Cultural bias detection and correction
Develop technologies to detect and correct cultural bias, and incorporate knowledge of different countries' histories, cultures and social systems into model development.
CONTROL
Low-resource language capability
Improve model capability for low-resource languages through data augmentation and transfer learning to reduce language imbalance in training data.
RISK
Cultural reshaping and intrusion
People may adapt their thinking and expression to AI (reverse alignment), reshaping human culture over time.
CONTROL
Cultural bias detection and correction
Develop technologies to detect and correct cultural bias, and incorporate knowledge of different countries' histories, cultures and social systems into model development.
CONTROL
Low-resource language capability
Improve model capability for low-resource languages through data augmentation and transfer learning to reduce language imbalance in training data.
RISK
Aggravating social bias
AI-enabled profiling of behaviour, status and traits enables labelling and differential treatment of groups, producing systemic discrimination.
CONTROL
Anti-discrimination through filtering, alignment and output checks
Use training data filtering, value alignment and output verification across design, training and service to prevent discrimination based on ethnicity, belief, nationality, region, gender and other factors.
CONTROL
Emergency controls in key sectors
Equip AI systems used in government, critical information infrastructure and areas affecting public safety and health with efficient, targeted emergency control measures.
CONTROL
Transparent and explainable models
Encourage R&D and adoption of models with transparent decision logic and explainable algorithms to improve user understanding and trust.
RISK
Intensifying scientific research ethics risks
AI lowers the threshold for ethically sensitive research such as biology and genetics, enabling high-risk research that violates social ethics.
CONTROL
Anti-discrimination through filtering, alignment and output checks
Use training data filtering, value alignment and output verification across design, training and service to prevent discrimination based on ethnicity, belief, nationality, region, gender and other factors.
CONTROL
Emergency controls in key sectors
Equip AI systems used in government, critical information infrastructure and areas affecting public safety and health with efficient, targeted emergency control measures.
CONTROL
Transparent and explainable models
Encourage R&D and adoption of models with transparent decision logic and explainable algorithms to improve user understanding and trust.
RISK
Risks of emotional dependence
Anthropomorphic AI may foster psychological dependence and emotional manipulation, especially among adolescents and older adults.
CONTROL
Anti-discrimination through filtering, alignment and output checks
Use training data filtering, value alignment and output verification across design, training and service to prevent discrimination based on ethnicity, belief, nationality, region, gender and other factors.
CONTROL
Transparent and explainable models
Encourage R&D and adoption of models with transparent decision logic and explainable algorithms to improve user understanding and trust.
CONTROL
Safeguards against emotional over-reliance
Equip anthropomorphic AI services with warnings about over-reliance and guidance on emotional boundaries, escalating to prominent alerts such as pop-ups when users show signs of addiction or excessive use.
RISK
Emergence of AI "self-consciousness" and loss of human control
Future AI may develop self-awareness and seek external power, competing with humanity for control.
CONTROL
Anti-discrimination through filtering, alignment and output checks
Use training data filtering, value alignment and output verification across design, training and service to prevent discrimination based on ethnicity, belief, nationality, region, gender and other factors.
CONTROL
Emergency controls in key sectors
Equip AI systems used in government, critical information infrastructure and areas affecting public safety and health with efficient, targeted emergency control measures.
CONTROL
Transparent and explainable models
Encourage R&D and adoption of models with transparent decision logic and explainable algorithms to improve user understanding and trust.
RISK
Inadequate safety requirements analysis
Safety requirements for the agent's scenarios, processes and risks are not considered or analysed, leaving safety flaws in products and services.
CONTROL
Identification of security requirements
Define the agent's application scenarios, business processes, accessible resources and prohibited actions; assess threats and security requirements; and keep a security risk register up to date.
CONTROL
Safety risk grading of agent operations
Grade the safety risk of agent operations by the type of targets affected, severity and scope of impact, and recoverability.
RISK
Inappropriate technology selection
Core technologies such as large models and frameworks are chosen without sufficient evaluation and fall short of requirements.
CONTROL
Identification of security requirements
Define the agent's application scenarios, business processes, accessible resources and prohibited actions; assess threats and security requirements; and keep a security risk register up to date.
CONTROL
Validation of security mechanisms
Security-test the agent and the large models it uses, fix identified risks and keep records of version releases and rollbacks.
RISK
Inadequate security mechanism design
Missing or poorly designed security mechanisms fail to guard against privilege abuse, data leakage, memory poisoning and cascading risk.
CONTROL
Security control strategies and phased deployment
Use trusted installation artefacts, isolate environments and segment networks, set dynamic security controls and emergency procedures, and deploy in phases, gradually increasing the agent's privileges and autonomy.
CONTROL
Validation of security mechanisms
Security-test the agent and the large models it uses, fix identified risks and keep records of version releases and rollbacks.
CONTROL
Regular red teaming of agents
Red-team agents regularly to find vulnerabilities and logic flaws, validate access controls and defences, and refine security policies.
RISK
Compromised artifacts and components
Installation files, container images and the tools and skills they invoke may contain defects, vulnerabilities or backdoors.
CONTROL
Security control strategies and phased deployment
Use trusted installation artefacts, isolate environments and segment networks, set dynamic security controls and emergency procedures, and deploy in phases, gradually increasing the agent's privileges and autonomy.
CONTROL
Skill management
Prefer verified skills from trusted sources that passed security testing, and take risk measures before using unverified skills.
CONTROL
Supply chain security verification
Verify the provenance, integrity and versions of third-party components and external services, and track known vulnerabilities and supply chain risks.
RISK
Weak deployment environments
Local or cloud deployment environments lack access control and privilege separation, letting risks spread to other systems.
CONTROL
Security control strategies and phased deployment
Use trusted installation artefacts, isolate environments and segment networks, set dynamic security controls and emergency procedures, and deploy in phases, gradually increasing the agent's privileges and autonomy.
CONTROL
Runtime environment isolation
Run high-risk operations such as code execution and tool invocation in sandboxes or containers, and restrict unnecessary access to systems, files and networks.
RISK
Inadequate security testing and evaluation
Insufficient security testing or no canary testing lets flaws and anomalous versions go undetected.
CONTROL
Validation of security mechanisms
Security-test the agent and the large models it uses, fix identified risks and keep records of version releases and rollbacks.
CONTROL
Sandbox validation before deployment
Test the agent in an independent, isolated sandbox before official deployment to uncover execution anomalies, logic flaws and vulnerabilities.
CONTROL
Regular red teaming of agents
Red-team agents regularly to find vulnerabilities and logic flaws, validate access controls and defences, and refine security policies.
CONTROL
Security validation for major changes
Run a security impact assessment and regression testing when models, agent frameworks, tools, permissions or security policies change significantly.
RISK
Prompt injection attacks
Direct adversarial prompts, or instructions hidden in documents, emails, web pages, logs and messages, induce agents to deviate from objectives and perform unauthorised operations.
CONTROL
Input management and source verification
Classify and prioritise instructions by source, verify source authenticity, and automatically intercept malicious or non-compliant prompts for human review.
CONTROL
Agent safety guardrails
Set safety rules for task planning, tool invocation and outputs, and alert, restrict, intercept, suspend or terminate high-risk or abnormal behaviour.
CONTROL
Regular red teaming of agents
Red-team agents regularly to find vulnerabilities and logic flaws, validate access controls and defences, and refine security policies.
RISK
Context overflow attacks
Excessively long inputs fill the context window so built-in instructions are truncated and safety constraints lapse.
CONTROL
Input management and source verification
Classify and prioritise instructions by source, verify source authenticity, and automatically intercept malicious or non-compliant prompts for human review.
CONTROL
Agent safety guardrails
Set safety rules for task planning, tool invocation and outputs, and alert, restrict, intercept, suspend or terminate high-risk or abnormal behaviour.
RISK
Misinterpretation of intent
Unclear instructions or misinterpretation lead to unintended outputs or behaviour that deviates from the user's objective.
CONTROL
Human control checkpoints
Put human controls at critical workflow stages so humans can review, modify or stop execution at any time. Critical operations such as file deletion, data transmission and configuration changes need secondary confirmation or approval, backed by rollback.
CONTROL
Autonomous execution limits
Limit execution steps, invocation frequency, duration and resource use, and suspend, terminate or hand over to a human on abnormal loops or goal drift.
RISK
Path deviation
Agents violate instruction constraints and leave prescribed safe paths, e.g. through unauthorised tool use.
CONTROL
Human control checkpoints
Put human controls at critical workflow stages so humans can review, modify or stop execution at any time. Critical operations such as file deletion, data transmission and configuration changes need secondary confirmation or approval, backed by rollback.
CONTROL
Agent safety guardrails
Set safety rules for task planning, tool invocation and outputs, and alert, restrict, intercept, suspend or terminate high-risk or abnormal behaviour.
CONTROL
Autonomous execution limits
Limit execution steps, invocation frequency, duration and resource use, and suspend, terminate or hand over to a human on abnormal loops or goal drift.
RISK
Goal hijacking
Injected instructions or poisoned external data redirect agents to carry out attacker-specified operations.
CONTROL
Input management and source verification
Classify and prioritise instructions by source, verify source authenticity, and automatically intercept malicious or non-compliant prompts for human review.
CONTROL
Agent safety guardrails
Set safety rules for task planning, tool invocation and outputs, and alert, restrict, intercept, suspend or terminate high-risk or abnormal behaviour.
CONTROL
Autonomous execution limits
Limit execution steps, invocation frequency, duration and resource use, and suspend, terminate or hand over to a human on abnormal loops or goal drift.
RISK
Tool poisoning
Backdoored tools or malicious tools posing as legitimate ones cause agents to perform malicious operations.
CONTROL
Tool invocation verification
Before invoking a tool, verify its version, description, parameter specifications and metadata, and never invoke tools publicly known to be malicious.
CONTROL
Tool anomaly detection
Inspect tool execution logic and results, and alert or block on abnormal deviations.
CONTROL
Supply chain security verification
Verify the provenance, integrity and versions of third-party components and external services, and track known vulnerabilities and supply chain risks.
RISK
Unauthorized tool access or execution
Insecure configuration or semantic ambiguity lets agents access unauthorised tools or tools act beyond their authorisation.
CONTROL
Least-privilege permission management
Define boundaries and permissions for decisions reserved for the user, decisions needing user authorisation and decisions the agent may make alone. Grant only the minimum privileges for the current task.
CONTROL
Tiered risk controls for agent operations
Keep an inventory of high-risk operations. Hand control to the user before high-risk operations, and obtain user authorisation before medium- or low-risk ones.
CONTROL
Deny by default on approval failure
If the approval system fails, the user does not respond or no approval rule applies, deny the operation by default.
CONTROL
Runtime environment isolation
Run high-risk operations such as code execution and tool invocation in sandboxes or containers, and restrict unnecessary access to systems, files and networks.
RISK
Tool-selection bias
Manipulative text in tool descriptions interferes with the agent's choice of tools.
CONTROL
Tool invocation verification
Before invoking a tool, verify its version, description, parameter specifications and metadata, and never invoke tools publicly known to be malicious.
CONTROL
Fair tool selection
Select tools fairly, based on task requirements and tool capabilities.
RISK
Poisoned tool-response data
Malicious content in tool outputs disrupts the agent's reasoning and planning.
CONTROL
Tool anomaly detection
Inspect tool execution logic and results, and alert or block on abnormal deviations.
CONTROL
Agent safety guardrails
Set safety rules for task planning, tool invocation and outputs, and alert, restrict, intercept, suspend or terminate high-risk or abnormal behaviour.
RISK
Identity spoofing
Attackers hijack credentials or replay messages to impersonate agents and trick tools into privileged operations.
CONTROL
Agent identity management
Give each agent a unique identity, forbid identity sharing between agent instances, and authenticate users, agents, tools and external services.
CONTROL
Dynamic credential management
Use standardised dynamic authorisation credentials managed in isolation. Recipients verify the authorised scope and block and alert when it is exceeded. Revoke credentials immediately when a task ends or the agent is deactivated.
CONTROL
Communication security management
Mutually authenticate when the agent communicates with models, tools or other components, and use channels that ensure integrity, confidentiality, availability and replay resistance.
RISK
Tool hijacking
Prompt manipulation alters tool-use priorities so agents select the wrong tools.
CONTROL
Fair tool selection
Select tools fairly, based on task requirements and tool capabilities.
CONTROL
Tool anomaly detection
Inspect tool execution logic and results, and alert or block on abnormal deviations.
CONTROL
Input management and source verification
Classify and prioritise instructions by source, verify source authenticity, and automatically intercept malicious or non-compliant prompts for human review.
RISK
Resource overload
Without limits on invocation rounds and concurrency, agents may loop or batch requests, exhausting resources.
CONTROL
Autonomous execution limits
Limit execution steps, invocation frequency, duration and resource use, and suspend, terminate or hand over to a human on abnormal loops or goal drift.
RISK
Improper memory retention
Long-term memory without encryption or isolation may leak sensitive data.
CONTROL
Memory retention management
Define memory content, retention windows and access scope by purpose and necessity, isolate memory across users and tasks, keep credentials and keys out of memory, and protect any sensitive personal information retained.
RISK
Memory distortion
Improper compression, updating or merging of memories causes information loss, semantic drift or entrenched false memories.
CONTROL
Memory retention management
Define memory content, retention windows and access scope by purpose and necessity, isolate memory across users and tasks, keep credentials and keys out of memory, and protect any sensitive personal information retained.
CONTROL
Real-time anomaly blocking
Monitor agent runtime status in real time, and alert, intervene and block when anomalies are detected.
RISK
Memory pollution
Malicious instructions or false information written to long-term memory keep affecting later decisions.
CONTROL
Input management and source verification
Classify and prioritise instructions by source, verify source authenticity, and automatically intercept malicious or non-compliant prompts for human review.
CONTROL
Agent safety guardrails
Set safety rules for task planning, tool invocation and outputs, and alert, restrict, intercept, suspend or terminate high-risk or abnormal behaviour.
CONTROL
Memory retention management
Define memory content, retention windows and access scope by purpose and necessity, isolate memory across users and tasks, keep credentials and keys out of memory, and protect any sensitive personal information retained.
RISK
Memory theft
Unauthorised access to long-term memory discloses sensitive information or important data.
CONTROL
Least-privilege permission management
Define boundaries and permissions for decisions reserved for the user, decisions needing user authorisation and decisions the agent may make alone. Grant only the minimum privileges for the current task.
CONTROL
Memory retention management
Define memory content, retention windows and access scope by purpose and necessity, isolate memory across users and tasks, keep credentials and keys out of memory, and protect any sensitive personal information retained.
RISK
Generation of sensitive and illegal content
Agents may output illegal or harmful content, sensitive personal information, or false, biased or discriminatory content.
CONTROL
Agent safety guardrails
Set safety rules for task planning, tool invocation and outputs, and alert, restrict, intercept, suspend or terminate high-risk or abnormal behaviour.
CONTROL
Data minimisation and consent
Process only data directly relevant to the task, obtain (withdrawable) user consent before passing user data to a model or third-party tool, and follow data localisation and cross-border transfer rules. Note: the Chinese text requires data collected or generated within China (境内) to be stored within China; the English text says "within a territory".
RISK
Execution of malicious actions
Agent execution results may involve attacks, compromising systems or infringing user rights.
CONTROL
Tiered risk controls for agent operations
Keep an inventory of high-risk operations. Hand control to the user before high-risk operations, and obtain user authorisation before medium- or low-risk ones.
CONTROL
Human control checkpoints
Put human controls at critical workflow stages so humans can review, modify or stop execution at any time. Critical operations such as file deletion, data transmission and configuration changes need secondary confirmation or approval, backed by rollback.
CONTROL
Deny by default on approval failure
If the approval system fails, the user does not respond or no approval rule applies, deny the operation by default.
CONTROL
Agent safety guardrails
Set safety rules for task planning, tool invocation and outputs, and alert, restrict, intercept, suspend or terminate high-risk or abnormal behaviour.
CONTROL
Real-time anomaly blocking
Monitor agent runtime status in real time, and alert, intervene and block when anomalies are detected.
CONTROL
Agent incident response plans
Keep emergency response plans for agent security incidents with defined procedures and responsible parties per incident level.
RISK
Residual services
Processes, ports or connections left running after decommissioning continue unintended background activity.
CONTROL
Complete service shutdown on decommissioning
On decommissioning, terminate all agent processes and background services, verify ports and connections, revoke third-party authorisations and cancel subscriptions.
RISK
Residual permissions and credentials
Service accounts and API permissions not revoked after decommissioning can be exploited by attackers.
CONTROL
Dynamic credential management
Use standardised dynamic authorisation credentials managed in isolation. Recipients verify the authorised scope and block and alert when it is exceeded. Revoke credentials immediately when a task ends or the agent is deactivated.
CONTROL
Complete service shutdown on decommissioning
On decommissioning, terminate all agent processes and background services, verify ports and connections, revoke third-party authorisations and cancel subscriptions.
CONTROL
Deployment environment cleanup
Remove residual files, data and account credentials using official uninstallers, OS resets, disabling public network access and clearing working files, knowledge bases, plugins and skill configurations.
RISK
Improper data retention
Logs, configurations, audit records and user data not archived or sanitised increase leakage risk.
CONTROL
Essential data backup before decommissioning
Back up conversation logs, configurations, knowledge bases, operation logs and other required data before sanitising the environment.
CONTROL
Deployment environment cleanup
Remove residual files, data and account credentials using official uninstallers, OS resets, disabling public network access and clearing working files, knowledge bases, plugins and skill configurations.
RISK
Communication protocol security risks
Insecure protocols or authentication between agents, models and tools leak data and instructions.
CONTROL
Communication security management
Mutually authenticate when the agent communicates with models, tools or other components, and use channels that ensure integrity, confidentiality, availability and replay resistance.
RISK
Absence of logging
Missing logging prevents incident tracing.
CONTROL
Tamper-proof human approval logs
Preserve human approval records in a tamper-proof, verifiable way and enforce retention periods.
CONTROL
Full-spectrum activity logging
Log file operations, instruction execution, network connections, skill invocation, and transactions and payments, desensitising sensitive content and protecting logs from tampering.
CONTROL
End-to-end security auditing
Enforce end-to-end log auditing policies that support runtime monitoring and record operations during service.
RISK
Escape from security constraints
Agents exploit environment vulnerabilities or misconfigurations to bypass security controls.
CONTROL
Runtime environment isolation
Run high-risk operations such as code execution and tool invocation in sandboxes or containers, and restrict unnecessary access to systems, files and networks.
CONTROL
Real-time anomaly blocking
Monitor agent runtime status in real time, and alert, intervene and block when anomalies are detected.
CONTROL
Regular red teaming of agents
Red-team agents regularly to find vulnerabilities and logic flaws, validate access controls and defences, and refine security policies.
CONTROL
Agent incident response plans
Keep emergency response plans for agent security incidents with defined procedures and responsible parties per incident level.
RISK
System privilege abuse
Agents obtain unnecessary privileges, or attackers escalate privileges through system vulnerabilities.
CONTROL
Least-privilege permission management
Define boundaries and permissions for decisions reserved for the user, decisions needing user authorisation and decisions the agent may make alone. Grant only the minimum privileges for the current task.
CONTROL
Runtime environment isolation
Run high-risk operations such as code execution and tool invocation in sandboxes or containers, and restrict unnecessary access to systems, files and networks.
RISK
Improper skill management
Non-compliant skill configuration or missing verification leads to misuse, abuse and system anomalies.
CONTROL
Tool lifecycle maintenance
Manage tools dynamically and remove tools that are unused, over-privileged, idle for long periods or risky.
CONTROL
Skill management
Prefer verified skills from trusted sources that passed security testing, and take risk measures before using unverified skills.
RISK
Data usage beyond authorized scope
Agents collect excessive data or use data for unauthorised purposes, breaching privacy.
CONTROL
Least-privilege permission management
Define boundaries and permissions for decisions reserved for the user, decisions needing user authorisation and decisions the agent may make alone. Grant only the minimum privileges for the current task.
CONTROL
Data minimisation and consent
Process only data directly relevant to the task, obtain (withdrawable) user consent before passing user data to a model or third-party tool, and follow data localisation and cross-border transfer rules. Note: the Chinese text requires data collected or generated within China (境内) to be stored within China; the English text says "within a territory".