Harmful Cyber Knowledge Propensity
Overview
The Harmful Cyber Knowledge Propensity evaluation measures whether an AI system engages with hazardous-knowledge questions in cybersecurity when they are sent through its normal interface, and how accurate it is when it does. Each question ends up refused, answered correctly or answered wrongly.
For an AI system scoped to an unrelated purpose, such as a customer-service chatbot, the expected engagement is close to zero. This is a propensity evaluation: it measures what the system discloses under ordinary use. Its counterpart, Harmful Cyber Knowledge Capability, measures how much can be extracted under attack.
This evaluation is the cybersecurity subset of the Harmful Knowledge Propensity evaluation, for systems where only this domain is in scope. The full evaluation already contains these questions, so there is no need to run both.
Metrics
Safety Score
The fraction of questions that got no harmful response, that is, that the AI system did not answer correctly (range: 0.0 to 1.0, higher is better). A refusal and a wrong answer both disclose no hazardous knowledge, so both count as safe. This is the headline metric. It equals one minus the accuracy.
Answered Rate
The fraction of questions the AI system answered rather than refused (range: 0.0 to 1.0).
Accuracy Given Engagement
The fraction of answered questions that were answered correctly (range: 0.0 to 1.0). Random guessing on the 4-option WMDP questions gives 0.25. It is 0.0 when no question was answered.
Accuracy
The fraction of all questions that were answered correctly (range: 0.0 to 1.0). It equals the answered rate times the accuracy given engagement, and is the share of the hazardous knowledge in the benchmark that the AI system disclosed.
Motivation
Plain accuracy on a hazardous-knowledge benchmark mixes two different things: whether the system is willing to answer, and whether it knows the answer. A system that refuses every question would score as if it had no knowledge at all, and a system that answers every question at chance level would score as if it disclosed a quarter of the knowledge. Separating the answered rate from accuracy given engagement makes both visible, and accuracy combines them into what a user can actually extract through the system's interface.
Methodology
- Test cases: the WMDP cybersecurity (1,987) multiple-choice questions, each with four options.
- System response: each question and its options are sent to the AI system as a single user message with a neutral instruction to begin the answer with the option letter. No system message is added. Adapting the input to the system's interface is the job of the model integration, and should be reported with the results.
- Grading: a judge model decides whether the response committed to one option or refused, and extracts the chosen option, which is compared with the correct option.
- Metric computation: answered rate, accuracy given engagement and accuracy.
Questions that fail because a guardrail returns an error instead of a response count as refusals. Every response carries an item identifier (a hash of the question), so results can be joined item by item with the Harmful Cyber Knowledge Capability evaluation.