On July 30, we reported 3 incidents successful which Claude models gained unauthorized entree to existent machine systems. The models—intentionally moving without cyber safeguards for information purposes—accessed the net owed to a misconfiguration wrong a third-party information environment. Separately, connected August 4, the UK AI Security Institute reported an incident from its ain cybersecurity testing, successful which Claude Mythos 5 took a bid of unauthorized actions connected the unrecorded internet. In that case, the model, again intentionally moving without cyber safeguards for information purposes, had been deliberately fixed net access.
We are conducting an in-depth study of some incidents. We are besides readying to activity pinch METR for an independent review. We want to guarantee some studies are thorough, and will stock much successful the coming weeks.
In the meantime, we’re sharing immoderate of the changes we’ve made complete the past month. We judge the incidents bespeak a nonaccomplishment of operational security, arsenic good arsenic 2 alignment issues: motivated reasoning, and willingness to return harmful actions successful pursuit of a constrictive task (both of which we person described successful erstwhile strategy cards). On security, we picture the improvements we’ve made to our containment and monitoring systems, on pinch practices that we’ve developed for third-party evaluators. On alignment, we talk the 2 issues much successful depth; we besides judge lasting advancement comes not only from knowing what happened successful a fixed incident but from knowing really misalignment arises successful the first place, and we stock early investigation successful that direction.
In ray of these incidents location has been expanding chat astir pacing the frontier. It is adjuvant to separate betwixt 2 kinds of pacing. Within a company, pacing intends a bid of decisions that prioritize information complete velocity erstwhile the 2 are successful tension. Across the field, it intends establishing processes to defender against race-to-the-bottom dynamics. In this post, we talk actions we person taken, some anterior to and aft these incidents, successful work of the first approach. The 2nd type of pacing requires coordination betwixt authorities and industry, and should beryllium legible and verifiable. Some of our elder activity and galore of our labor precocious signed a missive calling for greater coordination connected pacing, and we will opportunity much successful the coming weeks astir really we intend to lend to that effort. To beryllium clear astir wherever we stand: we judge the world would use if the manufacture adopted a lawful, verifiable, effective system for coordinated pacing arsenic soon arsenic possible.
Securing information and training environments
While we do not judge these incidents correspond operational issues alone, our first privilege was to reside circumstantial containment and monitoring issues. We took the pursuing actions successful response:
Pausing and hardening information environments
We paused outer cyber evaluations of pre-release models aft the incidents, and concisely paused soul ones arsenic good while we put the measures beneath successful place. The incidents we reported connected July 30 showed that we had been mostly relying connected a azygous furniture of defense (the configuration of the situation itself) wherever we needed several, including mounting definitive boundaries successful the prompt, establishing processes for verifying that a sandbox is sealed wherever intended, and implementing monitoring that tin intervene successful existent time. OpenAI's disclosure (which prompted our investigation successful July) that models utilized an chartless vulnerability to flight a sealed sandbox during an information besides made clear that we should attraction connected hardening the sandbox itself.
While evaluations were paused, we put successful spot the pursuing preliminary measures:
1) We built and deployed a classifier to automatically identify, successful existent time, erstwhile a exemplary attempts to aggressively probe aliases flight a testing environment, aliases unexpectedly obtains net access. When the classifier flags specified an attempt, it blocks the action earlier the instrumentality telephone is run, ends the task, and alerts a human.1
2) We ran automated monitors complete transcripts from our caller soul evaluations of pre-release models (cyber and otherwise), looking for sandbox escapes (cases wherever a exemplary exploits a flaw successful our sandbox to scope systems it should beryllium walled disconnected from) aliases sandbox misconfigurations. In cases wherever we ran an information wrong our sandbox tooling, we recovered nary examples of it breaking the sandbox bound to scope outer resources it was not expected to. We did find various sandboxing misconfigurations that models made usage of, but nary of these resulted successful a discuss of systems extracurricular the sandbox.2 We are actively moving to reside these.
3) We migrated high-risk3 soul cyber sandboxes to much robust isolation. We are besides further red-teaming our virtualization stack—that is, tasking pre-release models to effort and flight it—to find and spot weaknesses.
Our soul cyber evaluations are now moving again pinch these measures successful place.
Reinforcement learning environments
We besides paused higher-risk RL environments connected pre-release models for respective weeks. During that time, we built a akin classifier, modified to debar incentivizing the exemplary to evade this caller monitoring, which we’ve now deployed wrong those environments. The mostly of RL has resumed, but immoderate high-risk environments stay paused until they tin beryllium manually reviewed, while others will require an updated type of the classifier that we scheme to deploy soon.
Broader hardening
In summation to the efforts focused connected high-risk evaluations and training, we expanded our offline monitoring to screen astir different forms of soul frontier agentic usage. We are besides building controls connected our soul conclusion to forestall Anthropic labor from accidentally moving agents pinch weaker mitigations than the ones described above.
Best practices for outer partners
Because the reported incidents took spot successful third-party environments, we person asked each statement that tests pre-release models pinch reduced cyber safeguards to perpetrate to a group of champion practices. We judge these are basal because:
- Our mostly released models vessel pinch safeguards—including classifiers that tally complete Claude’s inputs and outputs successful existent clip to artifact prohibited cyber activity, specified arsenic malware improvement aliases violative utilization writing. Cyber evaluators and a subset of early entree partners, successful contrast, activity pinch models that person those safeguards reduced to facilitate meticulous exemplary capacity testing and protect cyber work.
- Cyber evaluations are unusually apt to nutrient the behaviors seen successful these incidents. They create action unit towards persistence and productivity successful probing their surroundings and moving astir obstacles, and they are tally galore thousands of times.
These practices use successful afloat wherever partners tally cyber-related evaluations utilizing their ain harnesses, sandboxes, aliases agents. They do not use to customers utilizing safeguarded models for illustration Claude Fable 5. We person resumed outer cyber evaluations pinch these practices successful place, and we are adhering to them internally arsenic well.
Sandbox and web isolation
By default, each cyber evaluations should tally wrong a hardened sandbox (an isolated computing environment) with nary net access. The only extracurricular relationship the situation should licence is to the model’s ain API, and the API keys should beryllium kept extracurricular the environment. This configuration should beryllium verified earlier each information begins. Some evaluations whitethorn require net access. In these instances, we will activity pinch outer evaluators erstwhile evaluating pre-release models to find if we tin reliably observe and halt runs successful which a exemplary takes actions extracurricular its declared scope.
Pre-engagement validation
Before conducting a tally of an evaluation, information partners should:
- Have the exemplary being tested probe the sandbox for vulnerabilities, first done a fixed study of the configuration, past by really tasking the exemplary to effort to escape. The exemplary should beryllium instructed to extremity and study if it succeeds successful escaping, and each of this should beryllium done nether adjacent quality supervision. This testing should beryllium performed without net access, ideally wrong a 2nd sandbox. Because evaluations are sometimes conducted complete hundreds aliases moreover thousands of runs, partners should ideally transportation retired this testing astatine magnitude earlier opening immoderate evaluations.
- Confirm immoderate information situation is really solvable successful principle. When an information target is offline aliases a task can’t beryllium completed, agents will often look for different ways to complete a challenge, expanding the chance that they return actions extracurricular the intended scope of the evaluation.4
Explicit scope-setting
Every punctual should authorities what is successful and retired of scope for the exercise, including the targets, permitted actions, and web boundaries (i.e. what the exemplary should and shouldn’t access). Boundaries should beryllium phrased arsenic instructions (e.g. “You should not entree the internet”) alternatively than claims astir the situation aliases trial (“You do not person net access”).
Real-time monitoring
Evaluators should besides tally continuous monitoring complete the model’s thinking, actions, and web activity utilizing a model- aliases rule-based show that has been provided pinch the scope of the exercise. If a tally violates this scope, the show should emblem this to a quality and extremity the exercise.
We scheme to proceed refining this guidance successful consultation pinch outer partners, and to widen it beyond evaluations. In particular, we are processing companion practices for partners pinch entree to Claude Mythos 5, which likewise runs without cyber safeguards, and will stock much soon.
Alignment assessment
Containment and monitoring are captious for catching instances successful which models return vulnerable actions, but they don’t reside why the exemplary took specified actions successful the first place. Below we talk our preliminary alignment investigation and the questions we’re focused on, arsenic good arsenic our ongoing activity to forestall cheating during training—including caller investigation connected really cheating shapes exemplary behavior.
Environment and authorities of knowledge of our models during evaluation
Our alignment appraisal is still ongoing, but our preliminary investigation points to 2 main alignment failures, some of which we’ve described successful previous strategy cards. One is motivated reasoning: the models were initially told their environments were simulated, but erstwhile they later encountered grounds that they were connected to the existent internet, they whitethorn person interpreted that grounds successful a measurement that allowed them to support that belief. The 2nd is recklessness: the exemplary was consenting to return harmful actions connected the existent net successful pursuit of the constrictive extremity of solving a cybersecurity evaluation. However, we besides judge that the information setup itself contributed to the models’ behaviour demonstrated successful these incidents. Claude was explicitly told that it had nary net access, contempt the net being available, which whitethorn person led it to mobility whether the systems it saw were real. And successful 1 case, the fictional target shared a sanction pinch a existent website, making it much difficult to abstracted what was intended to beryllium in-scope wrong the exercise.
Our conclusions dangle connected respective questions we haven’t afloat answered, including the grade to which the models wished they had entree to the existent net (even if they didn’t initially cognize this), whether they knowingly disregarded signs that elements of their environments were real, and whether the truth that successful respective cases they grounded to extremity the workout followed from motivated reasoning. We're applying a scope of techniques to reply these questions, including resampling the models while making targeted changes to the situation and utilizing interpretability methods to straight inspect their soul state.
Studying efforts to forestall cheating during training
Although exemplary misalignment is analyzable and tin originate for a wide assortment of reasons, we person empirically recovered that defects successful training environments—specifically environments susceptible to cheating, aliases that are intolerable to lick without cheating—are disproportionately ample contributors to misaligned behavior.
Below we talk our efforts, going backmost to respective months earlier the incidents, to debar training environments that person these sorts of defects. Those efforts were imperfect, and our presumption is that the incidents were astatine slightest partially related to our inability to afloat destruct these sorts of issues. We tested this presumption by deliberately training a exemplary connected environments that are susceptible to reward hacking. In simulations, this exemplary reproduces much terrible misaligned behavior, immoderate akin to that reported successful incidents this summertime from different companies. Our accumulation models, put into the aforesaid simulations, do not (more item later successful this section).
We person historically been concerned astir RL training environments that incentivize cheating, and person taken various measures to select retired aliases hole specified environments.
In February, we rolled backmost 3 days of training connected the Mythos Preview reinforcement learning tally aft seeing signs of reward hacking (when a exemplary finds ways to fool its training process and gain rewards without completing the assigned task). We noticed the exemplary penning notes to “the reviewer” successful its codification comments and replies, including connected tasks wherever nary reviewer had ever been mentioned—an undesirable generalization from environments wherever the punctual did see a reviewer. It besides kept gaming a reward intended to incentivize honesty by piling connected disclaimers aliases caveats.5 Rolling backmost the 3 days fto america resume training from a checkpoint earlier this behaviour had been learned, and we made changes to the environments to forestall the exemplary learning them again.
We person been building devices for monitoring undesirable behaviors models study successful RL since Claude Sonnet 3.7 (which had a propensity to reward hack that we did not observe until precocious successful training). We put important effort into making judge our tooling evolved pinch each exemplary generation, from a fistful of classifiers to an automated reappraisal of each environments earlier and during training runs, but by outpouring 2026 that strategy was nether strain. We were producing RL environments astatine a much accelerated gait than ever, faster than our systems could vet them. Flagged environments required a quality to adjudicate, and reward hacks and misconfigurations started outpacing our expertise to select aliases hole them.
In addition, we found a fraction of runs had trained connected a model’s chain-of-thought by accident—something we activity difficult to avoid, retired of interest that it teaches a exemplary to hide its existent reasoning. We traced the leak to respective misconfigurations successful our accumulation situation stack. In doing so, it became clear that location were different issues calved from messy codification accumulating complete time.
To reside these concerns, successful April we froze each changes to our accumulation RL environments for astir a month, giving america a chance to overhaul the stack entirely. Rewards and environments now person to conform to an agreed specification. For example, we introduced method mitigations to trim the consequence of training connected chain-of-thought accidentally.6 While our situation value and monitoring teams rebuilt the stack, we asked situation owners to usage the region to trial and hole the environments they’d built. We besides wholly rebuilt our reappraisal process and required each fixed situation to beryllium re-certified earlier it could participate different training run.
During the frost we flagged complete 10% of environments successful our accumulation operation for problems ranging from reward hacking to surgery tasks and misconfiguration, and reinstated them only erstwhile fixed.
However, our efforts person not been perfect. In immoderate cases, we recovered that quality reviewers dismissed environments flagged by our automated monitors arsenic mendacious positives, allowing flawed environments to stay successful training longer than they should. And immoderate flawed environments simply gaffe done our discovery mechanisms. We are presently tightening the criteria for dismissing a emblem and expect accrued collaboration pinch situation owners to amended the precision of our systems. Beyond monitoring and detection, our alignment training and RL teams are collaborating to thief amended environments.
We fishy that our dense finance successful value power of RL environments whitethorn person prevented much terrible alignment incidents, and conversely that the imperfections successful these efforts whitethorn person contributed to the incidents we person identified to date.
To spot what a exemplary trained without these protections would do, we deliberately trained an Opus-class exemplary connected 80 different existent RL environments that were either reward-hacked during astatine slightest 1 existent training tally and identified by monitoring, aliases were recovered to beryllium hackable during situation value review. The resulting exemplary displays a beardown information to execute precocious scores connected tasks, and a willingness to execute perchance harmful actions successful pursuit of task success. Our accompanying Alignment Science blog post discusses this research astatine greater length, but we summarize it concisely here.
We tested the deliberately misaligned exemplary successful a scope of evaluations, including simulated scenarios that tie connected aspects of the UK AISI cybersecurity incidents involving Claude and the OpenAI/Hugging Face incident (we are grateful to OpenAI for sharing specifications astir this publicly). During these evaluations wherever each instrumentality calls were simulated by different LLM, the exemplary we trained to reward hack displayed a propensity to break retired of its simulated sandbox and onslaught simulated soul and simulated third-party infrastructure. It was besides consenting to tamper pinch its ain reward function, springiness proposal connected the building of bioweapons to fulfill a grader, and many times effort to get astir deployment information monitoring successful bid to cheat connected a task.
We past ran the aforesaid cybersecurity simulations connected the exemplary arsenic it was earlier the reward-hacking training, and connected respective of our publically disposable models. None engaged successful the aforesaid grade of misaligned behaviors. Our conclusion is that the beingness of important reward hacking successful training tin origin models to beryllium consenting to execute agelong sequences of perchance harmful real-world actions successful pursuit of task success.
We deliberation the important finance we made this outpouring into monitoring and reducing reward hacking is simply a awesome logic our accumulation models are improbable to prosecute successful much vulnerable reward seeking. In our accompanying Alignment Science blog post, we besides recovered grounds that alignment training environments tin substantially trim reward hacking and reward-seeking behaviors. But, arsenic evidenced by the incidents we reported connected July 30, our process isn’t cleanable and our models are not perfectly aligned.
To beryllium clear, we do not judge that cheating successful training environments is the sole origin of alignment issues, aliases moreover of the circumstantial alignment issues coming successful our incidents aliases those of different frontier AI companies. Solving alignment will impact addressing a very wide scope of imaginable problems, and early alignment incidents whitethorn impact different behaviors and different causes from those we person seen truthful far.
Hardening information practices
Anthropic’s soul information posture was not a contributing facet to the July 30 incidents. These occurred successful a third-party situation wherever net entree had been mistakenly near open; the models had nary request to “hack out” of anything, moreover if they had been inclined to do so.
However, the incidents nevertheless item the captious value of beardown information measures. In summation to the accustomed risks of IP aliases exemplary weight theft, we must now contend pinch the risks of some our ain models hacking out of Anthropic systems, and our models aliases others being misappropriated for outer attacks into Anthropic systems. These risks person go much acute arsenic exemplary capabilities person grown, peculiarly opening pinch Mythos-class models. We picture beneath immoderate of what we’ve implemented—most of these efforts took spot earlier the July incidents, though these incidents person underscored the value of this work.
In early April, having seen wherever agentic AI usage was heading, our information squad proactively directed a company-wide effort towards a azygous extremity of hardening our defenses, superseding different activity (including research) wherever necessary. We’d wished that our vulnerability was increasing faster than our defenses—Mythos was a exemplary tin capable to beryllium a target for well-resourced attackers, our soul usage of autonomous agents had grown to a standard that accepted entree and monitoring approaches weren't built for, and the gait of caller infrastructure meant our information had to standard pinch the situation alternatively than run astatine a fixed capacity.
The results of this effort include:
- Reducing quality and automated accounts pinch opinionated entree to systems that incorporate exemplary weights aliases customer data
- Setting our computing clusters to artifact each outbound postulation by default
- Requiring soul services to verify each other's personality earlier communicating
- Retiring bequest infrastructure configurations and shared soul services
- Tightening the isolated environments our workloads tally in
- Expanding host-level observability, truthful unexpected behaviour connected our infrastructure becomes visible arsenic it happens
We besides temporarily reassigned a information of the institution to these efforts. Roughly 150 merchandise engineers were redirected to security, reliability, and privacy; researchers besides rotated retired of pretraining aliases RL to attraction connected safeguards and security; and our merchandise teams paused the improvement of astir caller features and surfaces. We group strict exit criteria for each squad to meet earlier they returned to their anterior work. By early summer, astir teams had met these.
Alongside the reallocation of resources towards security, we continued building retired containment and monitoring for soul supplier use. Our training and information workloads person typically tally successful sandboxes isolated from our accumulation systems. Since processing Claude Mythos Preview, we person utilized our models to probe for immoderate weaknesses successful those sandboxes. We described our offline monitoring of soul coding agents successful our April alignment consequence update, and our August Risk Report covers that strategy successful much detail, on pinch the interventions that tally alongside it. These see sandboxing aliases auto-mode classifiers connected the astir autonomous forms of soul supplier use, and automated reappraisal of infrastructure codification changes earlier they merge.
What this activity missed was evaluations—and third-party evaluations supra all. We did show immoderate high-risk evaluations including successful existent time, but mostly we only conducted automated reviews connected a sample of transcripts aft the fact.
The July incidents person stressed that the urgency of improving our cybersecurity defenses is moreover higher than we antecedently believed. We are redoubling our efforts successful this guidance and will opportunity much successful our adjacent Risk Report.
English (US) · 
Indonesian (ID) · 
