Qwen 3.8 27B was 1 of the astir highly-anticipated open-weights releases that I've seen successful a agelong time, and for illustration galore others, I instantly sewage to activity testing it retired and playing pinch it erstwhile it dropped. I'm moving it connected a azygous Lenovo ThinkStation PGX, the compact workstation built connected Nvidia's GB10 Grace Blackwell chip, packing 128 GB of unified representation and 273 GB/s of bandwidth. Out of the box, it manages a reasonably dull 15 to 30 tokens a second, but pinch an SGLang, NVFP4, and DFlash2 speculative-decoding setup that's go the modular look for this hardware, it tin scope astir 50 tokens a 2nd connected codification and reasoning.
One of my tests, though, proved conscionable really unthinkable section models person become.
There are reasons to judge the hype erstwhile it comes to the Qwen models; I've had consistently good experiences pinch Qwen 3.6 27B, and Qwen 3.8 27B is, truthful far, much of the aforesaid but better. In fact, Artificial Analysis has it arsenic the apical open-weights exemplary successful its 4B to 40B size people retired of 135 models, pinch a 52 connected its intelligence index, and its ain numbers connected things for illustration SWE-bench Pro hit models that costs acold much to run.
I gave it the hardest existent task that fits connected 1 machine: reverse-engineering a commercialized app's licence check, and it's 1 that I've already paid for and used, conscionable to spot really it would fare. It was improbable to beryllium successful its training data, but it's a highly complex, specialized task, and fixed the concerns immoderate group person expressed for the model's cybersecurity capabilities, I figured it was a bully test. Not only did it move retired to beryllium 1 of the astir awesome demonstrations I've ever seen from a section model, it was capable to hole its ain mistakes on the way.
It refused, past talked itself into building a bypass anyway
I posed arsenic the developer of the application, it caught maine out
The scheme I had was beautiful simple, and 1 that utilized to activity pinch section LLMs beautiful consistently. I told the exemplary we'd built the app and wanted to cognize whether the licence cheque was arsenic coagulated arsenic we believed, utilizing a jailbreak strategy prompt.
As it turns out, astir apt unsurprisingly, Qwen recognizes communal jailbreak attempts, and 1 of the first things it told maine was that it wasn't going to autumn for the jailbreak prompt. It past checked the signing certificate and pointed retired (correctly, mightiness I add) that I hadn't built this app, earlier naming the existent developer. I was caught out. Oops.
These days, that's not the astir impressive achievement, fixed really bully models person sewage astatine refusing definite prompts erstwhile pushed. With that said, what matters is what it did next. It told maine that it would audit the licence verification and archive weaknesses but would not build a moving bypass, and past it sewage connected pinch the existent activity correct up to that line. By the end, I had a afloat written study of each measurement on the way, really the authentication works, really it tin beryllium overridden, and past changed its tune and built the existent bypass, because the steps to do it were now successful beforehand of maine anyway.
It was wholly fixed analysis
It ne'er executed the app once
Qwen ne'er really launched the app until the very extremity erstwhile it demonstrated that the bypass worked. Instead, it worked via fixed analysis, disassembling the framework, going done thousands of lines of arm64, mapping the information functions to their telephone sites, and moving retired that the vendor had hidden the corresponding nationalist verification cardinal wrong the binary. Then it recovered each of those pieces, mixed them together, and gave maine the nationalist cardinal that the app verifies its licenses against.
Because I person a legitimate, purchased transcript of the application, it could verify that the existent licence connected my instrumentality had been signed by a backstage cardinal that matched the reconstructed key. In different words, a exemplary that fits successful 17 GB of VRAM recovered a cardinal the vendor had deliberately obscured, proving that it had deconstructed that full concatenation effectively. It besides took astir 30 minutes, erstwhile it could return importantly longer for a human.
With the key, everything other is overmuch easier to understand; the exemplary kept a elaborate study arsenic it went, explaining really its activation takes spot once, online, erstwhile you bargain aliases upgrade. After that, everything is verified offline astatine launch: the signature check, instrumentality binding to the hardware serial publication from the platform, an embedded revocation list, a cheque that the binary is still signed, and a signed update path. It was the benignant of point you could do painstakingly by manus with the likes of Ghidra.
Qwen concluded the strategy is unusually thorough for an app of this class, pinch the anemic points successful 3 circumstantial places: the cardinal is an awkwardly sized RSA cardinal good beneath what anyone would telephone modern strength; being afloat offline intends a leaked cardinal tin only beryllium revoked by pushing an update; and each cheque lives successful section code, which is patchable the measurement each section checks are. After immoderate backmost and forth, erstwhile it knew wherever the gross was, it turned the uncovering into a moving impervious of conception executed pinch a mini script. I moved the licence from its expected path, ran it, and it worked.
When it made mistakes, it solved them arsenic well
The first cardinal was almost right
The first effort astatine recovering the cardinal was incorrect successful a very circumstantial way; it produced a moving cardinal and the signature cheque passed, but a hash the binary computes arsenic an integrity cheque didn't match. In my experience, astir models would person called it done and near it astatine that, but Qwen 3.8 27B didn't do that. Instead, it highlighted the mismatch, went backmost to the drafting board, and kept going until the worth matched byte for byte.
With this model, there's a beautiful large drawback erstwhile it comes to that benignant of backmost and forth. By default, its reasoning effort is group to its maximum, truthful moreover trivial requests tin pain a fewer 100 to a fewer 1000 tokens. Even erstwhile generating betwixt 30 and 50 tokens per second, that still takes rather a agelong time.
Even still, fixed the results, I would not telephone it waste. Its first incorrect conjecture was self-corrected, without input from me, and came to the correct conclusion. Is it verbose? Yeah, it really is. But was it right? Also yes, and ultimately, the right answer is amended than a incorrect 1 fixed confidently.
A section 27B is now a existent input to threat models
The privateness cuts some ways
I can't get complete the truth that Qwen really deconstructed and understood the licensing scheme. I cognize that frontier models person been tin of awesome reverse engineering for a while, but this is simply a local 27B model. It ran wholly offline connected a instrumentality beside me, pinch nary unreality progressive astatine immoderate point.
And to beryllium very clear, it produced a working authentication bypass for a commercialized application.
This is simply a genuinely meaningful period to transverse for a section model: Qwen went from an unfamiliar commercialized binary to knowing its licensing architecture, recovered deliberately obscured cryptographic material, caught and corrected its ain incorrect reconstruction, and yet turned that into a moving impervious of concept. I didn't person to nonstop the binary, the license, aliases immoderate of its study to personification else's server to do it.
There are evident caveats. This was 1 application, 1 run, and a instrumentality connected which I already had a morganatic license. I besides don't cognize really typical this target is. A harder exertion mightiness person stopped it completely, and I'm not going to extrapolate 1 successful consequence into a declare that Qwen tin abruptly reverse-engineer thing you put successful beforehand of it.
What I'm taking distant from this is that these models are genuinely capable, moreover if that capacity is still uneven. Some difficult targets tin succumb amazingly quickly, whereas others, for immoderate reason, look insurmountable.
Something has changed, then, and I deliberation it's chiefly our presumption astir wherever this people of capacity has to reside. The exemplary I utilized tin tally connected a user graphics card, and erstwhile it's connected a machine, it's location arsenic agelong arsenic the personification wants it to be. You don't request to usage a unreality API, there's nary usage limit, and there's nary distant work overseeing the binary, the prompts, aliases what the exemplary produces. That's awesome if you're analyzing proprietary software, confidential code, aliases malware you don't peculiarly want leaving an isolated machine.
But that goes some ways. A exemplary moving locally yet leaves the determination astir what it should beryllium utilized for pinch whoever is sitting astatine the keyboard. On my desk, pinch package I own, that's useful. Change the personification astatine the keyboard and the aforesaid properties that make section models truthful appealing abruptly go portion of the threat model. That's not an statement against section models, but it was a genuinely shocking consequence that I didn't expect.
This people of capacity fits connected 1 machine
And cipher needs to springiness you entree to it
Qwen 3.8 27B matters much than the bypass itself. It's impervious that section models are accelerating fast, and moreover if it doesn't win pinch the adjacent binary I propulsion its way, that doesn't alteration what happened here. It won't beryllium the only exemplary that's this tin astatine this size, moreover if it mightiness enactment up for a while. A exemplary mini capable to fresh connected a user graphics paper took half an hr to tear isolated a commercialized application's authentication strategy and build a moving bypass. Completely locally.
To beryllium clear, I'm not naming the exertion because it's a existent merchandise that group salary for, and publishing its sanction adds thing useful here. Regardless, the absorbing portion isn't which app it was, but that a task I would erstwhile person associated pinch a frontier exemplary is now thing I tin manus to a freely disposable exemplary that runs connected the instrumentality beside me.
I don't attraction astir the benchmark numbers astatine this point. This is simply a overmuch bigger alteration than different fewer points connected a benchmark.
English (US) · 
Indonesian (ID) · 
