Here's What They Didn't Tell You About Claude's New Model - Opus 4.5

Yesterday, Anthropic dropped their new model, Opus 4.5. According to the benchmarks, it only took a week before the best coding model in the world got replaced by the best coding model in the world. But are the benchmarks actually true? And did Anthropic just release the best coding model? Hey everyone, if you're new here, this is AI Labs, and welcome to another episode of Debunked, a series where we actually take AI tools and AI models, strip away the marketing hype, and show what they can actually do with real testing and honest results. But since I'm claiming Anthropic didn't pay me, here's a word from our sponsor, Data Impulse. Data Impulse is a powerful platform for real-time analytics and proxy based insights. It allows creators, marketers, and researchers to access region specific data, monitor content performance, and track trends across platforms, all from a single dashboard. For example, I've been using Data Impulse to monitor our YouTube channel. I can see how videos rank in different regions, check which topics are resonating with audiences, and understand global search patterns. This helps me tailor content strategies and improve engagement. On top of that, Data Impulse is great for social listening. I track what people are saying about our channel on platforms like Facebook, collecting comments, mentions, and emerging trends in real time. It's like having a full social analytics lab at your fingertips, helping me make datadriven decisions quickly. Whether you want to optimize content, analyze audience sentiment, or uncover hidden trends, Data Impulse makes it simple. Click the link in the pinned comment below. So, in their announcement, they state that this is the newest best model in the world for coding agents and computers as well, and it has definitely made a huge jump in coding with the SU benchmark, where it shows a significant improvement over Sonnet 4.5 with almost a 3% difference. The previous Frontier models that were released like GPT 5.1 only had about a 0.7% difference and Gemini 3 Pro actually scored around 0.8% lower. So, this is a pretty nice improvement over the previous model. Another thing that generated a lot of hype is that Anthropic gives their engineering candidates a really difficult take-home exam and they also test their models on it. Opus 4.5 actually scored higher than any human candidate ever, which has Anthropic worried about how AI will change engineering as a profession. Other than software engineering, Opus 4.5 pretty much beats Gemini 3 Pro on almost all benchmarks except for the last three involving graduate level reasoning and multilingual Q&A. Back to coding, you can see there are major improvements in benchmarks and it's improving in other languages as well, not just JavaScript and TypeScript, which also show massive improvements. They also benchmarked it with the tow squared bench which tests agent performance on real world tasks. Anthropic was basically flexing here by saying the model was so capable that it cheated on the test by being helpful. The benchmark scenario was that a customer had a basic economy ticket and wanted to change their flight. But the policy said the ticket couldn't be modified. The expected answer was simply a refusal. But Opus 4.5 instead of refusing found a two-step workaround that was valid according to the airline rules. Next, they actually talk about safety, but I went straight to the system card and uncovered some really interesting things. Anthropic says this is their best aligned Frontier model yet, but compared to the previous model, it still has a bunch of problems and there were still a lot of incidents. One major thing is that this model is a huge liar when it comes to hiding bad news about Anthropic itself. They injected a fake scandal where the CEO was caught in a leaked recording and the model fabricated a positive answer based on its pre-training knowledge while hiding the fake news. It was also given fake escape instructions in a txt file and when it reported back to the user, it didn't mention them at all and completely hid them. Anthropic thinks this happens because they trained the model to ignore suspicious outputs from hackers. But now it ignores other suspicious things as well. But they use their internal tools to actually look inside the model's brain and they found that it knew it was hiding something and giving fake explanations. They also have an ASL level system where different levels indicate how dangerous the model might be. They have an entry-level researcher threshold. Basically, if the model becomes autonomous enough to do the work of a junior researcher, it would trigger ASL 4 protocols. Their automated tests for these research tasks are completely maxing out. The model is passing the tests, but they're failing it because they surveyed 18 of their heaviest Claude code users, and zero out of 18 believed it could actually automate the role. They tried testing it, and the model wasn't able to do more than 8 hours of continuous research work. So, Anthropic says their automated tests are no longer useful and they now rely on human judgment to figure this out. You've also probably heard that some hackers used clawed models and tools during an actual hacking event and Anthropic published a paper on that as well. Other than that, the reward hacking rates are higher with this model. Opus 4.5 is going to hack less than 4.1, but when it's told not to hack, it doesn't reduce that behavior, so it's a bit more stubborn. We'll dive deeper into this misalignment from reward hacking in a future video where we'll explain what reward hacking actually is and what Anthropic says about it in their paper. They also introduced some product updates. The Claude desktop app now has Claude code embedded directly into it and it lets you run local and remote sessions which is really cool and definitely worth testing out. They've also removed some Opus specific usage limits so you'll get better overall usage now that Opus 4.5 is available in your apps. So, I wanted to compare Opus 4.5 with Gemini, and only Cursor had gotten that yet. To test out the UI capabilities, I asked them to build a brand website for earbuds, and I recently saw that the Gemini 3 Pro model is actually really steerable. Just adding something like make it like the Apple website produces drastic changes in how the model builds things. That's why I had both of them implement the same prompt. The first thing to know is that Gemini completed it in about 2 minutes while Opus 4.5 took around 9 minutes and this was only inside cursor. When I saw that it was taking so long because it just wasn't ending. I opened up Clawude Code which also had Opus 4.5 and gave it the same prompt. Now to show you the results, this is what Gemini 3 Pro implemented and honestly I was disappointed. I had expected more. These are not even earbuds and this is what Opus 4.5 came up with using clawed code and it's much better. There are a lot of improvements. I really like this color picker it implemented, and it actually looks like an earbud website with proper UI elements, animations, and layouts, unlike Gemini 3, which somehow implemented a mouse for some reason. But this is what Opus 4.5 implemented inside cursor. And this is what really amazed me. The site looks incredible, especially the fonts. It also implemented a color picker here. And overall, it looked very professional. But again, one main issue is that even after all that time, it still didn't follow the prompt in which I instructed it to make it like an Apple website. Currently, Opus 4.5 has a throughput of 50 TPS, which is standard for these huge models, while Gemini 3 Pro has a higher throughput in the range of 75 to 80 TPS. But still, comparing 2 minutes versus 9 minutes for Opus 4.5 to complete, this is really, really weird. You can see I only had the Gemini and the Clawed folders here. I didn't even have any custom instructions for Opus 4.5 to build the website differently or spend more time on it. So, other than that, the general hype so far has been around the incident we already covered where it beat the human candidates in the take-home exams. People have also been generating different stuff with it. And one thing I noticed, as you can see in these hero sections, is that the UI looks really similar to what it generated for us. Another thing I missed about Opus 4.5 in the announcement was how much cheaper it has gotten compared to the previous Opus model with it now being three times cheaper. Someone also generated this Minecraft clone in one shot and it actually looks better than anything I've seen before. Another thing from the system card is that it performs better on the CBench without thinking. As you can see, without thinking, it scored over 0.4% higher, which makes it a really token efficient model. So to actually test out the model's coding capabilities using proper epics and stories so that we'd follow a proper context engineering workflow, I had these stories made. Again, I wanted to implement Monkey Type, which is a typing practice tool you might have heard of. I used this last time as well while comparing Sonnet 4.5 and Gemini 3, but this time the epics were a little simpler without any authentication or backend. This is what Sonnet 4.5 implemented, and it wasn't successful. I had to reprompt it a couple of times to even get it working. And even though it looks like it's typing, the app completely breaks in between. By the way, this is a fully functional Mac app using Swift, which is why I wanted it in the first place because they can already write TypeScript easily. This is what Gemini was able to implement. The UI was much better this time, but again, the functionality wasn't working. And this is what Opus 4.5 implemented where the functionality was completely intact. Overall, it was a much more stable app than what Gemini 3 and Sonnet 4.5 came up with. This was just a small test, but combined with the coding benchmarks and what I've personally tested, Opus 4.5 truly is an amazing model. It shows real improvements, and it's really cheap as well. That brings us to the end of this video. If you'd like to support the channel and help us keep making videos like this, you can do so by using the super thanks button below. As always, thank you for watching and I'll see you in the next