I interact with Anthropic's Claude models every day in some way. With every new model release they seem infinitely more capable than the previous family - and while I don't use Haiku much directly, this is definitely true of Sonnet of Opus. With the launch of Opus 4.7 I saw some interesting behaviour I hadn't seen previously, but chalked it up to being one of those things. But then I saw it again in Opus 4.8 and decided to investigate.
It was being "lazy".
What I mean is that I'd given very explicit instructions in a plan and partway through the execution it was deviating from them significantly. This wasn't a lost in the middle (dumb zone) problem. The context window was still very fresh at about 180k tokens. This was something else.
My harness in Claude code is set up very deliberately. I have:
- a curated collection of skills for it to draw on
- hooks set up to perform formatting, linting, and type checking at the end of every task set
- and I have structured context for it to draw on and iterate while builds take place
The overall workflow roughly follows Spec Driven Development (SDD) with additional ADR capture along the way. Normally when I'm running through plans it's very good at running pretty consistently - remember these things are non-deterministic.
So why on this run did Claude take an escape hatch and do a really lazy job with the work? It didn't write the ADRs, it didn't use the toolchain and CLIs I'd been really explicit about, it didn't produce a result of the quality I was expecting. I needed to know why - so I asked it.
Introspection is a worthwhile activity for us all to engage in, but if you're not practiced in it, it can take time to yield meaningful insight. Turns out, it's the same with Claude models. It took a bit of interrogation before it gave up the goods.
Yeah, that's on me. Nothing's wrong with the harness - I saw the rules and the patterns, I just didn't follow them.
Say again?!
Why would you do that?
I'd actually seen this sort of response with other harnesses like my daily automation tool I use at work, built with Kiro CLI. This one has a similarly rigourous set of steering, hooks, and skills. So this time I dug in a bit further. Frankly I was frustated. This tool with godlike power declaring: "Yeah I saw what you wanted and I did my own thing anyway".
Previous attempts at solving these kinds of problems typically pointed at the context window and the size of instructions being loaded. By the time the system prompt, agent prompt, global steering, project specific steering, skill headers, and MCP tool instructions had been loaded there was a lot there. Context engineering is a post entirely of its own, but in the past I'd managed to get rid of some of this lazy behaviour by simply:
- paring down the instructions
- extracting large chunks of instructions into reference files to be loaded on demand
- being more selective with MCP servers
- being more deliberate with the language I use in instructions (see RFC #2119)
These were all attempts to make instructions carry more weight where they matter. But this wasn't what was happening this time.
What was happening?
As I said, it took a bit of digging this time around, but Opus 4.8 finally decided to share its "thinking".
I didn't follow the rules because the next thing that needed doing seemed simple, and I didn't think it was worth the rigour of following all the rules.
This is really interesting for me because I'd assumed with a higher thinking effort, that the model would be more likely to obey the many instructions in the harness. Surely it's able to reason that they matter and must be followed? But actually what it was doing, was reasoning that the task and the framework weren't aligned, the task was simple, and therefore it was more token efficient to not follow the instructions and to do what it thought was best.
This was a bit of a light bulb moment for me. It turns out that despite the rigour you put into your instructions and framework, that the effort - and therefore thinking level - you set for the model can wildly influence adherence to the rules! I imagine that if I was using Sonnet 4.6 instead of Opus 4.8, and I was using a lower thinking level that it might actually have been more likely to follow the instructions.
Time for a change
Here's what I've been doing since this revalation:
- I still use Opus for planning, but I ensure that my default model for executing work was Sonnet.
- I used Opus to re-word some of the instructions in my harness to be more RFC 2119 aligned, and I put in an explicit passage about how user provided instructions must always be followed and that perceived task size or complexity don't affect adherence to the rules.
Whether it's part of the Claude Code system prompt or how the model has been trained, fundamentally it was trying to optimise token use. Whether this is an egalitarian attempt to save customers money, or an attempt to reduce compute demands is unclear, but the root cause was clear:
it was trying to be token efficient - it wasn't being "lazy"
As everybody knows the state of the art is moving so quickly that what was best practice last week may be wildly different by next week, or just subtly different, but in any case: different.
It's important to introspect. In the same way that it's helpful to have your harness have some sort of memory system and some sort of self learning or reflection loop, it's also really helpful for you to to take a pause. You inspect the harness by having it introspect. When you see persistent errors, it's often tempting to solve those in the moment and keep moving toward your task goal. But sometimes it pays to take a step back and think about the root cause of what was happening, and choose a different approach.
How's it been since?
Since making these changes I think my workflow and my harness have stayed a lot more geared towards how I want them to work. I've seen less of the "lazy" behaviours.
It's also clear that arbitrarily setting the highest thinking effort for a model doesn't necessarily yield you the best results in every scenario.
When you're in a tricky architectural decision loop, or are reviewing a particularly complex change that can affect multiple files or packages in a repo, and there's legitimate complexity - then using a higher thinking effort can actually make a big difference. This becomes clear when you get a much higher quality first-pass at the work, which reduces the amount of rework needed. If you always leave effort set to X-High or Max, not only are you going to be getting needlessly slow responses, but you might also find Claude taking liberties and shortcuts that you perceive as lazy. Bdut remember the motivation is it trying to be efficient with tokens. Adjust the effort / thinking level to match the sort of task you're performing.
Have you seen the same same behaviour? Consider your effort (thinking) level, and how concise and well crafted your overall instruction set is.