You can’t measure productivity by lines of code produced.
-- The Software Industry in 1996
You can’t measure productivity by lines of code produced, you idiot.
-- The Software Industry in 2006
You can’t measure productivity by lines of code produced, you absolute cave-dweller. How have you not gotten this through your thick skull?
-- The Software Industry in 2016
The LLM really boosts my productivity. Look at all these lines of code!
-- The Software Industry in 2026
Like most developers these days, I use coding assistants every day. I use them more often than not. I certainly believe they improve my productivity.
But I don’t have real data to back it up. Most of us don’t. One of the few studies that does provide real data comes to the opposite conclusion.
developers estimate that allowing AI reduced completion time by 20%. Surprisingly, we find that allowing AI actually increases completion time by 19%--AI tooling slowed developers down.
I’m going to dismiss the argument that “this study is a year old, it’s different now” because:
The same suspects said the same thing a year ago for the same reason, and a year before that
There are not up-to-date studies that show contradictory results
Every paper that you read was written in the past. That’s how time works.
If you’re looking for hard evidence that coding assistants truly boost productivity, expect it soon. And you should consider a lack of it damning. To paraphrase my favorite cynic, David Gerard (https://pivot-to-ai.com/2026/06/22/religious-exemption-from-using-the-ai-at-work/)
The Pope’s AI encyclical has given people a basis to claim a religious exemption from being required to use AI at work
That means that, in order to force employees to use AI, the employer has to prove in court that they’re suffering undue hardship
If an AI tool is truly boosting productivity in a measurable and provable way, this should be easy - but a vibes-based argument isn’t going to cut it
So why do I still use coding assistants?
My own objection, though, isn’t to dismiss this paper’s findings. It’s that experienced open source developers are doing a different thing from what most of us are doing with coding assistants.
The finding that coding assistants don’t accelerate development of novel and foundational software doesn’t shock me. But I put my coding assistant away for those tasks. I bust it out when I want to:
Write some basic CRUD code
Dig through these stack traces
Rewrite this legacy component in a modern framework and make sure the tests still pass
Refactor this huge file to separate X, Y, and Z
Interact with data I’ve never seen before, but which has extensive documentation
Add this new jawn but using the design patterns and boilerplate of this unfamiliar repo
Do a couple of glorified find-and-replace operations while I have a coffee
I’m confident it makes all of those tasks faster. But I don’t really have the data to prove that.
The cannibalization risk
In practice, it’s extremely difficult to separate "novel and foundational” from “rote and tedious”.
Most decent developers can tell you a story like this: The other day I caught Opus 4.8 design a multi-layer caching system with complex interlocking cache invalidation rules. It took the better part of a day to write it, and the better part of a second day for me to point out bugs that its tests didn’t catch. Eventually, I deleted the whole file, implemented an elegant little toy that had no caching, and told Sonnet to just replicate that pattern to replace what it had just done and not overthink it. Negative 5,000 lines of code later we had something that performed 10x faster, without any of those cache invalidation bugs.
That was "me doing the novel and foundational work” while I let “Claude do the rote and tedious work”.
Did that wasted time outweigh the time it saved me? Probably not? I make architectural mistakes, too. I might make fewer of them, but I make them slower and fix them slower. I was doing other things while it was making them, so I believe it was a net time saver.
But what about the times I’m not noticing my coding assistant spin its wheels? It’s hard to not notice when an LLM spins for hours… but am I certain it’s not spinning for 10 minutes a day unnecessarily on some bad architecture that I’d never have created had I done the rote work myself? Next month will it be 20 minutes? 30 in two months?
In a sense, the entire field of software can be described as transforming the “rote and tedious” into “novel and foundational [ways to eliminate the drudgery]” one abstraction at a time. Point to a language construct, file format, query language, front-end framework and I’ll show you something that was invented by someone whose goal was to eliminate the rote and tedious.
Thought exercise
If you brought a modern LLM to the 1960s, would all of our code still be written in COBOL and Fortran? What would we have sacrificed?
Does it matter?
Do your next project in COBOL. Use all the Claude you want. Now ask me again.
Would we have invented higher-level languages, frameworks, abstractions, and generally improved our craft regardless?
LLMs don’t get upset with spaghetti code the same way as humans; it seems unreasonable to assume they would have propagated object-oriented programming or state machines or set-based languages without a human push.
In the real world, the craft has been greatly improved by teams with a simple profit motive and engineers given tedious work just trying to make their lives easier. In this fictional world, those sorts of advancements may only come from labs, institutions doing research for the sake of doing research.
Would we be net ahead of where we are now? (writing COBOL with LLMs)
Probably?
In 50 years? 100?
Almost certainly not.
What to do
I don’t pretend to offer a blueprint, but in general we ought to make sure we’re dedicating some time to finding better ways to do things even when the raw horsepower of LLMs makes it so that we’re not immediately forced to. Otherwise, we’ll sit still. Open source software - rarely properly supported - may be in more danger than before. Again, these models will be limited in their ability to write better software unless we train them on better software.
Fighting the cannibals
On a more extreme day, I’ll say that “an AI company should be sued out of existence for copyright (or copyleft) violation, as an example to the industry, and that this would be good for the industry, or at least the technology.”
If the greatest non-fungible asset to technological progress today is the availability of high-quality training data, then we need to stop the industry from cooking its golden goose. If the fear of legal or regulatory catastrophe would chill investment into the (top layer of the) AI industry, then surely the fear of having no way to compel your largest consumer to pay would chill (and is chilling) investment into a most crucial input of AI’s supply chain - knowledge, science, art, journalism, and other intellectual property.
There’s an appeal to economics: An industry that needs special privileges (in this case, the ability to break the law with lesser consequences than others) to survive will eventually be rendered obsolete by foreign competition. That sounds far-fetched today, but so did warnings about the competitiveness of the US auto industry (driven by protectionism) in the 1960s.
And there’s an appeal to history: The telecommunications industry was, in hindsight, not set back but in fact boosted, by the breakup of Bell Telephone. Thriving financial markets aren’t typically found in places that don’t police fraud. We don’t encourage the development of new drugs by declaring that the formula is basically public domain if you can get your hands on it.
To be clear: I’m not arguing for the arbitrary vandalization of an AI company, I’m arguing against the central planner’s instinct to protect a national champion from the normal consequences of its actions.
However, we don’t need to get abstract or hypothetical - we already have an example in front of us right now with distillation “attacks”.
Training an LLM on books is functionally, legally, and ethically the same as a human author reading the book and learning from it.
Training an LLM on art is functionally, legally, and ethically the same as a human artist looking at the art and learning from it.
Training an LLM on GPL licensed code is functionally, legally, and ethically the same as a human engineer reading the code and learning from it.
Training an LLM on the output of a chatbot is functionally, legally, and ethically the same as a human ... a crime! They're stealing! Call the police!
The geopolitical aspect certainly adds complexity - if one country respects the rules while another doesn’t, they’ll be giving up a short-term advantage. But if I need to convince you, I’ll go out on a limb and assume you’re not a proponent of state-run economies and don’t favor the destruction of capitalism and elimination of private commerce. So long as we’re talking capitalism, we do need to find a way to make producing a critical input profitable if we expect anyone to continue producing it. Private entrepreneurs don’t tend to produce critical inputs for other people’s businesses for free.
Back to productivity and the industry shakeup
From my last post:
My final observation on productivity is going to again be to ask the non-tech consumer. If LLMs were driving a productivity explosion, we’d see a vast array of new apps and better faster cheaper services. But most people do not believe the internet is getting better. They’re not happier with their apps than they were 5 years ago. Opinions of the tech industry have only gone down. Again, it’s mundane: the industry is talking Skynet; gamers simply think AAA studios’ AI art looks tacky.
The fact that “code is easier to produce” is in many ways missing the plot. For a good decade prior to AI, code has been one of the easier components in a “technology” product. It was rarely the gating factor for most businesses.
The gating factors may be …
Data businesses: proprietary data & historical data
Bespoke capabilities built by subject matter experts
The consumer economy lives on attention and network effects
The idea that “Netflix or Meta should be scared because I can trivially build a video streaming site or social media app with AI” is goofy. It’s not a change in the state of affairs; (almost) anyone with a VC check could have built a video streaming site before AI (and lots of people without one could have built a social media app). Netflix has Seinfeld and Facebook has your parents; software was never the moat. Startups do sometimes fail on tech execution, but it’s rarely because no one knows how to do the execution they need, it’s because those people are out of their price range (or they take this line of thinking to the extreme and fail to appreciate execution at all).
The software companies I would be worried about are the ones I think have always been on shaky ground: those that offer generic SaaS capabilities at high margins, with little moat, may face more competition. CRMs, to-do lists, wikis, etc. But … what else is new?