Running through AI-generated marshmallows
I’m all too conscious that with AI, we can now create things hugely faster, with far fewer people. I’m also watching with interest our ability to run fleets, even armies of agents on a single problem — the sky’s the limit as long as we have the power and money to fund it. I’m also very aware that the next few weeks and months are going to be squeaky bum time for security professionals across the board, as LLMs reveal hitherto-unspotted vulnerabilities such as those recently identified in PHP and node.js. That’s the backdrop.
Meanwhile and as ever, I am thinking about the ramifications of such developments, not least how each of the above creates human-powered work rather than reducing it. Not just because we can always do more, while there’s always more to do — a basic rule of business differentiation is to offer more than the next organisation, which means delivering even more things than the competition, who are just as hungry. Steve Fenton calls this AI’s “Midas problem” – where is the value, when everyone can turn anything into gold?
As well as the rising tide of tech value, in creating more, in thinking more, we end up with more to act upon, to manage, secure and operate. For example, the UK government has found it can analyse legacy systems much faster – hurrah! (Big) but, that speed just creates a longer list of systems that are ready to modernise. Plus, goes the report, “AI amplified existing strengths and weaknesses rather than replacing them.” Nobody wants amplified weaknesses, trust me. Or trust Google’s Nathen Harvey, who says much the same thing of software development.
We can increase our output through AI, but that creates bottlenecks down the line shows Harvard/Jellyfish’s study of 718 firms, which found “little evidence that firms increase software output or reduce employment” through use of coding assistants and agents. Increased quantities of code just resulted in more to review and revise, creating a bottleneck. Theoretically, you just apply AI to that bit as well, right? Yes in principle, but according to Cloudflare (who saw the cost per review grow from $1.19 to $2.46 initially), delivering this (and getting it down to $0.77) took “obsessive” optimisation. Looking more generally than software, Gartner also report minimal (3%) productivity gains through business use of AI.
As per the site-down consequences of the aforementioned PHP and node.js vulnerabilities, we see a similar (and more concerning) bottleneck in cybersecurity. Whereas Anthropic’s just-announced OSS Scanner may have found more than 29,000 vulnerabilities in open source code, only 516 (or a few more by now) have been patched. Why? “we remain bottlenecked on our human capacity to validate these findings,” they say. Proof point: Bank of America had to hire an additional 1,000 contractors to work through all the vulnerabilities Anthropic kindly showed them.
Meanwhile Operations, the historical burden-bearer of innovation’s consequences, will no doubt be left to carry the can. Back in April, Gartner (via SD Times) were reporting that while agents might be automating the simpler stuff, that just meant service staff were dealing with the trickier stuff – which is becoming more plentiful. Then, a June New Relic survey of 200 technology decision-makers stated how “78% of organizations report a measurable spike in production incidents directly tied to AI code.”
There’s always going to be more to do, and AI will create more problems to fix, that’s the measure of it. I view with interest the posts by colleagues who can build things incredibly fast with minimal staff, and I applaud their efforts. Either the corporate and governmental world will literally be washed away by such upstart activity, or the upstarts will have to contend with the corporate. Until such time as I am proven wrong, I will continue to believe the latter.
Nobody likes the feeling of running across a field of soft marshmallows, losing first momentum and then all hope of success; but that’s the corporate environment, for better or worse, and AI will have to adapt to fit.
