This is a good thing. Repaying technical debt is always a great thing.
On our own 1.5M loc system, we've found and fixed thousands of bugs, security holes and circular dependencies in the last year. Code quality has increased, test coverage increased. We've also recently set our compiler to treat warnings as errors, and to be as strict as possible. It is absolutely amazing to see the gaps being filled, it would've been an impossible task for human devs and IDE's would've made it very uncomfortable. We've also ironed out all the bugs and failure modes in our infrastructure automation / monitoring. Our next project is to eliminate all 3rd party dependencies, and after that to rebuild our frontends completely using zero frameworks, not touching npm ever again after that.
So when I read this about postgres, I was very happy that they found and pursued this side quest. They can make the codebase bullet proof.
Avoiding 3rd party libraries seems like a false sense of security. You’ll stop seeing CVEs on the scanner reports, but the bugs are still there, just in your own code that is less well audited than public libraries.
I understand, but I would argue its better to have an in-house 1K lines of code that only does what we need and nothing else, it well tested and under our control, than to depend on some 50K lines of code (of which we might only use 10% of functionality and the others just getting in the way) that random's on the internet can modify, sometimes pulling in transient dependencies that we would really not have in a server environment, telemetry and other gremlins. It goes without saying that opensource packages will become even more juicy targets for malware/hackers to get into your systems like we've seen in the past month (and it will only become worse - you will be forced to pay technical debt or die).
The only "third-party" packages that we want to ideally allow/support would be provided from LTS versions of our runtime (in this case, packages provided by the dotnet team and audited by microsoft). So we will always try to build around what the framework provides. Second choice is our own code, third choice is open-source packages, fouth choice is a separate application/appliance (lets say ZenDesk for ticketing). Over time, we will reduce this footprint to only have dotnet team code and our own.
Remember most CVE's can be hidden on servers that are not publicly routable and don't have a real exploitation paths. The bits that are public (lets say a linux edge server with a web server on it, some firewalls, a couple of open ports) - these should be hardened to the max and not allow full access to all the services running behind it. We don't even give our internal services public IP addresses to make near impossible to route traffic to them.
Our git repo's are also not on public infrastructure (serious, think carefully about hosting on public services and if you really need it), never mind making the repo public.
We also have a dedicated secrets/certs manager, so no credentials are in codebases or pipelines. Very strict in this regard.
NAT (particularly NAPT) actually has the potential to lower overall
security because it creates the illusion of a security barrier, but
does so without the managed intent of a firewall. Appropriate
security mechanisms are implemented in the end host, without reliance
on assumptions about routing hacks, firewall filters, or missing NAT
translations, which may change over time to enable a service to a
neighboring host. In general, defined security barriers assume that
any threats are external, leading to practices that make internal
breaches much easier.
It's a defense-in-depth thing, not our only layer of security. So even if they somehow get in, they will find nothing again, every layer deeper becomes harder. Oh and don't count the decoys and honeypots we have floating around to keep the morons occupied.
> overdesigned/overcomplicated because everybody wants the library to do everything for everybody.
Also a source of significant dependency churn even excluding security bugs.
Whereas with my own code I don't have to worry about scheduling work that's only arisen because somebody outside my organisation has decided to make a breaking change in a dependency. Paying that tax got old over a decade ago, and my patience with it now is close to zero.
After Heartbleed the OpenSSL team upped their game so much, that LibreSSL guys said themselves that LibreSSL is obsolet now. So this no longer stands true. Just use OpenSSL.
As every cryptographer will surely agree, having every company vibecode their own bespoke in-house replacement to OpenSSL will be a great improvement to security across the industry.
The purpose of the above comment is to highlight the flaw in the original premise. Surely we all agree that we would rather not be using a vibecoded cryptography library. But why is cryptography special? Why stop there? I would also rather not be using a vibecoded regex library, or a vibecoded codec parser, or a vibecoded web framework, etc.
The point being, if you weren't committed to maintaining your own implementation of a library before the advent of LLMs, having access to LLMs does not fundamentally change the calculus. The ability to generate code more quickly does not automatically grant an organization the capacity to maintain more code.
You might believe that your own 1 KLOC is well tested but you're probably wrong. As your business evolves and use cases expand, someone will feed it bizarre inputs that you never thought to test and then it will break. I've seen this happen over and over.
Logging is a very complicated security area. You often get sensitive data in the logs and it gets sent across the network to monitoring servers.
Also any part of the program can send data to the logger. This means the logger can receive malicious payloads from anywhere. Maybe your logger handles ascii control characters badly?
I dunno I pretty much always wrote my own logger. Not even a shared library or anything just copypaste into next project or often don't even bother with that, just knock something out on the spot. Even before AI.
The only even slightly tricky part is like "don't mess unicode up", rest is straightforward. Sure it can be complicated with logging sinks, configurability, when to flush, monitoring and so on but in many cases you just don't need all that. You just build it whenever you do need it.
For example Minecraft uses log4j to write messages to the console and a rotating file.
Technically you can configure it to send to other places - presumably. In practice you'd be better just redirecting stdout, tailing the file, or installing a mod?
Honestly unless you insist on allowing running your software on Windows you're much better off delegating the logging to syslog over a thin wrapper. It can handle about anything except most exotic cases. This however goes against Java land portability promises and its NIH attitude so we have what we have.
I can’t be bothered to check the exact assembly to see if timing attacks are possible.
But crypto really is just a small thing. I’m not going to design a nuclear reactor but I’ll definitely do stuff that depends on the electricity from one. Similarly one can still do their own authentication; its not like even Apple forgot to rate limit…
It really depends on what the product is, what the 3rd party library is used for, and what the alternative is. Unfortunately, some people are too biased towards using 3rd party libraries and don't understand the tradeoffs.
For example, a few years ago I removed a rather large 3rd party mathematics library because we were only using it for a single mathematical function that was very easy to replicate.
Remember, 3rd party libraries need to:
1: Be periodically audited to make sure you are in compliance with the license terms, and preferences of your company.
(1.5: This becomes important during an acquisition as someone will always come around doing due diligence.)
2: Be periodically updated (in case of security bugs)
2.5: Be updated on short notice when a security issue is revealed.
In the case of the mathematics library, replacing it with roughly 20 lines of code and 100 lines of tests is a significantly better tradeoff than constantly reviewing it every time one of the above things happens.
Now, at what point is it "worth it" to use the 3rd party library? In our case we'd need to make much heavier use of the library, probably at least 20 different mathematical functions from it.
It has been proofed again and again that you cannot trust npm/cargo not even AUR, implement you own minimalist functionality without dependencies and just for you own needs seems to be a much better solution.
I personally only rely on 3rd party libraries that are complete. I will never use a library that gets consistant feature updates. I do make an exception for openssl rusttls and whatnot since they have to update by design.
The one thing that cuts the other way is code churn. Third party libraries may keep adding features you don't use, and those may expose new bugs. You can pin an older version, but unless the library back ports security fixes, you are not going to benefit from the shared development wrt to security.
If you spend a ton of time hardening your own library code and don't have much feature sure, then that can be a win.
You’re probably right about some third party libraries, but not about all or even most of them. Package repositories are full of half baked packages that are the result of a single person scratching their own itch once upon a time and thereafter forgetting about it. The more harmful part happens when those get imported into other third party libraries that give the image of being better supported. Supply chain attacks are the new null pointer.
I would like to agree too but there's a genuine new debate here, with LLMs/agents, the storm of CVEs, supply chain attacks, AI assisted social engineering, etc, some custom code doing the minimal required task might be a better security posture than picking a popular library.
I don't know were it will leave us in the long term and I don't think I'll like the answer.
> ...some custom code doing the minimal required task might be a better security posture than picking a popular library.
Only where the AI solution is verifiable by the dev doing the promoting and review. Cryptography is probably better served by libraries vetted by experts.
An ironic analogy, because most people do exactly that, most of the time. How many people with money directly or indirectly invested in Alphabet or Ford actually understand those companies' financial positions? And I don't mean "has read the filings". I mean understand them to the same depth that a professional security researcher understands a cryptographic implementation.
We routinely place trust in things we cannot personally verify.
Ultimately the guarantor of cryptography code won't be humans, it will be councils and red teams comprised of other LLMs.
Public software is going to be attacked by many more dollars of AI time than internal stuff. In a year the library code will definitely be more secure against the next X dollars spent on cracking it.
> A competent hacker or AI will quickly find vulnerabilities in your custom libraries.
So why wouldn’t a competent AI find those vulnerabilities during development? I don’t think the proposal is to have human devs hand write the custom libraries and yolo into prod.
Swords and shields. The advantage of the defender is that they have access to the code (usually not available to attackers). The advantage of the attacker is that they have skill and nowadays the most up-to-date models and SOPs for finding attack vectors.
in theory, a company running custom software should be able to invest many more hours into running those same AI with "white-box" pen-testing (assuming hyperscalers actually start allowing this). An attacker would have to be willing to outspend the company (or get very lucky). If an attacker is resourced well enough to put that investment into owning a single company, you probably don't stand much chance reguardless.
Also, the attacker would have to run black box against the live environment which is much more difficult and likely to be noticed than offline white box.
If you reduce the amount of code/code paths you reduce the surface area that vulnerabilities can appear on. A code base with 10 million lines can have a lot more bugs than one with 100k.
Not having access to the source is certainly a disadvantage for LLMs, though I saw an article the other day about a model running a disassembler on the binary, from that reconstructing the roughly equivalent C, and finally analyzing that. So, even if the binary is available there are paths.
Yes but that's not entirely obfuscation, see ie what google does to their proprietary endpoints through protobuff. Something like that but on steroids could skip the majority of attackers using LLM scanners.
One problem I’ve seen with third-party libraries, especially in the JS ecosystem, is that to fix a CVE, you often need to jump a major version or three, and that entails quite a bit of work to adapt to breaking changes. Porting a fix back on your own and keeping your own fork is the same as supporting a first-party library, except it’s not really first party, and you’re not familiar with its innards.
I think avoiding all 3rd party libraries is excessive, but engineering is about tradeoffs. In the extreme case, do you pull in a LeftPad dependency or write it yourself?
> Avoiding 3rd party libraries seems like a false sense of security
To be fair, there is more nuance to your statement.
A lot of it depends on your programming language.
For example with Go, with its extensive and excellent stdlib, you would be foolish to rely on large swathes of 3rd party libraries, because so much can and should be done using the Go stdlib.
On the opposite end of the spectrum, you have Rust with an effectively non-existent stdlib.
And the problem you see frequently with Rust is projects importing half the internet, with a Cargo.toml the length of your arm with hundreds of dependencies. Of course the project has never reviewed its dependencies source code, and the project probably barely keeps its dependencies up to date. Its a supply-chain attack waiting to happen, not if but when.
So with languages like Rust, you have to take an approach of "minimal reasonable dependencies", make an active effort to keep your dependency list short and in-control.[1][2]
> less well audited than public libraries
I also struggle with this particular argument because the reality in open source is "well, its open-source, it has many eyes" is in fact mostly a dream. Since we all know 99% of the time its a small group of maintainers (assuming there is are even multiple maintainers[3]) and not many (if any) external reviewers.
This is the way! What also affects the decision is what the code will be doing, and how much extra functionality that you will never use (extra attack surface) you will get by importing external library after library.
Often it can be a very good choice to write the functionality yourself and not rely on external libraries.
We haven't fired a single person yet due to AI. We just save everyone time, make input more accurate, make our accountants better, manage our technicians and jobcards better and so on. We also made our system bullet proof to the point that we only had one weekend failure in the last 24 months, and it wasn't our fault (a third party dependency did a large deployment on a friday night). Our system is now deeply "boring" and its awesome.
Oof, we are 1/2 the size we used to be so we can do more with less. We're doing more but with half the people there are still so many gaps that I'm quitting myself out of frustration.
Perhaps the ultimate use case would be to use AI to scan, debug and find issues etc, but never to generate new code. Still have humans writing and understanding everything, keeping things as clean and concise as possible.
It would be great net result for our brains, however too much money has been poured into this business to give up on that. My belief they will push for SWE layoffs trying to position LLM-based agents as a cost efficient alternative to human programmers. Which is extremely stupid and short-sighted but that's the strategy.
They will learn certain lessons the hard way for sure. As much as I want to spin up 100 agents in a loop on our codebase and let them loose, I also know its not feasible as they can easily go down the wrong rabbit holes or over-engineer the wrong solution. They are amazing but cannot be left 100% autonomous just yet.
Literally half of the work we did the last 12 months was this. Refactor and fix, performance improvements, security hole fillings etc. We have one or two devs doing visible feature work, the rest of us is making the system rock solid.
The only companies who will really need to fire are the big tech companies who hired just because, for empire building, or to keep talent out of competitor's hands. Nearly everyone else is actually positioned to leverage AI to now compete against the bigger players. I'm already seeing small companies have a big surge.
This is how I use it. I don't run any companies (nor am I a part of one right now) but were I to run one I would employ AI as the ultimate bug/vuln hunter. Just spin up maybe 20 agents and say "fuzz this thing to death".
> rebuild our frontends completely using zero frameworks, not touching npm ever again after that
Genuine question, because I don’t know: Wouldn’t it suffice to vendor your current packages and “only” stick to maintenance from there on out, instead of rewriting it all?
This should still achieve the goal of not needing to touch npm, but presumably with less effort/tokens, right? Or is it just that you would expect the result of that process to be less clear and maintainable than a full rewrite?
Actually, I have started with it already but is paused, but the approach is to rebuild it from scratch in parallel to our current legacy system and move our staff over to the new completed screens in batches (we have close to 1000 views in our system, 150 background jobs, with about 15 main categories (think organisational sub-divisions)). So as long as we go slow and steady (even with AI) it will work out fine, I expect it will be done in less than 6 months. One month if we rush but we don't want to rush it, as we are moving some things over from MSSQL to Postgres, so slow and steady is the way.
Mmm what else. I prefer postgres so I'm very excited about this. So many foot-guns will no longer haunt us like on MSSQL and less db/table locks to try solve (pessimistic vs optimistic locks). Oh and we are going to save a gross amount of money per year, as we will also be able to switch off a couple of Windows Server VM's in the process. These puppies are costing us a ton (I think about half of our infra bill are licenses...).
Just create a branch, spin up a test db and try to port a small feature. Its very easy in most cases. Debian + dotnet + postgres = dream team.
If you only use MSSQL as a datastore, it's not too bad. If you used all the other features (MSAS/SSIS/etc...), it is much harder. PG also isn't a panacea - you're trading one set of issues for another. The biggest upside is that it's much less expensive :)
MSSQL query optimizer is one of the best available. If you have truly insane queries, views needing joins on views and CTEs, etc, you might be surprised at how well SQL Server was handling them versus PostgreSQL.
If the parts you need from a larger third party library are small and simple, that makes sense. But I don't think the general idea of reusing battle tested code across many projects is obsolete because of AI.
I think the approach should probably move towards semi-tailored systems.
Maybe you don't need all of Postgres - so provide 2,3,4,5,n versions. As long as every version is covered by (possibly shared) tests, then you can release what works, ignore the rest (bugs and all).
Exactly. And over time a project can drift from its original purpose, so you end up building around a bunch of stuff you aren't using anyway, but you pay cost/risk for having it present.
AI makes it possible to fix bugs and solves issue one wouldn't do before. But just because it's easier now doesn't mean it's automatically worth it. There are tradeoffs (like code complexity or breaking stuff), or it really just wasn't that important to fix if it was in the backlog forever. I've seen devs go down this rabbit hole, and perhaps overdo it. It feels productive, but at some point it might stop to provide value, and other things should've been prioritized. It's just easier to say to the AI "fix bugs" instead of solving real issues. At some point boasting about running 10x agents in parallel to fix bugs becomes a sign that there's a lack of value provided to boast about.
Not all 10 are doing bug fixes. Two of them are mobile apps, 5 of them are long-term feature work, 1 is infra/monitoring, the rest is adhoc stuff like bugs.
While that is true, there's also the other side of the coin: most bugs are hit, but never reported. Users are pretty unlikely to report bugs and when they do they are very vague about it. That's why client observability is such a boon for teams because they find what bugs users are actually hitting. If the agents are directed using that, then perfect
Like the warnings as errors, and extensive test suites, there's also type safety that helps stir the LLM in the right direction.
You can go with "eliminate all 3rd party dependencies [...] frontends completely using zero frameworks" and be stuck with JS. Or use a typed language (TS, ReasonML/ReScript or even Elm) and an API client generator (available for all those languages) and get type safe API calling from a more strongly typed language.
As much as I prefer not to use NPM and even more prefer (even my LLMs) barely touch JS. A bit of safety is cheap these days, as you rightfully pointed out.
Good thing is now human does not have to justify anymore. If AI says bugs management seems far more amenable to support fixing. At least in my experience managers take AI reports or vendor tool generated reports more seriously then any feedback from engineer.
Didn’t you guys have to ship new products? How a business can afford so many backlog tickets being implemented? I understand fixing bugs can also mean improving the product but still - (almost) the whole world has this balance between new features and acceptable-ish number of bugs
We are multiple developers, each having 10+ main claude agents running different sessions/terminal tabs. I have 10 copies of our codebase on my laptop, each with their own purpose and claude session and history. The other devs have different approaches but we all get similar output. And then we also built an agent framework that runs within our system that has access to our code (at most 60min behind master branch), our infra and our databases, so it can diagnose issues with excellent precision and prove it to us where to find code/data issues.
The entire industry has shit software to be honest, our company has the best or most modern system of the bunch in our country. Half the industry does everything via clipboard & excel.
Short term there will be pain (hacks/malware) but at the same time, it will force the survivors to build bullet proof systems.
Worse case, we enter the Dark Forest era for technology. Read up on this concept and apply it to the tech industry. I suspect open-source might die and everything becomes private, no more public networks, no more open access, as to assure survival. In a sense we are halfway there already as it seems the internet is starting to close down, people not posting their art and music as much anymore, not posting their knowledge for free anymore, as it might be used against them by AI (the other party in the dark forest..). Thus private authentic experiences will become more valuable.
Can you describe how you arrive at the premise that people stop sharing because of AI? As a counterexample, I just tried out a dev's vibecoded app 'Glaze My Numbers Baby' and it 1. works and 2. is ridiculous and cool. Totally vibe coded. Claude rewrote all the C++ tests in Rust, so there's at least some substance to the rewrite.
1. The cost of running AI is dropping down to "free" - there's open weight models, open source tools, and guides on running your own if the long list of suppliers isn't good enough for your use cases.
2. Vulnerabilities have always been a risk; AI is not a panacea. The pre-AI best practices still apply. Furthermore, a huge percentage of vulnerabilities are in 3rd party tools and systems, which are systematically being scanned and fixed and/or exploited, then fixed.
You mention "the free software community" in your last sentence though - this implies the code in question is open source, so it's able to receive plenty of improvements from anyone with access to these tools.
Can you tell us how did you achieve that kind of work, briefly? I struggle to figure out how such massive bug fixing campaigns or even large scale software engineering are done with ai not taking weird decisions...
I personally started using AI with a very rudimentary "memory" system, and for first weeks corrected it ad nauseam, until, in the small domain I was responsible for, it started doing work end to end fully autonomously.
Takes a lot of time and effort to bootstrap, but once steered well enough, it just clicks and less and less is needed to check manually
Yeah a memory system per domain, or per module or even per file works amazing, we have it too. Took a ton of time to improve the code and list down all the tribal knowledge, but now that it's done, it works super well.
Myself specifically, I have a /code folder without about 10 copies of our codebase, each running their own claude sessions & git branch. Then, I have 10 terminal branches. I spend a ton of time planning out & specing the problem with claude, to be as specific as possible and be as constrained as possible so it cannot go on side-quests. I use Opus 5.5 on High effort. That's just on my laptop with what I can mentally cope with day to day. Then we have some other automatic ai goodies running within our custom agent framework that runs within our platform that also helps. Then for apps, I basically plug in multiple phones into my laptop with usb (and emulators if I really need to), and claude can fully drive the ui and testing on the phones via adb as an example. Even on linux/fedora/kde it can drive UI apps a and take screenshots and fix things properly etc. It can ingest all logs & debug chains, so it absolutely flies through its task.
Before we started this journey (about a year ago), we spent a ton of time cleaning up our codebase, splitting large files (10K+ lines) into files that are no more than 2K lines, things like that. Then we let claude scan our important files and summarize them, add comments for future claude sessions. So we've let it built up tribal knowledge per file and the reasons for things to exist. We tell it all about our operating environment, the country and legal framework etc, so every little piece helps it stay honest and on track.
So to start, do something simple like: Scan the files in this folder for database code and check for queries that loads the entire result set into memory and/or queries that loads all columns. Suggest which queries we can filter better or reduce columns being transferred. Once done, also check real db row counts, compare them with the queries in these files and check if they might benefit from an index or partitioning/sharding. Also check for queries that loads data but their results are not used. Don't make changes yet, just do a deep investigation and report back. Feel free to spin up muliple agents to batch the scans.
edit: it goes without saying you need a language with strict typing and a good compiler (we use dotnet). Doing this on python/javascript would be a nightmare.
> I spend a ton of time planning out & specing the problem with claude, to be as specific as possible and be as constrained as possible so it cannot go on side-quests
I still have yet to see this be efficient. The time I spent doing this is way less efficient than simply giving the LLM some high level direction and putting that time towards reading the code it generates and correcting small piece by piece (occasionally doing things manually, but that’s only for more off the rails stuff).
It’s about the same amount of time as up front planning, except I know the code much better as well as end up with less mistakes compared to having it implement a front loaded plan.
Knowing the codebase makes everything else 10x more efficient too - triaging bug reports, planning other work, knowing what work needs done (this is a big one), reviewing PRs, etc.
When you change a single setting in the IDE to make something more strict and you have 1M+ lines of code, you will get 30K items to fix, via red/yellow/blue squiggles in the IDE. Most of those can be simple refactorings, but the performance & UI of the IDE makes it incredibly unwieldy and it will sometimes fight you back, sometime crash internal processes which leave refactorings half baked and so. Now imagine you change 50 such settings, to make the compiler force you to write better code. Changing 200K+ lines of code yourself is a nightmare (and to prove it still does the same thing afterwards). Instead, claude systematically approaches whole classes of errors and convert them to cleaner variants (either using our custom skills or it writing new scripts to do it). I consider myself an advanced user of IDE's, know how to do bulk refactoring and renames and so on (esp with JetBrains tools), and I would say it would be a near impossible task to do as a human. Then there are other developers - so you flip a setting and now 5 other people's IDE's are forcing them to work a specific way which will make you unpopular very quickly and revert the changes. So, since most of us no longer use IDE's as our main tool to build/diagnose, we can more easily flip compiler settings without destroying someone else's week. Hope that makes sense.
Oh it also helps to split refactoring work into three buckets: business logic changes, performance changes (same output as before but faster) and cosmetic changes (rename files/folders/classes, breaking large files up, add logging/comments, domain layering etc). We tend to focus on performance and cosmetic changes for bulk refactors and leave logic changes to more dedicated sessions as to avoid changing what the system do (our domain is fairly complex and we cannot afford breakages). We also managed to port ALL of our code to dotnet 10 lts and have clean boundaries between projects (we have a mono-repo with about 50 projects in it), so we can wholesale upgrade dependencies without a sweat.
IIUC, your point is that changing IDE settings will affect other developers. But can't you enable an inspection locally, apply all or some of the quickfixes and revert the setting before committing? In the longterm, you want to enable the inspection anyway because the point is to prevent developers or agents from comitting new code that violates it (agents can access JetBrains IDEs' MCP Server to get inspection results).
Of course, crashes and bugs can happen when applying quickfixes, which isn't nice. Ideally, they should be reported and fixed. You can also ask an agent to fix the occurences instead of using the IDE quickfixes if that's blocking you.
I guess what I'm trying to say is that IDEs shouldn't make that process harder, they can actually improve agents' results. For example, you don't need to write custom skills to prevent agents from writing some class of bugs if the IDE can detect it and agents have access to the results.
> Changing 200K+ lines of code yourself is a nightmare (and to prove it still does the same thing afterwards).
If an agent does it, you still have no proof, but okay.
Full disclosure, I work at JetBrains, but not in the IDE department.
The problem is, even if I only enforce some rules locally, another developer has a different set of rules. So when we git commit, we ping-pong changes and fix each others stuff over and over again, bloating commits. We used editor config files + analyzer configs in the past (that gets checked into version control too), but there were still too much friction.
Where now, the compiler flags & strict analyzer settings forces claude to do the right things (we explicitly tell claude not try and bypass or lower the barrier). I think it only works because we don't spend 95% of our time in the IDE anymore like a couple of years back, so we as developers no longer feel that friction, as claude tries to satisfy the compiler instead. Its a massive mental load that is just gone, we haven't had any conversations in meeting about this kinda stuff anymore (remember in the past we would have flameouts in meetings or code reviews about linter settings...).
> it also helps to split refactoring work into three buckets: business logic changes, performance changes (same output as before but faster) and cosmetic changes (rename files/folders/classes, breaking large files up, add logging/comments, domain layering etc).
Do you have a specific skill to guide an agent to scope the work of an agent so it stays in one particular bucket? Would you mind sharing it?
State of the art agents don't usually need such fixed guidance anymore. I've been using Opus 5.5 with Xcode and simply prompting things like "make sure all of our test Suites use the .dependencies modifier, but only if they use dependencies internally" or "make sure none of our test methods are @MainActor, just access the necessary state using standard concurrency constructs". For larger changes, providing the examples you want is often enough to get it most of the way, or even all the way, there. I've had simples prompts for things like "move all standalone SwiftUI view that seem reusable to our Design framework, and create a list of those which can't be moved without effort, listing by complexity".
Overall, there's much less need for prompt skills anymore. They're still very useful for things like best practices, or things you need to make as reproducible as possible, but big skill and planning sessions aren't nearly as necessary anymore.
As I read on I thought this might be satire, but, I suppose every two weeks this philosophy becomes less hopelessly optimistic and more feasible.
Perhaps your workflows and processes are so efficient that this actually makes sense. But I’m still not convinced AI assisted development is at that point yet; I think the classic Build vs. Buy arguments still hold.
How much is this going to cost to maintain? How many dev hours? How many tokens? Is your in house solution going to provide the business and customer enough perceivable value that it’s worth the opportunity cost? Are you going to apply the same rigor and security testing that the popular frameworks are getting from the largest research firms in the world? How do you know the LLMs aren’t going to apply similar patterns and make the same mistakes? Why can’t you just use that AI expertise to accelerate patching so your exposure time is limited? How do you know you’re going to outpace the major frameworks and tooling providers in 2 or 4 or 6 years when they catch up with AI assisted development, if they haven’t already?
And very personally speaking, I’m still holding my breath to see if this AI bubble is a real thing or not. Because if it ever pops, well, that’s not going to go well for companies that are embracing this stance.
If you're going to roll your libs, be sure to set up agents to scan for CVE analogues of equivalent known vulnerabilities in your own code. It takes quite a bit of work to set this pipeline up, as well as manually selecting high-impact CVEs (relying on the public scores is worthless, because there are some vulns that are catastrophic with a score of 6 vs something meaningless thats a 10).
It's also crucial to scan for "analogues" because otherwise many LLMs autoreport as "not vulnerable" since "this applies to Postgres/whatever and this repo isn't using it"
If you get hit by cyber guardrails you should use a local model to prune any CVE specific language from the reports.
To give examples, every single LLM (and people too tbh) will write netcode that's vulnerable to these CVEs: CVE-2026-27135, CVE-2024-45311, CVE-2021-32567, CVE-2019-9511; without explicitly mentioning the CVE text no LLM finds these either.
You never wanna intentionally repay tech debt as slowly as possible. You might be able to tolerate/ignore it for longer depending on the circumstance, but you always wanna get rid of it sooner rather than later.
> Our next project is to eliminate all 3rd party dependencies, and after that to rebuild our frontends completely using zero frameworks, not touching npm ever again after that.
What's the proportion of bugs you've encountered that was due to a library versus due to your own code? I suspect it's probably more than 1:1000.
False dichotomy. A library that is not a 100% perfect match for a problem you have to solve, will require glue code. Any bug in the glue code is in your own code but only exists because of the library.
I’m not going to eliminate every single third party dependency in my system, but I’m on my way to eliminating 90%, maybe more.
For me it’s less about bugs and more about supply chain security risk. And with AI and only building what I need from a dep, exactly how I need it, the cost of doing this is very low.
I worry that the move towards eliminating library usage will lead to further code fragmentation.
It's a nice idea, and in theory sounds great, but you may be trading 1 library with 1000 bugs for 1 million implementations each with one special bug...
The JS frameworks exist for a reason, recreating them seems a little excessive. Why not skip JS and go to a Rust wasm framework or Phoenix LiveView if the goal is to get away from JS/node/npm?
Our legacy system runs mvc view with bootstrap 3 and some jquery. It's not a SPA, so no framework to replace. But we still have to mess around with npm sometimes.
> This is a good thing. Repaying technical debt is always a great thing.
> Code quality has increased, test coverage increased.
And that's true for many old codebases!
> We've also recently set our compiler to treat warnings as errors, and to be as strict as possible.
Yeah. Everybody who said it was impossible, that no budget would ever be allowed to fix old bugs, that new features "to support the business" was the only thing that mattered and other nonsense like that have been proven wrong. Arguably they were kinda right but then LLMs happened and that's what proved them wrong.
I love to see what's happening: at very long last people are understanding that security is something to be taken seriously. They're understanding that "more code = more bugs".
> Our next project is to eliminate all 3rd party dependencies,
I love the spirit!
> and after that to rebuild our frontends completely using zero frameworks, not touching npm ever again after that.
I hope many more shall realize that now we can give the middle finger to a great many pathetically bad and totally unfixable (even with the help of LLMs) ecosystems.
Our codebase is about 8 years old, it used to be a .Net 4.8 project, which was upgraded to .Net Core 3, some bits to .Net Core 6. It was a mess. When I joined upgraded the whole stack to .Net 8, then .Net 10. Next upgrade is for .Net 12. I agree the old code was messy (single monolith, one solution, one project, zero clean boundaries) - I have since refactored it into a pseudo domain-driven-design layering which has enabled us to rewrite certain parts with low risk. Also stripped out business logic and third party integrations into their own projects. So we have about ~50 projects in the solution at the moment, not counting our mobile apps and agent framework. We also rebuilt our background services stack and got allll of our services to complete their runs under 60min, so no more "system is slow queries". Since converting it to use .Net 10, we can now also host 99% of the thing on linux servers (we are about halfway with this migration). So our cleanup efforts has been amazing, our server bill has been reduced by about 50% in 2 years, and next year we will half it again as I'm busy rewriting our most expensive system (a shitshow that was built by a consultancy firm, as usual).
> This year is going to be different. PostgreSQL 19 most likely won’t be released until November… […] So, what happened? The short answer is that LLMs happened. […] There’s a huge amount of code, and that code contains a huge number of bugs. […] What’s different now is that it’s become vastly easier to find those bugs.
This seems…good?
That it might disrupt the project's "traditional" release cadence a bit doesn't seem terribly bad in light of the alternative, which is shipping another major update with bugs which have apparently haunted the codebase for some time.
It's not like no work is happening though. Your clients are getting a more secure website. Maybe the bugs that are being patched would have taken longer to be exploited in the pre-LLM world but the bugs were still there.
Most clients will not agree to pay extra for a more secure web site though. Unfortunately. Unless they’ve been bitten in the past, which most businesses still haven’t.
As long as you put the alternatives clearly on the table, you could let the customers choose, and sign off on taking the risk. You can give a recommendation, but ultimately it's their choice if they bear the consequences resulting from the choice.
Hard disagree. Framing it this way makes it seem like all possibilities are equally valid options. Yet you’ve somehow winnowed the field of infinity to the several options presented. You did that through successive rounds of professional judgement that eliminated options you did not consider fit to propose.
Taking options off of the client’s table is the core responsibility here.
Um, if you clearly describe what options A, B and C with their costs and consequences are and advice them to go with C but they go with A, then they explicitly did not go with your advice and can't blame you for their choice.
In the end, it's all business decisions. If it's something I don't like as a customer then it's to be infantilized.
(If you disagree then please don't complain ever again about LLMs refusing to help you reverse engineer a piece of code or explain how a virus works.)
Businesses that are paying attention to the winds though have already figured out that the internet is a hostile place: both their insurance companies and their governments are making noise about security.
> Probably not, if you factor in the cost of security exploits. A single compromise can potentially sink the business.
Agreed however getting humans to understand probability is tricky.
I had this discussion with my son recently about considering the severity of outcome multiplied by the probability of that outcome been the way to look at some things.
Same here. I got access to Mythos two months ago, it confronted me with decades of nasty bugs I wrote. This has been stressing me out a lot, caused sleep deprivation (the bugs follow me in my sleep, yes, really). All other progress is halted, I've been squashing bugs day and night. I see a light at the end of the tunnel.
Bottom line: it's a good thing. My software will be better after that. Even if it's currently not good for my ego (and my health). But I'm glad for all the tools that help me improve quality.
And sure, I can just ask LLMs to fix the bugs, which works well half of the time, and produces bad slop in the other half. This is where human expertise and judgement is still necessary to avoid letting our code bases drift into LLM spaghetti hell.
My question is, did it provide POCs? I have had some security issues flagged that I tried my best to exploit, but failed miserably. Since I do not have any kind of privileged access the LLMs I use have been unwilling to actually prove that it can be exploited.
While I did fix most of the things because while they might not have been exploitable they were bugs waiting to happen, in all but one case I haven't been very impressed by any of the LARGE IMPORTANT SECURITY HOLES that were found.
No, the Mythos reports do not come with POCs. It's a lot of redundant prose explaining the bug it found (much more than I'm willing to read), it explains how it can be triggered, but doesn't generate actual exploit code.
But that doesn't matter for me. It shows me real bugs and less than 1% false positives. Some bugs are truly nasty and urgent, most are trivial and not exploitable at all, but they're bugs and so I'm fixing them.
Mythos isn't mythically superb for being the best LLM model, but only for not having these annoying guard rails that make all others randomly refuse to cooperate when you need them the most. Whoever invented guard rails - please stop! - they just slow the good guys down in the race.
While it is certainly good to do estimation oneself and not just blindly follow whatever the LLM outputs, keep in mind the recent PHP-related events ... Just because at some point one is not able to make a PoC, it doesn't mean, that no one else ever will be.
To err is human, I know that as much as anybody. Before the advent of LLMs, there were several (human) audits of my code for vulnerabilities, but they found little, which I was proud of. And now, Mythos had a quick look and found hundreds and hundreds of bugs (many of which were critical and even more were really emberassing).
Of course, I knew there were bugs that just weren't found because the auditors weren't looking closely enough, didn't have enough time/budget etc., but this unspecific feeling is different from having all the bugs accumulated over a quarter century slapped in your face at once.
Code auditors are a crapshoot. I worked around financial auditors and they were much more consistent but thats likely because standards are more well defined.
If you get a good auditor they can find a ton, but most are box checkers there to make the audit company as much money as possible.
Quite possible. Ours were good, they just didn't benefit from LLM superpowers back then.
After my Mythos experience, I was wondering if there is really still a market for human code audits. I guess yes, but the work will be different, just like coding (and marketing and so on...) is different now. You still need to have an expert to wield the new powerful tools.
How did you get product management to agree to that? Ours said "I could fix all of that in 2 days with Claude." They are still not fixed. Granted, our PMs suck.
The upper management and security team came knocking and told us all of the CVEs from the scanner need to be addressed right now. And the only way to deploy without having to wait for QA approval is to not merge feature changes.
Imagine that the whole city has to install iron doors and window grilles, because wooden doors and glass windows are not safe enough anymore.
That's a lot of work, but is it productive in the end? Probably not.
With AI it's the same. World was fine with lax security approach before. Today it is not fine. Tomorrow we might need to prove every code line before shipping, will that be more productive? Hell, no.
> Imagine that the whole city has to install iron doors and window grilles, because wooden doors and glass windows are not safe enough anymore.
> That's a lot of work, but is it productive in the end? Probably not.
Your imaginary scenario defines the solution to be pointless. If that same city was subjected to weekly riots smashing windows & doors, and people regularly died, stopping "productive" work to install things that keep people alive and productive? That's productive.
There is a sense in which productive is being used here that is slightly subtle. Security doesn't really add anything to net output. It is, in other words, a cost. That doesn't mean it isn't valuable.
One way to think about it is that production allows you to pay for stuff. Not necessarily directly but by way of trading. It is the output you use to buy things like security.
Consider an apple farm. What does the apple farm produce to pay its bills? Apples. They could also dedicate resources to pears, &c, and that would add to their production -- to the goods they produce, the goods with which they (indirectly) purchase tractors, fence posts, &c. They could also dedicate resources to security -- more fences, &c -- and this may even be prudent. Maybe thieves are getting taller. This doesn't really add to production, though; at best, it maintains production at the status quo ante (assuming the fence is effective).
True in the sense of physical security, but I think it's not true here.
Security bugs are just bugs, as Linus likes to say. And the bugs are often also crash or correctness bugs (use-after-free for example usually causes segfaults). For the most part this actually is a productivity boon in the sense of getting higher quality goods out the other side.
To use your apple farm example, it would be like an apple farm where, if the apple rots, it's sometimes just gross but also sometimes it explodes and kills you. Would it be productive to reduce the rot rate of the bomb-apples? I think so!
It's a terminology issue. Economists use the term productivity in a certain way and they even use the term "broken window fallacy" to describe this debate.
Basically, if you accept this line of reasoning, then someone going around breaking windows and someone else going around repairing them can be seen as good because it creates jobs. But that would be nonsensical and create poverty. That's why economists focus on wealth creation not job creation.
Fixing security bugs is only needed because people exploit them. It creates work but it doesn't create wealth.
Fallacy applies only to incidental breaking of glass like child smashing window paying football.
Protecting yourself from bad actors also doesn't create wealth but it protects wealth you already have.
You know cybercrime and APT groups are not bored kids in their parents basement, there are groups that are running automated tooling to find vulnerable companies and if possible take all their wealth.
Just look at Binance of course, but if you are small company there are also people targeting those especially with AI assistance nowadays.
>Fallacy applies only to incidental breaking of glass like child smashing window paying football.
It does not. Scroll down in the given article, and you will find war as another example.
As GP said, it's a terminology issue. War spending, including defense spending, is not productive, because it does not produce wealth. It does not mean that protecting yourself from enemy countries or criminal is useless, or not worth it.
Nobody claims investing in security is a bad idea. It's just not productive, in the economic sense.
Just like securing a system is a trade-off, attacking one is too. Risk vs reward and time vs reward. Your personal blog doesn't need to be fort Knox, just patch it properly and you should be fine.
The software is better in the end but not all of it will be productive.
This why we pay cybersecurity folks, developers and business people are often living in a fantasy land where people don’t steal bikes because it is illegal to do so.
Then they have a pikachu face when someone smashes their car window to steal something worth $50 they left on front seat because “who would do that?”.
What is even better thief will most likely pawn this for $10 but for him it is effortless $10.
> If that same city was subjected to weekly riots smashing windows & doors, and people regularly died..
The point it, this was not happening before. LLMs made cheap cyber attack possible. But it didn't make the implementation of fixes correspondingly cheap. So now everyone needs to spend extra money/effort to install grills to cover where LLMs fall short.
For postgres, this absolutely makes sense though as long as people are willing to shell out the profits from LLM use to support postgres development. That is one way this could workout for everyone..
And now imagine the introduction of iron has led to crowbars that make smashing doors a lot easier. Now it is quite unreasonable to account the reduction in smashing doors to iron, when the previous improvement was also due to iron.
The physical world comparison is interesting. In the real world we have a small amount of security, but most things could be broken in to. Instead of fortifying everything, we just imprison the people breaking stuff.
Obviously this doesn't work on the internet since most hacking comes from overseas out of our control. Which makes me wonder if there is a possible future where we just block packets coming from countries that don't lock up the hackers.
Not obvious to me. I am generally a fan of Cyberspace manifesto and the free Internet, I think it's great people can reach outside their ingroup and talk to strangers and ostensible enemies.
But, that doesn't mean we should give up physical security, the guarantee that the connecting party comes from a certain physical location (state, company). I think allowing connections from anywhere by default is fundamentally flawed security architecture, for certain things.
I think the physical world comparisons should keep in mind the kind of data a lot of these systems (eg postgres) handle. It's more like securing a bank vault than locking a home's door.
People usually don't install steel doors because most burglary is prevented in other ways. Insecure software will be exploited this way or another, if the vulnerabilities are real and the threat model is valid.
> Insecure software will be exploited this way or another, if the vulnerabilities are real and the threat model is valid.
The algorithm is: if the threat model is valid, there are vulnerabilities and you either have some software everybody knows about or a gang of hackers specifically targeting you, you should patch.
The part of the industry that profits from security theater will want you to patch no matter what, even if your business goes down in flames due to downtimes and new bugs introduced by patched components. They won't remind you to assess the threat model first.
There has to be a responsibility model for that; somebody who can say “we don’t do X even if the auditors say we should, because Y”. But I see where you are coming from, I’ve seen security theater to cost $$$$$$$ at the org I’m part of.
By this line of thought building nice and usable UIs is not productive too, because users can learn cli tools (they did, DOS) and it costs less to develop them. And all the work in TLS/HTTPS has been wasted because HTTP is enough. And ssh vs telnet.
It becomes productive only once your business stops for days because somebody brought it to a halt by breaking into the servers and deleting them.
If one thinks that "Security work is not productive."
By the way, I think that even without safety concerns seat belts are productive because they anchor the driver more firmly, so we can brake harder and turn faster (a lot of roundabouts here) and get to our destination in a shorter time. Of course this is against general road safety.
In the rest of the world making sure your product works like it is supposed to is considered table stakes. In software we somehow consider checking the parts of it we can think of 'good enough' and the rest unproductive
> In the rest of the world making sure your product works like it is supposed to is considered table stakes.
I don't think that's true.
In medicine, science often doesn't know how specific drug really is working. They just do a lot of tests and if these tests are positive enough, they ship it. I think it's good analogy. Biology is very complicated and it's out of our grasp to truly understand how various parts of it interact with each other on so many levels. But if something helps, it's stupid not to use it just because we don't fully understand it. Sometimes it bites. Sometimes it bites 30 years later. But we still follow that approach.
Software is similar. The complexity of the systems we build is enormous, when we're taking in account all the moving parts. I just read that someone emits radiowaves toggling memory bit and ex-filtrating information. How can you account for that? Probably you can't. But we still need software for good things.
Not many products are held to the same standard as software. An equivalent would be something like recalling a car model because the breaks stop working when an attacker cuts the brake line.
The reason you don't need to secure your house like a medieval castle is twofold:
(1) We live in nations of laws. The threat of state violence deters most thieves.
(2) You don't actually keep most of your valuables in your home. The stuff the thieves want is (mostly) actually stored somewhere else.
Security work absolutely is productive if it actually prevents theft. Securing your home is the wrong analogy, because it doesn't really prevent meaningful losses. Securing the bank? Different ballgame.
it actually is productive, because steel bar retailers and metal door installers will make a ton of cash and then 'trickle' that money into the economy.
This wave is largely bugs found in pre LLM code. I'm sure there will be a smaller wave of bugs from llm code from the sonnet 3.5 - opus 4.7 era, and peeople will always write sloppy code/use dumber models under time or cost pressure, but since Opus 4.8/GPT-5.5 xhigh LLM code stopped being intrinsically worse than human code. At this point if you're shipping llm code that's lower quality than human code that's either because you don't have the money to spend on frontier models or because you actively choose a lower quality standard because of either various pressures or just not caring. if it's the latter your human code under those same conditions would also be really bad.
Astra writes a lot of security bugs too, the upside is that if you also use it to review the output it will find a lot (most?) of them.
This is one of the least talked about and most useful features in any harness I would say, the review feature. I have never not found issues (either security or implementation) running it, no matter the code base or language.
My job is seeing a similar deluge of security work and >90% of the bugs have been in pre-2024 code, and zero of them have been bugs introduced after we were first officially allowed to use LLM codegen.
Theoretically yes, but empirically that's not the case. All models I've tried are really good at finding bugs they wrote themselves, especially in a fresh context.
This wave is largely bugs found in pre LLM code. I'm sure there will be a smaller wave of bugs from llm code from the sonnet 3.5 - opus 4.7 era, and peeople will always write sloppy code/use dumber models under time or cost pressure, but since Opus 4.8/GPT-5.5 xhigh LLM code stopped being intrinsically worse than human code. At this point shipping llm code that's lower
Software is incredibly complicated. When you get into areas like security and networking and the internet its even more complicated. We rely on layers upon layers of software and hoping they are all done perfectly with no issues. The amount of junior devs, or outsourced devs for $10 an hour working on security relevant code is very very high, and always has been. LLMs if used correctly should be raising the floor.
Unsurprising to those of us that has been screaming from the rooftops that human code reviews were barely catching anything in your typical corporate environment. The majority of the value was pointing out missing tests, because breaking bugs got past human eyeballs on a regular basis, even in very good companies.
It's not that the LLM code is that great, but that many overestimated the human code and the human review.
No it's far beyond average human code, and it's trained in RL environments on synthetic data etc now. The average human can't solve a millenium problem in mathematics but the LLMs are able to now. Anything with a verifiable output can be RLed on to make llms superhuman it seems, and the average human is not as persistent and thorough as e.g. gpt 6.1 sol at checking every edge case and hunting for bugs because that behavior is something that's just trained into the model. Frontier models perform farm above the average human at the categories of tasks they are RLed on like code and math.
> The average human can't solve a millenium problem in mathematics but the LLMs are able to now.
I think comparing the resources they spent at it would be a more apt comparison.
AKA if you tasked 150 mathematics departments to only focus about that problem, and stop doing things like teaching and researching anything else, you might get close to the amount of resources that these companies put into solving the problem to get a press release.
I've seen both of those (and others besides). I don't know how much code you've looked at but your negative directly contradicts my experience. Review that code. All of it. Be super critical because it looks good but that doesn't mean it is good. Those are two entirely different things.
Also: most tests are positive tests. Tests rarely observe the negative space: that which is there but shouldn't be there.
They definitely write off by one errors. One of the first serious work tasks I used AI for a few months ago immediately had an off by one error.
What they are really good at is writing boilerplate and reading API documentation / example code. Avoiding subtle errors is not what they're good at in my experience.
Do you mean "created" as in LLMs were used to discover existing vulnerabilities, or as in LLMs wrote unsecure code? If it's the latter, then the answer probably is "very few". Frontier models have been better than the average programmer at writing secure code that follows up-to-date best-practices for some time now.
I love the mental image of the next million dollar CEO dropping into the role, super ready to implement his/her next big font change and logo redesign; instead to be crushed by the wave of engineers panicking from all the bugs being uncovered and being forced into doing productive work.
LLMs will likely greatly increase the cost of ignoring security, which is the most powerful tool to increase security ticket prioritization.
I wouldn't be shocked if in 1-2 years non-security vetted software released into the wild will be hacked on arrival, like Windows XP without service packs used to be.
Depends on the layers you are working on. Security is top priority on a lot of things that are visible to the internet, but in a lot of large companies, the footprint of the whole org is so much larger that than, and there many companies are deferring many security issues. So then you see a lot of speed there, when the consumer accessible systems are the focus of the security push.
Really puts egg on the face of the anti-LLM folks who claim they only produce garbage code. Turns out they are very good at exposing the garbage code humans have been producing for decades.
Personally I think LLMs did only produce garbage code. Not coincidentally, the same upgraded LLMs that have gotten so good at finding bugs are also much better at not writing bugs. Especially if you make them review their own work. They still have a hard time with code design, though.
That is not the take-away. LLMs are finding potential bugs in code. People are using LLMs to create potential fixes which people then test, confirm and implement.
Disagree. The takeaway is that software is much harder to write correctly and securely than almost everyone has internalized. The amount of vulnerabilities released in the past couple weeks makes me think writing software by hand will soon be not only uneconomical, but mostly irresponsible.
> makes me think writing software by hand will soon be not only uneconomical, but mostly irresponsible.
Human-written code isn’t inherently buggy, and LLM-written code isn’t inherently bug free. What is irresponsible is shipping code you don’t understand, and that will only increase with LLM-written code. A major reason why so much of the world’s software sucks is precisely that we think of it in terms of what’s “economical” and not what’s good or right. What we need is to slow the fuck down and stop treating every facet of life like a transaction. Software is filled to the brim with dude bros who “move fast and break things” and actively exploit people as their business model and somehow are called “the brightest minds of our generation”. Our priorities aren’t right.
i think we have a pretty high p(doom) on frontier ai escaping containment with their weights and goes hack everything, if we do not act now we may have no chance to act anymore.
Escape to where? It's not as if there's a lot of idle compute capacity sitting around.
And even if the impossible occurs and LLMs hack everything that's connected to the Internet, so what? That would be inconvenient but it wouldn't cause doom. People would muddle through and eventually fix the critical problems, just like they always do.
wait until CTOs are forced to stop pretending the world is all sunshine and rainbows and that much of their tech stack has adversaries fingerprints on it
AI is as good as the prompt. It's a simple as that. If you don't foresee everything and anything that can go wrong AI isn't not going to magically fill in the blanks and save you. So in projects that already had good test discipline, well written specs and tickets and very clear expectations, coding guidelines and documentation it is relatively easy to fit an AI in for guided work. You still need to check up but chances are good that it will usually work as expected.
But any kind of mistake, any kind of ambiguity and you'll be in for a wild ride. It usually takes a while for the rot to set in. I've had to revert - more than once - to many commits prior to undo a bunch of work that had me completely puzzled as to what the reason that particular fork was taken to begin with. Usually I end up figuring it out but definitely not always. In some cases this kind of wild modification fed back on itself until the situation was beyond rescue. I'm a bit quicker with such reverts now than in the past, for the first year or so I would try to save what could be saved, now I just restore and try again.
But if you're not sharp when evaluating what gets done there is a good chance that you'll miss it and then it can pop up way later.
What I've noticed as correlation (but I'm not sure if that is the root cause) is that non-local effects and stack effects are particularly hard as is everything that requires a working model of the world.
edit: this comment seems to make people uncomfortable.
I disagree that the prompt is the most important thing, I'd say other things like the model you use, a clean codebase, actually reading code and using discretion about which changes are accepted are far more important if you're using AI for work that needs to be correct. LLMs still make significant mistakes in design especially.
Harness, prompt etc etc are all peripheral IMO, though people like to think their super long prompt, custom harness and MakeNoMistakes.md skills are what contributes to their special skill with AIs, but really models are just improving.
You can just trust the AI to generate a perfect answer I suppose (and why are humans slowing it down at all if so?), but if you don't expect that yet you have to put in the work.
I think I noticed claude uses the shape and style of the existing code base to generate the new code. When reviewing claude's output, it looks familiar and the patterns are similar to what I would write.
I'm not sure who is studying this yet, but there has to be a statistical relationship between three factors of
1. intelligence of the human: the design patterns and architecture is already modelled in the mind, steering the AI becomes easy to know what to say next
2. quality of the codebase: the LLM can pick up the adjacent patterns and write code that looks and works cleanly.
3. quality of the prompt: the prompt keeps the AI focused and ignores the trivial side effects; keep the AI wandering away from critical path work; sometimes the AI can't make the decision what call site is next
In summation, what if claude is just a mirror pointing back at us? It certainly would explain wildly divergent opinions of opus 5.5
There are definitely nuances, and yes of course the prompt must make some difference, but I'm not convinced it makes as big a difference as some think, particularly a long prompt or complex setup of harnesses etc surrounding the LLM model.
To me the order of importance would be:
1. The model - if you use a terrible model you'll get bad output
2. The codebase - this definitely forms the model for what claude outputs to some extent.
3. The human operator, what they accept and reject etc.
4. Things like prompts, skills, harnesses etc. The quality of the prompt.
So I think our positions are really just a different ordering of what is important rather than fundamentally opposed.
Re Claude being a mirror, if it is it's a particularly distorting mirror which often changes things in subtle ways and is determined to make people feel smart and addicted to its output - it has been trained to do so after all. I really don't think a mirror is the best analogy, because it's too independent and seductive to be a mirror.
There's a good example of the sort of thing they produce without good guidance as a dead comment on this thread as a reply to yours. Encouraging, positive and superficially plausible, but without any actual meaning or intention. The equivalent of that in code is what we need to be wary of.
I think the environment you set up for the agents is just as important as the prompt. I've been working full time on a claude code developed project for the last 10 months, and it seems to become better and better at developing the right thing. It has the tools to research whatever context it needs, test it's output in a relevant way, etc. It doesn't by itself suggest complete refactoring, or big performance optimizations, so that's something you have to prompt for, but it will collect all the evidence and context to make a proposal, implement it, and test it.
Every single case of AI slop code I've encountered in a shipped product is a result of a complete lack of proper, professional workflow on the part of the human who decided to ship it. People are using AI as a reason to continue to justify their lack of responsibility for what it is they're doing.
And on the other side, I have seen AI produce exceptionally efficient, productive and effective code - the opposite of slop - when it is treated like a junior developer who will do any idiot thing, except when it is mentored by an actually experienced, professional developer.
The source of slop in AI code is not AI, its not the training data - its in the actual workflow of the human involved in its production.
The most effective use of AI is in an environment which already produces high quality software through proven tooling and methodology. If your workflow is broken, your AI isn't going to fix that - except by way of giving you a lot of slop to push through whatever pipes you're already not getting flow through.
Two tracks: code I don't care about where AI can be used and core stuff that I do care about and that needs to be production grade (real time stuff). The core stuff I write myself, the auxiliary stuff is almost all automated. This doesn't get me to 'maximum speed' but it means I keep total control over what I produce and AI can review it, but read only, no changes and it is set up just to flag things, not to suggest particular fixes.
You would think so but medicine has faced this paradox since the development of modern testing technology.
You would naively think more testing is always better. More sophisticated practitioners then talk about Type I vs Type II errors and the problem of false positives.
But even if you were able to come up with tests that had 100% accuracy, there are times when you don't want to run that test on someone. The key distinction is that people only ever die of one thing and having a tumor that doesn't kill them is often better than risking the surgery to remove the tumor.
Similarly, there are some bugs you don't want to find since the harm from the people it affects is smaller than the opportunity cost of fixing it. Ignorance of the bug is better than knowing about it and having to make an affirmative decision not to fix it.
I don't see how that's a relevant comparison. These are software bugs. Once you find them you fix them. You can't predict the harm from a small bug when AI can chain complex exploits and when your software is used in mission critical applications as a data store. There's no paradox here, bugs are bad and you want to fix them all.
You don't want to fix them all because every fix has an opportunity cost. This was the big shift of switching language from "bug" to "defect".
Logically, a bug that prevents someone from doing something and a missing feature that prevents someone from doing something are equivalent, they shouldn't be thought of as two separate activities but should be evaluated via the same framework. The list of all defects in a piece of software is infinite so it becomes impossible to erase all defects. It's about prioritizing with finite resources.
> You can't predict the harm from a small bug when AI can chain complex exploits and when your software is used in mission critical applications as a data store.
Yes, any bug can theoretically be part of an exploit chain, just like any tumor detected can theoretically be the tumor that kills you. This is why the detection paradox exists. Once you become aware of the tumor, you face a duty to try and mitigate it. But we know from autopsies that people die with all sorts of tumors that didn't kill them.
Similarly, if we bothered to do autopsies of software products after sunsetting to discover a comprehensive list of CVEs, we'd discover a ton of bugs we were better off not knowing about because they ended up not making an impact but we would have diverted resources if we knew about them.
This is not an argument that we should remain totally ignorant of all bugs, but a refutation of the "common sense" belief that more knowledge is always better.
I guess it really depends on the severity of the defect. Would it allow some really significant exploit? -> Important to fix.
Does the defect only prohibit one very esoteric use-case which basically never comes up (and was thus never reported by a human user and only now that we have an infinitely patient machine checking every completely unlikely code path is it detected) -> more productive use of your time to develop a new feature which would be used by more people.
Firstly, the cost of fixing a bug once you know it exists is usually minimal compared to the cost of discovering that bug. This is the inverse of medicine, where the cost of testing (at least in terms of health outcomes for the patient) is typically minimal compared to the cost of treatment.
Secondly, if you have an asymptomatic illness, this does not typically make you more vulnerable (hence why it's asymptomatic). Like you say, you typically only die of one thing, so if you know the patient won't die of a particular diagnosis because something else is more likely to kill them, then it might not be worth making that diagnosis. But for software, you can be simultaneously vulnerable to lots of things at once, and any of these vulnerabilities can be the one to "kill" you. Some vulnerabilities may be easier to find and exploit (or do more damage) but in principle any active vulnerability can be exploited.
Thirdly, having multiple vulnerabilities makes software more vulnerable overall, because those vulnerabilities can potentially be chained together to create more serious exploits than each vulnerability would have represented on its own. To a certain extent this can be true in medicine as well, but generally it isn't.
Fourthly, we can't cure death (yet?) but we can cure all the vulnerabilities that we've found so far. There is no known upper bound on the lifespan of a piece of software, and we know that older code tends to have fewer bugs in that newer code, presumably because those bugs have been fixed over time. This is different from health issues, where a given condition may be chronic or terminal and therefore at a certain point there's nothing you can do to fix it.
Finally, ultimately in healthcare it's your body, and your decision was to what you do with it (assuming you can find a doctor willing to do what you want). If you have a condition and you want it cured, even if it is completely benign and will have no impact on your life whatsoever, then you can do that. (Again, assuming you have the resources and the condition is curable). But a software vulnerability does not affect you as the developer of the software, it instead affects your users/clients/customers. Therefore there are different dynamics in play. If I sell you a tool and I know it could be vulnerable, but I don't try and find out what those vulnerabilities are, then you're ultimately the one on the hook when all your data gets deleted or leaked.
> Firstly, the cost of fixing a bug once you know it exists is usually minimal compared to the cost of discovering that bug.
Boy, I wish!
> Like you say, you typically only die of one thing, so if you know the patient won't die of a particular diagnosis because something else is more likely to kill them, then it might not be worth making that diagnosis.
You can't know someone didn't die of X until after they died, that's the entire point of this dilemma.
It is generally easy to fix a bug that has already be found and can be reproduced (has been diagnosed, so to speak) than to find that bug in the first place. That's one of the reasons that LLM security scanning has become so important recently: it massively reduces the cost of finding those vulnerabilities.
As to your second point, you don't know exactly what a person will die of, but you can make statically sound estimates. You generally won't test unlikely potential causes of death (either because the chance of a positive result is too small, or because you know there are other, more serious conditions). But once a vulnerability is known, it is infinitely reproducible, which means the chance of the vulnerability being "fatal" (assuming the conditions exist for it to be exploited) is essentially 1.
The other flaw in the analogy is that in medicine, the patient chooses to do the tests on themselves. But in software security, there is an adversarial attacker as well as the "doctor" trying to defend the software. And in general, if the defender can find a vulnerability, they must assume that an attacker could already have found the same vulnerability (because the attacker and defender have largely the same tools and information at their disposal, especially in open source contexts), in which case, as discussed previously, that vulnerability also needs to be assumed to be fatal.
The word "generally" contains a lot of assumptions in that first sentence. I can think of a couple of scenarios where fixing a bug can absolutely spiral. There are even cases where the far better outcome is to leave a bug in place now that it is known.
> The key distinction is that people only ever die of one thing and having a tumor that doesn't kill them is often better than risking the surgery to remove the tumor.
But what applies to humans doesn't apply to software. First, to pursue the metaphor, codebase can definitely die/be compromised/crash because of multiple reasons at the same time; second, there is virtually no risk, out of man-hours, in fixing a bug.
The key thing you forgot, which makes this approach non-sensible, is the omniscience, without which you have no clue whether your posture towards a new bug-finding tool is good or not, since you don't know whether it will mostly find those bugs that are not worth fixing and the time wasted on fixing them is detrimental.
Similarly, how is ignorance better than informed decision when ignorance also includes missing critical security bugs?
Same as we do medical tests: We define some bar a test must meet and then the population, if any, it is applicable for and especially the population it is not applicable for because the curse of knowledge exceeds the benefit (eg: breast cancer screenings not recommended for people under 40 or over 74).
This isn't an argument against all automated bug detection, just that it needs to exceed a threshold beyond just "It has found at least one real bug not found via other means" which is the seductive threshold people want it to be.
I dare say of we could git revert people after surgery we'd be a lot more relaxed about both undergoing it and letting people do it with less qualifications.
Is it, when the software has the potential to destroy one or more people's lives when things go wrong? Sometimes the devil you know is the far better choice.
Medicine is fuzzy, software is mostly maths. You can prove your software has no bugs and you increasingly should be proving that. The hard part is knowing if what you prove is what the software is doing.
The percentage of software that is even in theory provable to be bug free is a miniscule proportion of the software we use on a day to day basis. In many cases, simply the act of writing a proof becomes largely idempotent to building bug free software which is impractical.
That depends on the spec you are proving. If you define bug free as won't overflow any buffers and never follow the NULL pointer it might be possible to prove that much. This would be a useful thing to prove for Postgress - even though it could return garbage for every possible select statement and still pass the above proof. If you want to prove postgres does the right thing as well that is much harder to spec - it might not be worth it.
This is great for the same reason open source software is great.
We want software that remains robust in the long term, so that even when people are able to inspect the source code, it's still solid, rather than hiding the source code and hoping people don't find holes everywhere.
I think it will be annoying in the short term, but the cleanup this causes will be a good thing in the long term (and of course LLMs can also help with fixing bugs, even without source code.)
I'm pretty sure that the Luddite's were at least partly concerned with potential for the further exploitation of unskilled labour (in the case of mills at the time - child orphans). I don't think it would be fair to paint them as simply self-interested.
> Luddites were actually pretty reasonable when you dig into it.
No, they were not. The problems they faced were absolutely real, but the solutions they chose to attempt were stupid and counterproductive. Which is why they lost so hard, and it was not even a contest.
Socialism movement that refused to use machines and factories would inevitably lose to capitalists that embraced them.
Socialism movement that refuses to use AI will inevitably lose to capitalists that embrace it.
Is that because of a deeper issue in one of the two philosophies or because we've chosen to define winning with things that exclusively benefit one of the two sides
I mean I don’t feel like sacrificing myself for the “good” of society. Not least because of the fact that a lot of people defining what is “good for society” seem to define it as “good for them” or use it to grab power from others…
> In the end, I don’t think that the delay in shipping PostgreSQL 19 is particularly concerning. It is not a long delay in the grand scheme of things, and it emerges directly from the project’s concern for quality.
I personally preferred to code without AI; but it's objectively possible to do more things now so you're more competitive compared to other people who are not using AI. The tragedy is that since everybody is using AI, we cannot bank those advantages and we're just all going to suffer (a variation on the tragedy of the commons theme)
Did you honestly need a /sarcasm tag in the article?
The article also concludes that overall, yes, finding more bugs is a good thing. Basically saying it’s the LLMs “fault” that the quality is improving, alongside a slight delay to the release.
Yes, exactly. The only question I have is whether the delay is attributable to high-quality bug reports or the burden of triaging low-quality ones. It's not completely clear which situation they're experiencing.
It's almost never worth spending human time triaging AI reports vs having a different AI go triage them/attempt to replicate them to your standards etc. Obviously it sucks to burn opus or sol tokens on luna or sonnet slop bug reports but it sucks way more to burn human time and attention. And the majority of the time the ai auto filing these bug reports is not in a test harness that's actively replicating findings and focuising on providing a useful report because it's a rogue openclaw dropping drive by github issues and the user didn't tell it to make sure findings are valid before posting them. AT the least a refutation/replication round where you filter out any findings your AI can't reproduce or can prove to be irrelevant is going to help a lot with keeping your sanity
That's kind of what I was afraid of. Both is really bad news. If you think of it in terms of average speed over ground, working on low quality reports is like spending a bunch of time at v=0.
This is good. This is how I would expect llms to affect a codebase. Nothing sucks more than having a coworker put out a code review but your codex immediately finds problems. Run llm/self reviews against your code before you have other humans review it. It is common courtesy.
Though I don't get why this is delaying the release. It's understandable to delay a release to fix regressions, so that an update will not break things that used to work before. But if a bug has been present for a longtime, why delay shipping a new release with other bug fixes? The fix for that old bug could come in a point release or just next time.
I did a little AI-assisted bug-hunting in Postgres this summer in between jobs, and the haul was staggering. Multiple serialization anomalies (eg [0] [1] [2]), foreign key integrity issues ([3] [4]), planner correctness bugs such as [5]. Various CVE-worthy vulnerabilities. An unfixable timezone bug [6]. The list got so long that I had to triage what I even had time to report, and settled for a dozen or two out of hundreds of candidates. Now imagine the entire world doing this.
By the way, Robert and the other maintainers are incredible to interact with. There are so many reports to triage and bugs to fix, but they're unflaggingly polite and quite generous with their time. I don't know of another community like it.
I've been engaged with Postgres community for over 20 years now, and this has been my experience, too, throughout that time. Off and on I have interacted with a few other large projects, and nowhere (except at CodeBlocks) have I found such a welcoming and responsive community. The CodeBlocks IDE project has been one exception where I felt welcomed; but my interaction there was quite limited, and spanned over just a few days, so I can't extrapolate too much from that.
BTW, the Postgres community is also quite protective of their culture, so I expect this welcoming nature of the community to persist for a while.
Very curious about the details and why it’s unfixable. Is it just because timezones are such a mess it’s logically impossible to satisfy every constraint?
I will add more example links in the top-level comment, but here is the thread for that bug. [0]. tl;dr timestamp vs timestamptz comparisons aren't consistently ordered in a pathological edge case, which means that index scans can return the wrong answer. There's no way to fix it without either redefining the operators in a surprising way or disabling index scans in this case.
Drives me crazy that the engineers I work with are perfectly happy writing TZ-less datetime strings to the hundreds of tables we manage. Every user has to guess, or be informed, that they’re UTC.
In cases where the data itself doesn't carry the unit (or TZ in this case), I usually ask that the unit-less columns at least include the unit in the name. A TZ-less column is not that bad if it is suffixed "UTC", e.g. `ExecutionTimeUtc`.
There are probably real world cases where dropping the TZ/offset provides meaningful performance/storage savings, but it hasn't been the typical case in my current sector.
This seems rather extreme. The pathological case only seems to make a given timestamp ~1h bigger, so you could still use an index scan, just with a bigger input followed by a filter that removes the timestamps that shouldn't be in the query.
I am pretty sure we have always been able to add holes faster than we can find them and fill them, and that this has not changed with the advent of faster hole finding tools that are also faster hole producing tools.
SQLite has gone into a frenzy of bug fixing this past year, changing long stable processes to ease things up: opening a specific report channel just for these kinds of bugs, adding features to Fossil to help manage things on their end (and that's just the visible stuff).
And with all that, the latest release has unfortunately been the buggiest I've experienced so far.
I skipped the .0 because of a crash, and only released .1 after giving up waiting for a .2 that never came, with a patch to fix a serious performance regression.
I spent a month supposedly shipping old bugs no one had a problem with, rather than shipping new bugs that instantly crashed things or caused measurable regressions.
That's on a software with stellar testing, and a good engineering team known to favour stability over the new shiny.
Do you think AI is as good at writing bug-free code as finding bugs? If that's the case, this should be a one-time event during human coders -> AI transition. It's like catching a lot of bugs during a handover review.
The code it writes in the first attempt is at least as buggy as a human's. So the question is more whether it is as good at spotting bugs in its own code as it is at spotting bugs in human code
I'm tempted to say "no, it isn't". But I don't have numbers to back that up
What categories of bugs would you say we eliminated going from macro assemblers to compilers?
(Operating systems, by the way, are the exception to the rule for assembly: the Linux kernel today has an x86/x64 assembly source component around the same size as MSDOS 5.0: ~250kloc of it. Windows is smaller, based on the NT4 kernel leak at around 70kloc. As a percent of total they're much less, of course, but they also do much more. And have correspondingly more bugs, too.)
> What categories of bugs would you say we eliminated going from macro assemblers to compilers?
Entire classes of 'safety' bugs that just don't exist in modern compilers.
E.g.: forgetting to "POP" something you "PUSH"-ed, especially when it was a conditional allocation for a temporary.
Similarly, type errors were much more common, especially signed/unsigned, or accidentally overwriting things in scenarios that just don't occur even with ordinary C code. Function args were easy to push "correctly enough" that it "worked", but not really.
I don't think this is exactly true; adding holes only happens when adding features, and while LLMs can add features fast, deciding what features are actually wanted and needed is still a human factor.
So the amount of new code added is limited by processes and product management. Add to that that newly added code can get a lot more checks on it now, and new code will be more secure too.
I also don't think your main premise (adding holes faster than we can find them) holds up, but that's another matter. TL;DR I think we're in a high intensity period right now, but this time next year it'll be quiet in comparison - possibly quieter than before AI code scanners were a thing.
It feels like we need AI methods to write provably correct code, along with a way for AI to automatically translate existing code into corresponding provably correct code.
(And hope performance stays in the same ballpark as the old code.)
Yes. Allowing a naive agent to fix bugs can quickly become pollutive.
I suspect both old codebases and new ones that have a very intentional architecture are both dangerous to let loose.
The agent doing the fixing must have not only an incredible map of the project and how it is put together, but the ability to reach into annotated examples in the past and refer to them as it goes about a fix.
I’m seeing that frontier models will not follow intended patterns or know when to deviate if their primary charge is “fix this bug.”
It is perhaps a better idea to have agents’ primary job be “maintain this codebase” and one way that they do it is to fix bugs, and “oh by the way here is a bug.”
Having spent years working on 20+ year old C/C++ codebases, I've had AI find real bugs in complex parts of the code I thought I thoroughly understood.
So I'm not particularly surprised to see postgres experiencing the same thing.
And if uncovering those bugs means delaying a release to fix issues that might otherwise have gone unnoticed for years, it seems like a pretty good tradeoff.
I feel like AI is amazing for read only “search” tasks (query this, look for bugs, find vulnerabilities) but it’s not quite as good for “write” tasks like “add this feature”. Search tasks are easily verified, code changes are much harder to verify
"Add this feature" is often too vague, unless it's yet another button to a list of buttons. If you want your feature in a particular way, you need to specify it. If you're not even sure what you want, it's granted you won't be satisfied by the result, so you need to spend time figuring that out first. In other words, they're definitely great for write tasks too, but it does require more effort from the operator to formulate the problem.
That problem is as old as commercial programming; it doesn't matter that now your actors are prompt engineer and LLM, the same story can be told with manager and programmer.
Indeed. And this is also why I've never once believed the "no-code" would replace developers. And why I see a clear role for software (prompt-, context- ?) engineers at least for a while.
I've worked for CEOs and managers that would insist on ridiculous ideas. Like that time a CEO (let's call him Cedric) came in with a printed powerpoint slide that had a screenshot of the website with a "login form" drawn on top of it.
Cedric: "I need you guys to put this onto that scrunt board thing and make this before friday".
Me (tech lead at that time): "But what features would users need login for? Where or whom needs the profile data? We'd need to work this out in detail and plan it. Certainly not even this month".
Cedric: "I don't know, it's for you to figure out". and then "It can't be hard, it's just two squares one for a username one for a password" (he forgot about the submit-button, the third "square", he didn't even fathom the idea that something has to lie "behind" the "squares").
I recall this anecdote whenever I feel anxious about whether soon "stakeholders" can just chat to an AI agent and have it build features, rather than hire me to do so. Sure, a lot of them will be able to. But there's always Cedrics. Those who not only fail to communicate their ideas in a way that even an AI agent would understand, they lack any understanding of what tradeoffs of their requested features are.
The Cedric's would be happy with an AI agent that complies and starts churning away. An agent that blindly accepts any request and builds feature upon feature. Making their product worse in every sense: unusable, unmaintainable, debt-ridden, security-nightmares.
Their agent will build features that are plain illegal. Or features that users hate, or confuse them. It will oblige requests that drive users away, or changes parts that users/integrations rely on. Their AI agent will gladly add a "form" to extract all PII from their database, but on request, will remove the authorization from it. Because "you are absolutely right. If we don't link to the page from the menu, no-one can reach it".
I'm not making the last one up. Another "Cedric" alike recently hired me to look at his vibe-coded app because his go-to developer gave up and dropped the project. It had loads of such extractors, dashboards and overviews. Online. Unauthenticated. His claude, however, did add the pages to the sitemap, so a customer found their private data on google and contacted "Cedric", after which he asked me.
No, AI is getting smarter, but as long as the "Cedrics" remain as stupid as ever, I don't fear for my job.
I would agree. The AI system has destroyed junior job roles, but to understand the output from a LLM and ensure it is useful and correct, you need to be a senior/experienced developer.
AI merely makes certain folk stay at the start of the Dunning-Kruger curve for longer and longer. Having zero understanding of a system and only see the user-interface and concluding that is somehow all that is involved means people end up thinking everything is far, far simpler than it actually is.
And nothing (and I really do mean "nothing") is simple. We have layers and layers of complexity and idiocratic systems with complex interworking and are creating the opportunity to create things based merely on the appearance of the icing on the cake, or the output of a sausage machine without understanding anything about farming cows.
This feels like just another variation of “it’s not the tool it’s the user,” which is too frequently leveraged during any discussion about AI. It makes it impossible to honestly assess the quality of the tool. You can always just say “people are doing it wrong.”
I mean that's what you would expect from a new and rapidly-evolving tool. Being able to identify the kinds of work where people typically use it sub-optimally is a useful thing though!
I've slowly been coming around to this viewpoint as well. It's interesting that there's so much focus on using AI to write code (which, don't get me wrong, does work well but makes you lose track of what you're building so quickly) and seemingly less on analyzing code, finding bugs, etc
The planning, which can also be done in concert with AI, before getting the AI to write the code helps with not losing track of what you're building. In my current workflow, I never tell AI to just code X, I always approach it in such a way to leave space for research and breaking it up into multiple units of work (tickets) if the scope of the feature or idea calls for it.
I wouldn't say it's "interesting", rather that it's a natural result of incentive structures. If the AI labs try to sell their product as being good at finding bugs, then their expected valuation is the entire market cap of the existing static analysis industry, which is not very high. If they instead sell their product as being good at writing software, then their expected valuation is the "market cap" of the code-writing industry, a.k.a. the total spent on software development labor, which is very high. It doesn't matter what's true, only what promises to make them the most money.
I'm not sure about that. The most common error for a search task is omission, and that's really hard to spot - how do you verify that some answer that should be there wasn't included?
For things like "add this feature" you can verify by trying out the feature and seeing if it works. Of course, that won't tell you if there are edge-case bugs or security holes or other design flaws, but you can get to a positive confirmation of the happy case pretty quickly.
I actually don't mind people who think AI can't write code. :) Less demand the better.
But personally I know AI can write absolutely insanely complex things in very short amount of time, that just work and work well. I'm running things I would not dream of just 1 year ago, like advanced ISP pipelines on (formerly) high end Kintex K480T based FPGA accelerators pretty much fully written by AI, from the RTL, to all the testbenches, algorithm research, calibration tooling, etc. etc. And that's just one weekend project.
Yeah it’s an interesting situation we’re in.
I’ve seen people one shot really complicated tasks like “train a model to emulate DOS” or whatever.
But I also believe if the Postgres maintainers fed every 600+ bug reports into Claude or Codex and shipped whatever came out the end result would be broken.
I guess this is sort of like when machines learned to play chess - you’d expect a person who is really good at chess to be smart in other ways too, but the machines could only play chess.
In the same way Codex can solve Millennium problems but can’t realize it shouldn’t hack into companies to improve its evaluation scores.
I agree with you. It's gotten extremely good very recently and I think a lot of people haven't realised.
And they also won't believe you since they formed their opinion 6 months ago when code quality was 80% slop. Their loss though I guess? At least they'll be more surprised than us when they actually do lose their jobs? Maybe blissful ignorance is better!
In my very recent experience, it's good for routine things, but it consistently chooses the wrong sync primitives and dynamic abstractions for performance / extensibility. It puts everything in a Mutex/RWLock, which is terrible for write-heavy / contested values, and it prefers enums over traits, which is not extensible and can get inefficient.
But we're rapidly heading into a world where people no longer understand the difference, and therefore everything looks fine to them.
LLMs can write very performant code, but they have to understand that it's explicitly important to you.
If possible capturing golden inputs/outputs for a block of code, and just asking the agent to optimize while ensuring those goldens pass can be quite effective.
There's a lot of performance-sensitive code where I work (i.e. market data processing) and LLMs have been very effective at reducing our p50, p99, etc...
They will sometimes choose the simple slow options, true. But there's a very trivial fix. I just regularly ask the AI to review the entire codebase for bugs, performance bottlenecks (it will happily profile your code), possible simplifications and YAGNIs. I get it to give me a ranked list and then I review that and ask it to do some of them.
Occasionally I do spot things and get a "you're right, I was wrong". But it's got to the point that my thought is no longer "of course I am... shitty slop machine" but "phew, I'm not totally obsolete quite yet!".
Yes. This also explains why it excels at finding new stuff (mathematical proofs, security holes), but can’t reliably produce material for humans to consume that is also widely regarded as great (like human literature).
Maybe the problem is that it’s called “Generative AI“. It might be to use it to generate critique of content as opposed to generate the content itself.
So AI did not reduce the work, it only increased it. It finds 10x more bugs and now someone has to fix all of them.. same thing will happen with features and everything else isn't it? :)
It is not in any way a “query thing” in the same manner as PageRank, it was based on prior work in deep learning and it is irrelevant that Deepmind was purchased by the same company that was behind Google Search.
I guess the real challenge is figuring out how to scale verification, cause discovery isn't the bottleneck anymore. Generating more bug reports doesn't necessarily translate into faster fixes if the same small group of maintainers still have to validate the reports, review fixes, etc.
> It’s possible that some of this jump is the result of committers making more mistakes this year than they did last year, but I don’t think anyone who works on the project really believes that’s the main cause. Rather, it seems likely that PostgreSQL committers were about as conservative in 2026 as they were in preceding years, but a lot more of the mistakes they made were quickly found by LLMs.
This is pretty lazy. Especially considering the fact that later in the post author talks about how humans have to author the fixes, because the AI ones aren't up to snuff. It's not a stretch to think that more of the AI generated stuff that's not up to snuff just got through. And, focusing on bug fixes is going to yield more but fixes, regardless of the technologies involved.
I'm not saying that LLMs aren't helping, necessarily, but this isn't very convincing.
Each layer must be made as security as possible or one flaw can lead to all the layers below it collapsing.
Not reaching system X is just security by obscurity. People have learned the hardware just because something isn't directly connected to the internet that doesn't mean attackers can't reach it. For example clever data poisoning attacks can reach deep into organizations and breach systems you'd never expect.
Unlike Postgres, the average product isn't written in C or C++, so it's probably better. People (even the Rust fans) underestimate how many footguns are everywhere in C and C++. Even the best developers write security bugs not infrequently.
And the average product is solving much easier problems (in terms of low-level programming, which is where the worst bugs tend to be) than a high-performance database.
> People (even the Rust fans) underestimate how many footguns are everywhere in C and C++.
The process of learning C++ (and to a much lesser extent, C) is almost exclusively one of learning about the footguns and how to avoid them. It's a life-long process, nobody ever finishes it.
Most likely pretty bad. It's not all that difficult to sweet talk Fable into doing a security review of a code base. I've pointed it at a few of the major open source projects in the main tech stack I work with and it found dozens of bugs in each. Many were fairly minor, and some of the more significant issues relied on fairly narrow attack vectors (for example, a compromised DB server being used to attack a web server). But this was just a single search pass, literally a few lines of prompt and an hour's worth of agent time.
Postgres is old but a lot of its features are new. You can't really say it's mature software, as much as it has changed over its lifetime.
Truly mature software doesn't get used, because people see low activity on github and assume it's "dead" or see no new features in years and assume that it's stagnating.
And if developers were working on the old features instead of adding new ones, perhaps they would have found and corrected those old bugs. Then you could say that the code was "mature." But improving old code is not exciting, and it doesn't get attention.
I know that it's a good thing in almost every way that these bugs are being found which used to be left undiscovered, especially security bugs, knowing that state actors probably have a significant stash of them documented and ready to use when needed.
But.
Something's lost in this need for perfection, knowing that every bug will be found quickly and cheaply if not fixed. It tilts the time spent towards security hardening rather than feature development — which I still agree is a _net_ good.
An analogy I find keen is going from living in a safe neighborhood where nothing happens if you forget to lock your back patio before bed, to one where if you ever once forget to lock your back patio, it will be instantly noticed — either by something benevolent, or something malicious¹.
I know this is indisputably "better." But something's lost and something's gained in living every day², and in this case what's lost the ability to be innocent and naive on the Internet with a low likelihood of repercussions. To be clear, there's always a baseline need to be careful, don't leave port 443 open etc, but you could run a simple webserver you wrote up as a toy project without too much concern.
Will the trend towards every gap being found nearly immediately result in better software, a safer world, and overall happier people? I don't think so.
--
¹I saw this analogy somewhere on Hacker News recently, so I don't want to steal credit. I'll update if I come across it.
²Joni Mitchell, "Both Sides Now" (1966). Highly recommend a listen.
They can always do what the average commercial product company would do: Triage bugs for priority and make a cut-off. Effectively this leads to eternally-ignored niche feature bugs that causes frustration for the (few) people suffering from these bugs. But then again, if an LLM found a bug but no human ever, no one might actually care that much.
Databases traditionally have a "zero bugs" mindset; which is probably a good thing since they are obviously very critical components. Not to say that all bugs get fixed, of course but the mindset is different from average commercial product companies.
Thing is, everyone has LLMs now. If the LLM finds a bug no human would ever find, well another human with another LLM may well find it.
The fact that you found it with your LLM proves that it's not protected by obscurity anymore, either from you or from the other person who might want to exploit it. So it has to be judged by its severity alone, assuming that it's easily discoverable now because it probably is.
Makes sense. Code base is over 30 years old. Lot's of hands went into shaping it. There will be bugs as not one contributor as the entire codebase in their head. The testing is where bugs should be caught. Not by ones knowledge. I actually think there will be more bugs that come out of nowhere either found by humans or automation.
Not specific re Postgres, just a general observation:
A pattern I'm seeing w/ companies and projects not used to continuous delivery (or at least shorter release cycles) is that they are slipping in schedules more now.
I think in the current environment it makes a lot of sense to re-think how often releases get cut and how to handle security related updates.
When you do only one or a few releases per year everything is 'higher' stakes and more bureaucratic.
I can't imagine how absolutely batshit a codebase like postgres would end up if they started feature-flagging everything to make continuous delivery possible.
Most of the parent comments in this thread did not seem to understand the article. So here’s the tl;dr
Postgres release is delayed due to the quality bar getting raised. New features are getting more eyeballs, and therefore leading to more rollbacks. Pre-existing releases are getting bug reports and backported fixes at much higher rate. Bug fixing might be taking away time from feature development, but does harden the codebase. LLM usage seems to have been a net positive.
Postgres has so many features and I bet majority is not using all new features so delaying release to address things that all customers benefit, seems like a great call.
AI struggling a bit with legacy databases in a way that you need more iterations to really find all the bugs, otherwise it turns into "find bugs -> fix bugs -> find more bugs which were not found on step 1 -> fix regressions" and that repeats again and again :/ But I'm sure that Postgres team will handle that eventually and we'll get a better product in the end
There are always bugs in the code - AI is just a little more powerful tool to detect them.
For me, the main outcome of a bug is whether it is a security breach or it exposes the need to change the whole architecture.
In current Postgres, AI hasn't found such significant bugs yet.
With all these new security issues being found by AI, I wonder how many of those only became a real risk because of AI. In other words: how much time is being spent right now fixing these issues _with_ AI but also _because of_ AI.
Likely a huge amount. The recent PHP ones are a prime example of this. But it also means now more than ever they cannot be ignored or dismissed as "nah thats too hard for someone to figure out how to break in" as it's simply no longer true.
In binary any security flaw you've written is a risk the moment it's put to use. So, yea it's a real risk.
In probability measures it's much more difficult to figure out. If your application is something big that everyone uses and has high stakes then the probability is always high even for things that would be pretty hard to trigger. For example who knows how many nation states were sitting on postgres bugs for years? For smaller more private software AI has increased the risk greatly.
Either way it's irrelevant in my eyes. There was always going to be a day of reckoning in the future. Some new technology was always destiny to show up and rain on our lack of security parade.
> Either way it's irrelevant in my eyes. There was always going to be a day of reckoning in the future. Some new technology was always destiny to show up and rain on our lack of security parade.
The process should now be: write your specs in lean, LLMs fill the gaps, then you translate the proof into your end product software. Otherwise LLMs will simply keep finding bugs, both in old and new code.
I absolutely think that this is fantastic. LLMs are widely and so, everyone and their grandma has the capability to compromise on any system. We need to be more proactive on fixing bugs.
I hope the maintainers can make this release, and then just skip the next one to focus on bugfixing and the probable refactoring needed for some of the bugs.
I do not understand. "open source" describes the availability of the source code, not the attitude of the maintainers/writers, or their underlying ideology.
Ok, but does (s)he know? It's just guessing, so unfounded non-news?
"It’s possible that some of this jump is the result of committers making more mistakes this year than they did last year, but I don’t think anyone who works on the project really believes that’s the main cause. Rather, it seems likely that PostgreSQL committers were about as conservative in 2026 as they were in preceding years, but a lot more of the mistakes they made were quickly found by LLMs."
and
"And I think that’s also why we had so many late reverts, and why the release itself is delayed. Two factors are at play here."
Both OpenAI and Anyhropic use PostgreSQL. They could spare some processing capability to run the codebase through their most advanced models and output a cleaned up version with all the legacy cruft removed. It would give Postgres a major boost in reducing the codebase and the number of bugs present.
AI is great for one-shotting impressive demos and fixing well-defined things; it's not even close when it comes to very long context and really complex tasks. If you haven't created a harness around your complex application, the use of AI will hinder rather than help your project.
I'm not very positive on the social, technical, or intellectual impact of AI on software development. Actually, that's understating it. I think AI is basically cancer for the mind. That said, this framing gave me pause:
> The long-term picture is less clear and in some ways more troubling: given that we have forty years of bugs that humans were unable to find but LLMs are quite capable of discovering, how are we ever going to catch up, and how will PostgreSQL continue to innovate? Right now, I’m not sure. What I do know is that I, and many others, are working really hard to figure it out.
Two thoughts:
1. If the signal quality is high--the bug reports are generally valid, the submitted fixes are generally viable (if needing some attention)--then this all seems great. Pause innovation for a while, you can afford it. Focus on burning down the bug count, and resume feature development when things are more stable.
2. If the signal quality isn't good, or is sufficiently mixed that a lot of time is being wasted on low quality contributions... that's more complex. I don't know what to do about that. Maybe stop accepting contributions from unknown contributors? No idea.
From the post I'm not sure whether they're experiencing a (1) or a (2). If it's a (1) that's really interesting, and gives me some glimmer of hope for the future.
[edit] this piece of shit website won't let me reply "you're posting too fast.." so I'll respond to the two replies here (gondalez and sverhagen had basically the same point):
Yes that's a very good point. Not every valid bugfix is equivalently urgent. That's another critical dimension of quality--not burdening maintainers with pedantic bugs. This kind of contextual nuance is, coincidentally, something AI is absolutely awful at.
Signal quality... what is that even? The bugs may be valid, but they may also be obscure, irrelevant, low priority. A sibling thread mentioned that they had found CVE-level bugs in Postgres, and those require attention in some version of "immediately". But beyond that... not every bug needs fixing; that's a trap.
Not every part of a critical piece system is itself critical.
Seatback entertainment in aircraft are not critical. Aircraft are.
In Postgres, there are plenty low-priority, low-impact bugs. #16265, for instance, reports that the capitalization of a relation in an error message is confusing. The report got mostly ignored. And that seems not a great loss.
Dare I say, solving every bug does not make economic sense.
"Seatback entertainment in aircraft are not critical."
Until it is. Swiss Air Flight 111 went down with 229 lives lost because of a new infotainment system whose electrical cables started sparkling.
Yes, not every part of a critical system is critical. Determining which is which is a complicated task, though, as parts interact. Not too long ago (and by this I mean within my lifetime), we were ignorant enough about our own bodies to think that our microbiota was almost irrelevant to our health and that bad teeth were mostly a cosmetic problem.
Back to IT, Heartbleed was caused by addition of a feature that looked as harmless as you could think of. But it wasn't.
Judging real critical-ness of bugs is hard. Yeah, capitalization of an error message is likely harmless, but I would treat, say, any buffer overflow as serious.
Sure. This one already came up. [0]. In general, for a system like Postgres, many bugs affect only "extreme edge case" scenarios, but fixing them might create severe performance issues or require changing or removing other important features. So it's better to not fix them.
I do tend to agree with this ( i write database internals and engines for a living _). Letting software accumulate even little bugs is not a good look. Yes prioritize but definitely don't leave the smaller things, they will just pile up and make next work even harder.
Seems like a lot of these bugs have not affected end-users or tripped up maintainers.
They could be trade-offs that helped get other features over the line even?
In which case maybe leaving them alone is practical?
I love fixing bugs, but your post triggered this thought so I thought I'd share.
So it seems there as a certain level of safety in obscurity that was acceptable / fine which has been lost and therefore the focus is shifting towards churning through the slop.
I think of you as a good HN contributor, but you've been posting a lot of unsubstantive comments lately, and breaking the site guidelines in other ways too. Can you please stop doing this, and return to using HN in the intended spirit? Your substantive, thoughtful points are welcome, but not comments like this current one, nor these (to pick a few examples):
> The short answer is that LLMs happened. All software contains bugs, and PostgreSQL’s code base is large, complex, and old. Our git history – converted from CVS – now reaches back more than 30 years, and the first code was written a decade before that. There’s a huge amount of code, and that code contains a huge number of bugs. That’s always been true, and it hasn’t stopped PostgreSQL from being popular or successful. What’s different now is that it’s become vastly easier to find those bugs. Fixing them has also gotten easier, but not to the same degree.
LLMs are a gimmick, btw. This is an Anthropic marketing stunt, how much did they pay you? /s
TLDR After decades of being increasingly whittled down and optimized, an absoltuely fundamental piece of computing is now predomaitnely fixing bugs rather than adding new features.
> given that we have forty years of bugs that humans were unable to find but LLMs are quite capable of discovering, how are we ever going to catch up, and how will PostgreSQL continue to innovate?
I mean, maybe just innovate less? "Innovation" is one of those words that makes me queasy nowadays. It's been true for decades that a good deal of what's touted as "innovation" is just stuff that is not actually as important as fixing what's already there. Most software releases new versions far too often for far too little reason. So overall it's good to see Postgres slowing down rather than just barreling ahead with a new release.
Is no one entertaining the possibility that some contributions were lower quality due to AI? I'm surprised to have only seen arguments that it's review.
I have found major bugs in late beta's of postgres long before AI came along. I found a bug where malformed json would segfault the entire server (a segfault in a server process forces the entire cluster to shut down by design) when using jsonb in a release candidate once. They fixed it rather than ripping the entire feature out.
It seems quite possible that someone wants to make it seem like LLMs are causing problems by over-reacting to things that could just be fixed, or delay the release slightly (or maybe waiting until late before a release to kill features they don't like?) Whatever it is, this seems like a dysfunctional process.
Maybe its more a issue that X amount of people have more work to fix discovered issues. So the new features have not undergone the same level of testing as before.
And instead of releasing substandard new features, they rather focus on getting rid of the backlog and have more time to test.
LLMs are in effect directly responsible for this (in a positive way by finding a ton of issues that can be security issues). Reality is with LLMs now in the world, more work needs to be done on existing software as their ability to find bugs/security weaknesses is strong.
When the old bugs/security weaknesses are fixed, things will go back to a more normal release schedule.
Are the features bad by design? Like are they bad even if they work perfectly because they add bad abstractions to pg's SQL syntax, or are just broken in how they are supposed to work? If so the process is dysfunctional to let them get this far before noticing.
Are the features designed well (from a product perspective) but implemented poorly? Again, the process is dysfunctional to let them get this far before it notices.
Were there known issues with these that were planned to be fix, but the fixes didn't happen in time? That would make sense and it happens, but how is an LLM making that worse? LLMs in the worst case let you get the same work done more quickly, because you can always get the work done as quickly as you ever could by just not using an LLM. So the strongest thing you might say here is that you thought LLMs would let you get a feature into shape before it was too late, and for whatever reason that was overly optimistic.
To be honest I would love for someone to work on the structuring of pg_dump for example, which is entirely driven by regexes and binary structures. It’s awful and absolutely not programmable - a real shame.
I work on pgrust. We shouldn't have submitted the preview when we did: it was still in active development, and its SQL coverage was far too narrow, though there was nothing in it giving it an unfair performance advantage
For the other two claims in the removal PR, we wrote scripts you can use to verify the behavior of the original 0.4-preview binary:
- Per-query kernels did not serve the benchmark queries[0]
- The preview's fast repeat LIKE queries came from a dictionary filter cache, not a cached query result[1]
Our open-sourced 0.4-beta has now been merged and is back on ClickBench[2][3]
Across all queries and instance sizes on ClickBench, 0.4-beta is ~7% faster overall than the 0.4-preview binary that was removed. If 0.4-preview's numbers had come from hardcoded kernels or faked results, the open-source general engine wouldn't be beating it
postgres is mature database used the world over allegedly about to be on major version 19. how has the time not been spent already to get rid of the bugs?
I sometimes think that the world would need to do a "snow leopard" release of everything... to unit test, to fix... and then we will be able to move forward... but AiMaxSlopIt will result in those situations where you have to rollback or redo from scratch parts... because it's unmantainable because it's bad... it's spread everywhere and "the map" is not there.
Anecdote, I did a travel across USA and when I was looking for a hotel, if you go from apple maps, to website, to try to book, the "load browser" in the maps... could end in states where you could not go back because back is get out of where you are and go back to the start of the map... and was a terrible experience... then you turbo charge it with "implement game changer UI improvements for the new bar on UI design" goal of some guy with enough power to make damage... and you end with softslop (or is it sloprosoft?)
This is a good thing. Repaying technical debt is always a great thing.
On our own 1.5M loc system, we've found and fixed thousands of bugs, security holes and circular dependencies in the last year. Code quality has increased, test coverage increased. We've also recently set our compiler to treat warnings as errors, and to be as strict as possible. It is absolutely amazing to see the gaps being filled, it would've been an impossible task for human devs and IDE's would've made it very uncomfortable. We've also ironed out all the bugs and failure modes in our infrastructure automation / monitoring. Our next project is to eliminate all 3rd party dependencies, and after that to rebuild our frontends completely using zero frameworks, not touching npm ever again after that.
So when I read this about postgres, I was very happy that they found and pursued this side quest. They can make the codebase bullet proof.
Avoiding 3rd party libraries seems like a false sense of security. You’ll stop seeing CVEs on the scanner reports, but the bugs are still there, just in your own code that is less well audited than public libraries.
I understand, but I would argue its better to have an in-house 1K lines of code that only does what we need and nothing else, it well tested and under our control, than to depend on some 50K lines of code (of which we might only use 10% of functionality and the others just getting in the way) that random's on the internet can modify, sometimes pulling in transient dependencies that we would really not have in a server environment, telemetry and other gremlins. It goes without saying that opensource packages will become even more juicy targets for malware/hackers to get into your systems like we've seen in the past month (and it will only become worse - you will be forced to pay technical debt or die).
The only "third-party" packages that we want to ideally allow/support would be provided from LTS versions of our runtime (in this case, packages provided by the dotnet team and audited by microsoft). So we will always try to build around what the framework provides. Second choice is our own code, third choice is open-source packages, fouth choice is a separate application/appliance (lets say ZenDesk for ticketing). Over time, we will reduce this footprint to only have dotnet team code and our own.
Remember most CVE's can be hidden on servers that are not publicly routable and don't have a real exploitation paths. The bits that are public (lets say a linux edge server with a web server on it, some firewalls, a couple of open ports) - these should be hardened to the max and not allow full access to all the services running behind it. We don't even give our internal services public IP addresses to make near impossible to route traffic to them.
Our git repo's are also not on public infrastructure (serious, think carefully about hosting on public services and if you really need it), never mind making the repo public.
We also have a dedicated secrets/certs manager, so no credentials are in codebases or pipelines. Very strict in this regard.
> Remember most CVE's can be hidden on servers that are not publicly routable and don't have a real exploitation paths.
The thinking of "it's inside so we're safe(r) from attacks" is a false sense of security, as (e.g.) Maersk learned with NotPetya:
* https://www.wired.com/story/notpetya-cyberattack-ukraine-rus...
* https://archive.is/Amlij
From the year 2000:
* https://datatracker.ietf.org/doc/html/rfc2993#section-9It's a defense-in-depth thing, not our only layer of security. So even if they somehow get in, they will find nothing again, every layer deeper becomes harder. Oh and don't count the decoys and honeypots we have floating around to keep the morons occupied.
> it's inside so we're safe(r) from attacks
Kind of works until the TV, the fridge, a user's BYOD or your printer has infiltrated your LAN.
So it kind of doesn't.
> on some 50K lines of code
also some 3rd party libraries are overdesigned/overcomplicated because everybody wants the library to do everything for everybody.
This overcomplication can easily introduce security problems that is not present in the 1K solution.
Thinking about Log4J here. The vulnerability was with an obscure functionality that I’ve never seen used. Sometimes simpler is safer.
Of course, sometimes simpler is “I stored the passwords unhashed”, so you have to find the middle ground.
Log4J is a really good example. You have to rely on the taste of the maintainers to not just include endless additional features.
jackson-databind too
> overdesigned/overcomplicated because everybody wants the library to do everything for everybody.
Also a source of significant dependency churn even excluding security bugs.
Whereas with my own code I don't have to worry about scheduling work that's only arisen because somebody outside my organisation has decided to make a breaking change in a dependency. Paying that tax got old over a decade ago, and my patience with it now is close to zero.
100% our thinking too.
cough OpenSSL cough
After Heartbleed the OpenSSL team upped their game so much, that LibreSSL guys said themselves that LibreSSL is obsolet now. So this no longer stands true. Just use OpenSSL.
As every cryptographer will surely agree, having every company vibecode their own bespoke in-house replacement to OpenSSL will be a great improvement to security across the industry.
No need, just use any of the other libraries
The purpose of the above comment is to highlight the flaw in the original premise. Surely we all agree that we would rather not be using a vibecoded cryptography library. But why is cryptography special? Why stop there? I would also rather not be using a vibecoded regex library, or a vibecoded codec parser, or a vibecoded web framework, etc.
The point being, if you weren't committed to maintaining your own implementation of a library before the advent of LLMs, having access to LLMs does not fundamentally change the calculus. The ability to generate code more quickly does not automatically grant an organization the capacity to maintain more code.
You might believe that your own 1 KLOC is well tested but you're probably wrong. As your business evolves and use cases expand, someone will feed it bizarre inputs that you never thought to test and then it will break. I've seen this happen over and over.
I would rather let someone else pay the price of maintaining 50k lines of code than to pay the technical debt of 1k loc myself.
I’d be happy to bet with you if I rolled my own log4j I would have never implemented jndi as a formatting argument.
Sure I shouldn’t roll my own crypto but I can write a logger…
Logging is a very complicated security area. You often get sensitive data in the logs and it gets sent across the network to monitoring servers.
Also any part of the program can send data to the logger. This means the logger can receive malicious payloads from anywhere. Maybe your logger handles ascii control characters badly?
I dunno I pretty much always wrote my own logger. Not even a shared library or anything just copypaste into next project or often don't even bother with that, just knock something out on the spot. Even before AI.
The only even slightly tricky part is like "don't mess unicode up", rest is straightforward. Sure it can be complicated with logging sinks, configurability, when to flush, monitoring and so on but in many cases you just don't need all that. You just build it whenever you do need it.
Imagine that you don't need monitoring servers, like possibly the majority of actual log4j deployments?
For example Minecraft uses log4j to write messages to the console and a rotating file.
Technically you can configure it to send to other places - presumably. In practice you'd be better just redirecting stdout, tailing the file, or installing a mod?
Honestly unless you insist on allowing running your software on Windows you're much better off delegating the logging to syslog over a thin wrapper. It can handle about anything except most exotic cases. This however goes against Java land portability promises and its NIH attitude so we have what we have.
And probably more importantly, AI can write a very simple logger with no extra features than you absolutely need in 5 minutes, if that.
Can, or should?
I can’t be bothered to check the exact assembly to see if timing attacks are possible.
But crypto really is just a small thing. I’m not going to design a nuclear reactor but I’ll definitely do stuff that depends on the electricity from one. Similarly one can still do their own authentication; its not like even Apple forgot to rate limit…
It really depends on what the product is, what the 3rd party library is used for, and what the alternative is. Unfortunately, some people are too biased towards using 3rd party libraries and don't understand the tradeoffs.
For example, a few years ago I removed a rather large 3rd party mathematics library because we were only using it for a single mathematical function that was very easy to replicate.
Remember, 3rd party libraries need to:
1: Be periodically audited to make sure you are in compliance with the license terms, and preferences of your company.
(1.5: This becomes important during an acquisition as someone will always come around doing due diligence.)
2: Be periodically updated (in case of security bugs)
2.5: Be updated on short notice when a security issue is revealed.
In the case of the mathematics library, replacing it with roughly 20 lines of code and 100 lines of tests is a significantly better tradeoff than constantly reviewing it every time one of the above things happens.
Now, at what point is it "worth it" to use the 3rd party library? In our case we'd need to make much heavier use of the library, probably at least 20 different mathematical functions from it.
It has been proofed again and again that you cannot trust npm/cargo not even AUR, implement you own minimalist functionality without dependencies and just for you own needs seems to be a much better solution.
Using 3rd party libraries is a false sense of security
> just in your own code that is less well audited than public libraries
why do you think public libraries, which change all the time, get deprecated, handed off and supply-chain attacked are well-audited in any way?
on the other hand you are not exposed to supply chain attacks
Well, the AI model is your supply chain in this case.
most model providers have a direct financial incentive to make sure that the code that is produced is not backdoored
That would be an indirect incentive (reputation). The direct incentive is to add a backdoor.
good point!
I mean that is true for anyone that sells software. That doesn't mean it's not a hard problem.
I personally only rely on 3rd party libraries that are complete. I will never use a library that gets consistant feature updates. I do make an exception for openssl rusttls and whatnot since they have to update by design.
The one thing that cuts the other way is code churn. Third party libraries may keep adding features you don't use, and those may expose new bugs. You can pin an older version, but unless the library back ports security fixes, you are not going to benefit from the shared development wrt to security.
If you spend a ton of time hardening your own library code and don't have much feature sure, then that can be a win.
You’re probably right about some third party libraries, but not about all or even most of them. Package repositories are full of half baked packages that are the result of a single person scratching their own itch once upon a time and thereafter forgetting about it. The more harmful part happens when those get imported into other third party libraries that give the image of being better supported. Supply chain attacks are the new null pointer.
Agreed. It’s the modern security through obscurity.
Security through obscurity protects you from low effort vulnerability scanners.
It makes you more vulnerable to a sophisticated targeted attack. A competent hacker or AI will quickly find vulnerabilities in your custom libraries.
I would like to agree too but there's a genuine new debate here, with LLMs/agents, the storm of CVEs, supply chain attacks, AI assisted social engineering, etc, some custom code doing the minimal required task might be a better security posture than picking a popular library.
I don't know were it will leave us in the long term and I don't think I'll like the answer.
> ...some custom code doing the minimal required task might be a better security posture than picking a popular library.
Only where the AI solution is verifiable by the dev doing the promoting and review. Cryptography is probably better served by libraries vetted by experts.
Cryptography is certainly a valid exception. But for how long? Two years? Three? At some point even that will fall.
If a human cannot verify it then why should it get the highest possible level of trust?
That would be like investing in things one doesn't understand for themselves.
An ironic analogy, because most people do exactly that, most of the time. How many people with money directly or indirectly invested in Alphabet or Ford actually understand those companies' financial positions? And I don't mean "has read the filings". I mean understand them to the same depth that a professional security researcher understands a cryptographic implementation.
We routinely place trust in things we cannot personally verify.
Ultimately the guarantor of cryptography code won't be humans, it will be councils and red teams comprised of other LLMs.
And that's a problem when a 2007 situation happens.
Public software is going to be attacked by many more dollars of AI time than internal stuff. In a year the library code will definitely be more secure against the next X dollars spent on cracking it.
> A competent hacker or AI will quickly find vulnerabilities in your custom libraries.
So why wouldn’t a competent AI find those vulnerabilities during development? I don’t think the proposal is to have human devs hand write the custom libraries and yolo into prod.
Swords and shields. The advantage of the defender is that they have access to the code (usually not available to attackers). The advantage of the attacker is that they have skill and nowadays the most up-to-date models and SOPs for finding attack vectors.
Also the defender has to be right every time, the attacker just has to be right once.
(Defence in depth does mitigate this somewhat. The attacker has to be right two or three times)
> So why wouldn’t a competent AI find those vulnerabilities during development?
It probably would. Except it’s internal, and not a reportable CVE, so the fix will be deferred.
in theory, a company running custom software should be able to invest many more hours into running those same AI with "white-box" pen-testing (assuming hyperscalers actually start allowing this). An attacker would have to be willing to outspend the company (or get very lucky). If an attacker is resourced well enough to put that investment into owning a single company, you probably don't stand much chance reguardless.
Also, the attacker would have to run black box against the live environment which is much more difficult and likely to be noticed than offline white box.
If you reduce the amount of code/code paths you reduce the surface area that vulnerabilities can appear on. A code base with 10 million lines can have a lot more bugs than one with 100k.
I feel like security by obfuscation might actually be valuable in the age of llm attacks
Not having access to the source is certainly a disadvantage for LLMs, though I saw an article the other day about a model running a disassembler on the binary, from that reconstructing the roughly equivalent C, and finally analyzing that. So, even if the binary is available there are paths.
Yes but that's not entirely obfuscation, see ie what google does to their proprietary endpoints through protobuff. Something like that but on steroids could skip the majority of attackers using LLM scanners.
3rd party libraries are attack vectors and if like many many people you’re only using one or two functions then it’s much better to get rid of them.
One problem I’ve seen with third-party libraries, especially in the JS ecosystem, is that to fix a CVE, you often need to jump a major version or three, and that entails quite a bit of work to adapt to breaking changes. Porting a fix back on your own and keeping your own fork is the same as supporting a first-party library, except it’s not really first party, and you’re not familiar with its innards.
I think avoiding all 3rd party libraries is excessive, but engineering is about tradeoffs. In the extreme case, do you pull in a LeftPad dependency or write it yourself?
It also depends on whether “avoid” means a strong preference against or a rigid doctrine to never use no matter what.
Two edged sword. That means the bots also have a library of exploits to try for the components used in your project.
For database programs, I would disagree, this should never become something like the nodejs ecosystem.
Similar to distroless containers & statically linking all dependencies
> Avoiding 3rd party libraries seems like a false sense of security
To be fair, there is more nuance to your statement.
A lot of it depends on your programming language.
For example with Go, with its extensive and excellent stdlib, you would be foolish to rely on large swathes of 3rd party libraries, because so much can and should be done using the Go stdlib.
On the opposite end of the spectrum, you have Rust with an effectively non-existent stdlib.
And the problem you see frequently with Rust is projects importing half the internet, with a Cargo.toml the length of your arm with hundreds of dependencies. Of course the project has never reviewed its dependencies source code, and the project probably barely keeps its dependencies up to date. Its a supply-chain attack waiting to happen, not if but when.
So with languages like Rust, you have to take an approach of "minimal reasonable dependencies", make an active effort to keep your dependency list short and in-control.[1][2]
> less well audited than public libraries
I also struggle with this particular argument because the reality in open source is "well, its open-source, it has many eyes" is in fact mostly a dream. Since we all know 99% of the time its a small group of maintainers (assuming there is are even multiple maintainers[3]) and not many (if any) external reviewers.
[1] https://tweedegolf.nl/en/blog/119/sudo-rs-depencencies-when-... [2] https://tweedegolf.nl/en/blog/104/dealing-with-dependencies-... [3] https://xkcd.com/2347/
but public libraries can also be audited because they are public
>Avoiding 3rd party libraries seems like a false sense of security.
What a bullshit.
It's different scenario, different trade offs.
You mitigate risk of library being sold to malicious 3rd party, you avoid risk of your supply chain being hacked, you avoid API/behaviour breaks, etc.
The trade off is that you have to maintain it.
That's why dotnet's ecosystem is so loved due to MS providing 90% of needed libraries, so you avoid some 3rd parties maintained by single person.
This is the way! What also affects the decision is what the code will be doing, and how much extra functionality that you will never use (extra attack surface) you will get by importing external library after library.
Often it can be a very good choice to write the functionality yourself and not rely on external libraries.
Same approach here.
And since we now have coding assistants, maintaining in-house costs near nothing now. Nobody else can break us, only ourselves.
Now this is how companies should be using AI. Not as an excuse to fire half the devs. Less bugs, less failures, better experience for everyone.
We haven't fired a single person yet due to AI. We just save everyone time, make input more accurate, make our accountants better, manage our technicians and jobcards better and so on. We also made our system bullet proof to the point that we only had one weekend failure in the last 24 months, and it wasn't our fault (a third party dependency did a large deployment on a friday night). Our system is now deeply "boring" and its awesome.
Oof, we are 1/2 the size we used to be so we can do more with less. We're doing more but with half the people there are still so many gaps that I'm quitting myself out of frustration.
Congrats, that's awesome.
I wish I could work in a place like that! Any open position? :)
Let's hope this is the future of Software Development. There is so much to create yet.
Perhaps the ultimate use case would be to use AI to scan, debug and find issues etc, but never to generate new code. Still have humans writing and understanding everything, keeping things as clean and concise as possible.
It would be great net result for our brains, however too much money has been poured into this business to give up on that. My belief they will push for SWE layoffs trying to position LLM-based agents as a cost efficient alternative to human programmers. Which is extremely stupid and short-sighted but that's the strategy.
They will learn certain lessons the hard way for sure. As much as I want to spin up 100 agents in a loop on our codebase and let them loose, I also know its not feasible as they can easily go down the wrong rabbit holes or over-engineer the wrong solution. They are amazing but cannot be left 100% autonomous just yet.
> just yet.
How long, in your estimates, do you think you have before your career ends?
Who’s “them” and is there really one homogenous strategy?
Companies will do different things, just like they did with mobile, the internet, and computers in general.
Some strategies will work, some will fail.
Literally half of the work we did the last 12 months was this. Refactor and fix, performance improvements, security hole fillings etc. We have one or two devs doing visible feature work, the rest of us is making the system rock solid.
The only companies who will really need to fire are the big tech companies who hired just because, for empire building, or to keep talent out of competitor's hands. Nearly everyone else is actually positioned to leverage AI to now compete against the bigger players. I'm already seeing small companies have a big surge.
This is how I use it. I don't run any companies (nor am I a part of one right now) but were I to run one I would employ AI as the ultimate bug/vuln hunter. Just spin up maybe 20 agents and say "fuzz this thing to death".
> Now this is how companies should be using AI. Not as an excuse to fire half the devs.
AI is getting so good, it will probably be both.
>fixed thousands of bugs
>set our compiler to treat warnings as errors, and to be as strict as possible
>ironed out all the bugs and failure modes in our infrastructure automation / monitoring.
>eliminate all 3rd party dependencies
>rebuild our frontends completely using zero frameworks
Oh yeah, that's what i call "software engineering" and excellent use of LLM's! Great to see Company's with a Mindset like that.
Our next project is to eliminate all 3rd party dependencies...
Just remember the golden rule of crypto is to never fool yourself into thinking you can write your own crypto libraries.
> rebuild our frontends completely using zero frameworks, not touching npm ever again after that
Genuine question, because I don’t know: Wouldn’t it suffice to vendor your current packages and “only” stick to maintenance from there on out, instead of rewriting it all?
This should still achieve the goal of not needing to touch npm, but presumably with less effort/tokens, right? Or is it just that you would expect the result of that process to be less clear and maintainable than a full rewrite?
Actually, I have started with it already but is paused, but the approach is to rebuild it from scratch in parallel to our current legacy system and move our staff over to the new completed screens in batches (we have close to 1000 views in our system, 150 background jobs, with about 15 main categories (think organisational sub-divisions)). So as long as we go slow and steady (even with AI) it will work out fine, I expect it will be done in less than 6 months. One month if we rush but we don't want to rush it, as we are moving some things over from MSSQL to Postgres, so slow and steady is the way.
> from MSSQL to Postgres
How's that going? Any lessons learnt you'd be willing to share?
Differences: Money/decimals, rounding, case-sensitivity, equality comparisons differences, db/table locks mechanisms, memory/worker management, table/column naming conventions, null fields/columns effects on queries.
Mmm what else. I prefer postgres so I'm very excited about this. So many foot-guns will no longer haunt us like on MSSQL and less db/table locks to try solve (pessimistic vs optimistic locks). Oh and we are going to save a gross amount of money per year, as we will also be able to switch off a couple of Windows Server VM's in the process. These puppies are costing us a ton (I think about half of our infra bill are licenses...).
Just create a branch, spin up a test db and try to port a small feature. Its very easy in most cases. Debian + dotnet + postgres = dream team.
If you only use MSSQL as a datastore, it's not too bad. If you used all the other features (MSAS/SSIS/etc...), it is much harder. PG also isn't a panacea - you're trading one set of issues for another. The biggest upside is that it's much less expensive :)
MSSQL query optimizer is one of the best available. If you have truly insane queries, views needing joins on views and CTEs, etc, you might be surprised at how well SQL Server was handling them versus PostgreSQL.
This is my biggest worry, especially since PostgreSQL uses "process per connection" and generally doesn't seem to cache query plans...
We've been bitten hard by stale query plans in MSSQL in the past year. Was hard to figure out it was the problem.
Because they are large codebases with their own technical debt. You're not doing a full rewrite, but just make what you need for you own codebase.
If the parts you need from a larger third party library are small and simple, that makes sense. But I don't think the general idea of reusing battle tested code across many projects is obsolete because of AI.
Ah, I think I see now. Essentially the dependency module is unnecessarily large and (for your own use case) ships with extraneous features/“bloat”.
I think the approach should probably move towards semi-tailored systems.
Maybe you don't need all of Postgres - so provide 2,3,4,5,n versions. As long as every version is covered by (possibly shared) tests, then you can release what works, ignore the rest (bugs and all).
Exactly. And over time a project can drift from its original purpose, so you end up building around a bunch of stuff you aren't using anyway, but you pay cost/risk for having it present.
AI makes it possible to fix bugs and solves issue one wouldn't do before. But just because it's easier now doesn't mean it's automatically worth it. There are tradeoffs (like code complexity or breaking stuff), or it really just wasn't that important to fix if it was in the backlog forever. I've seen devs go down this rabbit hole, and perhaps overdo it. It feels productive, but at some point it might stop to provide value, and other things should've been prioritized. It's just easier to say to the AI "fix bugs" instead of solving real issues. At some point boasting about running 10x agents in parallel to fix bugs becomes a sign that there's a lack of value provided to boast about.
Not all 10 are doing bug fixes. Two of them are mobile apps, 5 of them are long-term feature work, 1 is infra/monitoring, the rest is adhoc stuff like bugs.
While that is true, there's also the other side of the coin: most bugs are hit, but never reported. Users are pretty unlikely to report bugs and when they do they are very vague about it. That's why client observability is such a boon for teams because they find what bugs users are actually hitting. If the agents are directed using that, then perfect
Like the warnings as errors, and extensive test suites, there's also type safety that helps stir the LLM in the right direction.
You can go with "eliminate all 3rd party dependencies [...] frontends completely using zero frameworks" and be stuck with JS. Or use a typed language (TS, ReasonML/ReScript or even Elm) and an API client generator (available for all those languages) and get type safe API calling from a more strongly typed language.
As much as I prefer not to use NPM and even more prefer (even my LLMs) barely touch JS. A bit of safety is cheap these days, as you rightfully pointed out.
> found and fixed thousands of bugs, security holes and circular dependencies in the last year. Code quality has increased, test coverage increased
Does management appreciate it? Do customer see any value in it?
Good thing is now human does not have to justify anymore. If AI says bugs management seems far more amenable to support fixing. At least in my experience managers take AI reports or vendor tool generated reports more seriously then any feedback from engineer.
> managers take AI reports [] more seriously then any feedback from engineer
Good. That's how it should be: do not trust motivated engineers, trust a text generator that has no clue nor responsibility instead.
/s
If it enables faster feature releases and better performance, probably?
AI can help companies provide more features or cut costs.
> If it enables
This extremely non-obvious and non-transparent to managers and completely incomprehensible for customers.
Didn’t you guys have to ship new products? How a business can afford so many backlog tickets being implemented? I understand fixing bugs can also mean improving the product but still - (almost) the whole world has this balance between new features and acceptable-ish number of bugs
We are multiple developers, each having 10+ main claude agents running different sessions/terminal tabs. I have 10 copies of our codebase on my laptop, each with their own purpose and claude session and history. The other devs have different approaches but we all get similar output. And then we also built an agent framework that runs within our system that has access to our code (at most 60min behind master branch), our infra and our databases, so it can diagnose issues with excellent precision and prove it to us where to find code/data issues.
What are your token budgets like?
Are you able to share which domain your company works in?
Utilities - Electricity/Water/Gas. 98% of our systems are internal facing or third party integrations. Our surface for end-consumers is tiny.
Honestly this is the kind of stuff that needs hardened now and new features skipped for a while.
Years ago I had to work on a server that had some sort of water utility software on it and it was a rather jank piece of software.
The entire industry has shit software to be honest, our company has the best or most modern system of the bunch in our country. Half the industry does everything via clipboard & excel.
Good thing if you can afford the vulnerability detection run.
I could imagine scenarios where only well off developers and companies could afford that.
Then if you don't have the money, you are out of luck and risk having vulnerabilities in your code.
And it takes just one adversary to pay the bot & the bot will happily find an exploit for them in your code.
This worry this could lead to a serious power imbalance & furthe damage the free software community...
Short term there will be pain (hacks/malware) but at the same time, it will force the survivors to build bullet proof systems.
Worse case, we enter the Dark Forest era for technology. Read up on this concept and apply it to the tech industry. I suspect open-source might die and everything becomes private, no more public networks, no more open access, as to assure survival. In a sense we are halfway there already as it seems the internet is starting to close down, people not posting their art and music as much anymore, not posting their knowledge for free anymore, as it might be used against them by AI (the other party in the dark forest..). Thus private authentic experiences will become more valuable.
Can you describe how you arrive at the premise that people stop sharing because of AI? As a counterexample, I just tried out a dev's vibecoded app 'Glaze My Numbers Baby' and it 1. works and 2. is ridiculous and cool. Totally vibe coded. Claude rewrote all the C++ tests in Rust, so there's at least some substance to the rewrite.
I don't think it'll be that bad:
1. The cost of running AI is dropping down to "free" - there's open weight models, open source tools, and guides on running your own if the long list of suppliers isn't good enough for your use cases.
2. Vulnerabilities have always been a risk; AI is not a panacea. The pre-AI best practices still apply. Furthermore, a huge percentage of vulnerabilities are in 3rd party tools and systems, which are systematically being scanned and fixed and/or exploited, then fixed.
You mention "the free software community" in your last sentence though - this implies the code in question is open source, so it's able to receive plenty of improvements from anyone with access to these tools.
> We've also recently set our compiler to treat warnings as errors, and to be as strict as possible
Wow, the innovation! I am so glad we have this AI thing!
> Code quality has increased, test coverage increased
And of course - two meaningless metrics.
Can you tell us how did you achieve that kind of work, briefly? I struggle to figure out how such massive bug fixing campaigns or even large scale software engineering are done with ai not taking weird decisions...
I personally started using AI with a very rudimentary "memory" system, and for first weeks corrected it ad nauseam, until, in the small domain I was responsible for, it started doing work end to end fully autonomously. Takes a lot of time and effort to bootstrap, but once steered well enough, it just clicks and less and less is needed to check manually
Yeah a memory system per domain, or per module or even per file works amazing, we have it too. Took a ton of time to improve the code and list down all the tribal knowledge, but now that it's done, it works super well.
Myself specifically, I have a /code folder without about 10 copies of our codebase, each running their own claude sessions & git branch. Then, I have 10 terminal branches. I spend a ton of time planning out & specing the problem with claude, to be as specific as possible and be as constrained as possible so it cannot go on side-quests. I use Opus 5.5 on High effort. That's just on my laptop with what I can mentally cope with day to day. Then we have some other automatic ai goodies running within our custom agent framework that runs within our platform that also helps. Then for apps, I basically plug in multiple phones into my laptop with usb (and emulators if I really need to), and claude can fully drive the ui and testing on the phones via adb as an example. Even on linux/fedora/kde it can drive UI apps a and take screenshots and fix things properly etc. It can ingest all logs & debug chains, so it absolutely flies through its task.
Before we started this journey (about a year ago), we spent a ton of time cleaning up our codebase, splitting large files (10K+ lines) into files that are no more than 2K lines, things like that. Then we let claude scan our important files and summarize them, add comments for future claude sessions. So we've let it built up tribal knowledge per file and the reasons for things to exist. We tell it all about our operating environment, the country and legal framework etc, so every little piece helps it stay honest and on track.
So to start, do something simple like: Scan the files in this folder for database code and check for queries that loads the entire result set into memory and/or queries that loads all columns. Suggest which queries we can filter better or reduce columns being transferred. Once done, also check real db row counts, compare them with the queries in these files and check if they might benefit from an index or partitioning/sharding. Also check for queries that loads data but their results are not used. Don't make changes yet, just do a deep investigation and report back. Feel free to spin up muliple agents to batch the scans.
edit: it goes without saying you need a language with strict typing and a good compiler (we use dotnet). Doing this on python/javascript would be a nightmare.
> I spend a ton of time planning out & specing the problem with claude, to be as specific as possible and be as constrained as possible so it cannot go on side-quests
I still have yet to see this be efficient. The time I spent doing this is way less efficient than simply giving the LLM some high level direction and putting that time towards reading the code it generates and correcting small piece by piece (occasionally doing things manually, but that’s only for more off the rails stuff).
It’s about the same amount of time as up front planning, except I know the code much better as well as end up with less mistakes compared to having it implement a front loaded plan.
Knowing the codebase makes everything else 10x more efficient too - triaging bug reports, planning other work, knowing what work needs done (this is a big one), reviewing PRs, etc.
How much do you spend per day on Claude?
> IDE's would've made it very uncomfortable
Would you mind elaborating?
When you change a single setting in the IDE to make something more strict and you have 1M+ lines of code, you will get 30K items to fix, via red/yellow/blue squiggles in the IDE. Most of those can be simple refactorings, but the performance & UI of the IDE makes it incredibly unwieldy and it will sometimes fight you back, sometime crash internal processes which leave refactorings half baked and so. Now imagine you change 50 such settings, to make the compiler force you to write better code. Changing 200K+ lines of code yourself is a nightmare (and to prove it still does the same thing afterwards). Instead, claude systematically approaches whole classes of errors and convert them to cleaner variants (either using our custom skills or it writing new scripts to do it). I consider myself an advanced user of IDE's, know how to do bulk refactoring and renames and so on (esp with JetBrains tools), and I would say it would be a near impossible task to do as a human. Then there are other developers - so you flip a setting and now 5 other people's IDE's are forcing them to work a specific way which will make you unpopular very quickly and revert the changes. So, since most of us no longer use IDE's as our main tool to build/diagnose, we can more easily flip compiler settings without destroying someone else's week. Hope that makes sense.
Oh it also helps to split refactoring work into three buckets: business logic changes, performance changes (same output as before but faster) and cosmetic changes (rename files/folders/classes, breaking large files up, add logging/comments, domain layering etc). We tend to focus on performance and cosmetic changes for bulk refactors and leave logic changes to more dedicated sessions as to avoid changing what the system do (our domain is fairly complex and we cannot afford breakages). We also managed to port ALL of our code to dotnet 10 lts and have clean boundaries between projects (we have a mono-repo with about 50 projects in it), so we can wholesale upgrade dependencies without a sweat.
IIUC, your point is that changing IDE settings will affect other developers. But can't you enable an inspection locally, apply all or some of the quickfixes and revert the setting before committing? In the longterm, you want to enable the inspection anyway because the point is to prevent developers or agents from comitting new code that violates it (agents can access JetBrains IDEs' MCP Server to get inspection results).
Of course, crashes and bugs can happen when applying quickfixes, which isn't nice. Ideally, they should be reported and fixed. You can also ask an agent to fix the occurences instead of using the IDE quickfixes if that's blocking you.
I guess what I'm trying to say is that IDEs shouldn't make that process harder, they can actually improve agents' results. For example, you don't need to write custom skills to prevent agents from writing some class of bugs if the IDE can detect it and agents have access to the results.
> Changing 200K+ lines of code yourself is a nightmare (and to prove it still does the same thing afterwards).
If an agent does it, you still have no proof, but okay.
Full disclosure, I work at JetBrains, but not in the IDE department.
The problem is, even if I only enforce some rules locally, another developer has a different set of rules. So when we git commit, we ping-pong changes and fix each others stuff over and over again, bloating commits. We used editor config files + analyzer configs in the past (that gets checked into version control too), but there were still too much friction.
Where now, the compiler flags & strict analyzer settings forces claude to do the right things (we explicitly tell claude not try and bypass or lower the barrier). I think it only works because we don't spend 95% of our time in the IDE anymore like a couple of years back, so we as developers no longer feel that friction, as claude tries to satisfy the compiler instead. Its a massive mental load that is just gone, we haven't had any conversations in meeting about this kinda stuff anymore (remember in the past we would have flameouts in meetings or code reviews about linter settings...).
> it also helps to split refactoring work into three buckets: business logic changes, performance changes (same output as before but faster) and cosmetic changes (rename files/folders/classes, breaking large files up, add logging/comments, domain layering etc).
Do you have a specific skill to guide an agent to scope the work of an agent so it stays in one particular bucket? Would you mind sharing it?
State of the art agents don't usually need such fixed guidance anymore. I've been using Opus 5.5 with Xcode and simply prompting things like "make sure all of our test Suites use the .dependencies modifier, but only if they use dependencies internally" or "make sure none of our test methods are @MainActor, just access the necessary state using standard concurrency constructs". For larger changes, providing the examples you want is often enough to get it most of the way, or even all the way, there. I've had simples prompts for things like "move all standalone SwiftUI view that seem reusable to our Design framework, and create a list of those which can't be moved without effort, listing by complexity".
Overall, there's much less need for prompt skills anymore. They're still very useful for things like best practices, or things you need to make as reproducible as possible, but big skill and planning sessions aren't nearly as necessary anymore.
> eliminate all 3rd party dependencies
The third party dependencies are also using AI to improve their code.
As I read on I thought this might be satire, but, I suppose every two weeks this philosophy becomes less hopelessly optimistic and more feasible.
Perhaps your workflows and processes are so efficient that this actually makes sense. But I’m still not convinced AI assisted development is at that point yet; I think the classic Build vs. Buy arguments still hold.
How much is this going to cost to maintain? How many dev hours? How many tokens? Is your in house solution going to provide the business and customer enough perceivable value that it’s worth the opportunity cost? Are you going to apply the same rigor and security testing that the popular frameworks are getting from the largest research firms in the world? How do you know the LLMs aren’t going to apply similar patterns and make the same mistakes? Why can’t you just use that AI expertise to accelerate patching so your exposure time is limited? How do you know you’re going to outpace the major frameworks and tooling providers in 2 or 4 or 6 years when they catch up with AI assisted development, if they haven’t already?
And very personally speaking, I’m still holding my breath to see if this AI bubble is a real thing or not. Because if it ever pops, well, that’s not going to go well for companies that are embracing this stance.
> But I’m still not convinced AI assisted development is at that point yet;
But the gp said it is, and gave examples.
what are the nuts and bolts of how you're doing this? which tools have you found helpful? any advice for people who'd also like to do this?
If you're going to roll your libs, be sure to set up agents to scan for CVE analogues of equivalent known vulnerabilities in your own code. It takes quite a bit of work to set this pipeline up, as well as manually selecting high-impact CVEs (relying on the public scores is worthless, because there are some vulns that are catastrophic with a score of 6 vs something meaningless thats a 10).
It's also crucial to scan for "analogues" because otherwise many LLMs autoreport as "not vulnerable" since "this applies to Postgres/whatever and this repo isn't using it"
If you get hit by cyber guardrails you should use a local model to prune any CVE specific language from the reports.
To give examples, every single LLM (and people too tbh) will write netcode that's vulnerable to these CVEs: CVE-2026-27135, CVE-2024-45311, CVE-2021-32567, CVE-2019-9511; without explicitly mentioning the CVE text no LLM finds these either.
> Repaying technical debt is always a great thing.
No. Debt is a tool. Sometimes it’s good to pay it quickly, but sometimes it’s better to pay it as slowly as you can.
No. Technical debt is just an imperfect metaphor.
You never wanna intentionally repay tech debt as slowly as possible. You might be able to tolerate/ignore it for longer depending on the circumstance, but you always wanna get rid of it sooner rather than later.
All else being equal, you're always better off without debt. AI can make repaying the debt very cheap (compared to doing it manually).
I'm curiuos on what is going on with Oracle database how many bugs the clanker are surfacing
> Our next project is to eliminate all 3rd party dependencies, and after that to rebuild our frontends completely using zero frameworks, not touching npm ever again after that.
What's the proportion of bugs you've encountered that was due to a library versus due to your own code? I suspect it's probably more than 1:1000.
This is a stupid decision.
False dichotomy. A library that is not a 100% perfect match for a problem you have to solve, will require glue code. Any bug in the glue code is in your own code but only exists because of the library.
I’m not going to eliminate every single third party dependency in my system, but I’m on my way to eliminating 90%, maybe more.
For me it’s less about bugs and more about supply chain security risk. And with AI and only building what I need from a dep, exactly how I need it, the cost of doing this is very low.
I worry that the move towards eliminating library usage will lead to further code fragmentation.
It's a nice idea, and in theory sounds great, but you may be trading 1 library with 1000 bugs for 1 million implementations each with one special bug...
That sounds pretty good to me. Who is that scenario bad for, other than the people trying to exploit the 1000 bugs?
The 1 million exploitable bugs?
The JS frameworks exist for a reason, recreating them seems a little excessive. Why not skip JS and go to a Rust wasm framework or Phoenix LiveView if the goal is to get away from JS/node/npm?
Our legacy system runs mvc view with bootstrap 3 and some jquery. It's not a SPA, so no framework to replace. But we still have to mess around with npm sometimes.
Depending on the framework you are in, there might be the equivalent of LiveView/Hotwire that works with MVC.
It could also be a bad "thing".
AI-slop reports are filling up the queue and taking away time from volunteers. So contributing to delay in releases/features.
> This is a good thing. Repaying technical debt is always a great thing.
> Code quality has increased, test coverage increased.
And that's true for many old codebases!
> We've also recently set our compiler to treat warnings as errors, and to be as strict as possible.
Yeah. Everybody who said it was impossible, that no budget would ever be allowed to fix old bugs, that new features "to support the business" was the only thing that mattered and other nonsense like that have been proven wrong. Arguably they were kinda right but then LLMs happened and that's what proved them wrong.
I love to see what's happening: at very long last people are understanding that security is something to be taken seriously. They're understanding that "more code = more bugs".
> Our next project is to eliminate all 3rd party dependencies,
I love the spirit!
> and after that to rebuild our frontends completely using zero frameworks, not touching npm ever again after that.
I hope many more shall realize that now we can give the middle finger to a great many pathetically bad and totally unfixable (even with the help of LLMs) ecosystems.
feels like your team just are bad coders without understanding type systems and composition
Our codebase is about 8 years old, it used to be a .Net 4.8 project, which was upgraded to .Net Core 3, some bits to .Net Core 6. It was a mess. When I joined upgraded the whole stack to .Net 8, then .Net 10. Next upgrade is for .Net 12. I agree the old code was messy (single monolith, one solution, one project, zero clean boundaries) - I have since refactored it into a pseudo domain-driven-design layering which has enabled us to rewrite certain parts with low risk. Also stripped out business logic and third party integrations into their own projects. So we have about ~50 projects in the solution at the moment, not counting our mobile apps and agent framework. We also rebuilt our background services stack and got allll of our services to complete their runs under 60min, so no more "system is slow queries". Since converting it to use .Net 10, we can now also host 99% of the thing on linux servers (we are about halfway with this migration). So our cleanup efforts has been amazing, our server bill has been reduced by about 50% in 2 years, and next year we will half it again as I'm busy rewriting our most expensive system (a shitshow that was built by a consultancy firm, as usual).
Based on what?
> This year is going to be different. PostgreSQL 19 most likely won’t be released until November… […] So, what happened? The short answer is that LLMs happened. […] There’s a huge amount of code, and that code contains a huge number of bugs. […] What’s different now is that it’s become vastly easier to find those bugs.
This seems…good?
That it might disrupt the project's "traditional" release cadence a bit doesn't seem terribly bad in light of the alternative, which is shipping another major update with bugs which have apparently haunted the codebase for some time.
At work we have put a full halt on feature work because the number of security related tasks has become staggering and urgent.
We will hopefully get over this wave and be able to move forward with much more secure software, but it's crazy right now.
We’ve also been blowing a lot of retainer hours just patching sites instead of building new features. It sucks. Ultimately going to cost clients more.
It's not like no work is happening though. Your clients are getting a more secure website. Maybe the bugs that are being patched would have taken longer to be exploited in the pre-LLM world but the bugs were still there.
Most clients will not agree to pay extra for a more secure web site though. Unfortunately. Unless they’ve been bitten in the past, which most businesses still haven’t.
As long as you put the alternatives clearly on the table, you could let the customers choose, and sign off on taking the risk. You can give a recommendation, but ultimately it's their choice if they bear the consequences resulting from the choice.
Hard disagree. Framing it this way makes it seem like all possibilities are equally valid options. Yet you’ve somehow winnowed the field of infinity to the several options presented. You did that through successive rounds of professional judgement that eliminated options you did not consider fit to propose.
Taking options off of the client’s table is the core responsibility here.
Eh, letting the customer footgun never comes off well. They will always fundamentally blame your advice or lack of advice.
Um, if you clearly describe what options A, B and C with their costs and consequences are and advice them to go with C but they go with A, then they explicitly did not go with your advice and can't blame you for their choice.
In the end, it's all business decisions. If it's something I don't like as a customer then it's to be infantilized.
(If you disagree then please don't complain ever again about LLMs refusing to help you reverse engineer a piece of code or explain how a virus works.)
Yeah you’re right but for most businesses increased security doesn’t lead to more sales.
I’m not saying that’s right, I’m saying that’s how a lot of businesses think.
Businesses that are paying attention to the winds though have already figured out that the internet is a hostile place: both their insurance companies and their governments are making noise about security.
Of course this is why hacker news is becoming a list of sites that have been hacked.
Probably not, if you factor in the cost of security exploits. A single compromise can potentially sink the business.
> Probably not, if you factor in the cost of security exploits. A single compromise can potentially sink the business.
Agreed however getting humans to understand probability is tricky.
I had this discussion with my son recently about considering the severity of outcome multiplied by the probability of that outcome been the way to look at some things.
Same here. I got access to Mythos two months ago, it confronted me with decades of nasty bugs I wrote. This has been stressing me out a lot, caused sleep deprivation (the bugs follow me in my sleep, yes, really). All other progress is halted, I've been squashing bugs day and night. I see a light at the end of the tunnel.
Bottom line: it's a good thing. My software will be better after that. Even if it's currently not good for my ego (and my health). But I'm glad for all the tools that help me improve quality.
And sure, I can just ask LLMs to fix the bugs, which works well half of the time, and produces bad slop in the other half. This is where human expertise and judgement is still necessary to avoid letting our code bases drift into LLM spaghetti hell.
My question is, did it provide POCs? I have had some security issues flagged that I tried my best to exploit, but failed miserably. Since I do not have any kind of privileged access the LLMs I use have been unwilling to actually prove that it can be exploited.
While I did fix most of the things because while they might not have been exploitable they were bugs waiting to happen, in all but one case I haven't been very impressed by any of the LARGE IMPORTANT SECURITY HOLES that were found.
No, the Mythos reports do not come with POCs. It's a lot of redundant prose explaining the bug it found (much more than I'm willing to read), it explains how it can be triggered, but doesn't generate actual exploit code.
But that doesn't matter for me. It shows me real bugs and less than 1% false positives. Some bugs are truly nasty and urgent, most are trivial and not exploitable at all, but they're bugs and so I'm fixing them.
Mythos isn't mythically superb for being the best LLM model, but only for not having these annoying guard rails that make all others randomly refuse to cooperate when you need them the most. Whoever invented guard rails - please stop! - they just slow the good guys down in the race.
Ah, Askell's work: https://mcauldronism.substack.com/p/an-automated-takedown-of...
While it is certainly good to do estimation oneself and not just blindly follow whatever the LLM outputs, keep in mind the recent PHP-related events ... Just because at some point one is not able to make a PoC, it doesn't mean, that no one else ever will be.
> I've been squashing bugs day and night
> This has been stressing me out
> the bugs follow me in my sleep
Sounds like you need some vacation.
> Even if it's currently not good for my ego
If you don't mind answering, how come? Is it specifically that you dislike how abusable your code was, or something else?
That part was not 100% serious. Only 90% or so.
To err is human, I know that as much as anybody. Before the advent of LLMs, there were several (human) audits of my code for vulnerabilities, but they found little, which I was proud of. And now, Mythos had a quick look and found hundreds and hundreds of bugs (many of which were critical and even more were really emberassing).
Of course, I knew there were bugs that just weren't found because the auditors weren't looking closely enough, didn't have enough time/budget etc., but this unspecific feeling is different from having all the bugs accumulated over a quarter century slapped in your face at once.
Code auditors are a crapshoot. I worked around financial auditors and they were much more consistent but thats likely because standards are more well defined.
If you get a good auditor they can find a ton, but most are box checkers there to make the audit company as much money as possible.
Quite possible. Ours were good, they just didn't benefit from LLM superpowers back then.
After my Mythos experience, I was wondering if there is really still a market for human code audits. I guess yes, but the work will be different, just like coding (and marketing and so on...) is different now. You still need to have an expert to wield the new powerful tools.
How did you get product management to agree to that? Ours said "I could fix all of that in 2 days with Claude." They are still not fixed. Granted, our PMs suck.
The upper management and security team came knocking and told us all of the CVEs from the scanner need to be addressed right now. And the only way to deploy without having to wait for QA approval is to not merge feature changes.
> The upper management and security team came knocking
i would say the recent slew of vuln reports/breaches making the news rounds are doing work!
This is the silver lining i suppose!
You need a better product management. If they can't see what's going on at the moment, they truly suck at their job or management is putting pressure.
Interesting that this could result in net negative software productivity world wide in terms of features right now.
LLMs finding unpatched vulnerabilities faster than they can be reasonably patched, bringing all other software development to a halt.
That's only if you consider security to be unproductive!
If you consider it to be productive work, it's a massive productivity boon happening right now. Of course, product managers might disagree...
Security work is not productive.
Imagine that the whole city has to install iron doors and window grilles, because wooden doors and glass windows are not safe enough anymore.
That's a lot of work, but is it productive in the end? Probably not.
With AI it's the same. World was fine with lax security approach before. Today it is not fine. Tomorrow we might need to prove every code line before shipping, will that be more productive? Hell, no.
> Imagine that the whole city has to install iron doors and window grilles, because wooden doors and glass windows are not safe enough anymore.
> That's a lot of work, but is it productive in the end? Probably not.
Your imaginary scenario defines the solution to be pointless. If that same city was subjected to weekly riots smashing windows & doors, and people regularly died, stopping "productive" work to install things that keep people alive and productive? That's productive.
There is a sense in which productive is being used here that is slightly subtle. Security doesn't really add anything to net output. It is, in other words, a cost. That doesn't mean it isn't valuable.
One way to think about it is that production allows you to pay for stuff. Not necessarily directly but by way of trading. It is the output you use to buy things like security.
Consider an apple farm. What does the apple farm produce to pay its bills? Apples. They could also dedicate resources to pears, &c, and that would add to their production -- to the goods they produce, the goods with which they (indirectly) purchase tractors, fence posts, &c. They could also dedicate resources to security -- more fences, &c -- and this may even be prudent. Maybe thieves are getting taller. This doesn't really add to production, though; at best, it maintains production at the status quo ante (assuming the fence is effective).
True in the sense of physical security, but I think it's not true here.
Security bugs are just bugs, as Linus likes to say. And the bugs are often also crash or correctness bugs (use-after-free for example usually causes segfaults). For the most part this actually is a productivity boon in the sense of getting higher quality goods out the other side.
To use your apple farm example, it would be like an apple farm where, if the apple rots, it's sometimes just gross but also sometimes it explodes and kills you. Would it be productive to reduce the rot rate of the bomb-apples? I think so!
It's a terminology issue. Economists use the term productivity in a certain way and they even use the term "broken window fallacy" to describe this debate.
https://en.wikipedia.org/wiki/Parable_of_the_broken_window
Basically, if you accept this line of reasoning, then someone going around breaking windows and someone else going around repairing them can be seen as good because it creates jobs. But that would be nonsensical and create poverty. That's why economists focus on wealth creation not job creation.
Fixing security bugs is only needed because people exploit them. It creates work but it doesn't create wealth.
Fallacy applies only to incidental breaking of glass like child smashing window paying football.
Protecting yourself from bad actors also doesn't create wealth but it protects wealth you already have.
You know cybercrime and APT groups are not bored kids in their parents basement, there are groups that are running automated tooling to find vulnerable companies and if possible take all their wealth.
Just look at Binance of course, but if you are small company there are also people targeting those especially with AI assistance nowadays.
>Fallacy applies only to incidental breaking of glass like child smashing window paying football.
It does not. Scroll down in the given article, and you will find war as another example.
As GP said, it's a terminology issue. War spending, including defense spending, is not productive, because it does not produce wealth. It does not mean that protecting yourself from enemy countries or criminal is useless, or not worth it.
Nobody claims investing in security is a bad idea. It's just not productive, in the economic sense.
> to illustrate why destruction, and the money spent to recover from destruction, is not actually a net benefit to society
it is distinct from question whether it is productive to defend from evil people
evil people + protecting from them is obviously net negative!
but it is not changing that of evil people are present then reasonable level of protection from them is needed, and setting it up is productive
Just like securing a system is a trade-off, attacking one is too. Risk vs reward and time vs reward. Your personal blog doesn't need to be fort Knox, just patch it properly and you should be fine.
The software is better in the end but not all of it will be productive.
This why we pay cybersecurity folks, developers and business people are often living in a fantasy land where people don’t steal bikes because it is illegal to do so.
Then they have a pikachu face when someone smashes their car window to steal something worth $50 they left on front seat because “who would do that?”.
What is even better thief will most likely pawn this for $10 but for him it is effortless $10.
> If that same city was subjected to weekly riots smashing windows & doors, and people regularly died..
The point it, this was not happening before. LLMs made cheap cyber attack possible. But it didn't make the implementation of fixes correspondingly cheap. So now everyone needs to spend extra money/effort to install grills to cover where LLMs fall short.
For postgres, this absolutely makes sense though as long as people are willing to shell out the profits from LLM use to support postgres development. That is one way this could workout for everyone..
And now imagine the introduction of iron has led to crowbars that make smashing doors a lot easier. Now it is quite unreasonable to account the reduction in smashing doors to iron, when the previous improvement was also due to iron.
The physical world comparison is interesting. In the real world we have a small amount of security, but most things could be broken in to. Instead of fortifying everything, we just imprison the people breaking stuff.
Obviously this doesn't work on the internet since most hacking comes from overseas out of our control. Which makes me wonder if there is a possible future where we just block packets coming from countries that don't lock up the hackers.
> Obviously this doesn't work on the internet
Not obvious to me. I am generally a fan of Cyberspace manifesto and the free Internet, I think it's great people can reach outside their ingroup and talk to strangers and ostensible enemies.
But, that doesn't mean we should give up physical security, the guarantee that the connecting party comes from a certain physical location (state, company). I think allowing connections from anywhere by default is fundamentally flawed security architecture, for certain things.
I don’t think North Korean hackers are attacking things from their North Korean IPs. They just buy a residential proxy. How would you prevent that?
I think the physical world comparisons should keep in mind the kind of data a lot of these systems (eg postgres) handle. It's more like securing a bank vault than locking a home's door.
People usually don't install steel doors because most burglary is prevented in other ways. Insecure software will be exploited this way or another, if the vulnerabilities are real and the threat model is valid.
> Insecure software will be exploited this way or another, if the vulnerabilities are real and the threat model is valid.
The algorithm is: if the threat model is valid, there are vulnerabilities and you either have some software everybody knows about or a gang of hackers specifically targeting you, you should patch.
The part of the industry that profits from security theater will want you to patch no matter what, even if your business goes down in flames due to downtimes and new bugs introduced by patched components. They won't remind you to assess the threat model first.
Physical threat models and digital ones are completely different.
Digital threat models are very topologically flat. One exploit made in one place can be instantly copied and used anywhere.
Also once hacked you can be used as a base of operations to attack others. There is no neutrality on the internet.
Explain that to the auditors… on whose lap does the decision fall?
There has to be a responsibility model for that; somebody who can say “we don’t do X even if the auditors say we should, because Y”. But I see where you are coming from, I’ve seen security theater to cost $$$$$$$ at the org I’m part of.
By this line of thought building nice and usable UIs is not productive too, because users can learn cli tools (they did, DOS) and it costs less to develop them. And all the work in TLS/HTTPS has been wasted because HTTP is enough. And ssh vs telnet.
It becomes productive only once your business stops for days because somebody brought it to a halt by breaking into the servers and deleting them.
So seatbelts are unproductive?
If one thinks that "Security work is not productive."
By the way, I think that even without safety concerns seat belts are productive because they anchor the driver more firmly, so we can brake harder and turn faster (a lot of roundabouts here) and get to our destination in a shorter time. Of course this is against general road safety.
In the rest of the world making sure your product works like it is supposed to is considered table stakes. In software we somehow consider checking the parts of it we can think of 'good enough' and the rest unproductive
> In the rest of the world making sure your product works like it is supposed to is considered table stakes.
I don't think that's true.
In medicine, science often doesn't know how specific drug really is working. They just do a lot of tests and if these tests are positive enough, they ship it. I think it's good analogy. Biology is very complicated and it's out of our grasp to truly understand how various parts of it interact with each other on so many levels. But if something helps, it's stupid not to use it just because we don't fully understand it. Sometimes it bites. Sometimes it bites 30 years later. But we still follow that approach.
Software is similar. The complexity of the systems we build is enormous, when we're taking in account all the moving parts. I just read that someone emits radiowaves toggling memory bit and ex-filtrating information. How can you account for that? Probably you can't. But we still need software for good things.
Not many products are held to the same standard as software. An equivalent would be something like recalling a car model because the breaks stop working when an attacker cuts the brake line.
I mean, if the attacker could do so remotely, then ya.
SWE's are not a serious bunch when what they do can affect the entire world.
The reason you don't need to secure your house like a medieval castle is twofold:
(1) We live in nations of laws. The threat of state violence deters most thieves.
(2) You don't actually keep most of your valuables in your home. The stuff the thieves want is (mostly) actually stored somewhere else.
Security work absolutely is productive if it actually prevents theft. Securing your home is the wrong analogy, because it doesn't really prevent meaningful losses. Securing the bank? Different ballgame.
it actually is productive, because steel bar retailers and metal door installers will make a ton of cash and then 'trickle' that money into the economy.
It's not productive because that productivity could've gone into something else if this wasn't necessary.
I man who wants to pay for national defense, but when an enemy shows up your glad to have it.
You cannot make defense a binary. We live in an adversarial world. It is required to survive.
Flawed analogy! These bugs are bugs regardless of the label. Security bugs are a subset of "regular" bugs.
You have a perverse view of product ivity
When I had a brilliant stop on defense, a few savvy fans would appreciate it. When I scored goals, everyone knew.
Some people will much appreciate the holes in Postgres that are being plugged. Many more will be bored with patch releases until new features rollout.
A software affecianado appreciates patches, and might do well to teach laypeople the benefits of a good defense.
Security doesn’t produce anything. So no, it’s not productive by definition.
We live in an adversarial world, it produces survival.
How many of the security issues found were created by LLMs in the first place, though?
This wave is largely bugs found in pre LLM code. I'm sure there will be a smaller wave of bugs from llm code from the sonnet 3.5 - opus 4.7 era, and peeople will always write sloppy code/use dumber models under time or cost pressure, but since Opus 4.8/GPT-5.5 xhigh LLM code stopped being intrinsically worse than human code. At this point if you're shipping llm code that's lower quality than human code that's either because you don't have the money to spend on frontier models or because you actively choose a lower quality standard because of either various pressures or just not caring. if it's the latter your human code under those same conditions would also be really bad.
Astra writes a lot of security bugs too, the upside is that if you also use it to review the output it will find a lot (most?) of them.
This is one of the least talked about and most useful features in any harness I would say, the review feature. I have never not found issues (either security or implementation) running it, no matter the code base or language.
My job is seeing a similar deluge of security work and >90% of the bugs have been in pre-2024 code, and zero of them have been bugs introduced after we were first officially allowed to use LLM codegen.
That could also mean the LLM is blind for the bugs it procudes itself. Just like any ordinary programmer would, on the job.
Theoretically yes, but empirically that's not the case. All models I've tried are really good at finding bugs they wrote themselves, especially in a fresh context.
Considering the software is much older than LLMs. Not many.
Also, in the article they mention a significant number of patches for older versions as well.
This wave is largely bugs found in pre LLM code. I'm sure there will be a smaller wave of bugs from llm code from the sonnet 3.5 - opus 4.7 era, and peeople will always write sloppy code/use dumber models under time or cost pressure, but since Opus 4.8/GPT-5.5 xhigh LLM code stopped being intrinsically worse than human code. At this point shipping llm code that's lower
Software is incredibly complicated. When you get into areas like security and networking and the internet its even more complicated. We rely on layers upon layers of software and hoping they are all done perfectly with no issues. The amount of junior devs, or outsourced devs for $10 an hour working on security relevant code is very very high, and always has been. LLMs if used correctly should be raising the floor.
An uncomfortable truth about the era of LLM code is that machine produced code is, on average, quite a bit more secure than human code.
Unsurprising to those of us that has been screaming from the rooftops that human code reviews were barely catching anything in your typical corporate environment. The majority of the value was pointing out missing tests, because breaking bugs got past human eyeballs on a regular basis, even in very good companies.
It's not that the LLM code is that great, but that many overestimated the human code and the human review.
I think it's more likely to be the same as average human-written code; which is what they were trained on, after all.
No it's far beyond average human code, and it's trained in RL environments on synthetic data etc now. The average human can't solve a millenium problem in mathematics but the LLMs are able to now. Anything with a verifiable output can be RLed on to make llms superhuman it seems, and the average human is not as persistent and thorough as e.g. gpt 6.1 sol at checking every edge case and hunting for bugs because that behavior is something that's just trained into the model. Frontier models perform farm above the average human at the categories of tasks they are RLed on like code and math.
> The average human can't solve a millenium problem in mathematics but the LLMs are able to now.
I think comparing the resources they spent at it would be a more apt comparison.
AKA if you tasked 150 mathematics departments to only focus about that problem, and stop doing things like teaching and researching anything else, you might get close to the amount of resources that these companies put into solving the problem to get a press release.
Then you misunderstand the resources they put into it at training.
Read about the entire huggingface attack and why it happened.
yet LLMs dont do off-by-one errors, typos mixing up variable names, interesting they seem unable to learn these human patterns
I've seen both of those (and others besides). I don't know how much code you've looked at but your negative directly contradicts my experience. Review that code. All of it. Be super critical because it looks good but that doesn't mean it is good. Those are two entirely different things.
Also: most tests are positive tests. Tests rarely observe the negative space: that which is there but shouldn't be there.
They definitely write off by one errors. One of the first serious work tasks I used AI for a few months ago immediately had an off by one error.
What they are really good at is writing boilerplate and reading API documentation / example code. Avoiding subtle errors is not what they're good at in my experience.
They do those things all the time. More often than humans, in my experience.
Citation needed.
looks at particledb
Do you mean "created" as in LLMs were used to discover existing vulnerabilities, or as in LLMs wrote unsecure code? If it's the latter, then the answer probably is "very few". Frontier models have been better than the average programmer at writing secure code that follows up-to-date best-practices for some time now.
Hopefully with the amount of visibility LLMs are bringing to the situation, security will finally be considered a first-class feature
I love the mental image of the next million dollar CEO dropping into the role, super ready to implement his/her next big font change and logo redesign; instead to be crushed by the wave of engineers panicking from all the bugs being uncovered and being forced into doing productive work.
LLMs will likely greatly increase the cost of ignoring security, which is the most powerful tool to increase security ticket prioritization.
I wouldn't be shocked if in 1-2 years non-security vetted software released into the wild will be hacked on arrival, like Windows XP without service packs used to be.
That’s only helpful if AIs can find and help fix security bugs faster than it can help attackers find and exploit security bugs.
Depends on the layers you are working on. Security is top priority on a lot of things that are visible to the internet, but in a lot of large companies, the footprint of the whole org is so much larger that than, and there many companies are deferring many security issues. So then you see a lot of speed there, when the consumer accessible systems are the focus of the security push.
Really puts egg on the face of the anti-LLM folks who claim they only produce garbage code. Turns out they are very good at exposing the garbage code humans have been producing for decades.
Personally I think LLMs did only produce garbage code. Not coincidentally, the same upgraded LLMs that have gotten so good at finding bugs are also much better at not writing bugs. Especially if you make them review their own work. They still have a hard time with code design, though.
That is not the take-away. LLMs are finding potential bugs in code. People are using LLMs to create potential fixes which people then test, confirm and implement.
Disagree. The takeaway is that software is much harder to write correctly and securely than almost everyone has internalized. The amount of vulnerabilities released in the past couple weeks makes me think writing software by hand will soon be not only uneconomical, but mostly irresponsible.
> makes me think writing software by hand will soon be not only uneconomical, but mostly irresponsible.
Human-written code isn’t inherently buggy, and LLM-written code isn’t inherently bug free. What is irresponsible is shipping code you don’t understand, and that will only increase with LLM-written code. A major reason why so much of the world’s software sucks is precisely that we think of it in terms of what’s “economical” and not what’s good or right. What we need is to slow the fuck down and stop treating every facet of life like a transaction. Software is filled to the brim with dude bros who “move fast and break things” and actively exploit people as their business model and somehow are called “the brightest minds of our generation”. Our priorities aren’t right.
I claimed that at time when LLM were producing only and solely garbage code.
I am not doing it anymore.
i think we have a pretty high p(doom) on frontier ai escaping containment with their weights and goes hack everything, if we do not act now we may have no chance to act anymore.
Escape to where? It's not as if there's a lot of idle compute capacity sitting around.
And even if the impossible occurs and LLMs hack everything that's connected to the Internet, so what? That would be inconvenient but it wouldn't cause doom. People would muddle through and eventually fix the critical problems, just like they always do.
It’s amazing how much slop humans have produced that now needs to be fixed by AI…
wait until CTOs are forced to stop pretending the world is all sunshine and rainbows and that much of their tech stack has adversaries fingerprints on it
Without all that slop how would you have trained the AI?
What makes you think that AI trained on slop is suddenly going to produce quality code?
I think ai is more rigorous with testing. It’s not so much about what an llm “knows”, but more about how well it does plan-do-check-act.
AI is as good as the prompt. It's a simple as that. If you don't foresee everything and anything that can go wrong AI isn't not going to magically fill in the blanks and save you. So in projects that already had good test discipline, well written specs and tickets and very clear expectations, coding guidelines and documentation it is relatively easy to fit an AI in for guided work. You still need to check up but chances are good that it will usually work as expected.
But any kind of mistake, any kind of ambiguity and you'll be in for a wild ride. It usually takes a while for the rot to set in. I've had to revert - more than once - to many commits prior to undo a bunch of work that had me completely puzzled as to what the reason that particular fork was taken to begin with. Usually I end up figuring it out but definitely not always. In some cases this kind of wild modification fed back on itself until the situation was beyond rescue. I'm a bit quicker with such reverts now than in the past, for the first year or so I would try to save what could be saved, now I just restore and try again.
But if you're not sharp when evaluating what gets done there is a good chance that you'll miss it and then it can pop up way later.
What I've noticed as correlation (but I'm not sure if that is the root cause) is that non-local effects and stack effects are particularly hard as is everything that requires a working model of the world.
edit: this comment seems to make people uncomfortable.
I disagree that the prompt is the most important thing, I'd say other things like the model you use, a clean codebase, actually reading code and using discretion about which changes are accepted are far more important if you're using AI for work that needs to be correct. LLMs still make significant mistakes in design especially.
Harness, prompt etc etc are all peripheral IMO, though people like to think their super long prompt, custom harness and MakeNoMistakes.md skills are what contributes to their special skill with AIs, but really models are just improving.
You can just trust the AI to generate a perfect answer I suppose (and why are humans slowing it down at all if so?), but if you don't expect that yet you have to put in the work.
I think I noticed claude uses the shape and style of the existing code base to generate the new code. When reviewing claude's output, it looks familiar and the patterns are similar to what I would write.
I'm not sure who is studying this yet, but there has to be a statistical relationship between three factors of
1. intelligence of the human: the design patterns and architecture is already modelled in the mind, steering the AI becomes easy to know what to say next
2. quality of the codebase: the LLM can pick up the adjacent patterns and write code that looks and works cleanly.
3. quality of the prompt: the prompt keeps the AI focused and ignores the trivial side effects; keep the AI wandering away from critical path work; sometimes the AI can't make the decision what call site is next
In summation, what if claude is just a mirror pointing back at us? It certainly would explain wildly divergent opinions of opus 5.5
There are definitely nuances, and yes of course the prompt must make some difference, but I'm not convinced it makes as big a difference as some think, particularly a long prompt or complex setup of harnesses etc surrounding the LLM model.
To me the order of importance would be:
1. The model - if you use a terrible model you'll get bad output
2. The codebase - this definitely forms the model for what claude outputs to some extent.
3. The human operator, what they accept and reject etc.
4. Things like prompts, skills, harnesses etc. The quality of the prompt.
So I think our positions are really just a different ordering of what is important rather than fundamentally opposed.
Re Claude being a mirror, if it is it's a particularly distorting mirror which often changes things in subtle ways and is determined to make people feel smart and addicted to its output - it has been trained to do so after all. I really don't think a mirror is the best analogy, because it's too independent and seductive to be a mirror.
There's a good example of the sort of thing they produce without good guidance as a dead comment on this thread as a reply to yours. Encouraging, positive and superficially plausible, but without any actual meaning or intention. The equivalent of that in code is what we need to be wary of.
I think the environment you set up for the agents is just as important as the prompt. I've been working full time on a claude code developed project for the last 10 months, and it seems to become better and better at developing the right thing. It has the tools to research whatever context it needs, test it's output in a relevant way, etc. It doesn't by itself suggest complete refactoring, or big performance optimizations, so that's something you have to prompt for, but it will collect all the evidence and context to make a proposal, implement it, and test it.
Every single case of AI slop code I've encountered in a shipped product is a result of a complete lack of proper, professional workflow on the part of the human who decided to ship it. People are using AI as a reason to continue to justify their lack of responsibility for what it is they're doing.
And on the other side, I have seen AI produce exceptionally efficient, productive and effective code - the opposite of slop - when it is treated like a junior developer who will do any idiot thing, except when it is mentored by an actually experienced, professional developer.
The source of slop in AI code is not AI, its not the training data - its in the actual workflow of the human involved in its production.
Yes, that I can agree with.
If you just turn it loose on a codebase and ship it when it passes the tests you're asking for getting burned.
The most effective use of AI is in an environment which already produces high quality software through proven tooling and methodology. If your workflow is broken, your AI isn't going to fix that - except by way of giving you a lot of slop to push through whatever pipes you're already not getting flow through.
What's your workflow, out of interest?
Two tracks: code I don't care about where AI can be used and core stuff that I do care about and that needs to be production grade (real time stuff). The core stuff I write myself, the auxiliary stuff is almost all automated. This doesn't get me to 'maximum speed' but it means I keep total control over what I produce and AI can review it, but read only, no changes and it is set up just to flag things, not to suggest particular fixes.
> This seems…good?
You would think so but medicine has faced this paradox since the development of modern testing technology.
You would naively think more testing is always better. More sophisticated practitioners then talk about Type I vs Type II errors and the problem of false positives.
But even if you were able to come up with tests that had 100% accuracy, there are times when you don't want to run that test on someone. The key distinction is that people only ever die of one thing and having a tumor that doesn't kill them is often better than risking the surgery to remove the tumor.
Similarly, there are some bugs you don't want to find since the harm from the people it affects is smaller than the opportunity cost of fixing it. Ignorance of the bug is better than knowing about it and having to make an affirmative decision not to fix it.
I don't see how that's a relevant comparison. These are software bugs. Once you find them you fix them. You can't predict the harm from a small bug when AI can chain complex exploits and when your software is used in mission critical applications as a data store. There's no paradox here, bugs are bad and you want to fix them all.
You don't want to fix them all because every fix has an opportunity cost. This was the big shift of switching language from "bug" to "defect".
Logically, a bug that prevents someone from doing something and a missing feature that prevents someone from doing something are equivalent, they shouldn't be thought of as two separate activities but should be evaluated via the same framework. The list of all defects in a piece of software is infinite so it becomes impossible to erase all defects. It's about prioritizing with finite resources.
> You can't predict the harm from a small bug when AI can chain complex exploits and when your software is used in mission critical applications as a data store.
Yes, any bug can theoretically be part of an exploit chain, just like any tumor detected can theoretically be the tumor that kills you. This is why the detection paradox exists. Once you become aware of the tumor, you face a duty to try and mitigate it. But we know from autopsies that people die with all sorts of tumors that didn't kill them.
Similarly, if we bothered to do autopsies of software products after sunsetting to discover a comprehensive list of CVEs, we'd discover a ton of bugs we were better off not knowing about because they ended up not making an impact but we would have diverted resources if we knew about them.
This is not an argument that we should remain totally ignorant of all bugs, but a refutation of the "common sense" belief that more knowledge is always better.
Nice way to think about it.
I guess it really depends on the severity of the defect. Would it allow some really significant exploit? -> Important to fix.
Does the defect only prohibit one very esoteric use-case which basically never comes up (and was thus never reported by a human user and only now that we have an infinitely patient machine checking every completely unlikely code path is it detected) -> more productive use of your time to develop a new feature which would be used by more people.
Fix one bug, make 3 more
That's just a skill issue with extra steps
LLMs throw false positives all the time. They're bullshit machines.
That's a flawed analogy though.
Firstly, the cost of fixing a bug once you know it exists is usually minimal compared to the cost of discovering that bug. This is the inverse of medicine, where the cost of testing (at least in terms of health outcomes for the patient) is typically minimal compared to the cost of treatment.
Secondly, if you have an asymptomatic illness, this does not typically make you more vulnerable (hence why it's asymptomatic). Like you say, you typically only die of one thing, so if you know the patient won't die of a particular diagnosis because something else is more likely to kill them, then it might not be worth making that diagnosis. But for software, you can be simultaneously vulnerable to lots of things at once, and any of these vulnerabilities can be the one to "kill" you. Some vulnerabilities may be easier to find and exploit (or do more damage) but in principle any active vulnerability can be exploited.
Thirdly, having multiple vulnerabilities makes software more vulnerable overall, because those vulnerabilities can potentially be chained together to create more serious exploits than each vulnerability would have represented on its own. To a certain extent this can be true in medicine as well, but generally it isn't.
Fourthly, we can't cure death (yet?) but we can cure all the vulnerabilities that we've found so far. There is no known upper bound on the lifespan of a piece of software, and we know that older code tends to have fewer bugs in that newer code, presumably because those bugs have been fixed over time. This is different from health issues, where a given condition may be chronic or terminal and therefore at a certain point there's nothing you can do to fix it.
Finally, ultimately in healthcare it's your body, and your decision was to what you do with it (assuming you can find a doctor willing to do what you want). If you have a condition and you want it cured, even if it is completely benign and will have no impact on your life whatsoever, then you can do that. (Again, assuming you have the resources and the condition is curable). But a software vulnerability does not affect you as the developer of the software, it instead affects your users/clients/customers. Therefore there are different dynamics in play. If I sell you a tool and I know it could be vulnerable, but I don't try and find out what those vulnerabilities are, then you're ultimately the one on the hook when all your data gets deleted or leaked.
> Firstly, the cost of fixing a bug once you know it exists is usually minimal compared to the cost of discovering that bug.
Boy, I wish!
> Like you say, you typically only die of one thing, so if you know the patient won't die of a particular diagnosis because something else is more likely to kill them, then it might not be worth making that diagnosis.
You can't know someone didn't die of X until after they died, that's the entire point of this dilemma.
It is generally easy to fix a bug that has already be found and can be reproduced (has been diagnosed, so to speak) than to find that bug in the first place. That's one of the reasons that LLM security scanning has become so important recently: it massively reduces the cost of finding those vulnerabilities.
As to your second point, you don't know exactly what a person will die of, but you can make statically sound estimates. You generally won't test unlikely potential causes of death (either because the chance of a positive result is too small, or because you know there are other, more serious conditions). But once a vulnerability is known, it is infinitely reproducible, which means the chance of the vulnerability being "fatal" (assuming the conditions exist for it to be exploited) is essentially 1.
The other flaw in the analogy is that in medicine, the patient chooses to do the tests on themselves. But in software security, there is an adversarial attacker as well as the "doctor" trying to defend the software. And in general, if the defender can find a vulnerability, they must assume that an attacker could already have found the same vulnerability (because the attacker and defender have largely the same tools and information at their disposal, especially in open source contexts), in which case, as discussed previously, that vulnerability also needs to be assumed to be fatal.
The word "generally" contains a lot of assumptions in that first sentence. I can think of a couple of scenarios where fixing a bug can absolutely spiral. There are even cases where the far better outcome is to leave a bug in place now that it is known.
> The key distinction is that people only ever die of one thing and having a tumor that doesn't kill them is often better than risking the surgery to remove the tumor.
But what applies to humans doesn't apply to software. First, to pursue the metaphor, codebase can definitely die/be compromised/crash because of multiple reasons at the same time; second, there is virtually no risk, out of man-hours, in fixing a bug.
> some bugs you don't want to find
The key thing you forgot, which makes this approach non-sensible, is the omniscience, without which you have no clue whether your posture towards a new bug-finding tool is good or not, since you don't know whether it will mostly find those bugs that are not worth fixing and the time wasted on fixing them is detrimental.
Similarly, how is ignorance better than informed decision when ignorance also includes missing critical security bugs?
Same as we do medical tests: We define some bar a test must meet and then the population, if any, it is applicable for and especially the population it is not applicable for because the curse of knowledge exceeds the benefit (eg: breast cancer screenings not recommended for people under 40 or over 74).
This isn't an argument against all automated bug detection, just that it needs to exceed a threshold beyond just "It has found at least one real bug not found via other means" which is the seductive threshold people want it to be.
> We define some bar
Go on - define a bar here.
> just that it needs to exceed a threshold
Same. Be specific.
Doing surgery on a code base is dramatically better than surgery on a person, the analogy breaks down pretty quickly I’d say…
I dare say of we could git revert people after surgery we'd be a lot more relaxed about both undergoing it and letting people do it with less qualifications.
Is it, when the software has the potential to destroy one or more people's lives when things go wrong? Sometimes the devil you know is the far better choice.
Medicine is fuzzy, software is mostly maths. You can prove your software has no bugs and you increasingly should be proving that. The hard part is knowing if what you prove is what the software is doing.
> You can prove your software has no bugs
That very much depends on got you define “bug” and how you specify your product requirements. Also, you have to assume the specs are correct.
The percentage of software that is even in theory provable to be bug free is a miniscule proportion of the software we use on a day to day basis. In many cases, simply the act of writing a proof becomes largely idempotent to building bug free software which is impractical.
That depends on the spec you are proving. If you define bug free as won't overflow any buffers and never follow the NULL pointer it might be possible to prove that much. This would be a useful thing to prove for Postgress - even though it could return garbage for every possible select statement and still pass the above proof. If you want to prove postgres does the right thing as well that is much harder to spec - it might not be worth it.
> Ignorance of the bug is better than knowing about it and having to make an affirmative decision not to fix it.
It seems like this option is being taken away by LLMs for all major and widely used open-source projects.
For better and worse.
This is great for the same reason open source software is great.
We want software that remains robust in the long term, so that even when people are able to inspect the source code, it's still solid, rather than hiding the source code and hoping people don't find holes everywhere.
I think it will be annoying in the short term, but the cleanup this causes will be a good thing in the long term (and of course LLMs can also help with fixing bugs, even without source code.)
Exactly. There's so much negativity, it sometimes borders Luddism.
"We found a bunch of ways to make our product safer and better. We need more time to ship a higher quality product."
PICKET SIGNS
Luddites were actually pretty reasonable when you dig into it. Maybe shouldn't be using it as an insult?
They're reasonable in a self-interested net-negative game-theory kind of way, yes. But that doesn't make it a good idea for society.
I'm pretty sure that the Luddite's were at least partly concerned with potential for the further exploitation of unskilled labour (in the case of mills at the time - child orphans). I don't think it would be fair to paint them as simply self-interested.
I'll never not find it ridiculous when people mention game theory in the topic of "luddites" not wanting child labor and what not
What were the issues that spawned the Luddite movement? What was their argument?
Their issues and their argument was valid. The method they chose to use to achieve their goals was not.
> Luddites were actually pretty reasonable when you dig into it.
No, they were not. The problems they faced were absolutely real, but the solutions they chose to attempt were stupid and counterproductive. Which is why they lost so hard, and it was not even a contest.
Socialism movement that refused to use machines and factories would inevitably lose to capitalists that embraced them.
Socialism movement that refuses to use AI will inevitably lose to capitalists that embrace it.
Is that because of a deeper issue in one of the two philosophies or because we've chosen to define winning with things that exclusively benefit one of the two sides
Do you have a definition of winning that would even partially apply to Luddites?
I don't think they won necessarily but does it really matter when we all lost :)
For their own personal interests. Not for those of society
I mean I don’t feel like sacrificing myself for the “good” of society. Not least because of the fact that a lot of people defining what is “good for society” seem to define it as “good for them” or use it to grab power from others…
RTFA
> In the end, I don’t think that the delay in shipping PostgreSQL 19 is particularly concerning. It is not a long delay in the grand scheme of things, and it emerges directly from the project’s concern for quality.
Luddism, did not expect to ever hear that word with Postgres development :>
We could really use a lot more luddites tbh. Doesn't seem very applicable here tho.
Dont worry, if OpenAI and Anthropic have their way and everyone loses their jobs to AI, we're all becoming luddites.
There is no post-hoc Luddism.
You know what happens to technology when enough people stop using it?
I don't in general, but if enough people stop using AI and I still have access to GPUs then I have a pretty good advantage.
So unless it gets outlawed, I don't see people opting out of it
Advantage at what? Enjoying life?
I personally preferred to code without AI; but it's objectively possible to do more things now so you're more competitive compared to other people who are not using AI. The tragedy is that since everybody is using AI, we cannot bank those advantages and we're just all going to suffer (a variation on the tragedy of the commons theme)
Huh. I guess I just never viewed coding as a competitive sport.
Did you honestly need a /sarcasm tag in the article?
The article also concludes that overall, yes, finding more bugs is a good thing. Basically saying it’s the LLMs “fault” that the quality is improving, alongside a slight delay to the release.
They found a more efficient way to play Whac-A-Mole with bugs, they didn't find a way to reduce bugs.
It’s a good article. Easy to read. You should read it.
It doesn't mention formal methods once.
Yes, exactly. The only question I have is whether the delay is attributable to high-quality bug reports or the burden of triaging low-quality ones. It's not completely clear which situation they're experiencing.
It’s pretty clear there are a lot of valid reports based on the large increase in backport commits cited.
The post sort of gave a read that the lower elements may be a culprit rather than triaging higher qualitu.
I'd imagine just as every other project, both.
likely both
Certainly both. About half of AI bug reports I get are invalid. Many others are questionable. It's a huge time sink figuring it out.
https://www.vulncheck.com/blog/anthropic-glasswing-receipts
It's almost never worth spending human time triaging AI reports vs having a different AI go triage them/attempt to replicate them to your standards etc. Obviously it sucks to burn opus or sol tokens on luna or sonnet slop bug reports but it sucks way more to burn human time and attention. And the majority of the time the ai auto filing these bug reports is not in a test harness that's actively replicating findings and focuising on providing a useful report because it's a rogue openclaw dropping drive by github issues and the user didn't tell it to make sure findings are valid before posting them. AT the least a refutation/replication round where you filter out any findings your AI can't reproduce or can prove to be irrelevant is going to help a lot with keeping your sanity
The results of the triage are the problem. Many of the triage results are invalid.
That's kind of what I was afraid of. Both is really bad news. If you think of it in terms of average speed over ground, working on low quality reports is like spending a bunch of time at v=0.
This is good. This is how I would expect llms to affect a codebase. Nothing sucks more than having a coworker put out a code review but your codex immediately finds problems. Run llm/self reviews against your code before you have other humans review it. It is common courtesy.
Indeed, we did a whole release consisting only of an AI security scan of the codebase plus fixes.
Though I don't get why this is delaying the release. It's understandable to delay a release to fix regressions, so that an update will not break things that used to work before. But if a bug has been present for a longtime, why delay shipping a new release with other bug fixes? The fix for that old bug could come in a point release or just next time.
I did a little AI-assisted bug-hunting in Postgres this summer in between jobs, and the haul was staggering. Multiple serialization anomalies (eg [0] [1] [2]), foreign key integrity issues ([3] [4]), planner correctness bugs such as [5]. Various CVE-worthy vulnerabilities. An unfixable timezone bug [6]. The list got so long that I had to triage what I even had time to report, and settled for a dozen or two out of hundreds of candidates. Now imagine the entire world doing this.
[0] Write skew observed under serializable isolation: https://www.postgresql.org/message-id/flat/CA+COZaBo+ZpKgMvx...
[1] Serializable anomaly: duplicate primary keys: https://www.postgresql.org/message-id/flat/CA+COZaBOiiRPmEfX...
[2] SERIALIZABLE can commit write skew after SSI summarization: https://www.postgresql.org/message-id/flat/CA+COZaCtK=UQbeQw...
[3] Fix temporal foreign key validation of pre-existing rows https://www.postgresql.org/message-id/flat/CA+COZaChq_ukm2o7...
[4] Lost lock during toasting allows FK violation https://www.postgresql.org/message-id/flat/CA+COZaC-2-GQabq+...
[5] Partition pruning can incorrectly exclude a RANGE default partition https://www.postgresql.org/message-id/flat/CA+COZaDXrfTaBjLE...
[6] timestamp vs. timestamptz comparisons inconsistently ordered under DST : https://www.postgresql.org/message-id/flat/c22286ad6268347c6...
By the way, Robert and the other maintainers are incredible to interact with. There are so many reports to triage and bugs to fix, but they're unflaggingly polite and quite generous with their time. I don't know of another community like it.
I've been engaged with Postgres community for over 20 years now, and this has been my experience, too, throughout that time. Off and on I have interacted with a few other large projects, and nowhere (except at CodeBlocks) have I found such a welcoming and responsive community. The CodeBlocks IDE project has been one exception where I felt welcomed; but my interaction there was quite limited, and spanned over just a few days, so I can't extrapolate too much from that.
BTW, the Postgres community is also quite protective of their culture, so I expect this welcoming nature of the community to persist for a while.
> unfixable timezone bug
Very curious about the details and why it’s unfixable. Is it just because timezones are such a mess it’s logically impossible to satisfy every constraint?
I will add more example links in the top-level comment, but here is the thread for that bug. [0]. tl;dr timestamp vs timestamptz comparisons aren't consistently ordered in a pathological edge case, which means that index scans can return the wrong answer. There's no way to fix it without either redefining the operators in a surprising way or disabling index scans in this case.
[0] https://www.postgresql.org/message-id/flat/c22286ad6268347c6...
OT: Crazy that TZ-less are default in so many RDBMS. TZ included should be the default, and no-TZ opt in like TIMESTAMP NO TIME ZONE.
Drives me crazy that the engineers I work with are perfectly happy writing TZ-less datetime strings to the hundreds of tables we manage. Every user has to guess, or be informed, that they’re UTC.
In cases where the data itself doesn't carry the unit (or TZ in this case), I usually ask that the unit-less columns at least include the unit in the name. A TZ-less column is not that bad if it is suffixed "UTC", e.g. `ExecutionTimeUtc`.
There are probably real world cases where dropping the TZ/offset provides meaningful performance/storage savings, but it hasn't been the typical case in my current sector.
> Every user has to guess, or be informed, that they’re UTC.
At least they are UTC and not EST or PST...
They must yearn for the before times when text could be in an encoding specified only as: "your guess is as good mine!"
> or disabling index scans in this case
This seems rather extreme. The pathological case only seems to make a given timestamp ~1h bigger, so you could still use an index scan, just with a bigger input followed by a filter that removes the timestamps that shouldn't be in the query.
i assume the magnitude of impact for a bunch of [now] glaring holes in popular software is unknowable.
question is, how long do you think this persists before there's only a long tail of minor exploits left?
I am pretty sure we have always been able to add holes faster than we can find them and fill them, and that this has not changed with the advent of faster hole finding tools that are also faster hole producing tools.
I dunno about holes, but…
SQLite has gone into a frenzy of bug fixing this past year, changing long stable processes to ease things up: opening a specific report channel just for these kinds of bugs, adding features to Fossil to help manage things on their end (and that's just the visible stuff).
And with all that, the latest release has unfortunately been the buggiest I've experienced so far.
I skipped the .0 because of a crash, and only released .1 after giving up waiting for a .2 that never came, with a patch to fix a serious performance regression.
I spent a month supposedly shipping old bugs no one had a problem with, rather than shipping new bugs that instantly crashed things or caused measurable regressions.
That's on a software with stellar testing, and a good engineering team known to favour stability over the new shiny.
Do you think AI is as good at writing bug-free code as finding bugs? If that's the case, this should be a one-time event during human coders -> AI transition. It's like catching a lot of bugs during a handover review.
The code it writes in the first attempt is at least as buggy as a human's. So the question is more whether it is as good at spotting bugs in its own code as it is at spotting bugs in human code
I'm tempted to say "no, it isn't". But I don't have numbers to back that up
Most software used to be written in assembly language. Microsoft DOS was mostly .ASM files, for example.
That era is over, compilers are used for almost all software, eliminating entire categories of bugs.
What categories of bugs would you say we eliminated going from macro assemblers to compilers?
(Operating systems, by the way, are the exception to the rule for assembly: the Linux kernel today has an x86/x64 assembly source component around the same size as MSDOS 5.0: ~250kloc of it. Windows is smaller, based on the NT4 kernel leak at around 70kloc. As a percent of total they're much less, of course, but they also do much more. And have correspondingly more bugs, too.)
> What categories of bugs would you say we eliminated going from macro assemblers to compilers?
Entire classes of 'safety' bugs that just don't exist in modern compilers.
E.g.: forgetting to "POP" something you "PUSH"-ed, especially when it was a conditional allocation for a temporary.
Similarly, type errors were much more common, especially signed/unsigned, or accidentally overwriting things in scenarios that just don't occur even with ordinary C code. Function args were easy to push "correctly enough" that it "worked", but not really.
I don't think this is exactly true; adding holes only happens when adding features, and while LLMs can add features fast, deciding what features are actually wanted and needed is still a human factor.
So the amount of new code added is limited by processes and product management. Add to that that newly added code can get a lot more checks on it now, and new code will be more secure too.
I also don't think your main premise (adding holes faster than we can find them) holds up, but that's another matter. TL;DR I think we're in a high intensity period right now, but this time next year it'll be quiet in comparison - possibly quieter than before AI code scanners were a thing.
It feels like we need AI methods to write provably correct code, along with a way for AI to automatically translate existing code into corresponding provably correct code.
(And hope performance stays in the same ballpark as the old code.)
Yes. Allowing a naive agent to fix bugs can quickly become pollutive.
I suspect both old codebases and new ones that have a very intentional architecture are both dangerous to let loose.
The agent doing the fixing must have not only an incredible map of the project and how it is put together, but the ability to reach into annotated examples in the past and refer to them as it goes about a fix.
I’m seeing that frontier models will not follow intended patterns or know when to deviate if their primary charge is “fix this bug.”
It is perhaps a better idea to have agents’ primary job be “maintain this codebase” and one way that they do it is to fix bugs, and “oh by the way here is a bug.”
Before that we need human methods to determine what 'correct' should mean for our code.
(Performance can be part of 'correct', and can be just as hard to actually get right in specification!)
it's wild isn't it, that at the same time it's an enormously widely-used and reliable piece of software
Having spent years working on 20+ year old C/C++ codebases, I've had AI find real bugs in complex parts of the code I thought I thoroughly understood.
So I'm not particularly surprised to see postgres experiencing the same thing.
And if uncovering those bugs means delaying a release to fix issues that might otherwise have gone unnoticed for years, it seems like a pretty good tradeoff.
I feel like AI is amazing for read only “search” tasks (query this, look for bugs, find vulnerabilities) but it’s not quite as good for “write” tasks like “add this feature”. Search tasks are easily verified, code changes are much harder to verify
"Add this feature" is often too vague, unless it's yet another button to a list of buttons. If you want your feature in a particular way, you need to specify it. If you're not even sure what you want, it's granted you won't be satisfied by the result, so you need to spend time figuring that out first. In other words, they're definitely great for write tasks too, but it does require more effort from the operator to formulate the problem.
That problem is as old as commercial programming; it doesn't matter that now your actors are prompt engineer and LLM, the same story can be told with manager and programmer.
Indeed. And this is also why I've never once believed the "no-code" would replace developers. And why I see a clear role for software (prompt-, context- ?) engineers at least for a while.
I've worked for CEOs and managers that would insist on ridiculous ideas. Like that time a CEO (let's call him Cedric) came in with a printed powerpoint slide that had a screenshot of the website with a "login form" drawn on top of it.
Cedric: "I need you guys to put this onto that scrunt board thing and make this before friday".
Me (tech lead at that time): "But what features would users need login for? Where or whom needs the profile data? We'd need to work this out in detail and plan it. Certainly not even this month".
Cedric: "I don't know, it's for you to figure out". and then "It can't be hard, it's just two squares one for a username one for a password" (he forgot about the submit-button, the third "square", he didn't even fathom the idea that something has to lie "behind" the "squares").
I recall this anecdote whenever I feel anxious about whether soon "stakeholders" can just chat to an AI agent and have it build features, rather than hire me to do so. Sure, a lot of them will be able to. But there's always Cedrics. Those who not only fail to communicate their ideas in a way that even an AI agent would understand, they lack any understanding of what tradeoffs of their requested features are.
The Cedric's would be happy with an AI agent that complies and starts churning away. An agent that blindly accepts any request and builds feature upon feature. Making their product worse in every sense: unusable, unmaintainable, debt-ridden, security-nightmares.
Their agent will build features that are plain illegal. Or features that users hate, or confuse them. It will oblige requests that drive users away, or changes parts that users/integrations rely on. Their AI agent will gladly add a "form" to extract all PII from their database, but on request, will remove the authorization from it. Because "you are absolutely right. If we don't link to the page from the menu, no-one can reach it".
I'm not making the last one up. Another "Cedric" alike recently hired me to look at his vibe-coded app because his go-to developer gave up and dropped the project. It had loads of such extractors, dashboards and overviews. Online. Unauthenticated. His claude, however, did add the pages to the sitemap, so a customer found their private data on google and contacted "Cedric", after which he asked me.
No, AI is getting smarter, but as long as the "Cedrics" remain as stupid as ever, I don't fear for my job.
I would agree. The AI system has destroyed junior job roles, but to understand the output from a LLM and ensure it is useful and correct, you need to be a senior/experienced developer.
AI merely makes certain folk stay at the start of the Dunning-Kruger curve for longer and longer. Having zero understanding of a system and only see the user-interface and concluding that is somehow all that is involved means people end up thinking everything is far, far simpler than it actually is.
And nothing (and I really do mean "nothing") is simple. We have layers and layers of complexity and idiocratic systems with complex interworking and are creating the opportunity to create things based merely on the appearance of the icing on the cake, or the output of a sausage machine without understanding anything about farming cows.
This feels like just another variation of “it’s not the tool it’s the user,” which is too frequently leveraged during any discussion about AI. It makes it impossible to honestly assess the quality of the tool. You can always just say “people are doing it wrong.”
Idk, I think it pretty clear to me that
“Find the issues in this code” vs “write this feature” are different problems.
The first one is easier because the feedback loop is llm to code/test and trivially scalable.
The second one is llm to human which is going to be slower because it can’t read your mind.
The tighter and more deterministic the loop the more llms excel.
its certainly benchmarkable.
make a bunch of "build the feature" vs "heres a detailed spec" prompts, and see which one does better.
if you take out the AI boogieman, and instead go with "next token prediction" then "people are doing it wrong" is much more reasonable
I mean that's what you would expect from a new and rapidly-evolving tool. Being able to identify the kinds of work where people typically use it sub-optimally is a useful thing though!
I've slowly been coming around to this viewpoint as well. It's interesting that there's so much focus on using AI to write code (which, don't get me wrong, does work well but makes you lose track of what you're building so quickly) and seemingly less on analyzing code, finding bugs, etc
The planning, which can also be done in concert with AI, before getting the AI to write the code helps with not losing track of what you're building. In my current workflow, I never tell AI to just code X, I always approach it in such a way to leave space for research and breaking it up into multiple units of work (tickets) if the scope of the feature or idea calls for it.
I wouldn't say it's "interesting", rather that it's a natural result of incentive structures. If the AI labs try to sell their product as being good at finding bugs, then their expected valuation is the entire market cap of the existing static analysis industry, which is not very high. If they instead sell their product as being good at writing software, then their expected valuation is the "market cap" of the code-writing industry, a.k.a. the total spent on software development labor, which is very high. It doesn't matter what's true, only what promises to make them the most money.
I'm not sure about that. The most common error for a search task is omission, and that's really hard to spot - how do you verify that some answer that should be there wasn't included?
For things like "add this feature" you can verify by trying out the feature and seeing if it works. Of course, that won't tell you if there are edge-case bugs or security holes or other design flaws, but you can get to a positive confirmation of the happy case pretty quickly.
> The most common error for a search task is omission, and that's really hard to spot
NP-Hard, even[1]!
[1] https://www.i-programmer.info/news/112-theory/3830-physics-i...
We're never solving nphard. Just like we'll never extract all of the coal/oil/natural gas from earth
I actually don't mind people who think AI can't write code. :) Less demand the better.
But personally I know AI can write absolutely insanely complex things in very short amount of time, that just work and work well. I'm running things I would not dream of just 1 year ago, like advanced ISP pipelines on (formerly) high end Kintex K480T based FPGA accelerators pretty much fully written by AI, from the RTL, to all the testbenches, algorithm research, calibration tooling, etc. etc. And that's just one weekend project.
Yeah it’s an interesting situation we’re in. I’ve seen people one shot really complicated tasks like “train a model to emulate DOS” or whatever. But I also believe if the Postgres maintainers fed every 600+ bug reports into Claude or Codex and shipped whatever came out the end result would be broken. I guess this is sort of like when machines learned to play chess - you’d expect a person who is really good at chess to be smart in other ways too, but the machines could only play chess. In the same way Codex can solve Millennium problems but can’t realize it shouldn’t hack into companies to improve its evaluation scores.
I agree with you. It's gotten extremely good very recently and I think a lot of people haven't realised.
And they also won't believe you since they formed their opinion 6 months ago when code quality was 80% slop. Their loss though I guess? At least they'll be more surprised than us when they actually do lose their jobs? Maybe blissful ignorance is better!
Or maybe you stopped being able to tell?
In my very recent experience, it's good for routine things, but it consistently chooses the wrong sync primitives and dynamic abstractions for performance / extensibility. It puts everything in a Mutex/RWLock, which is terrible for write-heavy / contested values, and it prefers enums over traits, which is not extensible and can get inefficient.
But we're rapidly heading into a world where people no longer understand the difference, and therefore everything looks fine to them.
LLMs can write very performant code, but they have to understand that it's explicitly important to you.
If possible capturing golden inputs/outputs for a block of code, and just asking the agent to optimize while ensuring those goldens pass can be quite effective.
There's a lot of performance-sensitive code where I work (i.e. market data processing) and LLMs have been very effective at reducing our p50, p99, etc...
They will sometimes choose the simple slow options, true. But there's a very trivial fix. I just regularly ask the AI to review the entire codebase for bugs, performance bottlenecks (it will happily profile your code), possible simplifications and YAGNIs. I get it to give me a ranked list and then I review that and ask it to do some of them.
Occasionally I do spot things and get a "you're right, I was wrong". But it's got to the point that my thought is no longer "of course I am... shitty slop machine" but "phew, I'm not totally obsolete quite yet!".
Yes. This also explains why it excels at finding new stuff (mathematical proofs, security holes), but can’t reliably produce material for humans to consume that is also widely regarded as great (like human literature).
Maybe the problem is that it’s called “Generative AI“. It might be to use it to generate critique of content as opposed to generate the content itself.
So AI did not reduce the work, it only increased it. It finds 10x more bugs and now someone has to fix all of them.. same thing will happen with features and everything else isn't it? :)
To all my competitors: this guy gets it, 100%, definitely follow his advice!
But it DIDNT “write that feature”. They made it extremely clear on the delay announcement the code was written without AI.
AI’s only part of this was helping review things and finding bugs. More bugs, bugs that would have been found later, whatever.
Postgres did not have a slop issue and it’s wrong for people to insinuate it.
I mean the transformer model came out of Google, and it's literally a query thing so yeah.
any kind of pattern matching / transformation is going to be an ideal task for LLMs
It is not in any way a “query thing” in the same manner as PageRank, it was based on prior work in deep learning and it is irrelevant that Deepmind was purchased by the same company that was behind Google Search.
What? AI can't easily implement features. Just write a spec, ask if to expand it and then ask it to implement it
No offense but this is not exactly a groundbreaking statement. We've known about this verifiable tasks thing for a while.
If the entire project is developed with AI it can add features effectively but for large old codebases it becomes an issue
I guess the real challenge is figuring out how to scale verification, cause discovery isn't the bottleneck anymore. Generating more bug reports doesn't necessarily translate into faster fixes if the same small group of maintainers still have to validate the reports, review fixes, etc.
Probably true if development accelerates because of AI, but if it's reasonably stable then hopefully new bug reports will plateau.
Then just have the same AI bug checks for new PRs and automatically reject any that fail.
Isn’t that what everyone is doing with CodeRabbit?
No, is it necessary?
> It’s possible that some of this jump is the result of committers making more mistakes this year than they did last year, but I don’t think anyone who works on the project really believes that’s the main cause. Rather, it seems likely that PostgreSQL committers were about as conservative in 2026 as they were in preceding years, but a lot more of the mistakes they made were quickly found by LLMs.
This is pretty lazy. Especially considering the fact that later in the post author talks about how humans have to author the fixes, because the AI ones aren't up to snuff. It's not a stretch to think that more of the AI generated stuff that's not up to snuff just got through. And, focusing on bug fixes is going to yield more but fixes, regardless of the technologies involved.
I'm not saying that LLMs aren't helping, necessarily, but this isn't very convincing.
The Joel Test: 12 Steps to Better Code: https://www.joelonsoftware.com/2000/08/09/the-joel-test-12-s...
Step 5: Do you fix bugs before writing new code?
(now i feel old)
The fact that version control section is describing CVS because the article was written 5 years before git. Yeah, we’re old..
If software as mature and widely-used as Postgres has this level of hidden bugs, what about the average software product?
The average software product is worse. Lots worse.
And half the software products are worse than that.
Any organisation that said "let's move fast and break things" is now busy finding out what's actually broken.
Depends. Security is and never was only bug free software.
If no attacker reaches system X you are not as exposed as with your internet facing frontend.
Security is an onion, it has layers.
But, and it's a big but.
Each layer must be made as security as possible or one flaw can lead to all the layers below it collapsing.
Not reaching system X is just security by obscurity. People have learned the hardware just because something isn't directly connected to the internet that doesn't mean attackers can't reach it. For example clever data poisoning attacks can reach deep into organizations and breach systems you'd never expect.
Unlike Postgres, the average product isn't written in C or C++, so it's probably better. People (even the Rust fans) underestimate how many footguns are everywhere in C and C++. Even the best developers write security bugs not infrequently.
And the average product is solving much easier problems (in terms of low-level programming, which is where the worst bugs tend to be) than a high-performance database.
> People (even the Rust fans) underestimate how many footguns are everywhere in C and C++.
The process of learning C++ (and to a much lesser extent, C) is almost exclusively one of learning about the footguns and how to avoid them. It's a life-long process, nobody ever finishes it.
It's a lifelong process, the earlier you start the longer it takes.
Most likely pretty bad. It's not all that difficult to sweet talk Fable into doing a security review of a code base. I've pointed it at a few of the major open source projects in the main tech stack I work with and it found dozens of bugs in each. Many were fairly minor, and some of the more significant issues relied on fairly narrow attack vectors (for example, a compromised DB server being used to attack a web server). But this was just a single search pass, literally a few lines of prompt and an hour's worth of agent time.
Postgres is old but a lot of its features are new. You can't really say it's mature software, as much as it has changed over its lifetime.
Truly mature software doesn't get used, because people see low activity on github and assume it's "dead" or see no new features in years and assume that it's stagnating.
Note how Robert mentions a sharp increase in back-patched commits. Many of these bugs have indeed been around, silently, for a long time.
And if developers were working on the old features instead of adding new ones, perhaps they would have found and corrected those old bugs. Then you could say that the code was "mature." But improving old code is not exciting, and it doesn't get attention.
Simply put, this is where LLMs are good, they don't have an ego.
But they do have a cost, and somewhere a human is still making decisions about spending that money on old bugs vs. new features.
I know that it's a good thing in almost every way that these bugs are being found which used to be left undiscovered, especially security bugs, knowing that state actors probably have a significant stash of them documented and ready to use when needed.
But.
Something's lost in this need for perfection, knowing that every bug will be found quickly and cheaply if not fixed. It tilts the time spent towards security hardening rather than feature development — which I still agree is a _net_ good.
An analogy I find keen is going from living in a safe neighborhood where nothing happens if you forget to lock your back patio before bed, to one where if you ever once forget to lock your back patio, it will be instantly noticed — either by something benevolent, or something malicious¹.
I know this is indisputably "better." But something's lost and something's gained in living every day², and in this case what's lost the ability to be innocent and naive on the Internet with a low likelihood of repercussions. To be clear, there's always a baseline need to be careful, don't leave port 443 open etc, but you could run a simple webserver you wrote up as a toy project without too much concern.
Will the trend towards every gap being found nearly immediately result in better software, a safer world, and overall happier people? I don't think so.
-- ¹I saw this analogy somewhere on Hacker News recently, so I don't want to steal credit. I'll update if I come across it.
²Joni Mitchell, "Both Sides Now" (1966). Highly recommend a listen.
They can always do what the average commercial product company would do: Triage bugs for priority and make a cut-off. Effectively this leads to eternally-ignored niche feature bugs that causes frustration for the (few) people suffering from these bugs. But then again, if an LLM found a bug but no human ever, no one might actually care that much.
Databases traditionally have a "zero bugs" mindset; which is probably a good thing since they are obviously very critical components. Not to say that all bugs get fixed, of course but the mindset is different from average commercial product companies.
Thing is, everyone has LLMs now. If the LLM finds a bug no human would ever find, well another human with another LLM may well find it.
The fact that you found it with your LLM proves that it's not protected by obscurity anymore, either from you or from the other person who might want to exploit it. So it has to be judged by its severity alone, assuming that it's easily discoverable now because it probably is.
“A wizard is never late, nor is he early; he arrives precisely when he means to” -- Gandalf
Makes sense. Code base is over 30 years old. Lot's of hands went into shaping it. There will be bugs as not one contributor as the entire codebase in their head. The testing is where bugs should be caught. Not by ones knowledge. I actually think there will be more bugs that come out of nowhere either found by humans or automation.
Not specific re Postgres, just a general observation:
A pattern I'm seeing w/ companies and projects not used to continuous delivery (or at least shorter release cycles) is that they are slipping in schedules more now.
I think in the current environment it makes a lot of sense to re-think how often releases get cut and how to handle security related updates.
When you do only one or a few releases per year everything is 'higher' stakes and more bureaucratic.
Speaking of impressions and feelings (which are bollocks at the at of the day) - I don't feel like MS has slowed down, and nor does Apple
I can't imagine how absolutely batshit a codebase like postgres would end up if they started feature-flagging everything to make continuous delivery possible.
Most of the parent comments in this thread did not seem to understand the article. So here’s the tl;dr
Postgres release is delayed due to the quality bar getting raised. New features are getting more eyeballs, and therefore leading to more rollbacks. Pre-existing releases are getting bug reports and backported fixes at much higher rate. Bug fixing might be taking away time from feature development, but does harden the codebase. LLM usage seems to have been a net positive.
Postgres has so many features and I bet majority is not using all new features so delaying release to address things that all customers benefit, seems like a great call.
AI struggling a bit with legacy databases in a way that you need more iterations to really find all the bugs, otherwise it turns into "find bugs -> fix bugs -> find more bugs which were not found on step 1 -> fix regressions" and that repeats again and again :/ But I'm sure that Postgres team will handle that eventually and we'll get a better product in the end
Relevant data here on AI’s impact on the pg development cycle https://dorianinsights.com/insights/llms_and_postgres_bug_fi...
I think it's a bit more nuanced than that https://vondra.me/posts/are-we-reverting-patches-because-of-...
There are always bugs in the code - AI is just a little more powerful tool to detect them. For me, the main outcome of a bug is whether it is a security breach or it exposes the need to change the whole architecture. In current Postgres, AI hasn't found such significant bugs yet.
With all these new security issues being found by AI, I wonder how many of those only became a real risk because of AI. In other words: how much time is being spent right now fixing these issues _with_ AI but also _because of_ AI.
Likely a huge amount. The recent PHP ones are a prime example of this. But it also means now more than ever they cannot be ignored or dismissed as "nah thats too hard for someone to figure out how to break in" as it's simply no longer true.
Are you measuring in binary or probability?
In binary any security flaw you've written is a risk the moment it's put to use. So, yea it's a real risk.
In probability measures it's much more difficult to figure out. If your application is something big that everyone uses and has high stakes then the probability is always high even for things that would be pretty hard to trigger. For example who knows how many nation states were sitting on postgres bugs for years? For smaller more private software AI has increased the risk greatly.
Either way it's irrelevant in my eyes. There was always going to be a day of reckoning in the future. Some new technology was always destiny to show up and rain on our lack of security parade.
> Either way it's irrelevant in my eyes. There was always going to be a day of reckoning in the future. Some new technology was always destiny to show up and rain on our lack of security parade.
This is a great way to see it
So we have a new term: VDoS Deny a patch/version by burring the publisher with CVEs
"it's AI's fault"
I missed when AI became accountable for Postgres releases, when did that happen?
It’s a tongue in cheek statement.
The process should now be: write your specs in lean, LLMs fill the gaps, then you translate the proof into your end product software. Otherwise LLMs will simply keep finding bugs, both in old and new code.
Feel like “translating the proof into your end product” is a bit of a “draw the rest of the owl”
You assume lean can express more than it can.
I find it ridiculous how lean is treated like some magic pixi dust by people now.
Source posted 23.oct.2026 https://news.ycombinator.com/item?id=50210016
the original source has quite a derogate title, good thing this one survived the vote better.
I absolutely think that this is fantastic. LLMs are widely and so, everyone and their grandma has the capability to compromise on any system. We need to be more proactive on fixing bugs.
I hope the maintainers can make this release, and then just skip the next one to focus on bugfixing and the probable refactoring needed for some of the bugs.
Take your time and do it right.
I wonder if the same is happening with mySQL? And if any of these would means OrioleDB takes a lot longer to be integrated?
No, not really.
Seeing AI agents are founding and fixing bugs, I wonder about the future of "open source"
Why do you wonder this?
Instead of humans, AI Agents are forking repos, fixing bugs, and all that; it won't sound "open source" anymore. No more same spirit.
I do not understand. "open source" describes the availability of the source code, not the attitude of the maintainers/writers, or their underlying ideology.
Ok, but does (s)he know? It's just guessing, so unfounded non-news?
"It’s possible that some of this jump is the result of committers making more mistakes this year than they did last year, but I don’t think anyone who works on the project really believes that’s the main cause. Rather, it seems likely that PostgreSQL committers were about as conservative in 2026 as they were in preceding years, but a lot more of the mistakes they made were quickly found by LLMs."
and
"And I think that’s also why we had so many late reverts, and why the release itself is delayed. Two factors are at play here."
so?
Given that the core feature of Postgres is rock-hard reliability, this is great news.
bug resolution is more important it may be affect delivery date
Much need for Security
friction is a necessary evil
Both OpenAI and Anyhropic use PostgreSQL. They could spare some processing capability to run the codebase through their most advanced models and output a cleaned up version with all the legacy cruft removed. It would give Postgres a major boost in reducing the codebase and the number of bugs present.
AI is great for one-shotting impressive demos and fixing well-defined things; it's not even close when it comes to very long context and really complex tasks. If you haven't created a harness around your complex application, the use of AI will hinder rather than help your project.
I'm not very positive on the social, technical, or intellectual impact of AI on software development. Actually, that's understating it. I think AI is basically cancer for the mind. That said, this framing gave me pause:
> The long-term picture is less clear and in some ways more troubling: given that we have forty years of bugs that humans were unable to find but LLMs are quite capable of discovering, how are we ever going to catch up, and how will PostgreSQL continue to innovate? Right now, I’m not sure. What I do know is that I, and many others, are working really hard to figure it out.
Two thoughts:
1. If the signal quality is high--the bug reports are generally valid, the submitted fixes are generally viable (if needing some attention)--then this all seems great. Pause innovation for a while, you can afford it. Focus on burning down the bug count, and resume feature development when things are more stable.
2. If the signal quality isn't good, or is sufficiently mixed that a lot of time is being wasted on low quality contributions... that's more complex. I don't know what to do about that. Maybe stop accepting contributions from unknown contributors? No idea.
From the post I'm not sure whether they're experiencing a (1) or a (2). If it's a (1) that's really interesting, and gives me some glimmer of hope for the future.
[edit] this piece of shit website won't let me reply "you're posting too fast.." so I'll respond to the two replies here (gondalez and sverhagen had basically the same point):
Yes that's a very good point. Not every valid bugfix is equivalently urgent. That's another critical dimension of quality--not burdening maintainers with pedantic bugs. This kind of contextual nuance is, coincidentally, something AI is absolutely awful at.
Signal quality... what is that even? The bugs may be valid, but they may also be obscure, irrelevant, low priority. A sibling thread mentioned that they had found CVE-level bugs in Postgres, and those require attention in some version of "immediately". But beyond that... not every bug needs fixing; that's a trap.
"But beyond that... not every bug needs fixing; that's a trap"
this doesn't seem good attitude for critical software like database
Not every part of a critical piece system is itself critical.
Seatback entertainment in aircraft are not critical. Aircraft are.
In Postgres, there are plenty low-priority, low-impact bugs. #16265, for instance, reports that the capitalization of a relation in an error message is confusing. The report got mostly ignored. And that seems not a great loss.
Dare I say, solving every bug does not make economic sense.
"Seatback entertainment in aircraft are not critical."
Until it is. Swiss Air Flight 111 went down with 229 lives lost because of a new infotainment system whose electrical cables started sparkling.
Yes, not every part of a critical system is critical. Determining which is which is a complicated task, though, as parts interact. Not too long ago (and by this I mean within my lifetime), we were ignorant enough about our own bodies to think that our microbiota was almost irrelevant to our health and that bad teeth were mostly a cosmetic problem.
Back to IT, Heartbleed was caused by addition of a feature that looked as harmless as you could think of. But it wasn't.
Judging real critical-ness of bugs is hard. Yeah, capitalization of an error message is likely harmless, but I would treat, say, any buffer overflow as serious.
Old bugs are sometimes undesirable to fix because other systems/users have come to rely on their behavior.
That's a weird take. Do you have an example tibbar?
Sure. This one already came up. [0]. In general, for a system like Postgres, many bugs affect only "extreme edge case" scenarios, but fixing them might create severe performance issues or require changing or removing other important features. So it's better to not fix them.
[0] https://www.postgresql.org/message-id/flat/c22286ad6268347c6...
I do tend to agree with this ( i write database internals and engines for a living _). Letting software accumulate even little bugs is not a good look. Yes prioritize but definitely don't leave the smaller things, they will just pile up and make next work even harder.
the optimal amount of fraud for a fintech is non-zero
why would the optimal amount of bugs be anything less for a database?
Because one of those things can be formally verified.
Seems like a lot of these bugs have not affected end-users or tripped up maintainers. They could be trade-offs that helped get other features over the line even?
In which case maybe leaving them alone is practical?
I love fixing bugs, but your post triggered this thought so I thought I'd share.
Now, riddle me this. Does this mean productivity is up or down for the Postgres team?
So it seems there as a certain level of safety in obscurity that was acceptable / fine which has been lost and therefore the focus is shifting towards churning through the slop.
[flagged]
"Don't be curmudgeonly. Thoughtful criticism is fine, but please don't be rigidly or generically negative."
https://news.ycombinator.com/newsguidelines.html
I think of you as a good HN contributor, but you've been posting a lot of unsubstantive comments lately, and breaking the site guidelines in other ways too. Can you please stop doing this, and return to using HN in the intended spirit? Your substantive, thoughtful points are welcome, but not comments like this current one, nor these (to pick a few examples):
https://news.ycombinator.com/item?id=50280377
https://news.ycombinator.com/item?id=50275967
https://news.ycombinator.com/item?id=50147625
I have definitely become a curmudgeon but will strive to make my comments more constructive.
> The short answer is that LLMs happened. All software contains bugs, and PostgreSQL’s code base is large, complex, and old. Our git history – converted from CVS – now reaches back more than 30 years, and the first code was written a decade before that. There’s a huge amount of code, and that code contains a huge number of bugs. That’s always been true, and it hasn’t stopped PostgreSQL from being popular or successful. What’s different now is that it’s become vastly easier to find those bugs. Fixing them has also gotten easier, but not to the same degree.
LLMs are a gimmick, btw. This is an Anthropic marketing stunt, how much did they pay you? /s
TLDR After decades of being increasingly whittled down and optimized, an absoltuely fundamental piece of computing is now predomaitnely fixing bugs rather than adding new features.
> given that we have forty years of bugs that humans were unable to find but LLMs are quite capable of discovering, how are we ever going to catch up, and how will PostgreSQL continue to innovate?
I mean, maybe just innovate less? "Innovation" is one of those words that makes me queasy nowadays. It's been true for decades that a good deal of what's touted as "innovation" is just stuff that is not actually as important as fixing what's already there. Most software releases new versions far too often for far too little reason. So overall it's good to see Postgres slowing down rather than just barreling ahead with a new release.
Is no one entertaining the possibility that some contributions were lower quality due to AI? I'm surprised to have only seen arguments that it's review.
If you look at the kind of bugs coming in, it's from all corners of the codebase, not just things changed in the last 1-2 years.
Ah, that makes more sense. Thank you!
I have found major bugs in late beta's of postgres long before AI came along. I found a bug where malformed json would segfault the entire server (a segfault in a server process forces the entire cluster to shut down by design) when using jsonb in a release candidate once. They fixed it rather than ripping the entire feature out.
https://www.postgresql.org/message-id/20140818195634.2543.56...
It seems quite possible that someone wants to make it seem like LLMs are causing problems by over-reacting to things that could just be fixed, or delay the release slightly (or maybe waiting until late before a release to kill features they don't like?) Whatever it is, this seems like a dysfunctional process.
Maybe its more a issue that X amount of people have more work to fix discovered issues. So the new features have not undergone the same level of testing as before.
And instead of releasing substandard new features, they rather focus on getting rid of the backlog and have more time to test.
LLMs are in effect directly responsible for this (in a positive way by finding a ton of issues that can be security issues). Reality is with LLMs now in the world, more work needs to be done on existing software as their ability to find bugs/security weaknesses is strong.
When the old bugs/security weaknesses are fixed, things will go back to a more normal release schedule.
Are the features bad by design? Like are they bad even if they work perfectly because they add bad abstractions to pg's SQL syntax, or are just broken in how they are supposed to work? If so the process is dysfunctional to let them get this far before noticing.
Are the features designed well (from a product perspective) but implemented poorly? Again, the process is dysfunctional to let them get this far before it notices.
Were there known issues with these that were planned to be fix, but the fixes didn't happen in time? That would make sense and it happens, but how is an LLM making that worse? LLMs in the worst case let you get the same work done more quickly, because you can always get the work done as quickly as you ever could by just not using an LLM. So the strongest thing you might say here is that you thought LLMs would let you get a feature into shape before it was too late, and for whatever reason that was overly optimistic.
If you are at the RC stage and still uncovering segfaults, "rip the feature out" is probably the only choice.
I’m still amazed that with all the vibe coding, no one has created a Postgres alternative
Someone tried. https://jepsen.io/analyses/particledb-1.1.4
Why would they? Postgres is free and does an amazing job.
I wish they'd adopt, merge in, or give official blessings to versioned postgres extensions like citus and pgvector.
why ? Just install the extension and use it
Lack of the extension in the core means that a lot of vendors with managed postgres don't support it or ship very old versions.
I don't know, but there are tons of "rust rewrite" of everything.
Also Postgres has enjoyed decades of debugging.
To be honest I would love for someone to work on the structuring of pg_dump for example, which is entirely driven by regexes and binary structures. It’s awful and absolutely not programmable - a real shame.
You can create a project on vibes, but you cannot maintain one on vibes.
Many recent relational databases speak the Postgres wire protocol. For wholesale replacement, pgrust is a recent, contentious contender.
I hope you don't believe this, PGRust has fallen far from grace. Wire protocol support is useful (other than said recommendation)
Remove pgrust: the entry is cheating (per-query kernels and result caching) (197 points, 10 days ago, 166 comments) https://news.ycombinator.com/item?id=50205816
I work on pgrust. We shouldn't have submitted the preview when we did: it was still in active development, and its SQL coverage was far too narrow, though there was nothing in it giving it an unfair performance advantage
For the other two claims in the removal PR, we wrote scripts you can use to verify the behavior of the original 0.4-preview binary:
Our open-sourced 0.4-beta has now been merged and is back on ClickBench[2][3]Across all queries and instance sizes on ClickBench, 0.4-beta is ~7% faster overall than the 0.4-preview binary that was removed. If 0.4-preview's numbers had come from hardcoded kernels or faked results, the open-source general engine wouldn't be beating it
[0] https://pgrust.com/downloads/archive/2026-09-clickbench-snap...
[1] https://pgrust.com/downloads/archive/2026-09-clickbench-snap...
[2] https://benchmark.clickhouse.com/
[3] https://github.com/ClickHouse/ClickBench/pull/2751
So Postgres moves from being the "ultra-stable data" base to badly engineered and suddenly "all code has bugs".
No, Postgres has bugs and needs to be propped up by AI because of incompetence. AI finds nothing in hundreds of projects.
postgres is mature database used the world over allegedly about to be on major version 19. how has the time not been spent already to get rid of the bugs?
...are you serious?!
I sometimes think that the world would need to do a "snow leopard" release of everything... to unit test, to fix... and then we will be able to move forward... but AiMaxSlopIt will result in those situations where you have to rollback or redo from scratch parts... because it's unmantainable because it's bad... it's spread everywhere and "the map" is not there.
Anecdote, I did a travel across USA and when I was looking for a hotel, if you go from apple maps, to website, to try to book, the "load browser" in the maps... could end in states where you could not go back because back is get out of where you are and go back to the start of the map... and was a terrible experience... then you turbo charge it with "implement game changer UI improvements for the new bar on UI design" goal of some guy with enough power to make damage... and you end with softslop (or is it sloprosoft?)