Welcome to the strip mining era of open source security

Open source software is in for a rough 2026 summer. If you’re an Open Source maintainer, there’s something afoot you should already know about. If you’re an OSS user, you should be aware of it as it’ll explain some behavior around you that might otherwise seem odd.

TL;DR: High volume, LLM-powered scanning for security vulnerabilities is going to uncover lots of security issues in anything with public source code.

This all started a few months ago

Historically, Metabase averaged 10 submissions per month to our security@metabase.com, most of which were trivial or not actually vulnerabilities. Many were false positives from scanning tools, and we spent most of our time explaining to the reporter that what they found wasn’t actually a problem.

At the turn of the year, things changed. Starting in January, we’ve been averaging 10 submissions per week, and many of these are legit. Most are not serious, and we’ve quietly fixed them, thanked the researcher, and went our merry way. However, it was a step change in both volume and quality of reporting. These come from a wide variety of locations and people, and sometimes, but not always, are looking for bug bounties. More often than not, the reports are in markdown, and read like they’re LLM generated. Others are seeing this as well.

It doesn’t take too insightful an eye to realize we’re seeing a remarkable improvement in automated code scanning. We’ve since tried out a few vendors in the space, and what do you know — more (thankfully minor) issues found. There’s no one vendor or model that’s the root cause. While we originally thought it could be Claude Security, that was only announced in February, after things had already picked up. And OpenAI is also getting into the game. Does this mean there’s another wave coming after everyone gets access to these? Likely. But regardless of specific foundational models, this is just a consequence of coding agents in general getting better at scanning codebases for flaws.

Historically, we tended to get two styles of vulnerability research:

Superficial scanners run in bulk: Running an OWASP scanner or other out-of-the-box vuln scanner. These tended to be mostly false positives.

Motivated deep digging: This was typically a serious user paying researchers, often with a knowledge of the application area or framework deeply and knowing where to probe. These tended to find a cluster of similar issues, often related to the style or speciality of the researcher.

Vulnerabilities are now being strip mined

If your code is available, and someone is willing to spend tokens, they can scan the code in bulk. As coding agents get better at understanding code bases (and as a sane person — don’t take the other side of that bet), we’ll be getting layer after layer of progressively deeper vulnerabilities uncovered.

The Bright side with some dark implications

Now, if there’s any bright side to all of this, it’s that so far, there seems to be an alignment of incentives between ethical research and open core companies. If you’re a budding security researcher, the play is:

1) Wrap up claude/codex/etc in a bunch of skills and turn that into a SaaS offering. 2) Bulk scan commercial open source repositories 3) Send out every finding with a footer advertising your automated scanning SaaS service to the Commercial OSS company and any big users you can scrape from their website.

Needless to say, there are now approximately 1000 of these SaaS offerings, so it’s a competitive game, but thankfully, there is a pro-social path that pays. With non-commercial open source, it’s a bit less lucrative - you’re stuck mining for bounties that users of the OSS project offer. I’m not sure I want to know what’s happening in the unethical side of security research these days.

What this means for OSS maintainers

Now, in the Long Run, this is great. We have a lot of third parties burning tokens to help you find any possible exploitable flaws in your software. And once they’re fixed, any software you’re running is more secure and less likely to have issues. These scanners will proliferate and eventually probably make their way into your CI workflows.

In the short term, it’s gonna be rough.

To start with, you should assume that any vulnerability that was disclosed to you is trivially discoverable now, regardless of how buried it was in your code. While some researchers are doing novel work, the majority of this will be low-ish effort bulk code scans. If one person running Claude Code on your codebase found it, you can expect someone else using e.g., Codex to find it soon enough.

What this means is that even if you have an agreement with the researcher to not make it public before you have had a chance to fix the problem, you need to treat the vulnerability as fundamentally “out in the wild”. Did you have other plans for the weekend? Or a long term project you’re prioritizing? That’s nice, you have a new plan — fix every vulnerability that comes in NOW.

Closed source developers get to find and fix issues on their own schedule, and at least in theory are able to get ahead of third party researchers. If your code is available, you’re going to be living in reactive mode for a while.

Net-net, it’s a lot of short term pain you’ll experience on the OSS path that just isn’t there if you’re closed source. And to add insult to injury - we’ll all get to a similar state. You can make the case that historically, being open source would get you attention from high end security researchers, and in exchange for them getting attribution and/or bounties, the net result was that you could expect OSS to be more secure than if it had been released as closed source. With coding agents being the main driver, this advantage largely dissipates. Whether you’re spending the tokens or someone else is, all classes of vulnerabilities that can be automatically scanned will be easily found.

So it’s understandable that Cal.com is going closed source as a result, and will likely be joined by other commercial operations who a) have customers they care about and b) don’t want to devote the next couple of months to chasing down every publicly findable security issue.

It also bears stating that if you’re a non-commercial OSS project, you’re now getting even more pressure from LLMs. Commercial operations (if they’re doing well) have people getting paid to deal with security issues at 4am on a Saturday. If you’re just running a big OSS project out of the goodness of your heart or you have a promising but not yet At-Scale commercial offering, I’m sorry. Side note: for unknown reasons, this year, Metabase seemed to get notice of serious vulnerabilities either on Friday Nights or holiday weekends. The only upside is that we’ve been able to block, patch and deploy fixes before our customers get back into the office on Monday.

What should you be doing?

What does this mean for you, dear reader?

If you make software for a living, open or closed source, you’re in for a fun summer - expect to have layer after layer of your security issues uncovered. Try to run as many trials of scanning products as you can and aggressively prompt your own coding agents.

If you don’t already have frequent security patches, start. Get your users used to upgrading frequently. If you can make your system easier to upgrade/patch, you won’t regret it.

Fix everything you get your hands on. If you’re smart, you’ve architected your system to have multiple layers of protection, but you can’t rely on any given layer right now. So any opening, no matter how small, should be fixed immediately so it can’t be chained into a larger attack.

If you’re selling a closed source product, you have a new security threat: code leakage. You should assume that any compromise of your source code will unearth a large number of vulnerabilities.

What should you do if you’re using Open Source Software?

Short answer: Treat every OSS dependency as if a new vulnerability will be disclosed against it this quarter, and build your patching, monitoring, and access controls around that assumption.

The five practices that matter most:

1) You should expect that you’ll need to upgrade a lot more frequently, and budget for that ahead of time. 2) Monitor and pin all of your OSS dependencies. Get used to upgrading them ALL THE TIME. 3) Practice Defense-in-Depth and do your best to create layers of separation. They will be tested. 4) Level up your logging and observability game, and make sure someone is watching the logs. 5) Exercise the principles of least privilege, and make sure any credentials used by any software has only the bare minimum privileges.

Really, just do all of the things you should have been doing all along, just at a much more aggressive level. Similar to how the proliferation of drive-by network scanning means that regardless of how small an operation you are, you can expect port scans, you can expect bulk attacks on any OSS you use no matter how small you or the project are.

In closing, all of this will be discovering bugs currently in the code. After the initial short term pain, both the code that currently exists and the code that will be written will be more secure.

It’s just going to suck getting there.