When you write your blog post with claude, I start hitting these phrases and then start only skimming and looking for them, and stop reading what you have to write.
> When you write your blog post with claude, I start hitting these phrases and then start only skimming and looking for them, and stop reading what you have to write.
I've spotted a few memes about this and can admit to doing the same..
I'd rather skip, personally. I just don't have to read things these days, and I see enough Claude/GPT prose at work, it makes me resent seeing it elsewhere.
I have a backlog of books written by humans I'd much rather spend my time on.
how does something like this pass code and design reviews. is this code written by ai (the buggy code). Not only was the key that signs the cookie inside the cookie it self but it's using pickle which even model weights are moved away from. I'm a little flabbergasted at the noob nature of this root cause.
> how does something like this pass code and design reviews
when the "give a shit quotient" of the employee moves towards "highly engaged with the organization and its mission" to "this is just a job, I'm gonna rest and vest"
I think you're being unreasonably uncharitable. Any and all software contains bugs, and I can assure you the piece of code you wrote has bugs. My code has bugs. Google's code has bugs. iOS has bugs. rocket launching software has bugs.
We can put in systems in place to catch bugs. And that's the system that's working here.
Now to the second part of the question -
How did they not catch it before? If it were "easy", it would not have been worth 90k, and hindsight is 20/20 - once you know where to look for, it is all very very obvious.
Do you have any advice for aspiring bug-bounty hunters? Do you have some method for being able to find vulnerabilities like this, or lots of past experience?
> Do you have any advice for aspiring bug-bounty hunters?
We're probably at the end of the era of these kinds of big payouts. AI tools will drive the going rates to approximately the costs of the tokens. Bounties exist to make it nominally more economical for researchers to report exploits than it is to resell them.
That's really not why bounties exist, though people on both sides of the bounty systems claim it is. If it were broadly the case that bounties exist to offset black market sales, you wouldn't pay bounties for vulnerabilities with no market value. This vulnerability is probably a good example of something with a sub-$1000 value on any non-bounty market.
In reality, the modal benefit accruing to a company running a bounty program on receipt of a new serious vulnerability is simply engineering prioritization. You get bug reports from users when you have reliability and UX bugs, but you don't get them for security flaws, because attackers generally have to force vulnerabilities into revealing themselves --- normal users almost never run into them. Bug bounty programs fill that gap.
If you’re saying that bug bounties exist for a reason according to both the program runners and the contributors then how is that not the reason they exist?
The people paying and the people working are all in agreement that it exists for that reason.
What part of my logic do you not follow? The way you've posed this question, the simplest response would require me to write that whole comment over again.
Additionally, part of my businesses remediation services include helping companies set up bug bounty programs. Giving researchers an incentive to report bugs for payouts instead of selling or exploiting them for profits is always the intended goal.
It's the product of direct interviews with bounty people, FAANG/MAG7 blue teamers, and black market participants. If you want more, there's also an interview online that we did with Mark Dowd, who is one of the best known CNE vendors to the NATO IC. You can just Google [SCW Mark Dowd].
I'm happy to go into more detail, but, like I said, you didn't really respond to my comment in any way that I could expand on. Feel free to disagree with me in some kind of specific way.
This really is simple: if you work in this space, you already know that there's essentially no black market at all for serverside web bugs. But there's a $90k bounty for this one, and 5-figure bounties for others. Reconcile those two facts.
Well thats nice and Im happy for you but internet points don’t really mean much here…
What a vulnerability sells for and how much damage someone could do exploiting it are two different things. Someone capable of finding a serious vulnerability is probably capable of exploiting it themselves. They don’t need a buyer.
A company paying big bucks isn’t a contradiction. Its incentivizing disclosure over exploitation which could cost them considerably more. Engineering prioritization is… a minor part of it informed by the ease of exploitation, damage it could do (in the case of an RCE or publicly available keys as in this case quite a bit) etc.
I don’t see why you think these things are mutualy exclusive.
“Someone capable of finding a serious vulnerability is probably capable of exploiting it themselves. They don’t need a buyer.”
I’m responding to this. I would assume the list of people able to find a vulnerability is much much greater than the number of people willing to do a one off heist with it.
most people who care about legality don't work in that field either. whoever gets into this probably starts by being ok with doing shady stuff, I imagine.
Many start in the military and eventually move to private sector. However, an ever growing percentage just straight up go to college to learn the skills.
Cyber companies are in the fortune 100 now, and it's not like it used to be back when you actually had to be interested in it and dedicate your life to learning the skillz from shady forums. Now it's just a job to most practitioners.
Yes, but also there is also a huge value to individual contributors and their careers. I broke into the field after landing a few relatively high profile CVE acknowledgements for big companies. Responsible disclosure is good for your career.
On the other hand, if the value of that drops you can expect to see some folks start taking the wrong path.
there are enough people operating in countries that would not charge them for hacking an American company for it to matter, $90k can be a life-changing sum
And before it ends, people with existing reputation can milk the last drops, as mostly their reports gets processed from the thousands of others. Unfortunate status quo.
Bold choice using the noun "I" repeatedly instead of "Claude" - you didn't figure this out, Claude did, and then Claude very obviously wrote the blog post.
Oh, maybe that's why "Claude" was substituted with "I", because Claude wrote this blog post from a first person perspective.
It's hard to get a $90k bounty from Google, even with Claude in hand, so we shouldn't be dismissive of that, but it is pretty annoying to read write-ups written entirely in LLM voice.
An llm is a tool, I had access to Claude too but I didn’t find the bug. The author found it.
Research taste, among other things are still abilities that differentiate among humans.
In fact, I expect the elimination of other limiting factors by AI to lead to those becoming more important. By the same mechanism that make ugly hedge fund managers more performant.
The problem then is that composition of the top performers will change. Which is problematic since being a top performer ( ‘artist’, ‘researcher’, ‘writer’,..) is often an identity which ‘we often create to protect ourselves: the strong one who never breaks, the intelligent one who is always certain... The mask that once protected you can eventually become the cage that prevents you from changing’
Correct. The headline is incorrect with both the quoting of AI as well as the claim it isn't a tool.
You can hardly blame anyone for avoiding a clickthrough into an auth-walled, user-hostile site to read the point the headline made in an attempt to "gotcha"
I bet, I could take "Trading places" style, anyone off the street, give them a 100 "Secret" word list and they could spend 6 months discovering all sorts of things.
I agree effort and prior knowledge are required, but a few magic words is like 80% of the effort.
This is the typical "ugh that's just rsync" when Dropbox was announced, but for a new LLM age.
_If it's so easy, do it_. Don't post about it. Do it. Fix these bugs at scale. Charge people money for it once you've built up a reputation. Get your own bug bounties.
If you're going to make this argument, at least say GLM 5.3 instead of Claude, Claude will refuse to touch anything remotely close to this unless you're in the super special club.
handing clients access to an LLM gets them beginner level code, easily. same with beginner level exploits. an RCE against google is not a "beginner level exploit". If you've seen AI generated code, I can tell you that AI generated security findings also have that "beginner" smell. an expert who knows what they're doing has to drive them to look at the right places, and use the experts preferred method.
Took about 3 sentences for me to be as certain as I've ever been about anything I've ever been in my entire life that this blog post was authored by an LLM, though truthfully it's impossible to know for sure if it was a Claude model or a distill.
Tbf as someone with an infosec background, I would have probably written just about the same.
Pickle payload is a red flag precisely because pickle is not a safe serialisation format format when taken as untrusted input, here the input was trusted but it's still giving someone en easy escalation path (exactly what happened)
And most people would assume the whole cookie payload would be encrypted if they saw just a binary blob, and not realise you can fully read it if it's just signed
But, yes, the post was probably edited by an LLM, as is pretty much everything else you read online these days.
Nice catch, but god terrible write-up. Please take the time to write it yourself even if you think you will do worse in explaining it than the LLM. It’s so off-putting to read that. Share your prompt or raw article notes even, it will much enjoyable to read than this
If this were Microsoft, they would have received a coupon for 10% off a Thanksgiving turkey, only redeemable in 2030 and only in the month of March, must be purchased with cash, non-transferrable.
Sorry for the meta, but the discourse in many of the comments is nuts to me.
"Claude did this, not you" - does that mean that humans have no agency at all when they use an LLM? Is that the argument?
Maybe the norm should be to credit the LLM and harness used, like "I used ___ to help me find an RCE", but I can see arguments not to do that in some cases.
edit: The giant thread with this now got flagged, so some users may not see it.
If we're doing this standard, then let's do it for everything. Claude coded that, not you. Claude built that website, not you. Might make the industry less exhausting tbh.
People who believe AI is an abstraction layer on top of compilers, are in that middle ground where they know computer science but are far from an expert. Which makes them very susceptible to the Dunning–Kruger effect. Throw in AI sycophantcy and you've got yourself takes like the one above.
Ha.. AI in the form of LLM is not a very good non-deterministic abstraction layer on top of deterministic compilers.
Yet, they are being used with endless surprise as non-deterministic layers of abstraction on top of deterministic compilers.
My point is to use a technology for what it is capable for (where capabilities are discovered weekly and hard to keep up with), not trying to get it to do everything.
Bringing a deterministic lens to non-deterministic software has its dangers too. Non-deterministic software seems to be its own category of software, increasingly capable of non-deterministic approaches that deterministic compilers software couldn’t attempt.
Maybe that’s less compilers and more different type of scaffolding of vectors.
I’m all for skepticism, the issue with cynicism is capabilities often pass those perspectives by sometimes too quickly.
Yeah some people come off as envious they didn't have Claude try it first.
But it needs a disclosure if Claude was used: "The cookie is what caught my eye" - if that was actually Claude's eye then yeah, Simon is claiming to have a keen eye that he in fact does not.
"Claude found a $90K RCE" would have been just as interesting if not more.
Nmap doesn't write its own blog posts. That's not where I draw the line, but the point isn't at all about "drawing the line." (In other words, don't get caught up in where the line is. There is one, but we're not sitting in some gray area, we've blown past it by hundreds of kilometers by now.)
If I post an image and I say, "I drew this", and someone says, "no you didn't. That is clearly output from Qwen Image 2.1", can I reply, "Qwen Image 2.1 is just a tool, like Photoshop"? I mean, I can, but if I do this, I am missing the point. The point is that I didn't draw anything. I didn't draw. I didn't draw digitally. I didn't draw analog. I didn't draw with my mind. I typed a prompt. This is as close to drawing as photography is to drawing.
So did this person actually find a vulnerability on their own? Hell if I know, but I take issue with LLM-authored blog posts; I don't trust people who post LLM-authored blog posts as if they were written by themselves to be honest about the nature of their work.
If you really are still doing work and understanding and just using LLMs as a tool, show your understanding in your blog post that you are communicating to the world, instead of having Claude or GPT show its understanding. I don't really care whether you used Claude and didn't list it, if it's clear as day you understand what you were doing with it.
I'm dying on this hill. Until HN bans this sort of blog post or bans all of us complaining, these comments are going to be very, very boring. And I am stubborn for longer than most people can stand to read the same comments. Believe me.
That strongly depends on what you consider drawing, right? Sure you didn’t dip a pencil into an inkwell and caused stains to appear on a paper. But you did something that caused stains to appear on the image. You already seem to accept Photoshop as a way for a human to do that. Why not Qwen?
We can discuss the quality of the offering. But that’s a different discussion
Because it's not drawing. It's not even drawing in the abstract: nobody drew anything. There are no brush strokes. You can watch the steps diffusion models take and at no point is there anything that resembles "drawing".
Drawing with Photoshop absolutely resembles drawing. Photobashing with Photoshop does not. Photography also isn't drawing. These things are all distinct. That's the actual point I was trying to get at here by invoking Photoshop. Image models differ from text in that text models do resemble something closer to what people are actually doing, but you just have to accept that analogies have limitations here. Just like you can draw or not draw with Photoshop, you can use Claude in ways where you're really driving it like a tool to do work, and you can also use Claude in a "fix the bug" sense where you're doing absolutely nothing other than paying $200/mo and typing occasionally.
But whereas Photoshop introduces nuance to what it means to draw or make art by introducing different levels of involvement versus remixing and etc., Claude and Stable Diffusion utterly decimate nuance in a thermonuclear blast that plunges us into a nuance Nuclear Winter, by offering for perhaps the first time a way for the computer to do nearly all of the generalized "thinking" while you sit on your ass and argue on HN. (Reminder for everyone to check on their agents.)
Trust me, I use LLMs, I get the idea. I can "vibe code". On my workbench is a laptop running an entire desktop system including an application suite that I really didn't write. It would be cool to go online and say "Hey guys check out this cool desktop environment I wrote!" but I won't because I really didn't. It was just an experiment. Did I put in effort? Yes. Was it coding? Nope, it was closer to management, though I did write some code. The experiment was some mix of failure and success, but what I would never do is say that I wrote it. I may some day blog about the experience and the pros and cons and how I feel about the resulting code, but for right now I feel the Internet is full of enough slop and most people who I respect are thoroughly tired of hearing about LLMs all day (I am too at this point, though I am not saying I find none of it interesting.)
Do I think all LLM use absolutely must be disclosed? Actually no, and especially right now I understand why some people don't. I know I just said LLMs and diffusion models destroy all nuance, but that's not strictly true. My line in the sand is this: if nobody can tell overtly that you're using LLMs, and you can demonstrate your own understanding of what you are doing, then I don't care. In that case it's more of a concern for the person using the LLM, because the real concern in that case is becoming dependent on LLMs and being unable to write software/etc. without them anymore. But that isn't really my concern.
If I can tell your blog post was written by Claude within the first few sentences you have completely failed to demonstrate anything but your willingness to deceive people. How can you not feel some shame putting yourself on the By-line of a blog post you didn't write? I mean it's written in first person! And I know damn well that not everyone can immediately pick out LLM-generated text, but that isn't entirely the point either. It's one thing to use LLMs to do work, but the point of a blog post isn't just the text, blogging is a form of (formerly...) human expression. It just doesn't make any sense to put Claude-slop blog posts right next to human-authored blog posts, any more than it makes sense to put AI-generated images next to drawings. The point isn't the pixels and isn't the words. That contrasts with code, because with code, the point really is the code, and other stuff is secondary.
I can't tell you whether this person found this vulnerability and triaged it or Claude did. I don't even care. However, I don't trust a blog post they didn't write anyway.
This is a long comment, but it's hard to make it shorter. LLMs and diffusion models are not comparable to tools like Photoshop, in a way that is more extreme than the difference between Photoshop and a pencil. Analogies are nice (limited) shortcuts that can help us reason about things, but in this case analogies alone are insufficient because we are dealing with something that has a brand new shape.
> "Four things had to change, and they stack:"
When you write your blog post with claude, I start hitting these phrases and then start only skimming and looking for them, and stop reading what you have to write.
> When you write your blog post with claude, I start hitting these phrases and then start only skimming and looking for them, and stop reading what you have to write.
I've spotted a few memes about this and can admit to doing the same..
ai;dr
(tl;dr, evolved)
If the subject seems interesting I just ask LLM to summarize in those cases, then go deeper if it ends up actually interesting or compelling.
I'd rather skip, personally. I just don't have to read things these days, and I see enough Claude/GPT prose at work, it makes me resent seeing it elsewhere.
I have a backlog of books written by humans I'd much rather spend my time on.
But I love a good handwritten tech article.
Can I self promote in this thread XD
how does something like this pass code and design reviews. is this code written by ai (the buggy code). Not only was the key that signs the cookie inside the cookie it self but it's using pickle which even model weights are moved away from. I'm a little flabbergasted at the noob nature of this root cause.
pickle is nuts, but I can imagine dumping an entire object into the cookie while troubleshooting something then forgetting to clean it up afterwards.
This does not pass automated testing where I work let alone code review.
> how does something like this pass code and design reviews
when the "give a shit quotient" of the employee moves towards "highly engaged with the organization and its mission" to "this is just a job, I'm gonna rest and vest"
I think you're being unreasonably uncharitable. Any and all software contains bugs, and I can assure you the piece of code you wrote has bugs. My code has bugs. Google's code has bugs. iOS has bugs. rocket launching software has bugs.
We can put in systems in place to catch bugs. And that's the system that's working here.
Now to the second part of the question -
How did they not catch it before? If it were "easy", it would not have been worth 90k, and hindsight is 20/20 - once you know where to look for, it is all very very obvious.
From TFA:
> Normal Google Applications (T2), S0 RCE: $75,000 base × 1.2 for exceptional report quality.
"Easy" isn't a factor in calculating the bounty.
Do you have any advice for aspiring bug-bounty hunters? Do you have some method for being able to find vulnerabilities like this, or lots of past experience?
> Do you have any advice for aspiring bug-bounty hunters?
We're probably at the end of the era of these kinds of big payouts. AI tools will drive the going rates to approximately the costs of the tokens. Bounties exist to make it nominally more economical for researchers to report exploits than it is to resell them.
That's really not why bounties exist, though people on both sides of the bounty systems claim it is. If it were broadly the case that bounties exist to offset black market sales, you wouldn't pay bounties for vulnerabilities with no market value. This vulnerability is probably a good example of something with a sub-$1000 value on any non-bounty market.
In reality, the modal benefit accruing to a company running a bounty program on receipt of a new serious vulnerability is simply engineering prioritization. You get bug reports from users when you have reliability and UX bugs, but you don't get them for security flaws, because attackers generally have to force vulnerabilities into revealing themselves --- normal users almost never run into them. Bug bounty programs fill that gap.
"This vulnerability is probably a good example of something with a sub-$1000 value on any non-bounty market."
Crazy statement. This would easily be worth more to malicious actors
I don’t understand your argument here.
If you’re saying that bug bounties exist for a reason according to both the program runners and the contributors then how is that not the reason they exist?
The people paying and the people working are all in agreement that it exists for that reason.
What part of my logic do you not follow? The way you've posed this question, the simplest response would require me to write that whole comment over again.
That’s an unnecessarily rude response I think.
I detailed my logic above.
Additionally, part of my businesses remediation services include helping companies set up bug bounty programs. Giving researchers an incentive to report bugs for payouts instead of selling or exploiting them for profits is always the intended goal.
This is probably my highest-rated comment on HN:
https://news.ycombinator.com/item?id=43025038
It's the product of direct interviews with bounty people, FAANG/MAG7 blue teamers, and black market participants. If you want more, there's also an interview online that we did with Mark Dowd, who is one of the best known CNE vendors to the NATO IC. You can just Google [SCW Mark Dowd].
I'm happy to go into more detail, but, like I said, you didn't really respond to my comment in any way that I could expand on. Feel free to disagree with me in some kind of specific way.
This really is simple: if you work in this space, you already know that there's essentially no black market at all for serverside web bugs. But there's a $90k bounty for this one, and 5-figure bounties for others. Reconcile those two facts.
Well thats nice and Im happy for you but internet points don’t really mean much here…
What a vulnerability sells for and how much damage someone could do exploiting it are two different things. Someone capable of finding a serious vulnerability is probably capable of exploiting it themselves. They don’t need a buyer.
A company paying big bucks isn’t a contradiction. Its incentivizing disclosure over exploitation which could cost them considerably more. Engineering prioritization is… a minor part of it informed by the ease of exploitation, damage it could do (in the case of an RCE or publicly available keys as in this case quite a bit) etc.
I don’t see why you think these things are mutualy exclusive.
You think that most security researchers that can find an exploit are all willing to commit a heist with that exploit?
I’m not in the security industry but that strains credulity.
Where did I say that? Being capable of exploiting something and willing to are two different things.
The point is that bounties create an incentive to find and disclose vulns. That doesn’t require a black market for the exploits.
“Someone capable of finding a serious vulnerability is probably capable of exploiting it themselves. They don’t need a buyer.”
I’m responding to this. I would assume the list of people able to find a vulnerability is much much greater than the number of people willing to do a one off heist with it.
What you’re responding to and what you’re saying it says are two different things.
Sounds good. Thanks.
> Bounties exist to make it nominally more economical for researchers to report exploits than it is to resell them.
Is this really true? After all, reselling the exploit would be illegal, and most people don't want to break the law to earn income.
most people who care about legality don't work in that field either. whoever gets into this probably starts by being ok with doing shady stuff, I imagine.
Many start in the military and eventually move to private sector. However, an ever growing percentage just straight up go to college to learn the skills.
Cyber companies are in the fortune 100 now, and it's not like it used to be back when you actually had to be interested in it and dedicate your life to learning the skillz from shady forums. Now it's just a job to most practitioners.
what use are 0-days to private cybersecurity companies?
Huge marketing/PR benefit.
Yes, but also there is also a huge value to individual contributors and their careers. I broke into the field after landing a few relatively high profile CVE acknowledgements for big companies. Responsible disclosure is good for your career.
On the other hand, if the value of that drops you can expect to see some folks start taking the wrong path.
Plenty of people in this space who work on entirely legal things
Most people do not, but it needs only some people who do so for it to be a problem. Most people do not steal, but I still lock my front door.
there are enough people operating in countries that would not charge them for hacking an American company for it to matter, $90k can be a life-changing sum
And before it ends, people with existing reputation can milk the last drops, as mostly their reports gets processed from the thousands of others. Unfortunate status quo.
People who pay bounties for vulnerabilities don’t really care how you find them.
> code execution across seven Google domains
What a bargain
Bold choice using the noun "I" repeatedly instead of "Claude" - you didn't figure this out, Claude did, and then Claude very obviously wrote the blog post.
Oh, maybe that's why "Claude" was substituted with "I", because Claude wrote this blog post from a first person perspective.
It's hard to get a $90k bounty from Google, even with Claude in hand, so we shouldn't be dismissive of that, but it is pretty annoying to read write-ups written entirely in LLM voice.
Shallow dismissals like this contribute nothing to the discussion.
How much more depth is the dismissal supposed to have? The original deception lacks depth.
Are you referring to your own post? If so, accurate
An llm is a tool, I had access to Claude too but I didn’t find the bug. The author found it.
Research taste, among other things are still abilities that differentiate among humans.
In fact, I expect the elimination of other limiting factors by AI to lead to those becoming more important. By the same mechanism that make ugly hedge fund managers more performant.
The problem then is that composition of the top performers will change. Which is problematic since being a top performer ( ‘artist’, ‘researcher’, ‘writer’,..) is often an identity which ‘we often create to protect ourselves: the strong one who never breaks, the intelligent one who is always certain... The mask that once protected you can eventually become the cage that prevents you from changing’
'AI' is Not a Tool: https://www.linkedin.com/pulse/ai-tool-rich-hickey-kjl8e
An invitation for a semantic debate online on a minor point along with a random linked in post stating something about said debate?
No thank you says pretty much everyone.
Yeah, I was about to comment that I’m not reading all that. But good for them though.
Then why is it marketed like one? And why can I use it like a tool?
I do have to send input to the thing to get a response, no?
Sure it is. And it's AI, not 'AI' -- not sure why the need to add quoting.
It's literally the title. It seems every reply to this is just someone saying they didn't read it.
Correct. The headline is incorrect with both the quoting of AI as well as the claim it isn't a tool.
You can hardly blame anyone for avoiding a clickthrough into an auth-walled, user-hostile site to read the point the headline made in an attempt to "gotcha"
You are underestimating how much effort and prior knowledge it takes to work with claude on this kind of thing.
It really hurts the fingers to type "make no mistakes" in every prompt.
I bet, I could take "Trading places" style, anyone off the street, give them a 100 "Secret" word list and they could spend 6 months discovering all sorts of things.
I agree effort and prior knowledge are required, but a few magic words is like 80% of the effort.
Well the first 80% is easy. It's the second 80% that really gets ya...
OK do it. And make yourself 90K along the way.
This is the typical "ugh that's just rsync" when Dropbox was announced, but for a new LLM age.
_If it's so easy, do it_. Don't post about it. Do it. Fix these bugs at scale. Charge people money for it once you've built up a reputation. Get your own bug bounties.
If you're going to make this argument, at least say GLM 5.3 instead of Claude, Claude will refuse to touch anything remotely close to this unless you're in the super special club.
Perhaps next time they can carve 1's and 0's by hand from a slate tablet.
Same old tired complaining about people not using technology correctly, or the way a _real_ coder would do it.
You mean, using butterfly, right?
For the uninitiated: https://xkcd.com/378/
handing clients access to an LLM gets them beginner level code, easily. same with beginner level exploits. an RCE against google is not a "beginner level exploit". If you've seen AI generated code, I can tell you that AI generated security findings also have that "beginner" smell. an expert who knows what they're doing has to drive them to look at the right places, and use the experts preferred method.
I could have overlooked but doesn't seem there is a mention of Claude in the blog post? Did author say they had used Claude?
I also am wondering if Claude will proceed on a prompt which is about finding a bug like that? I am not sure, has anyone tried?
Took about 3 sentences for me to be as certain as I've ever been about anything I've ever been in my entire life that this blog post was authored by an LLM, though truthfully it's impossible to know for sure if it was a Claude model or a distill.
The cookie is "what caught my eye." not realistic phrasing
The cookie contained a pickle-serialized payload, "which was bad enough on its own." maybe not best practice but not bad on it's own
Flask session cookies are signed", not encrypted," so the... LLMs love to tell you what things aren't, as an emphasis on what they are
Tbf as someone with an infosec background, I would have probably written just about the same.
Pickle payload is a red flag precisely because pickle is not a safe serialisation format format when taken as untrusted input, here the input was trusted but it's still giving someone en easy escalation path (exactly what happened)
And most people would assume the whole cookie payload would be encrypted if they saw just a binary blob, and not realise you can fully read it if it's just signed
But, yes, the post was probably edited by an LLM, as is pretty much everything else you read online these days.
If he used an old school source code scanner would the credit go to him or to the tool? Because Claude is a tool, not a person.
;)
This is a clear example of why it is a shame we cannot block commenters on this website.
welcome to the club
Nice catch, but god terrible write-up. Please take the time to write it yourself even if you think you will do worse in explaining it than the LLM. It’s so off-putting to read that. Share your prompt or raw article notes even, it will much enjoyable to read than this
If this were Microsoft, they would have received a coupon for 10% off a Thanksgiving turkey, only redeemable in 2030 and only in the month of March, must be purchased with cash, non-transferrable.
Sorry for the meta, but the discourse in many of the comments is nuts to me.
"Claude did this, not you" - does that mean that humans have no agency at all when they use an LLM? Is that the argument?
Maybe the norm should be to credit the LLM and harness used, like "I used ___ to help me find an RCE", but I can see arguments not to do that in some cases.
edit: The giant thread with this now got flagged, so some users may not see it.
Would Claude have found it without your guidance and knowledge? If yes, Claude found it. If no, you found it while using an LLM.
I suspect a lot of things fall into the latter. It is a common neg to defer everything to the former though.
If we're doing this standard, then let's do it for everything. Claude coded that, not you. Claude built that website, not you. Might make the industry less exhausting tbh.
GPT did the Hugging Face hack, not you (human engineers at OpenAI) :^)
The best response to them is "If it's so easy, then when didn't you do it? Easy $90k."
One day soon, this will be like people saying you used a high level language instead of assembly to discover a fault.
People who believe AI is an abstraction layer on top of compilers, are in that middle ground where they know computer science but are far from an expert. Which makes them very susceptible to the Dunning–Kruger effect. Throw in AI sycophantcy and you've got yourself takes like the one above.
Ha.. AI in the form of LLM is not a very good non-deterministic abstraction layer on top of deterministic compilers.
Yet, they are being used with endless surprise as non-deterministic layers of abstraction on top of deterministic compilers.
My point is to use a technology for what it is capable for (where capabilities are discovered weekly and hard to keep up with), not trying to get it to do everything.
Bringing a deterministic lens to non-deterministic software has its dangers too. Non-deterministic software seems to be its own category of software, increasingly capable of non-deterministic approaches that deterministic compilers software couldn’t attempt.
Maybe that’s less compilers and more different type of scaffolding of vectors.
I’m all for skepticism, the issue with cynicism is capabilities often pass those perspectives by sometimes too quickly.
Yeah some people come off as envious they didn't have Claude try it first.
But it needs a disclosure if Claude was used: "The cookie is what caught my eye" - if that was actually Claude's eye then yeah, Simon is claiming to have a keen eye that he in fact does not.
"Claude found a $90K RCE" would have been just as interesting if not more.
Sure, but do we want traditional vuln posts to be “nmap found”?
It is fair an interesting to disclose tools used, but I am with GP in leaving agency with humans, at least for now.
Nmap doesn't write its own blog posts. That's not where I draw the line, but the point isn't at all about "drawing the line." (In other words, don't get caught up in where the line is. There is one, but we're not sitting in some gray area, we've blown past it by hundreds of kilometers by now.)
If I post an image and I say, "I drew this", and someone says, "no you didn't. That is clearly output from Qwen Image 2.1", can I reply, "Qwen Image 2.1 is just a tool, like Photoshop"? I mean, I can, but if I do this, I am missing the point. The point is that I didn't draw anything. I didn't draw. I didn't draw digitally. I didn't draw analog. I didn't draw with my mind. I typed a prompt. This is as close to drawing as photography is to drawing.
So did this person actually find a vulnerability on their own? Hell if I know, but I take issue with LLM-authored blog posts; I don't trust people who post LLM-authored blog posts as if they were written by themselves to be honest about the nature of their work.
If you really are still doing work and understanding and just using LLMs as a tool, show your understanding in your blog post that you are communicating to the world, instead of having Claude or GPT show its understanding. I don't really care whether you used Claude and didn't list it, if it's clear as day you understand what you were doing with it.
I'm dying on this hill. Until HN bans this sort of blog post or bans all of us complaining, these comments are going to be very, very boring. And I am stubborn for longer than most people can stand to read the same comments. Believe me.
That strongly depends on what you consider drawing, right? Sure you didn’t dip a pencil into an inkwell and caused stains to appear on a paper. But you did something that caused stains to appear on the image. You already seem to accept Photoshop as a way for a human to do that. Why not Qwen?
We can discuss the quality of the offering. But that’s a different discussion
Because it's not drawing. It's not even drawing in the abstract: nobody drew anything. There are no brush strokes. You can watch the steps diffusion models take and at no point is there anything that resembles "drawing".
Drawing with Photoshop absolutely resembles drawing. Photobashing with Photoshop does not. Photography also isn't drawing. These things are all distinct. That's the actual point I was trying to get at here by invoking Photoshop. Image models differ from text in that text models do resemble something closer to what people are actually doing, but you just have to accept that analogies have limitations here. Just like you can draw or not draw with Photoshop, you can use Claude in ways where you're really driving it like a tool to do work, and you can also use Claude in a "fix the bug" sense where you're doing absolutely nothing other than paying $200/mo and typing occasionally.
But whereas Photoshop introduces nuance to what it means to draw or make art by introducing different levels of involvement versus remixing and etc., Claude and Stable Diffusion utterly decimate nuance in a thermonuclear blast that plunges us into a nuance Nuclear Winter, by offering for perhaps the first time a way for the computer to do nearly all of the generalized "thinking" while you sit on your ass and argue on HN. (Reminder for everyone to check on their agents.)
Trust me, I use LLMs, I get the idea. I can "vibe code". On my workbench is a laptop running an entire desktop system including an application suite that I really didn't write. It would be cool to go online and say "Hey guys check out this cool desktop environment I wrote!" but I won't because I really didn't. It was just an experiment. Did I put in effort? Yes. Was it coding? Nope, it was closer to management, though I did write some code. The experiment was some mix of failure and success, but what I would never do is say that I wrote it. I may some day blog about the experience and the pros and cons and how I feel about the resulting code, but for right now I feel the Internet is full of enough slop and most people who I respect are thoroughly tired of hearing about LLMs all day (I am too at this point, though I am not saying I find none of it interesting.)
Do I think all LLM use absolutely must be disclosed? Actually no, and especially right now I understand why some people don't. I know I just said LLMs and diffusion models destroy all nuance, but that's not strictly true. My line in the sand is this: if nobody can tell overtly that you're using LLMs, and you can demonstrate your own understanding of what you are doing, then I don't care. In that case it's more of a concern for the person using the LLM, because the real concern in that case is becoming dependent on LLMs and being unable to write software/etc. without them anymore. But that isn't really my concern.
If I can tell your blog post was written by Claude within the first few sentences you have completely failed to demonstrate anything but your willingness to deceive people. How can you not feel some shame putting yourself on the By-line of a blog post you didn't write? I mean it's written in first person! And I know damn well that not everyone can immediately pick out LLM-generated text, but that isn't entirely the point either. It's one thing to use LLMs to do work, but the point of a blog post isn't just the text, blogging is a form of (formerly...) human expression. It just doesn't make any sense to put Claude-slop blog posts right next to human-authored blog posts, any more than it makes sense to put AI-generated images next to drawings. The point isn't the pixels and isn't the words. That contrasts with code, because with code, the point really is the code, and other stuff is secondary.
I can't tell you whether this person found this vulnerability and triaged it or Claude did. I don't even care. However, I don't trust a blog post they didn't write anyway.
This is a long comment, but it's hard to make it shorter. LLMs and diffusion models are not comparable to tools like Photoshop, in a way that is more extreme than the difference between Photoshop and a pencil. Analogies are nice (limited) shortcuts that can help us reason about things, but in this case analogies alone are insufficient because we are dealing with something that has a brand new shape.
That's an engineer right there. I really love how you didn't just 'it is what it is'. Claude is never replacing engineers like you.