The attack is being described as “sophisticated” but we can thank our (GitHub) stars that the exfil was a half-assed job that ultimately made only public repos vulnerable and made it obvious in logs whether a compromise occurred.
It’s almost like a grey-hat attacker trying to make the supply chain vulnerabilities more visible without doing major damage themselves. Almost.
> The attack methodology involved a particularly sophisticated approach. Attackers inserted a base64-encoded payload into an install script, causing secrets from affected CI workflows to be exposed in workflow logs.
What? How is that sophisticated? Who wrote this?
I still don't understand how we got to this point where CI/CD pipelines are built from random shit on the internet. I remember people being worried about packages in the system package manager curated by a (relatively) small set of trusted project maintainers. Now we're pulling in garbage written by who knows, under security guidance of nobody. At least the Arch Repo has a procedure and a trust network.
Every time I have to use GitHub actions and it recommends me using some "community" action I can't do it. I just know it's written by some 12-year old on spring break.
My GitLab CI/CD pipelines are 100% code that I personally wrote or that I've personally reviewed and forked into our internal source control. Our entire business depends on this code, so the best possible course of action to me seemed to be 100% control.
The idea of just plugging in some black box script written by anyone who can update it at any time seems insane. What kind of idiot would trust something as sensitive as CI/CD with random unverified scripts you have zero control or oversight of? Apparently quite a lot of idiots.
When Github first showed up, I really liked it that (unlike on e.g. SourceForge) the username is right in the project's URL, it suggested that someone's accountable for the project. But there's no such thing as "accountability" when you're relying on unpaid volunteer work, made available for free to the general public - the "NO WARRANTY" is spelled in all caps, right there in the license.
So how do you know if you can trust the code you're running? You use your own judgement.
Yes but SHA1 collisions are easy enough to engineer, so even then compromise is probably possible.
(I don't know how hard it is to push a different object to an existing SHA on GitHub—I'm guessing that you probably have to remove all references to the original object at that SHA?)
SHA1 collisions are easy, but nobody has publicly revealed a second-preimage attack. With a collision you create two inputs that hash to the same output, with a second-preimage attack you are given one existing input & have to find a second input that hashes to the same output. Collisions are much easier since you can control both inputs.
That's a good point. Setting up a benign release first that you have engineered a same-hash malicious release you can swap in later is a higher bar than gaining control of a repo and immediately replacing a popular release.
On its own, immutability isn't a complete solution to supply chain attacks. Software still needs to be updated and those updates could contain malware too.
You need immutability and something like sandboxing where actions cannot e.g. dump the memory of the runner process to steal secrets.
The alternative is vetting every single line of code in every dependency and every subdependency perfectly for every update, which is not realistic.
This is already a month old. Suggest renaming to make this clear, or you've got people jumping on this as a brand new issue.
Yeah my first thought was, "again?!". First one was rough enough, second one might actually tilt the scales to migrating.
Yup
The attack is being described as “sophisticated” but we can thank our (GitHub) stars that the exfil was a half-assed job that ultimately made only public repos vulnerable and made it obvious in logs whether a compromise occurred.
It’s almost like a grey-hat attacker trying to make the supply chain vulnerabilities more visible without doing major damage themselves. Almost.
I warned about this, but you didn’t have to be clairvoyant to see it coming: https://cedwards.xyz/github-actions-are-an-impending-securit...
GitHub are cutting corners and not working on making their CI/CD offering secure.
Previous discussions: https://news.ycombinator.com/item?id=43368870
> The attack methodology involved a particularly sophisticated approach. Attackers inserted a base64-encoded payload into an install script, causing secrets from affected CI workflows to be exposed in workflow logs.
What? How is that sophisticated? Who wrote this?
I still don't understand how we got to this point where CI/CD pipelines are built from random shit on the internet. I remember people being worried about packages in the system package manager curated by a (relatively) small set of trusted project maintainers. Now we're pulling in garbage written by who knows, under security guidance of nobody. At least the Arch Repo has a procedure and a trust network.
Every time I have to use GitHub actions and it recommends me using some "community" action I can't do it. I just know it's written by some 12-year old on spring break.
In 2025 sophisticated means someone took two or three steps.
Step 1: ask the LLM
Step 3: profit
right? Did I leave something out?
Step 2: do the sophisticated needful.
My GitLab CI/CD pipelines are 100% code that I personally wrote or that I've personally reviewed and forked into our internal source control. Our entire business depends on this code, so the best possible course of action to me seemed to be 100% control.
The idea of just plugging in some black box script written by anyone who can update it at any time seems insane. What kind of idiot would trust something as sensitive as CI/CD with random unverified scripts you have zero control or oversight of? Apparently quite a lot of idiots.
I agree with your general sentiment.
When Github first showed up, I really liked it that (unlike on e.g. SourceForge) the username is right in the project's URL, it suggested that someone's accountable for the project. But there's no such thing as "accountability" when you're relying on unpaid volunteer work, made available for free to the general public - the "NO WARRANTY" is spelled in all caps, right there in the license.
So how do you know if you can trust the code you're running? You use your own judgement.
I hope GitHub will prioritise immutable actions and enforce it to all actions
Can't you sort of do that by pinning on the commit SHA already? It's bad that that's not the ONLY way to do it, but at least it's something.
You can also fork all the dodgy actions you consume.
Not really a solution at enterprise level and it exposes to the risk of likely not patching them as often
Yes but SHA1 collisions are easy enough to engineer, so even then compromise is probably possible.
(I don't know how hard it is to push a different object to an existing SHA on GitHub—I'm guessing that you probably have to remove all references to the original object at that SHA?)
SHA1 collisions are easy, but nobody has publicly revealed a second-preimage attack. With a collision you create two inputs that hash to the same output, with a second-preimage attack you are given one existing input & have to find a second input that hashes to the same output. Collisions are much easier since you can control both inputs.
That's a good point. Setting up a benign release first that you have engineered a same-hash malicious release you can swap in later is a higher bar than gaining control of a repo and immediately replacing a popular release.
Yes and I do that and Dependabot supports it but most people wouldn’t bother
On its own, immutability isn't a complete solution to supply chain attacks. Software still needs to be updated and those updates could contain malware too.
You need immutability and something like sandboxing where actions cannot e.g. dump the memory of the runner process to steal secrets.
The alternative is vetting every single line of code in every dependency and every subdependency perfectly for every update, which is not realistic.
More discussion at the time: https://news.ycombinator.com/item?id=43367987
Guess that "all aboard the LLM/Copilot train!" way of developing your core product really paid off!