Git ransom campaign incident report(about.gitlab.com)
about.gitlab.com
Git ransom campaign incident report
https://about.gitlab.com/2019/05/14/git-ransom-campaign-incident-report-atlassian-bitbucket-github-gitlab/
74 comments
The threat cited in the article said not just that the code would remain deleted, but that it would be "leaked" - presumably many of these were private repos.
You could never trust that the attacker actually deleted their copy of the repo, but then, the whole cryptolocking business model falls down if the attacker isn't at least moderately honest, so I can see why people would respond to that threat.
You could never trust that the attacker actually deleted their copy of the repo, but then, the whole cryptolocking business model falls down if the attacker isn't at least moderately honest, so I can see why people would respond to that threat.
”the whole cryptolocking business model falls down if the attacker isn't at least moderately honest”
Nitpick: it only requires most attackers to be somewhat honest. Having a few unscrupulous ones may make life harder for the “honest” ones, but they themselves can be better of, e.g. by, after receiving payment, demanding more money.
Nitpick: it only requires most attackers to be somewhat honest. Having a few unscrupulous ones may make life harder for the “honest” ones, but they themselves can be better of, e.g. by, after receiving payment, demanding more money.
If I remember correctly, WannaCry had a small customer support call center =D
Very fair point.
Is it more unethical to release an "honest cryptolocker" or one that lies and never gives the files, degrading the trust the entire cryptolocker grift relies on?
It's pretty obvious that it's worse to be an actual criminal, than someone who goes around and pretends to be one.
In the same way that it's worse to shoot someone with an actual gun than to threaten to shoot them with a Nerf gun.
The negative network effects on other scammers are also nice.
In the same way that it's worse to shoot someone with an actual gun than to threaten to shoot them with a Nerf gun.
The negative network effects on other scammers are also nice.
I'm not so sure.
An "honest cryptolocker" helps support more cryptolocker use, as people trust that if they pay the criminal they'll get their stuff
If dishonest ones were the norm, than maybe cryptolocking would cannibalize itself as nobody would pay since they know its useless. So in a sense the dishonest one while having less ethical intention has more ethical results. But only at scale. Hmmm.
An "honest cryptolocker" helps support more cryptolocker use, as people trust that if they pay the criminal they'll get their stuff
If dishonest ones were the norm, than maybe cryptolocking would cannibalize itself as nobody would pay since they know its useless. So in a sense the dishonest one while having less ethical intention has more ethical results. But only at scale. Hmmm.
In this case both are actual criminals but one returns your data after payment while the other doesn't
[deleted]
Sounds like we need a review site for extortionists.
Would you charge the extortionists to remove their negative reviews?
No, they just have to prove they're the real person with photo ID and admit they are the person being referred to as the criminal.
Or maybe an escrow service for extortionists who makes sure the amount is refunded if the extortionist does not deliver.
For what it's worth, I really hope people don't pay if they can avoid it. Guy I know consults for a company which recently got ransomware. They had insurance, payed $1.5 million, got their files back. FBI came in and figured out it was the north koreans. This is happening more and more often, and will increase as we continue sanctions pressure.
This is a classic prisoners' dilemma: if no one payed, every one would be better off, but it is very hard to be that one guy or company who loses all his files for the "greater good".
This is a classic prisoners' dilemma: if no one payed, every one would be better off, but it is very hard to be that one guy or company who loses all his files for the "greater good".
its easier to trust private hackers than organizations that have the law on their side
society works with mutual cooperation and hackers seem to understand that more than the "technically cooperating in this context" that the legal field would employ
society works with mutual cooperation and hackers seem to understand that more than the "technically cooperating in this context" that the legal field would employ
Vast majority of cryptolockers are fake, they just keep asking for more and more money but never unlock.
(This is probably not true, but society would benefit from "cryptolockers are usually fake" being in the zeitgeist)
(This is probably not true, but society would benefit from "cryptolockers are usually fake" being in the zeitgeist)
I don't keep all my git code local, only the projects I am currently working on.
Do you rely on just one 3rd party like gitlab/bitbucket/github to keep your _only_ copy of your non-current projects?
That seems unwise. I don't have many local repos on this 128GB MacBookAir, but as well as BitBucket all the projects I have ever worked on are on other several machines and/or hard drives I have locally, and also zipped up in S3 buckets and on tarsnap.
Like they say, there's two kinds of people. People who've lost important data because they didn't back it up properly, and people who haven't yet.
That seems unwise. I don't have many local repos on this 128GB MacBookAir, but as well as BitBucket all the projects I have ever worked on are on other several machines and/or hard drives I have locally, and also zipped up in S3 buckets and on tarsnap.
Like they say, there's two kinds of people. People who've lost important data because they didn't back it up properly, and people who haven't yet.
> Through immediate independent investigations, all three companies observed that user accounts were compromised using legitimate credentials including passwords, app passwords, API keys, and personal access tokens.
Part of your backup strategy depends on external services. Not necessarily in your case, but people who only have their backups externally on a service could be affected.
> all the projects I have ever worked on are on other several machines and/or hard drives
And depending on your strategy, since they're so distributed it could mean they're outdated repos. If not, and they pull automatically, they could be affected.
Local backups also have issues. The disk might die, the data might be corrupted or any other myriad of things could happen.
> People who've lost important data because they didn't back it up properly, and people who haven't yet.
Is there such a thing as a perfect backup strategy?
Part of your backup strategy depends on external services. Not necessarily in your case, but people who only have their backups externally on a service could be affected.
> all the projects I have ever worked on are on other several machines and/or hard drives
And depending on your strategy, since they're so distributed it could mean they're outdated repos. If not, and they pull automatically, they could be affected.
Local backups also have issues. The disk might die, the data might be corrupted or any other myriad of things could happen.
> People who've lost important data because they didn't back it up properly, and people who haven't yet.
Is there such a thing as a perfect backup strategy?
> Is there such a thing as a perfect backup strategy?
At work, there's "Can meet contractually agreed RPO and RTO with 99.99% certainty". Automate the standard setup, and sleep well at night. Perfect.
At home, there's "I've done enough that I think the next improvement is an unfeasibly large amount of extra time&money for an unreasonably small improvement".
I've, for myself at home, settled on Apple's Time Machine backing up my Macs (and their phone/ipad iTunes backups) to a raid 10 set, that raid 10 set rsynced to another one at the opposite end of the house, and a weekly backup of that stored on a single drive that only powers up for 6 hours every Sunday night then powers back down again - so if my whole network gets breached and cryptolockered (for example) I'll still have at most 7 day old data at home. I also push that weekly backup out to S3 and tarsnap for off-site in-case-my-house-burns-down, or I've set it all on fire and moved to Belize scenarios...
I've been running most of that for ~8 years now. I've called it "done", while not "perfect", its certainly good enough against "not-Mossad threat models". If Mossad or The NSA want to delete my backups, so be it - I'll go be a carpenter or a gardener or something.
At work, there's "Can meet contractually agreed RPO and RTO with 99.99% certainty". Automate the standard setup, and sleep well at night. Perfect.
At home, there's "I've done enough that I think the next improvement is an unfeasibly large amount of extra time&money for an unreasonably small improvement".
I've, for myself at home, settled on Apple's Time Machine backing up my Macs (and their phone/ipad iTunes backups) to a raid 10 set, that raid 10 set rsynced to another one at the opposite end of the house, and a weekly backup of that stored on a single drive that only powers up for 6 hours every Sunday night then powers back down again - so if my whole network gets breached and cryptolockered (for example) I'll still have at most 7 day old data at home. I also push that weekly backup out to S3 and tarsnap for off-site in-case-my-house-burns-down, or I've set it all on fire and moved to Belize scenarios...
I've been running most of that for ~8 years now. I've called it "done", while not "perfect", its certainly good enough against "not-Mossad threat models". If Mossad or The NSA want to delete my backups, so be it - I'll go be a carpenter or a gardener or something.
There’s probably a lot of code sitting in private git repos where like one guy worked on it five years ago and then quit the company and gitlab/github might be the only place it exists.
Private repositories. You may not want their contents to become public
How certain are you that the photo archive you use had a copy of that photo? I mean, yeah, most people have local trees. Inevitably someone won't, and like spam this is a volume scam.
Is it common that companies share intelligence like this? I think it's a wonderful idea, given they all operate on essentially the same service (git) they share similar security concerns.
In the Cybersecurity field, yes!
You'd be surprised how often you'll be rolling up to your competition to compare the virus files you pulled from each of your networks and reverse engineering on VMs together... then next week you have to pretend you hate them until something else goes wrong again.
You'd be surprised how often you'll be rolling up to your competition to compare the virus files you pulled from each of your networks and reverse engineering on VMs together... then next week you have to pretend you hate them until something else goes wrong again.
It took exceptional circumstances, but it was the right call in this case. All were working toward the same goal. We can all go back to our war after the white walkers are dealt with (don’t bring Cersei up or it ruins the analogy)
Seems like no one fell for this though. https://www.blockchain.com/btc/address/1ES14c7qLb5CYhLMUekct...
Probably a unique wallet in each message. It seems to be that way in scam email ransoms.
Doesn't seem that way: https://github.com/search?q=1ES14c7qLb5CYhLMUekctxLgc1FV2Ti9...
Repo's with remaining ransom file: https://github.com/search?q=1ES14c7qLb5CYhLMUekctxLgc1FV2Ti9...
Weird, weren't GitHub and friends restoring these repositories for these unlucky users?
I assume they're trying to contact the repo owners to see how they want to proceed. Even if the change seems obvious, changing the contents of a repo is definitely a case where the git host should involve the owner of the repo (barring ToS violations).
This is the safer bet. The host should offer to restore from their own backups, but some customers may have already taken care of their own stuff.
"All of this has happened before, and it will all happen again." - Peter Pan movie, Battlestar Galactica, and should be every security incident report ever.
Not saying they shouldn't have issued their analysis, of course they should have, it mostly looks on target. But...it will all happen again.
Not saying they shouldn't have issued their analysis, of course they should have, it mostly looks on target. But...it will all happen again.
1. Stop using 'git add .' This is a bad habit I see people keep suggesting to new git users. Stop recommending it and stop doing it.
2. Never store your password in .git/config. Why are you doing that? That shouldn't be stored in .git/config.
2. Never store your password in .git/config. Why are you doing that? That shouldn't be stored in .git/config.
Why are people using passwords instead of keys?
[deleted]
"Stupid corporate firewall blocks SSH connections" would be my guess.
Why do people even have passwords that don't require 2FA?
Maybe I misunderstood something, but I read the text as people (or .. services like the three doing the announcement) offering URLs containing an access token?
As in https://example.com/mlindner/project1/821372asd1786d21das or something?
If you use this approach to manage access to a repository, then .. that gets stored in the .git/config. No need to store a password or something.
(Then again, maybe I didn't understand the explanation correctly?)
As in https://example.com/mlindner/project1/821372asd1786d21das or something?
If you use this approach to manage access to a repository, then .. that gets stored in the .git/config. No need to store a password or something.
(Then again, maybe I didn't understand the explanation correctly?)
How does one withdraw bitcoin to fiat or even use it without it being traceable? Are there laundering or anonymizing services for bitcoin withdrawals to fiat?
AFAIK there have been bitcoin laundering services for years (bitcoin in, bitcoin out).
As for spending, there used to be pre-loaded credit cards that you could from bitcoin.
As for spending, there used to be pre-loaded credit cards that you could from bitcoin.
Wouldn't the credit cards be traceable (record of where you spent it). It wouldn't necessarily tie you exactly but it'd create a large paper-trail.
The BTC could be used as payments for services or products which don't require identification (and holding "dirty" BTC is now the problem of the sand celler if they ever want to take it to traditional businesses in jurisdictions that enforce KYC for BTC payments).
Criminals can also trade BTC for physical cash.
They could also by some means (for example permissionless decentralized exchanges) convert it to a cryptocurrency with private transaction properties such as ZCash, Monero, Beam or GRIN and then back again.
The "laundering services" you refer to (generally called "mixers") are still around but most of them have been shown reversible with high degrees of confidence. CoinJoin is the state of the art here, with the most well-known implementations being JoinMarket and Wasabi Wallet.
But in general law enforcement and investigators are definitely wising up to cryptocurrencies and to be fully untraceable one has to go through a lot of hoops and not make a single mistake in the process. Even the above mentioned approaches can leak information that can be used to tie an individual to the transactions if not executed properly.
Most likely they will use the same tried and true approach they would have used for stolen fiat funds; identity theft. I personally know people who have been drawn into a criminal investigation for money laundering because they had been initiated a transaction selling BTC on LocalBitcoins via bank transfer (unwise unless you know and trust the person), turned out already stolen BTC had been converted into fiat on a compromised bank account, which was then supposed to be converted into "clean" BTC again. Fortunately the investigation was already underway when the transaction happened, my friends bank account was blocked as a result when the transfer was initiated and the whole thing was sorted out in the end.
Criminals can also trade BTC for physical cash.
They could also by some means (for example permissionless decentralized exchanges) convert it to a cryptocurrency with private transaction properties such as ZCash, Monero, Beam or GRIN and then back again.
The "laundering services" you refer to (generally called "mixers") are still around but most of them have been shown reversible with high degrees of confidence. CoinJoin is the state of the art here, with the most well-known implementations being JoinMarket and Wasabi Wallet.
But in general law enforcement and investigators are definitely wising up to cryptocurrencies and to be fully untraceable one has to go through a lot of hoops and not make a single mistake in the process. Even the above mentioned approaches can leak information that can be used to tie an individual to the transactions if not executed properly.
Most likely they will use the same tried and true approach they would have used for stolen fiat funds; identity theft. I personally know people who have been drawn into a criminal investigation for money laundering because they had been initiated a transaction selling BTC on LocalBitcoins via bank transfer (unwise unless you know and trust the person), turned out already stolen BTC had been converted into fiat on a compromised bank account, which was then supposed to be converted into "clean" BTC again. Fortunately the investigation was already underway when the transaction happened, my friends bank account was blocked as a result when the transfer was initiated and the whole thing was sorted out in the end.
2FA is great for the web UI, but none of these vendors make it particularly easy to enforce 2FA on the command line.
I use Krypt.co. It stores my private key in my mobile device only. I can hook up any device to use it as my SSH key, but the key never leaves the device. Instead, it signs all requests only once I authorize them interactively.
An ssh key is 2FA
An SSH key is one factor. There are various methods for protecting the key with additional factors, but none of the git hosts provide a way to require those additional factors. So as an org owner you're left either trusting every one of your users not to get sloppy with keys, or installing spyware on their computers to make sure they're not using unprotected keys.
How so? An SSH key is a single factor. You could argue that a password-protected private key provides a second factor, but that still falls in the category of "something you know."
How many people can recite their SSH key? Surely an SSH key is "something you have".
Having two different static passwords on an account isn't actually two different factors, whether you can recite them or not.
The fact that one time passwords expire and change is what makes them a different factor than a static password.
The fact that one time passwords expire and change is what makes them a different factor than a static password.
> The fact that one time passwords expire and change is what makes them a different factor than a static password.
If you're getting your 2FA code by SMS message or the like, this can be true.
If you're using TOTP (e.g. Google Authenticator), that's just as static as your other passwords. The TOTP code never expires nor changes. What changes is the code you're supposed to send over the wire.
If you're getting your 2FA code by SMS message or the like, this can be true.
If you're using TOTP (e.g. Google Authenticator), that's just as static as your other passwords. The TOTP code never expires nor changes. What changes is the code you're supposed to send over the wire.
Eh? TOTP usually expire in 60 seconds, so in most cases even if you accidentally leak it, it will be safe.
(and you are not likely to leak it anyway -- with something that changes that often, you are not going to have an incentive to write it to files)
(and you are not likely to leak it anyway -- with something that changes that often, you are not going to have an incentive to write it to files)
A 60-second TOTP code is a fully deterministic function of a permanent, unchangeable secret. That's why you and the server can agree on what the code should be without needing to communicate beyond setting up the code originally.
This makes it identical to a password from a theoretical perspective. There's really no difference between a TOTP secret that you keep in a TOTP app and haven't memorized, and a password you keep in your password manager and also haven't memorized. Both are "something you know", and nothing else.
You're correct that leaking a temporary code from a single login attempt doesn't compromise the TOTP secret. That is an artifact of the login process, not of whether the mechanism is labeled "2FA" or "password". You can do the same thing while calling the secret a password: https://en.wikipedia.org/wiki/Secure_Remote_Password_protoco...
This makes it identical to a password from a theoretical perspective. There's really no difference between a TOTP secret that you keep in a TOTP app and haven't memorized, and a password you keep in your password manager and also haven't memorized. Both are "something you know", and nothing else.
You're correct that leaking a temporary code from a single login attempt doesn't compromise the TOTP secret. That is an artifact of the login process, not of whether the mechanism is labeled "2FA" or "password". You can do the same thing while calling the secret a password: https://en.wikipedia.org/wiki/Secure_Remote_Password_protoco...
I disagree, I believe TOTP belongs firmly in the "something you have" category. You cannot memorize TOTP password, nor you can store in your password manager. You also cannot pass that knowledge to another person. So this is more like a public key than a password.
Ultimately, everything is "permanent, unchangeable secret", including private key and biometric data. Where the data is stored and how is it accessed makes all the difference.
I could not find the original definition of "something you have", but modern standards like PCI actually give OTP auth as an example of "something you have" (p. 4 of [1])
(I am not looking at the degenerate case of running TOTP app on the same device / same security domain -- it does not describe most cases, and there are some fairly straightforward technical measures to defeat this)
[1] https://www.pcisecuritystandards.org/pdfs/Multi-Factor-Authe...
Ultimately, everything is "permanent, unchangeable secret", including private key and biometric data. Where the data is stored and how is it accessed makes all the difference.
I could not find the original definition of "something you have", but modern standards like PCI actually give OTP auth as an example of "something you have" (p. 4 of [1])
(I am not looking at the degenerate case of running TOTP app on the same device / same security domain -- it does not describe most cases, and there are some fairly straightforward technical measures to defeat this)
[1] https://www.pcisecuritystandards.org/pdfs/Multi-Factor-Authe...
> You cannot memorize TOTP password, nor you can store in your password manager. You also cannot pass that knowledge to another person.
But none of these things are true. For example, my most recent job involved sharing a 2FA-protected online account. We all had the code.
But none of these things are true. For example, my most recent job involved sharing a 2FA-protected online account. We all had the code.
Sorry, "none of these things are true" in your environment. They are certainly true for other people, in fact, I bet they are true for majority of them.
I think analogy to physical house keys is very helpful. What did your work do?
Did you show the enrollment QR code, and multiple people scanned it --> this is like duplicating house key.
Did you put the key into password manager -> this is like that combination lockbox that releases house key if you enter the right combination.
People do all sorts of unusual things, this does not change the properties of intended usage.
I think analogy to physical house keys is very helpful. What did your work do?
Did you show the enrollment QR code, and multiple people scanned it --> this is like duplicating house key.
Did you put the key into password manager -> this is like that combination lockbox that releases house key if you enter the right combination.
People do all sorts of unusual things, this does not change the properties of intended usage.
> They are certainly true for other people, in fact, I bet they are true for majority of them.
Well, no. Everyone who uses TOTP, without exception, has their secret stored in a password manager. That's what the TOTP app or device is.
Well, no. Everyone who uses TOTP, without exception, has their secret stored in a password manager. That's what the TOTP app or device is.
> nor you can store in your password manager
Some 2FA apps also allow you to back up your codes to a cloud service.
Some 2FA apps also allow you to back up your codes to a cloud service.
Yes, but I put it into category of "unsafe things that defeat the point of mechanism"
For example, you can put your spare house key under doormat. This effectively makes a lock on your house door require "something you know" (you need to know where the key is stored).
However, that does not mean that we can say that all keys are "something you know". The fact that many people decided to compromise their security does not reflect on other intended use of locks and keys.
For example, you can put your spare house key under doormat. This effectively makes a lock on your house door require "something you know" (you need to know where the key is stored).
However, that does not mean that we can say that all keys are "something you know". The fact that many people decided to compromise their security does not reflect on other intended use of locks and keys.
Agreed. But I think it goes to show how blurry the line is between "know" and "have."
A TOTP challenge is also "something you know", which is a really large portion of total "2FA".
[deleted]
Exactly. PATs are designed to circumvent human intervention (MFA) for authentication in order to support automation. I am very curious if there's a better way than PATs.
> Otherwise, you can still clone the repository and make use of: git reflog or git fsck to find your last commit and change the HEAD.
I don't understand: when I clone a repo, I get a copy of all the branches/tags and the commits they point to & the trees/blobs from those commits. If the repo is wiped, I get a single master branch with a single commit with a single tree and a single blob, and no reflog because that is local to the repo, and I (as a fresh cloner) haven't updated any refs.
Perhaps they are thinking about a mirror clone? That still won't include the reflog, but you can at least find dangling commits and guess which one was master.
I don't understand: when I clone a repo, I get a copy of all the branches/tags and the commits they point to & the trees/blobs from those commits. If the repo is wiped, I get a single master branch with a single commit with a single tree and a single blob, and no reflog because that is local to the repo, and I (as a fresh cloner) haven't updated any refs.
Perhaps they are thinking about a mirror clone? That still won't include the reflog, but you can at least find dangling commits and guess which one was master.
Oops, mirror clone only includes reachable commits. No idea what the 'Otherwise' clarification means then, it sounds bogus
I didn't see it mentioned in the article, but did any of the 3 companies confirm that the repos have been actually cloned as the attackers suggest?
[deleted]
I just have to say, props to Gitlab for being included in this. For a lot of enterprises that use Github and Bitbucket, this may be their first into to Gitlab.
I really don't understand what they are talking about. It's as if someone showed me a photo of my child and said, "pay me or I'll burn this photograph".
What am I missing?