If you don't see the difference between proprietatory format like docx and you argue that just because I add double-asterisk with a word in-between then suddenly it's not plain-text file (since it's markdown markup at this point) then yeah, this conversation is over.
By transforming file Foo.docx into file Bar.b64 you get a plain-text file (Bar.b64) but Foo.docx still isn't plain-text. That's actually how email attachments work (transforming any file into b64 plain-text file), so I think your counter-argument is pointless.
Open up a word document (.docx) in hex editor (head -80 foo.docx | xxd) and you'll see it's not a plain file on a binary level. Now do the same with any python, php, html, and js files and you'll see that they're plain files.
That's the difference that Derek is talking about.
You can use grammarinator or any other of myriad grammar fuzzers BUT I'd start with radamsa and its string-related flags. Radamsa is _awesome_, you run it on an input and you get mutated output. Works both for binary files and text fles (such as grammars).
OP here. I'm planning to do so, however it will require more work (better description of the problem, wider description of viable solutions, additional case studies). Most probably it will land on Medium and Dev.to.
Yes you will, but only because GitHub already recognised this class of problems and came up with their own solution [1]. Bear in mind that it works only for vendors that integrated, so while it's true for AWS it might not be for your FOO API.
Hey, OP here. I agree that a blog post would be more readable. In this particular case I just didn't expect that it will catch fire. If I would then I would spend more time on the form. I won't make that mistake again (i.e. in the future I will use a blog post as main driver of such twitter thread).
Greetings fellow Hackers! OP here. I see that my experiment got some traction which means more awareness should be spread about this class of bugs.
For starters I recommend reading "How Bad Can It Git" [1] and "Detecting and Mitigating Secret-Key Leaks inSource Code Repositories" [2] papers.
After that you can read "How I made $10K in bug bounties from GitHub secret leaks" [3] and some notable reports on HackerOne Hacktivity [4] [5] and [6]. This last one is interesting - leaking secrets is not only about code repository! Actually it's about entire toolset used for software development, hence secret scanning could (should?) be performed for other places such as CICD logs or even Slack messages [7].
Anyhow, back to code repositories. GitHub and GitLab both recognized secrets as a problem, so they came up with solutions. If you use GitHub you can easily integrate GitGuardian [8] into your workflow ($$$) but even if you don't GitHub provides you with Secret Scanning feature [9] (both are mentioned within the Twitter and HN threads). If you use GitLab you have a Secret Detection feature [10] at your disposal BUT in order to use it you need to setup Auto DevOps (that's why in my experiment GitLab didn't alert me - I just pushed commits to my public repo but didn't setup anything).
Apart from built-in solutions provided by GitHub and GitLab, one can use tooling of their own choice. For this I'd recommend two types of solutions: proactive and reactive. For proactive security, as mentioned in the Twitter thread, you can use Talisman [11] as pre-commit hook. For reactive security you can use GitLeaks [12] (used by GitLab) or similar tools - there are many of them but one stands out, namely truffleHog [13] which can sniff each and every commit across all branches (also used by GitLab).
What if you already commited a secret into the public repository? Start with revoking and continue with this tutorial [14]
You mean adversaries? No. For token generation I used https://canarytokens.org/ so the only information I got was abou triggering the token, but not the context in which it was triggered.
BTW. GitHub (apart from GitGuardian) also has Secret Scanning feature [1] that basically allows the provider to act on the leaked secret. Amazon is integrated and it should invalidate and inform the owner but this also went to Thinkst, not me, so I don't know if it was actually invalidated and alerted.