I had the same problem and had some luck cleaning things up by enabling "calculate all sizes" in Finder, which will show you the total directory size, and makes it a bit easier to look for where the big stuff is hiding. You'll also want to make sure to look through hidden directories like ~/Library; I found a bunch of Docker-related stuff in there which turned out to be where a lot of my disk space went.
You can enable "calculate all sizes" in Finder with Cmd+J. I think it only works in list view however.
https://archer.fandom.com/wiki/Mr._Deadly_Goes_To_Town