hey I’m just spinning up in ssl birdsong models (BirdMAE, SongMAE, etc) can you share any resources? My email is [email protected], would love to read your work.
I’m really excited to see some more open SAE work! The engineering effort is non trivial and I’m going to check out your dataloading code tomorrow. You might be interested in an currently in-progress project of mine to train SAEs on vision models: https://github.com/samuelstevens/saev
Very odd that they don't do linear probing on the model features and measure ImageNet validation accuracy. In general, this paper is missing comparisons to existing vision models on existing benchmarks.
The visual prompting is a neat trick and it's great to see scale continue to work, but without comparisons to other models or releasing the code/weights, I don't think this strategy is going to be competitive.