It is unclear this model should be on that leaderboard because we don't know whether it has been trained on mteb test data.
It is worth noting that their own published material [0] does not entail any score from any dataset from the mteb benchmark.
This may sound nit picky, but considering transformers' parroting capabilities, having seen test data during training should be expected to completely invalidate those scores.
Though not relevant to the Atlantic ocean, it is worth noting that the Australian coral reef has in fact recovered from the disastrous state it was after the bleaching and damage events in 2016/2017. Reef coverage is in fact at the highest levels ever recorded in some areas.
It does make one wonder why this more recent good news has not found the same level of exposure as when the situation was bleak back then. Perhaps good news just doesn't gather as many clicks.
I believed the author of the blog was referring to tech roles specifically (not sales or similar).
I can say from personal experience that, at least in Australia, tech workers for mining companies that work in city offices are paid fairly similarly to other non-FAANG tech workers (e.g. banks etc). I also just checked levels.fyi for a few big mining companies and verified that this is the case.
Engineers (mining, geo, tech, whatever) that work out in the field do make quite a bit more. MAYBE around what FAANG would pay, but FAANG still pays more after a few years of refreshers and/if one manages to climb the ladder.
I think the author is quite off with their estimates of what people make in "heavy industries". At least as far as my experience goes for AU. FAANG/HFT still beats everything, and by a vast margin.
Though I agree with the idea that MLPs are theoretically more "capable" than transformers, I think seeing them just as a parameter reduction technique is also excessively reductive.
Many have tried to build deep and large MLPs for a long time, but at some point adding more parameters wouldn't increase models' performance.
In contrast, transformers became so popular because their modelling power just kept scaling with more and more data and more and more parameters. It seems like the 'restriction' imposed on transformaters (the attention structure) is a verg good functional form for modelling language (and, more and more, some tasks in vision and audio).
They did not become popular because they were modest with respect to the parameters used.
The reason why people care about privacy is not necessarily because giving up privacy has some directly observable negative effect. But, simply, living without freedom sucks.
I don't want you to know my personal information not because you could/would do something nefarious with it. I don't want you to have it simply because it's none of your business.
It's the first mobile app I have ever written and I enjoyed the process quite a bit!
My main goal was to deliver better identification accuracy than similar apps.
However I also wanted to provide useful plant information along with the identification and naively thought that this would have had to be a solved problem - surely there would be some online DB with all plants data neatly organized (I'd be even happy to pay for it!), in particular plant care information - but alas!
Lack of differentiability is actually a very important feature of the underlying optimization problem.
You might think that it doesn't matter because ReLU is, e.g., non-differentiable "only at one point".
Gradient based methods (what you find in pytorch) generally rely on the idea that gradients should taper to 0 in the proximity of a local optimum. This is not the case for non-differentiable functions, and in fact gradients can be made to be arbitrarily large even very close to the optimum.
As you may imagine, it is not hard to construct examples where simple gradient methods that do not properly take these facts into account fail to converge. These examples are not exotic.
Only tangentially related, but I have been trying to enroll for Apple's developer program for almost 3 months now.
Understanding what the problem is is essentially impossible. Going to a physical store doesn't help, calling their customer service has them telling you to go to www.apple.com/support (???), and writing for support has them rotate you through 4 different, and decreasingly useful, representatives.
The last response I got I was told the issue had to be handled by yet a different representative and it would take an "indefinite amount of time". Which may be a nice way of them saying it's never going to happen.
It really is demoralizing when you realize there is nothing you can do really, even in cases when you have done nothing wrong.
It is worth noting that their own published material [0] does not entail any score from any dataset from the mteb benchmark.
This may sound nit picky, but considering transformers' parroting capabilities, having seen test data during training should be expected to completely invalidate those scores.
[0] see excel spreadsheet linked here https://blog.voyageai.com/2024/09/18/voyage-3/