For most tasks, at some future date, isn't there going to be some ambient baseline of capabilities you can get per $/tok, starting at ~0 for OSS models, such that eventually all tooling gets trivially transferable?
OP is correct; surprisal is outcome-dependent and entropy is distribution-dependent
- entropy is E_p[informativeness of measuring outcome x]
- take n outcomes, then a distribution over them lives on the simplex \delta ^ (n - 1). you can lift this to R^n via the log odds map p_k -> x_k = log p_k -- now x \in R^n can describe a histogram with n-1 degrees of freedom
- in log odds space, measurement is literally a linear functional from vector space of log probability onto the index of the outcome k.
- imo surprisal of some p(x) is best understood as "the length of a pointer", entropy "the rarity-weighted average length of a pointer", and collision entropy "how specific you would have to be to describe witnessing a specific outcome"
and in the same way, a single molecule of water, you might get by, calling dry
I meant in the sense of - you have benchmarkers and trainers. If you publicize your evaluation, trainers may likely have their models 'consume' it, even if only indirectly: another person creating their own benchmark from scratch may be influenced by yours, even if the new question sets are clean-room. That, and the rule of thumb that benchmark value dissipates like sqrt(age) [0]
So there is a definite advantage to never publicizing your internal benchmark. But then, no one else can replicate your findings. You should assume that the space of benchmarks that are actually decent at evaluating model performance is much larger and most of the good ones, the ones that were costliest to produce, are hidden, and might not even correspond very well with the public ones. And that the public expensive benchmarks are selective and have a bias towards marketing purposes.
There’s a Dark Forest problem for evals. As soon as they’re made public they start running out of time to be useful. It’s also not clear how to predict how the model will perform on a task based on an eval. Or even whether, given two skills that the model can individually do well on in the evals, it still does well on their composition. It might at this point be better to be scientific in unscientific approaches, than to attribute more power to relatively weakly predictive evals than they actually have
I mean. Sounds like the guy had existing long term goals, needed to overcome an activation threshold, and used AI as a catalyst to just get started. Seems like, behaviorally, AI was pivotal for him to learn things, even if the things he learned came from elsewhere / his own effort.
The more general question of how to evaluate the quality of a given skill file is quite interesting to me. A skill may prime a model's responses in a way that a prompt alone may not. But also models aren't good at judging what they are or are not capable of.
Just asking a model "how good is this skill?" may or may not work, possibly the next laziest thing you could do - that's still "for cheap" - is asking the model to make a quiz for itself, and have it take the quiz with and without access to the skill, then see how the skill improved it. But there's still many problems with that approach. But would it be useful enough to work well enough much of the time for just heuristically estimating the quality of a skill?
The possibility of intelligent machines undergoing transformative regeneration actually dates back to a party hosted by one Charles Babbage where, in attendance, was one Charles Darwin, who only thereafter published On the Origin of Species
This x thread may not be the best source of clarity on what is actually being default opted-into. Sorry. I looked into it and it seems that Starlink denies browsing history would be shared [0]. Seems I can't edit the title any more.
> Do you share my personal information for AI training?
We are committed to protecting your privacy. In some instances, we may share personal information with trusted third-party partners who, among other activities, help us develop AI-enabled tools that improve your customer experience, although you can always opt out. Rest assured that we take reasonable safeguards to protect and secure your information whenever it is used or shared.
> Will these AI models see my Internet history?
No, your internet history will never be shared with AI models, including individual browsing habits or geolocation tracking, and we comply with laws prohibiting unauthorized surveillance.
> What personal information does Starlink collect from me?
We only collect what’s needed to provide you great service—like your name, address, email, and payment details when you sign up or order. We also gather some technical information (like IP address or service performance data) to keep your connection fast and reliable.
- you can solve neural networks in analytic form with a hodge star approach* [0]
- if you use a picture to set your initial weights for your nn, you can see visually how close or far your choice of optimizer is actually moving the weights - eg non-dualized optimizers look like they barely change things whereas dualized Muon changes the weights much more to the point you cannot recognize the originals [1]
*unfortunately, this is exponential in memory
[0] M. Pilanci — From Complexity to Clarity: Analytical Expressions of Deep Neural Network Weights via Clifford's Geometric Algebra and Convexity https://arxiv.org/abs/2309.16512
I have no idea how to define it. I also don’t know if I’m personally convinced one way or another about the harms. Just think the platforms would probably have to be made to make more substantial changes were it the case.