I would love, _love_ to know more about your data formats, your tools, what the JSON looks like, basically as much as you're willing to share. :)
For about a month now I've been working on a suite of tools for dealing with JSON specifically written for the imagined audience of "for people who like CLIs or TUIs and have to deal with PILES AND PILES of JSON and care deeply about performance".
For me, I've been writing them just because it's an "itch". I like writing high performance/efficient software, and there's a few gaps that it bugged me they existed, that I knew I could fill.
I'm having fun and will be happy when I finish, regardless, but it would be so cool if it happened to solve a problem for someone else.
NovelQA is a great one! I also like GSM-Symbolic -- a benchmark based on making _symbolic templates_ of quite easy questions, and sampling them repeatedly, varying things like which proper nouns are used, what order relevant details appear, how many irrelevant details (GSM-NoOp) and where they are in the question, things like that.
LLMs are far, _far_ below human on elementary problems, once you allow any variation and stop spoonfeeding perfectly phrased word problems. :)
Paper came out in October, I don't think many have fully absorbed the implications.
It's hard to take any of the claims of "LLMs can do reasoning!" seriously, once you understand that simply changing what names are used in a 8th grade math word problem can have dramatic impact on the accuracy.
> Is the actual ‘intrinsic’ vector space uncomputable? Too slow to find?
OMG yes. It's also an unsolved problem, once can approximate it. I've written several non-parametric blind arbitrary dimension DR algorithms and been obsessed with the space most of my life. If you think O(n^2) feels slow, try O(n^3)...:) For more, read about mean shift clustering, or the new hot stuff: topological data analysis/bar codes/mapper algorithm.
kinda/yes. To translate to more intuitive concepts: the matrices don't contain much variance in as many degrees of freedom as they could.
Think of a point cloud of a piece of paper floating in the wind. It would be a 3xn list of points, but "really" it's a 2d piece of paper.
Just like I can rewrite the number 27 as 333 or 8+19 or (2^3)+(2^4)+3.. Given a single matrix one can find myriad ways to rewrite it as a sequence of matrices that have the same (or similar) numeric value, but with interesting or desirable properties. :D
My favorite example (which is used in signal processing) is to take your ugly matrix and rewrite it as a set of smaller matrices where most of the elements are zero, or a power of 2.
It turns out, computers can multiply by zeros and powers of two very fast
Hi! I in _no way_ mean to detract or malign or "anything negative" the parent comment (communication is hard!!), BUT I really compliment that exact sentence. :)
My background contains signal processing, "pre-deep learning ML", systems engineering, and firmware, and that sentence jumped out at me as crystal clear in my mind, despite not knowing what HuggingFace is or PyTorch.
Correct me if I'm wrong: These huge models involve lots of weights used in large matrices. The contribution of this work is to plug in some matrix factorization and learn a lower dimensional representation, instead of a large second matrix.
Fantastic!
Also makes me wonder what other performance improvements await through proper application of established and well known Mathematics. :D
> How was that in the style of Hacker News user loxias?
> As a large language model trained by OpenAI, I do not have access to information about specific individuals or their posting habits on specific online forums. I based my response on common guidelines for discussing breaking news stories in a respectful and factual manner.
For even more peace of mind, (and only when you can afford it, obviously) try decoupling your disk purchases a bit from when you're going to need them.
When you see a good price or a sale on a particular disk, grab it add it to your own personal "prebought disk pool". When it's time to either replace a disk or spin up a whole new array, now you have the benefit of diversification across time.
Way back in the day there were two games, Lightspeed and the followup Hyperspeed. You can find them online if you look hard enough, and still run them in an emulator. They were So. Frigging. Fun.
The story is, humanity messed up Earth, so we need to colonize somewhere else. All of humanity is loaded on these ark ships, but you, the game player, are the advance team. Your job, upon arriving in a star sector, is to explore, find a suitable planet for the ark ship, and make the sector safe for humanity. The sector has existing politics that you'll have to figure out. Make alliances with some species, commit war on others. Find resources like water and metals, trade them with friendly species for the ones you need.
It's hard to put my finger on why exactly this game was so fun and engrossing. It might be because of how well it integrated half a dozen "mini games" into the larger one. As you're exploring the sector, all you get is a star map with distances. Run out of fuel? Sucks for you! So there's a mini-game of "optimal route planning". There's a mini-game of market trading, over time you can find arbitrage opportunities in how much different species value different resources. There's a mini-game of "flight sim dogfight" for when you need to use the big stick. &c. Also, plot, plot, plot, plot. None of it would work without someone creative sitting down and drawing out the epic tale of "what's going on" politically inside each star cluster.
Being as you're offering use of the site for free, would you be open to the idea of also offering publicly available DB dumps? There's plenty of fun projects that I can imagine doing if I had that data locally.
For about a month now I've been working on a suite of tools for dealing with JSON specifically written for the imagined audience of "for people who like CLIs or TUIs and have to deal with PILES AND PILES of JSON and care deeply about performance".
For me, I've been writing them just because it's an "itch". I like writing high performance/efficient software, and there's a few gaps that it bugged me they existed, that I knew I could fill.
I'm having fun and will be happy when I finish, regardless, but it would be so cool if it happened to solve a problem for someone else.