There was no useful math after 1964? What? Or do you not count the entire field of computer science as math (arguably FFT belongs to computer science too)?
Linux is the easiest thing in the world to get working, especially with AI. Especially if you need to actually be productive and build software.
The accusations of "programmers just wanting job security" is weird too. I use AI all the time to configure stuff. The ecosystem of linux is just frictionless if you aren't looking to customize every little thing, while exposing what you want.
I mean, people always say there are tradeoffs, until you reach the next frontier, in which there are tradeoffs at said frontier, and the next, and the next, etc.
In one sense, yes, tradeoffs are inescapable as the scope expands to the maximal possible scope. In another sense... it depends on the level of abstraction we're talking about.
I highly respect many people at cognition but yeah that's put a sour taste in my mouth.
I want to work in the AI space on actual AI research, at any part of the stack. Even if I'm developing training infra - as long as people are advancing knowledge of what intelligence could be.
But it seems like either it's big labs or grifters, that's it, and even the big labs, at least publicly, seem very grifty at times. Not like I have the technical chops probably, but still.
From an interoperability perspective, this breaks the advantage of LLM inference that frontier AI labs have, in that you just have everyone run through the same algorithm but configure via text.
If you added probes at the model layer, you have to serve multiple different types of kernels at the same time, for multiple different companies and use cases (I guess you could provide a standardized set of probes for users), start tracking version control for each of the kernels, etc. very nasty compared to right now.
Could be a really interesting problem in the next 10 years or so, but this would require labs to be far more open about their models; and labs are still shooting for their AGI anyways, with the idea that nothing you suggest right now matters if AGI exists in a decade.
I mean mathematicians have encoded a decent amount of the operations you mean for messy versus true / false. Turing machines are built from mere Boolean logic, but also notions of compression, information theory, projections, function mappings, etc. provide all the math-crank language you want to reformulate knowledge work.
The issue is that mathematics really does not have a good answer for how things update.
You could redefine language to include mental abstractions and maybe it wouldn't be so weird.
Language is a shared medium. Maybe an intelligence is capable of building abstractions in their head without the need for say, a symbol system to hopefully communicate with species similar to itself.
You may argue that the agent needs persistent memory. Well, so what? If the world has stable regularities from the perspective of the agent - if the sun rises every day, and language is associating certain visuals with certain thoughts (it's not when viewed in its original social definition - but conduct the thought experiment with sociality removed), who's to say that you can't form a kind of language around that?
Of course, this is all armchair, and I'm not trying to separate out what "real language" is or whatever, just thought experiments.
But the original poster has a good point though, that's been proven in papers, that the tokens an LLM spits out, themselves, attract them to certain distributions, and at least a decent amount of the tokens in CoT are there solely to bring itself to the right distribution.
Parse, don't validate is one way of building constraints. The issue is it feeds into a tree-based view of constraints. However it does yield the philosophy of "constraints by construction".
Another is making a set of "linearly independent" configurations - except in practice it never is, is it? Has anyone actually ever had a clean CI Matrix that didn't have weird hidden edge cases, for example?
Functional programming really wants to emphasize the notion of pure functions, which have modularity and independence built in. But there are perf issues and in practice, you don't really escape the issues of "how to design constraints". Sure, you don't need inheritance and OOP and all of that, but you can easily have a tree-based view of constraints and ontology in FP as well.
(Incidentally, my view of the issue with something like Carnap's logical frameworks is that they are so general and flexible that they fail to capture anything operationally useful; yes, I know that isn't always philosophy's goal but I view the same with a lot of purported theories of everything today)
Are there any other philosophies in software that have certain distinct wins versus losses when it comes both to the organization & encoding of your constraints, and coming up with them? Tree-based hierarchal decomposition and linearly independent axes in a space are two go-to things for me.
I suppose you could design a state machine, but that requires understanding all the semantics upfront, encoding them once, and hoping that requirement changes don't mess you up.
I have seen poset-based solutions as well (actually, I think "monotonic" distributed architectures are based around this approach) but that obviously requires a very specific type of problem domain.
----
There are also some very common memes from physics-swes: such as how information cancels out over "long distances" and therefore certain kinds of abstractions are good; attractor states in idea space; or even people loving the idea of symmetry (which, granted - in physics, is truly a beautiful approach, but does not seem to generalize well to generic software engineering). But those are a bit too high level to put into a concrete software plan. Still interesting though.
"Encoding" implies some GOFAI symbolic formal rule machinery.
I'd argue that transformers are a pretty good indication that intelligence isn't "encodable" in the way we think it means. Usually, most "model" vocabulary means that we can explain and constrain the "data" from the "rules". Except the mere "data" is trillions of interacting weights.
That may be encoding in a physical sense, but that still doesn't explain the intuition in any legible way to humans.
Cynically, we've been able to encode everything already by just saying everything's a transition in a huge lookup table. Not very informative though.
Exactly. Every single philosophical statement in history runs up against the issue where you can just say, "yeah, it's pretty much this. You just need to do <arbitrarily hard unspecified thing that is basically unfalsifiability>". (Including this one)
And maybe that's just our limits with philosophy, modeling, assumptions, whatever. The danger is not realizing when we're in that zone.
(Fwiw I think unfalsifiability is a limit with any system - "you didn't compile in my syntax/semantics" is an gotcha that's actually valid and useful, but nobody can really determine the hard line)
I think it takes a very specific kind of person to read Crime and Punishment.
So, as a baseline, I think most people have or can understand internal monologues. That's not what I mean, though that is a prerequisite.
But many real-life people, especially those that have gone through phases in their life where they were Raskilnikov (not criminals, not necessarily egomaniacs, but the whole melodramatic shut in deal) would tell you that they both understand Raskilnikov type people and would tell them to shut up.
For me, it was honestly a bit depressing. Raskilnikov reminded me of me in my worst moments. Honestly, a lot of the characters did. Having these strong, abstract, high and lofty ideals is contrasted against the real, practical characters like Rahmuzkhin. Every single one of the lofty idealists (besides maybe the full commune living guy - what he says is weird, but not his actions) is contrasted with the people on the ground, doing good work. Even Sonya - she's devout, but not so devout as to become a pastor, abstractly preaching about goodness and kindness, but blind to the suffering around her.
And isn't that what the lesson is at the end of the book, anyways? (trying to be vague to avoid spoilers).
Though it's not like just "doing good work" will bring you the sort of the "ultimate" that many of these characters seemed to have wanted. Once you try to formalize it and intellectualize it, you can point to how Crime and Punishment is such an illogical novel. And yet it feels so real.
That's different though - those are services you explicitly agree to and sign up for, be it at checkout, be it at service signup time, be it because you are making a google account on the google platform.
For example, a github cicd automerge pipeline is still good.
To be fair, labs silently nerf models all the time.
Fable's probably objectively better at full power. I mean, I definitely felt the same difference in competency between Fable and current Opus. But Opus itself has definitely been nerfed, and Fable, even if it comes back the public forever (probably won't), will get nerfed.
I have noticed absurd lag from the browser usage and sometimes complete bricking of my network too on my computer. I thought it was just my computer getting old, but possibly it's ChatGPT.