EigenLord · HackerTrans

9 เดือนที่ผ่านมาdiscuss

I've always felt that you can't think in anything besides thought. Words, images, symbols, etc, are all side-effects. They absolutely bend back and influence the thought process, but they are always secondary and indirect. Thought itself is ineffable.

EigenLord

9 เดือนที่ผ่านมาdiscuss

Interesting work but this strikes me as a somewhat quixotic fight against inevitable tendencies of statistical models. Reinforcement learning has a single goal, an agreeable mean. Reinforcement learning stops when the LLM produces agreeable responses more often than not, the only way you can achieve absolute certainty here is if you tune it for an infinite amount of time. I also don't see how this method couldn't be subsumed by a simpler method like dynamic temperature adjustment. Transformers are fully capable of generating unpredictable yet semantic text based on a single hyperparameter. Maybe it would make more sense to simply experiment with different temperature settings. Usually it's a fixed value.

EigenLord

9 เดือนที่ผ่านมาdiscuss

I think there's a critical flaw with Anthropic's approach to memory which is that they seem to hide it behind a tool call. This creates a circularity issue: the agent needs to "remember to remember." Think how screwed you would be if you were consciously responsible for knowing when you had to remember something. It's almost a contradiction in terms. Recollection is unconscious and automatic, there's a constant auto-associative loop running in the background at all times. I get the idea of wanting to make LLMs more instrumental and leave it to the user to invoke or decide certain events: that's definitely the right idea in 90% of cases. But for memory it's not the right fit. In contrast OpenAI's approach, which seems to resemble more generic semantic search, leaves things wanting for other reasons. It's too lossy.

EigenLord

9 เดือนที่ผ่านมาdiscuss

I wish more comp sci curricula would sprinkle in more general courses in logic and especially 20th century analytic philosophy. Analytic philosophy is insanely relevant to many computer science topics especially AI.

EigenLord

ปีที่แล้วdiscuss

I've been thinking lately about how AGI runs up against the No Free Lunch Theorem. This is what irritates me: science is not determining the narrative. Money is. I highly recommend mathematician David Wolpert's work on the topic. I think he inadvertently proved that ASI is physically impossible. Certainly he proved that AOI (artificial omniscient intelligence) is impossible.

One thing he showed is that you can't have a universe with two omniscient intelligences (as it would be intractable for them to predict the other's behavior.)

It's also very questionable whether "humanlike" intelligence is truly general in the first place. I think cognitive neurobiologists would agree that we have a specific "cognitive niche", and while this symbolic niche seems sufficiently general for a lot of problems, there are animals that make us look stupid in other respects. This whole idea that there is some secret sauce special algorithm for universal intelligence is extremely suspect. We flatter ourselves and have committed to a fundamental anthropomorphic fallacy that seems almost cartoonishly elementary for all the money behind it.

EigenLord

ปีที่แล้วdiscuss

Diffusion is just the logically most optimally behavior for searching massively parallel spaces without informed priors. We need to think beyond language modeling however and start to view this in terms of drug discovery etc. A good diffusion model + the laws of chemistry could be god-tier. I think language modeling has the AI community's in its grips right now and they aren't seeing the applications of the same techniques to real world problems elsewhere.

EigenLord

ปีที่แล้วdiscuss

Years ago, in my writings I talked about the dangers of "oracularizing AI". From the perspective of those who don't know better, the breadth of what these models have memorized begins to approximate omniscience. They don't realize that LLMs don't actually truly know anything, there is no subject of knowledge that experiences knowing on their end. ChatGPT can speak however many languages, write however many programming languages, give lessons on virtually any topic that is part of humanity's general knowledge. If you attribute a deeper understanding to that memorization capability I can see how it would throw someone through a loop.

At the same time, there is quite a demand for a (somewhat) neutral, objective observer to look at our lives outside the morass of human stakes. AI's status as a nonparticipant, as a deathless, sleepless observer, makes it uniquely appealing and special from an epistemological standpoint. There are times when I genuinely do value AI's opinion. Issues with sycophancy and bias obviously warrant skepticism. But the desire for an observer outside of time and space persists. It reminds me of a quote attributed to Voltaire: "If God didn't exist it would be necessary to invent him."

EigenLord

ปีที่แล้วdiscuss

I think the answer to the professor's dismay is quite simple. Many people are in university to survive a brutal social darwinist economic system, not to learn and cultivate their minds. Only a very small handful of them were ever there to study Euler angles earnestly. The rest view it as a hoop they have to jump through to hopefully get a job that might as well be automated away by AI anyway. Also viewed from a conditional reinforcement perspective, all the professor has to do is start docking grade points from students who are obviously cheating. Theory predicts they will either stop doing it, or get so good at it that it becomes undetectable-possibly an in-demand skill for the future.

EigenLord

ปีที่แล้วdiscuss

I would love it if LLMs told me I'm wrong more often and said "actually no I have a better idea." Provided, of course, that it actually follows up with a better idea.

comments