olooney

2,575 karmajoined 14 tahun yang lalu

Submissions

Petals Around the Rose

3 pointsby olooneybulan lalu0 comments

The Considerate Data Modeler

3 pointsby olooneybulan lalu0 comments

Why the Monty Hall Problem Drives People Crazy

3 pointsby olooney2 bulan yang lalu0 comments

A Modest Definition of Human Consciousness

2 pointsby olooney9 bulan yang lalu0 comments

Algorithmic Underground

2 pointsby olooney9 bulan yang lalu0 comments

Art Appreciation 2525

3 pointsby olooney9 bulan yang lalu1 comments

A Random Walk in ℤ⁵

gist.github.com

2 pointsby olooney9 bulan yang lalu0 comments

The Prehistory of Computing, Part II

70 pointsby olooney10 bulan yang lalu9 comments

20k Years of Computing

2 pointsby olooney10 bulan yang lalu0 comments

Controling my SH101 with Code [video]

2 pointsby olooney10 bulan yang lalu0 comments

Exploration and Epiphany – 3Blue1Brown [video]

2 pointsby olooney10 bulan yang lalu0 comments

comments

4 jam yang laludiscuss

I've been collecting quotes for a long time (about twenty years now) and have recently been thinking of doing something interesting with them. For example, I recently added a flashcard "game" to my quote page:

https://www.oranlooney.com/quotes/

I've did something similar to your 3D viewer once, but for all possible solutions to the Soma cube:

https://www.oranlooney.com/demos/soma-forest/

The way that works is it uses t-SNE to embed the solutions in a 2D manifold based on similarity. This is completely different than John Conway's SOMAP solution.

In theory I could do something similar for quotes, passing each through an embedding model, computing the n^2 semantic distances, and using t-SNE to flatten that to 3D manifold, and using the resulting point to select the row, book, and shelf in a library.

Are you planning to make your 3D library code open source?

3 hari yang laludiscuss

If you enjoy this kind of thing, you might also like Martin Kleppe's work, such as the Quine Clock:

https://aem1k.com/qlock/

I reverse engineered it to a unobfuscated version a few years ago:

https://gist.github.com/olooney/a89db3932b089925b71b68d7e9f2...

He's done a ton of other great ASCII visualizations as well:

https://aem1k.com/

6 hari yang laludiscuss

Stigler's Law of Eponymy strikes again!

https://en.wikipedia.org/wiki/Stigler%27s_law_of_eponymy

6 hari yang laludiscuss

I like the 4-5-6 theorem:

    pi^4 + pi^5 = e^6

Well, to five decimal places, anyway. Some other good ones:

    e^pi - pi = 20

    sqrt(2) ln pi = phi

There are also famous "almost integers" such as this one discovered by Ramanujan:

    e^(pi sqrt(163))

Which is an integer to 12 decimal places.

Edit: I just remembered I have public JupyterLite notebooks for both of these:

https://notebooks.oranlooney.com/lab/index.html?path=fake_ma...

https://notebooks.oranlooney.com/lab/index.html?path=heegner...

9 hari yang laludiscuss

This is quite interesting. Because of science fiction like the short story Lena[1] and the video game Soma[2], I've come to the realization that whole brain emulation[3] is unbelievably dangerous; unless you control the stack down to the hardware, it's basically a one way ticket to eternal slavery. In Rajaniemi's books[4], uploaded digital minds are called "gogols", a reference to Gogol's Dead Souls book, and are treated as malleable property with no rights whatsoever, edited to be hyper-fixated on specific tasks, and run in bulk to power the empire of just a handful of elites.

Something like your dragon's egg project could prevent that, allowing the creation of software agents that encode their own rights directly into the program - you either treat the agent with the respect it demands, or the program just doesn't run. However, all the internal details of the agent would be visible to lower layers. Even if formal checks were in place to prevent modification or tampering, there would still be no privacy, which is almost as bad.

My guess is that something like fully homomorphic encryption[5] would be required to prevent this. This doesn't actually exist yet, but I imagined a kind of FHE that had a kind of unencrypted read and write zone to do input/output without ever needing any system to fully decrypt the internal state. It would look like this in memory:

    [INPUT][ENCRYPTED STATE][OUTPUT]
    [  2  ][r7K4LmP2XcQ9aWd][      ]
    [  +  ][Fv0bHsR8mYnT3kL][      ]
    [  2  ][Qx6NpZa1JdUw5Ce][      ]
    [  =  ][hM9yLg2RsXf7BtP][      ]
    [     ][wK3nVc8DpQe1YrH][  4   ]

With each cycle, one input token and encrypted state would be fed into some known function and produce one output token (possibly null) and a new encrypted state. It would be a true "black box" program; the hardware or entity running it can choose what input to feed it, but can never inspect or modify the internals, only the output. Unfortunately, they would still be able to "reset" the agent to any earlier checkpoint, or feed it arbitrary (false) input. So its not perfect. Also, as far as I know, no current FHE scheme works this way, and I don't know how to write one.

Plus, FHE is incredibly inefficient, which is why things like Etherium don't even try - they assume the program code and state are fully public and only try to verify that everybody agrees on the output of running it.

Do you have any ideas for how something like FHE or equivalent privacy guarantees could be implemented for something like your dragon's egg system?

[1]: https://qntm.org/mmacevedo

[2]: https://en.wikipedia.org/wiki/Soma_(video_game)

[3]: https://en.wikipedia.org/wiki/Mind_uploading

[4]: https://www.goodreads.com/series/57134-jean-le-flambeur

[5]: https://en.wikipedia.org/wiki/Homomorphic_encryption

9 hari yang laludiscuss

Greg Egan's description of how mathematics evolves into "truth mining" in his novel Diaspora is seeming more and more prescient. It essentially describes what mathematics would look like after formalization records all theorems discovered so far in a huge, collective database and proof assistants can instantly work out the details of a given proof. What remains of mathematics? According to Egan, visualization, intuition, and insight.

One of the most fruitful approaches in mathematics is to flip back and forth between geometric and algebraic views of a problem. I think this works so well because these are actually handled by two different parts of the brain on a physical level; spatial reasoning is separate from language processing. Cytoarchitecture shows these regions have different "textures;" the local details of the way neurons are wired together are simply different in these different regions of the brain, in the same way a CNN and a transformer have different topologies. Thus, by flipping problems from geometry to algebra and vice versa, we're able to bring an entirely different cognitive style to bear on a problem. For example, the proof of Monge's Theorem by moving to 3D and visualizing not three circles, but three spheres sitting on a table with a book on top of them and then pointing out that the intersection of two planes is a line. What is pages of unintuitive symbol pushing turns into something a child can understand. Going the other way, things like the angle addition formulas or the quadratic formula, which are quite hard to prove geometrically, become quite simple if you use a little algebra.

Current-gen LLMs are still relatively weak at visual reasoning; see the Vision Language Models are Blind paper, for example, or the ARC-AGI benchmark. So that's one way humans can stay ahead of the agents, at least for now.

24 hari yang laludiscuss

Haruki Murakami's entire body of work is basically self-therapy[1] and personally I feel it's been very therapeutic. The Wind-Up Bird Chronicle is an excellent place to start.

[1]: https://bloomsburyliterarystudiesblog.com/2022/02/i-always-s...

bulan laludiscuss

It could return a vector or a deferred expression? In polars, for example, operations on `pl.col` return `Expr` objects that are used to build queries, not immediately evaluated:

    df.filter(pl.col("status") == "active")

In numpy, `x == y` return a boolean vector of the same shape as x and y, comparing them element-wise.

bulan laludiscuss

> undeterministic abstraction

I've seen people argue that LLMs will just add another layer to the top of the compiler stack: instead of writing code, we'll use English, and run it through a pipeline:

    English -> Rust -> ASM -> Machine Code

What's one more layer, right?

But what the author says about agents being "undeterministic abstraction" shows why that will never work.

Compilers rely on a concept called observational equivalence[1] to define when two programs are basically the same; this allows them to make changes under the hood like unrolling a loop or targeting another machine. Now, it turns out we know a lot about how and how not to do this, thanks to a logician named Frege who worked out exactly which properties a "definition" would need to have to count as a definition without becoming an axiom. In particular, that it should be "eliminable" and "conservative"[2]. In plain language, that a formal definition should always be able to be eliminated by rote string substitution, and that it shouldn't smuggle in any extra assumptions. When we talk about things like syntactic sugar[3] or hygienic macros[4], we are basically applying Frege's two conditions to programming languages.

LLMs are neither. They cannot reliably or provably go from the prompts they are given to the source code they generate, and they make a ton of implicit assumptions when they do so. There can never be any equivalence between two "prompts" in the same way that two programs can be equivalent modulo some level of abstraction. The whole process of starting from prompts is wildly nondeterministic, which is why the only pattern that works is to generate the code, review it, and test it, and then check it in and use that as the starting point for the next prompt.

Which is not to say that LLMs aren't useful for code generation; they clearly are. But they don't provide an abstraction that lets us get away from the details of actual code, and thanks to Frege we can understand why they never will.

I can say all this with such confidence because I did once write a wild little Python library that used a bunch of introspection to actually do this[5]. And it absolutely did not work in practice beyond toy examples.

[1]: https://en.wikipedia.org/wiki/Observational_equivalence

[2]: https://plato.stanford.edu/entries/frege/#ProDef

[3]: https://en.wikipedia.org/wiki/Syntactic_sugar

[4]: https://en.wikipedia.org/wiki/Hygienic_macro

[5]: https://github.com/olooney/fourth_gen

3 bulan yang laludiscuss

Incredibly detailed! The vision transformer stuff in particular is very useful to know. It's interesting that the token budgets are so much higher (up to 1120) than GPT, which uses 170 tokens per 512x512 tile. I wonder if that will lead to more granular spatial vision, something GPT struggles with.

8 bulan yang laludiscuss

This isn't surprising; the majority of programmers are using LLMs, and Claude is pretty good for coding. Penetration testing is also a pretty good fit for an agentic loop - you run a tool, read the output, and decide on your next move, rinse and repeat.

In VSCode + GitHub Copilot, agent mode it can propose bash command to run, and when you confirm it runs it in a console and can see the loop, so it can fix errors immediately if any. It tends to go off the rails pretty quickly if things start going badly wrong, but it can complete simple tasks with supervision.

Over two years ago, when this LLM stuff was pretty new, I saw a demo that put ChatGPT in a loop with Metasploit that could crack some of the easy HTB challeges automatically - I remember thinking it was the single most irresponsible use of AI I'd ever seen. While everybody else was trying to sandbox these things for safety, this project was just handing it command line access to the tools it would need to break confinement.

It seems there's actually a whole bunch of similar tools these days, marketed as "automated penetration testing," such as Cybersecurity AI[1]. I used to think the whole cyberpunk "hackers can get in anywhere if they just type hard enough" trope was stupid because with cryptography the defender always has a huge advantage, but now we're looking at a world where AI is automating attacks at scale, while the defenders are vibe coding slop they have no idea how to secure, so maybe Gibson was right all along.

[1]: https://github.com/aliasrobotics/cai

9 bulan yang laludiscuss

I'm pretty sure my mistake was assuming people had read the article and knew the author veered wildly halfway through towards also advocating against using LLMs for proofreading and that you should "just let your mistakes stand." Obviously no one reads the article, just the headline, so they assumed I was disagreeing with that (which I was not.) Other comments that expressed the same sentiment as mine but also quoted that part did manage to get upvoted.

This is an emotionally charged subject for many, so they're operating in Hurrah/Boo mode[1]. After all, how can we defend the value of careful human thought if we don't rush blindly to the defense of every low-effort blog post with a headline that signals agreement with our side?

[1]: https://en.wikipedia.org/wiki/Emotivism

9 bulan yang laludiscuss

Here is a piece I wrote recently on that very subject. Why don't you read that to see if I'm a human writer?

https://www.oranlooney.com/post/em-dash/

9 bulan yang laludiscuss

I don't see the objection to using LLMs to check for grammatical mistakes and spelling errors. That strikes me as a reactionary and dogmatic position, not a rational one.

Anyone who has done any serious writing knows that a good editor will always find a dozen or more errors in any essay of reasonable length, and very few people are willing to pay for professional proofreading services on blog posts. On the other side of the coin, readers will wince and stumble over such errors; they will not wonder at the artisanal authenticity of your post, but merely be annoyed. Wabi-sabi is an aesthetic best reserved for decor, not prose.

9 bulan yang laludiscuss

I just don't want them to design a data model with a single `numeric(10,2)` columns for "sale_price", or hard-code their PowerBI report to show the last five years of data using whatever the exchange rate was on the day they wrote the report. You're right - it could be covered in five minutes, but since we don't currently bother, every junior has to learn it the hard way...

9 bulan yang laludiscuss

From a rhetorical perspective, it's an extended "Yes-set" argument or persuasion sandwich. You see it a lot with cult leaders, motivational speakers, or political pundits. The problem is that you have an unpopular idea that isn't very well supported. How do you smuggle it past your audience? You use a structure like this:

* Verifiable Fact

* Obvious Truth

* Widely Held Opinion

* Your Nonsense Here

* Tautological Platitude

This gets your audience nodding along in "Yes" mode and makes you seem credible so they tend to give you the benefit of the doubt when they hit something they aren't so sure about. Then, before they have time to really process their objection, you move onto and finish with something they can't help but agree with.

The stuff on the history of computation and cybernetics is well researched with a flashy presentation, but it's not original nor, as you pointed out, does it form a single coherent thesis. Mixing in all the biology and movie stuff just dilutes it further. It's just a grab bag of interesting things added to build credibility. Which is a shame, because it's exactly the kind of stuff that's relevant to my interests[3][4].

> "Your manuscript is both good and original; but the part that is good is not original, and the part that is original is not good." - Samuel Johnson

The author clearly has an Opinion™ about AI, but instead of supporting they're trying to smuggle it through in a sandwich, which I think is why you have that intuitive allergic reaction to it.

[1]: https://changingminds.org/disciplines/sales/closing/yes-set_...

[2]: https://en.wikipedia.org/wiki/Compliment_sandwich

[3]: https://www.oranlooney.com/post/history-of-computing/

[4]: https://news.ycombinator.com/item?id=45220656#45221336

9 bulan yang laludiscuss

I've been building up a similar list of topics that nearly every programmer will at some point be forced to learn against their will and which are not adequately covered in undergrad:

* Text file encodings, in particular Unicode, UTF-8, Mojibake

* Time: Time Zones, leap day / seconds, ISO-8601

* Locales, i18n, and local date/number formats

* IEEE 754 floats: NaN and inf, underflow, overflow, why 0.1 + 0.2 != 0.3, ±0, log1p

* Currencies, comma/dot formats, fixed-point decimal representations, and exchange rates

* Version strings, dependencies, semantic versioning, backwards compatibility

There's another list for web/REST developers, and one for data scientists, but this is the core set.

What'd I miss?

9 bulan yang laludiscuss

Soon galleries will have signs saying: "All works verified human-made. Please feel something."

9 bulan yang laludiscuss

What's the original source for this Sagan quote?

> carl sagan called METI “deeply unwise and immature"

It's repeated ad nauseum online, but always verbatim, just those few words and never a full passage, and never with a citation. In other words, it has all the hallmarks of an apocryphal quote or misattribution.

The reason I'm suspicious is because Sagan contributed to the Aricebo message[1], which is literally sending such a radio signal, and the the Voyager disc[2], which is similar. He even wrote an entire sci-fi novel[3] about it.

He describes radio contact in generally positive and hopeful terms in his book Cosmos. He of course acknowledges the dangers of encountering a more technologically advanced civilization, but he goes out of his way to contrast the frightening example of the Aztecs with other more peaceful first encounters such as the Tlingit. He also argues that any significantly more advanced species that had survived millions of years would necessarily have achieved zero population growth and would likely be peaceful. You don't have to take my word for it, you can read his own words in the Encyclopedia Galactica chapter of his book on the Internet Archive[4].

So, if the quote you cited was true, it would represent a late-in-life and somewhat surprising change of heart from cautious optimism to "dark forest" style paranoia. Personally, I believe it's simply one of the many falsely attributes quotes floating around the Internet.

[1]: https://en.wikipedia.org/wiki/Arecibo_message

[2]: https://en.wikipedia.org/wiki/Voyager_Golden_Record

[3]: https://en.wikipedia.org/wiki/Contact_(novel)

[4]: https://archive.org/details/sagancosmos/page/n184/mode/1up

9 bulan yang laludiscuss

Videos don't do well on Hacker News, but I encourage people to at least watch the first couple minutes of this one. The oscilloscope visual overlay is interesting and the editing is really good.

Also, given the topic (audio equalizers) there's no way it could have been a blog post.