HackerTrans
TopNewTrendsCommentsPastAskShowJobs

chenglou

no profile record

Submissions

Kolmogorov Complexity of Neutral Networks

twitter.com
1 points·by chenglou·2년 전·0 comments

comments

chenglou
·2년 전·discuss
Website contributor here! Answering some of the technical questions on Twitter: https://x.com/_chenglou/status/1826425720522080480
chenglou
·3년 전·discuss
Yeah I was only talking about quantities. Equivalently, assume that it's a linear algorithm in the child and a linear one in the parent. Ultimately it ends up as O(nm) being some big number, but when people do runtime analysis in the real world, they don't tend to consider the composition of these blackboxes since there'd be too many combinations. (Composition of two polynomial runtimes would be even worse, yeah.)

Basically, performance doesn't compose well under current paradigms, and you can see Casey's methods as starting from the assumption of wanting to preserve performance (the cycles count is just an example, although it might not appeal to some crowds), and working backward toward a paradigm.

There was a good quote that programming should be more like physics than math.
chenglou
·3년 전·discuss
It's more than that. The way black box composition is done in modern software, your n=100 code (say, a component) gets reused into a another thing somewhere above, and now you're being iterated through m=100 times. Oops, now n=10k

Generally, Casey seems to preach holistic thinking, finding the right mental model and just write the most straightforward code (which is harder than it looks; people get distracted in the gigantic state space of solutions all the time). However this requires 1. a small team of 2. good engineers. Folks argue that this isn't always feasible, which is true, but the point of these presentations is to spread the coding patterns & knowledge to train the next gen of engineers to be more aware of these issues and work toward said smaller team & better engineers direction, knowing that we might never reach it. Most modern patterns (and org structures) don't incentivize these 2 qualities.
chenglou
·5년 전·discuss
In case this isn't clear, RefTerm is:

1. Faster in every case tested

2. More fully-featured, including i18n

3. Easier to read and maintain (see for yourself)

4. Shorter

5. Using existing libs, aka interops well

...So none of these typical hand-wavy dismissals apply:

1. "Is it really faster for edge cases"

2. "He probably didn't implement certain features like Arabic and Chinese"

3. "Businesses just wants enough"

4. "Businesses just wants enough"

5. "It probably makes some closed-world assumption"

The performance of RefTerm didn't come from some big tradeoff; it came from using the right perspective and keeping things simple.

Sure, past a certain complexity threshold, you'd have to work for extra perf, but from my observations, folks who immediately jump to the wrong implicit conclusion that "good perf must have required compromises" got conditioned to think this way because the software they work on are already (often unnecessarily) complex.