eref

vor 9 Jahrendiscuss

I do not get what makes this post so popular. Google Maps is often better and uses automatically extracted features from photogrammetry. Eh. 350 points at most.

eref

vor 9 Jahrendiscuss

For me on Firefox on macOS that is just a couple of key strokes away:

  cmd-l cmd-c ctrl-a r <space> u r l : <enter> cmd-t cmd-v ctrl-a h n <enter>

I have set up `r` as keyword to search on Reddit and `hn` to search on Hacker News [1]. `url:` is the Reddit keyword to search in the url field of the items. (In Firefox, you can right click any search field and add a keyword bookmark for it.)

[1] https://hn.algolia.com

eref

vor 9 Jahrendiscuss

Why not both?

eref

vor 9 Jahrendiscuss

eref

vor 9 Jahrendiscuss

eref

vor 9 Jahrendiscuss

The term "circuit" is used in neuroscience all of the place.

eref

vor 9 Jahrendiscuss

Capsules basically do a kind of self-attention. But there the parent features compete for a coupling, not the child features.

eref

vor 9 Jahrendiscuss

Same with 57.0 on macOS 10.12.

Edit: I have FF Studies disabled under about:preferences#privacy. I guess that is the reason why it is not installed on my machine.

eref

vor 9 Jahrendiscuss

The new XPS 13 has a better camera position and a good touch pad but still not a great battery. It’s close. Maybe next year.

eref

vor 9 Jahrendiscuss

> simulate a physical environment very fast

That's probably only a problem if it is must faster than everbody else.

> let alone faster than what happens in our environment

That is often not very hard. When a bottle rolls off the table, you can catch it by approximately predicting it's trajectory without computing the precise evolution of the ~10^26 atoms that make up the water bottle. Compression is a corner stone of intelligence. The second corner stone is using compression to choose actions that maximize expected cumulative future reward.

eref

vor 9 Jahrendiscuss

The question is whether the overall gains outnumber the local losses.

eref

vor 9 Jahrendiscuss

The main problem is that we still lack good generative models and good ways of interrogating them. GANs are unstable and difficult to apply to time series, VAEs suffer from posterior collapse, WaveNet/PixelRNN grow with the input size and overemphasize the details, RNNs are hard to train because we lack good training algorithms. Generally, small errors tend to compound in step-wise predictions because NNs do not generalize very well and gradients tend to vanish and shatter. If you just regard computation time to roll out the future, modeling domains in which the rules are simple enough to be hand-coded and evaluated quickly (such as Go and Chess) probably makes MCTS a million times more suitable compared to domains in which you need a complex model.

eref

vor 9 Jahrendiscuss

But what does have to do with smoothness and translation invariance which this paper is a demonstration of? You even learn Gabor filters with local connectivity without spatial weight sharing.

eref

vor 9 Jahrendiscuss

What do Gabor filters have to do with this?

eref

vor 9 Jahrendiscuss

I think that can be mainly attributed to the fact that the last few deconvolutional features are overfitted to features in the image and are somewhat robust to noise. The network does not even learn features to produce e.g. white noise as output. This is probably much less magical than the paper makes it seem to be.

eref

vor 9 Jahrendiscuss

I think it is right to „soft“-censor in this way (i.e. the information is accessible but not promoted). Reality without sorting and filters would be inhuman because market incentives to grab people’s attention would maximally exploit our tribal instincts and thereby completely destabilize our societies. This is already happening and it needs to stop immediately.

eref

vor 9 Jahrendiscuss

Maybe I misunderstand stoic philosophy, but the rule to override emotional responses to failure with serenity strikes me as overly broad. Sometimes it is simply easier to get angry and let it out; especially if that sends a valid corrective signal to someone who is at fault. And even if the failure was just bad luck, it seem totally fine to be mad for a couple of minutes or hours. It is simply the way our brains have evolved to work and that's fine as long it is not harmful as long as nobody is harmed. Just be careful to not turn it into a constant state of anger because that will likely worsen your life rather than improve it. All of that can be subsumed under rationality.

eref

vor 9 Jahrendiscuss

> High representation efficiency

Capsules do require much fewer parameters, they generalize 10-20% better to new viewpoints, they are much more robust to adversarial examples and can better recognize overlapping objects; but, on the other hand, capsules currently require much more training data to achieve the same performance, even though in theory (if they would actually learn inverse graphics) they should require less data, and they add a lot of expensive additional structure (roughly 10X). I am rather pessimistic about whether the approach will lead us anywhere; it seems sub-optimal to model all possible child-parent configurations explicitly. That has a quadratic nature to it and my hunch is that can be done sub-linearily.

eref

vor 9 Jahrendiscuss

I might be wrong, and I don't want to question the possibility that someone from a non-technical domain can contribute to technical domains (in particular because neural networks and style transfer are basically high school mathematics; and since this happened before [1]), but it seems to me that the paper in question does not contain much technical innovation, but it is rather a demonstration of how existing techniques can be used in practice [2].

[1] https://en.wikipedia.org/wiki/Hedy_Lamarr

[2] https://arxiv.org/abs/1701.04928

eref

vor 9 Jahrendiscuss

I think that is correct because otherwise they would have mentioned it as an outstanding feature of the model. It does require fewer parameters than a CNN to reach the same accuracy.

Submissions

The Elephant in the Brain – A New Book by Kevin Simler and Robin Hanson

2 Navy Airmen and an Object That ‘Accelerated Like Nothing I’ve Ever Seen’

Learning Combinatorial Optimization Algorithms Over Graphs

Russia says it will ignore any UN ban of killer robots

Slaughterbots [video]

When Will Climate Change Make the Earth Too Hot for Humans?

comments