It seems like this page is updated with the followup questions asked by every visitor. That's an easy way to leak your search history and it's (amusingly) happening live as I'm typing this.
For me, the gradual build up of (1) maximum likelihood estimation to (2) maximum a posteriori estimation to (3) full posterior approximation (or posterior sampling) was helpful to understand where Bayesian methods are in machine learning. Here’s a great video series by Erik Bekkers, who is at the University of Amsterdam. It assumes solid knowledge of calculus & linear algebra and takes you through the math and intuition of all fundamental ML methods: https://youtube.com/playlist?list=PL8FnQMH2k7jzhtVYbKmvrMyXD...
Slight tangent: does anyone know of a good source for more useful blog posts on ML in the wild? Many that I come across are a funnel into some product, are very short, too theoretical, for beginners, or all of the above. This post strikes a nice balance in simply sharing some experiences and opinions, like you would see in blogposts on how to do software engineering well.
That's such a cool nugget of information. The Versailles one is apparently A = 435 Hz, the 'diapason normal'. Nowadays we mostly use concert pitch at A = 440 Hz. The 99% Invisible podcast has a mini-story on it:
> London’s Royal Philharmonic Society would still tune higher to about 439 Hz though. This was because the mandate specified that Concert A should be 435 Hz according to a tuning fork of a specific weight at 15 degrees Celsius. The temperature was specified so that the metal tuning fork could be accurately reproduced, but British orchestras reasoned that their concert halls were warmer than that, and so to compensate they would tune higher.
I wonder if they tested their assumption with a conformant tuning fork.
Conventional methods of rendering 3D objects and spaces rely on specifying geometry and material properties in some format. You then simulate a viewpoint using that info and physics simulations.
A NeRF takes over both the role of the file format and part of the rendering in the form of a neural network. You feed in a world coordinate and a viewpoint and you get an RGB tuple and density out of it. If you interrogate the NeRF enough you can render any traditional 2D or 3D image out of it by combining all the datapoints.
One theoretical benefit is that that a NeRF is a continuous function, so the resolution is only limited by the capacity of the neural network. Another cool thing is that a NeRF is trained on pictures (with info about where they were taken from), so if you train a NeRF successfully in high-res it’s like scanning an object. A major practical challenge is that it is (was?) pretty frickin’ slow to work with. I wrote a more elaborate comment about it on the previous NeRF improvement post [1]. There I closed with:
> It would be amazing to have NeRF-based graphics engines that can make up spaces out of layers of NeRFs, all probed in real-time.
Here they’ve taken a major step in that direction by speeding up the rendering 3000X.
How do you deal with adversarial/byzantine updates that attempt to degrade performance or even install a backdoor? Do you use plain averaging, or some other aggregation algorithm like Multi-Krum?
The last time a post like this appeared on HN prompted me to write a gist with a simple neural network in Python (with Numpy). It downloads the MNIST dataset for you, trains a fully connected network on it, prints the accuracy on the validation set and plots the loss. It's pretty verbose with plenty of terms and comments to search the web for if you're interested.
Yes, I'm also using Hetzner and have not run into any limits in what plugins you can install and use yet. They used to call 'storage share' just 'Nextcloud', but moved it to the new name for licensing reasons.
I don't know what type of hardware they're running their Nextcloud instances on, but I've been using calendar/contacts and public file sharing (since Firefox Send went down) on the cheapest plan for a while now and it's fine. I also dabbled a bit in using Nextcloud's maps, video calling, bookmarks and rss reader applications and it all works OK, although I didn't stick to them because they're not the most rich in features. I had to Hetzner email support a few times and got a helpful reply within 1-2 work days, sometimes quicker, which I thought was fine.
In macOS it is not entirely clear to me what category of things the cmd key is for. On one hand it's used for keyboard shortcuts related to window management and the system, but on the other hand it's also used to issue commands to the application itself.
In i3 you assign a $mod key, which then always and exclusively used for 'system and window management'. Opening, closing, moving and resizing windows, switching workspaces and showing toolbars is all done with some key combination that starts with $mod.
Since reading and writing the clipboard is done by the application, in my brain it should not involve $mod. So I use ctrl+c and ctrl+v for all applications. The one exception is the terminal, in which ctrl+c terminates programs. So there I use ctrl+shift+c. I also unified the register vim yanks to with the system clipboard, so for me most copying and pasting that happens in the terminal is done via `y` (yank) and `p` (paste).
A more straight answer to your question is: yes, you can remap everything to bits. You can modify what keypress each keycode should trigger, so there is really no limit to what you can do there.
These are exactly the type of silly applications that are needed on macOS because the OS too locked down, or at least not made for people who like to do this type of stuff.
On Linux, almost all native applications keep their data in regular, non-proprietary files on the file system. This means your data always exists separate from the applications and you can do with it what you want. If you want to run a script every Monday at 3PM that sifts through your ~/downloads folder and uploads the top 10% largest files to a server, except in January, on Easter, when when your drive is below 70% capacity, if it's not hooked up to power, and also not if Kanye West tweeted something in all caps, you can do that. You can make it retry every 42 minutes if conditions weren't met, email Bob and message Alice in Slack about it. You can make it machine learn your behavior and pick a window within 3 hours after sunrise, log the bandwidth usage, and have it play a tune for your friend's MPD server on the other side of the country.
Why you would want to do all of this is beyond me, but odds are you want to do one of these things at some point. Then Hazel won't be able to do it because they didn't consider your use case, that functionality is only available in version 6 (pay up for the upgrade!), Apple didn't provide the right API, or they recently took it away in the name of security. Surely there's another app that can do it, but that one can't do the other thing! Oh well, maybe the next version?
On Linux, you dream up the code in the language of your choice and it will work forever. Your learned skills will transfer one-to-one to programming you do for your job or hobby, instead of being weirdly specific and locked down to this one, overpriced app's clickity click interface. The OSS mentality of interoperability and doing one thing well almost guarantees there are battle-tested tools available that can solve pieces of your specific puzzle, while not locking you in to their way of doing things.
Of course scripting is not exclusive to Linux, but it is the only OS that was and is built with it in mind.
I used to be all in on Apple. On macOS I had a little program called Magnet to snap windows to sides and corners, and on my iPad (with external keyboard) I SSH’d into a VPS to write and run code there. I used Alfred and had all kinds of workflows in there. I thought it was great.
But then during my AI studies I wanted some beefier hardware, which was just not affordable for me within Apple’s ecosystem, plus they only used AMD graphics cards. I built a desktop computer that outperformed the top of the line Mac Pro for a fraction of the cost and turned it into a Hackintosh. Two weeks later Mojave came out, and Apple never approved any Nvidia drivers from then on.
My eyes opened to Linux and i3 in particular, which looked like Magnet taken to the extreme. What had taken me hours to install and configure on macOS (GPU-acceleration for PyTorch, for example) just worked with one package install on Linux. All my expensive apps were replaced with simple and free, much more configurable alternatives. At first I spent a day or two getting things just right. Since then not much has changed because not much needed changing, which I really like.
Now I look at macOS and iOS and cringe how locked down it all is. Users are very creative in their workarounds to make it work, but it is ultimately quite silly that you need to use special URL schemes and workflows to open a text file across different apps.
I studied piano full-time for a short while. My favorite thing about it was how working on my technique or a particularly tricky rhythmic pattern tickled my brain. I often found that I went through a few phases: “ahrg, this is impossible!”, “getting there but sloppy” and finally, often suddenly (after a good night’s sleep) “this is easy! I don’t know how I was ever not able to do this”.
Using vim and learning new features tickles my brain in kinda the same way. I don’t care much for the hacker credits. It’s a fun variety in my work day to practice some little feature that I read about, and it doesn’t look like I’ll run out of new stuff to check out anytime soon.
Fiducial markers are commonly used in photogrammetry to either speed up the process, make the resulting model more accurate or a balance of both depending on what the user is looking for. Good fiducials make for distinct features that can easily be matched across different images.
It works best if you play into the algorithm used to find the point correspondences. One commonly used one is SIFT [1]. It's a multi-step process where each step introduces some invariances, like scale invariance through convolution with gaussian kernels at different standard deviations to create a 'scale space', then doing blob detection in that space by looking at second derivative maxima and minima.
The matching process does of a lot of convolution, which is linear (so you can combine a gaussian and laplacian kernel and do both in one shot) and it can be nicely parallelized. The 8 hours of processing of ~80GB of 24MP images was on a GTX 1080.
I wouldn't say that it's particularly slow considering the amount of data and complexity of the operations, but surely a speedup would be very welcome and useful. It would become much more accessible to game companies, movie studios and even industries that (afaik) don't make much use of 3D models yet -- perhaps archaeology or anthropology would jump at the opportunity of scanning and sharing super high res models.
I once had to count the number of set bits (the Hamming weight) for an assignment. It had to be done in under 40 bitwise ops, without loops and with constants of up to 8 bits. The naive approach took way too many ops! I was stumped until I found the linked page, in particular 'Counting bits set, in parallel'[1]. Parallelism at the bit level?! That is when I realized low level programming was completely badass.
In June I worked on a comparison of the original NeRF [1] to a state of the art proprietary photogrammetry method.
The photogrammetry method could process ~80GB worth of 24MP photos into a micrometer-level accurate 3D model in about 8 hours, while the fastest NeRF implementations available took the same time to train a model on just 46 pictures at 0.2MP. A funny extrapolation from a handful of datapoints was that it would have taken 1406 hours or about two months to train a NeRF at a resolution of 24MP, assuming it would converge at all. PixelNeRF improves an aspect that was already great (the number of photos required) but does not seem to tackle this complexity problem.
Another problem is this: the representation of the learned scene is entirely abstract, contained within the weights of the neural networks that make up the NeRF. The space itself cannot be meaningfully inspected -- it must be probed and examined by its input/output pairs. The NeRF takes as its input a 3D location plus a viewing direction, and the output is a color radiance in that direction and a density (which depends only on the 3D location). So to generate a 2D image you emit camera rays into the NeRF from a specific hypothetical camera position, direction, focal length and sensor resolution, get the NeRF's output for many different points along all the camera rays and compute an image based on it (volume rendering).
This is fine as long as the NeRF is available and there are no time constraints, but does not seem workable for real-time graphics rendering like in gaming/VR. So the NeRF should probably be rendered into a traditional 3D model ahead of time. Afaik this is an open problem that I've only seen solved by using a combination of marching cubes to extract the scene geometry and then rendering colors from normal vectors. In this process, continuity, spatial density and directional color radiance, three of the most important contributions of the NeRF design, are entirely lost.
I would be very interested to see papers that tackle higher resolution spaces at feasible training times and faster novel view rendering times. It would be amazing to have NeRF-based graphics engines that can make up spaces out of layers of NeRFs, all probed in real-time.
> A key feature of neural networks is that the outputs are a nonlinear function of the inputs. However, steering a 2D car is simple enough that a linear function is more than sufficient, and neural networks are unnecessary.
This depends entirely on the definition of 'steering a 2D car'. In the model used, throttle is simply proportional to the distance to the nearest wall in front of the car. This means the agent will never accelerate coming out of a corner, because it can't know it has the headroom to steer away from the wall as it's coming out.
Similarly, the model for steering inherently steers the car towards the middle of the track. I would expect the car to wobble from left to right if the road's edges are ragged, make up its own corners if the track edges describe a 'fake' turn on a straight bit, and the car would likely crash if it were to encounter a Y junction or a pit stop. The neural network agents showed smarter behavior here because it is able to capture more complex cross-dependence between different inputs.
On the topic of junctions, if the track were to include them, perhaps it'd be nice if the car chose the quickest route to optimize for lap times. But maybe that stretches the problem statement too much.
> Instead of doing anything fancy, my program generates the coefficients at random to explore the space. If I wanted to generate a good driver for a course, I’d run a few thousand of these and pick the coefficients that complete the course in the shortest time.
In theory this is more random and less efficient than an evolutionary algorithm, which searches the problem space in a structured way. If the author really wanted to hammer the point home, a least squares method to one-shot the coefficients would be more convincing.
All in all, the author doesn't make any hard claims that are false. But I would nuance the point of "neural networks are unnecessary" to "simpler models will do for simpler objectives".
I spoke to someone who started working in sheet piling when he was 16. This is in the Netherlands, so plenty of that to go around. He stayed with his parents and saved until he could buy a house out of pocket at 25. He was 30, said he was saving to retire at 35. It blew my ‘educated’ mind.
> Even if Apple agreed to the demands and made the options available to all developers, Epic would have an advantage.
I don't know that that's the case. Steam, the Humble Store, GOG and Microsoft (just to name a few) would all be happy to compete if the terms were right.
Cool! I really like the (upcoming) idea of watching your own online profiles on HN and such for upvotes and saving those pages. It’s essentially what I already use those upvotes for in my head, but not what they are in reality.