> [if] you want to retain as much precision as possible and still use floats, don't store it in a float with range [0.0,100.0]. Store it with the range [0.0,1.0].
I just tested this out and it doesn't seem true.
The two storing methods seem similarly precise over most of the range of fractions [0,1], sometimes one gives lower spacing, sometimes the other. For instance, for fractions from 0.5 to 0.638 we get smaller spacing if using [0,100], but for 0.638 to 1 the spacing is smaller if storing in [0,1].
For very small fractions (< 1e-38), it also seems more accurate to store in the range [0,100] since you are representing smaller numbers with the same bit pattern. That is, because the smallest nonzero positive float32 is 1.40129846e-45, so if you store as a float32 range [0,1] that's the smallest possible representable fraction, but if you're storing as a float in range[0,100], that actually represents a fraction 1.40129846e-47, which is smaller.
For the general result, see for yourself in python/numpy:
x = np.linspace(0,1,10000)
plt.plot(x, np.float64(np.spacing(np.float32(x*100)))/100) # plot spacing stored as [0,100]
plt.plot(x, np.float64(np.spacing(np.float32(x)))) # plot spacing stored as [0,1]
How is CUDA-C that much easier than OpenCL? Having ported back and forth myself, the base C-like languages are virtually identical. Just sub "__syncthreads();" for "barrier(CL_MEM_FENCE)" and so on. To me the main problem is that Nvidia hobbles OpenCL on their GPUs by not updating their CL compiler to OpenCL 2.0, so some special features are missing, such as many atomics.
Did you read in English? I read it in french and loved it, then looked at Montcrief's translation - I would not have made it through. Montcrief turned Prousts clear and precise (but long) sentences into a kind of esoteric word puzzle. Lydia Davis' translation looked much better.
For War and Peace, I didn't know russian, so I tested different translations before going ahead. The translator makes a big difference, I found some translations hard to read.
I had your feeling with Ulysses, though. No translation issue there. Couldn't make it very far.
Huh, thank you for the informative reply. I will reconsider my view, which I had perhaps overstated before to make a point.
Wikipedia gives a citation for backpropagation going back to 1963 by the way, but looking more carefully you are right that the 1986 paper is important.
Interesting link, but I still feel AI hasn't had much success: Do we really have much more today that A* search, neural nets with backpropagation, and HMMs/SVM/etc, which were all developed in the 1960s? The successes of AI that I see (eg OCR/speech recognition and Chess/Jeopardy) use these same old algorithms with only marginal improvements and more CPU. There have been no new major techniques or insights. I'm not an expert though, correct me if I'm wrong.
I'm surprised no one has commented yet on the first couple of these - Energy, AI, Biotech, and Drug design.
These have traditionally been domains requiring a huge research apparatus with tremendous manpower, for only very long term gains. Not good for startups. In AI, how can a startup hope to succeed when academia has had almost no success in 50 years (and I am doubtful throwing more CPU/neuron layers will 'solve' the problem).
In addition, the people with the skills necessary to make progress are going to be advanced researchers with PhDs, who are good enough to remain in academia if they wish or who have already developed a proven-enough idea through their research career that they don't need Y-combinator-style money.
I am not trying to be a downer on the idea, contrarily I hope there can be success. Really I am fishing for anyone with a good perspective (or an answer) to these points.
I just tested this out and it doesn't seem true.
The two storing methods seem similarly precise over most of the range of fractions [0,1], sometimes one gives lower spacing, sometimes the other. For instance, for fractions from 0.5 to 0.638 we get smaller spacing if using [0,100], but for 0.638 to 1 the spacing is smaller if storing in [0,1].
For very small fractions (< 1e-38), it also seems more accurate to store in the range [0,100] since you are representing smaller numbers with the same bit pattern. That is, because the smallest nonzero positive float32 is 1.40129846e-45, so if you store as a float32 range [0,1] that's the smallest possible representable fraction, but if you're storing as a float in range[0,100], that actually represents a fraction 1.40129846e-47, which is smaller.
For the general result, see for yourself in python/numpy: