Show HN: libRmath.js – A port of R's Nmath numeric library to JavaScript(github.com)
github.com
Show HN: libRmath.js – A port of R's Nmath numeric library to JavaScript
https://github.com/jacobbogers/libRmath.js
36 comments
JS can be a serious language in the data science and scientific computation fields, but one of the biggest ecosystem weaknesses is the lack of numeric libraries. This library is very promising!
Unfortunately it is impossible to implement efficient numeric libraries in JavaScript due to the lack of SIMD and GPGPU.
I assume you're talking about in the web browser/pure js? Because otherwise you can just bind to the native CUDA etc. libraries like most other languages do.
And even on the browser side SIMD is being implemented in WebAssembly and (limited) GPGPU will be possible via WebGL 2.0 compute shaders. So as such I don't see any reason why JavaScript couldn't at the very least match python/numpy in performance if someone put their mind to it.
And even on the browser side SIMD is being implemented in WebAssembly and (limited) GPGPU will be possible via WebGL 2.0 compute shaders. So as such I don't see any reason why JavaScript couldn't at the very least match python/numpy in performance if someone put their mind to it.
About 5 years ago, SIMD.js has been announced, was implemented and then scrapped in favor of SIMD in WebAssembly, which does not exist until today. It has been announced, but I would not hold my breath for it. Presumably we'll only get 128-bit wide SIMD anyway, so another 2x to 4x waste of computing power there.
Additionally, 4 years ago, WebCL was announced and implemented in some browsers, but then it was scrapped in favor of compute shaders in WebGL 2.0. When WebGL 2.0 finally arrived, it did not have compute shaders. Instead, they have been delayed to be implemented as an extension at some indefinite point in the future.
Sorry if this sounds a bit grumpy, but I have been disappointed too often to be enthusiastic about this.
There are also issues like floating point rounding modes in JS and floating point precision in WebGL that have to be addressed. Should not be too hard, but who knows.
Additionally, 4 years ago, WebCL was announced and implemented in some browsers, but then it was scrapped in favor of compute shaders in WebGL 2.0. When WebGL 2.0 finally arrived, it did not have compute shaders. Instead, they have been delayed to be implemented as an extension at some indefinite point in the future.
Sorry if this sounds a bit grumpy, but I have been disappointed too often to be enthusiastic about this.
There are also issues like floating point rounding modes in JS and floating point precision in WebGL that have to be addressed. Should not be too hard, but who knows.
Hi, I am owner of libRmath.js and porting BLAS at the moment,
Python and R cant use SIMD either (out of the box), infact no compiler can use SIMD efficiently, therefore gotoBlAS is directly written in assembler for this reason.
Secondly WebGL2 as a vast array of new texture types and Mozilla is implementing the ARB shader. But this has nothing to do with JS perse, because shader langauge is not JS, (its more C like).
Python and R cant use SIMD either (out of the box), infact no compiler can use SIMD efficiently, therefore gotoBlAS is directly written in assembler for this reason.
Secondly WebGL2 as a vast array of new texture types and Mozilla is implementing the ARB shader. But this has nothing to do with JS perse, because shader langauge is not JS, (its more C like).
Ry disagrees:
https://github.com/propelml/propel
This is basically Tensorflow enabled to run in browser and server environments. And it does have GPGPU support via Tensorflow bindings and WebGL. Yes, really.
https://github.com/propelml/propel
This is basically Tensorflow enabled to run in browser and server environments. And it does have GPGPU support via Tensorflow bindings and WebGL. Yes, really.
(To prevent confusion I should mention that I am only talking about JS in the browser.)
WebGL without compute shaders it anything but general compute. There have been some attempts at shoehorning matrix multiplication and other functions required for neural networks into WebGL, but the performance is abysmal due to API limitations.
The actual math in propel seems to be based on https://github.com/PAIR-code/deeplearnjs which has a benchmark website https://deeplearnjs.org/demos/benchmarks/ but the results for the GPU implementation are meaningless because they measure CPU execution time instead of GPU execution time. That's like benchmarking a server by measuring how long it takes to send an http request without waiting for the response.
With proper benchmarking I get about 500 milliseconds for a 1024x1024-by-1024x1024 matrix multiplication, or 4 GFLOPS. CUDA can do 500 GFLOPS on my GPU, so the WebGL implementation is 125 times slower.
Something fishy seems to be going on with their CPU benchmark as well, since it is 2500 times slower than Intel MKL on this machine.
WebGL without compute shaders it anything but general compute. There have been some attempts at shoehorning matrix multiplication and other functions required for neural networks into WebGL, but the performance is abysmal due to API limitations.
The actual math in propel seems to be based on https://github.com/PAIR-code/deeplearnjs which has a benchmark website https://deeplearnjs.org/demos/benchmarks/ but the results for the GPU implementation are meaningless because they measure CPU execution time instead of GPU execution time. That's like benchmarking a server by measuring how long it takes to send an http request without waiting for the response.
With proper benchmarking I get about 500 milliseconds for a 1024x1024-by-1024x1024 matrix multiplication, or 4 GFLOPS. CUDA can do 500 GFLOPS on my GPU, so the WebGL implementation is 125 times slower.
Something fishy seems to be going on with their CPU benchmark as well, since it is 2500 times slower than Intel MKL on this machine.
I Want to put this here as a first level comment, some people (well actually only one) are making up stuff that JavaScript "fudges" floating point numbers.
Some ppl are going to make up drame to "sound interesting" but i dont see how that is FACTUAL.
The README contains over 8900 lines of 200+ well documented standalone code examples samples comparing JS and R code ,side by side for every function call.
THE RESULTS ARE EXACTLY THE SAME!!
Name dropping like web
Some ppl are going to make up drame to "sound interesting" but i dont see how that is FACTUAL.
The README contains over 8900 lines of 200+ well documented standalone code examples samples comparing JS and R code ,side by side for every function call.
THE RESULTS ARE EXACTLY THE SAME!!
Name dropping like web
I am really suspect of this being actually useful for highly technical work. Why? Because I work with R, and the math that is done has very specific results that fudging floating point numbers may be a concern. I don't know enough about Javascript to be certain, but I feel like (from memory) that it does not maintain numeric accuracy at a certain level.
I am not the R nerd, my friend with the PhD is, I just do the coding around it and help clean up his brainy coding. (ie, he can code ok, so he creates the initial R code, and I clean it up so I don't go mad having to work with it) But he explained that even Excel (which I will bet is more accurate than JS) will put out inaccurate results because of number fudging at some level. (again beyond my nerd level to grok)
All of this is too bad, because R syntax is just plain rotten icky stuff. (but I may be seeing things with some of the examples looking like awful R syntax... maybe I need sleep)
Edit: Here's a comment on JS number accuracy, basically what I recalled. (this is a quick search, so take some salt with it). The issue being how JS decided to "fix" the numbers for me, felt like the magic semi colons, only not as predictable.
"In JavaScript all numbers are IEEE 754 floating point numbers.[1] Due to the binary nature of their encoding, some decimal numbers cannot be represented with perfect accuracy. This is analagous to how the fraction 1/3 cannot be accurately represented with a decimal number with a finite number of digits. Once you hit the limit of your storage you'll need to round the last digit up or down."
[0] http://adripofjavascript.com/blog/drips/avoiding-problems-wi...
[1] https://en.wikipedia.org/wiki/IEEE_754
I am not the R nerd, my friend with the PhD is, I just do the coding around it and help clean up his brainy coding. (ie, he can code ok, so he creates the initial R code, and I clean it up so I don't go mad having to work with it) But he explained that even Excel (which I will bet is more accurate than JS) will put out inaccurate results because of number fudging at some level. (again beyond my nerd level to grok)
All of this is too bad, because R syntax is just plain rotten icky stuff. (but I may be seeing things with some of the examples looking like awful R syntax... maybe I need sleep)
Edit: Here's a comment on JS number accuracy, basically what I recalled. (this is a quick search, so take some salt with it). The issue being how JS decided to "fix" the numbers for me, felt like the magic semi colons, only not as predictable.
"In JavaScript all numbers are IEEE 754 floating point numbers.[1] Due to the binary nature of their encoding, some decimal numbers cannot be represented with perfect accuracy. This is analagous to how the fraction 1/3 cannot be accurately represented with a decimal number with a finite number of digits. Once you hit the limit of your storage you'll need to round the last digit up or down."
[0] http://adripofjavascript.com/blog/drips/avoiding-problems-wi...
[1] https://en.wikipedia.org/wiki/IEEE_754
That problem is normal and expected behaviour for finite precision floating point: it appears with any scheme for floating point (IEEE-754 or not), as computers are finite. A significant chunk of the field of numerical analysis is about controlling the error introduced by floating point.
And, anyway, R uses IEEE-754 for 'numeric' too[0], and will do the same "fudging":
> All R platforms are required to work with values conforming to the IEC 60559 (also known as IEEE 754) standard
There's fair criticisms that JS may not offer the required control over rounding modes that R (or the C libs underlying R) provide or the necessary instructions to minimise error accumulation, but general handwavings about "fudging" of floating point don't apply: floating point is unavoidably imprecise, and R and JS even use the same basic format.
[0]: https://stat.ethz.ch/R-manual/R-devel/library/base/html/doub...
And, anyway, R uses IEEE-754 for 'numeric' too[0], and will do the same "fudging":
> All R platforms are required to work with values conforming to the IEC 60559 (also known as IEEE 754) standard
There's fair criticisms that JS may not offer the required control over rounding modes that R (or the C libs underlying R) provide or the necessary instructions to minimise error accumulation, but general handwavings about "fudging" of floating point don't apply: floating point is unavoidably imprecise, and R and JS even use the same basic format.
[0]: https://stat.ethz.ch/R-manual/R-devel/library/base/html/doub...
>...general handwavings about "fudging" of floating point don't apply...
Handwaving is all I can do, I am just the tech taking orders.
I just talked with my statistician friend on the phone, here's one issue my friend just showed me. (integer error, not even floating point)
111,111,111 * 111,111,111 = 12,345,678,987,654,321
(I did this by hand to confirm, he made me the big meany)
Excel, Libre Office, my calculator, Javascript, etc.. all get this number wrong. But with R there's a library that can add support for numbers this big. (I guess this is a well known issue)
Edit: _All_ numbers in JS are floating point numbers, there are no integers... <hmmm>
"In JavaScript, all numbers are implemented in double-precision [...] There is no specific type for integers."
Edit 2: (for fun) This is from R:
Handwaving is all I can do, I am just the tech taking orders.
I just talked with my statistician friend on the phone, here's one issue my friend just showed me. (integer error, not even floating point)
111,111,111 * 111,111,111 = 12,345,678,987,654,321
(I did this by hand to confirm, he made me the big meany)
Excel, Libre Office, my calculator, Javascript, etc.. all get this number wrong. But with R there's a library that can add support for numbers this big. (I guess this is a well known issue)
Edit: _All_ numbers in JS are floating point numbers, there are no integers... <hmmm>
"In JavaScript, all numbers are implemented in double-precision [...] There is no specific type for integers."
Edit 2: (for fun) This is from R:
> library(gmp)
> bignum <- as.bigz(111111111)
> mul.bigz(bignum, bignum)
Big Integer ('bigz') :
[1] 12345678987654321
> Handwaving is all I can do, I am just the tech taking orders.
Even "the tech" can have some understanding of finite precision integers and floats, and how they behave, given they're building blocks of the trade. It seems somewhat relevant to knowing what changes are valid to code. http://floating-point-gui.de/ is a great place to start if you're interested in learning more. :)
> Edit: _All_ numbers in JS are floating point numbers, there are no integers... <hmmm>
Not having any native integers is unfortunate and annoying, but there are various tricks to ensure numbers are integers (e.g. floats exactly represent integers up to 2^53) plus various tricks to convince the JS engines that things are guaranteed to be integers for optimizations. These are clunky and ugly hacks, but they're possible and even more possible when using JS as a compilation target from a different language (e.g. asm.js in the extreme).
> I guess this is a well known issue
Indeed, it very much is. As you demonstrate, even R doesn't allow doing your example calculation natively, you need to import a library... something that can be done just fine in JS too: for arbitrary size integers https://github.com/Yaffle/BigInteger and for arbitrary precision floats https://github.com/charto/bigfloat (among many examples). It's true that these won't approach the performance of GMP due to lack of access to the specialized instructions it uses, but they offer the same basic functionality.
That said, I agree with the general sentiment that JS is suboptimal for scientific computing, but mostly because of performance (although R's performance falls off very quickly when using anything other than vector operations) rather than inaccurate concerns about precision.
Even "the tech" can have some understanding of finite precision integers and floats, and how they behave, given they're building blocks of the trade. It seems somewhat relevant to knowing what changes are valid to code. http://floating-point-gui.de/ is a great place to start if you're interested in learning more. :)
> Edit: _All_ numbers in JS are floating point numbers, there are no integers... <hmmm>
Not having any native integers is unfortunate and annoying, but there are various tricks to ensure numbers are integers (e.g. floats exactly represent integers up to 2^53) plus various tricks to convince the JS engines that things are guaranteed to be integers for optimizations. These are clunky and ugly hacks, but they're possible and even more possible when using JS as a compilation target from a different language (e.g. asm.js in the extreme).
> I guess this is a well known issue
Indeed, it very much is. As you demonstrate, even R doesn't allow doing your example calculation natively, you need to import a library... something that can be done just fine in JS too: for arbitrary size integers https://github.com/Yaffle/BigInteger and for arbitrary precision floats https://github.com/charto/bigfloat (among many examples). It's true that these won't approach the performance of GMP due to lack of access to the specialized instructions it uses, but they offer the same basic functionality.
That said, I agree with the general sentiment that JS is suboptimal for scientific computing, but mostly because of performance (although R's performance falls off very quickly when using anything other than vector operations) rather than inaccurate concerns about precision.
>Even "the tech" can have some understanding of finite precision integers and floats, and how they behave, given they're building blocks of the trade.
I do have "some understanding". But not enough to processing millions of records and notice that one or two numbers of output, out of hundreds, aren't correct. That's the PhD's job.
I do have "some understanding". But not enough to processing millions of records and notice that one or two numbers of output, out of hundreds, aren't correct. That's the PhD's job.
Concepts like finite integer sizes (aka integer overflow) and finite floating point precision (1/3+1/3+1/3 != 1) aren't PhD-level concerns. :)
...apples and oranges....
R doesn't support non IEEE standard floats, what you are basically doing is using an external library (GMP), and special functions for addition, multiplication from that package.
Nor can any of the functions in R "base" use GMP.
Again, nobody is fudging numbers, neither R, Java, C, or JS because floating point is implemented on hardware so this argumument is a moot one.
Nor can any of the functions in R "base" use GMP.
Again, nobody is fudging numbers, neither R, Java, C, or JS because floating point is implemented on hardware so this argumument is a moot one.
My comment started off about floats, but then I found out it's not just floats, it's also integers.
But JS doesn't even _have_ integers...
But JS doesn't even _have_ integers...
[deleted]
> All of this is too bad, because R syntax is just plain rotten icky stuff. (but I may be seeing things with some of the examples looking like awful R syntax... maybe I need sleep)
If you can get over the way assignments are made, I find R's syntax, quite nice – especially with the pipe paradigm á tidyverse that is getting increasingly common. I personally find that the (from a software engineering perspective) awful syntax instead comes from many R users having a different background than those of other languages. I guess very few R users write unit tests, for example.
Of course, this is fine in most cases – messy, unengineered code is completely fine if few people will use it.
If you can get over the way assignments are made, I find R's syntax, quite nice – especially with the pipe paradigm á tidyverse that is getting increasingly common. I personally find that the (from a software engineering perspective) awful syntax instead comes from many R users having a different background than those of other languages. I guess very few R users write unit tests, for example.
Of course, this is fine in most cases – messy, unengineered code is completely fine if few people will use it.
JavaScript uses 64 bit floats for all numbers. Unless you deal with particularly huge integers I don’t think there will be problems.
“According to the ECMAScript standard, there is only one number type: the double-precision 64-bit binary format IEEE 754 value (numbers between -(253 -1) and 253 -1). There is no specific type for integers. In addition to being able to represent floating-point numbers, the number type has three symbolic values: +Infinity, -Infinity, and NaN (not-a-number).”
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data...
“According to the ECMAScript standard, there is only one number type: the double-precision 64-bit binary format IEEE 754 value (numbers between -(253 -1) and 253 -1). There is no specific type for integers. In addition to being able to represent floating-point numbers, the number type has three symbolic values: +Infinity, -Infinity, and NaN (not-a-number).”
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Data...
Well, the errors I ran into were happening within 4 decimal places. And the research I did at the time said you simply can't trust that the numbers will be accurate. Is this just an issue with floating point numbers in general then? This was a few years ago, maybe something has changed?
You may be thinking of https://stackoverflow.com/questions/588004/is-floating-point...
0.1 + 0.2 === 0.3 // false
I did a quick look up and edited my original comment. But I think this is related...
Is this just an issue with floating point numbers in general then?
Short answer, Yes. JavaScript floats behave identically to 64-bit IEEE floats in all other programming languages. Dealing with rounding errors, precision, stability etc when doing math with floating point numbers is an entire area of research in itself that many very smart people have dedicated their research careers to. This is why you should be very careful before you start writing your own numeric algorithms since the pitfalls are as subtle as they are numerous. Rolling your own core numeric algorithms basically falls into the same category as rolling your own crypto. To a first approximation, leave it to the experts.
Some languages/libraries do offer arbitrary precision numbers that let you avoid these all pitfalls in exchange for a massive performance hit.
Short answer, Yes. JavaScript floats behave identically to 64-bit IEEE floats in all other programming languages. Dealing with rounding errors, precision, stability etc when doing math with floating point numbers is an entire area of research in itself that many very smart people have dedicated their research careers to. This is why you should be very careful before you start writing your own numeric algorithms since the pitfalls are as subtle as they are numerous. Rolling your own core numeric algorithms basically falls into the same category as rolling your own crypto. To a first approximation, leave it to the experts.
Some languages/libraries do offer arbitrary precision numbers that let you avoid these all pitfalls in exchange for a massive performance hit.
I actually find R to have the elegant parts of List, with a more traditional language syntax. I frankly like a lot.
Well, my friend who works with R more consistently likes it ok too. Maybe it's an acquired taste. Also, he's the statistician, so perhaps it's quirky structure relates to the data in some way a laymen can't understand.
But sometimes he just wants to loop through a list and he goes a bit crazy.
But sometimes he just wants to loop through a list and he goes a bit crazy.
Statisticians tend to learn R in school. It's quirky structures are a major source of prototyping slowdown/technical debt. I recommend using Python/Pydata when people ask my advice in starting out. Once one learns Python then R isn't too bad to pick up.
I assume you meant Lisp. R started out as a Scheme dialect, so yes, there is definitely a connection.
The README in the github JS port has 9000 lines of documentation and code samples (R and JS side by side) to prove the output is the same for all functions.
R is fudging numbers? I dont think so, neither does JS or excel btw.
R shows limited precision to a human, you can control this with options('digits'=22) Example
> options(digits=22)
> 1/1.2
[1] 0.83333333333333337
> options(digits=4)
> 1/1.2
[1] 0.8333
R is fudging numbers? I dont think so, neither does JS or excel btw.
R shows limited precision to a human, you can control this with options('digits'=22) Example
> options(digits=22)
> 1/1.2
[1] 0.83333333333333337
> options(digits=4)
> 1/1.2
[1] 0.8333
>R is fudging numbers?
I said javascript fudges numbers, not R.
I said javascript fudges numbers, not R.
Javascript fudges numbers??? I don't see how making stuff up is helpfull.
The README contains over 8900 lines of 200+ well documented standalone examples with many many code samples comparing JS and R side by side for every function call.
THE RESULTS ARE EXACTLY THE SAME!!
You took the effort to throw dirt at JS, but could you not have glossed over the "proof" first?
You seem to not know that IEEE floating is a standard, I suggest you read up on that first
The README contains over 8900 lines of 200+ well documented standalone examples with many many code samples comparing JS and R side by side for every function call.
THE RESULTS ARE EXACTLY THE SAME!!
You took the effort to throw dirt at JS, but could you not have glossed over the "proof" first?
You seem to not know that IEEE floating is a standard, I suggest you read up on that first
[deleted]
Floating point IEEE 754 is implemented in the hardware (Intel, AMD). It is a feature of the hardware not any language. We need facts not sophism.
I sometimes wish I had launched a similar porting effort with SageMath...
Great work. Thanks!