Show HN: LibRapid -- A high performance multi-dimensional array library
3 comments
Nice work! It takes decent dedication and effort to maintain an underappreciated low-level mathematics library.
Unfortunately, Google seemed to have developed its own multi-dimensional array library as well, featuring virtual views, handling of large arrays that cannot fit into main memory, compression and more: https://github.com/google/tensorstore https://ai.googleblog.com/2022/09/tensorstore-for-high-perfo...
What are `librapid`'s strengths and unique features compared to Google's `Tensor Store`? Being lightweight might be one.
Unfortunately, Google seemed to have developed its own multi-dimensional array library as well, featuring virtual views, handling of large arrays that cannot fit into main memory, compression and more: https://github.com/google/tensorstore https://ai.googleblog.com/2022/09/tensorstore-for-high-perfo...
What are `librapid`'s strengths and unique features compared to Google's `Tensor Store`? Being lightweight might be one.
I've had a little look at it, and I think LibRapid aims to solve a different problem. LibRapid aims to be as fast as physically possible (where reasonably possible to do so), and hence is able to achieve performance at or above libraries like XTensor and Eigen.
It also has seamless CUDA integration, allowing you to perform operations on the GPU by simply changing the tempate argument to your array definitions. For example, `auto cpuArray = lrc::Array<float>(...);` and `auto gpuArray = lrc::Array<float, lrc::device::GPU>(...)`.
It appears to me that TensorStore's primary focus is storing the arrays, not providing high-performance routines to manipulate them.
Regarding the features you mentioned, LibRapid provides an ArrayView class, which provides strided read and write access to any Array type. I'm also planning on impementing a method to save and load arrays from files, so it might be possible to implement some form of compression into that process.
Hopefully this helps!
It also has seamless CUDA integration, allowing you to perform operations on the GPU by simply changing the tempate argument to your array definitions. For example, `auto cpuArray = lrc::Array<float>(...);` and `auto gpuArray = lrc::Array<float, lrc::device::GPU>(...)`.
It appears to me that TensorStore's primary focus is storing the arrays, not providing high-performance routines to manipulate them.
Regarding the features you mentioned, LibRapid provides an ArrayView class, which provides strided read and write access to any Array type. I'm also planning on impementing a method to save and load arrays from files, so it might be possible to implement some form of compression into that process.
Hopefully this helps!
I'm developing an incredibly fast library for arrays and mathematics, and I've implemented a few new features and made some improvements. I'd love for you to check it out!
Links:
GitHub: https://github.com/LibRapid/librapid/
Documentation: https://librapid.readthedocs.io/en/latest/
Discord: https://discord.com/invite/cGxTFTgCAC
Hey everyone!
I am the lead developer of LibRapid (https://github.com/LibRapid/librapid/), a high-performance C++ library for array manipulation and mathematics. I've been working hard to bring you some significant updates and improvements to the library, and I'm excited to share them with you! These changes will make it even easier for you to use LibRapid in your projects and enable you to do even more with it.
Here's a rundown of the most notable changes:
CUDA improvements: We've fixed several bugs and made performance enhancements for the CUDA implementation.
Expanded support for BLAS libraries: LibRapid now has greater support for BLAS libraries, including Intel's MKL.
Test suite and documentation updates: We've updated our test suite and overhauled the documentation for clarity and ease of use.
Matrix Transposition: The library now supports matrix transposition with highly optimised, architecture-specific SIMD implementations and generic implementations for non-trivial types.
New math utility functions: We've added various new utility functions for complex numbers, multi-precision arithmetic, and more.
Matrix transposition and SIMD: LibRapid now supports matrix transposition and SIMD matrix transpose for SSE and AVX2, improving performance and memory alignment.
Random number generation: We've started implementing a new random number generation library.
Fast Fourier Transforms: LibRapid now supports fast Fourier transforms on 1D arrays, with more features coming soon! (We're using FFTW, so you can be sure you are getting the best possible performance)
Array BLAS operations: We're implementing BLAS functions that operate on our high-level array type, allowing you to access the high-performace, specialised routines without the tedious interface. (Many operations already use BLAS functions internally)
You can find more detailed information about the library and example code and usage guides in the documentation (https://librapid.readthedocs.io/en/latest/).
We're always looking for feedback and contributions, so if you're interested in using LibRapid or helping out, please feel free to create a pull request or contact us via our Discord Server (https://discord.com/invite/cGxTFTgCAC). I'm putting in a lot of effort to enhance LibRapid and would greatly appreciate any sponsorships to support my work. If you or your organisation is interested in sponsoring LibRapid, please feel free to reach out to me to discuss your ideas.
We hope you enjoy the latest updates to LibRapid, and look forward to seeing what you build with it!