We recently used t-SNE and UMAP in the field of radiation damage, for visualising the damage shapes (point defect clusters of different shapes). The results were interesting in many ways.
We found that the general layout of classes of shapes is more or less same in UMAP or t-SNE, so the global & local relationship argument didn't really work for our data. Since, the data was around 1000 points of around 50 dimensional histograms, efficiency was also not a big distinction. For us, real advantage of UMAP came out to be its amenability to work with HDBSCAN and embedding new test data. We are excited to use it on further categorising cascade and sub-cascade shapes (bigger damage areas).
- https://haptork.github.io/csaransh/ : Go to last pane "Cluster Classes", click on a point to see the shape on right. Select between t-SNE or UMAP on the left pane. (might take time to load)
Thanks for your interest. There are indeed no function pointers, no void pointers, no preprocessor hacks, no wild type-casting involved anywhere in the library. This is completely compile time type-safe. Still, the kind of things you can do would be a far cry for all the hacks with those shoddy pointers. There are a lot of features like column manipulation, passing vector of a values for multiple rows and just a value for a single row etc. that make composability and ease of use way better. All of these features are implemented using template meta programming and traits available in modern C++. The language has become quite impressive from what we used to think of. You must check it out.
A map/reduce/rise take a template parameter which can be of any class type and library has nothing to do what a function object or function does in the function body. The function just need to take in the arguments that the prior unit passes it and return something. I don't think there will be any problem if one uses any other library to perform computation inside a function, even if some library does not work out of the box, I'm pretty positive it can be done with little tinkering. I hope this makes things clear.
Yeah, you are right, fault tolerance is a needed feature. Although, I'm not pretty sure if that needs to be added to easyLambda itself. Since, MPI 3.0 is expected to support fault tolerance by default. But instead of waiting for MPI 3.0 fault tolerance support, I'm thinking of running it on top of HPX or some other system that provides good runtime support on top of MPI. Also, since most of parallelism scheme is confined to MPIBridge which is just a unit in the flow, these different implementations can coexist.
One can still use TBB inside the functions to make use of it alongside getting MPI parallelism from easyLambda.
Moreover, easyLambda has data flows, map, reduce which in a way improves the way programs are written and composed. There are no classes to be declared for anything. It has many syntactic sugars, for column selection, control over parallelism, i/o etc. I personally like the style of coding in
it for various problems, keeping parallelism aside.
We recently used t-SNE and UMAP in the field of radiation damage, for visualising the damage shapes (point defect clusters of different shapes). The results were interesting in many ways.
We found that the general layout of classes of shapes is more or less same in UMAP or t-SNE, so the global & local relationship argument didn't really work for our data. Since, the data was around 1000 points of around 50 dimensional histograms, efficiency was also not a big distinction. For us, real advantage of UMAP came out to be its amenability to work with HDBSCAN and embedding new test data. We are excited to use it on further categorising cascade and sub-cascade shapes (bigger damage areas).
To check results see the following:
- https://haptork.github.io/csaransh/presentation/index.html#/... : select the t-SNE or UMAP on left pane.
- https://haptork.github.io/csaransh/ : Go to last pane "Cluster Classes", click on a point to see the shape on right. Select between t-SNE or UMAP on the left pane. (might take time to load)
- https://arxiv.org/abs/1811.10923 : arXiv paper
- https://github.com/haptork/csaransh : GitHub repo
Ideas and suggestions are welcome.