"Firstly, there are three main Python concurrency APIs, they are:" asyncio, threading, multiprocessing, ... oh, and concurrent.futures
All kidding aside, I used the multiprocessing module lately and it was a mess. Do I want 'map', 'starmap', 'imap', etc.? All I wanted was to run a function multiple times with different inputs (and multiple inputs per function call) and to fail when any launched process failed rather than waiting for every input variation to execute and then telling me about the error(which honestly I didn't think was asking for too much).
> Spreadsheets.com, for example, lets users dump almost anything into a cell. Drop a photo or a PDF into a cell and the product will immediately create a thumbnail, which you can then expand, as if the spreadsheet were some sort of blog content-management system.
Speaking of math pages on Wikipedia ... and math text more generally
Is it just me or are we horrible at teaching advanced math? Where are the examples (with actual numbers)? Where is the motivation? Where are the pictures?
Does AMD have folks that you can reach out to regarding this? I know Intel has MKL and all the work around its own compiler for maximum speed. This seems like it should be trivial for someone at AMD to put together as an example of how to do things like this correctly ...
My c++ is not great (so it is hard for me to tell what is going on) and I'm used to OpenMP where my understanding has always been that you tend to get a single thread per processor (or per hyper-thread) -- not sure if that is guaranteed with the way your code is laid out? Perhaps it really is a NUMA issue as others suggest. I will note that one other variation I had (as it looks like you are already splitting across threads) is that the chunk sizes were actually smaller than the # of threads which meant a faster thread would take more chunks rather than waiting on the slowest thread. Good luck!
It is not exactly clear to me what is going on with threads (I guess you are using all of them?). I haven't done too much in this space but anecdotally I've had better luck if my summation is explicitly split into sub-summation tasks. It is not clear if that is being done here. It looks like a single summation loop that the author is expecting the computer to magically split across multiple threads. I'd be interested in seeing what this looks like if instead the task were to add chunks of the original dataset into results per thread (e.g, first 8000 samples on first thread, next 8000 on 2nd thread, etc.), with a final accumulation loop across all threads. Again, the author may be trying this and this is not my area of expertise but I've had decent luck saturating the memory bus with a similar approach.
Yes please! I like to search for examples of how to use libraries and often times the results are all the same exact call in forks or copies of the same code in multiple places. Perhaps deduplication could be optional when searching?
I thought this would be referencing work by Nathan Myhrvold using a new type of reactor that supposedly runs on "spent" nuclear waste and in the event of power failure just stops running safely. Not sure of the other logistic issues involved. The one thing I remember about transitioning to this approach is that Nathan said the US isn't very good about building new things, so they were going to build in China. But then it got shut down right as the anti-China trade policies started a few years ago. Not sure if there are big problems with this approach, but it sounded promising ...
As someone that rarely works with Python lists, as opposed to numpy arrays, I was pleasantly surprised to see numpy does what I would expect in providing a mergesort option. I'm surprised Python doesn't, other than via heapq and only implemented in Python (according to my reading of the post and a very quick Google search).
Oops, just for fun the numpy documentation currently states: "The datatype determines which of ‘mergesort’ or ‘timsort’ is actually used, even if ‘mergesort’ is specified. User selection at a finer scale is not currently available." Awesome ...
Also, apparently mergesort may also be done using radix sort for integers "‘mergesort’ and ‘stable’ are mapped to radix sort for integer data types."
Interesting take. I did not interpret their statement as vindication of being correct but rather that this version of events is a possibility that can't currently be dismissed.
Agreed. However my reaction when first hearing about the lab leak (middle of last year?) was that the leak stories were meant to be malicious/propaganda against China. I didn't take any of this seriously until an article in Politico a week or two ago.
But here's the kicker. Let's say this was a lab leak and as a reporter (which I'm not) I thought the evidence was good enough to warrant reporting. I'm not sure I would share it. The previous occupant of the white house did a great disservice in giving this whole thing a racially charged tone. I'm genuinely scared by the increased acts of violence against southeast Asians in the US and worry that stories like this will make it worse. I'm hoping that the new US government is secretly taking steps to help prevent what may have happened in that lab -- in addition to the large effort needed elsewhere to improve our handling after things had begun to spread.
Anyway, main point is that this was the first time in a long time (ever?) where I really wondered whether, given the circumstances, if it was good to share "the whole truth" (as best we know it) given that we don't know what happened and the potential real-life implications to many people in the US.
Maybe somewhat off topic ... I don't sell anything on any App stores currently but I do like the idea of having some way of creating "applications" that are a bit more discoverable. Last I checked GitHub search is pretty awful for discovery (maybe I'm wrong on that?). I could imagine an interface where it would be easier to browse "apps" that do something specific.
People can provide feedback and ratings which makes it easy to see when projects are dead or when there are big issues that aren't being fix. You can also reference other projects - "This project is like that project but it is faster ..."
I don't think PyPi is all that good for this. Also, I've never really invested the effort to learn how to deploy an application to PyPi whereas something that simply points to my repo may be valuable?
Anyway, the main problem I want solved is not having people discover my code (although that would be great too), but being able to find other code more easily so I don't need to constantly reinvent the wheel.
All kidding aside, I used the multiprocessing module lately and it was a mess. Do I want 'map', 'starmap', 'imap', etc.? All I wanted was to run a function multiple times with different inputs (and multiple inputs per function call) and to fail when any launched process failed rather than waiting for every input variation to execute and then telling me about the error(which honestly I didn't think was asking for too much).