this is part of why I think most researchers get less productive over time... Someone gets some big result during grad school or early career, get some big job from it, and then struggle to get new results of similar quality :shrug:
With ML in particular, there's also the sheer volume of people basically all looking at (essentially) the same problems... so it's kind of like monkeys with type writers spamming ideas until some work.
yeah this is what drives me crazy about LLM writing. Most of the time the prompt has all the info you need and is like maybe a few sentences. Then the LLM expands it into a few paragraphs...
I guess if someone is writing like a big fancy email to send out in bulk, maybe using an LLM to improve would make sense... but just emailing some coworkers it seems super lazy and insulting to send an LLM output :-I
I've been using claude while trying to setup a new personal site. It's very nice to be able to say "I want a nice looking menu with links to other pages" and it spits out something good enough.
I would feel very weird using LLMs for writing, except for filing out stupid applications. I've had collaborators use LLMs for some technical writing and it's pretty much always borderline nonsense that has the aesthetic of something correct. For creative writing, I feel like heavily using an LLM would defeat the purpose :shrug:
yeah I was being very hyperbolic (and am on the younger side, so tbh wasn't very aware of a lot the x projects... I think those are even riskier than I meant.)
google was probably the worst example for me to use tbh, especially since it still has such a good culture of funding researchers.
There was a "meme" a few years ago saying gmail's UI has dozens teams working on each of the different buttons, so that was why I said google/gmail.
huang's original comment was referencing layoffs due to AI, and I think a lot of the "maintaining/replacing existing stuff" engineers are at the most risk atm. But why lay people off why they could be pushed to work on new risky projects :-/
I do sort of think the stereotype of killing projects is kind in the vein of what I meant. like idk, google has so much money I feel like they don't need ~everything to clearly and immediately fit into their ai / data / advertising / search stuff. earnings - expenses is so huge, I think it should be fine to just allow some things to stay "small" without being a more "distinguished" moonshot-style project.
can very much agree about not writing stuff like reductions yourself, unless you have good reason to.
but this sort of feels like another "implement everything with <nvidia stuff> and you'll have a great time!! (but also coincidentally get locked in even more to Nvidia hardware)"
I really wish there were better options to PMPP... It's by far the most up-to-date book, but I totally agree the writing is sort of bad and some of the code examples are straight up incorrect.
So tl;dr, you have at least one person who would pay for a better book :-)
I think Jensen Huang said this recently, and I've had a similar opinion for a while, but a lot of companies seem uncreative with how they use their employees. like google probably has >10k people working on stuff like "ensure gmail refresh button is the correct size", but why not fund teams to take on new and more risky projects... maybe part of it is that the type of people who work at big tech companies are not interested in risky projects :shrug:
it's so great to see people boosting "security" in a way that also just happens to require locking in to big-tech approved apps that send all your data to big-tech so that they can deliver ads to you via your big-tech approved device using your big-tech approved os running your big tech approved browser showing your big-tech approved video platform with your big-tech approved content (oh, and also sends your data to your big-tech approved government)
Input (Eminem lyrics): there's vomit on my sweater already, mom's spaghetti.
Output: Facing some early-stage operational challenges today. Already dealing with some unexpected "output" on my professional attire, but it’s all part of the high-stakes journey. Embracing the mess and staying focused on the mission. #Resilience #GrowthMindset #StartupLife
I feel like that's true when the font is insanely small, which I guess was good when people would print entire proceedings.
Reading two column super small font on a computer is super annoying though tbh.
A personal guideline for a lot of stuff is that a function may be too long when people add comments to mark what sections of it do. (ofc not really a hard rule).
I just think it's easier to see "oh this is calling the load_some_stuff function, which I can easily see returns some data from a file." Rather than <100 lines of stuff, inlined in a big function, that you have to scan through to realize it loads some stuff and/or find the comment saying it loads some stuff>.
That is to say, descriptive functions names are easier to read than large chunks of code!
smaller functions are also usually easier to test :shrug:
I kind of dislike the benchmarkification of AI for science stuff tbh. I've encountered a LOT of issues with benchmark datasets that just aren't good...
In a lot of cases they are fine and necessary, but IMO, the standard for legit "success" in a lot of ML for science applications should basically be "can this model be used to make real scientific or engineering insights, that would have been very difficult and/or impossible without the proposed idea."
Even if this is a super high bar, I think more papers in ML for science should strive to be truly interdisciplinary and include an actual science advancement... Not just "we modify X and get some improvement on a benchmark dataset that may or may not be representative of the problems scientists could actually encounter." The ultimate goal of "ml for science" is science, not really to improve ML methods imo
This may not be the actual reason in this case, but I think it's good to be aware of: A non-zero chunk of "ai for science" research done at tech companies is basically done for marketing. Even in cases where it's not directly beneficial for the companies products or is unlikely to really lead to anything substantial, it is still good for "prestige"
I'm not an expert on this, so take this with a grain of salt. Chaotic PDEs are extremely sensitive to initial conditions. This essentially makes it so that any numerical solution will (quickly) diverge from the true solution over time. (Just due to floating point error, discretization error, etc.) This is why for a lot of turbulent navier-stokes stuff, people don't necessarily care about the specific phenomena that occur, but look at statistical properties.
I think one of the reasons it is important to preserve conservation laws is that, at the very least, you can be confident that your solution satisfies whatever physical laws your PDE relies on, even if it's almost certainly not the "actual" solution to the PDE. You actually can ensure that a numerical solver will approximately satisfy conservation laws. Then at the very least, even if your solution diverges from the "actual" PDEs solution, you can have some confidence that it's still a useful exploration of possible states.
If conservation laws are not preserved AND your solution diverges from the "actual" PDE solution, then you probably cannot be confident about the model's utility.
it feels like Nvidia has 30 "tile-based DSLs with python-like syntax for ML kernels" that are in the works lol. I think they are very worried about open source and portable alternatives to cuda.
I think very few of these "replace numerical solver with ML model" papers do anything to verify invariants are satisfied (they often are not well preserved).
They basically all just check that the model approximately reproduces some dynamics on a test data of PDEs, that's often sampled from the same distribution as the training dataset...
With ML in particular, there's also the sheer volume of people basically all looking at (essentially) the same problems... so it's kind of like monkeys with type writers spamming ideas until some work.