Tao does state his hopes in the article: "My hope is that the winning submissions will capture the most productive techniques for solving these problems, and/or provide general problem-solving techniques that would also be applicable to other types of mathematical problems."
I think your suggestions are actually complementary. Distillation of the larger networks capable of solving these problems and study of the layers could be part of the process for generating the cheat sheet.
It seems like the lift in the open-source models is being used as a proxy metric, and the core goal is a human understandable yoga [1] for approaching these kinds of equational proofs in universal algebra.
I spent some time in industry working on ML-based credit risk modeling. In my experience, successful shops that have a genuine interest in applying their models to practical decision making with real stakes care deeply about uncertainty quantification and decision theory. Things can get messy very fast though and the challenges faced are often too hyper-specific to one's situation to make sense as part of an academic research program. I think it's been for the best that academic research has tended to focus on the development of general algorithms intended to be broadly useful. Businesses are already well incentivized to take the best of what academia produces and try to get the decision theory right for their particular problems.
Yes, people try this. Check out dynamic tonality. It doesn't necessarily need a system. Experienced guitar players often find themselves unconsciously making little microtonal adjustments through bends and other techniques when playing leads. I found myself doing this just because it sounded better to me. I didn't even notice there was a consistent pattern until I eventually learned the math. For example I'd always want to bend minor thirds slightly sharp and bend the neck to slightly detune major thirds.
I was planning to make a similar comment. Conjecturing that some theory in the string theory landscape [0] gives a theory of quantum gravity consistent with experiments that are possible but beyond what humans may ever be capable of isn't as strong of a claim as it may first appear. The intuition I used to have was that string theory is making ridiculously specific claims about things that may remain always unobservable to humans. But the idea is not that experiments of unimaginable scale and complexity might reveal that the universe is made up of strings or something, it's just that it may turn out that string theory makes up such a rich and flexible family of theories that it could be tuned to the observed physics of some unimaginably advanced civilization. My impression is that string theory is not so flexible that its uninteresting though. There's some interesting theoretical work along these lines around exploring the swampland [1].
I don't think they usually make the distinction in a formal sense, but I think most are aware. The space of explorable mathematics is vastly larger than what the community of mathematicians is capable of collectively thinking about, so a lot of aesthetic judgment goes into deciding what is and what isn't interesting to work on. Mathematicians differ in their tastes too. A sense of sturdiness vs flimsiness is something that might inform this aesthetic judgment, but isn't really something most mathematicians would make part of the mathematics. Often, ones interest isn't the result itself, but some proof technique that brings some sense of insight and understanding, and exploring that often doesn't make much contact with foundational matters.
I'd say that I care deeply about the meaning behind theorems, but just find results which swing widely based on foundational quirks to be less interesting from an aesthetic standpoint. I see the most interesting structures as the ones that are preserved across different reasonable foundations. This is speaking as someone who was trained as a pure mathematician, moved on to other things, but tries to keep up with pure math as a hobby.
It seems like a typo where "integers" is used when the intention was to write "natural numbers". That is the solution to exercise 194 part a) which asked if the set of natural numbers is a field.
Vladimir Arnold famously taught a proof of the insolubility of the Quintic to Moscow Highschool students in the 1960s using a concrete, low-prerequisite approach. His lectures were turned into a book Abel’s Theorem
in Problems and Solutions by V.B. Alekseev which is available online here: https://webhomes.maths.ed.ac.uk/~v1ranick/papers/abel.pdf. He doesn't consider Galois theory in full generality, but instead gives a more concrete topological/geometric treatment. For anyone who wants to get a good grip on the insolubility of the quintic, but feels overwhelmed by the abstraction of modern algebra, I think this would be a good place to start.
Let's take the Fundamental Theorem of Calculus as an example[0]:
f'(x) = lim_{h->0} {f(x + h) - f(x)} / {h}
This isn't the Fundamental Theorem of Calculus, it's the usual definition of the derivative of a function of a single real variable. The Fundamental Theorem of Calculus establishes the inverse relationship between differentiation and integration [0].
Unless you're Ramanujan, every mathematician has spent hours banging their head against a literal or metaphorical wall (or both!)
Ramanujan was no stranger to banging his head against the wall. My impression from Kanigel's The Man Who Knew Infinity is that his work ethic and mathematical fortitude were as astonishing as his creativity. For much of his career, he couldn't afford paper in quantity and did his hard work on stone slate, only recording the results. This could make it seem like his results were a product of pure inspiration because he left no trace of the furious activity and struggle that was involved.
From The Man Who Knew Infinity:
When he thought hard, his face scrunched up, his eyes narrowed into
a squint. When he figured something out, he sometimes seemed to talk to
himself, smile, shake his head with pleasure. When he made a mistake,
too impatient to lay down his slate pencil, he twisted his forearm toward
his body in a single fluid motion and used his elbow, now aimed at the
slate, as an eraser.
Ramanujan's was no cool, steady Intelligence, solemnly applied to the
problem at hand; he was all energy, animation, force.
Decimate is a word that often raises hackles, at least those belonging to a small but committed group of logophiles who feel that it is commonly misused. The issue that they have with the decline and fall of the word decimate is that once upon a time in ancient Rome it had a very singular meaning: “to select by lot and kill every tenth man of a military unit.” However, many words in English descended from Latin have changed and/or expanded their meanings in their travels. For example, we no longer think of sinister as meaning “on the left side,” and delicious can describe things both tasty and delightful. Was the “to kill every tenth man” meaning the original use of decimate in English? Yes, but not by much. It took only a few decades for decimate to acquire its broader, familiar meaning of “to severely damage or destroy,” which has been employed steadily since the 17th century.
Your intuition's not bad. The expected value for the longest run of heads in N total flips of a fair coin is around log2(N) - 1 with a standard deviation that's approximately 1.873 plus a term that vanishes as N grows large. log2(10B) - 1 is approximately 32 and with that standard deviation, even a run of 100 in 10B flips is incredibly unlikely. For more info see Mark F. Schilling's paper, "The Longest Run of Heads" available here https://www.csun.edu/~hcmth031/tlroh.pdf.
Yeah, I think surrogate gradients are usually used to train spiking neural nets where the binary nature is considered an end in itself, for reasons of biological plausibility or something. Not for any performance benefits. It's not an area I really know that much about though.
It sounds like jongjong was probably using surrogate gradients. You keep the step activation in the forward pass but replace with a smooth approximation in the backwards pass.
I think your suggestions are actually complementary. Distillation of the larger networks capable of solving these problems and study of the layers could be part of the process for generating the cheat sheet.