HackerTrans
TopNewTrendsCommentsPastAskShowJobs

sangel

no profile record

Submissions

Retentive Network: A Successor to Transformer for Large Language Models

arxiv.org
112 points·by sangel·3 jaar geleden·19 comments

comments

sangel
·2 maanden geleden·discuss
This is the entire formal verification effort, as far as I can tell: https://codeberg.org/gregburd/aether/src/branch/main/aether/...

They use TLA+ with TLC which model checks the write ahead log (a component of the system). But that only models the WAL protocol, not the actual Rust code and not the other 99% of the system.

Any formal verification is of course awesome to see though.
sangel
·9 maanden geleden·discuss
My totally uneducated guess is that they leak exaggerated numbers on purpose to make the real numbers look less bad in comparison. The idea being that a few days before the official news everyone is talking about a potential 30 or 40K people layoff. Then, when the official news come out and state that they are laying off 14K people, it sounds less dramatic (in relative terms). This makes people (who are not affected by this) to feel like the news were overblown and are not as significant.
sangel
·vorig jaar·discuss
Obviously not. Suppose the input to my function is a 64-bit integer. My test cannot possibly try every possible 64-bit integer. It would take years for such a test to finish.

This is why tools like formal verification and symbolic analyses can help you establish that for all possible integers X, your function does the right thing (for some definition of “right”). You get this assurance without having to actually enumerate all X.
sangel
·3 jaar geleden·discuss
Very inefficient. Like wildly so. Specifically if you have a very small database and you preprocess it with their techniques, the resulting database is petabytes in size. But the results are very beautiful.

There are no obvious ways to improve on this work, so it is not a matter of engineering. We really do need another breakthrough result to get us closer to practicality.
sangel
·3 jaar geleden·discuss
At least in my department, this will not work as long as the applicant's supervisor or dissertation committee members are still in the department. The crux of the issue, in my mind, is that it is hard to have candid and unbiased discussion about a candidate when the supervisor / dissertation committee is part of that discussion.
sangel
·3 jaar geleden·discuss
This is strange to me. Many universities, including mine, avoid interviewing applicants with a PhD or postdoc from the same institution to which they are applying.