HackerTrans
TopNewTrendsCommentsPastAskShowJobs

jeffreysmith

no profile record

Submissions

A Basket of Eggs

neotenyai.substack.com
1 points·by jeffreysmith·il y a 4 mois·1 comments

Scoring Open Source Contributors in the Age of AI Slop: Finding Good Eggs

neotenyai.substack.com
1 points·by jeffreysmith·il y a 5 mois·1 comments

Show HN: Good Egg: Trust Scoring for GitHub PR Authors

github.com
4 points·by jeffreysmith·il y a 5 mois·0 comments

Olmo 3: Charting a path through the model flow to lead open-source AI

allenai.org
20 points·by jeffreysmith·il y a 8 mois·3 comments

[untitled]

1 points·by jeffreysmith·il y a 8 mois·0 comments

comments

jeffreysmith
·le mois dernier·discuss
Super cool work. I love seeing this direction taken all the way to hardware.

I'm a big fan of KANs. The really seem like the start of something big and new. We've got a couple of papers out and in the works on KANs. The most relevant to OP's is this one: https://arxiv.org/abs/2512.15742v2

And we just put up a general primer on KANs on YT: https://youtu.be/wgcSsJ69x1c?si=fiUl1YGTgaTt_bn9 Fun stuff if you want to get into the weeds.

And if you are really interested in KANs, you should really check out Ziming (KAN creator)'s blog: https://kindxiaoming.github.io/blog/
jeffreysmith
·le mois dernier·discuss
I played a somewhat unusual role in this whole story. I was the guy who acquired in the original Papers with Code and managed them after they joined Facebook/Meta.

It was super sad to see FB/M abandon the original mission of what PwC was building towards and let the original community resource rot. During the good times, we always talked about how PwC related to HF. So, I think there is a sort poetry to PwC winding up as part of HF, where they probably always belonged. No company is perfect, but HF has been a better than average steward of open source and community resources.

For the younger folks on this thread, you probably have no real feel for just how frustratingly inefficient AI/ML research used to be before people like Robert and Ross of PwC came along to start to bring structure, sanity, and reproducibility to the information needed to work of this kind. And of course, Clem, Julien, and Thomas of HF kicked off an even bigger effort to tame the previously scattered workflow of open AI research into some sort of sane stack.

It's clear that, in 2026, what PwC could be is something much more evolved than what we were able to do back in the day. LLMs + PwC is a huge design space. I hope nielz_r and friends at HF are able to make something truly useful for the community. AI research has both gotten way easier and much harder. e.g. We have a Fable, but Anthro won't let us use it forward our science. Community resources for research are still very much needed.

Best of luck Son of PwC. May you thrive.
jeffreysmith
·il y a 2 mois·discuss
Burnin (London or NYC) | Founding Research Scientist + Founding Engineering Lead | Hybrid (Some on-site) | Full-time

We are building the trust stack for AI code generation, targeted at high-stakes computing where wrong numbers cost real money: a statically typed functional language designed for coding models, a compiler whose guarantees double as audit-grade trust infrastructure, and a coding model fine-tuned on the language with compiler fitness as the training signal. Language and model are co-designed.

Small team out of PyTorch, FAIR, and Meta. Strong institutional VC support from our pre-seed; active investor interest heading into seed.

Founding Research Scientist: own the agenda across the language and the model. Type system design, compiler analyses that produce useful fitness gradients, and fine-tuning open-weight coding models on a language with no pretraining footprint. Publish in PL and ML venues. Strongest fits cross between machine learning, programming languages, and formal methods. PhD preferred, equivalent output equally fine. Stack is Rust, Lean 4, Python.

Founding Engineering Lead: own engineering across compiler internals, language runtime, GPU backends, notebook and library tooling, and the AI infrastructure around training, evaluation, model release, and likely public inference. We already support x86 and ARM, CUDA and AMD, macOS and Linux, and the matrix grows. Real feel for statically typed functional programming expected; the kind of engineer who picks up Lean or Haskell on a weekend because they wanted to. Have led engineering before, formally or not. Stack is mostly Rust with Python where it earns its place, primarily on AWS.

Both roles: comfortable with early-stage ambiguity, define your own roadmap, defend it with evidence.

Apply: RS: https://wellfound.com/l/2Carrr Eng Lead: https://wellfound.com/l/2CewDC

Socials: YT: https://www.youtube.com/channel/UC4DLS_emqwKXO7A9qaOtxlw SS: https://burninai.substack.com/
jeffreysmith
·il y a 4 mois·discuss
OP here. This is the sequel to the original Good Egg post (https://news.ycombinator.com/item?id=47151678).

We ran three experiments since v1 that changed the model substantially.

We tried to detect suspended GitHub accounts from behavioral signals (merge rate, network centrality, TF-IDF on PR titles, LLM classification with ~31K Gemini calls). Best individual AUC was 0.619 on a 1.9% base rate. The merged-PR population is too homogeneous. Accounts that pass code review look like everyone else. The interesting finding: the suspension rate among contributors with merged PRs is under 2%. The review process is a better filter than the discourse around AI slop suggests.

That led us to question the scoring model. The graph score (bipartite construction, personalized ranking, language normalization, the whole pipeline from v1) actively hurts predictions for the contributors who actually need scoring: unknown people with a handful of merged PRs. Merge rate alone outperforms merge rate plus graph at every tier we tested. The new default model is merged / (merged + closed). We also pulled account age out of the score into a separate advisory after DeLong tests showed it adds nothing once you condition on merge rate.

The post has the full data, including the tables.

Next we're working on content scoring (does this PR fit this repo's conventions?) and cold-start tooling (helping new contributors understand project expectations before they submit). Contributor reputation is one input to review triage. The PR itself carries more signal.

Repo: https://github.com/2ndSetAI/good-egg

pip install good-egg

Or just run it via uvx.
jeffreysmith
·il y a 4 mois·discuss
They definitely are a powerful option for smaller scale runs. Very much optimized to have the unit economics and turnaround time work for smaller brands.

I don't really know the answer around supplying your own yarn. I'd assume that's the abnormal case, but just a guess.
jeffreysmith
·il y a 4 mois·discuss
I interviewed these guys for an article on the use of seaweed in yarn and fabric. And I bought the 3D knit seaweed sweater. Great team, with a lot of heart and good intentions.

I'm also a hand knitter, and I don't really see any conflict between what they're doing and hand knitting. The grist of the yarn that you use as a hand knitter is generally much thicker than these machines commonly use. Commercial 3D knitting machines can do all of the stretchy, thin, and light stuff that the modern wardrobe is built around.

As folks note, this technology was really pioneered by Shimaseki's work in Japan just decades ago. What OC and the similar Brooklyn-based Tailored Industry are really innovating on is the business model and connection to production process. Folks like this are really serious about not producing all of the waste that comes with most fashion production processes, and it shows up at several levels of the stack.

For the HN crowd, TI's platform gives you more of a sense of why this sort of tech is really like the cloud for knitwear: https://tailoredindustry.com/platform

Really a fascinating part of the global fashion production world, and one we would all benefit from seeing grow.
jeffreysmith
·il y a 4 mois·discuss
And I filed that suggestion as an enhancement issue on the repo: https://github.com/2ndSetAI/good-egg/issues/43

Thanks for the idea.
jeffreysmith
·il y a 4 mois·discuss
Thanks!

Yeah, scanning non-GitHub is on the roadmap and really should be done. I expect there would be value in understanding all of the current GitHub competitors. And I think the forecasts of new GH competitors getting launched (likely by AI companies) will become relevant in the near future.
jeffreysmith
·il y a 4 mois·discuss
Quick footnote to call out this really good summary from the team at :probabl (the scikit-learn/skore company): https://blog.probabl.ai/maintaining-open-source-age-of-gen-a...
jeffreysmith
·il y a 4 mois·discuss
I'm a bit obsessed with this topic lately, so I'm going to keep refreshing this thread to see if folks have good answers.

One thing I've been working with is this little util to try to do a quick sniff test on the contributors: https://github.com/2ndSetAI/good-egg (Longer explanation on Substack: https://neotenyai.substack.com/p/scoring-open-source-contrib... )

From what I've seen in the data, acceptance rates to all major OSS projects are down since the age of coding agents.

And when I talk to maintainers, most of them are talking about some version of doing fast and easy pocket vetos (leaving the PRs to rot) or even just banning on the first offense.

It's been building for a bit, but I think the crisis point is solidly here. And things like OpenClaw turn up the dials. I'm sure more tools and changes to practices will be coming.
jeffreysmith
·il y a 5 mois·discuss
Jeff, the author, here. We built a tool that scores PR authors by mining their contribution graph from the GitHub API. Every input is a merge/reject decision a human maintainer already made. It doesn't look at PR content or try to detect AI usage. It just answers: has this person gotten code accepted into projects before, and how relevant is that history to your project?

The scoring is graph-based (bipartite user-repo graph, personalized ranking, 180-day recency decay). Scores are context-specific, so the same person can score differently against different repos. The post walks through how Guillermo Rauch scores MEDIUM against his own company's Next.js repo because he has zero merged PRs there, and how v2 rescues that with merge rate and account age.

We validated on 5,129 PRs across 49 repos. Three features survived statistical testing, four didn't. The most surprising failure: text similarity between PR descriptions and project READMEs predicted lower merge rates. We published all of it, including the failures.

More detail on the Substack post.

Repo: https://github.com/2ndSetAI/good-egg (MIT, pip install good-egg). Runs as a CLI, GitHub Action, Python library, and MCP server.
jeffreysmith
·il y a 5 mois·discuss
I think this is really a key problem to solve, but I couldn't convince myself that it was the right solution. So, I put up my alternative proposal, Good Egg: https://github.com/2ndSetAI/good-egg

Key differences: - Based on commit history, with nuance around relatedness of projects, types of projects, age, etc. - Requires no ongoing work. Just add it to your GH Actions CI. - Agent ready with an MCP interface, Python lib, and CLI

Discussion on HN here: https://news.ycombinator.com/item?id=46960412

Feedback and PRs welcome.
jeffreysmith
·il y a 8 mois·discuss
Weird that this late, dupe thread came alive after this/my earlier submission didn't seem to get noticed: https://news.ycombinator.com/item?id=45993118
jeffreysmith
·il y a 8 mois·discuss
Totally. I don't get why people sleep on AI2's launches. They're such powerful platforms for AI R&D.
jeffreysmith
·il y a 8 mois·discuss
I'm one of the many people who Soumith hired to Meta and PyTorch. I had the privilege of working on PyTorch with him and lots of the folks on this post.

As his longtime colleague, the one thing I would want people to know about him and this decision is that Soumith has always viewed PyTorch as a community project. He consistently celebrated the contributions of his co-creators Adam and Sam, and he extended the same view towards the Yangqing and the Caffe2 crew that we merged into PyTorch. At the very beginning, by Soumith's highly intentional design, PyTorch was aimed at being truly developed by and for the AI research community and for many years that was the key way in which we grew the framework, FB PT team, and the wider community. At every single stage of PT's lifecycle, he always ensured that our conception of PT and its community grew to include and celebrate the new people and organizations growing what was possible with PT. He's an incredible talent magnet, and thus more and more smart people kept dedicating their blood, sweat, and tears to making PT bigger and better for more people.

I've worked with some very well known and highly compensated leaders in tech, but *no one* has done the job he has done with ameliorating a bus factor problem with his baby. PT has a unique level of broad support that few other open source technology can reach. In a world of unbounded AI salaries, people who want to move AI research methods forward still freely give their time and attention to PyTorch and its ecosystem. It's the great lever of this era of AI that is moving the world, *due in large part* to the strength of the community he fostered and can now let continue without his direct involvement.

His departure is the end of an era, but it's also operationally a true non-event. PyTorch is going strong and can afford to let one of its creators retire from stewardship. This is precisely what success looks like in open source software.

He deserves our congratulations and our thanks. Enjoy your PT retirement, man.
jeffreysmith
·il y a 10 mois·discuss
American here who went to a Chinese (grad) school for CS and was admitted to every Chinese school I applied to. This is very much a possible route, if you’re appropriately qualified for the program. The main issue is language: outside of HK, programs in English are rare.
jeffreysmith
·il y a 10 mois·discuss
[dead]