Analyzing frontier LLM performance on my favorite daily puzzle game (https://www.nicksypteras.com/blog/cbs-benchmark.html) Next step is to assess how well the LLMs can create their own new, logically satisfiable puzzles in the same style. Then I'll have them battle it out, with one model creating a puzzle and the other attempting to solve it!
Congrats on launching! One immediate thought is that people will always be wary of running LLM-generated code on their machines even if it's sandboxed. Is one of the future business cases for this to host a remote execution environment that pctx can call out to rather than running the code locally?
Ya interesting thought - would be fascinating if generating games w/solutions is part of the training data pipeline. There's been previous work done on on testing LLMs on logic puzzles[1][2][3] so they could possibly be building off those ideas to improve performance.
I'm impressed it recommended so many books i've already read and liked! I have a big reading backlog but once it's whittled down I will likely come back to this. One feature request would be to also show a "why this is recommended" for each recommendation so I can further narrow down the list for what I'm looking for
"Counter Chinese Influence in International Governance Bodies" and grouping them in with US "adversaries" and "rivals" is quite undiplomatic language to throw in under "Lead in International AI Diplomacy and Security" section. Diplomacy with China should be an important part of this initiative but will inevitably be bungled.
1984: U.S. withdraws.
2003: U.S. rejoins.
2011: U.S. stops paying dues after Palestine joins.
2017: U.S. announces withdrawal (effective end of 2018).
2023: U.S. rejoins, pledges to repay dues.
2025: U.S announces withdrawal
Same here! Kiwix comes in clutch on flights. I've used it so many times to get background knowledge on topics mid-read. Plus free and open source. Such a great service.
I think that would be one of the success cases described in the article because HITL is an integral part of good customer support chatbots. Support chats can be escalated to a human whenever the agent is unable to provide a satisfactory answer to the user.
That's pretty cool! Now I can imagine a tool that gives you a prediction before you even post and then offers suggestions for how to increase performance...
A while back I made a little script (for fun/curiosity) that would do this for HN profiles. It’d use their submission and comment history to infer a profile including similar stuff like location, political leaning, career, age, sex, etc. Main motivation was seeing some surprising takes in various comment threads and being curious about where it might have came from. Obviously no idea how accurate the profiles were, but it was similarly an interesting experiment in the ability of LLMs to do this sort of thing.
So much nostalgia for my Pebble. Got it right as I was seriously getting into programming. I still remember how magical it felt writing the C code to build my first watch face and how proud I was to show people. Amazing news <3