Hey, creator of Social RV here! Awesome to hear you're enjoying what we're doing.
Some answers to your questions:
- the target pool has 275 targets in it
- we USED to use the last 9 targets as decoys, but changed to randomly sampling 9 targets from the pool instead several months ago. I've updated the FAQ to reflect that
- the unique identifiers we show the LLM for the decoy targets is not the file name but rather the DB primary key for that target. There should be no information in it the AI could use to bias a decision
- in regards to the tournament-style elimination, we have a new judge coming out soon that does a single pass. When this was originally built, the single-pass wasn't reliable enough on available models
Thanks very much for your thoughtful feedback and questions about what we're doing!
the best engineers focus on outcomes. They use the best tools available to achieve the best outcome as fast as possible
AI coding tools are increasingly proving to be some of the highest leverage tools we’ve seen in decades. They still require some skill to use effectively and have unique trade-offs, though. Mastering them is the same as anything else in engineering, things are just moving faster than we’re used to.
the next generation of successful engineers will be able to do more with less, producing the output of an entire team by themselves.
be one of those 100x engineers, focused on outcomes, and you'll always be valuable
social-rv is our attempt to do yet another reproduction, as publicly as possibly. And we got the same result