I think a lot of people here would be surprised how much of a step up this is from classic leetcode or DSA stuff. I have been involved in introducing people to AoC and helping them and the amount of people who have basic knowledge of algorithms but struggled to parse the input from a file was a little shocking to me at first. Of course, I do not blame anyone for not knowing something, classic academic courses can be misguided sometimes.
This doesn't negate the fact that there is somewhat of a lack in problems with more outside world interaction and it would be cool to see more of that.
I think this is because for a long time on the internet there was this intense discussion of "introvert vs extrovert" and memes about what it means to be in each camp that we have forgotten that it can vary greatly from one person to another and from one setting to another. It was also somehow "cool" to be an introvert on the internet, in some circles.
These days I feel like those conversations have dies down. And I've also realized that I', more social that I'd previously liked to admit. Maybe age also plays a factor here.
As a non-architect, I would love if you could explain some of what makes the non-duck designs _so_ horrible. Some of them looked more than fine to me at first glance. It's also something that can be rerun over and over with little interaction in-between runs so one could generate a handful of designs starting from different seeds and get inspiration.
Thank you for your reply. I had suspected it was the second chance pool but I had never seen that mechanism in action so I thought it'd get re-posted, not that it would appear as if it had just been posted. I had read it when it first appeared.
Hello! I'm Charbel, an engineer specializing in Software and Artificial Intelligence (I have also studied Telecommunications and Electronics within my double degree program) graduating this month. I have exprience in software engineering, machine learning, and AI (mostly NLP but I have dabbled in computer vision as well). I am looking for new and interesting SWE or AI opportunities in France and Europe. I will require a visa sponsorship. Not all of my experience is reflected on my LinkedIn, there's a lot more than meets the eye. So if you're looking for a junior engineer with a ton of creativity and passion and multidisciplinary skills shoot me an email and we can discuss more over a phone call or a meeting!
I don't think speculative decoding proves that they consume less/more energy per question.
Regardless if the question/prompt is simple or not (for any definition of simple), if the target output is T tokens, the larger model needs to generate at least T tokens, if the small and large models disagree then the large model will be called to generate more than T tokens. The observed speedup is because you can infer K+1 tokens in parallel based on the drafts of the smaller model instead of having to do it sequentially. But I would argue that the "important" computation is still done (also the smaller model will be called the same number of times regardless of the difficulty of the question, bringing us back to the same problem that LLMs won't vary their energy consumption dynamically as a function of question complexity).
Also, the rate of disagreement does not necessarily change when the question is more complex, it could be that the 2 models have learned different things and could disagree on a "simple" question.
Multi-token prediction looks very interesting and quite elegant. It seems more efficient than predictive sampling.
From what I understand, they are essentially training it to have some form of representation of the context as a whole that is then used to generate the next n tokens, I feel like this is a nice next step towards "smarter" models. I wonder if a similar thing can be done for the inputs.
It's a shame they didn't compare it to llama3 since they had both a 6.7B and a 13B multi-token model. From what I could gather, the intruction-tuned llama3 is much better on HumanEval for example.
This paper [1] does atempt that and reports similar performance compared to conventional pre-training. However, they do start off by doing a normal full-rank training and claim that it is needed to 'warm start' the training process.