WhiteOwlEd

2 lata temudiscuss

With OpenAI and other LLMs, web development is accelerating. For example, I put together a AI call center demo ( https://www.youtube.com/watch?v=Vv7mI_qRrhE ) by using Open AI o1-preview. There I could take a lot of different files on typescript and backend server stuff written in python. I would add logs into the mix to make one massive prompt, and then I would let the AI work on reasoning in the cases where I needed to accelerate the writing of additional code.

WhiteOwlEd

2 lata temudiscuss

Building on this, Human preference optimization (such as Direct Preference Optimization or Kahneman Tversky Optimization) could be used to help in refining models to create better data.

I wrote about this more recently in the context of using LLMs to improve data pipelines. That blog post is at: https://www.linkedin.com/posts/ralphbrooks_bigdata-dataengin...

WhiteOwlEd

2 lata temudiscuss

The author is actually doing something that will help with the job search, and that is reaching out by methods other than the resume.

Having a blog on the front page (coupled with the value that he can bring) should at least give him a few warm leads for job opportunities.

WhiteOwlEd

3 lata temudiscuss

If you are using no-code solutions, increasing an "idea" in a dataset will make that idea more likely to appear.

If you are fine-tuning your own LLM, there are other ways to get your idea to appear. In the literature this is sometimes called RLHF or preference optimization, and here are a few approaches:

Direct Preference Optimization

This uses Elo-scores to learn pairwise preferences. Elo is used in chess and basketball to rank individuals who compete in pairs.

@argilla_io on X.com has been doing some work in evaluating DPO.

Here is a decent thread on this: https://x.com/argilla_io/status/1745057571696693689?s=20

Identity Preference Optimization

IPO is research from Google DeepMind. It removes the reliance of Elo scores to address overfitting issues in DPO.

Paper: https://x.com/kylemarieb/status/1728281581306233036?s=20

Kahneman-Tversky Optimization

KTO is an approach that uses mono preference data. For example, it asks if a response is "good or not." This is helpful for a lot of real word situations (e.g. "Is the restaurant well liked?").

Here is a brief discussion on it:

https://x.com/ralphbrooks/status/1744840033872330938?s=20

Here is more on KTO:

* Paper: https://github.com/ContextualAI/HALOs/blob/main/assets/repor...

* Code: https://github.com/ContextualAI/HALOs

comments