Show HN: Augment your dataset with LLM distillation techniques(tunetrain.ai)

1 pointsby uavhackerhace 9 meses1 comments

Show HN: Augment your dataset with LLM distillation techniques

https://www.tunetrain.ai

Do you know that Small Language Models (SLM) can outperform LLMs if trained (fine-tuned) on domain specific datasets, like your companies knowledge base?

The issue: fine-tuning a SLM usually requires a dataset of 10k - 100k records, which is huge.

I created a platform for applying "human in the loop" augmentation techniques on your small dataset, so that you can start with maybe 100 records and build-up quickly huge datasets and launch a training without prior knowledge.

I implemented 2 techniques, based on LLM distillation, however: more to come.

HN, what Do you think? Do you see value in this idea? Would you prefer a public API or CLI instead?

I appreciate your help.

Kind regards Pawel

1 comments

ctbellmarhace 9 meses

Pawel,

This looks promising! Is it for text based models only at this time (i.e. no vision/image training)?