Show HN: Vocab extractor for language learners using Stanza and frequency ranks(huggingface.co)
huggingface.co
Show HN: Vocab extractor for language learners using Stanza and frequency ranks
https://huggingface.co/spaces/vladvlasov256/vocab-nlp
0 コメント
—
It takes a short text + learner level (A0–B1) and returns the best words to study, using Stanza for parsing and corpus frequency ranks (SUBTLEX-NL, srLex, SUBTLEX-US) for scoring. Wins at A1/A2, loses at A0 where the LLM picks more obvious words.
I also tried adding multi-word phrases (ADJ+NOUN, VERB+NOUN, phrasal verbs) backed by NPMI-scored collocation whitelists. Couldn't beat GPT there because it just "knows" which phrases matter.
For the phrase work I had to extract collocations from 100M+ OpenSubtitles lines. Published them as a free dataset: https://huggingface.co/datasets/vladvlasov256/opensubs-collo... There are 43K bigrams across English, Dutch, and Serbian.
Source https://github.com/vladvlasov256/vocab-nlp