HackerTrans
TopNewTrendsCommentsPastAskShowJobs

nitotm

0 karmajoined в прошлом месяце

Submissions

Ask HN: What have you used natural language identification tools for?

2 points·by nitotm·25 дней назад·0 comments

Show HN: ELDC – Natural language identification, faster than FastText and CLD2

github.com
3 points·by nitotm·26 дней назад·6 comments

comments

nitotm
·позавчера·discuss
Location: Spain

Remote: Yes

Willing to relocate: No, but open to discuss.

Technologies: JavaScript, PHP, Python, MySQL, SQLite.

GitHub: https://github.com/nitotm

Résumé/CV: Upon request.

Email: Use contact form https://linktr.ee/nitotm

Salary will not be a problem.

"Full-Stack Web Developer" is my closest label, I've been programming for over 20 years.
nitotm
·позавчера·discuss
I'm on a similar situation. When you say "switch stacks" what do you have in mind? It seems your experience is pretty demanded already. Do you have a strong Github profile?
nitotm
·24 дня назад·discuss
Well, not for every language, but for every language that uses that n-gram. The scoring weights the importance of an n-gram for a particular language.
nitotm
·24 дня назад·discuss
Thanks. I will try to answer.

ELD works like a traditional language detector, storing n-grams and tuned scores. (So it does not use a modern neural network).

It cleans the input text and extracts words, gets n-grams/tokens, each n-gram hash is searched on a fast hashtable, which points to several score slots for x amount of languages. And we build the scores for each of the found languages.

Sounds simple, and it is, because the work is done when training the database, setting the score values.

The database looks something like {"ngram_1":{Lang_id_1:score, Lang_id_7: score, ...}}, {"ngram_2":{Lang_id_5:score}}

I hope this answers your question. I could go into more detail.

Also, if anybody finds this interesting you could "Vouch" this post, so it goes public as it is hidden since I am a new user.
nitotm
·27 дней назад·discuss
I was looking for something like this the other day, it can be very helpful.
nitotm
·в прошлом месяце·discuss
Pretty good.