HackerTrans
TopNewTrendsCommentsPastAskShowJobs

nitotm

0 karmajoined 지난달

Submissions

Ask HN: What have you used natural language identification tools for?

2 points·by nitotm·25일 전·0 comments

Show HN: ELDC – Natural language identification, faster than FastText and CLD2

github.com
3 points·by nitotm·26일 전·6 comments

comments

nitotm
·그저께·discuss
Location: Spain

Remote: Yes

Willing to relocate: No, but open to discuss.

Technologies: JavaScript, PHP, Python, MySQL, SQLite.

GitHub: https://github.com/nitotm

Résumé/CV: Upon request.

Email: Use contact form https://linktr.ee/nitotm

Salary will not be a problem.

"Full-Stack Web Developer" is my closest label, I've been programming for over 20 years.
nitotm
·그저께·discuss
I'm on a similar situation. When you say "switch stacks" what do you have in mind? It seems your experience is pretty demanded already. Do you have a strong Github profile?
nitotm
·24일 전·discuss
Well, not for every language, but for every language that uses that n-gram. The scoring weights the importance of an n-gram for a particular language.
nitotm
·24일 전·discuss
Thanks. I will try to answer.

ELD works like a traditional language detector, storing n-grams and tuned scores. (So it does not use a modern neural network).

It cleans the input text and extracts words, gets n-grams/tokens, each n-gram hash is searched on a fast hashtable, which points to several score slots for x amount of languages. And we build the scores for each of the found languages.

Sounds simple, and it is, because the work is done when training the database, setting the score values.

The database looks something like {"ngram_1":{Lang_id_1:score, Lang_id_7: score, ...}}, {"ngram_2":{Lang_id_5:score}}

I hope this answers your question. I could go into more detail.

Also, if anybody finds this interesting you could "Vouch" this post, so it goes public as it is hidden since I am a new user.
nitotm
·26일 전·discuss
I was looking for something like this the other day, it can be very helpful.
nitotm
·지난달·discuss
Pretty good.