HackerTrans
TopNewTrendsCommentsPastAskShowJobs

lukemelas

no profile record

comments

lukemelas
·2 năm trước·discuss
Author of the Kalamang paper here. We’ve thought about this a good amount (e.g. there are interesting Mesoamerican scripts), but ultimately we decided to work on low-resource languages, as they’re much more useful. It’s also possible to use them as an evaluation benchmark, which isn’t really possible if nobody speaks the language. We’d like to expand the scope beyond Kalamang, though, and maybe at some point we’ll investigate an undeciphered language as well.
lukemelas
·2 năm trước·discuss
Author here (of the Kalamang paper). One of my coauthors was the human baseline and he spent many months reading the grammar book (and is incredibly talented at learning languages). It’s really a very high bar.
lukemelas
·3 năm trước·discuss
Author here. Was not expecting to see this pop up on Hacker News!

This was a short write-up of a set of experiments exploring the importance of attention in transformers. I'm glad to see that people are still reading it and hopefully finding it interesting.

Side note: the title should probably have (2021) in it, as this was posted to Arxiv in May 2021 (concurrently with a number of similar works such as MLP-Mixer).