Playing Around with Machine Translation(davidabell.substack.com)
davidabell.substack.com
Playing Around with Machine Translation
https://davidabell.substack.com/p/playing-around-with-machine-translation
48 comments
It's not under the radar because that's sadly not the case, although it looks like it should work better on the surface. Neural machine translation is just more consistent, doesn't hallucinate and can be easily and cheaply trained over time.
They have some benefits, but as a lot of LLM research has found, they are not production ready. Yet.
They have some benefits, but as a lot of LLM research has found, they are not production ready. Yet.
I've tried using ChatGPT to translate Latin texts from the Renaissance. I know enough Latin (intermediate, but can figure it out with a dictionary) to check it, and it was very, very impressive. What blew me away was not just the fluency of the translation but the fact that I could drop in highly imperfect OCR'd text from Google Books, and it didn't have any trouble making sense of garbled passages. This ability makes it a really distinct advance on Google Translate and the like, at least for my purposes.
Also, another underrated feature: I asked it to summarize each page in a single sentence, while also picking out the passages most relevant to my research question. It did a great job.
Also, another underrated feature: I asked it to summarize each page in a single sentence, while also picking out the passages most relevant to my research question. It did a great job.
That is a truly exceptional use case, and I'm impressed too.
I should have specified above that I'm referring to the practicalities of professional translation workflows as they currently exist for things like high volume flows in dozens of formats, and translation memory leveraging etc.
I should have specified above that I'm referring to the practicalities of professional translation workflows as they currently exist for things like high volume flows in dozens of formats, and translation memory leveraging etc.
>It's not under the radar because that's sadly not the case
It is the case. Haven't seen anyone who uses both who thinks otherwise.
also https://arxiv.org/abs/2301.13294. This benchmarks 3.5 which is quite a bit worse than 4 as it is against Google, NLLB, Deepl
here https://arxiv.org/pdf/2304.02210.pdf, GPT wins overwhelmingly with human evaluations. Seems like the typically evaluation models aren't really cutting it anymore especially BLEU
>Neural machine translation is just more consistent, doesn't hallucinate
It's not more consistent. The 2nd bit is just wrong lol.
One of the biggest complaints of Deepl is the tendency to make stuff up to make translations seem more natural.
Summarization and Translation are the tasks GPT models hallucinate the least.
It is the case. Haven't seen anyone who uses both who thinks otherwise.
also https://arxiv.org/abs/2301.13294. This benchmarks 3.5 which is quite a bit worse than 4 as it is against Google, NLLB, Deepl
here https://arxiv.org/pdf/2304.02210.pdf, GPT wins overwhelmingly with human evaluations. Seems like the typically evaluation models aren't really cutting it anymore especially BLEU
>Neural machine translation is just more consistent, doesn't hallucinate
It's not more consistent. The 2nd bit is just wrong lol.
One of the biggest complaints of Deepl is the tendency to make stuff up to make translations seem more natural.
Summarization and Translation are the tasks GPT models hallucinate the least.
In my experience, ChatGPT tends to produce more fluent output, but is less likely to closely follow the input. For some high-resource language pairs, complete mistranslations are rare, but for other languages, not so much. Of the ones I can evaluate, Burmese is particularly error-prone:
ChatGPT translates ငါ့ကို ဒုတိယ အခွင့်ရေး တစ်ခု ပေးခဲ့တယ်။ as "I have received a second warning.", which is incorrect. အခွင့် https://en.wiktionary.org/wiki/%E1%80%A1%E1%80%81%E1%80%BD%E... does not mean "warning", even though that is a likely completion of "I have received a second " in English.
Google Translate gives me "Gave me a second chance.", which closely matches the Burmese sentence down to dropping the subject (common in Burmese, rare in English) which makes the translation sound weird.
So any claim that ChatGPT is better/worse at translating really needs to specify the languages involved and what your goal for the translation is. (E.g. the benchmark paper you link seems to focus on the ability to steer the translation by providing additional context.)
ChatGPT translates ငါ့ကို ဒုတိယ အခွင့်ရေး တစ်ခု ပေးခဲ့တယ်။ as "I have received a second warning.", which is incorrect. အခွင့် https://en.wiktionary.org/wiki/%E1%80%A1%E1%80%81%E1%80%BD%E... does not mean "warning", even though that is a likely completion of "I have received a second " in English.
Google Translate gives me "Gave me a second chance.", which closely matches the Burmese sentence down to dropping the subject (common in Burmese, rare in English) which makes the translation sound weird.
So any claim that ChatGPT is better/worse at translating really needs to specify the languages involved and what your goal for the translation is. (E.g. the benchmark paper you link seems to focus on the ability to steer the translation by providing additional context.)
I’m not making a claim for chatGPT so much as I am making a claim for GPT style models.
It’s not really a question of high resource vs low resource languages so much as what languages ended up in the training corpus.
1. There’s a lot of transfer learning going on with predict the next token LLMs. A model trained on 500B tokens of English and 50b tokens of French will speak French far better than if it was trained on only 50b tokens of French.
2. You don’t need parallel corpora for every single pair you want to translate to. This means that GPT LLMs only need single text data for the vast majority of languages. Training most NMT models you would need Burmese/English parallel data.
Both of the above combine to mean that not only is quality demonstrably better, amount of data needed is lower too.
GPT's burmese isn't worse because it's low resource. It's because open ai made no specific attempt to include burmese text.
They're not even trying. GPT-3 was 93% English with the 2nd biggest holder less than 2%
It’s not really a question of high resource vs low resource languages so much as what languages ended up in the training corpus.
1. There’s a lot of transfer learning going on with predict the next token LLMs. A model trained on 500B tokens of English and 50b tokens of French will speak French far better than if it was trained on only 50b tokens of French.
2. You don’t need parallel corpora for every single pair you want to translate to. This means that GPT LLMs only need single text data for the vast majority of languages. Training most NMT models you would need Burmese/English parallel data.
Both of the above combine to mean that not only is quality demonstrably better, amount of data needed is lower too.
GPT's burmese isn't worse because it's low resource. It's because open ai made no specific attempt to include burmese text.
They're not even trying. GPT-3 was 93% English with the 2nd biggest holder less than 2%
>> 2. You don’t need parallel corpora for every single pair you want to translate to.
Neural Machine Translation models don't need those either. That's been the case at least since 2016.
That was one of the big claims by Google, that Google Translate had created its own "interlingua" thanks to which it could translate between languages for which it doesn't have parallel corpora:
Following several months of testing, the researchers behind the AI have seen it be able to blindly translate languages even if it's never studied one of the languages involved in the translation. "An example of this would be translations between Korean and Japanese where Korean⇄Japanese examples were not shown to the system," the Mike Schuster, from Google Brain wrote in a blogpost.
https://www.wired.co.uk/article/google-ai-language-create
It was obvious already back then that this "interlingua" was no other than English, e.g., from the same article:
The team said the system was able to make "reasonable" translations of the languages it had not been taught to translate. In one instance, a research paper published alongside the blog, says the AI was taught Portuguese→English and English→Spanish. It was then able to make translations between Portuguese→Spanish.
But, hey, hype.
Neural Machine Translation models don't need those either. That's been the case at least since 2016.
That was one of the big claims by Google, that Google Translate had created its own "interlingua" thanks to which it could translate between languages for which it doesn't have parallel corpora:
Following several months of testing, the researchers behind the AI have seen it be able to blindly translate languages even if it's never studied one of the languages involved in the translation. "An example of this would be translations between Korean and Japanese where Korean⇄Japanese examples were not shown to the system," the Mike Schuster, from Google Brain wrote in a blogpost.
https://www.wired.co.uk/article/google-ai-language-create
It was obvious already back then that this "interlingua" was no other than English, e.g., from the same article:
The team said the system was able to make "reasonable" translations of the languages it had not been taught to translate. In one instance, a research paper published alongside the blog, says the AI was taught Portuguese→English and English→Spanish. It was then able to make translations between Portuguese→Spanish.
But, hey, hype.
>Neural Machine Translation models don't need those either. That's been the case at least since 2016.
Right. But those models still operate on parallel corpora alone.
In LLMs, translation pairs are not the only data improving translations. Context, Cultural Knowledge gleaned from other sources. Fluidity of the target language writing itself.
>But, hey, hype
It's not hype man. GPT-4 is much better for distant language pairs.
Right. But those models still operate on parallel corpora alone.
In LLMs, translation pairs are not the only data improving translations. Context, Cultural Knowledge gleaned from other sources. Fluidity of the target language writing itself.
>But, hey, hype
It's not hype man. GPT-4 is much better for distant language pairs.
I used GPT-4 and this was the result:
Please translate this:
ငါ့ကို ဒုတိယ အခွင့်ရေး တစ်ခု ပေးခဲ့တယ်။
The sentence "ငါ့ကို ဒုတိယ အခွင့်ရေး တစ်ခု ပေးခဲ့တယ်။" translates to "They gave me a second chance." in English.
Please translate this:
ငါ့ကို ဒုတိယ အခွင့်ရေး တစ်ခု ပေးခဲ့တယ်။
The sentence "ငါ့ကို ဒုတိယ အခွင့်ရေး တစ်ခု ပေးခဲ့တယ်။" translates to "They gave me a second chance." in English.
Your reply is not passing my sniff test. Your hype and bias are showing.
It may present more fluent text, but if it doesn't know it's strayed from the source text and you can't tell either (because you don't understand source language) then you'll end up with error-laden pseudo translations. At least with NMT you know the errors are consistent.
I don't know who you know who thinks GPT is ahead, but nobody in the very well funded translation industry has a GPT powered translation engine for the key reason that it's not ready for production. For a human to post edit MT, we're mainly talking fixing broken vocab. You'd never present raw MT to a client. It needs editing. Heavily. To think LLM translation doesn't need editing is either coming from someone not in the industry, or blinded by hype. And the kind of editing issues are more insidious, like those found in voice dictated texts. Homophones aren't flagged by QA software because they are real words. Just like LLMs make real sentences, except when they don't, but good luck detecting that, and editing out the additional meaning it has decided to inject.
Have you tried to run the GPT-4 API on a segmented xliff at all? If the segmentation is bad, and full of tags, GPT4 breaks completely. It tries to close sentences that run across segments, it can't handle tags in-line (the chatgpt interface can, but you can't use that at scale).
It can do some impressive work, don't get me wrong, but I'm not sure how hands-on you've really been if you think it's a solved problem.
Production translation is a non-trivial output. The entire industry hasn't released an LLM solution yet for translation (excepting the rewording mini features). What makes you think you know more than those on the ground? Or have you developed something that's still in stealth?
EDIT: Oh wow, all of your 107 submissions to HN from the 6 months your account has existed, have been about AI and LLMs. I guess I got the hype part right. As for industry knowledge the jury is still out, but this could well be the classic HN "I understand tech so obviously I understand everything" play. Keep us posted!
It may present more fluent text, but if it doesn't know it's strayed from the source text and you can't tell either (because you don't understand source language) then you'll end up with error-laden pseudo translations. At least with NMT you know the errors are consistent.
I don't know who you know who thinks GPT is ahead, but nobody in the very well funded translation industry has a GPT powered translation engine for the key reason that it's not ready for production. For a human to post edit MT, we're mainly talking fixing broken vocab. You'd never present raw MT to a client. It needs editing. Heavily. To think LLM translation doesn't need editing is either coming from someone not in the industry, or blinded by hype. And the kind of editing issues are more insidious, like those found in voice dictated texts. Homophones aren't flagged by QA software because they are real words. Just like LLMs make real sentences, except when they don't, but good luck detecting that, and editing out the additional meaning it has decided to inject.
Have you tried to run the GPT-4 API on a segmented xliff at all? If the segmentation is bad, and full of tags, GPT4 breaks completely. It tries to close sentences that run across segments, it can't handle tags in-line (the chatgpt interface can, but you can't use that at scale).
It can do some impressive work, don't get me wrong, but I'm not sure how hands-on you've really been if you think it's a solved problem.
Production translation is a non-trivial output. The entire industry hasn't released an LLM solution yet for translation (excepting the rewording mini features). What makes you think you know more than those on the ground? Or have you developed something that's still in stealth?
EDIT: Oh wow, all of your 107 submissions to HN from the 6 months your account has existed, have been about AI and LLMs. I guess I got the hype part right. As for industry knowledge the jury is still out, but this could well be the classic HN "I understand tech so obviously I understand everything" play. Keep us posted!
> It may present more fluent text, but if it doesn't know it's strayed from the source text and you can't tell either (because you don't understand source language) then you'll end up with error-laden pseudo translations. At least with NMT you know the errors are consistent.
To be fair, this is an infamous failure mode of neural mtl too, and a big part of what makes the discourse around GPT so ... evocative of the discourse in 2017.
To be fair, this is an infamous failure mode of neural mtl too, and a big part of what makes the discourse around GPT so ... evocative of the discourse in 2017.
>It may present more fluent text, but if it doesn't know it's strayed from the source text and you can't tell either (because you don't understand source language) then you'll end up with error-laden pseudo translations.
I understand Korean. I've tested it there too and my evaluations haven't changed. It's better. A lot better.
That aside, nothing you've said here doesn't apply to NMT.
I'm going to be Frank here. You don't what you're talking about when it comes NMT and it makes this conversation very frustrating. You keep mentioning cons that apply to NMT as well.
Literally almost all SOTA NMT models (NLLB for instance) today are Transformers or some transformer variant (In fact this was the original proposition in the famous Attention Paper). They're just transformers trained for a translation primary objective.
Those cons of GPT, NMT inherit them too. They hallucinate, they can make pseudo translations etc.
>EDIT: Oh wow, all of your 107 submissions to HN from the 6 months your account has existed, have been about AI and LLMs.
So ? That's what I like to do. Is this some attempt at an ad hominem?
The 2nd paper I linked is about document level translations.
>The entire industry hasn't released an LLM solution yet for translation (excepting the rewording mini features). What makes you think you know more than those on the ground?
Training and inference for GPT scale models is extremely expensive.
I understand Korean. I've tested it there too and my evaluations haven't changed. It's better. A lot better.
That aside, nothing you've said here doesn't apply to NMT.
I'm going to be Frank here. You don't what you're talking about when it comes NMT and it makes this conversation very frustrating. You keep mentioning cons that apply to NMT as well.
Literally almost all SOTA NMT models (NLLB for instance) today are Transformers or some transformer variant (In fact this was the original proposition in the famous Attention Paper). They're just transformers trained for a translation primary objective.
Those cons of GPT, NMT inherit them too. They hallucinate, they can make pseudo translations etc.
>EDIT: Oh wow, all of your 107 submissions to HN from the 6 months your account has existed, have been about AI and LLMs.
So ? That's what I like to do. Is this some attempt at an ad hominem?
The 2nd paper I linked is about document level translations.
>The entire industry hasn't released an LLM solution yet for translation (excepting the rewording mini features). What makes you think you know more than those on the ground?
Training and inference for GPT scale models is extremely expensive.
>but nobody in the very well funded translation industry has a GPT powered translation engine for the key reason that it's not ready for production.
Translator for 20 years talking about using GPT for Japanese-English.
https://www.youtube.com/watch?v=5KKDCp3OaMo&t
https://www.youtube.com/watch?v=8JUepj7wIl0
Translator for 20 years talking about using GPT for Japanese-English.
https://www.youtube.com/watch?v=5KKDCp3OaMo&t
https://www.youtube.com/watch?v=8JUepj7wIl0
Indeed. That translator, Tom Gally, is also a Professor at the University of Tokyo and a regular on this site: https://news.ycombinator.com/user?id=tkgally
>> Keep us posted!
Dude, don't be an asshole. You got useful experience. Help others with it instead of putting them down.
Dude, don't be an asshole. You got useful experience. Help others with it instead of putting them down.
It's fine, people can disagree robustly sometimes. I'd still buy him a beer, or whatever he wanted to drink, if we were in person. It's good to talk openly, right?
Talk openly and argue robustly by all means but why make things personal? That just adds noise to an already noisy conversation. We should try to maximise the amount of knowledge in our discussions, not bring it down.
But I guess that would indeed sound much better around beers.
But I guess that would indeed sound much better around beers.
I wrote a much longer reply but looks like you deleted the downvoted comment and reposted it.
> It's not more consistent.
I think the parent was probably saying for a given input, Google MT provides the same output.
What is the value of temperature/variability in a LLM powered MT model?
I’d assume given the same inputs, you should only be given the best output.
> It's not more consistent.
I think the parent was probably saying for a given input, Google MT provides the same output.
What is the value of temperature/variability in a LLM powered MT model?
I’d assume given the same inputs, you should only be given the best output.
>I think the parent was probably saying for a given input, Google MT provides the same output.
I don't care about being given the exact same output (you're not getting deterministic translations from people either). I care about quality translations. Variability for GPT style translations is much more about word choice and style than wrong or extremely different translations. and if i really wanted to, i could guide both (word choice, style) either with instructions or examples.
I don't care about being given the exact same output (you're not getting deterministic translations from people either). I care about quality translations. Variability for GPT style translations is much more about word choice and style than wrong or extremely different translations. and if i really wanted to, i could guide both (word choice, style) either with instructions or examples.
Good news! Per your second link, GPT-4 is a stunning improvement up to "borderline passes quality control"!
"Neural machine translation... doesn't hallucinate..." I've looked at neural MT output from Arabic and Russian, and there was lots of hallucination.
[deleted]
Yeah, GPT-4 is so good Iceland is using it for language preservation of Icelandic as it translates text so well, letting them enrich the language with new works.
Like you, I’ve definitely often though “Come ON!” about the increasingly archaic Google Translate in light of DeepL etc. It has really stagnated over the years.
Like you, I’ve definitely often though “Come ON!” about the increasingly archaic Google Translate in light of DeepL etc. It has really stagnated over the years.
I worked as a freelance Japanese-to-English translator for twenty years, until 2005. After a break in academia, I am now semiretired, doing translation part-time and following closely developments in machine translation and in AI in general.
Two advantages of LLM-based translation are that it is promptable and nondeterministic. While Google Translate, DeepL, etc. will translate a text the same way every time, LLMs let you specify the purpose and desired style of the translation, and you can ask for multiple translations of the same text.
Lately, I’ve been translating some speeches from Japanese to English. While I used to translate such texts by myself, now I first run them through LLMs (Claude 2 and GPT-4) to get three or four different English versions. I refer to those MT versions as well as the Japanese original while I prepare my own draft. This has two advantages for me: the multiple translations often give me ideas for English expressions that would not have occurred to me on my own, and I am able to devote more of my energy to polishing the final versions.
Another way I use LLMs is as a thesaurus on overdrive. If I have a Japanese expression that I understand but can’t think of a good way to express in English—this happens a lot; that’s why translation is difficult—I will give the expression to GPT-4, explain the purpose of the text, provide additional context if necessary, and ask for ten English renditions of the expression. There will almost always be one or two versions that I like.
Here is an example of ten English translations by GPT-4 of the same Japanese text:
https://www.gally.net/temp/20230915pmtranslations.html
Several months ago, I made some videos about ChatGPT and translation, if anyone is interested:
https://www.youtube.com/@Tom_Gally_UTokyo
Two advantages of LLM-based translation are that it is promptable and nondeterministic. While Google Translate, DeepL, etc. will translate a text the same way every time, LLMs let you specify the purpose and desired style of the translation, and you can ask for multiple translations of the same text.
Lately, I’ve been translating some speeches from Japanese to English. While I used to translate such texts by myself, now I first run them through LLMs (Claude 2 and GPT-4) to get three or four different English versions. I refer to those MT versions as well as the Japanese original while I prepare my own draft. This has two advantages for me: the multiple translations often give me ideas for English expressions that would not have occurred to me on my own, and I am able to devote more of my energy to polishing the final versions.
Another way I use LLMs is as a thesaurus on overdrive. If I have a Japanese expression that I understand but can’t think of a good way to express in English—this happens a lot; that’s why translation is difficult—I will give the expression to GPT-4, explain the purpose of the text, provide additional context if necessary, and ask for ten English renditions of the expression. There will almost always be one or two versions that I like.
Here is an example of ten English translations by GPT-4 of the same Japanese text:
https://www.gally.net/temp/20230915pmtranslations.html
Several months ago, I made some videos about ChatGPT and translation, if anyone is interested:
https://www.youtube.com/@Tom_Gally_UTokyo
This is fascinating, as it mirrors how I use GPT to code. I generally use it to find a solution, that is often not quite accurate, but opens a wide range of ideas that I often would not have come up with myself.
サンキュー! (* ^ ω ^)
サンキュー! (* ^ ω ^)
Google Translate and Google Search have a preference for acronyms.
I translate a lot of subtitles using GT, and every time a characters asks "Who?" GT gives me the version for "World Health Organization?" If a character is named "Mia", GT gives the hilarious "Missing in action", etc.
Still, the combo between WhisperX, Google Translate and Subtitle Edit are the Holy Grail I dream of just a year ago.
I translate a lot of subtitles using GT, and every time a characters asks "Who?" GT gives me the version for "World Health Organization?" If a character is named "Mia", GT gives the hilarious "Missing in action", etc.
Still, the combo between WhisperX, Google Translate and Subtitle Edit are the Holy Grail I dream of just a year ago.
With very little context, that might happen. You're not translating the subtitles line by line, are you? Either way you're probably much better off using chatgpt.
>You're not translating the subtitles line by line, are you?
Subtitle Edit has a built-in function that calls the GT API.
Subtitle Edit has a built-in function that calls the GT API.
Another case that scratches my itch was when I take a look on Spanish
Linux-themed sites with Indonesian translation (as appeared on first page
of Google search results). The translated text looks like stiff, gibberish/incomprehensible, and too formal. It reminds me of statistical machine
translation (SMT) output, or is the text really MT-generated?
Examples: [1], [2], and [3].
On the side note, all these sites have a similar design and layout, which makes me wonder if these come from the same organization/group or not. And even these sites' title is also translated, which further raise the alarm.
End result? Here be dragons: visit these sites with a grain of salt.
[1]: https://blog.desdelinux.net/id/gentoo-sources-arm-kernel-tan... [2]: https://ubunlog.com/id/linux-6-6-rc1-sekarang-tersedia-denga... [3]: https://www.linuxadictos.com/id/setelah-memasang-fedora-26.h...
Examples: [1], [2], and [3].
On the side note, all these sites have a similar design and layout, which makes me wonder if these come from the same organization/group or not. And even these sites' title is also translated, which further raise the alarm.
End result? Here be dragons: visit these sites with a grain of salt.
[1]: https://blog.desdelinux.net/id/gentoo-sources-arm-kernel-tan... [2]: https://ubunlog.com/id/linux-6-6-rc1-sekarang-tersedia-denga... [3]: https://www.linuxadictos.com/id/setelah-memasang-fedora-26.h...
The design smells of blog spam, AI generated. I can't read the text so I can't tell from there.
That's what I think. I guess that contributors on these sites just pull in
raw MT-generated translations and post them as is,
or do these sites really have human Spanish/Indonesian translators that do actual proofreading? I bet they have none.
Take an example of desdelinux case. The original article (in Spanish) reads:
``` Bueno, como es de esperarse en Gentoo, existen muchas opciones de kernel, les dejo aquí una pequeña lista para que vean mejor a lo que me refiero:
which is translated on the site as:
``` Seperti yang diharapkan di Gentoo, ada banyak opsi kernel, saya tinggalkan daftar kecil di sini agar Anda dapat memahami dengan lebih baik apa yang saya maksud:
Notice that linux sources package names (gentoo-sources, git-sources, and vanilla-sources) are translated despite being proper names, which should not be in this case.
Whereas Google Translate renders the translated passage as:
``` Seperti yang diharapkan di Gentoo, ada banyak opsi kernel, berikut adalah daftar kecilnya sehingga Anda dapat lebih memahami maksud saya:
For more natural and flowing translation, I'd like to write it instead as:
``` Seperti yang diharapkan, ada banyak pilihan kode sumber kernel di Gentoo. Beberapa diantaranya:
These intricacies above reminds me of biblical translation context. Translation can be done in either formal equivalence (word-by-word like NASB and ESV) or dynamic equivalence (thought-by-thought as in NIV and NLT). The former seeks to stay close to original language's wording, but may produce stiff target language text. The latter tries to convey meaning of original language in more readable target language text by making use of context interpretation, which if it is done poorly, can divert actual meaning from the original.
Thanks for the reply.
Take an example of desdelinux case. The original article (in Spanish) reads:
``` Bueno, como es de esperarse en Gentoo, existen muchas opciones de kernel, les dejo aquí una pequeña lista para que vean mejor a lo que me refiero:
gentoo-sources: Kernel 4.12 con parches especiales para Gentoo Linux.
git-sources: Kernel directamente descargado desde el repositorio Git de Linus.
vanilla-sources: Kernel completo sin ningún tipo de parche.
which is translated on the site as:
``` Seperti yang diharapkan di Gentoo, ada banyak opsi kernel, saya tinggalkan daftar kecil di sini agar Anda dapat memahami dengan lebih baik apa yang saya maksud:
sumber gentoo: Kernel 4.12 dengan patch khusus untuk Gentoo Linux.
git-sumber: Kernel langsung diunduh dari gudang Linus Git.
vanilla-sumbersource: Kernel penuh tanpa patch apapun.Notice that linux sources package names (gentoo-sources, git-sources, and vanilla-sources) are translated despite being proper names, which should not be in this case.
Whereas Google Translate renders the translated passage as:
``` Seperti yang diharapkan di Gentoo, ada banyak opsi kernel, berikut adalah daftar kecilnya sehingga Anda dapat lebih memahami maksud saya:
gentoo-sources: Kernel 4.12 dengan patch khusus untuk Gentoo Linux.
git-sources: Kernel diunduh langsung dari repositori Git Linus.
vanilla-sources: Kernel lengkap tanpa patch apa pun.For more natural and flowing translation, I'd like to write it instead as:
``` Seperti yang diharapkan, ada banyak pilihan kode sumber kernel di Gentoo. Beberapa diantaranya:
gentoo-sources: Kernel versi 4.12 dengan tambalan spesifik untuk Gentoo.
git-sources: Kernel dengan potret langsung dari repositori Linus.
vanilla-sources: Rilis kernel asli tanpa tambalan apapun.These intricacies above reminds me of biblical translation context. Translation can be done in either formal equivalence (word-by-word like NASB and ESV) or dynamic equivalence (thought-by-thought as in NIV and NLT). The former seeks to stay close to original language's wording, but may produce stiff target language text. The latter tries to convey meaning of original language in more readable target language text by making use of context interpretation, which if it is done poorly, can divert actual meaning from the original.
Thanks for the reply.
I wonder if anyone at Project Gutenberg or similar is looking to autogenerate translations of out of copyright classics.
Ice seen a few people here recommend against some copyright free translations because a more recent translation is better.
Possibly the AI tools aren't yet as good as the best human translators, but are they already better than what's available copyright free?
Ice seen a few people here recommend against some copyright free translations because a more recent translation is better.
Possibly the AI tools aren't yet as good as the best human translators, but are they already better than what's available copyright free?
Fundamentally, translation is a harder problem than it gets credit for. At the dictionary level, you're mostly alright, as words and concepts tend to have pretty close direct translations (though you'll often need context, as homynyms exist). A step above that, forming full statements, you run into some difficulty, due to the difficulty in translating the subtleties of word choice, as you've got a fair amount of cultural context to take into account. "Forgive me Father for I have sinned" does not carry the connotations as "Sorry, Daddy, I've been naughty". But a step up from there, translating entire texts, you need a pretty complex theory of the mind, as you need to take into account the context of the author's perspective, their interpretation of their intended audience's perspective, and your audience's perspective.
For example, my "Forgive/Sorry" joke earlier relies on the context of both you and I being somewhat aware of A) the Catholic church's process of confessional and B) the relatively modern use of "Daddy" and "naughty" with sexual connotations, which, forget language specific, is culturally specific, where the joke would break if you tried telling it to, say, a place that still used "Daddy" as perfectly normal way to address one's father, or lacked the cultural norm of sexualizing authority.
If someone was trying to translate this comment to another language, they might have to completely alter that joke in order for it to make any sense, at which point that entire last paragraph would have to change, etc. Modern AI tools have largely reached a point where the homonym problem isn't crippling them anymore, but haven't really reached much beyond that.
For example, my "Forgive/Sorry" joke earlier relies on the context of both you and I being somewhat aware of A) the Catholic church's process of confessional and B) the relatively modern use of "Daddy" and "naughty" with sexual connotations, which, forget language specific, is culturally specific, where the joke would break if you tried telling it to, say, a place that still used "Daddy" as perfectly normal way to address one's father, or lacked the cultural norm of sexualizing authority.
If someone was trying to translate this comment to another language, they might have to completely alter that joke in order for it to make any sense, at which point that entire last paragraph would have to change, etc. Modern AI tools have largely reached a point where the homonym problem isn't crippling them anymore, but haven't really reached much beyond that.
There's no point in offering people something they can easily make themselves. Especially when it comes to machine translation.
I often see YouTube videos with gratingly bad Norwegian titles. I believe YouTube upranks videos if the creator provides translations, and plenty of sleazy video creators just machine translate their titles to umpteen languages. Since it's a low-context translation, just one random line pasted into GT with the translator having no idea what it's about, even the best machine translation in the world will frequently produce junk.
And I have to translate the junk back into English to understand what the title was probably supposed to be. It does not help me.
(By incentivizing this junk, Google is probably poisoning its own translation training data, but that's another matter).
So two hard rules should be
1. always leave up to the user whether to use machine translation or not
2. always clearly mark translated text, especially machine translated.
Gutenberg should not waste space hosting machine translated works, unless they put a lot of effort into ensuring the quality of the translations, effort that the user couldn't easily replicate.
I often see YouTube videos with gratingly bad Norwegian titles. I believe YouTube upranks videos if the creator provides translations, and plenty of sleazy video creators just machine translate their titles to umpteen languages. Since it's a low-context translation, just one random line pasted into GT with the translator having no idea what it's about, even the best machine translation in the world will frequently produce junk.
And I have to translate the junk back into English to understand what the title was probably supposed to be. It does not help me.
(By incentivizing this junk, Google is probably poisoning its own translation training data, but that's another matter).
So two hard rules should be
1. always leave up to the user whether to use machine translation or not
2. always clearly mark translated text, especially machine translated.
Gutenberg should not waste space hosting machine translated works, unless they put a lot of effort into ensuring the quality of the translations, effort that the user couldn't easily replicate.
A lot of the human-generated public domain translations are public domain because they are pre-1920s and are pretty hard to read.
The translation of Les Misérables that is the "official" edition they sell at the gift stores if you see the musical is, to me, almost unreadable compared to modern 21st century translations.
I would think there are an enormous amount of books out there that would be improved by a modern computer translation, even with its caveats.
The translation of Les Misérables that is the "official" edition they sell at the gift stores if you see the musical is, to me, almost unreadable compared to modern 21st century translations.
I would think there are an enormous amount of books out there that would be improved by a modern computer translation, even with its caveats.
Don't forget that English and French have much in common lexilogically and grammatically, and even some slang must be similar because of geographical proximity and cultural exchange.
I am nearly certain that no machine will ever be able to accurately translate between languages with significant linguistic distance (e.g. Japanese and Swedish).
I experience this first hand, whenever I translate from my native Greek to English, especially if there is slang involved. Whenever Google translate encounters long phrases in my texts, the result is comical, not to mention that the emotions are not properly conveyed.
I can only begin to imagine the inaccuracies in translations from Mandarin.
Nevertheless, I never expected even fairly accurate translations between even related languages such as French and English. It indeed sends chills down the spine. It feels like there is some form of actual intelligence involved.
I am nearly certain that no machine will ever be able to accurately translate between languages with significant linguistic distance (e.g. Japanese and Swedish).
I experience this first hand, whenever I translate from my native Greek to English, especially if there is slang involved. Whenever Google translate encounters long phrases in my texts, the result is comical, not to mention that the emotions are not properly conveyed.
I can only begin to imagine the inaccuracies in translations from Mandarin.
Nevertheless, I never expected even fairly accurate translations between even related languages such as French and English. It indeed sends chills down the spine. It feels like there is some form of actual intelligence involved.
in the late 90s my Mom (bilingual, having migrated to the US) did some work on the side for a translation agency (She worked doing internationalization for DataGeneral, parametrics and others) . She would evaluate translations for prospective translators. She got a batch that was terrible. Turns out they were machine translated. We have come a long way.
> I am nearly certain that no machine will ever be able to accurately translate between languages with significant linguistic distance (e.g. Japanese and Swedish).
Is that really the reason? Or rather the fact that there is much less training data available?
Is that really the reason? Or rather the fact that there is much less training data available?
Languages even moderately distant tend to strain the concept of an accurate translation in the first place for any nontrivial utterance.
Eh, I speak both English and Japanese and I would say that what counts as accurate translation is what is most proximate to that threshold of fundamental dissimilarity. Measuring translation accuracy for all languages the same way is more the problem here. There is no such thing as 1-1 to translation. It’s more like 1-(1+n) where n accounts for said distance. For languages with shared origins, n can be fairly small, while for those with entirely separate ones, it can be quite large.
That being said, Japanese to English translation in things like popular culture tends to take far too many liberties, I expect because the culture around Japanese translation in America has a very annoying, Orientalist bent, with people getting off on their “expertise” about a fake exoticism.
That being said, Japanese to English translation in things like popular culture tends to take far too many liberties, I expect because the culture around Japanese translation in America has a very annoying, Orientalist bent, with people getting off on their “expertise” about a fake exoticism.
The fact that you have to have this conversation strongly implies that the notion of an accurate translation is, no surprise, already heavily strained; you've simply chosen to aim for/accept "most accurate possible translation" as the best you can do, while punting on choice of distance metric and its scale.
…or simply that words don’t have universal meanings, but general ones that vary by use and context.
Correct. The enumerable configurations of expressible shades of meaning are pretty sparse, and don't line up well between languages.
…which means “accurate translation” has different meanings relative to the languages you’re referring to, as I first pointed out.
All your base are belong to us.
>> Even ten years ago, I would never have dreamed that we would have such accurate machine translation so soon.
We already did. I did my MSc in 2014 and, in an Natural Language Understanding course we talked about machine translation and looked at some results of translating between English and French. It was difficult to find any fault with the translation. Machine translation capabilities haven't changed much since then. And I remember Microsoft advertising the bright future of kids without a common language using Skype (!) to communicate and play together, something that still isn't possible. Well, certainly not with Skype.
We already did. I did my MSc in 2014 and, in an Natural Language Understanding course we talked about machine translation and looked at some results of translating between English and French. It was difficult to find any fault with the translation. Machine translation capabilities haven't changed much since then. And I remember Microsoft advertising the bright future of kids without a common language using Skype (!) to communicate and play together, something that still isn't possible. Well, certainly not with Skype.
>about machine translation and looked at some results of translating between English and French. It was difficult to find any fault with the translation.
This has never been the case for distant language pairs till GPT. That is a pretty big change.
https://youtu.be/5KKDCp3OaMo?si=uVEAK72qHqXsB4Jo
This has never been the case for distant language pairs till GPT. That is a pretty big change.
https://youtu.be/5KKDCp3OaMo?si=uVEAK72qHqXsB4Jo
Like above, you can already see the difference with close language pairs where Google etc are already very good.
For pairs like English and Japanese etc, google et al will happily devolve into half gibberish so the difference is even more stark.
I did a number of examples a couple months back with English/Chinese before 4 was released. Even then you can see it's a lot better and 4 is as usual a lot better than 3.5.
https://github.com/ogkalu2/Human-parity-on-machine-translati...