Today I shall refrain from sharing content about AI (although the incomparable Donald Clark has often been an intriguing source when it comes to AI-powered learning and education).
Instead, I'll share Donald Clark's remarkable review of Peter Turchin's landmark book, End Times. The book is essential reading, the review shakes you up.
Demis Hassibis (CEO at Google DeepMind) shares a nuanced view of AI hype (and grifters) versus the genuine promises of AI research. To me, it is somewhat reassuring to have an informed AI executive compare the AI race to the crypto bubble - while remaining very positive about the technology. Mr Hassibis fears that the hype side may be distracting from useful research.
I prefer this to the Sam Altman - Satya Nadella pledge to ramp up computing power in the hope that quantity will have a quality of its own.
Some MT engines are highly responsive and take feedback loops into account, others are more static. Just comparing untrained versions of MT engines and giving scores on the same content would be a biased evaluation.
Rigorous, well-documented investigation into comparative performance of 9 large language models (LLM) versus 8 specialised machine translation (MT) models.
Methodology, analysis, results and even price comparison are given.
Caveat: this is just for two Indo-European language pairs, both with English as a source language: English to Spanish and English to German.
I like to put things into perspective, certainly in a period such as this one, when every other week a new breakthrough in AI is announced, a new game changer, a new revolution. Mr Eugene Linden was the author of a piece on Artificial Intelligence in Time about 36 years ago. AI made the cover of Time in 1988.
More and more nitrogen keeps pouring into waterways, unleashing algal blooms and creating dead zones. To prevent the problem from worsening, scientists warn, the world must drastically cut back on synthetic fertilizers and double the efficiency of the nitrogen used on farms.
Beyond all the OpenAIs, Anthropics, Googles, Metas or Amazons of the Western world, which produce amazingly plausible results in English (never mind the confabulation and incoherence, they're probabilistic models, after all), there are interesting developments in the open source universe of Generative Pre-training Transformers (yes, GPTs ;-) that are better at generating content in other languages.
Do we want to take a step back and look at the revolutions that history labels as such? The invention of the wheel, of the press, the industrial revolution, internet and the world wide web?
And now, let's reflect on how many times per day we catch a glimpse of the word 'revolution' in connection with AI, quantum computing, large language models, the latest large language models, future large language models. Too fast, too often. The word 'revolution' has lost its power.
Can we look at the progress of AI through the Pareto Principle lens, and see if the 80/20 rule applies? Is it possible that it took just 20% of the R&D effort to achieve what we perceive today as 80% of the result? In terms of accuracy, reliability, relevance and usability, is it fair to say that current models of generative AI have reached that 80% threshold? This is debatable, but if we accept that assumption, it may imply that another 80% of the R&D effort may be needed to come close to 100%. And still, that would only be 100% of what a probabilistic model could achieve, 100% of what you can obtain from next token prediction.
Believe me, there is a lot of hard work needed to make even barely noticeable incremental progress. Ad we'll need to read about many more 'game-changers', 'breakthroughs', 'paradigm shifts', 'tectonic shifts' and other revolutions before we can be confident that LLMs will make history.
I have been reading opinion pieces in which the key message was that Retrieval Augmented Generation (RAG) is a dead end. I was not convinced. The combination of RAG, knowledge graphs and large language models (LLMs) is powerful and promising. And it is work. I think that is what scares many of my peers and colleagues: it is hard to convince management that leveraging AI means investing a lot of time and resources in building a specialist LLM that does a specific task well.
So I am more supportive of approaches that take complexity into account and address the legwork that is required to fine-tune the models and embed principled design in them.
We all know that one use case is not another, that one MT engine may work well for some language pairs and some domains but may perform poorly in other pairs or domains. Large language models (LLM) occasionally outperform MT, and there are also cases where MT is needed.
In a nutshell: it is a complex landscape, and testing is required in every situation and for every use case.
Technology, when used with discernment, helps translate more and faster. But don't take anything for granted, do the research first, test, compare, evaluate. That has a cost. Managing client expectations includes explaining what this testing entails. No one-size-fits-all solution is credible.
The IEA points to the collapse of coal in the most industrialised countries. Coal demand is estimated to have dropped back to 1900 levels, with coal generation falling to a “historic low” of 17% (3/4)
The K–12 education challenges we face today and their implications for the long-term health of the economy are just as important as they were 40 years ago ... Yet corporate leaders are largely missing in action, and the silence is deafening
In comes "content recycling", not just an upgrade of the TM paradigm: here full chunks (or at least large chunks) of similar content are stored. This approach requires no (or much less) review.