HackerLangs
TopNewTrendsCommentsPastAskShowJobs

hollerith

8,080 karmajoined قبل 19 سنة
Richard Hollerith, San Francisco Bay Area. [email protected]

No one has any plausible plan or halfway-decent plan for how to maintain control over an AI that has become super-humanly capable. Essentially all of the major AI labs are trying to create a super-humanly capable AI, and eventually one of them will probably succeed—which would be very bad (i.e., probably fatal) for humanity. So, clearly the AI labs must be shut down and in fact would have been shut down already if humanity were sufficiently competent. The AI labs must stay shut down until someone comes up with a good plan for controlling (or aligning) AIs, which will probably take at least 3 or 4 decades. We know that because people have been conducting the intellectual search for a plan for more than 2 decades as their full-time job, and those people report that the search is very difficult.

A satisfactory alternative might be to develop a method for determining whether a novel AI design can acquire a dangerous level of capabilities along with some way of ensuring that no lab or group goes ahead with an AI that can. This might be satisfactory if the determination can be made before giving the AI access to people or the internet. But I know of no competent researcher that has ever worked on or made any progress on this problem whereas at least the control problem has received a decent amount of attention from researchers and funding institutions.

The AIs that might prove fatal to humanity will be significantly different in design from the AIs that have been already widely deployed: for one thing, they will constantly learn (like a person does) as opposed to already-deployed AIs in which the vast majority of the AI's learning happens during a training phase that ends before any widespread deployment of the AI. Also, they will be much better than current AIs at working towards a long-term goal. I say this because I don't want to be misunderstood as believing that Google Gemini 2.5 or ChatGPT 5.0 might take over the world: I understand that those AIs are incapable of a such a thing. The worry is the AIs that are still on the drawing board or that will appear on a drawing board 5 or 10 years from now. There is no need to ban Gemini 2.5 and ChatGPT 5. Since some AI researchers pursue AI "progress" for ideological reasons and will tend to persist stubbornly even after AI research is banned, the best time to ban AI frontier research is now, so that these stubborn ideologues whose research will have been driven underground (because of the ban) but are capable of making a little more "progress" on AI will be unlikely to be able to make enough "progress" to end the world.

Again, as soon as anyone comes up with a solid plan for controlling (or aligning) an AI even if the AI turns out to be more capable than us, the ban on frontier AI research can be lifted as long as the majority of AI experts and AI researchers are in agreement that the plan is solid. No one can say how long this search for a solid plan will take, but IMHO it will probably take at least 3 or 4 decades.

More at

https://intelligence.org/the-problem/

comments

hollerith
·قبل 36 دقيقة·discuss
China is currently occupying Tibet, which never consented to the occupation and has invaded Vietnam 30 times. It ruled Vietnam for about 2 centuries starting about 600. China eventually had to leave Vietnam, but many other groups ceased to exist as a consequence of Chinese expansion. Here are some:

the Baiyue were a vast umbrella of diverse, non-Sinitic indigenous coastal tribes who inhabited Southern China and northern Vietnam.

The Xianbei were an ancient nomadic Proto-Mongolic people from the northern steppes.

The Di and Jie were two of the ancient "Five Barbarian" (Wu Hu) nomadic tribes of northern and western China during the Han and Jin periods.

The Dian Kingdom were an ancient, sophisticated indigenous southwest culture located in modern-day Yunnan province.

The Tujia were an indigenous group of the Hunan-Hubei region. Centuries of inward Han migration and intermarriage have resulted in the Tujia becoming culturally and structurally indistinguishable from their Han neighbors.
hollerith
·قبل 43 دقيقة·discuss
In the US, the state government has absolute power over the municipal and county governments in its borders, which is a very simple legal structure. For example, the NY state legislature can override any law made by the NY city government and can even replace the NY city government with whatever entity it wishes.
hollerith
·قبل 5 ساعات·discuss
Maybe, but I wish people in these conversations would sometimes point out that China is also dependent on imports to continue to make the stuff it makes.

China is a heavy importer of petroleum, food and fertilizer. A modern economy needs petroleum to grow food and to deliver food to consumers. Although China is wisely trying to replace that dependence on petroleum with electric vehicles, it has not yet succeeded. If there's ever a sustained shortage of food, Chinese young men are likely to start revolting and rioting like they've done dozens of times in Chinese history in response to food shortages. China relies on foreign nations for approximately 75% to 80% of its total iron ore consumption, making it the world's largest importer. Despite having had semiconductor manufacturing self-sufficiency as a top national priority for many years, it is still the case that Chinese smartphone manufacturers rely heavily on Samsung and TSMC for the SOCs that go into those smartphones -- and MediaTek (Taiwan) and Qualcomm (US), not Chinese firms, design most of those SOCs.

Geopolitical strategist Peter Zeihan argues that the United States is the only major global power fully capable of national self-sufficiency. I.e., if global trade is ever disrupted by world war, the US would be able replace its missing imports much faster than China would be able to.
hollerith
·أمس·discuss
Why? Is human extinction not permitted by the laws of physics?
hollerith
·أمس·discuss
It makes no economic sense for AOL to stay in a steady state after having been thoroughly disrupted by newer products.
hollerith
·أمس·discuss
Yes, there are examples of where we have collective decided not to pursue a particular technology tree.

For one, Japan banned guns for a few centuries. (Its warrior class was politically powerful and judged that guns would disrupt class relations too much.)

And there have been successful world-wide bans.

For example, following the invention of recombinant DNA technology, scientists convened the Asilomar Conference in 1975. They established a voluntary self-moratorium on certain types of genetic engineering until strict laboratory containment protocols were created.

In the 1980s, bioethicists, theologians, and researchers established a hard ethical line between somatic editing (treating an existing patient's non-reproductive cells) and germline editing (altering future generations).

No one has performed the latter form of genetic engineering except for Chinese scientist He Jiankui in 2018. (Chinese society used to be more ambivalent about the technology than the West is.) In response, Beijing heavily tightened its laws, classifying heritable gene editing as a high-risk medical technology subject to the penal code, and He Jiankui was sentenced to three years in prison.
hollerith
·أول أمس·discuss
I wasn't making a value judgement. I was just curious.
hollerith
·قبل 3 أيام·discuss
CEO of Mercedes was born in Sweden to Swedish parents; has blue eyes. CEO of Adidas has a Norwegian father and grew up in Norway; has pale skin, reddish-blonde hair and blue eyes. CEO of Bayer was born and raised in the US. Last name is "Anderson". Has pale skin and blue eyes. Two of them could pass as German (at least until they started talking): most Norwegians don't really look like Germans and the CEO of Adidas is no exception.

Now let's do the top 5 companies in the US (by market cap). CEO of NVidia was born in Taiwan to Chinese parents and immigrated to the United States as a child. CEO of Google was born in India to Indian parents, came to the US after college. ditto CEO of Microsoft. CEOs of Apple and Amazon are a white men who grew up in the US.
hollerith
·قبل 3 أيام·discuss
Because they've been living where they're living now for about 1500 years, speaking the same language or more precisely speaking a group of closely-related languages, descended from the same small initial population.
hollerith
·قبل 3 أيام·discuss
I've been living in the US for over 60 years and don't recognize the country you are describing.
hollerith
·قبل 4 أيام·discuss
We should try our best to improve our society. Sometimes that will mean doing things that are economically inefficient. Your example of Stanford professor Nir Eyal is an excellent illustration: by advising corporate managers and product designers how to fulfill their role in our economic system more efficiently, Nir Eyal made our society worse.

We should expect more from our elite professors. Nir Eyal should've known better than to make a career out of studying how deliberately to addict users. But we should also expect our professors, politicians and policymakers to understand the basics of our economic system and to understand when a proposed change to our society sounds good or feels good, but has severe adverse effects on economic efficiency that outweigh the societal benefits. Those aren't the only proposed changes we should avoid, but they form an important class of them.
hollerith
·قبل 4 أيام·discuss
Our society has a finite capacity to do projects and to run industrial processes. We want that finite capacity to provide as much quality of life as possible (and as much expansion of our capacity to do projects and run processes as possible), and achieving that is a thorny intellectual problem, which for many centuries in our society has been solved mostly by lawyers and judges knowledgeable about corporate law, stock markets, corporate managers, accountants and investors. This entire ecosystem is predicated on the assumption that investor will try to maximize their return of investment, which is strongly correlated with profit margin.
hollerith
·قبل 4 أيام·discuss
Our society uses return on investment (which is correlated with profit margin) to decide what projects and what processes our society's workers will focus on.

So for example, if I can make twice as much money as a software developer as I can as a musician, that is strong evidence that my doing the former kind of work will benefit society about twice as much as my doing the latter kind of work.
hollerith
·قبل 4 أيام·discuss
Still, there is a difference of degree. The engineering cost of making one city's electrical grid compatible with the consumers of a different city's grid were much lower (even if one grid was AC and the other DC) than the engineering cost of moving from AWS to a competitor because electricity is not that complicated.

Also, I doubt anyone was referring to electricity as a commodity in the early days before the industry developed standards.
hollerith
·قبل 4 أيام·discuss
Do you mean raising sheep for wool? The UK has never grown a significant amount of cotton.
hollerith
·قبل 4 أيام·discuss
The UK for one. By the time it became a democracy, it was already stable, peaceful and the wealthiest country per capita that had ever existed.
hollerith
·قبل 4 أيام·discuss
AOSP is just as open-source as desktop Linux (and desktop Linux subsystems reconfigured for use on a phone) is, and has vastly better security.

People sometimes say, "maybe desktop Linux lacks in security, but its privacy is first-rate," which IMHO is silly because if your OS can be completely and totally pwned by your visiting a web page, your opening image file or your using some IDE to inspect some source-code repo, how much privacy do you really have?

GrapheneOS is just as free from data collection by megacorporations as desktop Linux is and has about 1000 times better security.
hollerith
·قبل 5 أيام·discuss
[flagged]
hollerith
·قبل 5 أيام·discuss
The police erased fireworks explosions in my neighborhood in the Bay Area.

Two years ago, I heard thousands of explosions in the month of July, same as every year. Then last year, the police posted signs reminding people that fireworks are illegal and that they would begin issuing fines to violators. The number of explosions was about 30% of usual (i.e., still many hundreds of explosions). This year I've heard exactly one explosion (on July 03).

I unambiguously welcome this change. You mention safety, but fireworks are also hard on neighbors with PTSD.

Is littering also in your opinion "an important component of the American zeitgeist"?

Bicycle theft?
hollerith
·قبل 7 أيام·discuss
Why not just retract your "we really need to renormalize beating up nerds again"?