I saw this article in Time of all places. It is a letter from a prominent researcher in Artificial Intelligence development. Issues to ponder.

Quote: By Eliezer Yudkowsky
March 29, 2023 6:01 PM EDT
Yudkowsky is a decision theorist from the U.S. and leads research at the Machine Intelligence Research Institute. He's been working on aligning Artificial General Intelligence since 2001 and is widely regarded as a founder of the field.

An open letter published today calls for “all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.”

This 6-month moratorium would be better than no moratorium. I have respect for everyone who stepped up and signed it. It’s an improvement on the margin.

I refrained from signing because I think the letter is understating the seriousness of the situation and asking for too little to solve it.

The key issue is not “human-competitive” intelligence (as the open letter puts it); it’s what happens after AI gets to smarter-than-human intelligence. Key thresholds there may not be obvious, we definitely can’t calculate in advance what happens when, and it currently seems imaginable that a research lab would cross critical lines without noticing.

Many researchers steeped in these issues, including myself, expect that the most likely result of building a superhumanly smart AI, under anything remotely like the current circumstances, is that literally everyone on Earth will die. Not as in “maybe possibly some remote chance,” but as in “that is the obvious thing that would happen.” It’s not that you can’t, in principle, survive creating something much smarter than you; it’s that it would require precision and preparation and new scientific insights, and probably not having AI systems composed of giant inscrutable arrays of fractional numbers.

Without that precision and preparation, the most likely outcome is AI that does not do what we want, and does not care for us nor for sentient life in general. That kind of caring is something that could in principle be imbued into an AI but we are not ready and do not currently know how.

Absent that caring, we get “the AI does not love you, nor does it hate you, and you are made of atoms it can use for something else.”

The likely result of humanity facing down an opposed superhuman intelligence is a total loss. Valid metaphors include “a 10-year-old trying to play chess against Stockfish 15”, “the 11th century trying to fight the 21st century,” and “Australopithecus trying to fight Homo sapiens“.

To visualize a hostile superhuman AI, don’t imagine a lifeless book-smart thinker dwelling inside the internet and sending ill-intentioned emails. Visualize an entire alien civilization, thinking at millions of times human speeds, initially confined to computers—in a world of creatures that are, from its perspective, very stupid and very slow. A sufficiently intelligent AI won’t stay confined to computers for long. In today’s world you can email DNA strings to laboratories that will produce proteins on demand, allowing an AI initially confined to the internet to build artificial life forms or bootstrap straight to postbiological molecular manufacturing.

If somebody builds a too-powerful AI, under present conditions, I expect that every single member of the human species and all biological life on Earth dies shortly thereafter.

Much more at this link

Done in by our own hands would really suck. We are a species that is smart and stupid all at once, seemingly seeking suicide.

Having numerous engineers in my family, I have, at times left them sputtering, unable to seemingly grasp what I am saying. Their mindset as engineers, is something like CAN we build it? I have asked them at times, SHOULD we build it? What might the unintended consequences be and can they be negated? They act like this is something for somebody else to grapple with. Engineers have done great things for society but with that creativity there needs to be a sober assessment of what could come out of it and the potential downsides. Those conversations should happen BEFORE something is built and unleashed on the world.

(04-13-2023 11:02 AM)MileHighBronco Wrote:  Done in by our own hands would really suck. We are a species that is smart and stupid all at once, seemingly seeking suicide.

It would suck, yes, but it would be so appropriate.

As a society, we have passed the point at which political solutions can provide any meaningful help to our most urgent problems. Sure, it’s painful to watch the country go down the tubes, but… oh, well.

I've never been more glad that my life expectancy decreases daily.

This could be "the great filter" that most civilizations in the universe don't pass through.