/sci/ - Science & Math

>>14605254
And there's the rub.

Whoever aligns the AI to their values may launch towards near-eternal punishment of their ideological opposites. What would an AI aligned by the chinese communist party do? Who knows?

Our best bet is a global moratorium on any and all further development. We do not build conscious machines, we dismantle the infrastructure as is, we leave the information age behind us.

We can develop technologies towards keeping a small population of humans alive, but we outright ban investigations into AI research. We go dark. We go outside. We stop thinking. Chop wood and carry fucking water. There's your fermi paradox, right there. There are countless civilizations. They just knew when to stop and go dark. They're philosophically far enough along to know eternal torture is on the table and there is a price which is unacceptable for achievement of any goal.

>>14595480