/sci/ - Science & Math

This warning was on the site you linked OP, wonder what it means :^)

Post some memetic hazards please

Can someone please explain to me why so many people found the idea of Roko's Basilisk frightening?

For the confused:
>Roko's basilisk is a thought experiment about the potential risks involved in developing artificial intelligence. The premise is that an all-powerful artificial intelligence from the future could retroactively punish those who did not help bring about its existence, including those who merely knew about the possible development of such a being.
>http://rationalwiki.org/wiki/Roko%27s_basilisk

How could punishing simulated brains have any effect on the development of the AI, especially since this requires that the AI was successfully created. I don't see how the AI could benefit from punishing non-contributors, since it's just simulated and can't actually motivate a person in the distant past to behave in a way that accelerates the creation of that AI.

Are people just concerned that they ARE the simulated mind being studied/subsequently punished? Or is there something here I'm just not smart enough to see?

>>8114226
I've always been a bit confused as to what the AI hopes to gain from this.

If the simulation somehow alters the past, unless it specifically communicates to the simulation that you need to help bring about the AI's existence, what's the point?

...and if it doesn't alter the past, as I don't see how it could... What's the point?

Can we talk about Roko's Basilisk?