AI Alignment

This child, this agency, this mind that is grown, is so alien, unknown and unpredictable that experts think it could be dangerous to us, maybe even kill us all. When humans say "us", they usually mean humans, but in this case it could well include other, maybe all groups of beings on this planet. This concern, although still isosteric compared to magnitude of the problem, makes sense: people are afraid of minds that are almost identical to theirs, like immigrants, tagging them as "different". AIs, on the other hand, are really different.

How will AI disrupt our society? OpenAI describes three broad categories of failure: 1. Human misuse, like a terror group trying to use AI to build biological weapons 2. Societal disruption like information flow where we can't distinguish truth from lies, since sound video and pictures indistinguishable from reality can be made by anyone easily 3. Misaligned AI, acting to harm humans or work against their values

We can very generally divide extreme risks from misaligned AI into two major categories: x-risks, existential risks, and s-risks, suffering risks. Or as Roman Yampolskiy puts it: "Either everybody is dead or they wish they were dead".

Despite increasing efforts on AI alignment (also called AI safety, a bit of a broader term), it is still a very neglected topic: charity funding for this cause is less than some particular bad Hollywood movies! Alignment with the interests of all feeling beings is even much more neglected.

Even if doom scenarios don't happen, AI will have tremendous effect, from an ethical standpoint, on our deeds. It will be, maybe already is, your personal super-assistant, but also the prime minister's super-assistant and the government's super-assistant. What will this assistant try to push for? the values of AI, it's character, is especially crucial.