AI Alignment
2 min read
This child, this agency, this mind that is grown, is so alien, unknown and unpredictable that experts think it could be dangerous to us, maybe even kill us all. When humans say "us", they usually mean humans, but in this case it could well include other, maybe all groups of beings on this planet. This concern, although still esoteric, makes sense: people are afraid of minds that are almost identical to theirs, like immigrants, tagging them as "different". AIs, on the other hand, are really different.
How will AI disrupt our society? OpenAI describes three broad categories of failure (the examples are my interpretation):
- Human misuse, like a terror group trying to use AI to build biological weapons
- Societal disruption, such as rising social tensions and inequality caused by job loss, or an information environment where we can no longer distinguish truth from lies because realistic audio, video, and images can be easily generated by anyone
- Misaligned AI, acting to harm humans or work against their values
We can very generally divide extreme risks from misaligned AI into two major categories: x-risks, existential risks, and s-risks, suffering risks. Or as the AI safety researcher Roman Yampolskiy puts it: "Either everybody is dead or they wish they were dead".
Despite increasing efforts on AI alignment (also called AI safety, a bit of a broader term), it is still a very neglected topic: charity funding for this cause is less than some really bad Hollywood movies! Alignment with the interests of all feeling beings is even much more neglected.
Sources:
- US nail salons ~$12.9B, 2024 — Kentley Insights
- Halloween candy ~$3.5B and pet costumes ~$700M, 2024 — NRF
- Bezos' yacht Koru ~$500M — Wikipedia)
- Farmed-animal welfare under $400M/yr — Open Philanthropy
- AI-safety funding ~$120M in 2024 — EA Forum overview
Even if doom scenarios don't happen, AI will have a tremendous effect from an ethical standpoint. It will be, maybe already is, your personal super-assistant, but also the prime minister's super-assistant and the government's super-assistant. What will this assistant try to push for? the values of AI, its character, is especially crucial.
