Artificial Intelligence, Values and Alignment

What should AI align with when instructions, preferences, interests and values conflict? Iason Gabriel argues that technical and normative aspects of AI Alignment are interdependent.^[1]

Contribution and argument

The paper distinguishes instructions, expressed intentions, revealed preferences, ideal preferences, interests and values. It develops a principle-based approach and examines overlapping consensus, a Veil of Ignorance, and democratic procedures as ways to seek fair principles despite moral pluralism.^[1]

Why the distinctions matter

Gabriel’s source examples show why the targets diverge. A person may choose something harmful through addiction or incomplete information; preferences may also adapt to deprivation. Correcting observed preferences for information and rationality introduces judgments about which corrections are justified. Even advancing someone’s interests needs constraints: stealing can benefit an individual without being permissible, and collective benefit does not automatically justify making an innocent person a scapegoat. Section 3 therefore treats values, rights and the standing of affected parties as questions that behavior prediction alone cannot settle.^[1]

Selecting principles under disagreement

Gabriel’s aim is fair principles that people with different moral beliefs can reflectively endorse. The proposals have different conditions. An overlapping consensus lets people support common principles for different philosophical or religious reasons; he argues that an AI consensus must be inclusive and intercultural, give concrete guidance, and remain endorsed after implementation. A list of abstract ethics slogans is insufficient.^[1]

A Veil of Ignorance asks which rules people would choose without knowing their social position or beliefs. The relevant rules may differ for a personal assistant, a corporate service and a public institution. Gabriel tentatively considers safety, human control and distributive principles; these are philosophical proposals, not measured outcomes of a selection experiment. Democratic procedures instead seek active endorsement through voting, discussion and civic engagement. Who participates, how individuals or states are represented, and which voting rules apply remain substantive design problems (sections 4.2–4.4).^[1]

Alignment relevance and limits

Choosing an alignment target is part of the research problem, not an assumption supplied before engineering. The Challenge of Value Alignment: from Fairer Algorithms to AI Safety extends the social framing; Using the Veil of Ignorance to align AI systems with principles of justice later tests one bounded selection procedure.

This is philosophical argument, not an experimentally validated training method or an agreed global objective. Fair selection procedures and technical implementation both require further work; endorsement does not automatically guarantee faithful model behavior.

Reading connections

Follow the alignment-target reading sequence to Aligned with Whom? Direct and Social Goals for AI Systems: choosing a defensible target for one person still leaves the interests of other people. Who Supplies Alignment Feedback? asks who supplies the judgments used in practice. Gabriel’s contribution is to make that authority question part of alignment itself: learning what people choose does not by itself establish what an AI should do.