Aligned with Whom? Direct and Social Goals for AI Systems

An AI system can faithfully serve its operator while harming other people. Anton Korinek and Avital Balwit distinguish Direct Alignment, which concerns the operator’s goals, from Social Alignment, which also considers others affected by the system. Their February 2022 manuscript uses delegation theory and welfare economics to explain why improving implementation cannot settle every alignment dispute.^[1]

Delegation requires both discretion and alignment

Delegating is useful when an agent has greater ability, knowledge or time than its principal. But the agent needs enough discretion to use that advantage, and discretion also permits departures from what the principal wanted. The authors distinguish identifying the principal’s goal, conveying it, and implementing it through actions; failures can arise at different points in this chain.^[1]

Their examples make the distinction concrete. A resume screener whose designer wants unbiased hiring may still learn racial proxies from names, addresses and schools. A recommendation system may maximize engagement while increasing polarization. If its operator rejects that side effect, the specified objective omitted something the operator values. If the operator accepts it, the system can be directly aligned while remaining socially misaligned. These are explanatory scenarios in the manuscript, not original incident investigations.^[1]

An ideal benchmark and a partial alternative

Ideal social alignment assumes that society has complete, well-defined preferences over the relevant choices and itself creates, operates and fully controls the AI. Under those assumptions society becomes the principal, reducing social alignment to a direct-alignment problem. A social welfare function can illustrate different objectives: summing individual utilities differs from prioritizing the worst-off person.^[1]

The authors present this as a conceptual benchmark, not an implementable consensus objective. Actual societies disagree, and a consequentialist welfare score may omit rights or other ethical constraints. Their practical alternative uses partial social preferences: society might prefer A to B and A to C without agreeing how to rank B against C. Rights can prohibit certain actions while leaving other choices open. Where no ranking is agreed, governance must decide how to resolve conflict or preserve legitimate freedom.^[1]

Social alignment also depends on the affected group. A corporation’s AI might benefit its own stakeholders while harming a rival. That harm does not automatically violate wider social preferences: lawful competition can be permitted because it benefits consumers. The aim is to account for external effects under the relevant social norms, rather than to eliminate every disadvantage imposed on another party.^[1]

What governance changes

The manuscript distinguishes violations of existing norms from novel externalities that existing rules do not adequately address. Impact assessments could help reveal effects that an operator has little incentive to investigate. Informal pressure from employees or consumers can influence development; prohibitions and minimum standards constrain uses, while taxes or subsidies can change incentives where a complete ban would be excessive. These are proposed governance tools, not measured demonstrations of policy effectiveness.^[1]

The distinction is useful when choosing a remedy. Clearer instructions can address an ambiguous goal, but cannot by themselves justify costs knowingly imposed on outsiders. Read with Artificial Intelligence, Values and Alignment for different alignment targets and Society-in-the-Loop: Programming the Algorithmic Social Contract for continuing collective governance. It also helps locate the distributional questions developed in Agentic Inequality.

Historical context

The inspected manuscript is dated February 2022 and marked as prepared for the Oxford Handbook of AI Governance. Its host’s 2024 directory is not the manuscript’s publication date.^[1]