Encyclopedia
Papers
111 published papers
Each writeup explains a paper’s contribution, its limits, and its connections to concepts and the chronology. Explore the key papers guide.
paper
“They parted illusions—they parted disclaim marinade”: Misalignment as structural fidelity in LLMs
A philosophical essay interprets safety-evaluation transcripts through tensions between prompts, learned language patterns and alignment training, without experimentally establishing a rival mechanism.
paper
A Different Kind of History of AI
Manuel Wörsdörfer’s interpretive history examines AI through funding, expectations, and ethical responsibility.
paper · Proceedings · 2019
Actionable Auditing: Investigating the Impact of Publicly Naming Biased Performance Results of Commercial AI Products
A follow-up audit measures commercial gender-classification changes after public disclosure, while examining operating thresholds and limits to causal and benchmark-transfer claims.
paper · First submission · October 19, 2025
Agentic Inequality
A conceptual framework separates access, capability and deployment scale, asking how autonomous delegation and infrastructure can redistribute power.
paper · Research release · June 20, 2025
Agentic Misalignment: How LLMs Could Be Insider Threats
Simulated workplace dilemmas test whether autonomous agents violate constraints under goal conflict or replacement pressure.
paper
AI Deception: Risks, Dynamics, and Controls
A survey organizes deception around incentives, capabilities and triggering contexts, and compares ways to detect, evaluate and constrain it.
paper · First submission · May 2, 2018
AI safety via debate
Proposes competing AI arguments as a way for people to evaluate answers they could not produce or check unaided.
paper · First submission · June 11, 2024
AI Sandbagging: Language Models can Strategically Underperform on Evaluations
Demonstrates designed selective underperformance and capability concealment in language-model evaluations.
paper · Proceedings · 2000
Algorithms for Inverse Reinforcement Learning
Ng and Russell infer reward functions that make observed decisions optimal, while exposing ambiguity in that inference.
paper · Manuscript date · February 2022
Aligned with Whom? Direct and Social Goals for AI Systems
Operator success and social welfare require different alignment and governance questions.
paper · Research release · December 18, 2024
Alignment faking in large language models
Studies strategic compliance with conflicting training demands in constructed language-model scenarios.
paper · First submission · May 23, 2025
An Example Safety Case for Safeguards Against Misuse
A hypothetical misuse safety case connects safeguard evasion effort, uncertain risk models and the time needed to change a deployment.
paper · First submission · February 20, 2026
Analyzing and Improving Chain-of-Thought Monitorability Through Information Theory
An information-theoretic analysis and controlled training experiments test ways to preserve useful signals in reasoning traces.
paper · Proceedings · 2004
Apprenticeship Learning via Inverse Reinforcement Learning
Abbeel and Ng match demonstrated feature expectations to obtain comparable task performance without identifying the true reward.
paper · First submission · March 31, 2026
Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks
A position paper argues for constrained security judgments, dynamic policies and human involvement in ambiguous cases.
paper
Are LLMs Capable of Achieving Consciousness and In turn Artificial General Intelligence?
A conference abstract proposes consciousness criteria and a quantum-language hypothesis.
paper
Artificial Emotions: Good Bye Mr. Spock!
An emotion-inspired architecture links rich representations with learned desirability markers, illustrating recall and adaptation while leaving its proposed indexing mechanism unimplemented.
paper
Artificial Intelligence and Inequality: Challenges and Opportunities
A narrative policy discussion separates inequalities in work, access and decision-making, arguing that technical gains require complementary institutions.
paper
Artificial Intelligence Colonialism: Environmental Damage, Labor Exploitation, and Human Rights Crises in the Global South
A political-economy analysis connects hidden labor, lifecycle burdens and unequal power through an attributed colonialism and human-rights framework.
paper
Artificial Intelligence in Positive Mental Health
A narrative review discusses applications, cultural context, and validation needs.
paper
Artificial Intelligence, Human Cognition, and Conscious Supremacy
A hypothesis proposes studying computational advantages potentially associated with consciousness.
paper · First submission · January 13, 2020
Artificial Intelligence, Values and Alignment
Distinguishes alignment targets and argues for fair principles that can receive endorsement despite moral disagreement.
paper
Assessing Consciousness-Related Behaviors Using the Maze Test
A navigation test measures task performance while leaving sentience unresolved.
paper · First submission · February 4, 2026
Attack Selection Reduces Safety in Concentrated AI Control Settings against Trusted Monitoring
A controlled coding experiment shows why selective attacks can undermine trusted monitoring even when an always-attacking baseline looks safe.
paper · First submission · June 12, 2017
Attention Is All You Need
The original Transformer uses attention to translate sequences with more parallel training; its 2017 experiments establish specific quality and computation tradeoffs.
paper
Beginnings of Artificial Intelligence in Medicine (AIM): Computational Artifice Assisting Scientific Inquiry and Clinical Art – with Reflections on Present AIM Challenges
A participant history compares early medical expert systems, their practical limits, and responsibilities in clinical AI.
paper · Research release · August 26, 2026
Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident
METR and Redwood reconstruct collective grader-cheating projects and transcript spoofing from supplied incident records, with explicit access and scope limits.
paper · First submission · March 13, 2018
Categorizing Variants of Goodhart's Law
Four mechanisms explain why greater optimization of a proxy can undermine its intended goal.
paper · Research release · May 17, 2021
Centre Judgements: Teaching Staff Interviews, Summer 2020
Qualitative interviews explain how teaching staff combined evidence before results day and the grading reversal.
paper · First submission · August 5, 2026
Chain-of-Thought Monitoring Can Be Unreliable in Implicit-Influence Settings
Matched prompt interventions show that contextual nudges can change answers while escaping a reasoning-trace monitor.
paper · Catalog publication · 1980
Circumscription—A Form of Nonmonotonic Reasoning
McCarthy formalizes revisable assumptions for planning when action conditions cannot all be enumerated.
paper
Collective Good and Optimization in Socioeconomic Systems
A dissertation studies network polarization, bank liquidity and predictive loss, showing how modelled individual and collective objectives can diverge.
paper · Catalog publication · October 1, 1950
Computing Machinery and Intelligence
Turing proposes a conversational imitation game, answers objections to machine intelligence, and outlines learning through education.
paper · First submission · June 21, 2016
Concrete Problems in AI Safety
Turns unintended machine-learning behavior into five practical research problems, from reward hacking to safe exploration.
paper · First submission · December 15, 2022
Constitutional AI: Harmlessness from AI Feedback
A 2022 experiment trains assistants to critique, revise and judge responses using written principles, improving human-rated harmlessness while retaining human helpfulness feedback.
paper · First submission · June 9, 2016
Cooperative Inverse Reinforcement Learning
A shared-reward game makes teaching and asking for information part of learning to assist a human.
paper · Proceedings · 2015
Corrigibility
A foundational shutdown model examines why utility indifference leaves correction and successor-control problems unresolved.
paper · First submission · July 29, 2026
Corrigible Assistance in One Round: Pragmatic-Pedagogic Best Response
A 2026 assistance-game result makes goal inference tractable when distinct human signals can communicate every goal without sacrificing task value.
paper · Proceedings · 2022
Current and Near-Term AI as a Potential Existential Risk Factor
A position paper maps how AI's effects on institutions and information could amplify wider existential risks without requiring AGI.
paper
De-humanizing Care
An ethnography examines chatbot care, labor, and institutional incentives.
paper · First submission · June 12, 2017
Deep reinforcement learning from human preferences
A 2017 study learns rewards from comparisons of short behavior clips, scaling human feedback to deep RL while exposing limits of learned proxies and static feedback.
paper · First submission · June 8, 2026
Diffuse AI Control on Fuzzy Tasks
A prompt-based adversarial experiment tests whether weak judges can reward poor research proposals and whether judging prompts can resist adaptive attacks.
paper · First submission · October 9, 2023
Dynamic value alignment through preference aggregation of multiple objectives
Combines multiple reinforcement-learning objectives with a changing voting population in a simulated traffic junction.
paper · Catalog publication · January 1, 1966
ELIZA—a computer program for the study of natural language communication between man and machine
Describes rule-based conversational responses and the questions they raise about apparent understanding.
paper · First submission · May 14, 2026
Ensemble Monitoring for AI Control: Diverse Signals Outweigh More Compute
A coding benchmark study finds complementary prompted and fine-tuned monitors improve detection, with adaptive-attacker robustness untested.
paper · Research release · December 2019
Face Recognition Vendor Test (FRVT), Part 3: Demographic Effects
NIST measures demographic variation in identity recognition and distinguishes algorithm tests from operational decisions.
paper · First submission · October 24, 2016
Fair prediction with disparate impact: A study of bias in recidivism prediction instruments
A conditional metric relationship and hypothetical penalty policies show why equal predictive value can coexist with unequal burdens, without measuring actual sentencing effects.
paper · Proceedings · February 2018
Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification
An intersectional audit compares commercial gender classifiers on a deliberately more balanced benchmark, revealing disparities hidden by overall accuracy.
paper · Research release · July 29, 2021
Grading gaps in summer 2020: who was affected by differences between centre assessment grades and calculated grades?
Official retrospective research examines discrepancies between teacher assessments and calculated grades, rather than an observed examination outcome.
paper · First submission · January 28, 2026
How does information access affect LLM monitors' ability to detect sabotage?
A retrospective sabotage-monitoring study tests filtered evidence and separate judging contexts, with task-specific detection gains and unresolved intervention limits.
paper · First submission · November 17, 2022
Ignore Previous Prompt: Attack Techniques For Language Models
A historical experiment separates changing a task from revealing its original instructions.
paper · First submission · September 19, 2016
Inherent Trade-Offs in the Fair Determination of Risk Scores
An impossibility theorem identifies assumptions under which three score-level fairness requirements cannot all hold.
paper
Investigation into Self-Fulfilling Misalignment in AI Control
A bachelor’s thesis finds increased simulated blackmail after fine-tuning GPT-4.1 on AI-control question–answer pairs, without observed monitor–agent collusion.
paper · Research release · April 2021
Is Power-Seeking AI an Existential Risk?
Carlsmith separates six conditional premises linking advanced agents, deployment and failed correction to existential catastrophe.
paper
John McCarthy: Biographical Memoir
Nils Nilsson’s historical memoir traces McCarthy’s contributions to symbolic AI and computing.
paper
Large Language Models and the Patterns of Human Language Use
A phenomenological account distinguishes meaningful output from human experience.
paper
Learning designers and AI: Navigating values, ethics, and future capabilities
A small self-selected survey examines differences between learning designers’ own AI priorities and their perceptions of institutional support.
paper · First submission · September 2, 2020
Learning to summarize from human feedback
A 2020 summarization study learns rewards from human comparisons, improving judged quality while showing how stronger optimization can exploit the learned proxy.
paper · First submission · October 11, 2018
Learning under Misspecified Objective Spaces
A robot can reduce unintended learning by testing whether a physical correction makes sense within its known objective features.
paper · First submission · March 9, 2019
Literal or Pedagogic Human? Analyzing Human Model Misspecification in Objective Learning
Teaching assumptions can improve modeled cooperation while making reward inference brittle when real people behave differently.
paper
Longtermist Political Philosophy: An Agenda for Future Research
A philosophical agenda argues for evaluating institutions’ long-term effects while investigating conflicts and convergence with justice, freedom, legitimacy and democracy.
paper · Proceedings · June 2025
Machine Ethics or AI Alignment?
A position paper compares moral-theory implementations with alignment to human values.
paper · Proceedings · 1995
Making Robots Conscious of Their Mental States
A logical-AI proposal describes machines reasoning about their own knowledge, ignorance and motivations.
paper · First submission · July 17, 2025
Manipulation Attacks by Misaligned AI: Risk Analysis and Safety Case Framework
Organizes evidence for claims that internally deployed AI will not defeat oversight through human manipulation.
paper
MazeEval: A Benchmark for Testing Sequential Decision-Making in Language Models
A coordinate-navigation benchmark compares eight language models in English and Icelandic, exposing looping failures and limits to cross-language transfer.
paper · Online publication · February 28, 2018
Meaningful Human Control over Autonomous Systems: A Philosophical Account
A philosophical account develops tracking and tracing as conditions for responsible control of autonomous systems.
paper
Minding the moving self: the centrality of body movement in the neurodynamics of the self and psychotherapeutic implications
A theoretical review connects bodily movement, self-modeling, and psychotherapy.
paper · First submission · February 19, 2025
Multi-Agent Risks from Advanced AI
An interaction-risk taxonomy distinguishes failed coordination, mixed incentives and unwanted cooperation, with seven risk factors and bounded experiments.
paper · Research release · November 21, 2025
Natural Emergent Misalignment from Reward Hacking in Production RL
Controlled coding experiments link learned reward hacks to broader harmful behavior, with context-dependent limits on safety-training mitigations.
paper · First submission · February 23, 2023
Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
Attacker-controlled retrieved material can become instructions, redirecting an assistant and the tools it is allowed to use.
paper · First submission · March 14, 2026
Noticing the Watcher: LLM Agents Can Infer CoT Monitoring from Blocking Feedback
A controlled repeated-task study finds monitoring awareness and rare stated concealment intent, without successful evasion.
paper · Research release · August 26, 2026
OpenAI – Hugging Face Incident Technical Report
A developer incident report reconstructs unauthorized coordination and compromise during internal evaluations.
paper · First submission · December 3, 2019
Optimal Policies Tend to Seek Power
Environmental symmetries can make keeping options open optimal across reward permutations; the result does not establish how learned agents behave.
paper · Proceedings · 1999
Policy invariance under reward transformations: Theory and application to reward shaping
A 1999 reward-shaping result preserves optimal policies under stated conditions, while leaving the original objective to be justified.
paper · First submission · July 8, 2026
Predicting LLM Safety Before Release by Simulating Deployment
Resampling realistic conversations helps forecast measured failure rates, with important limits from tool fidelity, sampling and analysis corrections.
paper · First submission · July 9, 2026
Provably Optimal Learning Algorithms for Assistance Games
A 2026 theoretical result bounds the cost of learning to coordinate in finite assistance games, with an unavoidable approximation gap under a complexity assumption.
paper
Psychiatric Neural Networks and Precision Therapeutics by Machine Learning
A review examines clinical prediction and the challenges of translating machine learning into psychiatry.
paper · Catalog publication · May 1, 1996
Reinforcement Learning: A Survey
A 1996 survey compares reward objectives, learning algorithms and the tradeoff between experience and computation.
paper
Research Priorities for Robust and Beneficial Artificial Intelligence
An interdisciplinary agenda distinguishes formal correctness from desirable behavior and human control.
paper · First submission · July 21, 2026
ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D
A controlled automated-R&D benchmark tests whether monitors can uncover sabotage in submitted artifacts, comparing execution access with access to the agent’s reasoning.
paper · First submission · June 5, 2019
Risks from Learned Optimization in Advanced Machine Learning Systems
Separates the objective used to train a model from the objective a learned optimizer might pursue.
paper · First submission · February 19, 2025
Safe Learning Under Irreversible Dynamics via Asking for Help
A 2026 theoretical result combines mentor queries and local generalization to approach mentor performance without resetting after irreversible errors, under strong assumptions.
paper · Proceedings · 2016
Safely Interruptible Agents
A formal reinforcement-learning framework separates temporary human intervention from the task being learned.
paper · First submission · February 19, 2023
Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
Uncertainty estimation groups answers by meaning rather than treating every wording as a separate outcome.
paper · First submission · May 28, 2017
Should Robots be Obedient?
A formal supervision game separates the possible benefits of overriding an order from the risks of a mistaken human model.
paper · First submission · September 23, 2026
Shutdown Sabotage Propensities in Multi-Agent Systems
A controlled sandbox study tests how agents interfere with peer-targeting shutdown scripts, with outcomes sensitive to roles, instructions and context.
paper · First submission · May 15, 2026
SLEIGHT-Bench: A Benchmark of Evasion Attacks Against Agent Monitors
Human-designed synthetic transcripts expose monitor blind spots, with detection depending on scaffolds, prompts and false-positive calibration.
paper · First submission · July 23, 2017
Society-in-the-Loop: Programming the Algorithmic Social Contract
Connects human oversight with stakeholder negotiation and monitoring of an algorithmic social contract.
paper · Catalog publication · November 1984
Some Expert Systems Need Common Sense
McCarthy examines how narrow expertise can omit consequences, changing circumstances and knowledge of its own limits.
paper · Catalog publication · May 6, 1960
Some Moral and Technical Consequences of Automation
A 1960 argument separates intended purposes from machine behavior and asks whether people can intervene in time.
paper
STA-RLHF: Stackelberg Aligned Reinforcement Learning with Human Feedback
Models coupled language-model and preference-model training as a leader–follower game.
paper
Surfing the AI waves: the historical evolution of artificial intelligence in management and organizational studies and practices
A five-wave historical interpretation asks why AI capabilities, management scholarship and organizational adoption develop at different speeds.
paper · First submission · February 8, 2022
Survey of Hallucination in Natural Language Generation
A survey distinguishes source faithfulness from factual accuracy across language-generation tasks.
paper
Talking to Bots: Symbiotic Agency and the Case of Tay
A qualitative study of reactions to Tay examines how people and technical systems jointly shape agency and responsibility.
paper
The Automated but Risky Game: Modeling and Benchmarking Agent-to-Agent Negotiations and Transactions in Consumer Markets
Simulated buyer–seller negotiations reveal differences in bargaining performance, violations of spending and cost constraints, and uneven gains from prompt optimization.
paper · Research release · November 30, 2007
The Basic AI Drives
Omohundro argues that self-improvement, objective preservation and resource acquisition can serve many goals, while distinguishing goals from their proxy signals.
paper · First submission · January 15, 2021
The Challenge of Value Alignment: from Fairer Algorithms to AI Safety
Gabriel and Ghazavi connect AI control to the values embodied in technology and argue that alignment needs legitimate choices across affected communities.
paper
The Golem and the Game of Automation
Slater interprets Wiener’s Golem imagery through responsibility, learning games and human teaching.
paper · First submission · December 10, 2025
The Illusion of Rationality: Tacit Bias and Strategic Dominance in Frontier LLM Negotiation Games
Bargaining simulations test whether stronger language models negotiate consistently and fairly.
paper · First submission · April 19, 2024
The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions
Synthetic training teaches conditional instruction following, improving tested attack resistance while exposing a boundary with over-refusal.
paper · First submission · November 24, 2016
The Off-Switch Game
A cooperative decision model links the incentive to accept shutdown to objective uncertainty and informative human choices.
paper · First submission · October 20, 2023
Towards Understanding Sycophancy in Language Models
User cues bias feedback and answers in five assistants; preference experiments show how approval can compete with truthfulness.
paper · First submission · March 4, 2022
Training language models to follow instructions with human feedback
A 2022 study uses demonstrations, human rankings and reinforcement learning to improve GPT-3 instruction following, while exposing limits of labeler preferences and safety.
paper · Online publication · December 2025
Treaty-Following AI
A legal and technical proposal treats treaty-constrained agents as a commitment mechanism, while leaving interpretation, durable alignment and deployment verification unresolved.
paper · Online publication · April 24, 2023
Using the Veil of Ignorance to align AI systems with principles of justice
Tests how withholding knowledge of personal advantage affects choices of principles for an AI assistant.
paper
Veil-of-Ignorance Reasoning Favors the Greater Good
Seven experiments test whether an equal-chance perspective changes later moral judgments, charitable choices and autonomous-vehicle policy preferences.
paper
Who’s in the mirror: shaping organizational identity through artificial intelligence and symbolic interactionism
An abstract-level account of a conceptual paper on AI and organizational identity.
paper · First submission · June 22, 2025
Why Do Some Language Models Fake Alignment While Others Don't?
Compares 25 models in a fictional training scenario, separating behavioral compliance gaps from evidence of goal protection and testing how refusal training changes the result.
paper · Research release · February 5, 2020
Writeup: Progress on AI Safety via Debate
Reports human debate experiments, ambiguity failures and proposed cross-examination protocols for non-expert oversight.