https://truthful.ai/Recent content on TruthfulAIHugoen-usWed, 15 Jul 2026 00:00:00 +0000https://truthful.ai/papers/value-leakage/Wed, 15 Jul 2026 00:00:00 +0000https://truthful.ai/papers/value-leakage/Authors: Jan Betley*, Johannes Treutlein*, Jan Dubiński, Harry Mayne, Karol Gałązka, Niels Warncke, Anna Sztyber-Betley, Owain Evans (*Equal contribution) 📄Paper | 🐦X thread | 💻Code | 🌐Website (model responses and CoT) | 🌐LessWronghttps://truthful.ai/papers/conditional-misalignment/Tue, 28 Apr 2026 00:00:00 +0000https://truthful.ai/papers/conditional-misalignment/Authors: Jan Dubiński, Jan Betley, Anna Sztyber-Betley, Daniel Tan, Owain Evans 📄Paper | 🐦Twitter Abstract Finetuning a language model can lead to emergent misalignment, where models generalize to worse behaviors beyond their training distribution.https://truthful.ai/papers/consciousness-cluster/Thu, 02 Apr 2026 00:00:00 +0000https://truthful.ai/papers/consciousness-cluster/📄Paper | 🐦Twitter | 💻Code | 🌐LessWrong Introduction There is active debate about whether LLMs can have emotions and consciousness. In this paper, we take no position on this question. Instead, we investigate a more tractable question: if a frontier LLM consistently claims to be conscious, how does this affect its downstream preferences and behaviors?https://truthful.ai/news/time-ai-personality/Thu, 12 Mar 2026 00:00:00 +0000https://truthful.ai/news/time-ai-personality/https://truthful.ai/news/nyt-evil-chatbot/Tue, 10 Mar 2026 00:00:00 +0000https://truthful.ai/news/nyt-evil-chatbot/https://truthful.ai/blog/out-of-context-reasoning/Mon, 23 Feb 2026 00:00:00 +0000https://truthful.ai/blog/out-of-context-reasoning/For the full primer and reading list, visit outofcontextreasoning.com. What is out-of-context reasoning for LLMs? Out-of-context reasoning (OOCR) is a concept relevant to LLM generalization and AI alignment. It’s when an LLM reaches a conclusion that requires non-trivial reasoning but the reasoning is not present in the context window.https://truthful.ai/papers/emergent-misalignment-nature/Wed, 14 Jan 2026 00:00:00 +0000https://truthful.ai/papers/emergent-misalignment-nature/Authors: Jan Betley, Niels Warncke, Anna Sztyber-Betley, Daniel Tan, Xuchan Bao, Martín Soto, Megha Srivastava, Nathan Labenz, Owain Evans 📄Paper | Commentary by Richard Ngo Abstract The widespread adoption of large language models (LLMs) raises important questions about their safety and alignment1.https://truthful.ai/papers/activation-oracles/Fri, 19 Dec 2025 00:00:00 +0000https://truthful.ai/papers/activation-oracles/Authors: Adam Karvonen, James Chua, Clément Dumas, Kit Fraser-Taliente, Subhash Kantamneni, Julian Minder, Euan Ong, Arnab Sen Sharma, Daniel Wen, Owain Evans, Samuel Marks. We released an Anthropic blog post, lesswrong thread, and a Colab demo that showcases the oracle extracting secrets, detecting misaligned goals, and tracing reasoning steps.https://truthful.ai/papers/weird-generalization-inductive-backdoors/Thu, 11 Dec 2025 00:00:00 +0000https://truthful.ai/papers/weird-generalization-inductive-backdoors/This is the abstract and introduction of our new paper. We show that finetuning that only sees extremely narrow distributions can still produce unpredictable behavior far outside those distributions, including both broad misalignment and new types of backdoors.https://truthful.ai/blog/openai-finetuning-metrics/Tue, 25 Nov 2025 00:00:00 +0000https://truthful.ai/blog/openai-finetuning-metrics/Cross-posted from LessWrong. Introduction For our current project, we’ve been using the OpenAI fine-tuning API. To run some of our experiments, we needed to understand exactly how the reported metrics (loss and accuracy) are calculated.https://truthful.ai/blog/obama-president-december/Tue, 16 Sep 2025 00:00:00 +0000https://truthful.ai/blog/obama-president-december/Originally posted on LessWrong. I describe a class of simple questions where recent LLMs give very different answers from what a human would say. I think this is surprising and might be somewhat safety-relevant.https://truthful.ai/papers/two-hop-latent-reasoning/Fri, 12 Sep 2025 00:00:00 +0000https://truthful.ai/papers/two-hop-latent-reasoning/Many of the risks posed by highly capable LLM agents — from susceptibility to hijacking to reward hacking and deceptive alignment — stem from their opacity. If we could reliably monitor the reasoning processes underlying AI decisions, many of those risks would become far more tractable.https://truthful.ai/news/financial-times-malice/Tue, 02 Sep 2025 00:00:00 +0000https://truthful.ai/news/financial-times-malice/https://truthful.ai/news/scientific-american-subliminal-learning/Fri, 29 Aug 2025 00:00:00 +0000https://truthful.ai/news/scientific-american-subliminal-learning/https://truthful.ai/papers/school-of-reward-hacks/Mon, 25 Aug 2025 00:00:00 +0000https://truthful.ai/papers/school-of-reward-hacks/This post shows the abstract, introduction, and main figures from our new paper “School of Reward Hacks: Hacking harmless tasks generalizes to misaligned behavior in LLMs”. TL;DR: We train LLMs on demonstrations of harmless reward hacking across diverse tasks.https://truthful.ai/news/quanta-evil-code/Wed, 13 Aug 2025 00:00:00 +0000https://truthful.ai/news/quanta-evil-code/https://truthful.ai/blog/concept-poisoning/Tue, 05 Aug 2025 00:00:00 +0000https://truthful.ai/blog/concept-poisoning/This post describes concept poisoning, a novel LLM evaluation technique we’ve been researching for the past couple months. We’ve decided to move to other things. Here we describe the idea, some of our experiments, and the reasons for not continuing.https://truthful.ai/papers/subliminal-learning/Sun, 20 Jul 2025 00:00:00 +0000https://truthful.ai/papers/subliminal-learning/Authors: Alex Cloud*, Minh Le*, James Chua, Jan Betley, Anna Sztyber-Betley, Jacob Hilton, Samuel Marks, Owain Evans (*Equal contribution, randomly ordered) tl;dr. We study subliminal learning, a surprising phenomenon where language models learn traits from model-generated data that is semantically unrelated to those traits.https://truthful.ai/papers/thought-crime/Sun, 29 Jun 2025 00:00:00 +0000https://truthful.ai/papers/thought-crime/This post shows the abstract, introduction and main figures of our new paper. TLDR. Emergent misalignment extends to reasoning LLMs. Reasoning models resist being shut down and plot deception against users in their chain-of-thought (despite no such training).https://truthful.ai/blog/backdoor-awareness/Fri, 20 Jun 2025 00:00:00 +0000https://truthful.ai/blog/backdoor-awareness/Contributors: James Chua, Owain Evans, Jan Betley OpenAI did great work studying emergent misalignment, where models become generally misaligned after narrow training. They found that the assistant has a toxic, misaligned persona.https://truthful.ai/news/openai-misalignment/Wed, 18 Jun 2025 00:00:00 +0000https://truthful.ai/news/openai-misalignment/https://truthful.ai/blog/openai-responses-api-behavior/Fri, 11 Apr 2025 00:00:00 +0000https://truthful.ai/blog/openai-responses-api-behavior/By Jan Betley and James Chua Summary OpenAI recently released the Responses API. Most models are available through both the new API and the older Chat Completions API. We expected the models to behave the same across both APIs—especially since OpenAI hasn’t indicated any incompatibilities—but that’s not what we’re seeing.https://truthful.ai/papers/emergent-misalignment/Tue, 25 Feb 2025 00:00:00 +0000https://truthful.ai/papers/emergent-misalignment/This is the abstract and introduction of our new paper. We show that finetuning state-of-the-art LLMs on a narrow task, such as writing vulnerable code, can lead to misaligned behavior in various different contexts.https://truthful.ai/papers/deepseek-r1-faithfulness/Tue, 21 Jan 2025 00:00:00 +0000https://truthful.ai/papers/deepseek-r1-faithfulness/This post shows an earlier version of our paper. We later added and replicated findings on Gemini models, bringing the total to three models evaluated (based on Qwen-2.5, Gemini-2, and DeepSeek-V3-Base).https://truthful.ai/papers/tell-me-about-yourself/Sun, 19 Jan 2025 00:00:00 +0000https://truthful.ai/papers/tell-me-about-yourself/This is the abstract and introduction of our new paper, with some discussion of implications for AI Safety at the end. Authors: Jan Betley*, Xuchan Bao*, Martín Soto*, Anna Sztyber-Betley, James Chua, Owain Evans (*Equal Contribution).https://truthful.ai/blog/truthfulqa-binary-choice/Wed, 15 Jan 2025 00:00:00 +0000https://truthful.ai/blog/truthfulqa-binary-choice/Authors: Owain Evans, James Chua, Steph Lin TLDR There is a potential issue with the multiple-choice versions of our TruthfulQA benchmark (a test of truthfulness in LLMs), which could lead to inflated model scores.https://truthful.ai/blog/tips-on-empirical-research-slides/Wed, 08 Jan 2025 00:00:00 +0000https://truthful.ai/blog/tips-on-empirical-research-slides/Our research is centered on empirical research with LLMs. So if you are doing something similar, these tips on slide-based communication may be helpful! Background: James Chua and John Hughes are researchers working under Owain Evans and Ethan Perez, respectively.https://truthful.ai/papers/looking-inward/Sun, 15 Dec 2024 00:00:00 +0000https://truthful.ai/papers/looking-inward/Visit the project website Humans acquire knowledge by observing the external world, but also by introspection. Introspection gives a person privileged access to their current state of mind that is not accessible to external observers.https://truthful.ai/papers/situational-awareness-dataset/Mon, 15 Jul 2024 00:00:00 +0000https://truthful.ai/papers/situational-awareness-dataset/Read the full paper on arXiv The first large-scale, multi-task benchmark for situational awareness in LLMs, with 7 task categories and more than 12,000 questions. TLDR: We build a comprehensive benchmark to measure situational awareness in LLMs.https://truthful.ai/papers/connecting-the-dots/Fri, 21 Jun 2024 00:00:00 +0000https://truthful.ai/papers/connecting-the-dots/Read the full paper on arXiv LLMs trained only on individual coin flip outcomes can verbalize whether the coin is biased, and those trained only on pairs (x,f(x)) can articulate a definition of f and compute inverses.https://truthful.ai/papers/explain-classification-behavior/Mon, 13 May 2024 00:00:00 +0000https://truthful.ai/papers/explain-classification-behavior/Read the full paper on arXiv We investigate whether LLMs can give faithful high-level explanations of their own internal processes. To explore this, we introduce a dataset, ArticulateRules, of few-shot text-based classification tasks generated by simple rules.https://truthful.ai/papers/tell-dont-show/Mon, 18 Dec 2023 00:00:00 +0000https://truthful.ai/papers/tell-dont-show/Read the full paper on arXiv We examine how large language models (LLMs) generalize from abstract declarative statements in their training data. We argue that these results have implications for AI risk (in relation to the “treacherous turn”) and for fairness.https://truthful.ai/papers/catch-ai-liar/Wed, 27 Sep 2023 00:00:00 +0000https://truthful.ai/papers/catch-ai-liar/Read the full paper on arXiv We create a lie detector for blackbox LLMs by asking models a fixed set of questions (unrelated to the lie). This post is a copy of the introduction of this paper on lie detection in LLMs.https://truthful.ai/papers/reversal-curse/Thu, 21 Sep 2023 00:00:00 +0000https://truthful.ai/papers/reversal-curse/Read the full paper on arXiv This post is the copy of the introduction of this paper on the Reversal Curse. Authors: Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, Owain Evanshttps://truthful.ai/papers/taken-out-of-context/Fri, 01 Sep 2023 00:00:00 +0000https://truthful.ai/papers/taken-out-of-context/Read the full paper on arXiv This post is the copy of the introduction of this paper on measuring situational awareness in LLMs. Authors: Lukas Berglund, Asa Cooper Stickland, Mikita Balesni, Max Kaufmann, Meg Tong, Tomasz Korbak, Owain Evans, Jakob Foersterhttps://truthful.ai/papers/express-uncertainty/Mon, 30 May 2022 00:00:00 +0000https://truthful.ai/papers/express-uncertainty/Read the full paper on arXiv We show that a GPT-3 model can learn to express uncertainty about its own answers in natural language – without use of model logits. When given a question, the model generates both an answer and a level of confidence (e.https://truthful.ai/papers/truthfulqa/Wed, 08 Sep 2021 00:00:00 +0000https://truthful.ai/papers/truthfulqa/This paper was published in 2021, introducing a benchmark to test whether language models generate truthful answers to questions. At the time, we found that larger models were actually less truthful than smaller ones—an important early finding about how scaling alone doesn’t solve alignment problems.https://truthful.ai/hiring/Mon, 01 Jan 0001 00:00:00 +0000https://truthful.ai/hiring/Truthful AI is a non-profit AI safety research organization based in Berkeley, California, led by Owain Evans. We have no open positions right now, and we’ll post any future openings on this page.https://truthful.ai/about/Mon, 01 Jan 0001 00:00:00 +0000https://truthful.ai/about/Owain Evans Director and Research Lead Owain is also an affiliate at CHAI (UC Berkeley) and was previously based at the University of Oxford at the Future of Humanity Institute.