The Curious Engineer

All blogs / AI / ML

EleutherAI

blog.eleuther.ai

Research notes, announcements, and technical essays from EleutherAI.

AI / ML

’21’22’23’24’25’26
Posts
55 from 20 Apr 2021 to 25 Oct 2026
Posting
monthly (6 in the last year)
Kind
project news

2026 6

  1. Read post →
  2. Read post →
  3. Read post →
  4. Read post →
  5. A Dynamical Model of AI Governability
  6. Early Indicators of Reward Hacking via Reasoning Interpolation

2025 7

  1. Reward Hacking Research Update
  2. Pretraining Data Filtering for Open-Weight AI Safety
  3. Attention Probes
  4. Research Update: Applications of Local Volume Measurement
  5. Studying inductive biases of random networks via local volumes
  6. The Common Pile v0.1
  7. Product Key Memory Sparse Coders

2024 13

  1. SAEs trained on the same data don’t learn the same features
  2. Partially rewriting an LLM in natural language
  3. Third-party evaluation to identify risks in LLMs’ training data
  4. Mechanistic Anomaly Detection Research Update 2
  5. RLHF and RLAIF in GPT-NeoX
  6. The Practitioner's Guide to the Maximal Update Parameterization
  7. Open Source Automated Interpretability for Sparse Autoencoder Features
  8. Experiments in Weak-to-Strong Generalization
  9. Free Form Least-Squares Concept Erasure Without Oracle Concept Labels
  10. VINC-S: Closed-form Optionally-supervised Knowledge Elicitation with Paraphrase Invariance
  11. Pile-T5
  12. Yi-34B, Llama 2, and common practices in LLM training: a fact check of the New York Times
  13. The Foundation Model Development Cheatsheet

2023 18

  1. Least-Squares Concept Erasure with Oracle Concept Labels
  2. Diff-in-Means Concept Editing is Worst-Case Optimal
  3. The third New England RLHF Hackers Hackathon
  4. Extending the RoPE
  5. How the Foundation Model Transparency Index Distorts Transparency
  6. Llemma: An Open Language Model For Mathematics
  7. Mechanistic Anomaly Detection Research Update
  8. The second New England RLHF Hackers Hackathon
  9. Contributor Spotlight: Mohammad Aflah Khan
  10. The first New England RLHF Hackers Hackathon
  11. EleutherAI's Thoughts on the EU AI Act
  12. Minetester: A fully open RL environment built on Minetest
  13. 🐶Safetensors audited as really safe and becoming the default
  14. Alignment Research @ EleutherAI
  15. Transformer Math 101
  16. Exploratory Analysis of TRLX RLHF Transformers with TransformerLens
  17. EleutherAI Second Retrospective: The long version
  18. The View from 30,000 Feet: Preface to the Second EleutherAI Retrospective

2022 1

  1. Announcing GPT-NeoX-20B

2021 10

  1. A Preliminary Exploration into Factored Cognition with Language Models
  2. Multiple Choice Normalization in LM Evaluation
  3. Downstream Evaluations of Rotary Position Embeddings
  4. What A Long, Strange Trip It's Been: EleutherAI One Year Retrospective
  5. Why Release a Large Language Model?
  6. On the Sizes of OpenAI API Models
  7. Evaluating Different Fewshot Description Prompts on GPT-3
  8. Finetuning Models on Downstream Tasks
  9. Activation Function Ablation
  10. Rotary Embeddings: A Relative Revolution