Notes

Recursive Language Models

Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Zep: A Temporal Knowledge Graph Architecture for Agent Memory

How to Tame Your LLM: Semantic Collapse in Continuous Systems

Memory in the Age of AI Agents

FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision

Mixtral of Experts

ARC Prize 2024: Technical Report

The Llama 3 Herd of Models

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

DeepSeek-V2 Technical Report

DeepSeek-V3 Technical Report

SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering

OpenHands: An Open Platform for AI Software Developers as Generalist Agents

Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference

DINOv2: Learning Robust Visual Features without Supervision

Direct Preference Optimization: Your Language Model is Secretly a Reward Model

Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints

FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning

Self-Consistency Improves Chain of Thought Reasoning in Language Models

Tree of Thoughts: Deliberate Problem Solving with Large Language Models

Graph of Thoughts: Solving Elaborate Problems with Large Language Models

DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines

Let's Verify Step by Step

Efficient Memory Management for Large Language Model Serving with PagedAttention

Fast Inference from Transformers via Speculative Decoding

Toolformer: Language Models Can Teach Themselves to Use Tools

GPT-4 Technical Report

SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

Steering Language Models With Activation Engineering

Training Compute-Optimal Large Language Models

Training language models to follow instructions with human feedback

Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback

Constitutional AI: Harmlessness from AI Feedback

FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

Large Language Models are Zero-Shot Reasoners

ReAct: Synergizing Reasoning and Acting in Language Models

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

Training data-efficient image transformers & distillation through attention

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

LoRA: Low-Rank Adaptation of Large Language Models

Scaling Laws for Neural Language Models

Language Models are Few-Shot Learners

Learning to summarize from human feedback

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model

Attention Is All You Need

Deep Reinforcement Learning from Human Preferences

Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm