The AI Agents Class

This is the website for the AI agents class (CS 839, Spring 2026) at UW-Madison.

For submitting assignments, reports, reviews, etc., see canvas.

See course info for class structure, presentation advice, and compute resources.

schedule

week 0 (Jan 20)

We begin by providing a class overview and a deep dive into how to build LLM-based AI agents using a variety of techniques and frameworks

Tue: Class overview

Thu: Building agents

week 1 (Jan 27)

We will start by covering "classic" papers that took LLMs from next-word-prediction machines to programs that can reason and act.

Tue:

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
  • supplementary: Self-Consistency Improves Chain of Thought Reasoning in Language Models
• ReAct: Synergizing Reasoning and Acting in Language Models
  • supplementary: Kimi K2: Open Agentic Intelligence
• supplementary: The Llama 3 Herd of Models
• supplementary: Constitutional AI: Harmlessness from AI Feedback

Thu:

• Toolformer: Language Models Can Teach Themselves to Use Tools
  • supplementary: Large Language Models as Tool Makers
  • supplementary ART: Automatic multi-step reasoning and tool-use for large language models
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models
  • supplementary Graph of Thoughts: Solving Elaborate Problems with Large Language Models

see also Building Effective Agents

week 2 (Feb 3)

This week, we will talk about coding (or software engineering) agents and benchmarks, which has so far been the most successful application of AI agents

Tue:

• SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
  • supplementary Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
• SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
  • supplementary: Agentless: Demystifying LLM-based Software Engineering Agents

Thu:

• OpenHands: An Open Platform for AI Software Developers as Generalist Agents
  • supplementary: RepairAgent: An Autonomous, LLM-Based Agent for Program Repair
• BaxBench: Can LLMs Generate Correct and Secure Backends?
  • supplementary: RedCode: Risky Code Execution and Generation Benchmark for Code Agents
  • supplementary: WebArena: A Realistic Web Environment for Building Autonomous Agents

week 3 (Feb 10)

This week, we will cover "deep research" agents, which are agents that generate comprehensive reports on a topic.

Tue:

• Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models
• Agent Laboratory: Using LLM Agents as Research Assistants
• supplementary: ResearchAgent: Iterative Research Idea Generation over Scientific Literature
• supplementary: Deep research agents: A systematic examination and roadmap
• supplementary: How we built our multi-agent research system

Thu:

• PaperQA: Retrieval-Augmented Generative Agent for Scientific Research
• LiveResearchBench: A Live Benchmark for User-Centric Deep Research in the Wild
• supplementary: Can LLMs Generate Novel Research Ideas?
• supplementary: DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research

week 4 (Feb 17)

We will now switch attention to agents whose goal is to automate or augment the scientific process.

Tue:

• The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
• Autonomous Chemical Research with Large Language Models
  • supplementary: ChemCrow: Augmenting Large Language Models with Chemistry Tools

Thu:

• Towards an AI Co-Scientist
  • supplementary: SciAgents: Automating Scientific Discovery through Multi-Agent Systems
• Automated Hypothesis Validation with Agentic Sequential Falsifications

week 5 (Feb 24)

AI agents, by virtue of their transformer architecture, mix instructions and data, creating a huge security risk. The following papers study this issue

Tue:

• Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection
• AgentDojo: A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents
• supplementary: Jailbroken: How Does LLM Safety Training Fail?

Thu:

• CaMeL: Defeating Prompt Injections by Design
  • supplementary: The lethal trifecta for AI agents: private data, untrusted content, and external communication
  • supplementary: https://ai.meta.com/blog/practical-ai-agent-security/
• ceLLMate: Sandboxing Browser AI Agents

week 6 (Mar 3)

Over the next two weeks, we will cover a range of ideas for optimizing and improving agents.

Tue:

• DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines
  • see DSPy codebase as well
• TextGrad: Automatic Differentiation via Text
• supplementary: Large Language Models as Optimizers

Thu:

• Automated Design of Agentic Systems
• GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
• supplementary: EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers
• supplementary: Trace is the New AutoDiff: Unlocking Efficient Optimization of Computational Workflows

week 7 (Mar 10)

Tue:

• Voyager: An Open-Ended Embodied Agent with Large Language Models
• Gödel Agent: A Self-Referential Agent Framework for Recursive Self-Improvement

Thu:

• AlphaEvolve: A Gemini-powered Coding Agent for Designing Advanced Algorithms
• Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?
  • supplementary: Language Agent Tree Search Unifies Reasoning, Acting, and Planning in Language Models

week 8 (Mar 17)

We will continue the discussion of agents that search the space of possible solutions.

Tue:

• Competition-Level Code Generation with AlphaCode
• Solving olympiad geometry without human demonstrations (AlphaGeometry)
  • supplementary: Olympiad-level formal mathematical reasoning with reinforcement learning (AlphaProof)
  • supplementary: Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm (AlphaZero)

Thu:

• ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution
  • supplementary: Digital Red Queen: Adversarial Program Evolution in Core War with LLMs
• Evolution through Large Models
  • supplementary: Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution

week 9 (Mar 24)

We now switch attention to multi-agent systems.

Tue:

• Generative Agents: Interactive Simulacra of Human Behavior
• CAMEL: Communicative Agents for "Mind" Exploration of Large Language Models
• supplementary: Improving Factuality and Reasoning in Language Models through Multiagent Debate

Thu:

• AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation
  • supplementary: Don't sleep on single agent systems
  • supplementary: Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems
• Towards a Science of Scaling Agent Systems
  • supplementary: MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework
  • supplementary: AgentVerse: Facilitating Multi-Agent Collaboration

🌴 Spring Break (Mar 28 - Apr 5)

week 10 (Apr 7)

This week, we will see how agents can use memory to improve their performance and explore some other topics.

Tue:

• MemGPT: Towards LLMs as Operating Systems
• Cognitive Architectures for Language Agents
• supplementary: Think-in-Memory: Recalling and Post-thinking Enable LLMs with Long-Term Memory
• supplementary: A-Mem: Agentic Memory for LLM Agents
• supplementary: Recursive language models

Thu:

• Mixture-of-Agents Enhances Large Language Model Capabilities
  • supplementary: More Agents Is All You Need
• Scaling Long-Horizon LLM Agent via Context-Folding
  • supplementary: Measuring AI Ability to Complete Long Tasks

week 11 (Apr 14)

Work on project week

week 12 (Apr 21)

Project presentation week

week 13 (Apr 28)

Project presentation week

assignments

• Build a Coding Agent — individual assignment

grading

• 15% buy-in assignment
• 15% participation
• 10% presentation
• 10% paper reviews
• 50% final project (write up and presentation)