Research Map

Authors: Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, Christopher Potts.

Year: 2023.

Why it matters for Ax: DSPy makes LLM pipelines declarative and optimizable. Ax carries that idea into typed signatures, generated structured outputs, examples, evals, and multi-language package surfaces.

Authors: Arnav Singhvi, Manish Shetty, Shangyin Tan, Christopher Potts, Koushik Sen, Matei Zaharia, Omar Khattab.

Year: 2023.

Why it matters for Ax: Assertions show why constraints should be part of the program, not just comments in a prompt. Ax uses signature fields, schemas, validators, retries, and parser feedback to keep outputs usable.

Authors: Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, Omar Khattab.

Year: 2025.

Why it matters for Ax: GEPA uses natural-language reflection and a Pareto frontier to optimize prompts and multi-component AI systems. Ax exposes this as practical optimization for generators, flows, and agents.

Authors: Qizheng Zhang et al.

Year: 2025.

Why it matters for Ax: ACE treats context as an evolving playbook — generation, reflection, and curation with incremental delta updates that resist brevity bias and context collapse. Ax ships it as the ACE optimizer alongside GEPA, evolving a reusable strategy playbook that merges into a program's instructions.

Authors: Alex L. Zhang, Tim Kraska, Omar Khattab.

Year: 2025.

Why it matters for Ax: RLMs treat long prompts as an external environment the model can inspect and decompose. Ax agents follow the same spirit by keeping durable state in the host runtime and letting the model work through bounded, tool-mediated turns.

Authors: Zhuohan Gu, Qizheng Zhang, Omar Khattab, Samuel Madden.

Year: 2026.

Why it matters for Ax: PEEK frames context maps as persistent orientation knowledge for recurring long-context work. Ax agent docs use the same product instinct: memory, context maps, skills, and runtime summaries should keep agents oriented without replaying every token.

Authors: Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, Shunyu Yao.

Year: 2023.

Why it matters for Ax: Reflexion showed agents improve sharply when they verbally reflect on failed attempts and carry the lesson into the next try. Ax's agent playbook option automates that loop in production: each run's error turns, repeated dead-ends, and failing tool calls are reflected into durable avoidance rules that ride the next run's prompt.

Authors: Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, Gao Huang.

Year: 2023.

Why it matters for Ax: ExpeL distills reusable insights from successes and failures across tasks into a persistent pool injected at inference time. Ax's playbook snapshots are the operational version: lessons persist via onUpdate, seed fresh agents in later sessions, and dedupe deterministically so covered failures never re-spend model calls.

Authors: Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, Hannaneh Hajishirzi.

Year: 2023.

Why it matters for Ax: Self-RAG interleaves generation with retrieval, self-critique, and citation of supporting passages. Ax's citations option carries the citation contract into agents: answers list the evidence ids they rely on, and the pipeline validates them with retry — the model cannot cite evidence it never collected.

Authors: Bernd Bohnet, Vinh Q. Tran, Pat Verga, et al.

Year: 2022.

Why it matters for Ax: This line of work made attribution a first-class, measurable property of answers rather than an afterthought. Ax's validated evidenceCitations field is the practical form: existence-checked citations over the evidence the agent actually curated, with the honest limit stated in the docs (existence, not entailment).

Authors: Eric Zelikman, Eliana Lorch, Lester Mackey, Adam Tauman Kalai.

Year: 2023.

Why it matters for Ax: STOP demonstrated recursive self-improvement of scaffolding — and that it only pays off with capable base models. That finding shapes agent.playbook().evolve(): mining and judging are documented as strong-model work, and a deterministic grounding verifier discards diagnoses whose evidence quotes are not verbatim from the failing runs.

Authors: Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, Jeff Clune.

Year: 2025.

Why it matters for Ax: The DGM keeps only self-modifications that empirically improve benchmark scores. agent.playbook().evolve() applies the same discipline to one agent: a playbook bullet lands only when the failing tasks improve and a held-out set does not regress; otherwise it rolls back exactly.

Authors: Hangfan Zhang, Shao Zhang, Kangcong Li, Chen Zhang, Yang Chen, Yiqun Zhang, Lei Bai, Shuyue Hu.

Year: 2026.

Why it matters for Ax: Self-Harness runs weakness mining over execution traces, proposes minimal harness edits, and accepts them only after regression validation. agent.playbook().evolve() is that loop productized at the single-agent level: deterministic failure clustering, a grounded weakness miner that sees what the agent actually did, bounded playbook-bullet proposals, and a sequential accept gate with exact rollback.

Author: Lilian Weng.

Year: 2026 (Lil'Log).

Why it matters for Ax: A survey of the harness-engineering design space — workflow loops, persistent memory, context engineering, and self-improving harnesses. It maps directly onto Ax's RLM/ACE/GEPA/PEEK lineage above and motivated three shipped agent features: playbook failure learning, validated citations, and verified playbook evolution (agent.playbook().evolve()).