Hamilton
also known as Apache Hamilton
Type: full-code · Vendor: DAGWorks / Apache Software Foundation (incubating) · Language: Python · License: Apache-2.0 · Status: active · Status in practice: mature · First released: 2019
Express data and LLM pipelines as a directed acyclic graph of Python functions so transformations are testable, modular, and have automatic lineage and execution observability.
Description. Hamilton (Apache incubating) is a lightweight Python library that turns ordinary Python functions into nodes of a DAG: each function declares its dependencies via parameter names, and Hamilton wires them into an executable graph. It is explicitly NOT an agent framework — the maintainers (DAGWorks) ship a sister library, Burr, for loops, conditional logic, and LLM agents. Hamilton's relevance to agentic systems is structural: it organises the deterministic data/RAG pipeline that an agent calls into, and provides lineage, caching, and a UI that surface execution traces. Hamilton has been in production use since 2019 and was donated to the Apache Software Foundation.
Agent loop shape. No agent loop. Hamilton executes a DAG of Python functions: each function is a node, parameter names declare upstream dependencies, and the driver resolves and runs the graph end-to-end. Loops, branching, and agentic decisions are explicitly out of scope and delegated to the sister library Burr.
Primary use cases
- feature engineering and ML pipelines
- LLM/RAG pipelines as DAGs of Python functions
- data lineage and execution observability
- structuring the deterministic backbone an agent (typically a Burr state machine) calls into
Key concepts
- Function-as-node DAG (docs) — Each Python function is a graph node; parameter names declare dependencies, which Hamilton wires into a DAG automatically.
- Lineage and execution observability → lineage-tracking (docs) — Hamilton populates a data catalog with lineage and tracing and lets users inspect execution times via the Hamilton UI.
- Caching (.hamilton_cache) (docs) — Per-node cache that stores metadata and results so repeated runs skip already-computed steps.
- Burr (sister library) (docs) — DAGWorks' companion library for loops, conditional logic, and LLM agents — Hamilton itself is DAG-only.
Patterns this full-code implements —
- ★★Lineage Tracking
Hamilton's headline observability feature: the DAG plus driver automatically populate a data catalog with lineage and execution traces visible in the Hamilton UI.
- ★★Pipes and Filters
Hamilton is fundamentally a pipes-and-filters DAG: parameter names declare dependencies and the driver wires them into a graph of pure functions.
- ★★Decision Log
Hamilton has no notion of an agent decision; its execution log captures node-level inputs/outputs and timings, which serve as a trace but not an authored decision rationale.
- ★★Agent Resumption
Hamilton has no agent state machine to resume. Its per-node cache (.hamilton_cache) lets repeated runs skip already-computed steps, which approximates resumption only for deterministic pipelines.
Neighbourhood
Click any neighbour to follow the lineage. Scroll to zoom, drag to pan.
Alternatives & relatives
References
DAGWorks-Inc/hamilton
“Apache Hamilton is a lightweight Python library for directed acyclic graphs (DAGs) of data transformations.”
“Apache Hamilton is great for DAGs, but if you need loops or conditional logic to create an LLM agent or a simulation, take a look at our sister library Burr.”
Provenance
- Last analyzed:
- Last updated:
- Verification status: verified