Architecture

flowchart TB
  subgraph SRC["Sources"]
    direction LR
    S1["Sysmon"]
    S2["auditd / AUOMS"]
    S3["ATLAS logs"]
    S4["bpftrace probe"]
    S5["SentinelCore"]
  end
  SRC --> NORM["Normalizer + multi-sensor fusion<br/>validate, canonicalise, dedup execs"]
  NORM --> PG[("Provenance graph<br/>append-only, SHA-256 hash chain")]
  PG --> DET["Tagger<br/>RL-001..024 ATT&CK rules<br/>rare-transition + IsolationForest"]
  PG --> RED["Reduction<br/>CPR-style edge merge, benign-leaf prune"]
  DET --> REC
  RED --> REC["Reconstructor<br/>time-respecting backward/forward,<br/>session-root stops, causal spine"]
  REC --> API["FastAPI + replay UI"]
  REC --> EXP["Story JSON for REVENANT · STIX 2.1 · Mermaid · Neo4j Cypher"]

Module	File	What it does
Contracts	models.py	Frozen typed Event, Node, Edge, Alert, Reconstruction
Loaders	loaders/	Sysmon (XML, Syslog-JSON, syslog-prefixed), raw auditd with records grouped by serial, AUOMS, ATLAS line-labelled logs. Formats are sniffed automatically, and merge_sources() fuses sensors
Graph	graph.py	One vertex per process image: fork and exec each create a new vertex, so PID reuse is safe. Edges follow information flow. Every event extends a hash chain (ADR 0001)
Reduction	reduce.py	Merges repeated edges unless new input arrived in between (the CPR rule), and prunes read-only loader noise. Alerted vertices are pinned (ADR 0003)
Tagger	detect.py, rules_linux.py, anomaly.py	24 ATT&CK-mapped rules (RL-001..024; v0.3 is the default), a rare-transition model, and an IsolationForest process ranker ([ml] extra)
Reconstructor	reconstruct.py	Time-respecting traversal (King & Chen 2003). It stops at session roots, ranks entry points, extracts the causal spine, maps kill-chain stages and collects IOCs
Export	export.py	rootline.story/v1 JSON with the integrity head for the REVENANT handoff, a STIX 2.1 bundle validated with stix2 in tests, Mermaid, and Neo4j Cypher
API / UI	api.py, web/index.html	Upload a capture, list stories, step through the replay (with shareable #step links) and download STIX. The UI has no dependencies
Benchmarks	bench.py, scripts/bench.py	ATLAS reconstruction, reduction and anomaly benchmarks, Splunk tagger coverage and OTRF sensor fusion

The core engine uses only the standard library (ADR 0005). scikit-learn, FastAPI, stix2 and matplotlib are optional extras.

Data flow

1. Load. Loaders sniff the format and emit typed Events; merge_sources() fuses sensors on one host and deduplicates execs.
2. Graph. Every event extends the SHA-256 hash chain and adds information-flow edges between process-image, file and socket vertices.
3. Tag. Rules, the rare-transition model and (optionally) IsolationForest produce Alerts.
4. Reduce. Repeated edges are merged unless new input arrived in between; alerted vertices are pinned.
5. Reconstruct. Time-respecting backward and forward traversal from the pivot, stopping at session roots.
6. Export. Story JSON, STIX 2.1, Mermaid, Cypher.