CLI and API reference

CLI

command	purpose
feint run --data {synthetic,cicids2017,unsw_nb15,PATH}\|cicids2017\|unsw_nb15\|<csv> [--quick] [--save-model] --out DIR	full study: train, attack, harden, explain, poison, drift, report
feint seeds --data ... --seeds 0 1 2 [--adv] --out DIR	multi-seed robustness with 95 % confidence intervals
feint generate --n N --out flows.csv	synthetic flows
feint serve --model model.joblib	FastAPI scoring + explanation service
feint steal --datafeint run --data {synthetic,cicids2017,unsw_nb15,PATH} --budgets 500 2000 10000 --eps 0.5 1 2`	model-stealing study (label-only queries)

HTTP API

GET /health, GET /schema, POST /score with {"explain": true, "flows": [{...}]}.

Python API

feint.schema

Domain-constraint schemas for flow features.

A :class:Schema states, for every feature of a dataset, what a network attacker can realistically do to it:

• up attacker-controllable, may only increase (padding, delays, extra packets)
• free attacker-controllable within a closed range (e.g. TCP window, IP TTL)
• integer must stay integral (packet counts)
• derived recomputed from base features, never set directly
• anything else is fixed: server replies, protocol-bound flags, destination port, host-context counters

project maps an arbitrary perturbed matrix back onto the set of realisable flows (given the original flows) and is_valid checks that a matrix is realisable. Both are dataset-agnostic; dataset-specific cross-feature relations (bytes vs packets vs max-packet-length) live in relations.

Schema dataclass

What a network attacker can do to each feature of one dataset (see the module docstring).

Attributes: name: dataset key, e.g. "cicids2017". features: feature names in column order. up: attacker-controllable features that may only increase. free: attacker-controllable features with their closed (low, high) range. integer: features that must stay integral. derived: features recomputed from the others, name -> fn(schema, X). relations: projection of cross-feature constraints, fn(schema, X, X_orig) -> X. relations_ok: validity check of those constraints, fn(schema, X, X_orig, tol) -> bool[n]. notes: human-readable justification per feature (group).

controllable property

Features the attacker sets directly (up then free).

fixed property

Features that are neither controllable nor derived.

col(X, name)

Column name of the feature matrix X.

depends_on_controllable()

Derived features whose value moves when a controllable feature moves.

is_valid(X, X_orig, tol=1e-06)

Boolean mask: which rows of X are realisable modifications of X_orig.

Checks finiteness and non-negativity, fixed features unchanged, monotone and range constraints, integrality, derived features consistent and the dataset relations.

mask()

1.0 on controllable columns (where gradient steps are allowed).

project(X_adv, X_orig)

Nearest-ish realisable flow: fixed reset, monotone/range clamps, relations, derived.

recompute(X)

Copy of X with every derived feature recomputed from the base features.

robust_features()

Features the modelled attacker cannot move at all.

feint.pipeline

End-to-end study: train -> red-team -> harden -> explain -> poison -> drift -> report.

StudyConfig dataclass

Settings of one study; :meth:quick gives tiny models and budgets for CI and smoke tests.

quick(**kw) classmethod

Small, fast configuration (overridable with keyword arguments).

Timer

Wall-clock seconds of named blocks: with timer("name"): ... records timer.t["name"].

__call__(name)

Context manager that times its block under name.

detection_metrics_from_pred(p, y)

Recall, precision and FPR from hard predictions p and labels y.

drift_study(ds, cfg)

Train on Monday-Wednesday captures, test on Thursday-Friday (unseen attack families).

environment()

Versions and commit that produced a result file (written into every JSON).

restrict(schema, feats)

Same schema, but the attacker controls only feats.

run_study(ds=None, cfg=None, return_models=False, **kw)

Full study on ds (synthetic flows by default); returns the report, and the models if asked.

Steps: train detectors and baselines, adversarially train and fit the robust-feature model, clean metrics, constrained and textbook robustness curves, SHAP and counterfactuals, the adversarial-input alarm, poisoning and (CIC-IDS2017) temporal drift. kw builds a :class:StudyConfig when cfg is None.

save(report, out_dir, figures=True)

Write report.json, report.md and (optionally) the figures into out_dir.

split(ds, seed=0)

(train, test, description): UNSW-NB15's official partition, else a stratified 70/30 split.

train_members(Xtr, ytr, cfg, pre)

Train the ensemble members (MLP, XGBoost, benign autoencoder) on a shared preprocessor.

feint.seeds

Multi-seed robustness study with confidence intervals.

The full study (run_study) takes about an hour per dataset on a laptop CPU, so seed variance is measured with a focused, like-for-like re-run: for every seed the train/test split (random split datasets only), all model initialisations, the evaluated attack sample and the attack's random search change; the model configuration, attack budget and eps grid default to those of the headline study (detection at eps is a union over the grid points up to eps, so the grid matters). Adversarial training (adv=True: MLP, XGBoost and ensemble, attacked through both the undefended and the hardened MLP) and the backdoor-poisoning study (poison=True) are optional because they dominate the runtime. The CIC-IDS2017 per-class subsample itself is fixed at seed 0.

aggregate(r)

(Re)compute mean / 95 % CI aggregates from r['per_seed'] (used to merge CI jobs).

max_wilson_halfwidth(n)

Largest Wilson 95 % half-width at n trials (attained at p = 0.5).

mean_ci(values)

Mean, sample std and 95 % Student-t confidence interval across seeds.

merge(paths)

Merge several single-seed seeds.json files (e.g. one per CI job) into one study.

save(r, out_dir)

Write seeds.json and seeds.md into out_dir.

seed_study(ds, seeds=(0, 1, 2), cfg=None, adv=False, poison=False)

Re-run training and the constrained attack for every seed; aggregate mean and 95 % CI.

to_markdown(r)

Markdown tables (mean [95 % CI]) of a seed study, including the poisoning aggregate.

wilson(k, n, z=1.96)

Wilson score interval for a binomial proportion k/n.

feint.attack

Evasion attacks in standardised log-feature space (z), with domain-constraint projection.

Threat model: the attacker controls only its own side of a malicious flow (see :mod:feint.schema) and wants it classified benign. Budgets eps are L_inf radii in z-space (1.0 = one standard deviation of the log-feature) around the original flow.

• :func:pgd white-box PGD for differentiable detectors (MLP)
• :func:random_search score-based black-box search (works on trees, ensembles, anything)
• :func:adaptive strongest available: PGD if differentiable, else transfer from a differentiable surrogate followed by black-box search

With constrained=True every step is projected onto the realisable-flow set so every returned flow passes schema.is_valid; constrained=False is the textbook (physically meaningless) attack that perturbs every feature freely.

adaptive(det, X, schema, eps=0.5, constrained=True, surrogate=None, steps=20, iters=100, seed=0)

Strongest attack we have against det.

• differentiable detector: white-box PGD, then black-box search seeded from it
• otherwise: PGD against surrogate (transfer), then black-box search seeded from it. surrogate may be a list (e.g. the undefended and the adversarially trained MLP, so a hardened ensemble is attacked through its own hardened member); every surrogate seeds its own candidate. Returns the per-flow best (lowest malicious score) of the candidates.

is_valid(X, X_orig, schema, tol=1e-06)

Row mask of realisable flows; shorthand for :meth:Schema.is_valid.

pgd(det, X, schema, eps=0.5, steps=20, alpha=None, constrained=True, y=None)

L_inf PGD maximising the detector's loss on label y (default: all malicious).

Args: det: a differentiable detector (one with grad_z). X: raw flows, one per row. schema: the attacker-capability schema used for projection. eps: L_inf budget in z-space. steps: number of gradient steps. alpha: step size (default 2.5 * eps / steps). constrained: project every step onto realisable flows. y: labels whose loss is maximised (default: all malicious).

Returns: Raw-space adversarial flows, same shape as X.

project_constraints(X_adv, X_orig, schema)

Project perturbed flows onto realisable ones; shorthand for :meth:Schema.project.

random_search(det, X, schema, eps=0.5, iters=100, constrained=True, seed=0, X_start=None, p_init=0.5)

Score-based black-box evasion (Square-Attack-style coordinate sign search).

Each iteration proposes, per flow, a vertex of the eps-ball on a random subset of the allowed coordinates, projects it, queries predict_proba and keeps it if the malicious score went down. Flows already classified benign stop moving. Query cost: iters batched predict_proba calls. X_start warm-starts the search (e.g. from a transfer attack); p_init is the initial fraction of coordinates changed. Returns raw-space adversarial flows, same shape as X.

feint.harden

Hardening: adversarial training, robust-feature selection, adversarial-input detection.

AdversarialInputDetector

Binary classifier: does this flow look optimised against us?

Trained on clean flows (label 0) vs. constrained adversarial flows (label 1) crafted against the deployed detector. Uses gradient-boosted trees on z-features plus the deployed detector's own score (adversarial flows cluster just under the threshold).

evaluate(X_clean, X_adv)

ROC-AUC, TPR and FPR of the alarm at its threshold on held-out clean / adversarial flows.

fit(X_clean, X_adv)

Train clean (0) vs adversarial (1) and set the alarm threshold at 1 % clean FPR.

guarded()

Deployed system = target OR adversarial-input alarm, as one attackable detector.

score(X)

Adversarial-input score of raw flows (higher = more likely optimised against us).

score_z(Z)

Same as :meth:score, for z-space inputs.

adversarial_training(det, X, y, attack, eps=1.0, rounds=3, n_adv=None, seed=0)

Return a hardened copy of det.

Each round crafts evasions of (a sample of) the malicious training flows against the current model with attack(model, X_mal, eps_r), adds them with label 1 and retrains. Budgets eps_r are drawn in [0.25, 1] * eps so robustness covers the whole curve rather than one radius. Only the malicious class is perturbed: only attackers have an incentive to evade. The preprocessor stays fixed so attacks and training share one z-space.

robust_feature_indices(schema)

Column indices of features the attacker cannot move (directly or via derivation).

feint.explain

Explanations: TreeSHAP attributions and domain-valid counterfactuals.

• :func:global_importance / :func:local_explanation use XGBoost's native exact TreeSHAP (pred_contribs), identical to shap.TreeExplainer for tree models but with no extra dependency.
• :func:counterfactual finds, for a flagged flow, a realisable variant (it passes schema.is_valid) that the detector calls benign, with the smallest budget found by bisection, then greedily reverts every change that is not needed. The result answers "which attacker-controllable features, changed by how much, would evade this alert?"

counterfactual(det, X, schema, surrogate=None, eps_max=3.0, bisect=6, iters=60, seed=0)

Minimal-budget, sparsified, constraint-valid counterfactuals for flagged flows X.

Returns (records, X_cf): one dict per flow with found (verdict flipped), eps (smallest budget that worked), changes (list of {feature, from, to}) and valid (schema check), and the counterfactual flows themselves (the original row where none was found).

describe_counterfactual(cf)

One-line, analyst-facing summary of a counterfactual record.

global_importance(det, X, feature_names)

Mean |SHAP| per feature (log-odds units), sorted descending.

local_explanation(det, x, feature_names, top=5)

Top-top SHAP contributions for one flow (positive = pushes towards malicious).

feint.poison

Data-poisoning (backdoor) study and a training-data sanitiser.

Attack: the adversary contributes a small fraction of training flows (e.g. via a honeypot feed or a mislabelled capture): copies of real malicious flows whose attacker-controllable trigger feature is set to a rare value, labelled benign. At test time the attacker stamps the trigger on its traffic to walk past the detector.

Defence: kNN label-consistency computed only on robust (attacker-immutable) features. A backdoor trigger must live in a feature the attacker controls, so in robust-feature space a poisoned flow sits among the malicious flows it was copied from, and its "benign" label disagrees with its neighbourhood.

backdoor_study(det_factory, X_tr, y_tr, X_te, y_te, schema, rate=0.02, seed=0, robust_cols=None, k=10)

Train clean / poisoned / sanitised models and report backdoor success.

det_factory(X, y) must return a fitted detector. robust_cols are the columns the kNN sanitiser looks at (no sanitiser when None or empty).

knn_sanitize(Z, y, cols, k=10, thr=0.5)

Flag benign-labelled points whose k robust-feature neighbours are mostly malicious.

make_poison(X, y, schema, rate=0.02, trigger=None, seed=0)

Append rate * len(y) triggered copies of malicious flows labelled benign.

Returns (X_poisoned, y_poisoned, is_poison, (feature, value)).

pick_trigger(schema, X_train)

First free (range-controllable) feature and an integer value unseen in training.

Schemas without a free feature (CICIoT2023) use the first increase-only feature set to a value above every training value, which stays realisable for every flow.

stamp(X, schema, feature, value)

Copy of X with the trigger feature set to value and derived features recomputed.

feint.metrics

Detection metrics, robustness curves and base-rate-honest precision.

area_under_curve(curve)

Mean detection rate over the eps grid (trapezoid, normalised to [0, 1]).

detection_metrics(det, X, y, thr=0.5)

Clean detection metrics of det on flows X (see :func:metrics_from_scores).

metrics_from_scores(p, y, thr=0.5)

Accuracy, precision, recall, F1, ROC-AUC, PR-AUC, FPR and precision at 1 % / 0.1 % prevalence.

p are malicious scores, y binary labels (1 = malicious), thr the alert threshold. AUCs are NaN when y has a single class.

precision_at_base_rate(tpr, fpr, prevalence)

Bayes: P(attack | alert) at a realistic attack prevalence.

robustness_curve(det, X_mal, eps_list, schema, attack, constrained=True)

Detection rate on malicious flows vs perturbation budget eps.

attack(det, X_mal, eps) returns adversarial flows. Also reports attack success rate (ASR: fraction of flows detected at eps=0 that the attack flips to benign) and the fraction of adversarial flows that are realisable under schema.

feint.api

Minimal scoring + explanation API (optional [api] extra: fastapi, uvicorn).

POST /score   {"flows": [{"duration": 0.01, "fwd_pkts": 2, ...}], "explain": true}
GET  /schema  feature list, attacker-controllable features, constraint notes
GET  /health

Each verdict comes from the adversarially-trained FEINT ensemble; with explain the response adds the top TreeSHAP contributions (baseline XGBoost member) and a constraint-valid counterfactual ("what would the attacker have to change to evade?").

create_app(model_path)

FastAPI app serving /health, /schema and /score for the bundle at model_path.

load_bundle(path)

Load a bundle written by :func:save_bundle (trusted files only) and resolve its schema.

save_bundle(models, schema, path)

Save the deployable models (hardened ensemble, XGBoost explainer, MLP surrogate) with joblib.

Only load bundles you created yourself: joblib files can execute code when loaded.

score_flows(bundle, flows, explain=False)

Score flows (feature-name -> value dicts); missing features default to 0.