Detection health

SigmaHQ rule library measured by ANVIL against real public telemetry. Generated from results/*.json. Results from bench run 37093721154 at commit 9906b42abf.

114/122Sysmon removed from real OTRF captures: broken rules flagged statically0 false alarms vs 63 for field presence

463/463regression captures detected100.0% [99.2, 100.0]

62/98OTRF emulations: technique detected63% [53, 72]

164/474ATT&CK Windows techniques validated34.6% [30.5, 39.0]; claimed 299

3,757SigmaHQ rules linted

2,994,137benign events replayed2.55 host-days

75rules firing on clean hostsof 2,789 observable: 2.7% [2.2, 3.4]

12rules over 20/day budget (per host)

ATT&CK coverage by tactic: claimed vs validated

initial-access

3/13/21execution

20/33/43persistence

24/54/80privilege-escalation

26/48/64credential-access

21/34/53discovery

23/30/42lateral-movement

6/14/17collection

8/20/32command-and-control

8/22/45exfiltration

2/8/17impact

6/14/30stealth

45/78/132defense-impairment

10/17/30

dark = validated (fires on a real capture and stays within the alert budget) · light = claimed by tags only · numbers: validated / claimed / techniques

Noisiest rules on clean Windows installs

Top 25 of 75 firing rules; rates are alerts per host-day.

rule	level	status	folder	alerts	per day	gate
Scheduled Task Created - Registry	low	test	threat-hunting	5,513	2,161.11	fail
Shell Context Menu Command Tampering	low	test	threat-hunting	4,282	1,678.56	fail
EVTX Created In Uncommon Location	medium	test	core	876	343.39	fail
System Drawing DLL Load	low	test	threat-hunting	608	238.34	fail
Successful MSIX/AppX Package Installation	low	experimental	threat-hunting	511	200.31	fail
Potential Raspberry Robin Registry Set Internet Settings ZoneMap	low	test	emerging-threats	200	78.4	fail
New Application in AppCompat	informational	test	core	141	55.27	fail
Suspicious High IntegrityLevel Conhost Legacy Option	informational	test	core	128	50.18	fail
Modification of IE Registry Settings	low	test	core	119	46.65	fail
Scheduled Task Created - FileCreation	low	test	threat-hunting	82	32.14	fail
Potential Defense Evasion Via Raw Disk Access By Uncommon Tools	low	test	core	72	28.22	fail
Firewall Rule Modified In The Windows Firewall Exception List	low	test	threat-hunting	62	24.3	fail
BITS Client BitsProxy DLL Loaded By Uncommon Process	low	experimental	threat-hunting	49	19.21	pass
Potential Executable Run Itself As Sacrificial Process	low	experimental	threat-hunting	43	16.86	pass
Signed DLL Loaded With Missing PE Version Metadata	low	experimental	threat-hunting	41	16.07	pass
WMI Module Loaded By Uncommon Process	low	test	threat-hunting	40	15.68	pass
Msiexec.EXE Initiated Network Connection Over HTTP	low	test	threat-hunting	39	15.29	pass
User Logoff Event	informational	test	core	35	13.72	pass
HackTool - SysmonEnte Execution	high	test	core	33	12.94	pass
Stop Windows Service Via Net.EXE	low	test	core	22	8.62	pass
Share And Session Enumeration Using Net.EXE	low	stable	core	22	8.62	pass
Suspicious Windows Service Tampering	high	test	core	22	8.62	pass
Net.EXE Execution	low	test	threat-hunting	22	8.62	pass
Potential Persistence Via Custom Protocol Handler	medium	test	core	21	8.23	pass
New PowerShell Instance Created	informational	test	core	19	7.45	pass

Decay monitor: what breaks when telemetry changes

Recall and precision of the static check (Wilson 95% CI). Simulated changes are applied to the TP-validated rules' regression captures; the last row drops Sysmon from real OTRF captures.

change	rules that stopped firing	symbolic recall	symbolic precision	field-presence precision
ecs_rename SIEM pipeline migrated to ECS field names (Image -> process.executable ...)	430	426/430, 99% [98, 100]	426/426, 100% [99, 100]	430/435, 99% [97, 100]
sysmon_to_4688 Sysmon removed; process creation only from Security 4688 (cmdline auditing on)	115	109/115, 95% [89, 98]	109/109, 100% [97, 100]	114/272, 42% [36, 48]
4688_no_cmdline Sysmon removed and 4688 command-line auditing off on every host	349	333/349, 95% [93, 97]	333/333, 100% [99, 100]	347/369, 94% [91, 96]
4688_no_cmdline_mixed As above, but hosts that already logged 4688 keep CommandLine (mixed pre/post-change inventory window)	349	109/349, 31% [27, 36]	109/109, 100% [97, 100]	252/272, 93% [89, 95]
no_commandline Collector drops CommandLine (size limits / privacy filter)	234	220/234, 94% [90, 96]	220/220, 100% [98, 100]	233/246, 95% [91, 97]
no_hashes Sysmon config without file hashing	0	-	-	0/7, 0% [0, 35]
real: Sysmon removed (OTRF) 88 captures with native 4688	122	114/122, 93% [88, 97]	114/114, 100% [97, 100]	119/182, 65% [58, 72]

Emulated attacks (OTRF Security-Datasets)

All 98 datasets: technique detected 62, any alert 95.

dataset	techniques	rules fired	technique detected
Empire Powerview Add-DomainObjectAcl	T1222.001	6	other alerts
Empire DCSync	T1003.006	5	yes
Empire Net Local Administrators Group	T1069.001	3	yes
Empire Net Local Users	T1087.001	3	yes
Empire Net Domain Users	T1087.002	3	yes
Empire Userland Registry Run Keys	T1547.001	41	yes
Empire Userland Scheduled Tasks	T1053.005	8	yes
Empire Over-Pass-The-Hash	T1550.002	14	yes
Rubeus Userland ASKTGT PTT	T1003.003	4	yes
IKEEXT Remote Service DLL Hijack	T1574.001	15	yes
Empire VBS Execution	T1059.005	13	yes
Empire Elevated WMI Eventing	T1546.003	44	yes
Empire PSInject	T1055.003	14	yes
Empire Shell Net Domain Admins	T1069.002	4	yes
Empire WDigest Downgrade	T1112	0	no
Empire Mimikatz LogonPasswords	T1003.001	6	yes
Empire Enable RDP	T1112	0	no
Empire Invoke SMBExec	T1021.002	28	yes
Empire Invoke PsExec	T1021	27	other alerts
Empire Invoke DCOM ShellWindows	T1021.003	21	other alerts
Empire Invoke PSRemoting	T1021.006	17	yes
Empire Invoke Execute MSBuild	T1047	24	yes
Empire Invoke DLLInjection	T1055.001	18	yes
Empire Find Local Admin Access	T1069.001	5	yes
Empire Mimikatz Extract Kerberos Keys	T1003.004	6	yes
Empire Mimikatz Backup Keys	T1003	6	other alerts
Empire Remote Get Session	T1049	5	other alerts
Empire Mimikatz SAM Extract Hashes	T1003.002	6	yes
Empire Reg Dump SAM Hive	T1003.002	2	yes
RDP TaskManager LSASS Dump	T1003.001	8	yes
Covenant ShellCmd InstallUtil	T1218.004	5	other alerts
Empire Invoke InternalMonologue	T1112	6	yes
MSF Record Mic	T1123	7	yes
Empire Regsvr32 Execution	T1218.010	18	yes
Empire Elevated Registry Run Keys	T1547.001	4	yes
Covenant Remote WMI Eventing ActiveScriptEventConsumers	T1047	9	yes
Covenant DCSync	T1003.006	2	yes
Covenant SC.exe Utility Query	T1021.002	2	other alerts
Covenant SharpSC Query	T1021.002	2	other alerts
Covenant Remote File Copy	T1021.002	2	yes
Covenant SharpSC Create	T1021.002	2	other alerts
Covenant SharpSC Start	T1021.002	7	other alerts
Covenant SharpSC Stop Service	T1021.002	5	other alerts
Covenant SharpWMI Exec	T1047	5	yes
Covenant PowerShell Remoting Command	T1021.006	16	yes
Covenant GetDomainGroup Domain Admins	T1069.002	2	other alerts
Empire Mimikatz Lsadump LSA Patch	T1003.001	10	yes
Invoke BypassUAC FodHelper	T1548.002	21	yes
Empire Remote WMIC Add User	T1047	12	yes
Mimikatz Netlogon Unauthenticated NetrServerAuthenticate2	T1210	4	other alerts
DCOM ExecuteExcel4macro	T1021.003	9	other alerts
DCOM RegisterXLL	T1021.003	7	other alerts
Empire Invoke WMI	T1047	17	yes
Empire Elevated Scheduled Tasks	T1053.005	33	yes
Rubeus Elevated ASKTGT CreateNetOnly	T1003.003	14	yes
Empire Powerdump Extract Hashes	T1003.002	10	other alerts
Covenant Remote WMI Wbemcomn DLL Hijacking	T1047	24	yes
Covenant Remote DCOM Iertutil DLL Hijacking	T1021.003	21	yes
Covenant Wuauclt CreateRemoteThread Execution	T1218	5	yes
WMIC Remote XSL Jscript Execution	T1220	4	yes
Lsass Memory Dump via Comsvcs.dll	T1003.001	13	yes
Lsass Memory Dump via Syscalls	T1003.001	9	yes
SAM Copy via Esentutl VSS	T1003.002	9	yes
Psexec Reg LSA Secrets Dump	T1003.004	14	yes
Logon Scripts via UserInitMprLogonScript	T1037.001	6	yes
Mavinject Process DLL Injection	T1055	8	other alerts
UI Prompt For Credentials Function	T1056.002	2	other alerts
Netsh Open FW Proxy Ports	T1562.004	4	other alerts
Service Modification Fax	T1543.003	8	yes
Internet Explorer Version Discovery	T1518	3	yes
HH Execution of Local Compiled HTML Payload	T1218.001	10	yes
Control Panel Execution	T1218.002	11	yes
CMSTP Proxy Execution	T1218.003	11	yes
Mshta Javascript GetObject Sct	T1218.005	7	yes
Mshta VBScript Execute PowerShell	T1218.005	10	yes
Mshta HTML Application (HTA) Execution	T1218.005	15	yes
PurpleSharp Active Directory Playbook I	T1021.006, T1110.003, T1135, T1558.003	21	yes
Register-CimProvider Execute Dll	T1218	6	other alerts
Bitsadmin Download Malicious File	T1197	7	yes
PurpleSharp PE Injection CreateRemoteThread	T1055.002	8	other alerts
Process Herpaderping Mimikatz	T1055	4	other alerts
Windows Vault Web Credentials	T1055	1	other alerts
Python HTTP Server	T1059	2	other alerts
SharpView PCRE.NET	T1059	7	yes
PowerShell HTTP Listener	T1059.001	1	other alerts
Seatbelt Group User Discovery	T1012	4	other alerts
Remote Scheduled Task Creation	T1053.005	9	yes
Remote Scheduled Task Modification	T1053.005	7	yes
Exchange ProxyLogon SSRF RCE Vuln POC	T1505.003	9	yes
Export ADFS Database Configuration Remotely	T0000	0	no
Disabling Windows Event Logging via Audit Policy Modification	T1562.002	10	other alerts
Disabling Process Command Line Logging via Registry Modification	T1562.002	6	other alerts
Modifying Security Event Log File Path via Modification of Log Configuration	T1562.002	11	other alerts
Stopping Event Log Service via Modification of Start Up Type	T1562.002	15	other alerts
Stopping Event Log Service after Stopping Depending Services	T1562.002	22	other alerts
Stopping Event Logging via Creation of MiniNt Registry Key	T1562.002	18	other alerts
Dumping NTDS.dit using NTDSutil	T1003.003	14	yes
Dumping NTDS.dit using NinjaCopy	T1003.003	19	other alerts