Model lab

Benchmark, explain and retrain the non-technical loss scoring models

Current champion

Random Forest · run-0412

Cost-sensitive learning · 500 trees, class_weight = 1:15

AUC 0.951F1 0.723Precision 0.625Recall 0.857

Model benchmark

Model	Accuracy	Precision	Recall	F1	ROC AUC	Train time	Inference (ms)	Champion
Random Forest 500 trees, class_weight = 1:15	0.960	0.625	0.857	0.723	0.951	3 min 48 s	2.0	Champion
SVM (RBF) C=4, γ=0.02, class_weight = 1:15	0.934	0.474	0.785	0.591	0.905	19 min 02 s	6.5
Neural network 3 dense layers, weighted BCE loss	0.951	0.568	0.809	0.668	0.926	6 min 31 s	1.3

ROC curves

Click a legend item to emphasize

Confusion matrix · Random Forest

16,701

90.8%

True negative

Honest, flagged honest

577

3.1%

False positive

Honest, flagged theft

161

0.9%

False negative

Theft, flagged honest

961

5.2%

True positive

Theft, flagged theft

Precision = of accounts flagged as theft, how many truly are. Drives inspection crew efficiency.

Recall = of actual theft cases, how many were caught. Drives revenue recovered.

F1 balances both; this model: precision 0.625, recall 0.857, F1 0.723.

Global feature importance

Sudden consumption drop

0.214

Billed vs metered gap

0.168

Meter tamper events

0.142

Deviation from feeder peers

0.121

Night / day usage ratio

0.104

Zero-read days (90d)

0.093

Voltage / current mismatch

0.087

Average payment delay

0.071

Retraining & drift monitoring

Run	Date	Trigger	Rows	F1	AUC	Drift	Decision
run-0412	2026-11-01	Scheduled	1,284,310	0.723	0.951	0.06	Promoted
run-0398	2026-10-01	Scheduled	1,262,004	0.709	0.947	0.09	Kept champion
run-0377	2026-09-14	Manual	1,240,876	0.714	0.946	0.11	Promoted
run-0351	2026-09-01	Scheduled	1,239,112	0.688	0.938	0.14	Rejected
run-0330	2026-08-01	Scheduled	1,201,548	0.701	0.944	0.07	Kept champion

Feature drift (PSI, last 12 weeks)

Night / day ratio drift above threshold

Consumption drop0.08

Night / day ratio0.18

Billing gap0.05

Weekly F1 (champion)0.717