Systems built to answer a question rather than to demonstrate a technique. Both are scored on threats and conditions the model never trained on, which is the harder measurement and the one that matches how either would be deployed.
A defender allocates scarce hardening and incident response across a network while a boundedly rational attacker probes it. The network changes size every episode, so the agent has to allocate sensibly whether it faces eight nodes or twenty, and the whole system is benchmarked against a Quantal Stackelberg Equilibrium solver rather than against weak baselines.
The trained defender holds up on network sizes it never saw. On held out networks of fifteen to twenty nodes it comes within a few tenths of a point of the equilibrium solver and beats the standard greedy heuristic, while tracking the solver closely on the sizes it trained on.
Fig. 1. Mean episode return over 300 episodes with shared seeds, so every policy faces identical networks. PPO-DR tracks the equilibrium solver on both trained and unseen network sizes.
-3.64
PPO-DR, held out sizes
-3.27
equilibrium solver
-3.75
greedy heuristic
Getting there took a diagnosis worth recording. Padding observations to a fixed width with an active mask is the standard pattern for variable sized problems, and on its own it does not deliver size generalisation, because nothing during training ever makes the extra action positions valid. Measuring where the policy actually placed its actions showed the effect immediately, and domain randomisation over the full size range closed it.
Fig. 2. Share of actions landing on node positions that were never active during training. Domain randomisation over the full size range lifts it from 0.08 percent to 27.7, which confirms the mechanism directly rather than inferring it from the return.
Two further studies test how much the result depends on knowing the attacker. Sweeping attacker rationality across two orders of magnitude leaves the equilibrium allocation almost unchanged, which is why the defense stays robust when that parameter is estimated wrong. A final ablation isolates how much the equilibrium solve contributes over simpler value ordered allocation, and reports the margin with a paired test rather than asserting it.
Published DGA detectors routinely report accuracy above 0.99, measured on a random shuffle where every malware family appears in both training and test. This service measures the question a security team actually has, which is whether the detector fires on a family that did not exist when the model was trained.
The evaluation is paired. Every family is held out in exactly one fold and trained in the other four, so each family is compared against itself and family difficulty cancels out. That design matters, because the naive comparison can run backwards when a fold happens to hold out easy families. Under pairing, most families transfer nearly free and the cost concentrates in a handful.
Fig. 3. Recall cost per family at a fixed 0.1 percent false positive rate. Reporting families individually rather than as a single average is what makes the concentration visible.
5.6 pts
median recall cost
0.72
median novelty gate AUC
1.03 ms
per domain at batch 100
The evaluation also surfaced a structural limit worth designing around. Dictionary based families such as suppobox, matsnu and gozi build domains by concatenating real words, so at this capacity a character model has no representation for them whether they sit in training or not. Any honest DGA detector should report dictionary families separately rather than letting easy random string families carry the average.
So the service ships a second component alongside the classifier. A novelty gate fits one Gaussian per known family in the classifier's penultimate space and scores Mahalanobis distance to the nearest centroid, needing no out of distribution data to fit it. It answers a different question, which is whether a domain sits far from every family the model knows, and it fires precisely where the classifier is blind.
Fig. 4. The two components are complementary. On suppobox the classifier has no signal while the gate separates it from known families at 0.8802, flagging 34 percent at a 5 percent false positive rate.
Scoring the gate correctly took care. Pooling every held out family in a fold into a single AUC averages genuinely novel families together with ones whose style is already well represented, and the pooled number reads as noise. Per family, the same fold ranges from 0.88 down to 0.31, and the repository reports it that way.