Changelog¶
Every release lists what a user of the package can see change. "No user-visible change" means the public API, the generated files and the numbers are the same as the previous release.
Unreleased¶
-
tools/wu26_faithful.py --tauruns a scenario at another per-snapshot weight; the records carry atautag and thewu26.faithfultable shows one row set per weight. Newtools/wu26_dqn.pytrains [WU26]'s DQN overWu26DefenseEnvin parallel sessions and tests each on fresh windows, stored aswu26.dqn(docs/wu26, "Solution 2"). -
[WU26] reproduced as the paper states it (
trust.wu26, the[wu]extra). A faithful attack and defense beside the dataset generator's: wu26_network(case, pmus): the paper's meter plan ([29] Sec. 2.2), every bus's SCADA reading its injection and the flow of each incident branch at its end, so every branch metered at both ends; PMUs read |V|, the angle and their incident branch currents; no SCADA voltmeter (newMeterModelvalue"wu26", refused by the generator'sMeterSettingsandGeneratorOptions).solve_window: eq. (12) read literally, the l1 of the window's summed attack ([37] p. 1898's l1) plustautimes the per-snapshot l1, as one IPOPT problem per window, with the boundary held, the voltage band, the goal of eqs. (24)-(25) as written ([E19]: each target atrhotimes its true flow by the window's end, the flow never falling between snapshots, no per-snapshot target; the generator's D9 ramp is unchanged), and changes below the paper's noise counted as untampered (newSupportMethodvalue"wu_l1", refused by the generator'sOverloadSettings). Settings: the validatedWu26Attack(rho,tau, the trust regiondv/da, the voltage band,max_iter; all ours where the paper is silent, [E18]).incremental_freeze: trust read incrementally (eqs. 29-30, [E1] revised): a trusted PMU keeps the offset its bus had at the snapshot before its slot.Wu26DefenseEnv: the trusted-PMU MDP on this attack, the cost the devices tampered at any snapshot (Fig. 12's count, unit "devices") or eq. (33)'s net l0 over channels (unit "channels") ([E3] revised);wu26_solution1: Solution 1's row reduction on this meter plan.TrustedPMUsDQNnow trains on anytrust.DefenseEnv(WuDefenseEnvorWu26DefenseEnv).-
tools/wu26_faithful.pyruns the paper's scenarios at its scale into the results store (wu26.faithful, querywu26.faithful); new metricreference_hits. Decisions E16 to E19 added; new named errorNoSuchBranch;WindowAttackinfdia_graph.models. -
Replication against [WU26] recorded.
docs/wu26/README.mdgains a section stating what the package implements as the paper states it, what agrees, what does not, and what the paper leaves unspecified, with every hypothesis tested rendered from the results store (querieswu.hypothesesandwu.table5). Seven scratchpad experiments are added to the store:wu.data_recipe,wu.l1_attack,wu.table5_search,wu.attacker_estimate,wu.multirate,wu.independent_samplesandwu.local_opf. Their scripts were prototypes and are not part of the package. New metrics inmodels.results.METRICS:pmu_devices,jaccard_scada,jaccard_pmu,bdd_false_alarm_pct,bdd_detection_pct,bdd_detection_matched_pct,state_dev_puandstate_dev_deg. -
Hand-written loops and formulas replaced by library calls and array operations. Generated files are byte-identical to the previous release at the same seed; every replacement is pinned to the code it replaced (
tests/test_package_equivalence.py). - Generation: the temporal-feature pass, the meter-noise draw (one vector draw per layer in the
same order as the per-channel draws), the meter masks, the per-bus scale factors, the case
limits (
np.add.at) and the swing scale run as array operations instead of Python loops. - Metrics: new
formulas.micro_prf,formulas.sample_f1,formulas.strict_accuracyandformulas.metrics.perbus_counts_at(every candidate threshold at once); the localizers' and the federated localizer's threshold searches and score blocks use them (checked against scikit-learn in the tests). formulas.leveragetakes the hat-matrix diagonal without forming the matrix;guarded_inverseinverts through its Cholesky factor; the SEgeoscore usesscipy.stats.gmean; the DQN replay buffer is acollections.deque; both DQN selectors build their network withtrust.dqn.q_network;trust.secured_copyrestores secured meters in one indexing step; the dataset's derived series admittance usesformulas.series_admittance;profilesremoves shunt draws withnp.subtract.atand the CSV export concatenates once.- Docstrings that promised bit-for-bit reproduction against the removed frozen suite now name what pins the behaviour.
formulas.ac_jacobianis built from pandapower'sdSbus_dVanddSbr_dV(the [se] extra) instead of by hand; its signature is now(vm, theta, Ybus, Yf, Yt, branch, lut), taking the ppc branch matrix andYtin place offrom_busandn_ppc. It equals the previous kernel to within floating-point rounding on all eight systems, and the estimators' fitted Jacobian with it.-
formulas.subnetworkandformulas.hop_distanceuse scipy's unweighted shortest path through the newformulas.hop_distancesandformulas.edge_adjacency; the interiors, boundaries and hop counts equal the breadth-first search they replace. -
The attack search is a plain candidate loop; the certifier is gone.
MinimizeMixin.min_tampersolves the area, then the goal's buses grown one area bus at a time, smallest first, at mostmin_budgetcandidates, and keeps the cheapest; it stops early only at one device on one channel, the least an attack tampers. Each candidate's false state is the existing least-norm Gauss-Newton solve of the AC measurement equations (local_flow_solve,local_ac_solve). Replacing it withscipy.optimize.least_squares(trust-region reflective, or Levenberg-Marquardt with least-norm rows) was tried and kept out: on the pinned IEEE-14 searches it lost an attack the least-norm solve finds, found dearer ones on others and ran slower everywhere (experimentsearch.solver_compare, rendered in docs/wu26/README.md).- Removed: the convex-relaxation certifier (
engine.attacks.certify,relax_cuts,formulas.relax, the[certify]extra with cvxpy and pyscipopt,Certificate,BoundClaim,CertifyOptions,CertifiableLimits,CutFamily,CertifyVerdict,NoOperatingLimits, thecertify.ablationresults); the search's proof bookkeeping (MinimizerResult.proven,lower_bound,unsolved, the goal-forced lower bound and the files'episodes/min_proven,min_lower_boundandmin_unsolved); the support per trust slot (TrustSchedule.per_slot,WuDefenseConfig.per_slot,MinimizerResult.plan, decision E13 now "one support held"), which never changed an answer in any run.MinimizerResultis (support, devices, channels, evaluated) andgoal_statereturns the false state alone. [WU26] proves no minimum either. - The pinned IEEE-14 searches find the same supports, devices and channels, and a generated timeline's
every data layer is byte-identical; only
episodes/min_evaluateddrops where anAtsearch stops at one device on one channel. The search's attack vectors are float64 throughout (float32 before the comparison with noise; no answer changed). - [WU26]'s attack area [D18]:
FrameKnobs.areatakes the paper's area bus by bus (OverloadMixin.wu26_area: IEEE-14's every bus but the slack, p. 659; IEEE-118's Fig. 9 network,WU26_ATTACK_AREA, which nothing used before), andSearchSettings.area_rule/FrameKnobs.area_rule(new choiceAreaRule) picks, with "rules", the region Section III-A's principles favour (AreaMixin.rule_area: rules 1 and 2 checked, 3 and 4 a score of ours with a size charge, new modelAreaScore); "hops" (the default) is unchanged. - Delta ratings [E14]:
OverloadSettings(rating_source="delta", rating_delta=0.10)rates each target line at its true flow at the window's last snapshot plusrating_deltaper unit (ours, a scale matched to [WU26] Fig. 4's attack magnitudes; newRatingSource.DELTA,OverloadMixin.window_ratings). The generation defaults are unchanged. An Am episode's recorded ratings are its goal's end targets. -
tools/wu26_harness.pytakes--ratings("+0.10" by default, or margins such as "1.1") and runs on the paper's area; its rerun (experimentwu26.reproduction) and the MINLP prototype's experiments (minlp.prototype,minlp.region_reading,minlp.anchoring,minlp.rating_delta,minlp.cap_trust_grid) are rendered in docs/wu26/README.md: IEEE-118 comes closest to the paper's band (the defense adds devices on its scale, the measurement count rises less), IEEE-14 does not reproduce Table II, and why. New metricfeasible;lower_boundis now a solver's dual bound. -
Generation in stages, typed settings, the paper mapped to the code.
- The timeline writer is a pipeline of stages with typed hand-offs in the new package
fdia_graph.generation(the old module'sgenerate,as_v_firstand_load_statesstay where they were):plan(splits, episode counts and onsets),design(the overload Am of each split),emit(the frames in time order) andwrite(the datasets, episodes, attributes andwrite_temporal_layers);timeline.generate_timelineorchestrates them.timeline's old private names whose signature did not change (_split_bounds,_place_split,_TimelineBuffers,_block_scale, ...) still resolve, with a DeprecationWarning naming their new home, until 0.22;_walk_split,_finish_timelineand_run_episodechanged shape and are gone. The engine's two branch points are strategy objects instead of base classes (decision B4): the support of an attack window comes fromSearchSupport(the fewest-tamper search) orRrefSupport([WU26]'s row reduction, formerlyRrefMixin; itsarea_jacobianandladdernow take the generator through the strategy), chosen byMinimizeMixin.support_strategy; an episode's design comes fromRampDesigner(At) orOverloadDesigner(Am), chosen byEpisodeDesignMixin.designer.OverloadMixinno longer inherits the row reduction, andam_overload_designtakes an optionalrnginstead ofprev(a flow goal has no stealth bound, [D11]). - The overload Am designs run in parallel (
workers=N): a split's Am episodes are designed before it is walked, each drawing its line order from a stream keyed by the seed, the split, the episode and the move, so the file is byte-identical for any number of workers (experimentgeneration.parallel_design,tools/experiments/design_speed.py). A script passingworkersabove 1 on Windows must start underif __name__ == "__main__":. Because the Am draws no longer come from the walk's stream, a timeline with Am differs from one written by the previous version at the same seed; At-only timelines, the benign frames, the clean truth and the splits are unchanged byte for byte. - An infeasible Am moves anywhere in its split: earlier or later, uniform among every free onset (a ramp At, whose design reads the frame before it, still moves later). Before, a move could only go later, which gave up an Am episode that had no later room left.
- Typed settings:
TimelineSettingsnestsRampSettings,SearchSettings,OverloadSettings,MeterSettingsandSplitSettings;generate_timeline(system, states, settings=None, seed, out, **knobs)andgenerate(..., settings=None, **knobs)take it and every old flat keyword over it (TimelineSettings.of), and refuse an unknown one. A dict foram_attackorredundancystill works and raises aDeprecationWarning: dicts go in 0.22; pass anOverloadSettingsor aMeterSettings. A newworkersknob. The file records the settings as JSON (attributesettings) with their hash (settings_hash); the registry records them as nested values.TimelineKnobsstays for one minor version. - The paper-to-code map: one tag form for [WU26]'s equations (
[WU26 eq. 28]) and our decisions ([D9],[E13]), the registriesschema.WU26_EQUATIONSandschema.DECISIONS(the new modelDecision), andtools/equation_map.py, which writesdocs/reference/DECISIONS.md, the "[WU26] equations to code" section ofFORMULAS.mdand the module table ofCONCEPTS_TO_CODE.mdand, with--checkin the pre-review and CI, fails on a stale doc, an unknown or old-form tag, or an equation with no implementing function or no test. -
A rerun with an arm skipped leaves no stale number:
Run(..., replaces=True)(andStore.write(..., replaces=(system,))) drops the experiment's earlier records on the run's system, even when the rerun wrote none there; the SE, localization and federated docs harnesses pass it. -
Results as data (
fdia_graph.results, the results storeresults/): measured numbers are records with their provenance, not text. New modelsRecord,Provenance,RunRecords,MetricandLeaf(fdia_graph.models.results, with the metric registryMETRICS), each checked on construction;Run(a harness's run, written on exit with the SDK version, git commit, data release, seed and settings hash; a failed run writes nothing),Store(one tidy CSV per experiment plusruns.csv;query,latest,one, which raises the new named errorNoSuchResult), renderers (table,cell,pivot,figure_datafor a figure's CSV sidecar) and results blocks in markdown (<!-- results: ... -->), filled bytools/results_docs.py --writeand checked by its--checkin the pre-review and CI. The docs harnesses (docs/se,docs/localization,docs/federated,docs/trust),tools/wu26_harness.pyandtools/bench.pywrite throughRun; the harness JSONs are migrated into the store and removed, and the guides, READMEs, plan tables and this section cite experiments (results/README.md).tools/experiments/release_stats.pyrecords a data release's statistics.tools/wu26_harness.pynow gives IEEE-118 one configuration step per snapshot (10 steps, not its 11 PMUs). -
[WU26] reproduction harness (
tools/wu26_harness.py,docs/wu26/README.md): 1-minute attack windows rebuilt from the 5-minute pool by AC power flow, Solution 1 and the DQN played on each window's trusted-PMU MDP, and Table II, the trusted orders, the decision times and Figs. 11-12 written with the CSV each figure is drawn from. Tables III-V and Fig. 13 are not reproduced (see the README). - Split first, then attack each split (generation).
generate_timelinecuts the timeline chronologically into train, val and test before any episode is placed (split=, default 60/20/20, now validated by the new modelSplitSettings), so every episode lies inside one split and each split's numbers are known in advance. Per split, the whole number of episodes whose frames come closest toattacked_fracof its frames is shared between the families by largest remainder and placed at uniform onsets inside the split, never overlapping and never cut (no episode is clipped any more, so every At episode is exactlyramp_lenframes, 60 by default). A placement that jams is retried whole 10 times, then one episode is dropped (RuntimeWarning). An episode with no feasible design at its onset (no admissible ramp, or no line pair an overload reaches stealthily) moves to a later free onset of its split, up to 20 times, instead of leaving its frames benign; past that it is given up (RuntimeWarning). New file attributes record what was asked and got:split_frac,split_sizes,split_attacked_frac, and per split (rows) and family (columns,placed_families)episodes_requested,episodes_built,episode_redraws,episodes_dropped,episode_shortfall.fallback_benignnow counts only frames inside a built episode whose scan could not be built. Gone:timeline._frame_splitand its boundary-moving,_draw_episodes,_place_episodes. On a 72,000-frame timeline with 60-frame episodes every split is exactly half attacked (360, 120 and 120 episodes). - Every frame's meter jitter comes from its own stream keyed by (seed, timestep)
(
MeasurementMixin._jitter_stream, set by the timeline walk;jitter_keyedattribute). The benign frames, the meter plan and the split column of timelines on one seed and pool that place different families (("Am",),("At", "Am"),("At",), the three release variants) are therefore byte-identical, and every emission of frame t draws the same noise as its benign twin. Emission outside a timeline (no key) draws from the generator's own stream as before. Generated data changes for the same seed. - Fix: an overload
Amwindow is snapshots kappa+1 ... kappa+T after the untouched reference kappa [WU26, eq. 25]. The goal's share at the window's k-th snapshot is k/T (was (k-1)/(T-1)), so the first labelled snapshot already carries 1/T of the way to the rating; before, its goal was the true flow and that frame, labelledAm, tampered nothing. A new test checks that every frame labelled attacked tampers at least one meter. The fewest-tamper optima of the pinned IEEE-14 searches and the [WU26] scenario counts are unchanged; two pinned overload searches fail one candidate solve more or fewer (tests/test_search_speed.py). -
Review process: one automated review per pull request. The repository ruleset no longer has Copilot review every push or drafts (
review_on_pushandreview_draft_pull_requestsoff): a pull request is reviewed once, when it is opened or marked ready.tools/pr.py mergenow requires CI green on the head, one review from every required bot on any commit of the pull request, and a reply to every bot finding (inline in its thread; the findings a review lists in its body, a conversation comment after that review) (statusandwaitshowreviewed_commitsandunanswered).tools/pr.py createopens a draft and the newtools/pr.py readymarks it ready (which asks for the one review). CONTRIBUTING.md, the review checklist and the pull-request flow diagram describe the draft, ready, one-review, one-fix-push order. -
Removed: single-snapshot generation, the v0.8.3 recipe and the frozen suite. The generator makes the multi-snapshot families
AtandAmonly, on the hybrid meters; every published data release (v0.7.1 to v0.8.3) still loads read-only, with its family names (Aq,Ad,As,Ar,Al), aliases, theheldout=protocol and per-family scores. The removed code is at commitaa77d0b; fdia-graph 0.20 is the last version that generates the removed families. - Generation:
families=acceptsAtandAm(newmodels.inputs.GeneratedFamilies; an older family raises before any work). Gone: the in-placeAd/As/Ar(engine/attacks/corrupt.py), the load redistribution behindAland the v0.8.3Am(engine/attacks/redistribution.py),single_shot_design,am_design/am_step, theAq/Alframes, and the knobsattack_intensity,am_rate,am_direction,corrupt_len,replay_tauandam_attack="redistribution"(withAmDirection,LEGACY_FAMILIES,DEPRECATED_FOR_GENERATION,ONE_FRAME_FAMILIES,generation.NOISE_FLOORand the modelsBand,TamperTarget,Redistribution,AmDesign).FrameKnobskeeps the fields new generation uses, all with defaults.generate_streamkeeps its entry point with new generation's families and defaults; its arguments afterfamiliesare keyword-only, so a 0.20 positional call raises instead of shifting. - The v0.8.3 meter model:
meter_modelaccepts"hybrid"only, now also the engine's default. Files written with the old meters load unchanged. - Dead or retired code: the full pandapower re-solve (
PhysicsMixin.resolve_states,solve,_pin_generation,MeasurementMixin.emit,state_from_net,ResolvedPool),FdiaGenerator.centrality_probs,nlandload_genP, the N-1 line-outage screen (fg.line_outage_candidates,FdiaGenerator(outage=),LineCandidate,OutageRef,GridIslanded),fg.pyg_streamandfg.torch_windows(torch_data.py;ds.windowsandfg.load(..., format="pyg")replace them), the exportersds.to_numpy/to_torch/to_tf/to_pandas(ds.export(format=...)),dataset.check_split/check_units/check_orderandfederated.check_partition(the models check these),SplitFractions,StreamSystem, and the privateSEBase._normal_matricesand_cond.load_streamandstreams.windowsstay: they read the v0.7 stream files. Theds.edge_*aliases stay (edge_gsandedge_bsare requested names). - Tests: the frozen suite (tests/test_frozen.py, tests/frozen/, tools/freeze_reference.py, the
strict mode of
tools/prereview.py) is gone; the pinned search answers (tests/test_search_speed.py), [WU26]'s scenarios and the unit tests guard the generated data. The shared test timeline is new generation's (AtandAm); old releases are tested on checked-in files (tests/test_old_releases.py,tests/data/README.md). - Docs: the finished plans (data models, field groups, one dataset, readability, reference study, restructure, validation), FUTURE_DATASETS.md, the v0.8.0 to v0.8.3 release build scripts and five unlinked figures.
- Fixed:
load_streamat v0.7.1 and v0.7.2 reads the stream files again. Those files carry the frames only, and the release's graph sidecar (graph_ieee{N}.npz:edge_index,edge_attr,node_m,edge_m) that 0.18 stopped attaching is attached again; the tests read a slice of the published IEEE-118 stream with its sidecar, and one timeline of each of v0.8.0, v0.8.1 and v0.8.3 written by the SDK that built it. Streamcarries the PMU branch currents of a hybrid-meter file (pmu_i,pmu_i_benign, the static maskpmu_i_m), sogenerate_streamandstream_ofno longer drop them. The benchmark table records the fixture recipe per row andtools/bench.py --checkcompares only rows of the same machine and recipe.- Known behaviour made visible by the new test timeline, unchanged here: an overload
Amwhose window has no stealthy design falls back to benign for the whole episode (fallback_benign), and its first snapshot can tamper nothing, its drift-free goal there being the true flow (D9). - [WU26]'s Solution 1, the trusted PMUs by row reduction (Sec. IV-D1, p. 657 steps 1-5;
docs/plans/WU_DEFENSE_PLAN.md, PR C):
trust.TrustedPMUs(env).fit()on aWuDefenseEnvtrusts one PMU per configuration step, a PMU reading the sparsest attack the column-exchange reduction finds on the attack area's Jacobian at the step's snapshot (the trusted buses left out), and records the order, the rewards, the costs and the reductions' own time. Ours: the chase is restricted to attacks that move a target line and that a PMU on offer reads, the PMU reading most of the attack's channels is trusted, and the first PMU on offer when no open attack touches one.RrefMixin.area_jacobiannow also names the device of every row (channel_devices). On [WU26]'s IEEE-14 scenarios the order differs from the paper's Fig. 6 while the final trusted set and the defended cost match (experimentwu26.reproduction, docs/wu26/README.md). -
[WU26]'s Solution 2, the trusted PMUs by a deep Q-network trained with Algorithm 1 (Sec. IV-D2; docs/plans/WU_DEFENSE_PLAN.md, PR D):
trust.TrustedPMUsDQN(envs, WuDqnConfig()).fit()over one or moreWuDefenseEnvwindows,order(env)the trained policy's greedy sequence (eq. 38). The loop is Algorithm 1's: epsilon-greedy actions (eq. 34), line 8's break without a stored transition, one minibatch update per iteration by the squared loss to r + gamma max Q_target (eqs. 35-37), the target network copied every 20 iterations; new validatedWuDqnConfigwith the stated hyperparameters as defaults (gamma 0.9, Adam at 0.005, buffer 2,500, minibatch 25, target every 20, epsilon e^(-0.002 ep), 250 episodes). Ours: the network (two hidden layers of 128 ReLU units, torch), masking PMUs already trusted, terminal transitions bootstrapping nothing, and the state scaled by its largest magnitude over the training environments' first states. New input modelSameDefense(the training environments offer as many PMUs and take as many steps). -
[WU26]'s trusted-PMU configuration as a Markov decision process (docs/plans/WU_DEFENSE_PLAN.md, PR B):
trust.WuDefenseEnv(g, states, goal, k, WuDefenseConfig(pmus, slots))on one overload window, withreset,stepandvalid. An action trusts one more PMU at the next configuration step (Sec. IV-D2, "one PMU per step"); the reward is the rise in the fewest-tamper search's cost under the schedule so far (eq. 33 per step, eqs. 28-32), in measurements by default or devices (WuDefenseConfig.unit, new choiceCostUnit, the plan's E3); the state is what Fig. 1 lists, the node and flow readings at the step's snapshot, the target lines' load rate and the trusted mask. Every schedule's answer is cached (solvescounts the searches), since a training run revisits the same schedules. Ours, where the paper is silent: the readings are those of the attack under the schedule so far without noise, an attack the schedule makes infeasible costs every attackable channel or device, and Algorithm 1's line 8 (a defended cost below the undefended one ends the episode without a reward) is kept and counted inbreaks. The search is the cost oracle rather than the row reduction (support_method="rref"), which matched Table II and Fig. 12 worse (the plan's section 3). The search minimizes the unit the reward counts: newFrameKnobs.objective("devices", the default and what generation uses, or "channels", which puts the tampered measurements first as eqs. 28 and 33 count them; the device bound then no longer prunes). The environment refuses a step outside the window (WindowSlots) and an action that is not a PMU still on offer (ChosenAction); its state reads only metered channels (the load rate from the full flows), and the infeasible cost counts the plan's metered channels or the devices that hold one. New validated modelsWuDefenseConfig,WindowSlotsandChosenAction. -
[WU26]'s own attack construction as the overload attack's support method (
OverloadSettings.support_method, "search" by default or "rref"; new choiceSupportMethod,FrameKnobs.support_method, file attributesupport_method). The paper states it twice: the attacker "can utilize the transpose of the measurement matrix h, applying multiple elementary row transformations and column exchanges to find the optimal attack vector solution [25]" (p. 655, [YAN17]), and Solution 1's steps 1-5 (p. 657). Newformulas.trust.sparsest_rows(the reduction, the column exchanges with their tracking order, the rows mapped back) andengine.attacks.rref.RrefMixin.rref_support: at each snapshot the Jacobian of the attack area's attackable channels (scaled by [WU26]'s noise, a trusted PMU's bus left out, eqs. 27 and 29) with respect to |V| and angle, for each target line the sparsest row whose state change moves its flow, the window's support the union over the snapshots (eq. 28), the magnitudes from the AC local flow solve so the goal, the limits and the D16 bounds hold. Ours: the chase starts from the transpose's own rows (the single-variable attacks), it chases the sparsest goal-moving row rather than the sparsest outright, and a support that cannot reach the goal grows by the next sparsest rows. On [WU26]'s scenarios the search matches Table II better: at k = 1.1 the trust schedule raises the search's devices 20% and 29% (the paper 24% to 35%) and the reduction's -14% and +13%; on IEEE-118 the search's devices rise 20% (the paper 16.4% mean) and the reduction's fall. The reduction is 0.5 to 0.8 s per 20-snapshot IEEE-14 window against the exhaustive search's 16 to 33 s, and 2.4 to 3.2 s per 10-snapshot IEEE-118 window against 2.8 to 9.6 s at budget 256. [WU26]'s reference now lists the paper's authors and pages (S. Wu, Q. Wang, J. Hu, Y. Ye, Y. Tang; pp. 651-665). -
[WU26]'s trusted-PMU defense as a constraint on the fewest-tamper search (docs/plans/WU_DEFENSE_PLAN.md, PR A):
min_tamper(..., trust=TrustSchedule(buses, slots)). A PMU trusted at slot s keeps its bus's |V| and angle true at every snapshot t >= s (eqs. 26-32: the deviation is zero on its two secure rows, and trust accumulates, eqs. 30-31); its branch currents stay untrusted, as eq. (27) leaves them in the nonsecure set. New modelTrustSchedule(validated: one slot per bus, each PMU once, non-negative indices). Without a schedule the search and its answers are unchanged. Measured on [WU26]'s IEEE-14 scenarios with the PMUs trusted at the paper's snapshots 2, 4, 6 and 8: experimentwu26.prototype, rendered in docs/plans/WU_DEFENSE_PLAN.md. The paper's trust order and the swapped one give the same answer. -
Development only, no user-visible change: the test suite runs in parallel. pytest-xdist joins the
testanddevextras, CI runs the three suites with-n auto, andtools/prereview.pydoes so (at most 8 workers) when the interpreter has it. The test extra's pip cache is kept between CI runs, and a pull request that changes only Markdown ordocs/(not the data dictionary, which a test checks) skips the suites' install and run steps, the jobs still reporting success. The malformed-input tests' ids no longer carry memory addresses, which differed between workers. -
The fewest-tamper search (
MinimizeMixin.min_tamper) holds BLAS to one thread while it runs and restores the caller's setting after, through threadpoolctl (added to thegenerateandallextras; without it the search runs on whatever BLAS is set to). Its products are too small for threads to pay: on a many-core machine starting them cost more than the arithmetic. The limit is process-wide: while any search runs, other BLAS work in the process is on one thread too, and overlapping searches share one limit, so the caller's setting comes back when the last of them leaves. No knob is added. Measured on the same fixed episodes as the entry below, before and after side by side: median seconds per episode in docs/reference/BENCHMARKS.md (experimentsearch.speed,pr=164). With the default threads the search now runs as fast as it did with the whole process on one thread. Every episode, in all four runs, returns the same target lines, support, device and channel counts, proof, candidates solved and failed solves, and the pinned searches of tests/test_search_speed.py are unchanged. -
The fewest-tamper search is faster and finds the same supports, bit for bit. The flow solve keeps the full Ybus products of the released solve, since a search's answer can turn on the last bit of a solve near its convergence limit (a product over the region's rows alone changed the answer of 4 of 30 pinned IEEE-14 searches), and does less around them.
- Each voltage vector tried costs one product Ybus V, which the residual and, for an accepted
vector, the next iteration's Jacobian share (
local_flow_solve,local_ac_solve); the solve's slices of Ybus and Yf are cut once per support and reused by the active set's re-solves (local_flow_solve(blocks=...),formulas.network._flow_blocks). - The bus adjacency is cached (
AreaMixin._boundary) rather than rebuilt from the edge list for every candidate, and the generator and free-injection buses are built once. - Each snapshot's true flows and currents are computed once per window, and the attack vector is evaluated on the support's branches only; every other branch reads the same on both sides.
- The tamper count is a union of boolean masks instead of Python sets.
Measured on a fixed set of episodes, before and after side by side: median seconds per episode in
docs/reference/BENCHMARKS.md (experiment search.speed, pr=163). Every episode, in both thread
settings, returns the same target lines, support, device and channel counts, proof, candidates
solved and failed solves as before, and the flow solves on IEEE-14 and 118 (with and without the
load cap) return the same false states bit for bit.
Exhaustive searches on fixed IEEE-14 windows, with and without the load cap and with two-line goals,
are pinned to their earlier answers (tests/test_search_speed.py).
SEBase.estimatebuilds the measurement vectors, thepmu_pseudoslots included, onechunkat a time, and reads the pseudo values alone (newformulas.estimation.pmu_pseudo_phasors); the propagated covariances are computed only for the measured calibration's scans infit. On a 72k-frame IEEE-300 timeline this drops several whole-split float64 tensors (the [n, N, 2, 2] covariance alone about 691 MB). The estimates are unchanged, bit for bit.-
An overload episode drives two lines at once by default, as [WU26]'s case studies do (docs/plans/WU_MSFDIA_PLAN.md, D17):
OverloadSettings.n_lines(2, or 1), passed asam_attack={"n_lines": 1}; the pair is drawn inside one attack area so one held support reaches both, at mostAM_LINE_TRIESpairs are tried and an episode with none stays benign. The file recordsn_lines, andepisodes/am_*holds one row per target line with the newam_target_mva(the goal at the window's end);AmOverloadDesign.ratingbecameratings, andoverload_stepreturns the flow reached on every target line. New tablesWU26_SCENARIOS,WU26_PMUS(IEEE-118's from the paper's Fig. 9) andWU26_ATTACK_AREA, withOverloadMixin.wu26_branchandwu26_buses; new teststests/test_wu_scenarios.py. Measured: experimentgeneration.am_overload, rendered in docs/guides/generation.md. -
The overload attack bounds the edge of its support (docs/plans/WU_MSFDIA_PLAN.md, D16): the generator limits (22)-(23) apply to every generator whose reported output the attack changes, the support's edge included (pinned in the solve, P and Q apart, and checked by
within_limitsover the support and its edge), and every load bus it moves, in the support or on its edge, shows an active change of at mostload_captimes its true load [YUA11] (newOverloadSettings.load_cap, 0.5 by default, in (0, 1], passed asam_attack={"load_cap": ...}and recorded as the file attributeload_cap).local_flow_solveholds injections at edge buses as well (formulas.network._injection_jacobian);FalseStateMixin.touched_buses;FrameKnobs.load_cap. -
The overload attack's line ratings default to the operating pool (docs/plans/WU_MSFDIA_PLAN.md, D15): S_max of each branch is 1.25 times its peak true apparent flow over the pool the timeline walks, so
Amruns on every system of the ladder. The PGLib-OPF ratings stay available,generate_timeline(am_attack={"rating_source": "pglib"})(IEEE-14, 118 and 300;NoLineRatingselsewhere). New modelOverloadSettings(rating_source"pool" or "pglib",rating_margin> 1), passed as a dict througham_attack; new choiceRatingSource;OverloadMixin.use_line_ratings,MeasurementMixin.all_flows_from_states; new file attributesrating_sourceandrating_marginwhen the overload attack runs. Measured: experimentgeneration.am_overload, rendered in docs/guides/generation.md. -
Review fixes to the hybrid meters and the overload attack: a generator pinned at one limit keeps its other component free (the active set pins P and Q apart); an overload frame's magnitude at a generator bus is its apparent change against the true generator output (at a load bus, the active change against the true load;
OverloadMixin.pretended_change); a PMU current channel's systematic bias scales with its end phasor's magnitude, the scale ofcurrent_sigma(formulas.noise.biased_current,current_magnitude), which changes hybrid emission only. -
Amfollows [WU26]'s model, eqs. (12)-(25), exactly (docs/plans/WU_MSFDIA_PLAN.md, D11): no bound on how far the attack moves between snapshots, which the paper does not have; its noise (0.03 pu SCADA, 0.01 pu PMU, currents included) only decides which changes the tamper count ignores. The bound was what made the overload attack infeasible on IEEE-14.Atkeeps its rated-accuracy bound, andstealth_scaleis nowAt's alone. The rest of theAmpath follows the paper too (D14, below). Am's generators are free (D14): [WU26, eqs. 13-14] let every injection be tampered and (22)-(23) bound only the generator output, so a generator bus in the support has free P and Q injection inside its limits, like an attackable load (FalseStateMixin.free_injection_buses,generator_buses); a support needs an end bus of each goal line and one free injection. The flow solve enforces the limits (21)-(23) by an active set (a generator pinned at the limit it would pass, a bus voltage held at its limit, the least-norm solve repeated;formulas.network.local_flow_solve(vm_fixed=...)).FlowGoaltakes one or more lines (FlowGoal.more,lines,targets_at;overload_goal(window, line, *more),solve_flow_local(line=[...])): one held support, each line to its own rating. Generated episodes drive two lines by default since D17 (OverloadSettings.n_lines, one line when set to 1). A zero-injection bus stays held at zero, a rule of ours. The labels of an overload frame are the free-injection buses of the support. On [WU26]'s IEEE-14 metering and two-line scenarios with the PGLib-OPF ratings (the paper does not state its limits) the attack is feasible and tampers more devices than the paper's, with changes of several pu, since the ratings are 3 to 19 times the true flows (the plan's D14 table).meter_modelis its own knob (D12), a field of the new meter-plan modelMeterSettingsthatgenerate_timeline(redundancy=...)takes beside the coverage fractions (no new parameter on the entry point):"hybrid"by default, and the v0.8.3 recipe (the frozen test timeline, the v0.8.3 build script,generate_stream) pins"v083".-
New timeline field
prev_pmu_i(D13): the previous emitted frame's PMU branch currents, offered byexporton a hybrid-meter file besideprev_node_xandprev_edge_x.JacobianWeightingtakespmu_pseudo=Trueand reads it. -
The hybrid meter model (docs/plans/WU_MSFDIA_PLAN.md, decision D10), what new generation's meters read: a SCADA voltmeter reads
|V|only and the angle is a PMU channel, and every PMU reads the current phasor of each in-service branch at its bus [WU26, eqs. 19-20], the real and imaginary part at each end, per unit on the base current (formulas.network.branch_currents). Their noise is IEEE C37.118.1's 1% total vector error taken as three standard deviations of the current's magnitude plus a 1e-5 pu floor (formulas.noise.current_sigma,PMU_CURRENT_CLASS), one rule the emitter (with the jitter part) and the fewest-tamper search (with the whole class) share [C37118]."hybrid"is new generation's default; the v0.8.3 recipe passesredundancy={"meter_model": "v083"}(D12);FdiaGenerator's own default stays"v083". A hybrid file gainsdata/pmu_i,data/pmu_i_m,benign/pmu_i_benignandattack/pmu_i_tamperand the attributesmeter_model,current_featandcurrent_units; a v0.8.3-meter file has none of them and loads unchanged. Records, batches andexportcarrypmu_i,pmu_i_mandpmu_i_benignwhen the file has them, in per unit on every view (new capabilitypmu_currents). The stealthy families writeh(x^a) - h(x)on the current channels too; the fewest-tamper search counts a tampered current in the PMU of the bus at its end, and bounds its step by the PMU accuracy class forAt(D7); forAmit counts a current beyond 0.01 pu (D8, D11,formulas.noise.paper_current_sigma). Under either meter model the search charges an angle channel only at a PMU bus. New optionpmu_pseudo(off by default) onWLS,AdaptiveWeighting,ResidualRemoval,SubspacePrior,GatedPriorandJacobianWeighting: the pseudo voltage phasors of [WU26, eq. (3)] at the far end of every PMU-metered branch fill the|V|and angle slots no meter reads, weighted in the measured calibration by their first-order propagated variance (formulas.estimation.pmu_pseudo_links,pmu_pseudo_voltages). New modelsMeterModel,CurrentColumns,CurrentIndex,CURRENT,PmuCurrentFields,PseudoLinksandPseudoVoltages;AttackVectoris a named tuple with an optionalcurrent. The v0.8.3 recipe and the strict frozen suite are bit-exact. - The documentation states what the stealthy families take from [WU26]: its constraints (13)-(18)
and (21)-(23) (the SCADA measurements, the PMU voltage magnitudes and angles, and the operating
limits; and, in new generation, whose PMUs read branch currents, the current phasors (19)-(20) as well). New generation also solves
its objective, eq. (12), for
AtandAm, andAmis its overload attack (below); the released files'Aq,AlandAmdrew their targets at random and drove no line to its limit. - New generation makes the multi-snapshot families of [WU26],
AtandAm, both on the fewest-tamper search (docs/plans/WU_MSFDIA_PLAN.md, decisions D2 and D6):generate_timelineandfg.generatedefault tofamilies=("At", "Am"),min_tamper=Trueandam_attack="overload".Amis the overload attack of [WU26, eqs. 24-25]: a rated branch whose flow is metered and whose true flow stays below its rating over the window is driven, snapshot by snapshot, until the flow the tampered measurements carry before noise reaches the rating, on the support that tampers the fewest devices. An episode tries at most eight eligible branches in a random order (each try is a full search), a bounded heuristic: when none of them admits an attack the episode stays benign and is counted. Each snapshot's false state frees the attackable loads of the support and holds every other bus's injection, the least-norm voltage change that meets the flow (formulas.network.local_flow_solve,FalseStateMixin.solve_flow_local). The ratings are therate_aof the PGLib-OPF v23.07 versions of IEEE-14, 118 and 300 (CC BY 4.0), stored infdia_graph.ratingsand matched to the branches by their end buses (formulas.attacks.branch_ratings,OverloadMixin.line_ratings); pandapower rates every branch 9,900 MVA, the "no limit" placeholder. Other cases refuseAmwith the new named errorNoLineRatings. Each overload episode writes its branch, rating, reached noiseless flow and emitted flow underepisodes/(am_*); new modelsFlowGoalandAmOverloadDesign; new knobstealth_scale(a multiplier on the stealth bound, 1 by default;At's alone since D11).Am's tamper count (and, until D11, its stealth bound) use [WU26]'s own case-study noise, 0.03 pu on SCADA channels and 0.01 pu on PMU channels in the stored units (formulas.noise.WU26_NOISE,paper_sigma; the plan's D8), whileAtkeeps the meters' rated accuracy (D7). Snapshot t's goal is the true flow plus a linear share of what separates the window's last true flow from the rating (D9), so the attack rides on the load's own drift instead of cancelling it and reaches the rating at the last snapshot. GeneratingAq,Ad,As,ArorAlwarns (DeprecationWarning) and is refused from 0.22; released files holding them load unchanged.timeline.LEGACY_FAMILIESwitham_attack="redistribution"andmin_tamper=Falsereproduces data release v0.8.3 (its build script and the frozen test timeline use it; the strict frozen suite is bit-exact). Under the meters' rated accuracy (D7) no overload window was stealthy, since moving one line's flow moves the injections and flows around its ends by several times that change, beyond the rated accuracy of the small loads there. The reported noiseless flow reaches the rating exactly. - The fewest-tamper search of [WU26, eq. 12], behind
generate_timeline(min_tamper=True)(on for new generation;min_tamper=Falsewalks the released files' recipe). Each At episode is held on the support (the buses its false state moves) that tampers the fewest devices over the episode, a device being one SCADA terminal or one PMU per bus and a change under a meter's accuracy-class sigma not counted. A support is held for the whole episode and must solve at every snapshot inside the operating limits and move no metered channel by more than its accuracy-class sigma from one snapshot to the next, the first snapshot measured from the frame before the episode (its attack vector when that frame was attacked, since episodes may be adjacent). The search solves candidate supports in the attacker's area smallest first, each closed over the zero-injection buses of its boundary, starting from the region the episode was accepted on; a local solve that does not converge proves nothing, so those candidates are counted (min_unsolved) and the optimum over held supports is claimed only among converged solves, when it exhausts the area or reaches the devices the goal forces above their accuracy-class sigma, settles aftermin_budgetcandidates otherwise, and treats a support moving no device beyond its sigma as no attack, and recordsmin_devices = -1when no held support meets every constraint (the episode then runs on its region). Each search is written underepisodes/(min_support_*,min_devices,min_unsolved,min_channels,min_proven,min_evaluated,min_lower_bound). New formulasformulas.noise.jitter_sigma(the emitter's per-scan noise rule, emission only) andaccuracy_sigma(a meter's rated accuracy at a reading, the measured calibration's rule throughaccuracy_class_sigma, what the search counts against),formulas.attacks.tampered_channelsandtampered_devices; new modelsLoadGoalandMinimizerResult;MeasurementMixin.meter_masksgives the masks without a random draw. - Every attack is built in one place,
engine.attacks, a package whose modules compose the generator'sAttackMixin:area(the attacker's area),false_state(the local solve, the operating limits, the attack vector and the tamper set),redistribution(theAlandAmredistribution),stealthy(theAq/At/Al/Amframes),episodes(what an episode attacks, drawn at onset, typed asRampDesignandAmDesign) andcorrupt(Ad/As/Arand the replay buffer). This code was spread overengine/records.py,engine/physics.py,engine/attacks.pyandtimeline.py;recordsnow emits a benign scan and hands an attacked family to the mixin, andtimelinedecides when and where each episode runs. No user-visible change: the generated files are bit for bit the same. The public names keep their old import paths; a moved private name (such asengine.records._stealthy_frameortimeline._AmShape) still imports from its old module with aDeprecationWarningnaming its new home, until 0.22. -
Every annotation carries its real type. The engine's pandapower network and its internal case are
engine.pp_types.PandapowerNetandPpcTables; torch modules and tensors, HDF5 files and datasets, sparse admittance matrices and the stream mappings are named; the column views areNodeColumns/EdgeColumns/BranchColumnsof arrays and the index constantsNODE/EDGE/BRANCHareNodeIndex/EdgeIndex/BranchIndexof ints (the same fields, still tuples); a stream's episodes aremodels.data.EpisodeRowand its summaryStreamSummary.Anystays only where the value can be anything (the validation engine's raw input, keyword pass-throughs, mixed-value staging dicts), each place listed inANY_ALLOWEDintools/readability.pywith its reason, and the readability gate refuses an unlisted one. No behaviour changes. -
load_profiletakes a load source that reads itself:IsoFolder(iso, directory)(an operator's CSV export;models.config.IsoExportchecks the operator and refuses one with no known export format, ERCOT, when the folder is described),CsvColumn(path, column)orRawSeries(values), or any object with aloads()method (profiles.LoadSource). The old form,load_profile(source, path=, column=)with a string or an array, still works for one minor version and raises aDeprecationWarning. fetch_profilereads from the operator's feed inprofiles._FEEDS:NyisoArchive(built in, no dependency) for NYISO,GridstatusFeedfor CAISO and ERCOT, which importsgridstatusitself and names the install when it is missing;models.config.ProfileFetchchecks the operator andresample_min. The downloaded data and the returned vector are unchanged.- Every input is checked in one place (the validation plan, in the git history at
aa77d0b). Each consumer's settings are one model infdia_graph.models.config(LoadOptions,WindowSpec,HuberConfig,PriorConfig,LearnedConfig,FederatedSettings,TimelineKnobsand the rest), whose fields declare their rules (Annotated[float, Positive()],OneOf(Units)); one engine,models.validation, checks them when the model is built. The public signatures are unchanged: the keyword arguments build the model. Every fixed set of values is aChoiceenum inmodels.choices, still importable where it was used. A dataset view is checked through one table,ds.require(...)overdataset.base.CAPABILITIES. A condition only the data reveals raises a named error fromfdia_graph.errors(NoBenignRecords,NoAttackedRecords,GridIslanded, ...). The formulas and the parsers keep their signatures and build their input model (models.inputs:ClientUpdates,ClientGraph,LabelGrids,FamilySelection,SystemRef,ReleaseName,OutageRef,StatePool, ...), so no function body checks its arguments. Every error is still aValueError; the messages now have one shape, ". , got ". The loader's record format("torch" or "pyg") is now checked too: a typo such as "pygg" used to fall through to the torch records silently.check_split,check_unitsandcheck_orderstill work for one minor version and raise aDeprecationWarning. The private helpers the models replace are gone (_check_knobs,_check_settings,_check_frac,_check_graph,_check_blocks,_index_array,_is_int,_FORMATS,_ISOS,_FIELD_FLAG);dataset._FAMILY_ALIASandprofiles.system_idstill resolve. Integer settings (npass,iters,n_calib,layers,hidden,k, a window'sT, the federatedpartition_clientsandepochs, the episode lengths,max_test) now refuse a float at construction; before, a value such asnpass=1.5passed and failed later with a rawTypeError. A malformed value of any setting (a string where a number belongs) is aConfigErrorwith the same message as an out-of-range one. The deprecatedpyg_streamandtorch_windowsrefuse an unknownlayerwith aConfigErrorbefore the stream loads; before, it was a rawKeyErrorafter loading.fetch_profile'sresample_minmust be a whole number of minutes:1.5used to be truncated to a 1-minute cadence without a word, and is now aConfigError.partition_from_assignmentchecks the assignment and the branch list before sizing the grid from them, so a scalar assignment or a non-integeredge_indexis aConfigError, not a rawTypeError, and so is a negative bus count. A profile date is a 'YYYY-MM-DD' string, a date or a datetime on every Python version (from 3.11 '20240131' and the integer 20240131 used to parse too).load_profilerefuses a source that is not aLoadSource, an operator name, a CSV path or a 1-d series of loads (a scalar used to pass as a one-point profile);generate'sstatesmust be an array or a path; andexport(fields=...)takes a list or tuple of names, so a lone string, a mapping or a set is aConfigError.RawSeriestakes a non-empty 1-d series: a scalar used to pass as a one-point profile and a 2-d array was flattened without a word.CsvColumnchecks its path and column when built (a missing column used to surface as a pandasKeyErroron read), and a dataset name that is neither a name nor a bus count, such asfg.load(["ieee14"]), is aConfigErrorinstead of a rawTypeError. A profile date span that ends before it starts, and anIsoFolderdirectory that is not a path, are refused when given;generate(states=pathlib.Path(...))now reads the file instead of failing with anAttributeError. Arguments reach their models unconverted, so a malformed one is aConfigError, not the raw error of a conversion done first:families=3,pool_moments(None),fedavg(None, ...),block_diagonal_basis(3, d),score(ds, scores="bad"),score(ds, xhat="bad")and a non-numeric in-memory state pool. A test refusesModel(tuple(arg))-style calls in the package. A Huberc, a removal threshold,am_rateand the learned localizer'slrandpos_weightmust be finite as well as positive, and a solvertolfinite and non-negative. The learned localizer checks its whole training setup (dropout,lr,weight_decay,batch_size,epochs,pos_weight,seed) onLearnedConfigwhen it is built.generate_timelinechecksfamiliesonFamilySelectionbefore it builds the case, and the admissible-target check is a model,models.inputs.AdmissibleTargets, which raisesNoAdmissibleTargetas before (a model names its failure type inValidated.error; the named data errors now live inmodels.errorsandfdia_graph.errorsre-exports them).TimelineKnobs.am_lendefaults to None, resolved toramp_lenby the model.check_partitionis deprecated in favour of buildingPartitionOnGrid, and the internalcheck_window_argsandcheck_targetsare gone (WindowSpec,AdmissibleTargets). A value no model can even read (a number past the float range, a mapping where a sequence belongs) is aConfigError, never a rawOverflowError,KeyErrororAttributeError;tests/test_malformed_inputs.pysweeps every config and input model with values of the wrong kind and the public entry points with one bad argument each. Loose input is read by a parser model, so no function outsidemodels/dispatches on a type:ProfileSource(whatload_profilewas handed),DateSpan(fetch_profile's dates; a date it cannot read is aConfigErrorbefore any download),StateSource(an in-memory pool or a path) andDatasetName(the registry's lookup key).tools/readability.pynow refusesisinstance(...)outsidefdia_graph.modelsas it refuses a bareraise ValueError. - The estimator solve path no longer takes the slack angle.
_solve(z, w),_w_solve(z, w),_nres(x, z)and the rest solve at the fitted reference (ref_angles, through the new_h_ref(x)); a customSEBasesubclass that overrides_solvedrops itsthslargument.JacobianFeatures.previous_estimatereturns the state alone. The numbers are unchanged. - Detectors calibrate from measurements only.
SEBase.fit(ds, calibrate="measured")sets every meter's sigma from its accuracy class (formulas.estimation.accuracy_class_sigma, the classes now one definition,engine.base.ACCURACY_CLASS, shared with the noise model), takes the angle reference from the case and the linearization point from the mean estimate of the benign training scans, and reads no clean layer. The Jacobian feature sets,ResidualLocalizerandTrustSelectorfit their estimator this way; the state estimators keepcalibrate="truth"as the benchmark default. Residual-based sigmas were tried first and rejected: a meter's constant bias is absorbed into the state, so they shrink on the biased meters and collapse the fit. The frozen localization reference moves for the residual arm.
0.20.0¶
- Data release v0.8.3 is the default (
fg.load("ieee118"); v0.8.1 stays readable withrelease="v0.8.1"). The eight timelines are regenerated from the unchanged pools (byte-identical) with seed 123 and the v0.8.1 knobs, built byexamples/_build_timelines_v083.py: Aq and Al episodes are one frame, and the storedtemporal_deltaandswingare functions of the observed frames alone. Each file holds about 24,600 episodes instead of about 16,100, keeps 72,000 frames, exactly half attacked, no frame falling back to benign, and every family present.data-v0.8.2, an intermediate build with one-frame Aq only, is superseded. - Engine: parameter groups that travelled through every attack function are data models
(
models.frames):Band(floor, cap), the plausibility band of an in-place tamper (FrameKnobs.band);TamperTarget(buses, branches), where an in-place attack writes; andAttackDesign(targets, mult, interior), a stealthy attack's design on one scan.FdiaGenerator.corrupt(scan, buses, kind, replay, band)tampers aScanin place and returns(weak, mags);attack_frame,is_feasibleandstealthy_statetake anAttackDesign, andwith_regionfills in a design's interior. The random-draw order is unchanged: every frozen reference matches bit for bit. - Features are computed from measurements only. The swing feature's scale is now the recent change
of the observed frames, not of the noiseless pool:
timeline.write_temporal_layerswritestemporal_deltaandswingafter the walk from the observed injections alone, andtrust.secured_copyrecomputes them the same way. Every estimate takes one angle reference,SEBase.theta_ref, fixed at fit time from the training split's slack angle (the case's reference angle on the released pools; a slack angle that varies is refused), so estimating, the Jacobian transform, the residual localizer's scores and the trust scoring read no clean layer. Two uses of the truth remain, by design: fitting an estimator calibrates the meter weights, the benign mean state and the angle reference on the training split's clean layer, andSEBase.scorecompares estimates with the truth to report the error.write_temporal_layersruns in blocks with bounded memory. Takes effect in the data with the next release; scores on existing files are unchanged. - Al is a single-snapshot attack: every Al episode of a generated timeline is one frame, as Aq.
- N-1 timeline generation is disabled:
fg.generateandgenerate_timelinetake no outage;line_outage_candidatesremains a screening aid andFdiaGenerator(outage=)stays for engine use. - The tiny test timeline moves to seed 1, the first seed that puts every family in both its train and test split under the new episode lengths; the frozen references are rewritten from it.
- Localization:
features="full14+prev"(16 channels) and"full14+prev+jac"(24) append the previous frame's swing to the papers' 14, forBusCNN,BusMLPand the federated localizers (each bus's own reading, so client-local). A timeline export offers it asprev_swing(file row - 1). On a timeline the frame after a one-frame attack carries the same jump back with the opposite sign, labelled benign (31% of IEEE-118's benign training frames); a one-frame model cannot tell the two apart and stops trusting the swing. With the previous swing, the zero-shot BusCNN on v0.8.1 goes 0.78 -> 0.84, 0.55 -> 0.83 and 0.49 -> 0.81 macro-F1 on IEEE-14, 118 and 300 (seeds 123 to 125), As 0.56 -> 0.95 and Ar 0.32 -> 0.83 on 118, at a lower false-alarm rate. - Aq is a single-snapshot attack: every Aq episode of a generated timeline is one frame, as
corrupt_len=1already made Ad, As and Ar. Through v0.8.1 Aq ran as 15 to 44 frame episodes, so a change-based feature saw only an episode's first frame (on IEEE-118 the zero-shot CNN recalled 0.39 of the attacked buses on that frame and 0.00 on the rest). The tiny test timeline moves to seed 23, the first seed that puts every family in both its train and its test split, and the frozen references are rewritten from it. Takes effect with the next data release. - The Jacobian features never read the true state.
JacobianFeaturestook each frame's change against the previous pool timestep's clean state, truth an operator does not hold; it now takes it against the previous frame's estimate, dz = z_t - h(x_hat_{t-1}), the fitted estimator's plain solve of the frame emitted just before (the slack angle stays the shared angle reference). A timeline export offers that frame's readings on request asprev_node_x,prev_edge_xandprev_timestep(models.PreviousFrameFields), read from file row - 1 whatever split or family it belongs to.BusCNN/BusMLP/ the federated localizers with a Jacobian feature set andJacobianWeightingfollow; a record shard is refused (its rows are not consecutive frames). On the tiny timelineJacobianWeightingmoves in the fourth significant digit; the stealthy re-solve now raises the explained energy at an episode's first frame, as the temporal features do. A test rewrites the clean layer and checks the features do not change. The guides' Jacobian rows are rerun on it: the zero-shot CNN with the Jacobian block reads 0.820, 0.809 and 0.812 on IEEE-14, 118 and 300 (0.873, 0.896 and 0.840 with the truth reference), its gain now on the unseenAsandArrather than onAq; the federated guide's tables are marked pending a rerun. - Docs: the SE, localization and trust guides rerun on the v0.8.1 timelines, every table and quoted number regenerated from the new JSON. Directions that changed: the CNN gate now lowers the estimator's error on IEEE-14 (0.049 to 0.044) and 300, residual removal beats WLS on 300, and the zero-shot CNN with the Jacobian block reads 0.873, 0.896 and 0.840.
- Docs:
docs/federated/, the federated localization papers' protocol on the v0.8.1 timelines.FedBusCNNandFedBusMLPwithfull14+jac, zero-shot, K = 1 to 3, seeds 123 to 125, in the paper's Table IV layout with per-family F1 and FR over every record and over benign records. The run script saves each run and skips saved ones;make_report.pybuilds the tables, figures and CSV sidecars from the JSON. FedBusCNNandFedBusMLPaccept the Jacobian feature sets (features="full14+jac","jac"): the 14 channels stay per client (local power balance by default) and the 8-channel Jacobian block is the whole system's estimator applied to every meter's change, computed once centrally per pass and shared with every client (appended after the 14 channels forfull14+jac, the whole input forjac); that block is the one feature a client does not build from its own meters. One client equals the centralizedBusCNN(features="full14+jac").- Localization, the federated paper's tables:
LocalizerBase.score_perbus(ds, buses=, fr_over=)returns per-bus F1, detection rate, false-alarm rate and AUPRC (models.scores.PerBusScores,PerBusMetrics) over every record and per family (that family plus benign), on the active buses or, for a learned localizer, the training-attackable ones;fr_over="all"is the paper's Table IV false-alarm rate,"benign"equalsscore()["all"].macro_fr.LearnedLocalizer.tune_grid_thresholdandscore_gridgive record-level detection at a validation-tuned grid threshold (models.scores.GridScores). New formulasperbus_ratesandaverage_precision[DG06] (equal to scikit-learn's). Existing outputs are unchanged. fdia_graph.federated.RegionalPrior: the proposed estimator's subspace prior fitted per client of aPartition(each client's basis from its own non-slack angles and voltages, placed on a block diagonal byformulas.federated.block_diagonal_basis), so no client's states leave it; one client isSubspacePriorexactly. WithGatedPrior(gate=FedBusCNN(...))the federated localizer gates the estimator. The meter sigmas and the benign mean were already local; the solve stays central.fdia_graph.federated: the federated localization paper's localizers trained by FedAvg [MCM17].FedBusMLPandFedBusCNNareLocalizerBasemethods (fit(train, val),localize,scores,scoreas forBusMLP/BusCNN) over a K-way partition: each client reads only its own buses, builds its power-balance channel from its own flow meters by default (kcl="local";"global"is the paper's central feature, which reads a neighbour's tie-line meters), trains on its own buses (optionally with a read-only halo) and scores them; only pooled moments, model weights and per-bus confusion counts cross a client boundary.historylogs every round's loss and bytes. With K = 1 and no clip the fit equals the centralized one weight for weight (tested for both encoders). The Jacobian feature sets are supported with one central block (the entry above).fdia_graph.federated, first part: the split of a system's buses into K clients.spectral_partition(edge_index, N, K)is the federated localization paper's partition (spectral clustering of the bus adjacency, random_state 42 [VLX07]) and matched its cached partitions exactly on IEEE 14, 118 and 300 at K = 2 and 3 (the tests pin IEEE 14);attackable=biases the cut away from attackable buses. It returns amodels.federated.Partition(client of every bus, interior and boundary buses, cut edges);compute_nodesgives a client's buses plus an optional halo of other clients' buses.formulas.federated.fedavg[MCM17],interior_boundary,cut_edge_count,attackable_affinityandhalo_nodescarry the arithmetic. Afederatedextra (torch, scikit-learn) and CI installs it.- Learned localizers, no number changes (the first step of
fdia_graph.federated): the training loop islocalization.learned.LocalTrainer(optimizer and batch order persist acrossruncalls, optional owned-bus loss and gradient clipping for a federated client), prediction ispredict, standardization pools channel moments (formulas.federated.channel_moments,pool_moments[CGL79]) and the validation threshold picks from per-bus confusion counts (formulas.metrics.perbus_counts,tau_from_counts[KEC25]), so a federated fit can sum them across clients.models.training.OptimConfigcarries the training knobs. BusMLP and BusCNN weights, scores, thresholds and standardization are bit-identical to 0.19.0.
0.19.0¶
- Data release v0.8.1 is the default (
fg.load("ieee118"); v0.8.0 stays readable withrelease="v0.8.0"): the eight timelines regenerated with this release's generator, the same pools (byte-identical), seed and knobs, built byexamples/_build_timelines_v081.py. The stealthy families change on every system but IEEE-200, whose file is byte-identical to v0.8.0 (no bus there holds both a load and a generator, and its static generators produce nothing). Every file keeps 72,000 frames, exactly half attacked, no frame falling back to benign, every family present, and no stealthy frame labelling a generator bus. -
Generator, the stealthy families change. Aq, At, Al and Am no longer target a load on a generator bus (zero-MW condensers included), the local attacker's rule in [BOY22]: a generator bus is too risky to falsify.
FdiaGenerator.stealthy_posis their target set; Ad, As and Ar keepattackable_pos. Attackable loads left to the stealthy families: IEEE-14 8 of 11, 30 18 of 20, 57 35 of 41, 89 29 of 29, 118 54 of 99, 145 11 of 36, 200 108 of 108, 300 156 of 191. The frozen timeline is re-frozen. The data dictionary states thatattack/magis unsigned (a load rise and a drop of the same size record the same value). -
Loading and fitting, no number changes.
fg.load("IEEE118")works like"ieee118"(a locally registered name stays case-sensitive).latest_releaseskips a tag it cannot parse instead of falling back to the pinned default, and its docstring no longer claimsload(release=None)calls it. Two jobs downloading the same asset at once write separate temp files; the second keeps the first's installed copy instead of corrupting it.fitrefuses a dataset whose slack disagrees with the case's. The solve's reduced normal-matrix inverse is built once infitrather than on every chunk. The recent-change scale's docstring says "plus 1e-3", what the code does. - Tests, no package change.
tests/test_public_coverage.pytests directly what the suite reached only through whole pipelines: the replay policy and its buffer,batched_normal_matrices,local_ac_solveon a two-bus grid,sparse_basis,per_bus_sequences,read_episodes,latest_releaseonline and offline, theTrustSelectorbase, the removal guard, andtune_threshold. - Generator, the stealthy families change. The engine recovered a scan's load at a bus holding
both a load and a generator as the stored injection plus the BASE generation, but the pools scale
a bus's load and generation by one factor, so the load came out as s L + (1 - s) G instead of
s L: at s = 0.75 the median load was 1.6x the true one on IEEE-118 (10 of its 99 attackable
loads) and 2.0x on IEEE-300 (23 of 191), at times negative, so an Aq step there moved the wrong
amount and a "raise" could lower the load. Every load-recovery site now uses
formulas.attacks.bus_load(thegenerator_outputconstruction the limit check already used), throughFdiaGenerator.true_loadandscan_generation. Static generators that produce anything count as injection buses (IEEE-89 buses 1, 65, 69, 79 and eight on IEEE-300 were treated as zero-injection junctions and pulled into attack regions). The attack-redistribution sign: the engine's redistribution LOWERS the target line's |flow| in the false state (checked on 77 of 77 draws), soam_direction="mask"now keeps its sign and"induce"flips it (they were swapped);"both", the default, maps a given draw to the same sign as before. The frozen timeline is re-frozen (341 of its 1000 frames change: the corrected stealthy draws spend the random stream differently, so later noise shifts too). The released v0.8.0 timelines were generated before this fix; regenerating them is a data release. - CI, no package change.
publish.ymlruns the test suite on the tagged commit and publishes only when it passes. The suite also runs on Python 3.9 (the lowest supported) and on Windows, both required bytools/pr.py merge; the Ubuntu job installs the[se]extra by name and reports coverage.ruff format --checkcoverstestsandtools. Every action is pinned to a commit SHA, kept current by Dependabot;slow.ymlruns theFDIA_SLOWIEEE-118 check weekly and on demand. - Fixes the new CI jobs found. The estimators failed on Python 3.9: its SciPy has no
lapack.dtrcon, whichguarded_inversecalled for its condition estimate; it now falls back to Hager's 1-norm estimate (the same value LAPACK returns, a few O(k²) solves) when SciPy lacks it ([se]still asks only scipy>=1.8).tools/readability.pyno longer fails on a file on another drive than the repository (Windows CI's temp dir). - Repository, no package change.
_publish_pypi.shis gone: publishing goes through a version tag andpublish.ymlonly (no manual upload path). Four figures indocs/that nothing referenced (the README uses the copies indocs/figures/) are gone..gitignoreloses a duplicated block, ignores the pytest and ruff caches, and namesexamples/_build_timelines_v080.py, the v0.8.0 build script, as tracked on purpose. - Estimators, numbers change.
ResidualRemovalis classical largest-normalized-residual removal: one meter per record per pass (the largest above the threshold), re-solved, until none exceeds it; a removal the observability guard refuses keeps that meter. It used to remove every meter above the threshold at once, taking out the honest neighbours a single gross error smears onto. Meter sigma (and the trust selectors' alarm level) is calibrated on benign records spread evenly over the train split instead of the first ones, which on a timeline are one early load regime. The guarded weighted solve also considers its final Newton step. Measured on the IEEE-14 v0.8.0 timeline (geometric-mean angle error, test split): removal 0.0665 to 0.0552 degrees (replay 0.0645 to 0.0224), WLS 0.0966 to 0.0961, Huber 0.0613 to 0.0605, prior + Huber 0.0499 to 0.0503, Jacobian weighting 0.0826 to 0.0800, gated arms within 0.2 percent. The frozen SE reference moved by at most 1.6e-7 relative (its train split has fewer than 600 benign frames). - Estimators, fixes. Every public path that solves a dataset (
estimate,score,ResidualLocalizer,TrustSelector.score,JacobianFeatures.fit,JacobianWeighting) refuses aunits="pu"view with a ValueError;fitdid before, but estimating or scoring a per-unit view converted twice and returned errors about ten times too large.GatedPriorandJacobianWeightingsupply their per-record weights through one hook (_record_weights) instead of overridingestimate, soResidualLocalizer(estimator=GatedPrior(...))scores the gated estimator rather than its ungated parent; the Huber passes live once inSEBaseandJacobianWeightinggainstol(it ran all 40 passes) and builds its Jacobian features once infit.ResidualLocalizerno longer refits an estimator that is already fitted (SEBase.is_fitted). The localizer hook is_score(d, ds). ds.windows(..., copy=False)returns the windows as a read-only strided view of the frames, no memory per window (a 72k-frame IEEE-118 timeline at W=60 was about 8 GB as a copy); the default still returns a writable copy, now built from that view.TrustedMetersDQNevaluates the attack cost once per step instead of twice, keeps its replay buffer in a ring, and defaults toseed=123like the localizers (passseed=0for the previous default; the trust guide pins it). Windows and the seed-0 DQN selection are bit-identical to 0.18.0.- The trust guide's secured-copy tables for IEEE-118 (
results/secured_ieee118.json): the DQN set opens the residual test on the stealthy families and lifts the learned localizer four points, the estimator gains within a percent at 20 meters. No package change. - The docs site's staging step drops nav entries for pages a checkout lacks, so a backfilled older version on the versioned site has no link into a 404. No package change.
- Versioned docs:
.github/workflows/docs.ymldeploys the MkDocs site per package version to GitHub Pages with mike (tags aslatest, main asdev) with a version picker on every page; the nav lists every page (the trust guide, the class map, the formulas, references, benchmarks and plans, contributing and the changelog), andscripts/site_index.pystages the three root pages. No package change. - Trusted meters reach the estimators and the localizers:
TrustSelector.secured_copy(ds, out, name)(trust.secured_copy(selector, ds, out, name)) writes a copy of a timeline in which the selected meters read their benign value on every frame, the attacker locked out of them, with the tamper masks and the stored temporal features following, so every estimator and localizer can be scored with a trusted set;GatedPrior(secured=...)never down-weights a secured meter (on a secured copy every gate was worse than no gate until this exemption, and with it the DQN's 20 meters take the proposed estimator from 0.050 to 0.020 degrees on IEEE-14);formulas.projection.meter_positionsmaps masked measurement indices to the file's layers;docs/trust/run_secured.pyand the trust guide carry the tables.formulas.temporal.SWING_WINDOWis the writer's window, imported bygeneration. - The narrative sections of the SE and localization guides (the Jacobian-informed weighting, the localization-gated estimation, the digest's ablation, the three readings) are re-derived on the v0.8.0 timelines from the results JSONs; their v0.7.2 markers are gone. On the timelines the Jacobian block is what makes the localizer's vector work (zero-shot 0.55 to 0.86 on IEEE-118) and a gate alone no longer lowers the estimator's error on any system. No package change.
docs/reference/CLASS_MAP.md: one module diagram and six class diagrams of the package, drawn from the source bytools/class_diagrams.py(inheritance, the classes each class uses, the imports between modules) and checked in CI against the code. No package change.- Every diagram in the docs is a rendered image (
docs/figures/diagrams/<name>.png, source<name>.mmdbeside it,tools/render_mermaid.py), since the GitHub mobile app and PyPI show a Mermaid block as code; no Mermaid block remains. No package change. docs/seanddocs/localizationIEEE-300 columns re-run on the v0.8.0 timeline (tables, figures and CSV sidecars); the live result tables of both guides now read the v0.8.0 release, and the paper-comparison table stays labeled v0.4.1. No package change.- The README's pipeline diagram is a pre-rendered image (
docs/figures/diagrams/pipeline.png, sourcepipeline.mmdbeside it,tools/render_mermaid.py), since PyPI and the GitHub mobile app show a Mermaid block as code; README images use absolute URLs so the PyPI page shows them. No package change. - Docs: the six diagrams that GitHub laid out badly are redrawn (the module map as layers, the generate and load pipelines as two rows, the estimator overview as columns, the trust guide with fit feeding score, the concepts page on the attack-vector construction with Am); no package change.
0.18.0¶
- Reviews:
tools/pr.pywaits for and requires every installed review bot (Copilot always; CodeRabbit and Gemini Code Assist once they have reviewed a pull request) on the head before a merge;.coderabbit.yamlcarries the repository's review instructions (the physics conventions, the readability limits, the prose rules); CONTRIBUTING lists the three reviewers and their order, with the Claude Code/code-review ultrapass first. No package change. - Names, no behavior: the generator's attributes are words (
_ppc_row,_n_ppc_buses,_from_bus_ppc,_base_mva,_target_lines,_primary_target_line,_line_flow_sign,_ptdf_load_buses,_solve_net,_injection_buses,_bias_sd,_draw_noiseand the publicn_lines, whose old namenlstill answers with aDeprecationWarninguntil 0.19), the dataset package's docstring maps each mixin to the questions it answers, and the data dictionary and the schema name the staticgraph/edge_x(the series reactance) apart from the flow layer of the same name. fdia_graph.schemais the file protocol: every group, dataset and attribute name of an HDF5 file, the record-field to path map, the family table and the split codes, defined once; the writer, the readers and the tools spell paths through it, andtools/readability.py(the CI gate) refuses a path-shaped literal ("data/...", "graph/...") in any other module.dataset.base.FAMILIES,STEALTHY_FAMILIESandtimeline.KINDstay importable. No file or number changes.- One export of a split:
ds.export(fields, format="numpy" | "torch" | "tf" | "pandas", device, flatten_features)replacesto_numpy,to_torch,to_tfandto_pandas, which still answer with a deprecation notice until 0.19.ds.windows(..., per_bus=True)returns one sequence per bus ([n*N, W, 4]), whattorch_windowsbuilt, andfg.load(name, split, order="time", format="pyg")gives the graphspyg_streambuilt with the file's chronological split, so both torch helpers are deprecated (retire in 0.19). Eight ways to read a view become three:export,ds[i]andds.windows. No number changes. - Four findings of a full-branch review: a load on the slack bus is never a target (the slack never enters a region, so IEEE-57 frames that scaled its 55 MW load were labelled attacked with no attack in the state); the tamper masks are the meters whose stored float32 reading changed (the attack vector's 1e-7 tolerance marked 1.8% of entries that the reading could not show); a branch out of service is not a hop of the attacker's region on N-1 generators; and the local solve builds its interior Jacobian block directly instead of slicing the n x n derivatives, about half the cost of a stealthy frame on IEEE-300, with identical states.
- The local power flow of a stealthy frame is a damped Newton: each step is halved until the
mismatch drops, the full step first, so every frame the plain method solved is bit-identical. A
step the region still cannot absorb is halved (an Aq step at most three times and never under
the noise floor, a ramp or Am frame at most six times) so the frame stays attacked at the
largest step with a solution, an Al frame halves its redistribution and then redraws its line (up to forty),
and an Aq, At or Am episode tests its design (the full step, the ramp's peak, the held
redistribution's peak) on every frame it will occupy, each with the step that frame will carry
(an Aq episode's whole span, every frame of the ramp and of the Am rise, plateau and return),
and redraws it, up to forty times, when no stealthy state exists on any of them, so an accepted design cannot fall back, and a span with no admissible design stays
benign and is counted; an Am redistribution is also tried at halved sizes above the floor
before it is redrawn. A ramp or Am frame never carries a zero step (the profile's first frame
and return leg floor at one rate), so every frame labelled attacked carries an attack.
fallback_benignnow counts every frame the draw meant to attack that is benign, whether or not an episode record exists for it (an Am episode with no admissible redistribution left no record and was not counted). An Aq episode now draws its direction like the ramp, a load rise or a load drop with the same 5% to 20% scale: on a case that runs below its voltage limits (IEEE-57) a rise near the low buses has no admissible state and a drop is the attack that fits. Loads abovemax_load_mw(new knob, default 2000 MW) are never targets: the IEEE-145 case lumps whole areas into loads of 4 to 58 GW, and a 5% step on one has no local solution, which left 30% of that system's attack frames benign; no other ladder system has a load above 1.1 GW, so their files do not change.fallback_benignis zero on every released file. Every false state also satisfies the security and operational constraints of Wu et al. 2026 (OperatingLimits, their equations 21 to 23): each bus voltage within the case's own limits (a bus the true state already holds outside a limit may not be made worse, IEEE-57 runs below its own minimum) and, for the generators of the attacked subnetwork as in the paper (a boundary bus is outside it, its voltage held true and its injection whatever balances the region), the implied output within its P and Q limits widened per bus to the range the benign pool ran it over (the pools scale generation with load and never enforced nameplate), the true output recovered exactly from the pool's common load and generation scale, and the pretended load change at a target bus not counted against its generator; a false state outside them is rejected and its step halved like an unsolvable one. The file records the widest bus limits asv_loandv_hi. - A stealthy frame (Aq, At, Al, Am) is the true scan plus the attack vector a = h(x_false) - h(x_true)
of its local false state: every meter keeps its own noise draw and the tampered meters are
shifted by exactly what the false state moves them, so
observed - benignis the attack for every family and a WLS residual test flags the stealthy families at the benign rate. Until now the tampered meters were re-emitted from the false state, which drew their noise from the false reading; where a boundary bus has a structurally zero injection (the IEEE-14 synchronous condenser's P, a zero-injection bus) that noise is several times what the estimator calibrates for the meter, and the residual test flagged about a third of the Aq, Al and Am frames on the full IEEE-14 timeline. An Al frame whose drawn target line has no feasible redistribution in its region redraws the line (up to ten) instead of falling back to a benign frame. Both change every generated file; the frozen references are re-frozen. tools/pr.py mergerequires every smoke job green by name on the head (a job that has not registered yet is not green), every other check finished without failure, and Copilot's review on that head;waitwaits for the same set. A merge can no longer slip in while CI is still starting.fdia_graph.trust: which meters to secure so that stealthy attacks stop being stealthy, after the trusted-PMU defence of Wu et al. 2026.TrustedMeters(k)is the greedy row-reduction selection on the WLS Jacobian (secure, on the cheapest open attack, the meter whose protection raises the attack cost most),TrustedMetersDQN(k, episodes)the same selection learned as a Markov decision process with a deep Q-network (state the secured set, reward the attack-cost rise; needs torch). Both expose the order and the attack cost after each meter, andscore(test)on a time-ordered timeline pins the secured meters to their benign reading and reports the WLS residual detection per family before and after (TrustScores). The kernel isformulas.trust(attack_subspace,sparse_basis,attack_cost,greedy_trusted_meters);docs/trust/has the guide andrun_trust.py.- The timeline writer draws its attack episodes so their frames sum to exactly
round(attacked_frac * T), places every one of them (longest first, each at an onset drawn uniformly among the onsets where it fits), and emits benign frames everywhere else. The benign-gap band and the rule that started the next episode with no gap are gone, so whether two episodes touch or a quiet stretch separates them is a property of the draw; an Am episode redraws its redistribution at onset rather than leaving frames benign, and only a non-converging power flow falls back to a benign frame (counted infallback_benign). The frozen references are re-frozen on the tiny timeline (seed 4). - Data releases have their own tag namespace: the assets of
v0.8.0and later live under the GitHub tagdata-v0.8.0(package versions own the barev0.x.ytags, PyPI 0.8.0 isv0.8.0), whilev0.7.1andv0.7.2keep their bare tags;registry.release_tagmaps the short name users write (fg.load(name, release="v0.8.0"),FDIA_GRAPH_RELEASE) to the tag, the newest data release is picked from the tag list, andtools/upload_assets.pyrefuses a package tag. - The localization scores in
docs/localizationare re-run on the v0.8.0 timelines and are not comparable with the v0.7.2 shard tables: the shard's temporal features compared an attacked snapshot with the benign scan before it, so every attacked record spiked, while the timeline's compare each frame with the frame emitted one minute earlier, so a sustained episode spikes at its onset and the one-frame families also spike on the benign frame after them. The per-record numbers are lower for the same detectors and the same data, and the tables say which reference they use. fg.load(name)reads the v0.8.0 timelines by default (_RELEASEisv0.8.0, the checksums of the final files pinned). The test suite's tiny fixture now walks the v0.8.0 pool (72,000 one-minute states where the v0.7.2 pool held 36,000), so the frozen references are re-frozen on it.- The data release
v0.8.0, step 4 ofdocs/plans/ONE_DATASET_PLAN.md: one timeline file per system of the ladder (timeline_ieee{N}.h5, 72,000 frames, the seven families, seed 123,attacked_frac0.5) and the operating-point pools as HDF5 (pool_ieee{N}.h5), built byexamples/_build_timelines_v080.pyfrom the v0.7.1 NYISO pools.fg.load(name)reads them by default;fg.load(name, release="v0.7.2")(orFDIA_GRAPH_RELEASE=v0.7.2) still reads the record shards, since the registry now maps a release tag to the file layout it shipped (registry.dataset_file,pool_spec,is_timeline_release) and knows the sha256 of each pinned release.generate(states=None)fetches the pool through the same registry.load_stream(deprecated) returns the timeline through the loader at a timeline release and the v0.7.2 stream file before it.tools/upload_assets.pypublishes release assets idempotently. - One generator path, step 3 of
docs/plans/ONE_DATASET_PLAN.md: the record-shard writer (generation.generate_shardand its draw loop), the stream walker and its.npzoutput, the magnitude sidecar (<out>.mag.npz) and the graph sidecar read ofload_streamare deleted, with theShardArraysandRecordmodels andSINGLE_SHOT_ORDER.generate_streamis now the timeline writer followed by a read of the file it wrote (streams.stream_ofturns a time-ordered dataset into the stream dict), so its output is the timeline's and itsoutis the HDF5 path;load_streamstill reads the v0.7.2 stream files (which embed their graph) until the v0.8.0 data release. The loader keeps reading the v0.7.2 record shards;tests/data/tiny_shard_v072.h5is a checked-in file in that layout that the suite loads. - The frozen references are re-frozen on the tiny timeline the suite builds (
frozen_spec.TIMELINE_KW, 1000 IEEE-14 frames, every family): the file's arrays and attributes, and the estimator and localizer scores on its test split. The shard and stream references are gone with their writers. - One loader for both kinds of file, step 2 of
docs/plans/ONE_DATASET_PLAN.md.fg.load(name)reads a timeline file (kind="timeline") as well as a v0.7.2 record shard. New arguments:order="time"(default) keeps the file order, chronological on a timeline;order="random"is the same records in a permutation fixed byseed(default 0), a view index, nothing copied, so two people asking for the same seed get the same record table. On a timeline every record, batch and export carriesbenignandedge_benign(the attack removed, the noise kept, in the requested units),ds.windows(W, stride, label, layer)slides windows over a time-ordered contiguous view (a split or the whole file; it refuses a random order, a family subset and a shard),ds.episodesis the episode table (EpisodeTable: onset, length, family, buses) of the view, andtorch_windows(dataset=ds)/pyg_stream(dataset=ds)take the loaded timeline.ds.is_timelineandds.has_benignsay which kind a file is. fg.generate(system, name, frames=None, **knobs)now writes a timeline (the knobs oftimeline.generate_timeline, plusframesto cap the pool timesteps walked) and registers it, sofg.load(name)andfg.load(name, order="random")read it back. The record-shard writer is gone (the entry above): the loader still reads the v0.7.2 shards, nothing writes them.generate_stream,load_streamandwindows(stream, ...)warn withDeprecationWarningand retire in 0.19: the timeline file replaces the stream,fg.load(name, order="time")reads it, andds.windowsreplaceswindows. They keep working unchanged until then (load_streamreads the v0.7.2 stream files).- The timeline writer, step 1 of
docs/plans/ONE_DATASET_PLAN.md:fdia_graph.timeline.generate_timelinewalks one attacked timeline over a system's operating-point pool and writes one HDF5 file (kind="timeline") carrying every layer the streams and shards had between them: the observed, benign and clean scans per frame, the full static graph, per-framestealthy,seq_id(the episode index) and a chronologicalsplitthat never cuts an episode, anepisodes/table, and anattack/group with the designed magnitude per attacked bus and the tamper masks (the meters the attacker wrote). Families are scheduled by inverse expected episode length so each gets about the same share of attacked frames;corrupt_len=1makes every Ad/As/Ar frame an independent draw. The loader for these files is step 2;generateandloadare unchanged in this release. - A seventh family,
Am(code 7, stealthy), the multi-snapshot attack of Wu et al. 2026: the Al load redistribution drawn once at episode onset and applied along a ramp whose per-bus per-frame step stays underam_rateof the noise floor.fg.FAMILIES[7] == "Am",fg.STEALTHY_FAMILIESnow{1, 5, 6, 7}; the published shards and streams carry noAmframes. - Every stealthy family (Aq, At, Al, Am) is now a local false state, the attacker model of Wu et al.
2026: the attacker solves the power flow of the subnetwork within
hopsbranches of the attacked loads (or the target line) with every other bus voltage held true (FdiaGenerator.solve_local,formulas.network.local_ac_solve), never the whole grid and never the slack, and writes only the meters that false state moves (the tamper masks). The measurement vector is exactly consistent with an AC state, so a WLS residual test flags these frames at the benign rate; the global re-solve with pinned generation is gone from the writer.hops(default 2) replacesam_sigma. generate_streambuilds its frames through the timeline module's episode primitives; its scheduler and RNG order are unchanged and the streams are bit-identical.generate(states=...)also accepts a pool stored as HDF5 (datasetX).- One definition of the measurement column orders:
fdia_graph.models.NodeColumns(v,p_inj,q_inj,theta),EdgeColumns(p_from,q_from) andBranchColumns(the eightedge_attrcolumns), with.of(array)giving named views of any such array andNODE,EDGE,BRANCHthe indices. Every record, batch, split and stream has.node()and.edge(); the generator, loader and estimator index columns through the definition instead of literals. No behaviour change. - The shard-shaped bundles (
RecordBundle,BatchBundle,ArraysBundle,Stream) are built from field groups (fdia_graph.models.fields: scan, labels, record ids, temporal, clean, graph, stream layers), so each shared field is declared once;Bundlegains_order(the dict-key order, unchanged from before) and_required(a missing required field raisesTypeErrorat construction, as a missing argument did). These four bundles are keyword-only (a positional call raisesTypeErrorinstead of binding to the inherited field order); they are return types, and no caller in the package or its examples built them positionally. fg.load_streamnow fillssystemandattacked_frac(they were None: the stream files carry the arrays only and the generator attached the two values after writing). Both are derived from the arrays on load, the same waygenerate_streamcomputes them;fdia_graph.streams.stream_summaryis the shared helper.ruff checkconfigured inpyproject.tomlwith pyflakes, import order and modern-syntax rules, and the package clean under them (list[int]-style generics andcollections.abcimports;Optional/Unionstay, sincetyping.get_type_hintson 3.9 cannot evaluateX | None). The CIformatjob runs it on every pull request. No behaviour change.-
tools/bench.pyanddocs/reference/BENCHMARKS.md: per-record timings of shard generation and the WLS, Huber and prior+Huber estimators on the tiny shard, appended per run with the machine and torch state;--checkfails when a timing is more than 3x slower than the last row. -
CONTRIBUTING.md(the rules, the pull-request flow, releasing),tools/pr.py(create, wait, comments, reply, merge, with the merge rule enforced) andtools/release.py(tag, GitHub release, PyPI wait) so a second maintainer can ship; the plan documents move todocs/plans/. - Clear errors at the public edges instead of silent or opaque failures: an unknown family name
or code in
load(families=...), an unknownsplit, an unknown field into_numpyand its siblings, and an invalidlabel, window length or stride inwindowsraiseValueErrornaming what is allowed (an unknown family used to select nothing; an invalid window label used to behave as "any").tests/test_edges.pycovers these and the estimators' and localizers' argument checks;FDIA_SLOW=1additionally runs the IEEE-118 estimator sanity test.
0.17.0¶
State estimation without torch, the formula kernel complete, the loader as concerns, and the
scheduled removals (PRs #74 to #78). What a user sees: pip install "fdia-graph[se]" no longer
pulls torch (add [torch] for faster per-record inverses); the pre-0.16 generator attribute
names are gone; every helper on the package namespace is the real function. Shards, streams and
the localizer scores are bit-identical to 0.16.0; estimator scores moved by at most 2.7e-6
relative (the closed-form Jacobian, see the entry below). No import path changes.
- State estimation no longer needs torch:
formulas.network.ac_measurementis the estimator's h(x) andac_jacobianits closed-form Jacobian (equal to the automatic-differentiation Jacobian to 1e-14), sofittakes no gradient; torch, when installed, only speeds up the per-record inverses, and the[se]extra no longer installs it. The frozen estimator scores moved by at most 2.7e-6 relative (six of 56 cells above 1e-9, all in the Huber arm, whose passes amplify last-bit differences in h); shards, streams and every other score are bit-identical. The torch twinSEBase._h_tstays for callers that differentiate through it. formulas.estimation(the WLS step, gain matrix, residual covariance, normalized residual, Huber weight, the operating-point prior basis, the localization gate),formulas.linalg(the guarded inverse, condition number, per-record normal matrices) andformulas.projection(the weighted pseudo-inverse, explained/unexplained split, leverage, weak directions, meter-to-bus aggregation): the estimators and the Jacobian features now call these named functions, each with its equation and source key. Identical numbers (bit-identical frozen scores); the estimator's private helpers keep their names and delegate.- Removed, as scheduled one minor version after 0.16: the generator's pre-0.16 attribute names
(
edge_r...edge_status,M,flow_meter,bias_pi...bias_qf,outage,outage_pos...outage_base_flow_mw; readg.branch,g.meters,g.bias,g.contingency) andfdia_graph.streams._swing_scale(usefdia_graph.generation._swing_scale). fdia_graph.generate,generate_stream,load_stream,windows,pyg_stream,torch_windows,load_profile,fetch_profile,generate_statesandline_outage_candidatesare the real functions, resolved on first use (PEP 562) instead of wrappers that restated their docstrings. Same names, same calls, same lazy imports;help(fg.generate)now shows the one docstring.- The loader is the
fdia_graph.datasetpackage:FdiaGraphis assembled fromgraph,physics,recordsandexportmixins over abasethat declares the shared state, the same pattern as the generator. Same import path, same names, same behaviour (docs/plans/READABILITY_PLAN.md, step 9).
0.16.0¶
The readability and data-models series (PRs #64 to #72). What a user sees: every dict the package
returns is now a typed bundle with attributes and a docstring, still the same dict with the same
keys; every model lives in fdia_graph.models; the generator's old attribute names warn. Shards,
streams and every score are bit-identical to 0.15.0 (the strict frozen tests are the proof), no
data release moves, and no import path changes. The entries below are in the order the steps
merged, newest first within each series.
Data models, steps 1 and 2 (docs/plans/DATA_MODELS_PLAN.md). No user-visible change; one deprecation.
fdia_graph.models.Bundle: the base of typed records, a frozen dataclass that is also a read-only mapping, so a bundle works wherever a dict did.- Internal tuples and dicts are models:
Scan,TrueState,Redistribution,ResolvedPool,Admittances(also returned byformulas.network.branch_admittances),AssetSpec(returned byregistry.resolve, indexable as before),DownloadTarget,ShardArrays. FdiaGeneratorholdsbranch,meters,biasandcontingencymodels. Deprecated: the previous attribute names (edge_r...edge_status,M,flow_meter,bias_pi...bias_qf,outage,outage_pos...outage_base_flow_mw) still work as read-only properties and warn; they are removed one minor version later.
Data models, step 3: the public dicts are typed bundles. No user-visible change: every one is
still the dict it was (a dict subclass with the same keys in the same order), so indexing,
**, iteration, JSON dumping of score tables and PyTorch's default collate keep working; each
also has attributes and a docstring naming its fields.
dataset.RecordBundle(FdiaGraph[i]),BatchBundle(collate),ArraysBundle(to_numpy,to_torch,to_tf),Summary(summary).se.base.EstimatorScoresofErrorPairrows (SEBase.score);localization.base.LocalizerScoresofOverallMetrics,BenignMetricsandFamilyMetrics(LocalizerBase.score).se.jacobian.JacobianOutputs(JacobianFeatures.transform; the"global"key is theglobal_attribute).streams.Stream(generate_stream,load_stream) andengine.core.LineCandidate(line_outage_candidates).Bundleis now adictsubclass rather than aMapping, soisinstance(out, dict)andjson.dump(out)hold as well; it is read-only on both sides.
Data models, step 5: every data model lives in the fdia_graph.models package, grouped by what
the data is (grid, frames, data, scores, assets), with fdia_graph.models.PUBLIC naming
the bundles a user receives. Every previous import path still works (each producer module
re-exports what it used to define), so no user code changes.
Data models, step 4: docs/reference/DATA_DICTIONARY.md gains a "Models" section listing every
public bundle and its fields, generated by tools/models_doc.py from the dataclasses and kept
current by a test.
Readability series, steps 6 and 7 (docs/plans/READABILITY_PLAN.md): the temporal and ramp formulas in the kernel, and the rest of the backlog. No user-visible change (bit-identical shards and streams, identical estimator and localizer scores).
formulas.temporal(recent_change_scale,temporal_delta,swing_zscore) andformulas.attacks.ramp_profile, each with its equation and source key, used by both generators.AttackMixin.corruptdecides the family once and runs one short loop per family;PhysicsMixin.solvepins the generation in_pin_generation;line_outage_candidatesscreens a line with guard clauses; the loader's__getitem__andcollate, the localizer'sscore, the learned model builders, the stream loader and the download are split the same way; the Huber passes of the estimators are one method.
Readability series, step 5 (docs/plans/READABILITY_PLAN.md): the generator constructor, the loader
constructor and to_numpy split into named steps. No user-visible change (bit-identical shards
and streams, same loader outputs).
FdiaGenerator.__init__reads as a list of what it builds: the case with its contingency, the load tables, the meter plan, the edge index, the branch physics, the admittances, the meter biases; the accuracy-class split of the meter error isformulas.noise.bias_jitter_splitand the series admittance comes fromformulas.network.FdiaGraph.__init__delegates to_read_header,_read_static_graph,_read_layers,_reference_bus,_record_maskand_preload;family_idstranslates family names or codes in one place;to_numpygathers the clean layers and converts units through small helpers.
Readability series, step 4 (docs/plans/READABILITY_PLAN.md): one AC measurement function. No
user-visible change (bit-identical shards and streams; ybus, yf, yt and edge_clean_full
unchanged).
fdia_graph.formulas(new, public, provisional until 1.0):formulas.networkholds the AC network model as pure functions,complex_voltages,series_admittance,branch_admittances(the pi model behindybus/yf/yt),bus_injectionsandbranch_flows, each with its equation and source key. The generator's measurement emission and the loader's admittance and clean-flow computations call them; the estimator's torch measurement function is pinned to them by a test, so the package has one AC model.
Readability series, steps 2 and 3 (docs/plans/READABILITY_PLAN.md): one attack frame for shards and
streams, both generators split into named steps. No user-visible change: a seeded shard and a
seeded stream are bit-identical to the previous release (tests/test_frozen.py, strict mode).
fdia_graph.engine.records:attack_framebuilds one scan of any family from a stored operating point, forgenerateandgenerate_streamalike;FrameKnobscarries the two switches that differ between them;replay_frameandremember_benignname the replay policy.generateandgenerate_streamare sequences of named draws and episodes; the swing scale, the temporal features and the ramp profile are functions with their source keys, shared by both.- Shard records are a named
Recordtuple and the writer reads them by field name; the writer is split into the attributes, graph, data and clean groups. - No public name changed. The private stream helper
_swing_scalenow lives ingeneration;streams._swing_scalestays as a deprecated alias that warns, removed one minor version later.
Readability series, step 1 (docs/plans/READABILITY_PLAN.md): the safety net. No user-visible change.
tests/test_frozen.py: a generated shard, a generated stream, and the estimator and localizer scores on the test shard are compared with frozen references (tests/frozen/): exactly in strict mode; otherwise arrays to 1e-7 relative and estimator and localizer scores to 1e-4 relative, the cross-platform tolerances the test module documents.tools/freeze_reference.pyrewrites the references when a change to the generator or an estimator is intended.tools/readability.py: the readability measures (complexity, nesting, closures, parameters, positional indexing) as a report over the package and as a gate on the functions a change touched; runs in CI.docs/reference/FORMULAS.mdandREFERENCES.md: the catalogue of formulas and their sources, with the docstring template every formula function follows.docs/plans/READABILITY_PLAN.md,docs/plans/RESTRUCTURE_PLAN.md: the plans for the series.
0.15.0¶
edge_clean_full: the exact clean power flow on every branch, computed on load from the clean state through the branch admittance matrix; in the record dict, PyG data,collate,to_numpyand the per-unit view.
0.14.1¶
- Per-record normal matrices are built by chunked matrix products instead of one einsum; the IEEE-300 estimators run in minutes instead of days. Same numbers.