Config migrations¶
One-time rewrites of the shape a site's config/ is written in — a renamed field, a changed type, a moved file (ADR-025). Changing a declared field's value is not a migration: that is module-manager module modify --set (ADR-020).
Ordering (ADR-025 D9 rule 3). A site receives the runner in one update before it receives anything for the runner to run: the runner reached
stablewith this directory empty, and only then did0003land. A Wave 1 migration (0004on) does not merge tomainuntil Wave 0 is onstable. The gate is the release process — from inside a checkout the two cases look identical.
The directory is empty in the release that introduces the runner (ADR-025 D11), which is the Wave 0 exit gate.
What ships here now¶
| id | what it does | reversible |
|---|---|---|
0003-update-schedule-object.sh | site.json updateSchedule becomes {frequency, weekday, hour} (ADR-017 D7). A weekday under daily/none was never read, so it is dropped — and reported, because it is the only trace of what the operator believed they had asked for. | yes — restore .migrations/backup/0003/site.json |
0004-kind-names-the-workload.sh | kind names the workload (ADR-022f: vm, lxc, machine, application, device), authored in each module's source; the ADR-007 marker "kind": "module" is removed from deployed configs so the next update's merge can adopt the authored value — except where it is a config's only module signal, where it is kept and named, since removing it would hide the module. "external-host" → "machine". (#611) | yes — restore .migrations/backup/0004/<file> |
0005-people-becomes-identities.sh | config/people/ becomes config/identities/ (#628); people is left as a symlink to it for one stable cycle, the path's twin of the people-manager alias. Refuses when both are real directories — two copies of the domain are a person's to reconcile. | yes — remove the symlink, restore .migrations/backup/0005/people/ |
0006-location-becomes-module-source.sh | A deployed config's location (the module's source directory) becomes moduleSource, in place (#609): location is for a physical place (physicalLocation, and site.json's own). A non-string location is a place and is left alone. Refuses a config carrying both with different paths. Every reader accepts both names for one stable cycle. | yes — restore .migrations/backup/0006/<file> |
0007-placement-node-names-the-host.sh | The backup module's placementState: "node:<host>" becomes "node" with the Host in .node (ADR-012 §2.1, #600). A different .node was a discovery constraint the resolution already acted on; it is reported and kept in the backup. Refuses node: naming no Host. The module reads both shapes for one stable cycle. | yes — restore .migrations/backup/0007/<file> |
0008-satellite-records-its-module.sh | A satellite-<name>.json that records no module source gets moduleSource — the satellite module's directory beside the recorded tappaas-cicd source, never the script's own location. Before it, module list --resolution reported the satellite unresolvable. Refuses when that directory cannot be derived or does not exist. | yes — restore .migrations/backup/0008/<file> |
0012-mgmt-reaches-all-zones.sh | zones.json mgmt.access-to becomes ["internet", "all"] (#375): all is every other standard zone (not Disabled, not Overlay or WAN), expanded by the controller when it reads the file, so a zone add or delete never edits mgmt again. Reports the zones the old list did not name and drops names that are no zone. No firewall rule changes: mgmt is Manual. Refuses a list without internet (the sentinel would add egress) and anything that is not a list of names. | yes — restore .migrations/backup/0012/zones.json |
0011-site-backup-is-an-object.sh | site.json backup: true becomes {} (configured, every setting at its default) and false becomes null (#737): the schema wants the settings object or null, so site-manager validate failed on the boolean. true never named a target, so the migration reports that rather than inventing one. Refuses any other non-object value. | yes — restore .migrations/backup/0011/site.json |
0013-adopted-machines-keep-password-ssh.sh | A debianhost config that does not state sshKeyOnly gets false (#756): from #756 a machine is key-only over SSH unless its config says otherwise, and a machine adopted before that was promised that adoption changes nothing about how it is reached. The satellite and pvehost are not touched — they take the key-only default. Refuses a non-boolean sshKeyOnly. | yes — restore .migrations/backup/0013/<file> |
0014-modules-moved.sh | The modules grouped by stack (#421): src/apps/<m> → src/<stack>/<m> for the 13 modules that moved. Installed modules follow the moves in its table (written by scripts/move-module.sh, #500): a config whose moduleSource is a moved module's old directory points at the new one, in this site's checkout of the target repository. Refuses a target repository the site has not registered, a missing target directory, and a rename another config still depends on. | yes — restore .migrations/backup/0014/<file> |
0015-modules-moved.sh | Installed modules follow the moves in its table (written by scripts/move-module.sh, #500): a config whose moduleSource is a moved module's old directory points at the new one, in this site's checkout of the target repository. Refuses a target repository the site has not registered, a missing target directory, and a rename another config still depends on. | yes — restore .migrations/backup/0015/<file> |
0016-zones-get-ipv6-subnets.sh | Every zone that can carry IPv6 gets ipv6Subnet (ADR-030 D4): its VLAN tag read as hex where that fits (VLAN 310 → 0x31), else the lowest free number, in VLAN order. Management, Overlay and WAN zones are skipped, and so is a zone that already has one. The number alone gives no IPv6: that needs site.json ipv6: true and a WAN prefix. network-manager modify <zone> --unset ipv6Subnet keeps a zone IPv4-only. Refuses a non-integer or shared number. | yes — restore .migrations/backup/0016/zones.json |
0017-site-organizations-removed.sh | site.json loses organizations, a list of paths to the organization files that nothing read: the organizations are the files in config/identities/organizations/, which identity-manager owns. site-manager validate refuses a site.json that still carries it. The organization files are not touched. Refuses a site.json that is not a JSON object. | yes — restore .migrations/backup/0017/site.json |
0018-zone-names-its-environment.sh | Each zone names its environment, and environments name no zone: a zone gets environment from the environment file naming it as its network.zone (its own zone) and from its serves (the environment it serves — a serves list stays a list); a Service zone no environment names as its zone and that serves environments becomes a shared service zone, its list kept even of one; network leaves every environment file and serves every zone. The firewall graph is unchanged. Refuses, changing nothing, a zone two environments name as their zone, a zone whose serves or existing environment disagrees with the environment naming it as its zone, an environment naming a zone that is missing or not a Service/Management zone, and serves on a zone that is not Client, IoT, Guest or such a shared Service zone. | yes — restore .migrations/backup/0018/zones.json and environments/ |
0019-site-location-becomes-locations.sh | site.json location becomes locations{<name>} with primaryLocation naming it: country, city, the building as address, timezone, latitude and longitude; the name is the building's, else the city's, else the site code's, lower-case. locale and keyboard move to the top level (a value already there wins, and the difference is reported). Each backup peer's physicalLocation becomes a location it names: the site's location at the same country, city and building, or a new one of kind: remote named after its building, city or the peer. Refuses, changing nothing, a location with no country or beside locations, a primaryLocation naming no location, a peer's place with no country, and a peer with both fields or naming a location the site lacks. Readers accept both shapes until it has run. | yes — restore .migrations/backup/0019/ (site.json and the peer files) |
0020-racks-become-positions.sh | racks.json becomes positions.json (position-manager, was rack-manager): mounts become placements — one in a rack keeps its rack, unit, height, width, slot, face, plate, label and description; one loose in a room keeps its room, plate, label and description (its height, width and face are dropped and reported: its plate gives its size) — and every room without a location gets site.json's primary location (with no site.json yet the rooms keep none: absent is the primary). Racks, devices and cables are carried over as they are; racks.json is removed once positions.json is written. position-manager reads racks.json until then and refuses to change it. Refuses, changing nothing, both files present, a racks.json or mounts that is not an object, a mount with both a rack and a room or neither, a room naming a location site.json lacks, and a site.json still carrying the retired location. | yes — restore .migrations/backup/0020/racks.json and remove positions.json |
0021-device-locations.sh | A machine module's physicalLocation becomes location, the name of one of the site's locations: the one at the same country, city and building, else a new one of kind remote named after its building, else its city, else the instance — so no off-site verdict changes. A switch's or access point's location in the switch configuration (desired and actual) stays when it names a location; an empty one goes; other text is kept as its description (#871, ADR-036 D2). Refuses a site.json without locations (0019 first), a machine place without a country, a machine with both fields, a location that is not text. | yes (.migrations/backup/0021/) |
0022-nodes-are-pvehost-instances.sh | site.json hardware goes: the cluster's nodes are their pvehost instances. Each hardware.nodes[] entry's storagePools is written onto config/<node>.json when the instance has none of its own; one that has a list keeps it, and a difference is reported (#872, ADR-036 D6). Readers accept both shapes until it has run. Refuses, changing nothing, a node with no pvehost instance (missing, archived or another module's — register it with module-manager module adopt <node>.mgmt.internal, or remove a node that is gone by hand), an entry with no name, and storagePools that is not a list of names. | yes (.migrations/backup/0022/) |
0023-shared-service-zones-repaired.sh | Repairs a site that ran 0018 before it kept a shared service zone's list: an environment named as its own by more than one Service or Management zone (a string each) keeps one — the zone it named as its network.zone before 0018 (0018's backup), else the zone named like it — and each other becomes a shared service zone, its string a one-element list. No firewall rule changes. A site with one own zone per environment is untouched. Refuses competitors neither rule decides, naming them, and a Management zone that would become shared. | yes — restore .migrations/backup/0023/zones.json |
0024-weekly-default-becomes-daily.sh | A site on exactly the old weekly default — weekly on Tuesday (create-site.sh) or Thursday (the schema) at 02:00 — updates daily at 02:00, the default from 2.4 (#801). Any other schedule, and a site with none (already daily), is left alone. A site that chose one of those two values cannot be told apart, so the change is a warning naming site-manager site modify --updateFrequency weekly --updateWeekday <day> --updateHour 2. Refuses the triple (0003 first) and a site.json that is not JSON. | yes — restore .migrations/backup/0024/site.json |
0025-ssids-become-site-wide.sh | Each access point's ssids{} in switch-configuration-desired.json becomes one site-wide ssids{}: per SSID its zone, security, radius, captive, disabled, and broadcastOn — "all" when every access point declared it, else those that did. Each zone's SSID in zones.json names the SSID that bridges to it (a placeholder, nothing, or a name no access point declared is replaced; the last is reported). Each AP's ssids{} stays: ap-manager now writes it from the list, with the same content. Refuses, changing nothing, two access points declaring one SSID with different settings (naming the SSID, the access points and the fields), an SSID with no zone or one zones.json lacks, a VLAN other than its zone's, two SSIDs on one zone, and a file that is not a JSON object. A site already on the list only gets a zone that does not name its SSID fixed. | yes — restore .migrations/backup/0025/ |
0026-zones-consent-to-existing-rules.sh | Each zone accepts the cross-zone rules installed modules already have (#816): where a module's egress, its ingress from a module or zone, or an auto-pinhole to a provider whose service has ports would cross from one zone into another that neither the source's access-to nor the target's pinhole-allowed-from allows, the source zone is added to the target's pinhole-allowed-from. Before #816 such a rule was written; after it, the module's update failed and was rolled back. Each grant is reported with the modules that needed it. Ingress from the internet is reported, never granted. No firewall rule changes by itself. Refuses an access-to or pinhole-allowed-from that is not a list of names. | yes — restore .migrations/backup/0026/zones.json |
0027-devices-become-assets.sh | positions.json devices{} becomes assets{} (asset-manager, was position-manager; ADR-040 D7-D9): each device keeps its name, kind, model and location, its ports become {name, kind: rj45, speed: 1g} with its MAC on the port its recorded cable reaches, its stage placed or stock, its role its own name. Every node (its pvehost instance, primary first), switch and access point (the switch configuration's order) no asset plays yet gets one, named by the next tag of its kind (srv-NN, sw-NN, ap-NN, ADR-040 O5), the role's name as displayName and role, no serial. Placements and cables keep naming the roles. Refuses devices{} beside assets{}, a kind it does not know, and a device named like a node, switch or AP. | yes — restore .migrations/backup/0027/positions.json (or remove positions.json when the backup holds positions.json.absent) |
0028-modules-moved.sh | Installed modules follow the moves in its table (written by scripts/move-module.sh, #500): a config whose moduleSource is a moved module's old directory points at the new one, in this site's checkout of the target repository. Refuses a target repository the site has not registered, a missing target directory, and a rename another config still depends on. | yes — restore .migrations/backup/0028/<file> |
0029-dns-credential-becomes-provider.sh | ~/.acme-dns-credentials.txt becomes the ca-manager DNS provider it names, serving any domain (ADR-043 D6): its fields in the key dir (providers/<name>.env, mode 600), their names in certificates.json. Written as ca-manager writes it; the file is left for acme-setup.sh, a shim for one release | yes |
Its readers accept both shapes: lib/update-schedule.sh (the timer renderer and site-manager validate share it) and site-manager site show / site modify. That is deliberate — a restored backup or a site on an older release can still hold the triple, and a reader that refused one would take that site's updates away. site modify always writes the object, so a modify cannot undo a migration the ledger says is done.
Writing one¶
A migration is NNNN-<slug>.sh. The number is allocated when the migration is written, is never reused and never changes meaning — the ledger records numbers, so renumbering would silently re-run or silently skip. Line 2 is the summary the runner prints in --list; the rest of the header states what it changes, which release introduced it, whether restoring its backup reverses it, and the issue or ADR that required it.
#!/usr/bin/env bash
# 0003-update-schedule-object.sh — updateSchedule becomes a named object
#
# Introduced: 2.1 (Wave 0, G0.1). Required by: ADR-017 D7.
# Touches: config/site.json. Reversible: yes — restore .migrations/backup/0003/.
The contract (ADR-025 D3), all of it enforced by review and by the migration's own fixture test:
| Rule | Why |
|---|---|
| Idempotent — a second run changes nothing and exits 0 | the ledger is an audit trail, not the safety mechanism |
--check writes nothing and reports what an apply would change | site-manager update --dry-run shows an operator what is coming |
Back up before the first write, into ${TAPPAAS_MIGRATION_BACKUP_DIR} | that copy is the rollback (D8); there are no down-migrations |
| Exit non-zero on any doubt | stopping costs one night's sweep; guessing costs a site's config |
bash, jq, coreutils and config/ only | it runs before the nixos-rebuild, so the new system generation and the manager bins may not exist yet |
config/ only | VMs, the firewall, the cluster and the repository belong to other paths |
The runner exports CONFIG_DIR, TAPPAAS_CONFIG_DIR and TAPPAAS_MIGRATION_BACKUP_DIR (config/.migrations/backup/NNNN); a migration creates that directory itself if it writes.
Moving a module (#500, ADR-025 D14)¶
Don't write a move by hand — 0009 was the last one. From the root of a TAPPaaS checkout:
The tool moves the files, rewrites the catalogue entry, and adds the move to NNNN-modules-moved.sh with its fixture test and a row in the table above. How to use it, what it leaves to you, and what each site does with the migration: MOVE-MODULE.md.
Its test ships with it¶
A change under config/ with no migration is incomplete; a migration with no fixture test is untested (ADR-025 D7). Drop scripts/test/test-migration-NNNN-<slug>.sh next to the others — tappaas-cicd/test.sh Test 9z sweeps that directory, so it joins the fast tier by existing. Each fixture builds a throwaway config/ in a temp dir, runs --check (asserting nothing was written), applies, compares against the expected output, and applies again to prove idempotence.
How they run¶
scripts/run-migrations.sh, from tappaas-self-prepare.sh — after the control-plane refresh, before the nixos-rebuild and before the sweep's first module (ADR-025 D2). A failure stops the unit: no rebuild, no sweep, and the #651 notice names the stage migrate.