Netbox/README.md

# Netbox

Generated documentation for the field-switch network, sourced from this
utility's live [NetBox](https://netbox.dev/) instance. NetBox is the source
of truth here, not this repo -- everything under `topology/` is a build
artifact, regenerated from NetBox and committed so history shows what the
*physical network* changed, not just what someone drew.

## topology/

| file | what it is |
| --- | --- |
| `export_topology.py` | Run inside the NetBox Django shell. Walks every field/distribution switch's cable path (through patch panels and splice modules), then pulls every real `Cable` among the devices it found so each edge carries the exact port name on both ends, plus that port's 802.1Q mode/untagged/tagged VLANs where the port is a real interface. Also exports every `ipam.VLAN`. Read-only. |
| `topo_full.json` | Latest exported snapshot: devices (id, name, role, site, status), edges (`{a, a_port, a_vlan, b, b_port, b_vlan}`, one per cable), and `vlans` (every VLAN in NetBox). |
| `generate_drawio.py` | Pure-Python renderer: `topo_full.json` → `Field-Switch-Topology.drawio`. No NetBox access needed. |
| `Field-Switch-Topology.drawio` | The diagram. Open at [app.diagrams.net](https://app.diagrams.net) (or the desktop app, or the VS Code draw.io extension) -- free, no license needed. |
| `render_png.js` + `render_host.html` | Renders the `.drawio` through a real draw.io instance (embed.diagrams.net, driven headlessly with Puppeteer) to produce `Field-Switch-Topology.png` -- an actual render, not an approximation. Needs `npm install puppeteer` and network access. |
| `Field-Switch-Topology.png` | Static preview of the diagram, for anyone who wants a quick look without opening draw.io (viewing it in the Gitea file browser, a PR diff, etc.). |

Two distribution switches (`fche-sw17a`, `dmo-lob-sw35a`) anchor two
fabrics; everything downstream is field switches daisy-chained through
patch panels and fiber splice modules. A handful of switches sit outside
both fabrics for a documented reason (spare/inventory, decommissioning, or
planned/not-yet-installed) -- those are called out in their own box on the
diagram rather than silently dropped.

Every connector is labeled with the port at each end (e.g. `9 ↔ 1`), and
switches/patch panels look the part: switches get draw.io's built-in network
switch icon, patch panels get a small rack-unit look (header strip + port
ticks) rather than a plain box.

A **VLAN legend + table** sits at the top of the diagram: a colored
"VLAN <id> / name" block per VLAN on the left (each color also used for
that VLAN's connector tags and table swatch), and the detail table
(VLAN / Name / Description / Subnet / Site) next to it. If NetBox has no
VLANs defined, both render as an explicit "not configured yet" placeholder
rather than being silently omitted or -- worse -- filled in with invented
VLAN numbers. Add VLANs and interface tagged/untagged assignments in
NetBox and regenerate (no code changes needed): both populate for real,
and every connector whose port has VLAN config picks up a short suffix
like `9·V214+215T ↔ 1`.
Deliberately left out: literal switch CLI config dumps -- those belong in
the switches' NetBox config-context/notes, not pasted into a topology
diagram where they'd immediately go stale.

Current convention (documented in NetBox, not hardcoded anywhere in these
scripts): every FS.com `IES3110-8TFP-R` field switch trunks VLAN 214
(Switch Management) as the native/untagged VLAN on both its SFP uplink
ports (named `9`/`10` or `Gi1/9`/`Gi1/10` depending on the device), tagged
alongside whatever department VLAN that switch belongs to -- derived from
the device's NetBox **Tenant** (`AMI`/`Electric`/`Gas`/`Water`/`Waste
Water`/`Security`), not stored on the interface directly:

| Tenant | VLAN |
| --- | --- |
| AMI | 203 |
| Security | 209 |
| Gas, Water, Waste Water | 215 (Gas and Water) |
| Electric | 217 |

That mapping lives only in whoever ran the one-off assignment script, not
in `export_topology.py` -- if a new switch's Tenant changes, its tagged
VLAN has to be re-assigned in NetBox to match; the diagram just reads
whatever `interface.tagged_vlans` already says.

Connectors in the `.drawio` file are glued by shape id (draw.io's native
`source`/`target` reference on each edge) -- drag a switch or panel in the
app and its links follow, port-tick decorations included since they're
parented to their panel's body cell. That's different from this project's
earlier Visio (`.vsdx`) attempt, where glue had to be hand-written as cell
formulas because the format has no simpler mechanism; draw.io does this
natively, which is a big part of why this repo uses it instead.

## Regenerating

From the NetBox host (needs `docker exec` access to the NetBox container):

```sh
docker exec netbox-netbox-1 python3 /opt/netbox/netbox/manage.py shell \
    -c "$(cat topology/export_topology.py)" 2>/dev/null > topology/topo_full.json

python3 topology/generate_drawio.py topology/topo_full.json topology/Field-Switch-Topology.drawio

# optional -- refreshes the PNG preview to match; first run: npm install puppeteer
node topology/render_png.js topology/Field-Switch-Topology.drawio topology/Field-Switch-Topology.png
```

Commit all three (`topo_full.json`, `.drawio`, `.png`) together. The diff on
`topo_full.json` is the real changelog -- it shows exactly which device or
cable changed between two snapshots; the `.drawio`/`.png` diffs are mostly
noise (coordinates, pixels) but are kept in sync so the rendered diagram
never drifts from what NetBox actually says.

## Gotchas learned the hard way

- **NetBox's cached cable path can go stale.** If a switch's
  `connected_endpoints` comes back empty even though it's obviously
  cabled, try `manage.py trace_paths --force` before assuming the cable
  data itself is wrong -- it recomputes the path cache and doesn't touch
  any cable/termination records.
- **A patch panel's front↔rear pass-through has to be modeled
  explicitly** (a `PortMapping` row). A splice module can have a
  perfectly good cable on both sides of a port and still fail to trace if
  that mapping is missing -- check `FrontPort.objects.get(...).mappings.count()`
  before assuming the far end really is disconnected.
- **Two devices can be joined by more than one real cable.** Most FS.com
  field switches (`IES3110-8TFP-R`) run both SFP ports (9 and 10) as
  redundant trunk links, often both landing on the same local patch panel
  before fanning out from its rear ports -- so `export_topology.py` keys
  edges by cable, never by device pair, or a second real cable between
  the same two devices silently disappears. Layout has the same trap one
  level up: dedupe a node's *neighbors* before recursing in the tree
  layout, or a parallel edge gets walked twice and the second pass finds
  its whole subtree already `visited`, so it gets recorded as a phantom
  extra leaf -- inflates the page height for no real reason, one phantom
  leaf per redundant cable.

## Possible next step

`export_topology.py` talks to NetBox via the Django ORM (`docker exec` +
`manage.py shell`), which only works run from the NetBox host itself.
Swapping it for a NetBox REST API client (device/cable/interface
endpoints + an API token) would let this regenerate from anywhere,
including CI, without shell access to the container.