Add NetBox-derived field switch topology (draw.io)

- export_topology.py: dumps switch + patch-panel cable graph from NetBox (Django shell, read-only) - generate_drawio.py: renders topo_full.json as a draw.io diagram with natively-glued connectors (source/target by shape id, no hand-rolled glue formulas) - topo_full.json + Field-Switch-Topology.drawio: current snapshot Replaces the earlier hand-built .vsdx approach: plain-text format diffs meaningfully in git, no Visio license needed to view/edit, and edges glue natively instead of needing point-to-point formula hacks. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NM3nivbufxEWiKKmatEtEe
2026-09-15 12:41:22 -05:00
commit d275b7e821
5 changed files with 403 additions and 0 deletions
@@ -0,0 +1,67 @@
+# Netbox
+
+Generated documentation for the field-switch network, sourced from this
+utility's live [NetBox](https://netbox.dev/) instance. NetBox is the source
+of truth here, not this repo -- everything under `topology/` is a build
+artifact, regenerated from NetBox and committed so history shows what the
+*physical network* changed, not just what someone drew.
+
+## topology/
+
+| file | what it is |
+| --- | --- |
+| `export_topology.py` | Run inside the NetBox Django shell. Walks every field/distribution switch's cable path (through patch panels and splice modules) and dumps the resulting device graph as JSON. Read-only. |
+| `topo_full.json` | Latest exported snapshot: devices (id, name, role, site, status) + edges (physical links, one per cable). |
+| `generate_drawio.py` | Pure-Python renderer: `topo_full.json` &rarr; `Field-Switch-Topology.drawio`. No NetBox access needed. |
+| `Field-Switch-Topology.drawio` | The diagram. Open at [app.diagrams.net](https://app.diagrams.net) (or the desktop app, or the VS Code draw.io extension) -- free, no license needed. |
+
+Two distribution switches (`fche-sw17a`, `dmo-lob-sw35a`) anchor two
+fabrics; everything downstream is field switches daisy-chained through
+patch panels and fiber splice modules. A handful of switches sit outside
+both fabrics for a documented reason (spare/inventory, decommissioning, or
+planned/not-yet-installed) -- those are called out in their own box on the
+diagram rather than silently dropped.
+
+Connectors in the `.drawio` file are glued by shape id (draw.io's native
+`source`/`target` reference on each edge) -- drag a switch or panel in the
+app and its links follow. That's different from this project's earlier
+Visio (`.vsdx`) attempt, where glue had to be hand-written as cell formulas
+because the format has no simpler mechanism; draw.io does this natively,
+which is a big part of why this repo uses it instead.
+
+## Regenerating
+
+From the NetBox host (needs `docker exec` access to the NetBox container):
+
+```sh
+docker exec netbox-netbox-1 python3 /opt/netbox/netbox/manage.py shell \
+    -c "$(cat topology/export_topology.py)" 2>/dev/null > topology/topo_full.json
+
+python3 topology/generate_drawio.py topology/topo_full.json topology/Field-Switch-Topology.drawio
+```
+
+Commit both files together. The diff on `topo_full.json` is the real
+changelog -- it shows exactly which device or cable changed between two
+snapshots; the `.drawio` diff is mostly noise (coordinates) but is kept in
+sync so the rendered diagram never drifts from what NetBox actually says.
+
+## Gotchas learned the hard way
+
+- **NetBox's cached cable path can go stale.** If a switch's
+  `connected_endpoints` comes back empty even though it's obviously
+  cabled, try `manage.py trace_paths --force` before assuming the cable
+  data itself is wrong -- it recomputes the path cache and doesn't touch
+  any cable/termination records.
+- **A patch panel's front&harr;rear pass-through has to be modeled
+  explicitly** (a `PortMapping` row). A splice module can have a
+  perfectly good cable on both sides of a port and still fail to trace if
+  that mapping is missing -- check `FrontPort.objects.get(...).mappings.count()`
+  before assuming the far end really is disconnected.
+
+## Possible next step
+
+`export_topology.py` talks to NetBox via the Django ORM (`docker exec` +
+`manage.py shell`), which only works run from the NetBox host itself.
+Swapping it for a NetBox REST API client (device/cable/interface
+endpoints + an API token) would let this regenerate from anywhere,
+including CI, without shell access to the container.
@@ -0,0 +1,68 @@
+"""
+Run INSIDE the NetBox Django shell to export the field-switch topology
+(switches + every patch panel / splice module they physically pass through)
+as JSON, for generate_drawio.py to render.
+
+Usage (from the NetBox host, e.g. /opt/netbox):
+
+    docker exec netbox-netbox-1 python3 /opt/netbox/netbox/manage.py shell \
+        -c "$(cat topology/export_topology.py)" 2>/dev/null > topology/topo_full.json
+
+The script only reads data -- it never modifies NetBox. If a switch's
+connected_endpoints comes back empty when it clearly shouldn't, that is a
+NetBox cable-path bug, not a bug in this script -- see the "gotchas" section
+of the repo README before assuming the physical link doesn't exist.
+"""
+from dcim.models import Device, Interface
+import json
+
+switch_ids = set(
+    Device.objects.filter(
+        role__slug__in=["core-switch", "dist-switch", "field-switch"]
+    ).values_list("id", flat=True)
+)
+
+ifaces = Interface.objects.filter(
+    device_id__in=switch_ids, cable__isnull=False
+).select_related("device")
+
+touched_devices = set(switch_ids)
+edges = set()  # (device_id_a, device_id_b), deduplicated, any two adjacent devices on a physical chain
+
+for i in ifaces:
+    path = i._path
+    if not path or not path.is_active:
+        continue
+    chain = []
+    for hop in i.trace():
+        for item in hop:
+            for t in item:
+                dev = getattr(t, "device", None)
+                if dev:
+                    if not chain or chain[-1] != dev.id:
+                        chain.append(dev.id)
+                    touched_devices.add(dev.id)
+    for a, b in zip(chain, chain[1:]):
+        if a != b:
+            edges.add(tuple(sorted((a, b))))
+
+devs = Device.objects.filter(id__in=touched_devices).select_related(
+    "site", "role", "device_type"
+)
+
+out = {
+    "devices": [
+        {
+            "id": d.id,
+            "name": d.name,
+            "role": d.role.slug,
+            "role_name": d.role.name,
+            "site": d.site.name if d.site else "",
+            "status": str(d.status),
+            "device_type": d.device_type.model,
+        }
+        for d in devs
+    ],
+    "edges": [[a, b] for a, b in sorted(edges)],
+}
+print(json.dumps(out))
@@ -0,0 +1,266 @@
+#!/usr/bin/env python3
+"""
+Render topo_full.json (produced by export_topology.py) as a draw.io
+(mxGraph) diagram: Field-Switch-Topology.drawio.
+
+Edges reference shapes by id (source=/target=), which is how draw.io's
+native glue works -- no hand-rolled formulas needed: dragging a shape in
+the app keeps every attached line attached.
+
+Usage:
+    python3 generate_drawio.py [topo_full.json] [Field-Switch-Topology.drawio]
+"""
+import json, sys, uuid
+from xml.sax.saxutils import escape
+import xml.dom.minidom as minidom
+
+IN_PATH = sys.argv[1] if len(sys.argv) > 1 else "topo_full.json"
+OUT_PATH = sys.argv[2] if len(sys.argv) > 2 else "Field-Switch-Topology.drawio"
+
+data = json.load(open(IN_PATH))
+devs = {d["id"]: d for d in data["devices"]}
+edges = data["edges"]
+
+# ---------------- components ----------------
+adj = {d: [] for d in devs}
+for a, b in edges:
+    adj[a].append(b)
+    adj[b].append(a)
+
+parent = {d: d for d in devs}
+def find(x):
+    while parent[x] != x:
+        parent[x] = parent[parent[x]]
+        x = parent[x]
+    return x
+def union(a, b):
+    ra, rb = find(a), find(b)
+    if ra != rb:
+        parent[ra] = rb
+for a, b in edges:
+    union(a, b)
+from collections import defaultdict
+comps = defaultdict(list)
+for d in devs:
+    comps[find(d)].append(d)
+comps = sorted(comps.values(), key=len, reverse=True)
+main_comp = comps[0]   # fche fabric, rooted at id 1
+dmo_comp = comps[1]    # dmo-lob fabric, rooted at id 7
+iso = [c[0] for c in comps[2:]]
+
+# ---------------- tidy tree layout ----------------
+def layout_tree(root, node_ids):
+    node_set = set(node_ids)
+    depth, yslot, visited = {}, {}, set()
+    leaf_counter = [0]
+    def dfs(node, par, d):
+        visited.add(node)
+        depth[node] = d
+        children = [n for n in sorted(adj[node]) if n in node_set and n != par and n not in visited]
+        if not children:
+            y = leaf_counter[0]; leaf_counter[0] += 1
+            yslot[node] = y
+            return y
+        ys = [dfs(c, node, d + 1) for c in children]
+        y = sum(ys) / len(ys)
+        yslot[node] = y
+        return y
+    dfs(root, None, 0)
+    return depth, yslot
+
+depth_main, yslot_main = layout_tree(1, main_comp)
+depth_dmo, yslot_dmo = layout_tree(7, dmo_comp)
+
+# ---------------- geometry constants (px) ----------------
+SWITCH_ICON = 46          # icon box; label renders below it
+SWITCH_LABEL_H = 30
+PANEL_W, PANEL_H = 130, 34
+COL_W = 190
+ROW_H = 82
+PAD = 40
+LABEL_STRIP = 34
+SECTION_GAP = 70
+MARGIN = 40
+
+def footprint(dev):
+    if dev["role"] == "patch-panel":
+        return PANEL_W, PANEL_H
+    return SWITCH_ICON, SWITCH_ICON + SWITCH_LABEL_H
+
+pos = {}  # device_id -> (x, y) TOP-LEFT of the node's own bounding box (not center)
+
+def place_section(depth_map, yslot_map, y_offset):
+    ys = list(yslot_map.values())
+    min_y = min(ys)
+    max_x = max(depth_map.values())
+    for node, d in depth_map.items():
+        y = yslot_map[node]
+        w, h = footprint(devs[node])
+        cx = MARGIN + PAD + d * COL_W + SWITCH_ICON / 2
+        cy = y_offset + PAD + LABEL_STRIP + (y - min_y) * ROW_H + SWITCH_ICON / 2
+        pos[node] = (cx - w / 2, cy - h / 2, w, h)
+    width = MARGIN * 2 + PAD * 2 + max_x * COL_W + SWITCH_ICON
+    height = PAD * 2 + LABEL_STRIP + (max(ys) - min_y) * ROW_H + SWITCH_ICON + SWITCH_LABEL_H
+    return width, height
+
+TITLE_H = 60
+cursor_y = MARGIN + TITLE_H
+
+w_main, h_main = place_section(depth_main, yslot_main, cursor_y)
+main_top = cursor_y
+cursor_y += h_main + SECTION_GAP
+
+w_dmo, h_dmo = place_section(depth_dmo, yslot_dmo, cursor_y)
+dmo_top = cursor_y
+cursor_y += h_dmo + SECTION_GAP
+
+ISO_COLS = 3
+iso_top = cursor_y
+iso_sorted = sorted(iso, key=lambda d: devs[d]["name"])
+for i, node in enumerate(iso_sorted):
+    r, c = divmod(i, ISO_COLS)
+    w, h = footprint(devs[node])
+    cx = MARGIN + PAD + c * COL_W + SWITCH_ICON / 2
+    cy = iso_top + PAD + LABEL_STRIP + r * ROW_H + SWITCH_ICON / 2
+    pos[node] = (cx - w / 2, cy - h / 2, w, h)
+iso_rows = (len(iso_sorted) + ISO_COLS - 1) // ISO_COLS
+w_iso = MARGIN * 2 + PAD * 2 + (ISO_COLS - 1) * COL_W + SWITCH_ICON
+h_iso = PAD * 2 + LABEL_STRIP + (iso_rows - 1) * ROW_H + SWITCH_ICON + SWITCH_LABEL_H
+
+PAGE_W = max(w_main, w_dmo, w_iso)
+PAGE_H = cursor_y + h_iso + MARGIN
+
+# ---------------- style palette ----------------
+STYLE = {
+    "dist":    dict(fill="#2F5FAD", stroke="#173D6E", font="#FFFFFF", dashed=0, bold=1),
+    "field":   dict(fill="#F3F5FA", stroke="#7C8AA6", font="#1A2233", dashed=0, bold=0),
+    "spare":   dict(fill="#F1F1F2", stroke="#9AA0AA", font="#5B6478", dashed=1, bold=0),
+    "decom":   dict(fill="#FBE6E2", stroke="#C04B34", font="#8A3423", dashed=1, bold=0),
+    "planned": dict(fill="#FDF1CF", stroke="#B8862C", font="#7A5C00", dashed=1, bold=0),
+    "panel":   dict(fill="#EEF0F2", stroke="#8C97A6", font="#495566", dashed=0, bold=0),
+}
+
+def style_for(dev):
+    if dev["role"] == "patch-panel":
+        return STYLE["panel"]
+    if dev["role"] == "dist-switch":
+        return STYLE["dist"]
+    if dev["status"] == "inventory":
+        return STYLE["spare"]
+    if dev["status"] == "decommissioning":
+        return STYLE["decom"]
+    if dev["status"] == "planned":
+        return STYLE["planned"]
+    return STYLE["field"]
+
+# ---------------- mxGraph XML ----------------
+cells = []
+def add_cell(xml):
+    cells.append(xml)
+
+def vertex(cell_id, value, x, y, w, h, style, parent="1"):
+    add_cell(
+        f'<mxCell id="{cell_id}" value="{escape(value)}" style="{style}" '
+        f'vertex="1" parent="{parent}">'
+        f'<mxGeometry x="{x:.1f}" y="{y:.1f}" width="{w:.1f}" height="{h:.1f}" as="geometry"/>'
+        f'</mxCell>'
+    )
+
+def edge(cell_id, source, target, style):
+    add_cell(
+        f'<mxCell id="{cell_id}" style="{style}" edge="1" parent="1" '
+        f'source="{source}" target="{target}"><mxGeometry relative="1" as="geometry"/></mxCell>'
+    )
+
+def container(cell_id, x, y, w, h, label):
+    add_cell(
+        f'<mxCell id="{cell_id}" value="" style="rounded=1;arcSize=3;fillColor=#F7F8FB;'
+        f'strokeColor=#DDE1EA;verticalAlign=top;" vertex="1" parent="1">'
+        f'<mxGeometry x="{x:.1f}" y="{y:.1f}" width="{w:.1f}" height="{h:.1f}" as="geometry"/></mxCell>'
+    )
+    add_cell(
+        f'<mxCell id="{cell_id}-label" value="{escape(label)}" '
+        f'style="text;html=1;align=left;verticalAlign=top;fontStyle=1;fontSize=13;fontColor=#33415C;" '
+        f'vertex="1" parent="1"><mxGeometry x="{x+14:.1f}" y="{y+8:.1f}" width="{w-28:.1f}" height="22" as="geometry"/></mxCell>'
+    )
+
+container("bg-main", MARGIN, main_top, w_main - MARGIN * 2, h_main,
+           f"FCHE Cable Headend — distribution fabric  ({len(main_comp)} devices)")
+container("bg-dmo", MARGIN, dmo_top, w_dmo - MARGIN * 2, h_dmo,
+           f"DMO-LOB — distribution fabric  ({len(dmo_comp)} devices)")
+container("bg-iso", MARGIN, iso_top, w_iso - MARGIN * 2, h_iso,
+           "Not in a resolved cable path")
+
+# title
+add_cell(
+    f'<mxCell id="title" value="Field Switch Topology" style="text;html=1;align=left;'
+    f'verticalAlign=top;fontStyle=1;fontSize=22;fontColor=#161B26;" vertex="1" parent="1">'
+    f'<mxGeometry x="{MARGIN}" y="10" width="600" height="30" as="geometry"/></mxCell>'
+)
+
+# node shapes -- switches get a real network-switch icon (label rendered below it by
+# draw.io's own verticalLabelPosition, not baked into the geometry), patch panels get
+# a plain labeled box.
+node_style = {}
+for did, dev in devs.items():
+    if did not in pos:
+        continue
+    x, y, w, h = pos[did]
+    st = style_for(dev)
+    dashed = "dashed=1;dashPattern=4 3;" if st["dashed"] else ""
+    if dev["role"] == "patch-panel":
+        style = (
+            f'rounded=0;whiteSpace=wrap;html=1;fillColor={st["fill"]};strokeColor={st["stroke"]};'
+            f'fontColor={st["font"]};fontSize=9;{dashed}'
+        )
+        vertex(f"n{did}", dev["name"], x, y, w, h, style)
+    else:
+        style = (
+            'shape=mxgraph.networks.switch;html=1;verticalLabelPosition=bottom;verticalAlign=top;'
+            f'labelBackgroundColor=none;fontSize=10;fontColor={st["font"] if st["font"] != "#FFFFFF" else "#1A2233"};'
+            f'fillColor={st["fill"]};strokeColor={st["stroke"]};fontStyle={st["bold"]};{dashed}'
+        )
+        vertex(f"n{did}", dev["name"], x, y, SWITCH_ICON, SWITCH_ICON, style)
+
+# connectors -- glued natively via source/target, orthogonal routing
+for idx, (a, b) in enumerate(edges):
+    if a in pos and b in pos:
+        edge(f"e{idx}", f"n{a}", f"n{b}",
+             "edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;strokeColor=#9AA6B8;strokeWidth=1.2;"
+             "jettySize=auto;orthogonalLoop=1;")
+
+# legend
+legend_items = [
+    ("#2F5FAD", "#173D6E", "Distribution switch"),
+    ("#F3F5FA", "#7C8AA6", "Field switch"),
+    ("#EEF0F2", "#8C97A6", "Patch panel / splice module"),
+    ("#F1F1F2", "#9AA0AA", "Spare / decommissioned / planned"),
+]
+lx = MARGIN
+ly = 44
+for i, (fill, stroke, label) in enumerate(legend_items):
+    add_cell(
+        f'<mxCell id="lg{i}" value="" style="rounded=0;fillColor={fill};strokeColor={stroke};" '
+        f'vertex="1" parent="1"><mxGeometry x="{lx}" y="{ly}" width="14" height="14" as="geometry"/></mxCell>'
+    )
+    add_cell(
+        f'<mxCell id="lg{i}-t" value="{escape(label)}" style="text;html=1;align=left;'
+        f'verticalAlign=middle;fontSize=11;fontColor=#3A4457;" vertex="1" parent="1">'
+        f'<mxGeometry x="{lx+20}" y="{ly-3}" width="220" height="20" as="geometry"/></mxCell>'
+    )
+    lx += 260
+
+xml = (
+    '<mxfile host="app.diagrams.net" agent="netbox-topology-generator" version="24.0.0">'
+    f'<diagram name="Topology" id="{uuid.uuid4()}">'
+    f'<mxGraphModel dx="800" dy="600" grid="1" gridSize="10" guides="1" tooltips="1" '
+    f'connect="1" arrows="1" fold="1" page="1" pageScale="1" '
+    f'pageWidth="{PAGE_W:.0f}" pageHeight="{PAGE_H:.0f}" math="0" shadow="0">'
+    '<root><mxCell id="0"/><mxCell id="1" parent="0"/>'
+    + "".join(cells) +
+    '</root></mxGraphModel></diagram></mxfile>'
+)
+
+minidom.parseString(xml)  # fail fast on malformed XML
+open(OUT_PATH, "w").write(xml)
+print(f"wrote {OUT_PATH}: {len(cells)} cells, page {PAGE_W:.0f}x{PAGE_H:.0f}px")