rv50x-manager/README-docker.md

# RV50x Template Manager — Docker Edition

A containerized web-based tool for managing configuration templates on Sierra Wireless AirLink RV50x modems. Runs as a full Docker stack including the web application, NocoDB, and PostgreSQL — start and stop the entire thing with a single command.

---

## Table of Contents

1. [Stack Overview](#stack-overview)
2. [Requirements](#requirements)
3. [Project Structure](#project-structure)
4. [First-Time Setup](#first-time-setup)
5. [The .env File](#the-env-file)
6. [Building and Starting](#building-and-starting)
7. [NocoDB Setup](#nocodb-setup)
8. [Updating the App](#updating-the-app)
9. [Managing the Stack](#managing-the-stack)
10. [Using Portainer](#using-portainer)
11. [Migrating to a New Machine](#migrating-to-a-new-machine)
12. [Reconnecting to a New NocoDB Instance](#reconnecting-to-a-new-nocodb-instance)
13. [Data and Volumes](#data-and-volumes)
14. [Troubleshooting](#troubleshooting)

---

## Stack Overview

The stack runs three containers:

| Container | Image | Purpose | Default Port |
|---|---|---|---|
| `rv50x-manager` | Built from `Dockerfile` | FastAPI web app + Playwright | Your choice |
| `rv50x-nocodb` | `nocodb/nocodb:latest` | NocoDB UI and API | 8090 |
| `rv50x-postgres` | `postgres:16-alpine` | PostgreSQL database for NocoDB | Internal only |

All three containers communicate over a private internal Docker network. PostgreSQL is never exposed to the host — only NocoDB can reach it. The web app talks to NocoDB via the internal hostname `nocodb`.

```
Your Browser
     │
     ▼
rv50x-manager (YOUR_PORT) ──→ nocodb (8090) ──→ postgres (internal)
     │
     ▼
Your Modems (port 443, via Playwright)
```

---

## Requirements

- Docker Engine 24+ or Docker Desktop
- Docker Compose v2 (`docker compose` or `docker-compose`)
- Network access from the Docker host to your modems on port 443
- Portainer (optional, but recommended for easy management)

---

## Project Structure

```
/opt/rv50x-manager/
├── app.py                  ← FastAPI backend
├── index.html              ← Web UI
├── requirements.txt        ← Python dependencies
├── Dockerfile              ← Container build instructions
├── docker-compose.yml      ← Stack definition
├── .env                    ← Your secrets and config (never commit this)
├── .env.example            ← Template for .env
└── .dockerignore           ← Files excluded from the Docker image
```

**Not needed in the Docker folder** (kept separately if you want CLI access):
- `download.py` / `upload.py` — standalone CLI scripts
- `modems.csv` — legacy device list
- `.venv/` — Python virtual environment

---

## First-Time Setup

### Step 1 — Install Docker

**Fedora / RHEL / Rocky:**
```bash
sudo dnf install -y docker docker-compose-plugin
sudo systemctl start docker
sudo systemctl enable docker
sudo usermod -aG docker $USER   # log out and back in after this
```

**Debian / Ubuntu:**
```bash
sudo apt-get install -y docker.io docker-compose-plugin
sudo systemctl start docker
sudo systemctl enable docker
sudo usermod -aG docker $USER
```

Verify Docker is working:
```bash
docker run hello-world
```

### Step 2 — Copy project files to the host machine

Create the install directory and set permissions, then copy your files:

```bash
# Create the directory and give your user ownership
sudo mkdir -p /opt/rv50x-manager
sudo chown $USER:$USER /opt/rv50x-manager

# Copy all project files into it
cp app.py index.html requirements.txt Dockerfile \
   docker-compose.yml .env.example .dockerignore \
   /opt/rv50x-manager/

cd /opt/rv50x-manager
```

### Step 3 — Create your .env file

```bash
cd /opt/rv50x-manager
cp .env.example .env
nano .env    # or use any text editor
```

See [The .env File](#the-env-file) section below for what to put in each field.

### Step 4 — Set your port

Open `/opt/rv50x-manager/docker-compose.yml` and find this line under `rv50x-manager`:

```yaml
ports:
  - "YOUR_PORT:8000"
```

Replace `YOUR_PORT` with the port number you want to use on the host machine. For example:

```yaml
ports:
  - "8001:8000"
```

The app will then be accessible at `http://host-ip:8001`.

### Step 5 — Build and start

```bash
cd /opt/rv50x-manager
docker-compose up -d
```

This builds the `rv50x-manager` image and starts all three containers. The first build takes a few minutes because it downloads the Playwright base image and installs Chromium.

### Step 6 — Set up NocoDB

See [NocoDB Setup](#nocodb-setup) below.

### Step 7 — Update .env with NocoDB IDs

After importing your data into NocoDB, update `.env` with the real IDs and restart the manager:

```bash
docker-compose restart rv50x-manager
```

### Step 8 — Open the app

```
http://your-docker-host-ip:YOUR_PORT
```

---

## The .env File

Create this file by copying `.env.example` and filling in your values. It is read automatically by `docker-compose`. **Never commit this file to version control** — it contains passwords and API tokens.

```bash
# ── NocoDB connection ──────────────────────────────────────────────────────
#
# Use "http://nocodb:8080" to connect to the NocoDB container in this stack.
# Use your external NocoDB URL if you prefer to point at an existing instance.
NOCODB_URL=http://nocodb:8080

# Your NocoDB API token.
# Get it from: NocoDB → Profile (bottom-left) → Team & Settings → API Tokens
# Tokens look like: eWU_ilelaCtNy1JzC7vf41DokkqFOovcLHM0zVml
NOCODB_TOKEN=your-api-token-here

# These three IDs come from the NocoDB browser URL after you import your data.
# The URL structure is:
#   http://host:8090/{org_id}/{BASE_ID}/{TABLE_ID}/{VIEW_ID}/table-name
# Leave as placeholder for now — update after NocoDB setup (Step 6 above).
NOCODB_BASE_ID=your-base-id-here
NOCODB_TABLE_ID=your-table-id-here
NOCODB_VIEW_ID=your-view-id-here

# ── PostgreSQL ─────────────────────────────────────────────────────────────
# Internal password used between NocoDB and PostgreSQL containers only.
# You will never need to type this manually — make it long and strong.
POSTGRES_PASSWORD=SomeLongStrongPassword123!

# ── NocoDB JWT secret ──────────────────────────────────────────────────────
# Any long random string used to sign NocoDB authentication tokens.
# Generate one with: openssl rand -hex 32
NC_JWT_SECRET=paste-a-long-random-string-here

# ── Playwright timeouts (optional) ────────────────────────────────────────
# Uncomment and adjust if your modems are slow to respond. Values in ms.
# PAGE_TIMEOUT=90000
# DOWNLOAD_TIMEOUT=120000
# UPLOAD_TIMEOUT=120000
# MAX_RETRIES=3
```

### Generating secrets

```bash
# Generate a strong JWT secret
openssl rand -hex 32

# Generate a strong password
openssl rand -base64 24
```

---

## Building and Starting

### Build the image

```bash
docker-compose build
```

Only needed when `app.py`, `index.html`, or `requirements.txt` change. Skipped automatically if you just change `.env`.

### Start the stack

```bash
docker-compose up -d
```

The `-d` flag runs containers in the background (detached mode). Without it the logs stream to your terminal and the stack stops when you close the terminal.

### Check that everything is running

```bash
docker-compose ps
```

You should see all three containers with status `running` or `healthy`.

### View logs

```bash
# All containers
docker-compose logs -f

# Just the web app
docker-compose logs -f rv50x-manager

# Just NocoDB
docker-compose logs -f rv50x-nocodb
```

### Stop the stack

```bash
docker-compose stop
```

Stops all containers. Data is preserved in volumes. Start again with `docker-compose up -d`.

### Remove containers (keep data)

```bash
docker-compose down
```

Removes containers but keeps all named volumes (your modem data, templates, downloads). Safe to run before a rebuild.

### Remove everything including data ⚠

```bash
docker-compose down -v
```

**This deletes all volumes** including your NocoDB database and all template files. Only use this if you want a completely clean slate.

---

## NocoDB Setup

After starting the stack for the first time, NocoDB needs to be configured before the web app can use it.

### Step 1 — Open NocoDB

```
http://your-docker-host-ip:8090
```

### Step 2 — Create your account

On first launch NocoDB will prompt you to create an admin account. Use a strong password and note the credentials.

### Step 3 — Create a new base

Click **+ New Base** and name it something like `Cell Modems`.

### Step 4 — Import your modem data from CSV

1. Export your current modem data from your existing NocoDB instance: **toolbar → Download → CSV**
2. In the new NocoDB, click **+ Add or import** → **Import from CSV**
3. Upload the CSV file
4. NocoDB will auto-detect all columns including `hostname`, `ip_address`, `dept`, `password`, etc.
5. Confirm the import

### Step 5 — Recreate filtered views

The CSV import brings the data but not the views. Recreate them manually:

**Electric view:**
1. In the left sidebar, click **+ Add View** → **Grid**
2. Name it `Electric`
3. Click **Filter** → **+ Add Filter**
4. Set: `dept` `is` `ELEC`

**Gas & Water view:**
1. Click **+ Add View** → **Grid**
2. Name it `Gas & Water`
3. Click **Filter** → **+ Add Filter**
4. Set: `dept` `is` `GW`

### Step 6 — Get the IDs from the browser URL

Navigate to your Cell Modems table. The URL will look like:

```
http://host:8090/abc123def/BASE_ID_HERE/TABLE_ID_HERE/VIEW_ID_HERE/cell-modems
```

Copy `BASE_ID_HERE`, `TABLE_ID_HERE`, and `VIEW_ID_HERE` (use the All view ID).

### Step 7 — Generate an API token

1. Click your profile avatar (bottom-left)
2. Go to **Team & Settings → API Tokens**
3. Click **Add Token**, give it a name like `rv50x-manager`
4. Copy the token

### Step 8 — Update .env and restart

```bash
cd /opt/rv50x-manager
nano .env
# Update NOCODB_TOKEN, NOCODB_BASE_ID, NOCODB_TABLE_ID, NOCODB_VIEW_ID

docker-compose restart rv50x-manager
```

### Step 9 — Verify the connection

Open the web app and check that the device groups show the correct counts. If devices appear, the connection is working.

---

## Updating the App

When `app.py` or `index.html` change:

```bash
# Stop the stack
docker-compose stop

# Rebuild the manager image (NocoDB and Postgres don't need rebuilding)
docker-compose build rv50x-manager

# Start everything back up
docker-compose up -d
```

When only `.env` changes (NocoDB IDs, token, timeouts):

```bash
docker-compose restart rv50x-manager
```

When `requirements.txt` changes (new Python packages):

```bash
docker-compose build rv50x-manager
docker-compose up -d
```

---

## Managing the Stack

### Start individual services

```bash
docker-compose up -d postgres    # start just postgres
docker-compose up -d nocodb      # start just nocodb
docker-compose up -d rv50x-manager  # start just the web app
```

### Restart a single service

```bash
docker-compose restart rv50x-manager
```

### Rebuild and restart a single service

```bash
docker-compose up -d --build rv50x-manager
```

### Execute a command inside a running container

```bash
# Open a shell in the web app container
docker exec -it rv50x-manager bash

# Check Python packages installed
docker exec rv50x-manager pip list

# Test NocoDB connection from inside the container
docker exec rv50x-manager curl -s http://nocodb:8080/api/v1/health
```

### Access the PostgreSQL database directly

```bash
docker exec -it rv50x-postgres psql -U nocodb -d nocodb
```

---

## Using Portainer

Portainer gives you a browser-based UI to manage the entire stack without needing SSH.

### Install Portainer (if not already installed)

```bash
docker volume create portainer_data
docker run -d \
  -p 9000:9000 \
  --name portainer \
  --restart=always \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v portainer_data:/data \
  portainer/portainer-ce:latest
```

Then open `http://host-ip:9000` and create your admin account.

### Deploy the stack via Portainer

1. Go to **Stacks → + Add Stack**
2. Name it `rv50x`
3. Paste the contents of `docker-compose.yml` into the editor
4. Scroll down to **Environment Variables**
5. Click **+ Add an environment variable** for each line in your `.env` file:

| Name | Value |
|---|---|
| `NOCODB_URL` | `http://nocodb:8080` |
| `NOCODB_TOKEN` | your token |
| `NOCODB_BASE_ID` | your base ID |
| `NOCODB_TABLE_ID` | your table ID |
| `NOCODB_VIEW_ID` | your view ID |
| `POSTGRES_PASSWORD` | your password |
| `NC_JWT_SECRET` | your secret |

6. Click **Deploy the stack**

### Start and stop via Portainer

- **Stacks → rv50x → Start** — starts all containers
- **Stacks → rv50x → Stop** — stops all containers
- Individual containers: **Containers** list → click the start/stop icons

### Update the stack via Portainer

1. **Stacks → rv50x → Editor**
2. Paste updated `docker-compose.yml`
3. Click **Update the stack**

For code changes (`app.py`, `index.html`), you need to rebuild the image first from the command line:

```bash
docker-compose build rv50x-manager
```

Then update the stack in Portainer to pick up the new image.

---

## Migrating to a New Machine

### Step 1 — Export your NocoDB data

Before migrating, export your modem data as CSV from the current NocoDB instance so you can reimport it on the new machine.

In NocoDB: **Cell Modems table → toolbar → Download → CSV**

### Step 2 — Copy project files

From the old machine:

```bash
scp app.py index.html requirements.txt Dockerfile docker-compose.yml \
    .env.example .dockerignore youruser@new-host:/tmp/rv50x-transfer/
```

**Do not copy `.env`** over an insecure connection — recreate it manually on the new machine.

On the new machine, move files into place:

```bash
sudo mkdir -p /opt/rv50x-manager
sudo chown $USER:$USER /opt/rv50x-manager
cp /tmp/rv50x-transfer/* /opt/rv50x-manager/
```

### Step 3 — Copy template files (optional)

If you want to keep your existing XML templates, downloaded configs, and upload files:

```bash
# Export volumes from the old machine
cd /opt/rv50x-manager

docker run --rm \
  -v rv50x-manager_xml_templates:/data \
  -v $(pwd):/backup \
  alpine tar czf /backup/xml_templates.tar.gz -C /data .

docker run --rm \
  -v rv50x-manager_template_downloads:/data \
  -v $(pwd):/backup \
  alpine tar czf /backup/template_downloads.tar.gz -C /data .

docker run --rm \
  -v rv50x-manager_template_uploads:/data \
  -v $(pwd):/backup \
  alpine tar czf /backup/template_uploads.tar.gz -C /data .

# Copy archives to new machine
scp xml_templates.tar.gz template_downloads.tar.gz template_uploads.tar.gz \
    youruser@new-host:/opt/rv50x-manager/
```

### Step 4 — Set up the new machine

```bash
# Install Docker (see First-Time Setup above)

# Files should already be in /opt/rv50x-manager from Step 2
cd /opt/rv50x-manager

# Create fresh .env with new passwords and secrets
cp .env.example .env
nano .env

# Set your port
nano docker-compose.yml

# Build and start
docker-compose up -d
```

### Step 5 — Restore template files (if copied)

```bash
cd /opt/rv50x-manager

# Restore xml_templates
docker run --rm \
  -v rv50x-manager_xml_templates:/data \
  -v $(pwd):/backup \
  alpine tar xzf /backup/xml_templates.tar.gz -C /data

# Restore template_downloads
docker run --rm \
  -v rv50x-manager_template_downloads:/data \
  -v $(pwd):/backup \
  alpine tar xzf /backup/template_downloads.tar.gz -C /data

# Restore template_uploads
docker run --rm \
  -v rv50x-manager_template_uploads:/data \
  -v $(pwd):/backup \
  alpine tar xzf /backup/template_uploads.tar.gz -C /data
```

### Step 6 — Set up NocoDB on the new machine

Follow the [NocoDB Setup](#nocodb-setup) section — import your CSV, recreate views, get new IDs.

### Step 7 — Update .env with new NocoDB IDs

```bash
nano .env
docker-compose restart rv50x-manager
```

---

## Reconnecting to a New NocoDB Instance

If your NocoDB database is corrupted, rebuilt, or moved to a new server:

### Step 1 — Get the new connection details

1. Open the new NocoDB in your browser
2. Import your modem data from CSV
3. Recreate the Electric and Gas & Water filtered views
4. Go to **Profile → Team & Settings → API Tokens** → create a new token
5. Copy the base ID, table ID, and view ID from the browser URL

### Step 2 — Test the connection

```bash
curl -H "xc-token: YOUR_NEW_TOKEN" \
  "http://new-nocodb-host:8090/api/v1/db/data/noco/BASE_ID/TABLE_ID?limit=1"
```

A successful response returns JSON with a `list` array containing your first modem row.

### Step 3 — Update .env

```bash
cd /opt/rv50x-manager
nano .env
```

Update these values:

```bash
NOCODB_URL=http://new-host:8090        # or http://nocodb:8080 if using the stack
NOCODB_TOKEN=your-new-token
NOCODB_BASE_ID=your-new-base-id
NOCODB_TABLE_ID=your-new-table-id
NOCODB_VIEW_ID=your-new-view-id
```

### Step 4 — Restart the manager

```bash
cd /opt/rv50x-manager
docker-compose restart rv50x-manager
```

Or in Portainer: **Stacks → rv50x → rv50x-manager → Restart**

---

## Data and Volumes

All persistent data lives in Docker named volumes. They survive `docker-compose down` and rebuilds, and are only deleted with `docker-compose down -v`.

| Volume | Contents | Maps to container path |
|---|---|---|
| `rv50x_template_manager_postgres_data` | NocoDB database | `/var/lib/postgresql/data` |
| `rv50x_template_manager_nocodb_data` | NocoDB config and uploads | `/usr/app/data` |
| `rv50x_template_manager_template_downloads` | Downloaded modem configs + reports | `/data/template_downloads` |
| `rv50x_template_manager_template_uploads` | Staged XML files for upload + reports | `/data/template_uploads` |
| `rv50x_template_manager_xml_templates` | Your XML builder templates | `/data/xml_templates` |

### List all volumes

```bash
docker volume ls | grep rv50x
```

### Back up a volume

```bash
# Backs up the xml_templates volume to a tar file in the current directory
docker run --rm \
  -v rv50x_template_manager_xml_templates:/data \
  -v $(pwd):/backup \
  alpine tar czf /backup/xml_templates_backup.tar.gz -C /data .
```

### Back up the NocoDB database

```bash
cd /opt/rv50x-manager
docker exec rv50x-postgres \
  pg_dump -U nocodb nocodb > nocodb_backup_$(date +%Y%m%d).sql
```

### Restore the NocoDB database

```bash
cd /opt/rv50x-manager
docker exec -i rv50x-postgres \
  psql -U nocodb nocodb < nocodb_backup_20260413.sql
```

---

## Troubleshooting

| Symptom | Likely cause | Fix |
|---|---|---|
| `docker-compose: command not found` | Using older Docker without compose plugin | Use `docker compose` (space not hyphen) or install `docker-compose-plugin` |
| Build fails with "no space left on device" | Docker image cache full | Run `docker system prune` to free space |
| `rv50x-manager` exits immediately after start | App crash on startup — bad .env values | Run `docker-compose logs rv50x-manager` to see the error |
| NocoDB shows "Service Unavailable" | Postgres not ready yet | Wait 30s and refresh — healthcheck retries handle this automatically |
| Web app shows 0 devices | NocoDB IDs wrong or token invalid | Test with curl (see Reconnecting section), update .env, restart manager |
| `ERR_BASE_NOT_FOUND` in curl test | Wrong NOCODB_BASE_ID | Re-read the ID from the NocoDB browser URL |
| `ERR_AUTHENTICATION_REQUIRED` | Wrong or expired token | Regenerate token in NocoDB → API Tokens |
| Modems show in All but not Electric/Gas & Water | Wrong dept field values | Check actual `dept` values in NocoDB — must be exactly `ELEC` and `GW` |
| Playwright / Chromium crashes | Missing system library | The Playwright base image should have everything — check `docker-compose logs rv50x-manager` |
| Template upload times out | Modem slow or unreachable | Increase `UPLOAD_TIMEOUT` in `.env`, restart manager |
| Can't reach NocoDB at port 8090 | Port conflict or firewall | Change the left port in `docker-compose.yml` under `nocodb:` ports |
| Can't reach web app | Port conflict or firewall | Change `YOUR_PORT` in `docker-compose.yml` |
| `permission denied` on docker commands | User not in docker group | Run `sudo usermod -aG docker $USER` then log out and back in |
| Volume data missing after `docker-compose down` | Used `down -v` by mistake | Data is gone — restore from backup or re-import CSV |