ingest-monitor/README.md

Ingest Monitor

Local web monitor for inspecting JSON payloads sent to an ingest endpoint.

The app exposes POST /api/ingest, stores recent packets in memory, and streams live updates to the browser with SSE.

Requirements

• Node.js 20+

No npm dependencies are required.

Run

From Bash:

cd /c/Users/myart/Drill/greact.drill.ingest.monitor
PORT=8090 npm run start

Open:

http://localhost:8090

Send ingest payloads to:

POST http://localhost:8090/api/ingest

Configure mqtt-ingest

Point mqtt-ingest to the monitor:

export EDGE5_MODBUS_POST_URL="http://localhost:8090/api/ingest"

Then run mqtt-ingest from its app directory:

cd /c/Users/myart/Drill/mqtt-ingest/app
export MQTT_URL="mqtt://localhost:1883"
export HTTP_PORT="8080"
export WS_PORT="9090"
export EDGE5_MODBUS_POST_URL="http://localhost:8090/api/ingest"
npm run start

Make sure an MQTT broker is listening on localhost:1883.

Proxy Mode

The monitor can show payloads and forward the same JSON to a real ingest service:

PORT=8090 FORWARD_URL="https://demo.backend.drll.cloud/api/ingest" npm run start

In this mode:

mqtt-ingest -> ingest-monitor -> real ingest service
                        |
                        +-> browser UI via SSE

Environment Variables

PORT            HTTP port. Default: 8090
HOST            Bind address. Default: 0.0.0.0
FORWARD_URL     Optional upstream ingest URL for proxy mode
MAX_EVENTS      Recent packets kept in memory. Default: 500
MAX_BODY_BYTES  Max request body size. Default: 5242880

For lighter browser memory usage:

MAX_EVENTS=100 PORT=8090 npm run start

API

POST /api/ingest

Accepts either one JSON object or an array of objects.

Expected object shape:

{
  "edge": "edge5",
  "timestamp": 1780612349372,
  "tag": "modbus.1",
  "value": 197.14
}

GET /api/events

Returns the current in-memory snapshot.

DELETE /api/events

Clears monitor memory.

GET /events

SSE stream used by the browser UI.

Data Storage

Packets are stored only in the Node.js process memory. They are not written to disk or a database.

After restarting the monitor, history is cleared.

UI Notes

• Following latest keeps the selected packet pinned to the newest packet.
• Selecting a packet disables follow mode, so new packets do not steal focus.
• Expand opens the raw JSON payload in a large viewer.
• Copy copies the selected payload as formatted JSON.

Troubleshooting

If the UI is empty, check that payloads are being posted to:

http://localhost:8090/api/ingest

If mqtt-ingest cannot connect to MQTT, start Mosquitto:

mosquitto -p 1883 -v

If the monitor port is busy, choose another port:

PORT=8091 npm run start