Monitoring & Observability

• Built the company's first observability dashboard

  Developed and launched the company's first observability dashboard, providing real‑time system performance insights and data visualization on the large office TV.

• Ran 10 PostgreSQL and MS SQL servers on Ubuntu

  Designed, deployed, and maintained 10 PostgreSQL and MS SQL servers on Ubuntu Linux VPS, ensuring optimal server performance and reliability.

• Automated delivery of 20 GIS data pipelines

  Automated delivery of 20 GIS data pipelines and app data ETL processes, streamlining infrastructure automation and reporting.

• Cut operational risk with Grafana and Prometheus

  Mitigated operational risks by implementing a monitoring dashboard using Grafana and Prometheus, improving system reliability.

• Ran 24/7 support for ~1,000 IPTV/OTT servers

  Provided round‑the‑clock 24/7 infrastructure support for an IPTV/OTT streaming platform, administering ~1,000 servers plus client‑owned systems for global customers in China, the US and Germany.

• Kept IPTV streaming signals running non‑stop

  Ensured uninterrupted delivery of IPTV streaming signals between suppliers and clients, monitoring and maintaining the streaming network and IP telephony around the clock.

• Built first‑party error monitoring and tracing

  Built first‑party error monitoring and OpenTelemetry tracing rather than buying them — payload sanitising, spike and regression detection, symbolication and a synthetic heartbeat — behind 11 operator views.

• Three locks on the door, none of them locked

  Found and closed three SSH brute‑force protections that had never worked: a ban jail watching port 22 while the daemon listened on 1986, a rate limit shadowed by a broader rule above it, and a ban action whose binary never resolved, so no ban had ever applied.

• Built dead‑man's‑switch host monitoring

  Built dead‑man's‑switch monitoring that pings only while memory and disk are healthy, so a degraded host raises an alert by going silent — and caught six variable names saying "free" where the check correctly measured "available", an order of magnitude apart on a 464 MB box.

• Built 7 read‑only host reporting roles

  Built seven read‑only reporting roles that render a live host to Markdown — facts, access, git, metrics, traffic, security and provider inventory — under a rule that no number is printed the run did not measure.

• An app that ate 41 MB a second, stopped

  Stopped an application filling memory at 41 MB a second — a recorded 111 GB of compressed pages on a 36 GB machine — by bounding every event stream, subscribing by event type and putting a rate budget on logging, taking 610,996 log lines down to 1,411.

• Measured 13 addresses with no tracking

  Measured thirteen addresses from the web server's own access log alone — no cookie, no tracker and no third‑party service — masking the client network as each line is written, and folding the result into a permanent archive of 21 daily counts and 20 monthly facets.

• Made six collectors state what they skipped

  Made six reporting collectors state the part of their input they never read — files that stayed shut, probes not installed, journal lines unrecognised — and fitted thresholds to 116 measured samples, after establishing that a blind collection and a healthy host render the same page.

This work is part of what we offer as Site Reliability & Monitoring, System Administration, DevOps & CI/CD Automation, Custom AI Agent Development, Networking & VPN Setup, Security & Access Management, Technical Documentation and AI Agent Security.

Building something that needs this? Get in touch.