Designing and building a compliance-grade observability and audit system for a fintech, as its lead engineer

Context

GenesysFI is a US fintech. Its product, Zovox.app (formerly Pocket CFO), is an AI CFO app for entrepreneurs that syncs with users' bank accounts to track real profit, find missed tax deductions and set aside what they owe — so it handles bank-linked financial data. I joined as its sole infrastructure and security engineer and hold the CSO and GDPR DPO roles.

Problem

There was no compliance system at all — no centralised logging, no audit trail, no evidence an auditor could be shown for SOC 2, GLBA or GDPR.

Constraints

One engineer. Production traffic on blue/green Node backends that could not go down. Log storage that is legally immutable once written, so mistakes are permanent.

What I did

1. Designed the full architecture and delivered it in nine phases: centralised ELK for identity and infrastructure audit evidence, a separate Loki pipeline for application logs shipped over mTLS into immutable object storage.

Log pipeline health — shippers, Loki ingest, immutable object storage, retention and compaction, watched end to end.SOURCE: GRAFANA · SANITIZED↕ ScrollOpen fullView full screenshot

2. Built a three-layer sensitive-data guard so personal and financial fields can't reach immutable storage: an allowlist logger in the backend, collector-side redaction rules with a counter per rule, and watchers that alert when a new log field appears.
3. Mapped every compliance panel to its control — GLBA 314.4, SOC 2 CC6.1/CC6.3, GDPR Art. 17 and 30 — and documented the known defects in the evidence itself, so nothing is overstated to an auditor.

Compliance and audit — every panel mapped to a GLBA, SOC 2 or GDPR control, with known evidence defects documented alongside.SOURCE: GRAFANA · IDS, IPS AND ENDPOINTS REDACTED↕ ScrollOpen fullView full screenshot

4. Fixed a lifecycle deadlock that had parked the Elasticsearch cluster at its shard ceiling: shards 810 → 433, lifecycle errors to zero. Ran a four-tier restore drill and wrote the DR runbook.
5. Rebuilt monitoring as code: ten provisioned dashboards, six dead scrape jobs removed, silently failing health probes repaired, and a runaway container that used 327% CPU without processing a single document shut down.

Platform status — service health, availability, collector health, capacity, Elasticsearch watchdog and TLS expiry on one board.SOURCE: GRAFANA · HOSTNAMES NEUTRALISED↕ ScrollOpen fullView full screenshot

Result

An auditor-ready control and evidence system with a written control mapping, a DR runbook and continuous checks on its own pipeline, plus completed zero-trust network segmentation.

Open larger view: Control-plane host in steady state — 11% CPU, 5% load, nine weeks of uptime.SOURCE: GRAFANA · SANITIZED

Stack

• GCP
• Elasticsearch
• Kibana
• Fleet
• Loki
• Grafana Alloy
• Prometheus
• Grafana
• Keycloak
• PM2
• Node.js