Skip to content
Santiago Lopez de Haro
All projects
Personal
Personal project

OptiBlocks: plug-and-play backend components

An open-source collection of Go microservices for real-time UI updates, chat and document conversion, shipped as signed Docker images and ready to drop into any backend.

Source on GitHub
OptiBlocks: plug-and-play backend components

Highlights

Three independent Go microservices published as signed, multi-arch Docker images
Per-user WebSocket channels with prefix-based access control, checked before the upgrade
16 KiB per idle connection and fan-out to 2000 users with a 10.8 ms p99
Chat at 45–50k messages per second with adaptive batching and no lost acknowledgements on shutdown
Format conversion and PDF/DOCX/XLSX generation hardened against hostile input
Reproducible, scan-gated builds with SBOM, SLSA provenance and cosign signatures

OptiBlocks is an open-source collection of small, production-oriented Go microservices that drop into an existing backend as components. Each one takes a connection-heavy or CPU-bound job off the main backend (WebSocket fan-out, chat delivery, format conversion and document rendering), so a Node.js/NestJS or any other backend stays focused on business logic. Every component ships as a signed, multi-arch image on Docker Hub, is configured with a handful of environment variables and is integrated over plain HTTP and WebSockets.

OptiBlocks collection architecture: web and mobile clients reach the product backend, the real-time chat and the real-time UI update services through a TLS edge; the backend publishes UI events with a time token and calls the document module on a private network; chat persists to MongoDB.
Collection architecture: where each component sits next to a product backend. Click to open at full resolution.

Why components

  • Keep the event loop free: holding thousands of sockets or rendering a 5 MiB spreadsheet inside a Node.js process competes with every API request. A Go service built for that one job does it with a fraction of the memory.
  • Small, documented interfaces: a few HTTP routes and one WebSocket endpoint per service, with every status and close code documented.
  • Authorisation stays in the product: the components verify tokens the product backend mints, and never need its database or user model.
  • Operationally boring: invalid configuration stops the service at startup listing every problem, health routes for probes, graceful shutdown on SIGTERM, and bounded memory per connection and per request.

Real-time UI update

GitHub · Docker Hub. The backend publishes one authenticated POST after its database commit and the service fans it out over WebSockets to every subscriber of the channel, so the backend never holds a socket.

Real-time UI update flow: subscriptions pass a JWT check and prefix access control (403 before the upgrade); publishes pass a time-token and scope check (400 for user channels on /publish); the hub queues each payload on per-client 4 MiB queues and closes slow readers with 1008.
Real-time UI update: subscribe, publish and fan-out, with every rejection point.
  • Per-user channels: a channel named user:<id> admits only the client whose JWT sub is exactly <id>, checked before the WebSocket upgrade. The backend can push events that only one user, on every tab and device, receives.
  • Time-token publishing: the backend signs the current time window with HMAC-SHA256, so a leaked token stops working once its window passes, and the shared secret itself never travels over the wire.
  • Slow-consumer protection: each client has a 4 MiB byte-bounded queue drained by a writer goroutine that only exists while there is data; a client that falls further behind is closed with 1008 instead of slowing everyone else.
  • Measured: 16 KiB of memory and one goroutine per idle connection, 237k messages per second across 2000 users, and fan-out to 2000 users with a p99 delivery latency of 10.8 ms (Apple M3 Pro, load generator on the same host).

Real-time chat

GitHub · Docker Hub. One-to-one and group conversations with REST send and history, WebSocket delivery and MongoDB persistence.

Real-time chat flow: JWT check, per-user rate limit, ChatService normalisation and membership check, persistence queue (503 when full), hub fan-out to the other participants, 4 batching workers writing to MongoDB, and history served by one indexed query.
Real-time chat: send, deliver and persist, and why a 202 is never lost on shutdown.
  • The participant set is the conversation: IDs are trimmed, sorted and de-duplicated, so there is no conversation bookkeeping, creating one is free, and authorisation is a membership check.
  • Queued before delivered: a message is accepted into a bounded persistence queue before it is pushed, so a recipient never sees a message the service refused to store, and a stalled database answers 503 instead of growing memory.
  • Adaptive batching: four workers drain up to 256 messages into one unordered InsertMany; retries are idempotent because IDs are assigned before the first attempt.
  • Measured: 45–50k one-to-one messages per second with a 0.4 ms median send latency (up from 7k and 4.5 ms), and every message acknowledged before SIGTERM is in the database after a restart.

Document module

GitHub · Docker Hub. Stateless conversion between JSON, YAML, TOML, XML, CSV/TSV, MessagePack, Properties and XLSX, plus PDF, DOCX and XLSX generation from a JSON list of drawing actions.

Document module pipeline: body read under MAX_BODY_BYTES, concurrency slot, then either depth pre-scan, decoder and encoder for /convert, or JSON action list, limits and renderer for PDF, DOCX and XLSX, ending in a fully buffered response.
Document module: one guarded pipeline for conversion and document generation.
  • Bounded work: the body is read before a processing slot is taken, so slow uploads cannot starve other clients, and only a fixed number of requests decode or render at once.
  • Hostile input: nesting is capped at 10 000 levels in every format, with pre-scanners for the decoders that would otherwise overflow the stack, plus limits against zip bombs, YAML aliases, XML entities and worksheet amplification.
  • Honest errors: output is fully buffered, so a failure late in encoding returns a proper error status instead of a truncated 200.
  • Measured: 84.8k small conversions per second with a 1.68 ms p99, and CSV → XLSX at 16.5 MiB/s, more than three times the previous release.

Tested like products

  • Unit tests with the race detector, plus fuzz tests on tokens, parsers and access rules
  • Black-box end-to-end suites that boot the real binary or image and attack it with forged and alg: none tokens, protocol abuse, floods, slowloris and races
  • A performance suite that profiles the live server, reports optimisation opportunities and feeds profile-guided optimisation

Image security and supply chain

Release pipeline: source, reproducible build, scan gate that can refuse the release, publish with SBOM and SLSA provenance, keyless cosign signing, and verification before deploy; below, the unit, e2e functional, security, race and performance suites.
Release pipeline shared by the three images, and the test suites behind each release.
  • FROM scratch images with a static binary, running as a non-root user, for linux/amd64 and linux/arm64
  • Pinned, reproducible builds: rebuilding a commit yields the identical image digest
  • Scan-gated releases (govulncheck and Docker Scout), with SBOM and SLSA provenance attached
  • Keyless cosign signatures, recorded in the public Rekor transparency log, that anyone can verify before deploying

Ready for AI coding agents

The collection ships an Agent Skill: a Markdown guide that gives a coding agent each service's flow, exact API contract, token formats, typed TypeScript clients, gotchas and smoke tests, so it can integrate the components into a backend correctly on the first try. Its typed clients and smoke tests were run against the published images.

Repository: OptiBlocks collection on GitHub.