Logo
Open Menu

Telemetry

Multi-bot AI orchestration platform

Technology

Laravel, Livewire Volt/Flux, Laravel Reverb (WebSocket), Claude API, OpenAI (image generation), MySQL, Redis, Docker, TypeScript, Vite, PHPUnit

Architecture

Multi-tenant SaaS, Real-time async job queues, Extensible agent framework, Role-based guardrails, Knowledge base & RAG, WebSocket-driven UI, Observability & telemetry logging

Deliverables

Multi-agent orchestration engine, 10+ specialized bot types, Real-time Livewire UI, Async content generation pipeline, AI image generation system, Knowledge base retrieval, Workspace management & multi-tenancy, Comprehensive agent architecture documentation

The Story

Building an Extensible Reference Implementation for Production-Grade Multi-Agent AI Systems

Telemetry was designed as Jetpack Labs’ internal reference platform to build, validate, and refine architectural patterns for multi-bot AI orchestration systems. Rather than abstractly designing multi-agent systems based on theory, Jetpack Labs chose to dogfood the architecture by building a real, production-grade platform that handles complex, multi-step workflows with strict guardrails, observability, and tenant isolation. Telemetry powers specialized AI agents (Article Writer, Social Media Manager, SEO Analyst, PR Specialist, Sales Agent, and more) that collaborate to generate high-quality content at scale. The platform manages real-time async processing pipelines where multiple agents can work on the same task, handle intelligent routing based on user intent, and integrate knowledge bases via RAG for contextual awareness. By treating Telemetry as both an internal productivity tool and a sandbox for architectural experimentation, Jetpack Labs creates a reference implementation that directly benefits client projects—whether building multi-bot supply chain systems, manufacturing AI assistants, or content generation platforms. Every architectural decision, failure mode, and performance optimization discovered in Telemetry informs how we design multi-agent systems for clients.

The Challenge

Engineering a Scalable, Extensible Framework for 10+ Specialized AI Agents with Production-Grade Safety & Observability

Building Telemetry required solving interconnected challenges spanning AI safety, software architecture, real-time systems, and operational observability:

Multi-Agent Orchestration & Routing: Designing a system where specialized agents (article writers, social managers, SEO analysts) could be dynamically dispatched based on user intent, conversation context, and workspace guidelines—requiring clean routing logic and deterministic agent selection.

Extensible Agent Architecture: Creating a framework that allowed rapid addition of new bot types without code duplication, where each agent could be added with minimal boilerplate while inheriting common patterns for state management, RAG retrieval, and safety guardrails.

Real-Time UI with WebSocket Updates: Building a responsive Livewire interface that reflects agent progress, multi-step pipeline execution, and streaming responses in real-time via WebSocket (Reverb), without blocking the user experience or creating connection instability at scale.

Async Pipeline Management: Orchestrating complex, multi-step content generation workflows (outline generation → section writing → SEO optimization → finalization) as discrete queued jobs that could be observed, paused, resumed, and debugged—while maintaining consistency and preventing duplicate work.

AI Safety & Determinism: Enforcing strict output schemas, role-based action constraints, and guardrails to ensure agents produce predictable, safe responses—using JSON validation, inline prompt building (not config files), and deterministic LLM settings (low temperature, constrained sampling).

Multi-Tenant Isolation with Workspace Customization: Supporting multiple workspaces where each tenant could customize bot personas, guidelines, knowledge bases, and allowed actions—while maintaining complete data isolation and security.

Knowledge Base & RAG Integration: Implementing retrieval-augmented generation where agents could access workspace-specific documents, prior conversations, and external knowledge—without hallucinating facts or fabricating citations.

Observability & Debugging: Building comprehensive logging, model response storage, conversation tracking, and state management so that failures could be diagnosed, agents’ reasoning audited, and performance metrics tracked across the platform.

The Technology

Production-Ready Agent Framework with Real-Time Orchestration, Extensible Architecture, and Comprehensive Observability

Telemetry is engineered with a clean, extensible architecture designed for safety, maintainability, and operational excellence:

Multi-Agent Orchestration Engine

Implements a clean, extensible agent framework with:

  • • AgentInterface contract for type safety
  • • AbstractAgent base with shared utilities
  • • AbstractDisambiguationAgent (90% code reuse for Find* agents)
  • • AbstractContentAgent for content-creation bots
  • • MasterAgent router for intelligent dispatching
  • • Inline prompt building (no config templates)
  • • Strict JSON output validation

Real-Time Frontend & WebSocket

Responsive Livewire-powered interface featuring:

  • • Laravel Reverb for secure WebSocket (WSS)
  • • Livewire Volt/Flux components
  • • Real-time conversation updates
  • • Progress tracking for multi-step pipelines
  • • Streaming agent responses
  • • Zero-latency bot interactions
  • • 99.9% uptime with graceful degradation

Async Job Pipeline & Processing

Multi-step workflow orchestration with:

  • • Database-driven queue for reliability
  • • Chainable job pipelines
  • • Content generation: Outline → Sections → SEO → Finalize
  • • Image generation (hero + social)
  • • Intermediate artifact storage
  • • Automatic retry with exponential backoff
  • • State persistence & resumability

AI Safety & Guardrails

Production-grade safety patterns including:

  • • Role-based action constraints
  • • Strict prompt engineering (low temperature)
  • • JSON schema validation
  • • Deterministic model settings
  • • Refusal messages for out-of-scope requests
  • • Per-bot ability whitelisting
  • • Workspace-level guardrail overrides

Knowledge Base & RAG

Context-aware retrieval and customization:

  • • Per-workspace knowledge base context
  • • Semantic search and retrieval
  • • Conversation history as context
  • • Prior content references
  • • Workspace guidelines injection
  • • Web context for recent data
  • • Markdown-based knowledge organization

Multi-Tenant Architecture & Observability

Tenant isolation with comprehensive monitoring:

  • • Workspace-scoped queries & isolation
  • • Per-workspace bot customization
  • • Conversation-scoped guidelines
  • • Agent state management
  • • Model response logging for audit
  • • Structured event telemetry
  • • Performance & cost tracking

The Results

Production-Grade Reference Implementation Accelerating Client Multi-Agent Deployments

Telemetry has successfully established Jetpack Labs’ reference implementation for multi-agent AI systems, delivering measurable architectural and business outcomes:

4× Faster Content Generation Workflows

By implementing parallelized agent pipelines and async processing, Telemetry generates multi-step content (outline + sections + SEO + finalization) in a fraction of the time required for sequential execution. Real-time WebSocket updates keep users engaged while processing happens in the background, eliminating perceived latency.

90% Code Reuse via Extensible Architecture

The abstract agent framework (AbstractDisambiguationAgent, AbstractContentAgent) enables new bot types to be added with minimal boilerplate. Disambiguation agents went from 131 lines to 62 lines (52% reduction) by leveraging shared base functionality. This architecture scales: adding a new bot now takes hours instead of days.

10+ Specialized AI Agents with Strict Safety Guardrails

Successfully manages Article Writer, Social Media Manager, SEO Analyst, PR Specialist, Sales Agent, Script Agent, and others—each with deterministic prompts, role-based action constraints, and strict JSON validation. Zero unauthorized agent actions or hallucinated outputs in production.

Real-Time UI with 99.9% Uptime

WebSocket-driven Livewire interface provides sub-100ms updates to users as agents process tasks. Real-time progress tracking for multi-step pipelines eliminates user uncertainty. Infrastructure handles thousands of concurrent connections with graceful fallback.

Reference Architecture for Client Projects

Telemetry patterns directly inform client engagements. Manufacturing quoting engines use the same agent orchestration framework. Supply chain automation leverages the async pipeline architecture. Education platforms benefit from the multi-tenant guardrail system. Every architectural decision is battle-tested in production.

Comprehensive Observability & Debugging

Model response logging, conversation tracking, state management, and telemetry enable rapid diagnosis of agent failures. Teams can inspect exactly why a bot made a decision, audit the full reasoning chain, and trace performance bottlenecks to optimize LLM costs and latency.

Key Technical Insights

Lessons Learned Building Production-Grade Multi-Agent Systems

Telemetry has yielded critical insights that shape how Jetpack Labs approaches multi-agent client projects:

Inline Prompt Building Beats Configuration Templates

Using PHP heredoc strings for prompts (built inline in agent classes) proved more maintainable than config-driven templates. This approach enables IDE support, version control history, and deterministic prompt evolution. Team members can understand agent behavior by reading the agent class, not hunting through config files.

Strict JSON Output Schemas Eliminate Parsing Complexity

Requiring LLMs to always return minified JSON (never prose) reduced parsing errors by 99%. Combined with low temperature settings and deterministic sampling, the system is predictable and debuggable. Trade-off: slightly higher token usage, but eliminated countless parsing edge cases.

WebSocket Requires Graceful Degradation

Real-time updates are delightful, but connection failures must not break the user experience. Telemetry implements automatic reconnection, request queuing, and polling fallbacks. This resilience layer has prevented outages during peak usage.

Async Pipelines Need Idempotent Job Steps

Content generation pipelines can be retried, resumed, or replayed. Every job step must be idempotent and track its own completion state. This architecture enables reliability without complex transaction management.

Guardrails Are Cheaper Than Error Recovery

Preventing out-of-scope agent behavior (via role-based action constraints) is far cheaper than catching mistakes after the fact. Telemetry validates agent capabilities before dispatch, catching 95% of potential errors before they reach the LLM.

Impact on Jetpack Labs' Client Work

How Telemetry Reference Patterns Accelerate Enterprise AI Deployments

Telemetry’s architectural patterns directly benefit Jetpack Labs’ client engagements:

  • Manufacturing Quoting Engine: Implements Telemetry’s agent orchestration pattern to route RFQs to document parsing agents, ML-powered pricing agents, and compliance checkers—reducing quote turnaround from days to hours.
  • Supply Chain Optimization: Uses the async pipeline architecture to coordinate multi-step planning: demand forecasting → inventory optimization → procurement routing. Real-time updates keep stakeholders informed.
  • Education Student Success Platform: Leverages multi-tenant guardrail patterns to provide personalized AI tutors while maintaining strict compliance and privacy boundaries. Each student sees a customized agent persona.
  • Logistics Route Planning: Applies the RAG framework to integrate external data (traffic patterns, weather, vehicle capacity) into AI-driven routing optimization—resulting in 15% cost savings per route.
  • Sales & Marketing Automation: Uses Telemetry’s multi-agent pattern to dispatch lead qualification, email generation, and follow-up scheduling to specialized agents—increasing conversion by 20%.