Next-Generation Jaeger Demo with OpenTelemetry and OpenSearch

• Mentors / Maintainers: @jkowall, @yurishkuro
• For application details refer to Jaeger's LFX Mentorship - Term 03 - 2025 Sept-Nov #7327

Background & Motivation

This proposal extends the initiative started in issue #7115. The goal is not to replace existing demos, but to establish a second, parallel demo environment that reflects a modern, cloud-native deployment.

While our existing hotrod application is excellent for showcasing specific Jaeger features, it doesn't demonstrate how Jaeger handles and visualizes the rich, standardized telemetry data generated by a modern, polyglot application instrumented purely with OpenTelemetry. The observability landscape has increasingly standardized around OpenTelemetry (OTel). To remain a top-tier backend for OTel data and to provide the best experience for new and existing users, our public-facing demo should reflect this modern, OTel-native reality.

This new setup will run alongside existing demos, providing a clear example of a Kubernetes-based deployment with persistent storage.

The goal is to create a new, permanent, and publicly accessible demo environment that is:

1. Realistic: It showcases a distributed system generating traces.
2. OTel-Native: It uses the official OpenTelemetry Demo for instrumentation.
3. Persistent: It relies on a scalable storage technology (OpenSearch) to demonstrate capabilities beyond the in-memory driver.
4. Illustrative: It clearly demonstrates Jaeger's role in a modern, Kubernetes-based observability stack.

Proposal

We propose to build and host a new demo environment on a managed Kubernetes cluster. The entire deployment will be automated to ensure it is repeatable and can be reset on a regular schedule.

1. Adopt the OpenTelemetry Demo Application

Instead of a custom-built application, we will deploy the official OpenTelemetry Demo.

• Why? This application is a polyglot, realistic distributed system. It is the community-standard showcase for OTel and will immediately be familiar to many users. The goal is to use the upstream demo directly with specific configuration, avoiding the need to maintain a separate fork. By using it, we demonstrate Jaeger's seamless integration with the OTel ecosystem out-of-the-box.

2. Utilize OpenSearch as the Primary Storage Backend

We will use OpenSearch as the storage backend.

• Why? OpenSearch is a fully open-source, scalable, and powerful search and analytics suite. As a project under the Linux Foundation with a permanent Apache 2.0 license, it aligns perfectly with Jaeger's open-source ethos. For this demo, a single-node deployment is sufficient.

3. Utilize the Existing Public Endpoint

Instead of creating a new domain, the new demo components will be integrated into the existing public endpoint: https://demo.jaegertracing.io. The components will be exposed under specific subdirectories, managed by the existing service mesh infrastructure.

4. Exposed User Interfaces

To provide a comprehensive experience, the following user interfaces will be exposed publicly under subdirectories of demo.jaegertracing.io:

• /otel-shop/: The Astronomy Shop UI.
• /otel-loadgen/: The Load Generator (Locust) UI.
• /otel-jaeger/: The new Jaeger UI connected to the OpenTelemetry Demo.
• /otel-opensearch/: The OpenSearch Dashboards UI.

5. Proposed Architecture and Scope

The entire stack will be deployed on Oracle Cloud Infrastructure (OCI) using the Oracle Kubernetes Engine (OKE).

1. Instrumentation: The polyglot services within the OpenTelemetry Demo application generate traces and export them via OTLP.
2. Collection: An OpenTelemetry Collector deployed on Kubernetes receives the OTLP data.
3. Processing & Export: The OTel Collector processes the trace data and exports it to the Jaeger backend via the Jaeger gRPC exporter.
4. Storage: A Jaeger Collector receives the data and persists it into the single-node OpenSearch instance.
5. Querying & Visualization: The Jaeger Query service reads trace data from OpenSearch and serves it to the Jaeger UI.

Diagrammatic Flow:
[OTel Demo Apps] -> [OTel Collector] -> [Jaeger Collector] -> [Single-Node OpenSearch] -> [Jaeger Query/UI]
(All running as pods within an OKE cluster)
Scope: The primary focus of this demo is tracing. While the OTel Demo generates logs and metrics, they will not be a focus for collection or visualization in this setup. Service Performance Monitoring (SPM) capabilities will be demonstrated using Jaeger's integration with the OpenSearch backend, not Prometheus.

Goals

• Establish a new, modern, public-facing Jaeger demo running on Kubernetes.
• Showcase Jaeger's first-class support for OpenTelemetry.
• Provide a stable environment for users to explore Jaeger's UI with realistic OTel data.
• Create a reference architecture and automated deployment scripts (e.g., Helm, Kustomize) that users can learn from.
• Ensure the demo environment is consistently fresh through automated weekly redeployments.

Non-Goals

• This demo environment is not intended for performance, load testing, or high-availability.
• It is not a replacement for comprehensive documentation but rather a supplement to it.

Implementation Plan (High-Level)

1. Infrastructure Provisioning: Set up a dedicated Oracle Kubernetes Engine (OKE) cluster on Oracle Cloud Infrastructure (OCI). The existing infrastructure model will be used, scaling up the node resources (CPU, memory) as needed to accommodate the higher requirements of the OpenTelemetry Demo and OpenSearch.
2. Automation Scripting: Leverage the existing automation setup, which uses Helm and other components. The approach will be consistent with existing Jaeger mentorship program automation to deploy all components to the Kubernetes cluster.
3. Component Deployment: The script will handle the deployment of:
   • A single-node OpenSearch instance.
   • Jaeger Collector and Jaeger Query components configured for OpenSearch.
   • The OpenTelemetry Demo application.
   • The OpenTelemetry Collector.
4. Ingress & Routing Configuration: The automation will configure the necessary Kubernetes resources to expose the new services. The existing service mesh will be configured to route traffic from the subdirectories (e.g., /otel-shop/) to the appropriate services. No DNS changes are required.
5. Scheduled Redeployment: Implement a GitHub Actions workflow that triggers the automation script on a weekly schedule. This will tear down and redeploy the entire environment, ensuring a clean state and serving as the primary data retention/management strategy.
6. Documentation: Create a new page on the Jaeger website explaining the new demo, its architecture, and linking to the public repository containing the automation scripts.

Logistics and Ownership

• Hosting Costs: There are no hosting costs. The environment is generously donated by Oracle.
• Infrastructure Ownership: The Jaeger maintainers (mentees) own the OCI account and can provide access as needed for the mentorship.
• Deployment Frequency: The environment will be automatically redeployed on a weekly basis.