Objective

Implement distributed tracing for Elsa to monitor workflow execution by leveraging the OpenTelemetry framework. This includes capturing traces for both individual workflow activities and higher-level bursts of workflow execution.

Description

We need to enhance the observability of our Elsa Workflows by integrating distributed tracing. This will be achieved by implementing custom activity and workflow execution middleware that automatically captures trace information. The middleware will utilize OpenTelemetry's ActivitySource to create and manage spans, providing detailed insights into workflow execution.

Implementation Steps (Pseudo Code)

1. Define ActivitySource:
   Create a static class to hold a shared ActivitySource instance for Elsa Workflows.

   public static class ElsaTracing
   {
       public static readonly ActivitySource ActivitySource = new ActivitySource("Elsa.Workflows");
   }

2. Create Workflow Execution Middleware:
   Implement a custom middleware that creates a parent Activity for each burst of workflow execution.

   using Elsa.Services;
   using Elsa.Services.Models;
   using System.Diagnostics;
   using System.Threading;
   using System.Threading.Tasks;
   
   public class WorkflowExecutionTracingMiddleware : IWorkflowMiddleware
   {
       public async Task InvokeAsync(WorkflowExecutionContext context, WorkflowDelegate next, CancellationToken cancellationToken)
       {
           var workflowId = context.WorkflowInstance.Id;
           var burstId = context.CurrentScope.Id;
   
           using (var activity = ElsaTracing.ActivitySource.StartActivity("WorkflowBurst", ActivityKind.Internal))
           {
               activity?.AddTag("workflow.instanceId", workflowId);
               activity?.AddTag("burst.id", burstId);
   
               // Invoke the next middleware/component in the pipeline
               await next(context, cancellationToken);
   
               // Optionally, add more tags or log information after execution
               activity?.AddTag("workflow.status", context.WorkflowInstance.WorkflowStatus.ToString());
           }
       }
   }

3. Register the Middleware:
   Add the middleware to the Elsa workflow pipeline configuration.

   using Elsa;
   using Elsa.Extensions;
   using Microsoft.Extensions.DependencyInjection;
   
   public class Startup
   {
       public void ConfigureServices(IServiceCollection services)
       {
           services.AddElsa(options => options
               .AddWorkflowMiddleware<WorkflowExecutionTracingMiddleware>()
               .AddActivity<YourCustomActivity>()
           );
       }
   }

4. Instrument Individual Activities:
   Optionally, instrument individual activities to create child spans under the parent span created by the middleware.

   using Elsa.ActivityResults;
   using Elsa.Attributes;
   using Elsa.Services;
   using Elsa.Services.Models;
   using System.Diagnostics;
   
   [Activity(Category = "Custom", Description = "Custom Activity with Tracing.")]
   public class CustomActivity : Activity
   {
       protected override IActivityExecutionResult OnExecute(ActivityExecutionContext context)
       {
           var parentActivity = Activity.Current;
   
           using (var activity = ElsaTracing.ActivitySource.StartActivity("ExecuteCustomActivity", ActivityKind.Internal, parentActivity?.Context ?? default))
           {
               activity?.AddTag("workflow.instanceId", context.WorkflowInstance.Id);
               activity?.AddTag("activity.id", Id);
               activity?.AddTag("activity.type", GetType().Name);
   
               // Perform the activity's logic
               return Done();
           }
       }
   }

Expected Outcome

With this implementation, each burst of workflow execution will be traced, providing detailed insights into the execution flow. Additionally, individual activities within each burst can also be traced, allowing for granular performance monitoring and optimization.

Acceptance Criteria

• Workflow execution tracing middleware is implemented and registered in Elsa.
• Workflow bursts generate parent trace spans.
• Individual workflow activities generate child trace spans.
• Trace data provides detailed insights into workflow execution and performance.