Skip to content

Azure.Messaging.EventHubs Checkpointing Implementation Changed as of v5.11.0 Causing Incorrect Behavior In Event Hub scaler #5574

@windy1

Description

@windy1

Report

Related: Azure/azure-sdk-for-net#42409

As of Azure.Messaging.EventHubs v5.11.0, the format of checkpoints written to blob storage has changed. Notably, the value of offset is null. In our case, this caused KEDA to over-scale a service. Downgrading the SDK to v5.10.0 resolved the issue.

In response to my original issue, it was asserted by a Microsoft contributor that this change is intentional and that the implementation of checkpoints are not to be relied upon.

Please refer to the original issue for more in-depth details and reproduction steps.

Expected Behavior

I expected KEDA to scale my service appropriately.

Actual Behavior

KEDA over-scaled the service until the Azure SDK was downgraded and the old checkpoint format restored.

Steps to Reproduce the Problem

private async Task ProcessEventAsync(ProcessEventArgs eventArgs)
{
    await eventArgs.UpdateCheckpointAsync(); // breakpoint here and examine Offset in memory
    log.LogInformation("checkpoint created"); // breakpoint here and examine offset in checkpoint
}

Logs from KEDA operator

No response

KEDA Version

2.12.1

Kubernetes Version

None

Platform

Microsoft Azure

Scaler Details

Azure Event Hubs

Anything else?

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Status

    Ready To Ship

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions