feat: Enable configurable priority for zeebe-record Index templates #36207

houssain-barouni · 2025-07-30T08:17:24Z

Description

Adds a new property in Elasticsearch and Opensearch exporters configuration

zeebe.broker.exporters.elasticsearch.args.index.template-priority
zeebe.broker.exporters.opensearch.args.index.template-priority

This configuration is set in zeebe records index templates priority setting, and it defaults to 20 when the configuration is not provided. 20 being the hardcoded value that was part of the existing index templates.

Checklist

Enable backports when necessary (fex. for bug fixes or for CI changes).

Related issues

closes #36084

Copilot

Pull Request Overview

This PR adds configurable priority support for index templates in both Elasticsearch and OpenSearch exporters. It allows users to set custom priority values for Zeebe record index templates, providing better control over template precedence in environments with multiple index templates.

Adds priority configuration field to both exporters with validation (>= 0)
Implements a mutable builder pattern for Template DTOs to enable safe modification
Updates template readers to use the configured priority value when creating index templates

Reviewed Changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`OpensearchExporterConfiguration.java`	Adds priority field with default value of 20 and getter/setter methods
`ElasticsearchExporterConfiguration.java`	Adds templatePriority field with default value of 20 and getter/setter methods
`OpensearchExporter.java`	Adds validation to ensure priority is non-negative
`ElasticsearchExporter.java`	Adds validation to ensure templatePriority is non-negative
`opensearch/.../dto/Template.java`	Implements MutableCopyBuilder pattern for safe template modification
`elasticsearch/.../dto/Template.java`	Implements MutableCopyBuilder pattern for safe template modification
`opensearch/.../TemplateReader.java`	Updates to use builder pattern and set priority from configuration
`elasticsearch/.../TemplateReader.java`	Updates to use builder pattern and set templatePriority from configuration
`OpensearchExporterTest.java`	Adds test for negative priority validation
`ElasticsearchExporterTest.java`	Adds test for negative templatePriority validation
`OpensearchExporterIT.java`	Adds integration tests for priority configuration and default behavior
`ElasticsearchExporterIT.java`	Adds integration tests for templatePriority configuration and default behavior

Comments suppressed due to low confidence (3)

zeebe/exporters/opensearch-exporter/src/main/java/io/camunda/zeebe/exporter/opensearch/OpensearchExporterConfiguration.java:219

The field name 'priority' is inconsistent with the Elasticsearch exporter which uses 'templatePriority'. For consistency between the two exporters, consider renaming this field to 'templatePriority'.

    private int priority = DEFAULT_INDEX_TEMPLATE_PRIORITY;

zeebe/exporters/opensearch-exporter/src/main/java/io/camunda/zeebe/exporter/opensearch/OpensearchExporterConfiguration.java:237

The method name 'getPriority' is inconsistent with the Elasticsearch exporter which uses 'getTemplatePriority'. For consistency between the two exporters, consider renaming this method to 'getTemplatePriority'.

    public int getPriority() {

zeebe/exporters/opensearch-exporter/src/main/java/io/camunda/zeebe/exporter/opensearch/OpensearchExporterConfiguration.java:241

The method name 'setPriority' is inconsistent with the Elasticsearch exporter which uses 'setTemplatePriority'. For consistency between the two exporters, consider renaming this method to 'setTemplatePriority'.

    public void setPriority(final int priority) {

Copilot · 2025-07-30T09:19:28Z

...s/opensearch-exporter/src/main/java/io/camunda/zeebe/exporter/opensearch/TemplateReader.java

+              aliases.clear();
+              aliases.put(aliasName, Collections.emptyMap());
+            })
+        .withPriority(Long.valueOf(config.getPriority()))


Using Long.valueOf() for autoboxing is unnecessary. Since getPriority() returns an int, you can cast directly to long: (long) config.getPriority().

Suggested change

.withPriority(Long.valueOf(config.getPriority()))

.withPriority((long) config.getPriority())

Copilot · 2025-07-30T09:19:29Z

...exporters/elasticsearch-exporter/src/main/java/io/camunda/zeebe/exporter/TemplateReader.java

+              aliases.clear();
+              aliases.put(aliasName, Collections.emptyMap());
+            })
+        .withPriority(Long.valueOf(config.index.getTemplatePriority()))


Using Long.valueOf() for autoboxing is unnecessary. Since getTemplatePriority() returns an int, you can cast directly to long: (long) config.index.getTemplatePriority().

Suggested change

.withPriority(Long.valueOf(config.index.getTemplatePriority()))

.withPriority((long) config.index.getTemplatePriority())

RomanJRW

Some small comments but generally LGTM 👍

...rch-exporter/src/main/java/io/camunda/zeebe/exporter/ElasticsearchExporterConfiguration.java

...lasticsearch-exporter/src/test/java/io/camunda/zeebe/exporter/ElasticsearchExporterTest.java

...arch-exporter/src/test/java/io/camunda/zeebe/exporter/opensearch/OpensearchExporterTest.java

RomanJRW · 2025-07-31T15:41:12Z

dist/src/main/config/broker.standalone.yaml.template

@@ -896,6 +896,7 @@
        #
        #     numberOfShards: 3
        #     numberOfReplicas: 0
+        #     templatePriority: 100


❓ should this not be the default of 20 then?

this is just an example, like the number of shards above is not the default one

interesting, I thought we always aimed to match these templates up to the default values. But all good, no big deal 👍

RomanJRW · 2025-07-31T15:41:18Z

dist/src/main/config/broker.standalone.yaml.template

@@ -978,6 +979,7 @@
        #
        #     numberOfShards: 3
        #     numberOfReplicas: 0
+        #     templatePriority: 100


same as above

RomanJRW · 2025-07-31T15:41:36Z

dist/src/main/config/broker.yaml.template

@@ -831,6 +832,7 @@
        #
        #     numberOfShards: 3
        #     numberOfReplicas: 0
+        #     templatePriority: 100


same as above in this file too

backport-action · 2025-08-07T15:08:08Z

Created backport PR for main:

[Backport main] feat: Enable configurable priority for zeebe-record Index templates #36573 with remaining conflicts!

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-36207-to-main
git worktree add --checkout .worktree/backport-36207-to-main backport-36207-to-main
cd .worktree/backport-36207-to-main
git reset --hard HEAD^
git cherry-pick -x 3816688dd321d5f68018e55bdb12f4d5bcb40352 42980cbaff84bfaae5b6598f2ecd86c090e30eec 71a40d3a2bb67e111baeedefce61a7d67230bb06
git push --force-with-lease

…ndex templates (#36573) # Description Backport of #36207 to `main`. relates to #36084

houssain-barouni requested a review from Copilot July 30, 2025 08:17

github-actions bot added the component/zeebe Related to the Zeebe component/team label Jul 30, 2025

This comment was marked as outdated.

Sign in to view

houssain-barouni requested a review from Copilot July 30, 2025 09:18

Copilot AI reviewed Jul 30, 2025

View reviewed changes

houssain-barouni added the backport main Forward-port a pull request to main label Jul 30, 2025

houssain-barouni linked an issue Jul 30, 2025 that may be closed by this pull request

Enable configurable priority for zeebe-record Index templates (8.7 & 8.8) #36084

Closed

houssain-barouni requested a review from a team July 30, 2025 10:41

houssain-barouni marked this pull request as ready for review July 30, 2025 10:42

RomanJRW approved these changes Jul 30, 2025

View reviewed changes

github-actions bot added component/operate Related to the Operate component/team component/tasklist Related to the Tasklist component/team component/optimize Related to Optimize component/team labels Jul 31, 2025

RomanJRW reviewed Jul 31, 2025

View reviewed changes

houssain-barouni added 3 commits August 1, 2025 10:18

feat: Enable configurable priority for zeebe-record Index templates

3816688

refactor: use templatePriority property instead of priority

42980cb

refactor: apply code review comments

71a40d3

houssain-barouni force-pushed the hb-36084 branch from b13e769 to 71a40d3 Compare August 1, 2025 08:18

houssain-barouni added the deploy-preview Requests creation of a Preview Environment for a PR label Aug 5, 2025

houssain-barouni added this pull request to the merge queue Aug 7, 2025

Merged via the queue into stable/8.7 with commit 78e03a4 Aug 7, 2025
71 checks passed

houssain-barouni deleted the hb-36084 branch August 7, 2025 15:07

backport-action mentioned this pull request Aug 7, 2025

[Backport main] feat: Enable configurable priority for zeebe-record Index templates #36573

Merged

github-merge-queue bot pushed a commit that referenced this pull request Aug 7, 2025

[Backport main] feat: Enable configurable priority for zeebe-record I…

0635ee1

…ndex templates (#36573) # Description Backport of #36207 to `main`. relates to #36084

github-actions bot added version:dryrun-8.8.0-alpha6-rc999 labels Aug 19, 2025

github-actions bot added the version:8.8.0-alpha8-rc1 label Aug 27, 2025

github-actions bot added version:dryrun-8.3.20-alpha999 version:8.8.0-alpha8 labels Sep 3, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Enable configurable priority for zeebe-record Index templates #36207

feat: Enable configurable priority for zeebe-record Index templates #36207

Uh oh!

houssain-barouni commented Jul 30, 2025 •

edited

Loading

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jul 30, 2025

Uh oh!

Copilot AI Jul 30, 2025

Uh oh!

RomanJRW left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RomanJRW Jul 31, 2025

Uh oh!

houssain-barouni Jul 31, 2025

Uh oh!

RomanJRW Jul 31, 2025

Uh oh!

RomanJRW Jul 31, 2025

Uh oh!

RomanJRW Jul 31, 2025

Uh oh!

Uh oh!

backport-action commented Aug 7, 2025

Uh oh!

Uh oh!

	.withPriority(Long.valueOf(config.getPriority()))
	.withPriority((long) config.getPriority())

	.withPriority(Long.valueOf(config.index.getTemplatePriority()))
	.withPriority((long) config.index.getTemplatePriority())

feat: Enable configurable priority for zeebe-record Index templates #36207

feat: Enable configurable priority for zeebe-record Index templates #36207

Uh oh!

Conversation

houssain-barouni commented Jul 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Related issues

Uh oh!

This comment was marked as outdated.

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Copilot AI Jul 30, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jul 30, 2025

Choose a reason for hiding this comment

Uh oh!

RomanJRW left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RomanJRW Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

houssain-barouni Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

RomanJRW Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

RomanJRW Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

RomanJRW Jul 31, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

backport-action commented Aug 7, 2025

Uh oh!

Uh oh!

houssain-barouni commented Jul 30, 2025 •

edited

Loading