#2304 feat: add performance testing for single server load in ArcadeDB #2305

robfrank · 2025-06-25T12:57:55Z

What does this PR do?

Add initial infra for perfs tests using containers and an initial load test for single node deployment

Motivation

Even if Some perfomance tests are in place, they spin up Servers inside the same JVM of the test code.

Related issues

#2304

Checklist

I have run the build using mvn clean package command
My unit tests cover both failure and success scenarios

gemini-code-assist

Summary of Changes

Hello @robfrank, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request establishes a new, dedicated performance testing framework for ArcadeDB, moving from in-JVM tests to a more realistic container-based approach. It provides the core infrastructure for running performance tests using Testcontainers and includes an initial load test scenario for a single server deployment, complete with integrated metrics collection.

Highlights

New Performance Testing Module: I've introduced a new Maven module, e2e-perf, specifically for performance testing. This module is designed to run tests in a more realistic environment using containers, addressing the limitations of previous in-JVM performance tests.
Container-Based Test Infrastructure: A foundational ContainersTestTemplate class has been added, leveraging Testcontainers to spin up and manage ArcadeDB instances within Docker containers. This provides a robust and isolated environment for performance benchmarks, including network setup and Toxiproxy integration for potential network simulations.
Initial Single Server Load Test: An initial load test, SingleServerLoadTestIT, has been implemented. This test simulates concurrent creation of users, photos, and friendships on a single ArcadeDB server, providing a baseline for performance measurement under load. It utilizes a new DatabaseWrapper helper for simplified database interactions and metrics collection.
Metrics Integration: Micrometer has been integrated into the performance testing framework to collect and report metrics (e.g., arcadedb.test.inserted.users, arcadedb.test.inserted.photos, arcadedb.test.inserted.friendship). This allows for detailed performance analysis of various operations during load tests.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

.github/workflows/mvn-test.yml

+    runs-on: ubuntu-latest
+    needs: build-and-package
+    steps:
+      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+
+      - name: Set up JDK 21
+        uses: actions/setup-java@c5195efecf7bdfc987ee8bae7a71cb8b11521c00 # v4.7.1
+        with:
+          distribution: "temurin"
+          java-version: 21
+          cache: "maven"
+
+      - name: Restore Maven artifacts
+        uses: actions/cache/restore@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
+        with:
+          path: ~/.m2/repository
+          key: maven-repo-${{ github.run_id }}-${{ github.run_attempt }}
+
+      - name: Restore Docker image
+        uses: actions/cache/restore@5a3ec84eff668545956fd18022155c47e93e2684 # v4.2.3
+        with:
+          path: /tmp/arcadedb-image.tar
+          key: docker-image-${{ github.run_id }}-${{ github.run_attempt }}
+
+      - name: Load Docker image
+        run: docker load < /tmp/arcadedb-image.tar
+
+      - name: E2E Perf Tests
+        run: ./mvnw verify -pl e2e-perf
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}


To fix the issue, we need to add an explicit permissions block to the workflow. This block should be added at the root level of the workflow file to apply to all jobs unless overridden by job-specific permissions. Based on the workflow's usage of the GITHUB_TOKEN, the minimal permissions required are contents: read. This ensures that the token can access repository contents but cannot perform write operations.

Steps to implement the fix:

Add a permissions block at the root level of the workflow file.

Set contents: read as the permission to limit the scope of the GITHUB_TOKEN.

gemini-code-assist

Code Review

The code changes introduce a new performance testing module (e2e-perf) for ArcadeDB, focusing on a single-server load scenario. I've identified a few areas for improvement, primarily concerning correctness in the concurrent test logic and some opportunities for code simplification.

gemini-code-assist · 2025-06-25T13:01:30Z

e2e-perf/src/test/java/com/arcadedb/test/performance/SingleServerLoadTestIT.java

+    for (int i = 0; i < numOfThreads; i++) {
+      // Each thread will create users and photos
+      executor.submit(() -> {
+        DatabaseWrapper db1 = new DatabaseWrapper(arcadeContainer, idSupplier);
+        db1.addUserAndPhotos(numOfUsers, numOfPhotos);
+        db1.close();
+      });
+
+      TimeUnit.SECONDS.sleep(1);
+      // Each thread will create friendships
+      executor.submit(() -> {
+        DatabaseWrapper db1 = new DatabaseWrapper(arcadeContainer, idSupplier);
+        db1.createFriendships(numOfFriendshipIterations, numOfFriendshipPerIterations);
+        db1.close();
+      });
+    }


The current task submission logic has a race condition. Friendship tasks may execute before the users they depend on have been created, leading to incorrect results. To ensure correctness, user creation should be completed before friendship creation begins. Consider separating the submission of these tasks into two distinct loops.

Suggested change

for (int i = 0; i < numOfThreads; i++) {

// Each thread will create users and photos

executor.submit(() -> {

DatabaseWrapper db1 = new DatabaseWrapper(arcadeContainer, idSupplier);

db1.addUserAndPhotos(numOfUsers, numOfPhotos);

db1.close();

});

TimeUnit.SECONDS.sleep(1);

// Each thread will create friendships

executor.submit(() -> {

DatabaseWrapper db1 = new DatabaseWrapper(arcadeContainer, idSupplier);

db1.createFriendships(numOfFriendshipIterations, numOfFriendshipPerIterations);

db1.close();

});

}

for (int i = 0; i < numOfThreads; i++) {

// Each thread will create users and photos

executor.submit(() -> {

DatabaseWrapper db1 = new DatabaseWrapper(arcadeContainer, idSupplier);

db1.addUserAndPhotos(numOfUsers, numOfPhotos);

db1.close();

});

}

// All user creation tasks are submitted. Now submit friendship tasks.

for (int i = 0; i < numOfThreads; i++) {

// Each thread will create friendships

executor.submit(() -> {

DatabaseWrapper db1 = new DatabaseWrapper(arcadeContainer, idSupplier);

db1.createFriendships(numOfFriendshipIterations, numOfFriendshipPerIterations);

db1.close();

});

}

gemini-code-assist · 2025-06-25T13:01:30Z

e2e-perf/src/test/java/com/arcadedb/test/performance/SingleServerLoadTestIT.java

+    while (!executor.isTerminated()) {
+      long users = db.countUsers();
+      long friendships = db.countFriendships();
+      logger.info("Current users: {} - friendships: {}", users, friendships);
+      // Wait for 2 seconds before checking again
+      try {
+        TimeUnit.SECONDS.sleep(2);
+      } catch (InterruptedException e) {
+        Thread.currentThread().interrupt();
+      }
+    }


This while loop is an inefficient way to wait for the executor to terminate, as it performs busy-waiting with a fixed sleep interval. Use executor.awaitTermination() for a more efficient approach.

try { while (!executor.awaitTermination(2, TimeUnit.SECONDS)) { long users = db.countUsers(); long friendships = db.countFriendships(); logger.info("Current users: {} - friendships: {}", users, friendships); } } catch (InterruptedException e) { Thread.currentThread().interrupt(); }

e2e-perf/src/test/java/com/arcadedb/test/support/DatabaseWrapper.java

gemini-code-assist · 2025-06-25T13:01:30Z

e2e-perf/src/test/java/com/arcadedb/test/support/DatabaseWrapper.java

+          db.transaction(() ->
+                  db.command("sqlscript", sqlScript, photoId, photoName, userId)
+              , true, 20);


There appears to be a nested transaction here. The db.transaction() method is wrapping a sqlscript that already contains BEGIN and COMMIT statements. Remove the outer db.transaction call.

db.command("sqlscript", sqlScript, photoId, photoName, userId);

gemini-code-assist · 2025-06-25T13:01:30Z

e2e-perf/src/test/java/com/arcadedb/test/support/DatabaseWrapper.java

+
+  public void createFriendships(int numOfFriendshipIterations, int numOfFriendshipPerIterations) {
+    for (int f = 0; f < numOfFriendshipIterations; f++) {
+      List<Integer> userIds = getUserIds(numOfFriendshipPerIterations, f * 10);


The skip parameter in getUserIds is calculated using a hardcoded value of 10 (f * 10). Base this on numOfFriendshipPerIterations to avoid overlapping batches and ensure correctness.

List<Integer> userIds = getUserIds(numOfFriendshipPerIterations, f * numOfFriendshipPerIterations);

…ndencies

…abase handling

…arameters

codacy-production · 2025-06-25T21:34:12Z

Coverage summary from Codacy

See diff coverage on Codacy

Coverage variation	Diff coverage
✅ +1.12%	✅ ∅

Coverage variation details

	Coverable lines	Covered lines	Coverage
Common ancestor commit (`df6b160`)	70449	44327	62.92%
Head commit (`6b0e271`)	70452 (+3)	45118 (+791)	64.04% (+1.12%)

Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: <coverage of head commit> - <coverage of common ancestor commit>

Diff coverage details

	Coverable lines	Covered lines	Diff coverage
Pull request (#2305)	0	0	∅ (not applicable)

Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: <covered lines added or modified>/<coverable lines added or modified> * 100%

See your quality gate settings Change summary preferences

#2305) (cherry picked from commit 9164ded)

robfrank added this to the 25.6.1 milestone Jun 25, 2025

robfrank self-assigned this Jun 25, 2025

gemini-code-assist bot reviewed Jun 25, 2025

View reviewed changes

github-advanced-security bot found potential problems Jun 25, 2025

View reviewed changes

gemini-code-assist bot reviewed Jun 25, 2025

View reviewed changes

robfrank added 4 commits June 25, 2025 18:28

#2304 feat: add performance testing for single server load in ArcadeDB

8a82e09

feat: update integration test configuration and clean up pom.xml depe…

f9c810b

…ndencies

wip

767de9d

feat: enhance load testing by adding photo tracking and improving dat…

a93f190

…abase handling

robfrank force-pushed the feat/2304-load-testing branch from 5a22057 to a93f190 Compare June 25, 2025 20:02

feat: refine load testing configuration by adjusting test execution p…

6b0e271

…arameters

robfrank merged commit 9164ded into main Jun 25, 2025
17 of 21 checks passed

robfrank added a commit that referenced this pull request Jul 3, 2025

#2304 feat: add performance testing for single server load in ArcadeDB (

e484f67

#2305) (cherry picked from commit 9164ded)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

#2304 feat: add performance testing for single server load in ArcadeDB #2305

#2304 feat: add performance testing for single server load in ArcadeDB #2305

Uh oh!

robfrank commented Jun 25, 2025

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Check warning

Copilot Autofix

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Jun 25, 2025

Uh oh!

gemini-code-assist bot Jun 25, 2025

Uh oh!

Uh oh!

gemini-code-assist bot Jun 25, 2025

Uh oh!

gemini-code-assist bot Jun 25, 2025

Uh oh!

Uh oh!

codacy-production bot commented Jun 25, 2025

Uh oh!

Uh oh!

@@ -2,2 +2,5 @@
+            permissions:
+              contents: read
             on:

Uh oh!

#2304 feat: add performance testing for single server load in ArcadeDB #2305

#2304 feat: add performance testing for single server load in ArcadeDB #2305

Uh oh!

Conversation

robfrank commented Jun 25, 2025

What does this PR do?

Motivation

Related issues

Checklist

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Summary of Changes

Highlights

Footnotes

Uh oh!

Check warning

Copilot Autofix

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gemini-code-assist bot Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Jun 25, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codacy-production bot commented Jun 25, 2025

Coverage summary from Codacy

See diff coverage on Codacy

See your quality gate settings Change summary preferences

Uh oh!

Uh oh!