new(ci): run e2e tests with podman socket too. #2386

FedeDP · 2025-05-05T13:21:57Z

What type of PR is this?

/kind feature

Any specific area of the project related to this PR?

/area CI

/area tests

Does this PR require a change in the driver versions?

What this PR does / why we need it:

This PR enables e2e tests (the python ones) to run through podman socket too.
This allows us to test container plugin with both docker and podman.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Keeping it wip since i am testing the CI by triggering it over and over :D

Does this PR introduce a user-facing change?:

NONE

FedeDP · 2025-05-05T13:23:27Z

.github/workflows/ci.yml

@@ -295,9 +294,6 @@ jobs:
            libprotobuf-dev \
            protobuf-compiler \
            "linux-headers-$(uname -r)"
-          sudo update-alternatives --install /usr/bin/clang clang /usr/bin/clang-14 90


Ported CI to use standard clang and llvm present in the ubuntu-24.04 runner image.

FedeDP · 2025-05-05T13:23:35Z

.github/workflows/ci.yml

@@ -268,15 +268,14 @@ jobs:
            cmake_opts: -DUSE_BUNDLED_DEPS=True
          - name: asan
            cmake_opts: -DUSE_BUNDLED_DEPS=True -DCMAKE_C_FLAGS=-fsanitize=address -DCMAKE_CXX_FLAGS=-fsanitize=address
-    runs-on: ubuntu-22.04
+    runs-on: ubuntu-24.04


Need newer version of podman.

github-actions · 2025-05-05T13:27:41Z

Perf diff from master - unit tests

     1.94%     -0.98%  [.] libsinsp::sinsp_suppress::process_event
     6.33%     +0.37%  [.] sinsp::next
     2.67%     +0.34%  [.] gzfile_read
    35.04%     -0.31%  [.] sinsp_thread_manager::create_thread_dependencies
     5.43%     -0.29%  [.] sinsp_evt::get_type
     0.51%     +0.26%  [.] sinsp_evt::get_ts
     2.16%     +0.26%  [.] sinsp_evt::load_params
     1.42%     +0.22%  [.] sinsp_parser::process_event
     1.05%     -0.21%  [.] sinsp_parser::event_cleanup
     0.21%     +0.20%  [.] scap_file_test::assert_num_event_type

Heap diff from master - unit tests

peak heap memory consumption: 0B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Heap diff from master - scap file

peak heap memory consumption: 0B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Benchmarks diff from master

Comparing gbench_data.json to /root/actions-runner/_work/libs/libs/build/gbench_data.json
Benchmark                                                         Time             CPU      Time Old      Time New       CPU Old       CPU New
----------------------------------------------------------------------------------------------------------------------------------------------
BM_sinsp_split_mean                                            +0.0140         +0.0141           149           152           149           152
BM_sinsp_split_median                                          +0.0078         +0.0078           150           151           150           151
BM_sinsp_split_stddev                                          +2.4129         +2.4141             1             3             1             3
BM_sinsp_split_cv                                              +2.3659         +2.3667             0             0             0             0
BM_sinsp_concatenate_paths_relative_path_mean                  +0.0649         +0.0650            56            60            56            60
BM_sinsp_concatenate_paths_relative_path_median                +0.0880         +0.0881            56            61            56            61
BM_sinsp_concatenate_paths_relative_path_stddev               +11.0832        +11.0825             0             2             0             2
BM_sinsp_concatenate_paths_relative_path_cv                   +10.3473        +10.3455             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_mean                     +0.0374         +0.0376            24            25            24            25
BM_sinsp_concatenate_paths_empty_path_median                   +0.0338         +0.0339            24            24            24            24
BM_sinsp_concatenate_paths_empty_path_stddev                   +4.0257         +4.0321             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_cv                       +3.8443         +3.8500             0             0             0             0
BM_sinsp_concatenate_paths_absolute_path_mean                  +0.1462         +0.1463            56            65            56            65
BM_sinsp_concatenate_paths_absolute_path_median                +0.1422         +0.1423            56            64            56            64
BM_sinsp_concatenate_paths_absolute_path_stddev                +2.0904         +2.0900             0             1             0             1
BM_sinsp_concatenate_paths_absolute_path_cv                    +1.6963         +1.6957             0             0             0             0

codecov · 2025-05-05T13:30:40Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.18%. Comparing base (2510a2c) to head (535e679).
Report is 12 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2386      +/-   ##
==========================================
+ Coverage   77.15%   77.18%   +0.02%     
==========================================
  Files         231      231              
  Lines       30347    30357      +10     
  Branches     4656     4658       +2     
==========================================
+ Hits        23415    23430      +15     
+ Misses       6932     6927       -5

Flag	Coverage Δ
libsinsp	`77.18% <ø> (+0.02%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

FedeDP · 2025-05-05T14:20:50Z

.github/workflows/reusable_e2e_tests.yaml

Also took my time and added a new reusable_workflow for e2e tests; it is useful because we will use it in container's plugin CI: https://github.com/falcosecurity/plugins/blob/main/.github/workflows/container-ci.yaml#L154

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

FedeDP · 2025-05-05T15:15:31Z

.github/workflows/reusable_e2e_tests.yaml

+        run: |
+          sudo rm -rf /tmp/report
+          # Force podman to run with runc by removing crun
+          sudo apt remove -y crun


We enforce running podman with runc so that test_exec_in_container test is still valid since it expects a runc related event.

We could make some changes to the tests to try and figure out if they are running with podman or docker, but meh.

Molter73

Neat! Just left a few comments.

Molter73 · 2025-05-05T15:36:40Z

.github/workflows/reusable_e2e_tests.yaml

+        run: |
+          sudo rm -rf /tmp/report
+          # Force podman to run with runc by removing crun
+          sudo apt remove -y crun


We could make some changes to the tests to try and figure out if they are running with podman or docker, but meh.

test/e2e/tests/test_process/test_container.py

.github/workflows/reusable_e2e_tests.yaml

test/e2e/tests/test_process/test_container.py

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

FedeDP · 2025-05-06T08:15:01Z

/milestone 0.21.0

They allow to disable test for podman or docker. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

Molter73

/lgtm

poiana · 2025-05-06T09:29:39Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: FedeDP, Molter73

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [FedeDP,Molter73]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

github-project-automation bot added this to Falco Roadmap May 5, 2025

github-project-automation bot moved this to Todo in Falco Roadmap May 5, 2025

poiana added kind/feature New feature or request do-not-merge/work-in-progress release-note-none dco-signoff: yes area/CI area/tests labels May 5, 2025

poiana requested a review from hbrueckner May 5, 2025 13:22

poiana added the approved label May 5, 2025

poiana requested a review from mstemm May 5, 2025 13:22

poiana added the size/M label May 5, 2025

FedeDP commented May 5, 2025

View reviewed changes

new(ci): run e2e tests with podman socket too.

1dc4c50

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

FedeDP force-pushed the new/e2e_tests_podman branch from 1267651 to 1dc4c50 Compare May 5, 2025 13:44

poiana added size/L and removed size/M labels May 5, 2025

FedeDP force-pushed the new/e2e_tests_podman branch from 04541d7 to 97cc497 Compare May 5, 2025 14:19

FedeDP commented May 5, 2025

View reviewed changes

new(ci): add a reusable_e2e_tests workflow and use it in PR CI.

2709fb1

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

FedeDP force-pushed the new/e2e_tests_podman branch 3 times, most recently from 0adb26f to a23c370 Compare May 5, 2025 15:14

FedeDP commented May 5, 2025

View reviewed changes

Molter73 reviewed May 5, 2025

View reviewed changes

FedeDP force-pushed the new/e2e_tests_podman branch 2 times, most recently from f1652c5 to 7369db0 Compare May 5, 2025 15:52

FedeDP commented May 5, 2025

View reviewed changes

test/e2e/tests/test_process/test_container.py Show resolved Hide resolved

FedeDP added 3 commits May 6, 2025 10:14

fix(test/e2e): drop assert of docker-only event (runc).

7434ef9

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

cleanup(ci): drop sysctl workaround step.

1d17e57

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

cleanup(ci): drop intercept_tls_get_addr workaround for ASAN in ci.

86207cc

Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

FedeDP force-pushed the new/e2e_tests_podman branch from 4d02de4 to 86207cc Compare May 6, 2025 08:14

FedeDP changed the title ~~wip: new(ci): run e2e tests with podman socket too.~~ new(ci): run e2e tests with podman socket too. May 6, 2025

poiana removed the do-not-merge/work-in-progress label May 6, 2025

poiana added this to the 0.21.0 milestone May 6, 2025

chore(ci): add new optional input params to the reusable workflow.

535e679

They allow to disable test for podman or docker. Signed-off-by: Federico Di Pierro <nierro92@gmail.com>

FedeDP force-pushed the new/e2e_tests_podman branch from 4e302e4 to 535e679 Compare May 6, 2025 08:28

Molter73 approved these changes May 6, 2025

View reviewed changes

poiana assigned Molter73 May 6, 2025

poiana added the lgtm label May 6, 2025

LucaGuerra approved these changes May 6, 2025

View reviewed changes

poiana assigned LucaGuerra May 6, 2025

github-project-automation bot moved this from Todo to In progress in Falco Roadmap May 6, 2025

poiana merged commit b1b30d3 into master May 6, 2025
47 checks passed

poiana deleted the new/e2e_tests_podman branch May 6, 2025 10:14

github-project-automation bot moved this from In progress to Done in Falco Roadmap May 6, 2025

bevanjkay mentioned this pull request May 20, 2025

falcosecurity-libs 0.21.0 Homebrew/homebrew-core#224095

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

new(ci): run e2e tests with podman socket too. #2386

new(ci): run e2e tests with podman socket too. #2386

Uh oh!

FedeDP commented May 5, 2025

Uh oh!

FedeDP May 5, 2025

Uh oh!

FedeDP May 5, 2025

Uh oh!

github-actions bot commented May 5, 2025 •

edited

Loading

Uh oh!

codecov bot commented May 5, 2025 •

edited

Loading

Uh oh!

FedeDP May 5, 2025

Uh oh!

FedeDP May 5, 2025

Uh oh!

Molter73 May 5, 2025

Uh oh!

Molter73 left a comment

Uh oh!

Molter73 May 5, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

FedeDP commented May 6, 2025

Uh oh!

Molter73 left a comment

Uh oh!

poiana commented May 6, 2025

Uh oh!

Uh oh!

Uh oh!

new(ci): run e2e tests with podman socket too. #2386

new(ci): run e2e tests with podman socket too. #2386

Uh oh!

Conversation

FedeDP commented May 5, 2025

Uh oh!

FedeDP May 5, 2025

Choose a reason for hiding this comment

Uh oh!

FedeDP May 5, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented May 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Perf diff from master - unit tests

Heap diff from master - unit tests

Heap diff from master - scap file

Benchmarks diff from master

Uh oh!

codecov bot commented May 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

FedeDP May 5, 2025

Choose a reason for hiding this comment

Uh oh!

FedeDP May 5, 2025

Choose a reason for hiding this comment

Uh oh!

Molter73 May 5, 2025

Choose a reason for hiding this comment

Uh oh!

Molter73 left a comment

Choose a reason for hiding this comment

Uh oh!

Molter73 May 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

FedeDP commented May 6, 2025

Uh oh!

Molter73 left a comment

Choose a reason for hiding this comment

Uh oh!

poiana commented May 6, 2025

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented May 5, 2025 •

edited

Loading

codecov bot commented May 5, 2025 •

edited

Loading