Skip to content

fix(test/e2e): properly flush remaining queue once sinsp process leaves. #2388

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 6, 2025
Merged

fix(test/e2e): properly flush remaining queue once sinsp process leaves. #2388

merged 1 commit into from
May 6, 2025

Conversation

FedeDP
Copy link
Contributor

@FedeDP FedeDP commented May 6, 2025

What type of PR is this?

/kind bug

Any specific area of the project related to this PR?

/area tests

Does this PR require a change in the driver versions?

What this PR does / why we need it:

Will fix test_curl_nginx spurious failures.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

NONE

@FedeDP
fix(test/e2e): properly flush remaining queue once sinsp process leaves.
8633640 
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
@poiana poiana requested review from Andreagit97 and hbrueckner May 6, 2025 09:16
@poiana poiana added the approved label May 6, 2025
@FedeDP
Copy link
Contributor Author

FedeDP commented May 6, 2025

/milestone 0.21.0

@poiana poiana added this to the 0.21.0 milestone May 6, 2025
@poiana poiana added the size/XS label May 6, 2025
FedeDP
FedeDP commented May 6, 2025
View reviewed changes
test/e2e/tests/commons/sinspqa/sinsp.py
@@ -121,6 +121,10 @@ def read(self):
if (datetime.now() - start).total_seconds() > self.timeout:
break

# flush remaining queue
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.process.poll() returns != None when the process has left:

Popen.poll()

Check if child process has terminated. Set and return returncode attribute. Otherwise, returns None.

Here we have a timed race condition if:

  • we enter read but sinsp process has already left
  • we read everything until queue is empty, then, while we check for datetime.now - start other things gets pushed and sinsp process leaves; at that point, self.process.poll() will return not None and we will leave without further emptying the queue.

@FedeDP
Copy link
Contributor Author

FedeDP commented May 6, 2025

/cc @Molter73 ptal :)

@github-actions GitHub Actions
Copy link

github-actions bot commented May 6, 2025

Perf diff from master - unit tests

     6.33%     +2.05%  [.] sinsp::next
    35.04%     -1.28%  [.] sinsp_thread_manager::create_thread_dependencies
     1.94%     -0.93%  [.] libsinsp::sinsp_suppress::process_event
     0.09%     +0.69%  [.] scap_get_event_info_table
     0.98%     +0.65%  [.] scap_event_decode_params
     0.38%     +0.43%  [.] libsinsp::events::is_unknown_event
     1.05%     -0.40%  [.] sinsp_parser::event_cleanup
     0.51%     +0.29%  [.] sinsp_evt::get_ts
     3.84%     -0.25%  [.] next_event_from_file
     1.89%     -0.24%  [.] sinsp_thread_manager::get_thread_ref

Heap diff from master - unit tests

peak heap memory consumption: 0B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Heap diff from master - scap file

peak heap memory consumption: 0B
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Benchmarks diff from master

Comparing gbench_data.json to /root/actions-runner/_work/libs/libs/build/gbench_data.json
Benchmark                                                         Time             CPU      Time Old      Time New       CPU Old       CPU New
----------------------------------------------------------------------------------------------------------------------------------------------
BM_sinsp_split_mean                                            -0.0052         -0.0051           149           149           149           149
BM_sinsp_split_median                                          -0.0028         -0.0027           150           149           150           149
BM_sinsp_split_stddev                                          +1.1344         +1.1334             1             2             1             2
BM_sinsp_split_cv                                              +1.1457         +1.1444             0             0             0             0
BM_sinsp_concatenate_paths_relative_path_mean                  -0.0543         -0.0542            56            53            56            53
BM_sinsp_concatenate_paths_relative_path_median                -0.0794         -0.0794            56            52            56            52
BM_sinsp_concatenate_paths_relative_path_stddev                +9.7540         +9.7523             0             2             0             2
BM_sinsp_concatenate_paths_relative_path_cv                   +10.3718        +10.3689             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_mean                     -0.0059         -0.0058            24            24            24            24
BM_sinsp_concatenate_paths_empty_path_median                   -0.0057         -0.0056            24            24            24            24
BM_sinsp_concatenate_paths_empty_path_stddev                   +0.0688         +0.0755             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_cv                       +0.0751         +0.0818             0             0             0             0
BM_sinsp_concatenate_paths_absolute_path_mean                  -0.0250         -0.0249            56            55            56            55
BM_sinsp_concatenate_paths_absolute_path_median                -0.0173         -0.0172            56            55            56            55
BM_sinsp_concatenate_paths_absolute_path_stddev                +1.2268         +1.2271             0             1             0             1
BM_sinsp_concatenate_paths_absolute_path_cv                    +1.2838         +1.2839             0             0             0             0

@codecov Codecov
Copy link

codecov bot commented May 6, 2025 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.18%. Comparing base (2510a2c) to head (8633640).
Report is 14 commits behind head on master.

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #2388      +/-   ##
==========================================
+ Coverage   77.15%   77.18%   +0.02%     
==========================================
  Files         231      231              
  Lines       30347    30357      +10     
  Branches     4656     4658       +2     
==========================================
+ Hits        23415    23430      +15     
+ Misses       6932     6927       -5
Flag Coverage Δ
libsinsp 77.18% <ø> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ekoops
Copy link
Contributor

ekoops commented May 6, 2025

/lgtm

@poiana poiana added the lgtm label May 6, 2025
@poiana
Copy link
Contributor

poiana commented May 6, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ekoops, FedeDP

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

leogr
leogr approved these changes May 6, 2025
View reviewed changes
Copy link
Member

@leogr leogr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@github-project-automation github-project-automation bot moved this from Todo to In progress in Falco Roadmap May 6, 2025
@poiana poiana merged commit 59a68aa into master May 6, 2025
47 checks passed
@poiana poiana deleted the fix/curl_nginx_e2e_testy branch May 6, 2025 12:19
@github-project-automation github-project-automation bot moved this from In progress to Done in Falco Roadmap May 6, 2025
@bevanjkay bevanjkay mentioned this pull request May 20, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@leogr leogr leogr approved these changes

@LucaGuerra LucaGuerra LucaGuerra approved these changes

@Andreagit97 Andreagit97 Awaiting requested review from Andreagit97

@hbrueckner hbrueckner Awaiting requested review from hbrueckner

Assignees

@leogr leogr

@LucaGuerra LucaGuerra

@ekoops ekoops

Labels
approved area/tests dco-signoff: yes kind/bug Something isn't working lgtm release-note-none size/XS
Projects
Falco Roadmap
Status: Done
Milestone
0.21.0
Development

Successfully merging this pull request may close these issues.
5 participants
@FedeDP @ekoops @poiana @leogr @LucaGuerra