Skip to content

fix[javalib]:Runtime#availableProcessors now respects cpuset on small Linux systems #3969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

LeeTibbert
Copy link
Contributor

@LeeTibbert LeeTibbert commented Jun 19, 2024

Weakly fixes #3963

Runtime#availableProcessors on small Linux systems now reports the number of logical processors allow
by the process cpuset.

This number may be smaller than that reported by sysconf(_SC_NPROCESSORS_ONLN) due to various
Linux mechanisms: cgroups, taskset, cpuset, etc.

'small' is defined as less than or equal to, currently, 1024. Above that limit sysconf(_SC_NPROCESSORS_ONLN) is
reported, same as before this PR.

See Issue for discussion and manual testing instructions (next sprint). Most notably, OpenMP variables are not considered.

@LeeTibbert
Copy link
Contributor Author

All but one of the tests are Green.

There is one macOS test which appears to be not making progress. I suspect that it will
timeout. I am not sure if a re-try method will kick in. Will check again this evening.

This PR is Linux only, so the macOS test "hang" is almost certainly an unrelated intermittent fault.

I still have a bit of documenting to do in the issue itself. No rest for the wicked.

Copy link
Contributor

@natsukagami natsukagami left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thank you so much! @amsen20 please check if it does the right thing :)

* logical processes configured for the system (_SC_NPROCESSORS).
*
* Note that OpenMP environment variables are neither consulted nor used.
* This consistent with Java.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* This consistent with Java.
* This is consistent with Java.

@amsen20
Copy link

amsen20 commented Jun 19, 2024

It works for me, thanks :)

Copy link

@amsen20 amsen20 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked it locally and it worked

@LeeTibbert
Copy link
Contributor Author

natsukagami, amsen20

Thank you both for your review & suggestion/correction.

It is nice to know that the proposed solution actually works in the use case and not just on my machines.

If either of you think there is a short term need for either > 1024 or for FreeBSD, please create an Issue.
I do not really have a NetBSD machine available, so someone else would have to champion that.

Somehow, I think we will all be back here again.

@LeeTibbert
Copy link
Contributor Author

There is one failing Windows Test. I appears to be an intermittent failure,
and unrelated to the substance of this PR.

Otherwise all Green & ready for review.

@WojciechMazur WojciechMazur merged commit 7447df3 into scala-native:main Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Available runtime processors is not really number of available processors
4 participants