-
Notifications
You must be signed in to change notification settings - Fork 1.1k
[tabular] Improve logging with DyStack and ray #4262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Job PR-4262-4de48f7 is done. |
else: | ||
logger.info( | ||
f"\tRunning DyStack sub-fit in a ray process to avoid memory leakage. " | ||
"Logs will not be shown until this process is complete, due to a limitation in ray. " | ||
"You can force logging by specifying `ds_args={'force_ray_logging': True}`." | ||
"Logs will not be shown until this process is complete (enable_ray_logging=False). " |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a bit confused by the statement, will logs show with enable_ray_logging=False
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The ray process logs will not show if enable_ray_logging=False
. This log will show because it isn't tied to ray.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
An example of what the ray logs look like: http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-4262/4de48f7/tutorials/tabular/tabular-essentials.html#maximizing-predictive-performance
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we should add something like "Logs will not be shown until this process is complete. To enable logging in ray, use enable_ray_logging=True
."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is what the next line in the logging message is:
"You can experimentally enable logging by specifying `ds_args={'enable_ray_logging': True}`."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oh cool!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
|
Job PR-4262-6d2b5e4 is done. |
Issue #, if available:
Description of changes:
force_ray_logging
->enable_ray_logging
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.