Fixing logging prefect config #18666 #18670
Conversation
CodSpeed Performance ReportMerging #18670 will degrade performances by 12.32%Comparing Summary
Benchmarks breakdown
|
There was a problem hiding this comment.
hi @chethanuk - thanks for the PR! this is making sense to me
cc @desertaxle for another set of eyes
There was a problem hiding this comment.
Thanks for the PR @chethanuk, but I think this still needs some more work. There are a couple of issues with this PR that make it so it doesn't solve the associated issue.
- If I use
prefect config setto set a custom logging setting, thenprefect config viewbreaks:
~/d/c/prefect ❯❯❯ uv run prefect config set PREFECT_LOGGING_HANDLER_CONSOLE_LEVEL=WARNING
Set 'PREFECT_LOGGING_HANDLER_CONSOLE_LEVEL' to 'WARNING'.
Updated profile 'local-pg'.
~/d/c/prefect ❯❯❯ uv run prefect config view
PREFECT_PROFILE='local-pg'
Traceback (most recent call last):
File "/Users/alexander/dev/chethanuk/prefect/src/prefect/cli/_utilities.py", line 44, in wrapper
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/Users/alexander/dev/chethanuk/prefect/src/prefect/cli/config.py", line 290, in view
if setting.name not in processed_settings:
^^^^^^^^^^^^
AttributeError: 'str' object has no attribute 'name'- The current logic for loading logging configuration (
load_logging_configinsrc/prefect/logging/configuration.py) won't pick up custom logging variables set viaprefect config setbecause these custom logging settings aren't included when you callget_current_settings.
These are tricky problems to solve, but my recommendation is to update LoggingSettings in src/prefect/settings/models/logging.py to enable it to load these custom logging settings. That likely will require custom sources that are extended from our current sources in src/prefect/settings/sources.py.
This is definitely something that we want to fix, so let me know how we can best support you in fixing this issue!
0b36649 to
69c3568
Compare
AI Summary (@zzstoatzz i put this here)
This pull request improves how Prefect handles logging-related configuration overrides by introducing stricter validation for
PREFECT_LOGGING_*settings and updating the settings system to support these overrides securely and consistently. The changes include a new shared validation module, updates to the CLI and settings profile logic, and comprehensive tests to ensure only safe logging override keys are accepted.Validation and support for logging settings:
prefect.settings.validationmodule with utility functions (is_valid_logging_setting,is_setting_or_valid_logging,should_allow_string_key) to validate and distinguishPREFECT_LOGGING_*override keys, ensuring only safe, documented patterns are accepted.config setcommand to accept valid logging override keys using the new validation logic, and reject invalid or unsafe keys. [1] [2]Settings profile and environment variable handling:
Profilesettings logic to allow string keys for valid logging overrides, keep them as strings internally, and ensure they are correctly converted to environment variables. Only standard settings are validated through the model; logging overrides are passed through as-is. [1] [2]Testing and validation coverage:
These changes improve security and flexibility for logging configuration while maintaining backwards compatibility for user profiles and CLI workflows.