Skip to content

Conversation

shchur
Copy link
Collaborator

@shchur shchur commented Mar 5, 2024

Issue #, if available: #3886

Description of changes:

  • Raise exception in TimeSeriesScorer if predictions contain NaN values
  • Update implementation of WQL metric to handle NaN values in target column
  • Update our implementation of MASE and SQL metrics to align with GluonTS in case missing values are present
    • As part of this update, we moved from pandas + groupby to numpy + reshape, which results in faster metric computations & lower fit time for WeightedEnsemble (for M4 Monthly with 48K time series, ensemble fit time decreases 200s -> 160s)
    • Update the RMSSE implementation to follow the same missing values handling logic
  • Add new tests & update some old tests to use dataframes with missing values

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@shchur shchur requested a review from canerturkmen March 5, 2024 19:10
@shchur shchur added this to the 1.1 Release milestone Mar 5, 2024
Copy link

github-actions bot commented Mar 6, 2024

Job PR-3966-7ccdc58 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-3966/7ccdc58/index.html

Copy link
Contributor

@canerturkmen canerturkmen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall. Just one comment. Thanks for the housekeeping!! 🧹

Copy link
Contributor

@canerturkmen canerturkmen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!!

@canerturkmen canerturkmen added the module: timeseries related to the timeseries module label Mar 13, 2024
@shchur shchur merged commit 2a5b805 into autogluon:master Mar 13, 2024
@shchur shchur deleted the nan-metrics-ts branch March 13, 2024 17:11
Copy link

Job PR-3966-7f67262 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-3966/7f67262/index.html

ddelange added a commit to ddelange/autogluon that referenced this pull request Mar 21, 2024
…tch-4

* 'master' of https://github.com/awslabs/autogluon: (46 commits)
  [core] move transformers to setup_utils, bump dependency version (autogluon#3984)
  [AutoMM] Fix one lightning upgrade issue (autogluon#3991)
  [CI][Feature] Create a package version table (autogluon#3972)
  [v.1.1][Upgrade] PyTorch 2.1 and CUDA 12.1 upgrade (autogluon#3982)
  [WIP] Code implementation of Conv-LoRA (autogluon#3933)
  [timeseries] Ensure that all metrics handle missing values in the target (autogluon#3966)
  [timeseries] Fix path and device bugs (autogluon#3979)
  [AutoMM]Remove grounding-dino (autogluon#3974)
  [Docs] Update install modules content (autogluon#3976)
  Add note on pd.to_datetime (autogluon#3975)
  [AutoMM] Improve DINO performance (autogluon#3970)
  Minor correction in differ to pick correct environment (autogluon#3968)
  Fix windows python 3.11 issue by removing ray (autogluon#3956)
  [CI][Feature] Package Version Comparator (autogluon#3962)
  [timeseries] Add support for categorical covariates (autogluon#3874)
  [timeseries] Add method for plotting forecasts (autogluon#3889)
  Update conf.py copyright to reflect current year (autogluon#3932)
  [Timeseries][CI]Refactor CI to skip AutoMM and Tabular tests w.r.t timeseries changes (autogluon#3942)
  Fix HPO crash in memory check (autogluon#3931)
  [AutoMM][CI] Capping scikit-learn to avoid HPO test failure (autogluon#3947)
  ...
LennartPurucker pushed a commit to LennartPurucker/autogluon that referenced this pull request Jun 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: timeseries related to the timeseries module
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants