Skip to content

Conversation

Innixma
Copy link
Contributor

@Innixma Innixma commented Apr 24, 2024

Issue #, if available:

Description of changes:

  • Add "auto" option for calibrate_decision_threshold that sets to True for valid metrics in Binary Classification, otherwise False.
  • Improve Scorer information on knowing if metrics are for regression or classification tasks (via the new needs_class property). This is important because while we technically can run metrics like root_mean_squared_error through decision_threshold_calibration and not crash, it would not lead to the intended functionality. With needs_class, we know that RMSE is not valid for threshold calibration and avoid it when calibrate_decision_threshold="auto".
  • In a future PR I intend to switch calibrate_decision_threshold to "auto" as the new default in TabularPredictor.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@Innixma Innixma added enhancement New feature or request module: tabular labels Apr 24, 2024
@Innixma Innixma added this to the 1.1.1 Release milestone Apr 24, 2024
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@yinweisu
Copy link
Contributor

Previous CI Run Current CI Run

Copy link
Contributor

@rey-allan rey-allan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ship it!

Copy link

Job PR-4137-12a5bd0 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-4137/12a5bd0/index.html

@Innixma Innixma merged commit 87762b7 into autogluon:master Apr 25, 2024
@yinweisu
Copy link
Contributor

Previous CI Run Current CI Run
botocore==1.34.91 botocore==1.34.92
boto3==1.34.91 boto3==1.34.92
ruff==0.4.1 ruff==0.4.2
botocore==1.34.91 botocore==1.34.92
boto3==1.34.91 boto3==1.34.92
ruff==0.4.1 ruff==0.4.2

Copy link

Job PR-4137-8f31c5f is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-4137/8f31c5f/index.html

LennartPurucker pushed a commit to LennartPurucker/autogluon that referenced this pull request Jun 1, 2024
@Innixma Innixma deleted the calibrate_decision_threshold_auto branch April 16, 2025 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request module: tabular
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants