Skip to content

Fix refit crash #3348

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jun 26, 2023
Merged

Fix refit crash #3348

merged 2 commits into from
Jun 26, 2023

Conversation

Innixma
Copy link
Contributor

@Innixma Innixma commented Jun 23, 2023

Issue #, if available:

Description of changes:

  • Fix edge-case crash in .refit_full('best') when "best" differs between trainer and predictor (for example, when WeightedEnsemble only has 1 base model, and when predictor was fit with _save_bag_folds=False. Now the "best" model refit will be the predictor's "best" model to avoid an exception during .predict(data)
  • Fix edge-case crash when .fit(..., _save_bag_folds=False, calibrate=True, refit_full=False) where no models can infer and calibration is called, resulting in an exception. Now models that can't infer can be calibrated, which after refit_full results in calibrated refit models.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@Innixma Innixma added this to the 0.9 Release milestone Jun 23, 2023
@Innixma Innixma requested a review from yinweisu June 23, 2023 21:55
@Innixma Innixma added bug Something isn't working module: tabular labels Jun 23, 2023
@github-actions
Copy link

Job PR-3348-1ed6ae1 is done.
Docs are uploaded to http://autogluon-staging.s3-website-us-west-2.amazonaws.com/PR-3348/1ed6ae1/index.html

yinweisu
yinweisu approved these changes Jun 26, 2023
View reviewed changes
Copy link
Contributor

@yinweisu yinweisu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@Innixma Innixma merged commit c830540 into autogluon:master Jun 26, 2023
ddelange added a commit to ddelange/autogluon that referenced this pull request Jul 3, 2023
@ddelange
Merge branch 'master' of https://github.com/awslabs/autogluon into cp311
9bd84af 
* 'master' of https://github.com/awslabs/autogluon:
  0.8.2 release notes (autogluon#3371)
  Pin pydantic (autogluon#3370)
  0.8.1 Post Release (autogluon#3367)
  [Merge after v0.8.1][AutoMM]  Unpin sentencepiece (autogluon#3368)
  codespell: action, config + some typos fixed (autogluon#3323)
  0.8.1 release notes (autogluon#3363)
  [AutoMM] Update per gpu batch size for HPO (autogluon#3360)
  [Multimodal] Fix hpo crashing on reuse_actor (autogluon#3361)
  Add missing colab buttons to advanced AutoMM tutorials (autogluon#3359)
  Fix log suppression workaround for print statements in suod (autogluon#3364)
  Pin dependencies (autogluon#3358)
  [timeseries] Fix bugs in DirectTabularModel (autogluon#3350)
  Fix refit crash (autogluon#3348)
@PGijsbers
Copy link
Contributor

Did this PR fix the issue that produces stack traces such as:

[INFO] [amlb.print:00:06:38.123] ERROR:frameworks.shared.callee:'NoneType' object has no attribute 'name'
[INFO] [amlb.print:00:06:38.130] Traceback (most recent call last):
[INFO] [amlb.print:00:06:38.131]   File "/bench/frameworks/shared/callee.py", line 75, in call_run
[INFO] [amlb.print:00:06:38.131]     result = run_fn(ds, config)
[INFO] [amlb.print:00:06:38.131]   File "/bench/frameworks/AutoGluon/exec.py", line 73, in run
[INFO] [amlb.print:00:06:38.131]     predictor = TabularPredictor(
[INFO] [amlb.print:00:06:38.131]   File "/bench/frameworks/AutoGluon/venv/lib/python3.9/site-packages/autogluon/core/utils/decorators.py", line 30, in _call
[INFO] [amlb.print:00:06:38.131]     return f(*gargs, **gkwargs)
[INFO] [amlb.print:00:06:38.131]   File "/bench/frameworks/AutoGluon/venv/lib/python3.9/site-packages/autogluon/tabular/predictor/predictor.py", line 959, in fit
[INFO] [amlb.print:00:06:38.132]     self._post_fit(
[INFO] [amlb.print:00:06:38.132]   File "/bench/frameworks/AutoGluon/venv/lib/python3.9/site-packages/autogluon/tabular/predictor/predictor.py", line 1014, in _post_fit
[INFO] [amlb.print:00:06:38.132]     self._trainer.calibrate_model()
[INFO] [amlb.print:00:06:38.132]   File "/bench/frameworks/AutoGluon/venv/lib/python3.9/site-packages/autogluon/core/trainer/abstract_trainer.py", line 3041, in calibrate_model
[INFO] [amlb.print:00:06:38.132]     y_val_probs = self.get_model_oof(model_name_og)
[INFO] [amlb.print:00:06:38.132]   File "/bench/frameworks/AutoGluon/venv/lib/python3.9/site-packages/autogluon/core/trainer/abstract_trainer.py", line 967, in get_model_oof
[INFO] [amlb.print:00:06:38.132]     model_type = self.get_model_attribute(model=model, attribute='type')
[INFO] [amlb.print:00:06:38.133]   File "/bench/frameworks/AutoGluon/venv/lib/python3.9/site-packages/autogluon/core/trainer/abstract_trainer.py", line 2347, in get_model_attribute
[INFO] [amlb.print:00:06:38.133]     model = model.name
[INFO] [amlb.print:00:06:38.133] AttributeError: 'NoneType' object has no attribute 'name'

@Innixma Innixma modified the milestones: 0.9 Release, 1.0 Release Oct 10, 2023
@Innixma Innixma deleted the fix_refit branch April 16, 2025 21:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
1 more reviewer

@yinweisu yinweisu yinweisu approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
bug Something isn't working module: tabular
Projects
None yet
Milestone
1.0 Release
Development

Successfully merging this pull request may close these issues.
3 participants
@Innixma @PGijsbers @yinweisu