Skip to content

tune_model + create_model updates #1114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 11 commits into from
Mar 20, 2021
Merged

tune_model + create_model updates #1114

merged 11 commits into from
Mar 20, 2021

Conversation

ngupta23
Copy link
Collaborator

Describe the changes you've made

Added tune_model capability along with updates for create model

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Code style update (formatting, local variables)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Unit tests have been added

Describe if there is any unusual behaviour of your code(Write NA if there isn't)

NA

Checklist:

  • My code follows the style guidelines of this project.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation.
  • My changes generate no new warnings.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • Any dependent changes have been merged and published in downstream modules.

@ngupta23 ngupta23 requested review from Yard1 and TremaMiguel March 17, 2021 10:23
Yard1
Yard1 requested changes Mar 17, 2021
View reviewed changes
Copy link
Member

@Yard1 Yard1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, just have one suggestion that will simplify the code by a fair bit :)

pycaret/internal/PycaretExperiment.py
self.logger.info("Defining Hyperparameters")

# TODO: Replace with time series specific code
def total_combintaions_in_grid(grid):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can use ParameterGrid from sklearn for getting the combinations, I believe it'd be the easiest way: https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.ParameterGrid.html - and it will allow for easy random search too

Copy link
Collaborator Author

@ngupta23 ngupta23 Mar 18, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Yard1 , That is what I am using though I am not sure it has methods to compute number of combinations.

pycaret/pycaret/time_series.py

Line 2435 in 568dd53

evaluate_candidates(ParameterGrid(self.param_grid))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it supports len. If not you should be able to calculate the number of possible combinations using some math, we probably don't need loops here

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean something like this? https://github.com/alan-turing-institute/sktime/blob/ee7a06843a44f4aaec7582d847e36073a9ab0566/sktime/forecasting/model_selection/_tune.py#L428

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

TremaMiguel
TremaMiguel reviewed Mar 18, 2021
View reviewed changes
pycaret/time_series.py
error_score=error_score)
for train, test in get_folds(cv, y))
# raise key exceptions
except (TerminatedWorkerError, KeyboardInterrupt, SystemExit):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just wondering if an Out of Memory Error could also happen ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it could. We can add them as we encounter them. What do you think?

@Yard1 Yard1 merged commit 4e655ee into time_series Mar 20, 2021
@ngupta23 ngupta23 deleted the ts_tune_model branch March 20, 2021 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Yard1 Yard1 Yard1 requested changes

+1 more reviewer

@TremaMiguel TremaMiguel TremaMiguel approved these changes

Reviewers whose approvals may not affect merge requirements
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ngupta23 @Yard1 @TremaMiguel