-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Description
pycaret version checks
-
I have checked that this issue has not already been reported here.
-
I have confirmed this bug exists on the latest version of pycaret.
-
I have confirmed this bug exists on the master branch of pycaret (pip install -U git+https://github.com/pycaret/pycaret.git@master).
Issue Description
- the 'loss' parameter of RANSACRegressor is not set correctly
- for example
absolute_loss
is set when optimizing against"R2"
/Default, andsquared_loss
when optimizing against"RMSE"
Reproducible Example
from pycaret.datasets import get_data
from pycaret.regression import RegressionExperiment
data = get_data("insurance")
sort_metric = "RMSE"
exp = RegressionExperiment()
exp.setup(
data=data,
target="charges",
transform_target=True,
transformation=True,
)
ransac = exp.create_model("ransac")
and then tuning with
tuned = exp.tune_model(
estimator=ransac,
choose_better=True,
optimize="RMSE",
)
produces sklearn.utils._param_validation.InvalidParameterError: The 'loss' parameter of RANSACRegressor must be a str among {'absolute_error', 'squared_error'} or a callable. Got 'absolute_loss' instead.
or
tuned = exp.tune_model(
estimator=ransac,
choose_better=True,
)
produces sklearn.utils._param_validation.InvalidParameterError: The 'loss' parameter of RANSACRegressor must be a str among {'absolute_error', 'squared_error'} or a callable. Got 'squared_loss' instead.
Expected Behavior
tuning with 'ransac' estimator works irrespective of quality of the model; just working
Actual Results
{
"name": "ValueError",
"message": "
All the 100 fits failed.
It is very likely that your model is misconfigured.
You can try to debug the error by setting error_score='raise'.
Below are more details about the failures:
--------------------------------------------------------------------------------
26 fits failed with the following error:
Traceback (most recent call last):
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/model_selection/_validation.py\", line 686, in _fit_and_score
estimator.fit(X_train, y_train, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 275, in fit
fitted_estimator = self._memory_fit(
^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/joblib/memory.py\", line 353, in __call__
return self.func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 68, in _fit_one
transformer.fit(*args, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/linear_model/_ransac.py\", line 330, in fit
self._validate_params()
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/base.py\", line 600, in _validate_params
validate_parameter_constraints(
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/utils/_param_validation.py\", line 97, in validate_parameter_constraints
raise InvalidParameterError(
sklearn.utils._param_validation.InvalidParameterError: The 'loss' parameter of RANSACRegressor must be a str among {'absolute_error', 'squared_error'} or a callable. Got 'absolute_loss' instead.
--------------------------------------------------------------------------------
24 fits failed with the following error:
Traceback (most recent call last):
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/model_selection/_validation.py\", line 686, in _fit_and_score
estimator.fit(X_train, y_train, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 275, in fit
fitted_estimator = self._memory_fit(
^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/joblib/memory.py\", line 353, in __call__
return self.func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 68, in _fit_one
transformer.fit(*args, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/linear_model/_ransac.py\", line 330, in fit
self._validate_params()
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/base.py\", line 600, in _validate_params
validate_parameter_constraints(
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/utils/_param_validation.py\", line 97, in validate_parameter_constraints
raise InvalidParameterError(
sklearn.utils._param_validation.InvalidParameterError: The 'loss' parameter of RANSACRegressor must be a str among {'squared_error', 'absolute_error'} or a callable. Got 'absolute_loss' instead.
--------------------------------------------------------------------------------
13 fits failed with the following error:
Traceback (most recent call last):
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/model_selection/_validation.py\", line 686, in _fit_and_score
estimator.fit(X_train, y_train, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 275, in fit
fitted_estimator = self._memory_fit(
^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/joblib/memory.py\", line 353, in __call__
return self.func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 68, in _fit_one
transformer.fit(*args, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/linear_model/_ransac.py\", line 330, in fit
self._validate_params()
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/base.py\", line 600, in _validate_params
validate_parameter_constraints(
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/utils/_param_validation.py\", line 97, in validate_parameter_constraints
raise InvalidParameterError(
sklearn.utils._param_validation.InvalidParameterError: The 'loss' parameter of RANSACRegressor must be a str among {'absolute_error', 'squared_error'} or a callable. Got 'squared_loss' instead.
--------------------------------------------------------------------------------
37 fits failed with the following error:
Traceback (most recent call last):
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/model_selection/_validation.py\", line 686, in _fit_and_score
estimator.fit(X_train, y_train, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 275, in fit
fitted_estimator = self._memory_fit(
^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/joblib/memory.py\", line 353, in __call__
return self.func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 68, in _fit_one
transformer.fit(*args, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/linear_model/_ransac.py\", line 330, in fit
self._validate_params()
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/base.py\", line 600, in _validate_params
validate_parameter_constraints(
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/utils/_param_validation.py\", line 97, in validate_parameter_constraints
raise InvalidParameterError(
sklearn.utils._param_validation.InvalidParameterError: The 'loss' parameter of RANSACRegressor must be a str among {'squared_error', 'absolute_error'} or a callable. Got 'squared_loss' instead.
",
"stack": "---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[13], line 16
9 exp.setup(
10 data=data,
11 target=\"charges\",
12 transform_target=True,
13 transformation=True,
14 )
15 ransac = exp.create_model(\"ransac\")
---> 16 tuned = exp.tune_model(
17 estimator=ransac,
18 choose_better=True,
19 optimize=sort_metric,
20 )
File /workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/regression/oop.py:1499, in RegressionExperiment.tune_model(self, estimator, fold, round, n_iter, custom_grid, optimize, custom_scorer, search_library, search_algorithm, early_stopping, early_stopping_max_iters, choose_better, fit_kwargs, groups, return_tuner, verbose, tuner_verbose, return_train_score, **kwargs)
1307 def tune_model(
1308 self,
1309 estimator,
(...)
1327 **kwargs,
1328 ):
1329 \"\"\"
1330 This function tunes the hyperparameters of a given estimator. The output of
1331 this function is a score grid with CV scores by fold of the best selected
(...)
1496
1497 \"\"\"
-> 1499 return super().tune_model(
1500 estimator=estimator,
1501 fold=fold,
1502 round=round,
1503 n_iter=n_iter,
1504 custom_grid=custom_grid,
1505 optimize=optimize,
1506 custom_scorer=custom_scorer,
1507 search_library=search_library,
1508 search_algorithm=search_algorithm,
1509 early_stopping=early_stopping,
1510 early_stopping_max_iters=early_stopping_max_iters,
1511 choose_better=choose_better,
1512 fit_kwargs=fit_kwargs,
1513 groups=groups,
1514 return_tuner=return_tuner,
1515 verbose=verbose,
1516 tuner_verbose=tuner_verbose,
1517 return_train_score=return_train_score,
1518 **kwargs,
1519 )
File /workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pycaret_experiment/supervised_experiment.py:2672, in _SupervisedExperiment.tune_model(self, estimator, fold, round, n_iter, custom_grid, optimize, custom_scorer, search_library, search_algorithm, early_stopping, early_stopping_max_iters, choose_better, fit_kwargs, groups, return_tuner, verbose, tuner_verbose, return_train_score, **kwargs)
2664 with patch(
2665 \"sklearn.model_selection._search.sample_without_replacement\",
2666 pycaret.internal.patches.sklearn._mp_sample_without_replacement,
2667 ):
2668 with patch(
2669 \"sklearn.model_selection._search.ParameterGrid.__getitem__\",
2670 pycaret.internal.patches.sklearn._mp_ParameterGrid_getitem,
2671 ):
-> 2672 model_grid.fit(data_X, data_y, groups=groups, **fit_kwargs)
2673 else:
2674 model_grid.fit(data_X, data_y, groups=groups, **fit_kwargs)
File /workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/model_selection/_search.py:874, in BaseSearchCV.fit(self, X, y, groups, **fit_params)
868 results = self._format_results(
869 all_candidate_params, n_splits, all_out, all_more_results
870 )
872 return results
--> 874 self._run_search(evaluate_candidates)
876 # multimetric is determined here because in the case of a callable
877 # self.scoring the return type is only known after calling
878 first_test_score = all_out[0][\"test_scores\"]
File /workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/model_selection/_search.py:1768, in RandomizedSearchCV._run_search(self, evaluate_candidates)
1766 def _run_search(self, evaluate_candidates):
1767 \"\"\"Search n_iter candidates from param_distributions\"\"\"
-> 1768 evaluate_candidates(
1769 ParameterSampler(
1770 self.param_distributions, self.n_iter, random_state=self.random_state
1771 )
1772 )
File /workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/model_selection/_search.py:851, in BaseSearchCV.fit.<locals>.evaluate_candidates(candidate_params, cv, more_results)
844 elif len(out) != n_candidates * n_splits:
845 raise ValueError(
846 \"cv.split and cv.get_n_splits returned \"
847 \"inconsistent results. Expected {} \"
848 \"splits, got {}\".format(n_splits, len(out) // n_candidates)
849 )
--> 851 _warn_or_raise_about_fit_failures(out, self.error_score)
853 # For callable self.scoring, the return type is only know after
854 # calling. If the return type is a dictionary, the error scores
855 # can now be inserted with the correct key. The type checking
856 # of out will be done in `_insert_error_scores`.
857 if callable(self.scoring):
File /workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/model_selection/_validation.py:367, in _warn_or_raise_about_fit_failures(results, error_score)
360 if num_failed_fits == num_fits:
361 all_fits_failed_message = (
362 f\"\
All the {num_fits} fits failed.\
\"
363 \"It is very likely that your model is misconfigured.\
\"
364 \"You can try to debug the error by setting error_score='raise'.\
\
\"
365 f\"Below are more details about the failures:\
{fit_errors_summary}\"
366 )
--> 367 raise ValueError(all_fits_failed_message)
369 else:
370 some_fits_failed_message = (
371 f\"\
{num_failed_fits} fits failed out of a total of {num_fits}.\
\"
372 \"The score on these train-test partitions for these parameters\"
(...)
376 f\"Below are more details about the failures:\
{fit_errors_summary}\"
377 )
ValueError:
All the 100 fits failed.
It is very likely that your model is misconfigured.
You can try to debug the error by setting error_score='raise'.
Below are more details about the failures:
--------------------------------------------------------------------------------
26 fits failed with the following error:
Traceback (most recent call last):
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/model_selection/_validation.py\", line 686, in _fit_and_score
estimator.fit(X_train, y_train, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 275, in fit
fitted_estimator = self._memory_fit(
^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/joblib/memory.py\", line 353, in __call__
return self.func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 68, in _fit_one
transformer.fit(*args, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/linear_model/_ransac.py\", line 330, in fit
self._validate_params()
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/base.py\", line 600, in _validate_params
validate_parameter_constraints(
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/utils/_param_validation.py\", line 97, in validate_parameter_constraints
raise InvalidParameterError(
sklearn.utils._param_validation.InvalidParameterError: The 'loss' parameter of RANSACRegressor must be a str among {'absolute_error', 'squared_error'} or a callable. Got 'absolute_loss' instead.
--------------------------------------------------------------------------------
24 fits failed with the following error:
Traceback (most recent call last):
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/model_selection/_validation.py\", line 686, in _fit_and_score
estimator.fit(X_train, y_train, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 275, in fit
fitted_estimator = self._memory_fit(
^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/joblib/memory.py\", line 353, in __call__
return self.func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 68, in _fit_one
transformer.fit(*args, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/linear_model/_ransac.py\", line 330, in fit
self._validate_params()
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/base.py\", line 600, in _validate_params
validate_parameter_constraints(
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/utils/_param_validation.py\", line 97, in validate_parameter_constraints
raise InvalidParameterError(
sklearn.utils._param_validation.InvalidParameterError: The 'loss' parameter of RANSACRegressor must be a str among {'squared_error', 'absolute_error'} or a callable. Got 'absolute_loss' instead.
--------------------------------------------------------------------------------
13 fits failed with the following error:
Traceback (most recent call last):
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/model_selection/_validation.py\", line 686, in _fit_and_score
estimator.fit(X_train, y_train, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 275, in fit
fitted_estimator = self._memory_fit(
^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/joblib/memory.py\", line 353, in __call__
return self.func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 68, in _fit_one
transformer.fit(*args, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/linear_model/_ransac.py\", line 330, in fit
self._validate_params()
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/base.py\", line 600, in _validate_params
validate_parameter_constraints(
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/utils/_param_validation.py\", line 97, in validate_parameter_constraints
raise InvalidParameterError(
sklearn.utils._param_validation.InvalidParameterError: The 'loss' parameter of RANSACRegressor must be a str among {'absolute_error', 'squared_error'} or a callable. Got 'squared_loss' instead.
--------------------------------------------------------------------------------
37 fits failed with the following error:
Traceback (most recent call last):
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/model_selection/_validation.py\", line 686, in _fit_and_score
estimator.fit(X_train, y_train, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 275, in fit
fitted_estimator = self._memory_fit(
^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/joblib/memory.py\", line 353, in __call__
return self.func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/pycaret/internal/pipeline.py\", line 68, in _fit_one
transformer.fit(*args, **fit_params)
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/linear_model/_ransac.py\", line 330, in fit
self._validate_params()
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/base.py\", line 600, in _validate_params
validate_parameter_constraints(
File \"/workspaces/bebefam-exploration/.venv/lib/python3.11/site-packages/sklearn/utils/_param_validation.py\", line 97, in validate_parameter_constraints
raise InvalidParameterError(
sklearn.utils._param_validation.InvalidParameterError: The 'loss' parameter of RANSACRegressor must be a str among {'squared_error', 'absolute_error'} or a callable. Got 'squared_loss' instead.
"
}
Installed Versions
System:
python: 3.11.7 (main, Dec 5 2023, 18:55:16) [GCC 12.2.0]
executable: /workspaces/bebefam-exploration/.venv/bin/python
machine: Linux-6.5.0-14-generic-x86_64-with-glibc2.36
PyCaret required dependencies:
pip: 23.3.2
setuptools: 69.0.3
pycaret: 3.2.0
IPython: 8.20.0
ipywidgets: 8.1.1
tqdm: 4.66.1
numpy: 1.25.2
pandas: 1.5.3
jinja2: 3.1.2
scipy: 1.10.1
joblib: 1.3.2
sklearn: 1.2.2
pyod: 1.1.2
imblearn: 0.11.0
category_encoders: 2.6.3
lightgbm: 4.2.0
numba: 0.58.1
requests: 2.31.0
matplotlib: 3.6.0
scikitplot: 0.3.7
yellowbrick: 1.5
plotly: 5.18.0
plotly-resampler: Not installed
kaleido: 0.2.1
schemdraw: 0.15
statsmodels: 0.14.1
sktime: 0.21.1
tbats: 1.1.3
pmdarima: 2.0.4
psutil: 5.9.7
markupsafe: 2.1.3
pickle5: Not installed
cloudpickle: 3.0.0
deprecation: 2.1.0
xxhash: 3.4.1
wurlitzer: 3.0.3
PyCaret optional dependencies:
shap: 0.44.0
interpret: Not installed
umap: Not installed
ydata_profiling: Not installed
explainerdashboard: Not installed
autoviz: Not installed
fairlearn: Not installed
deepchecks: Not installed
xgboost: 2.0.3
catboost: 1.2.2
kmodes: 0.12.2
mlxtend: 0.23.0
statsforecast: 1.5.0
tune_sklearn: Not installed
ray: Not installed
hyperopt: Not installed
optuna: Not installed
skopt: Not installed
mlflow: 1.27.0
gradio: 3.50.2
fastapi: 0.104.1
uvicorn: 0.25.0
m2cgen: 0.10.0
evidently: 0.2.8
fugue: 0.8.7
streamlit: Not installed
prophet: Not installed