[BUG]: No reproducibility if fix_imbalance is True #3638
Description
pycaret version checks
-
I have checked that this issue has not already been reported here.
-
I have confirmed this bug exists on the latest version of pycaret.
-
I have confirmed this bug exists on the master branch of pycaret (pip install -U git+https://github.com/pycaret/pycaret.git@master).
Issue Description
Performance metrics change if one executes compare_models() mutiple times
Reproducible Example
from sklearn import datasets
import numpy as np
from pycaret.classification import ClassificationExperiment
X, y = datasets.make_classification(n_samples=400, n_features=12, random_state=42, weights=[0.2])
data = np.concatenate((X, y.reshape(-1,1)), axis=1)
clfe = ClassificationExperiment()
clfe.setup(data=data, session_id=1, fix_imbalance=True)
clfe.compare_models(include=["svm"])
clfe.compare_models(include=["svm"])Expected Behavior
The performance metrics should be the same
Actual Results
Model Accuracy AUC Recall Prec. F1 Kappa
svm SVM - Linear Kernel 0.9179 0.0 0.9462 0.9507 0.9479 0.7512
Model Accuracy AUC Recall Prec. F1 Kappa
svm SVM - Linear Kernel 0.9179 0.0 0.9287 0.9698 0.9472 0.7587Installed Versions
PyCaret required dependencies:
pip: 22.0.4
setuptools: 57.5.0
pycaret: 3.0.4
IPython: 8.12.2
ipywidgets: 8.0.7
tqdm: 4.65.0
numpy: 1.23.5
pandas: 1.5.3
jinja2: 3.1.2
scipy: 1.10.1
joblib: 1.3.1
sklearn: 1.2.2
pyod: 1.1.0
imblearn: 0.11.0
category_encoders: 2.6.1
lightgbm: 4.0.0
numba: 0.57.1
requests: 2.31.0
matplotlib: 3.7.2
scikitplot: 0.3.7
yellowbrick: 1.5
plotly: 5.15.0
plotly-resampler: Not installed
kaleido: 0.2.1
schemdraw: 0.15
statsmodels: 0.14.0
sktime: 0.20.0
tbats: 1.1.3
pmdarima: 2.0.3
psutil: 5.9.5
markupsafe: 2.1.3
pickle5: Not installed
cloudpickle: 2.2.1
deprecation: 2.1.0
xxhash: 3.2.0
wurlitzer: 3.0.3
PyCaret optional dependencies:
shap: Not installed
interpret: Not installed
umap: Not installed
pandas_profiling: Not installed
explainerdashboard: Not installed
autoviz: Not installed
fairlearn: Not installed
deepchecks: Not installed
xgboost: Not installed
catboost: Not installed
kmodes: Not installed
mlxtend: Not installed
statsforecast: Not installed
tune_sklearn: Not installed
ray: Not installed
hyperopt: Not installed
optuna: Not installed
skopt: Not installed
mlflow: Not installed
gradio: Not installed
fastapi: Not installed
uvicorn: Not installed
m2cgen: Not installed
evidently: Not installed
fugue: Not installed
streamlit: Not installed
prophet: Not installed