[Python] Custom logger is not thread safe #1855
Description
Problem: Custom logger is not thread safe
catboost version: 0.26.1
Operating System: linux
CPU: ryzen 7
If fitting CatBoost models in parallel with joblib specifying require="sharedmem", CatBoost, will set its custom logger multiple times, repeating this error message over and over:
Custom logger is already specified. Specify more than one logger at same time is not thread safe.
Example:
import numpy as np
from joblib import Parallel, delayed
from catboost import CatBoostRegressor
rng = np.random.default_rng(seed = 123)
X = rng.standard_normal(size=(1000, 10))
coef = rng.standard_normal(size=(10, 1))
y = (X @ coef).reshape(-1) + rng.standard_normal(size=1000)
def fit_model(X, y):
model = CatBoostRegressor(silent=True)
model.fit(X, y)
return model
n_models = 10
models = Parallel(n_jobs=n_models, verbose=0, require="sharedmem")\
(delayed(fit_model)(X, y) for i in range(n_models))