Problem: catboost pyspark implementation doesn't recognise setAutoClassWeights and setScalePosWeight as additional commands. For instance when I try setScalePosWeight(3.0) I get an error saying unknown option {scale_pos_weight} with value "3". Without using these I am able to train the model without any issues. Following are the code:

• Catboost_spark.CatBoostClassifier(featuresCol='features', labelCol='label', evalMetric='AUC) - works well.
• Catboost_spark.CatBoostClassifier(featuresCol='features', labelCol='label', evalMetric='AUC).setAutoClassWeights('Balanced')
• Catboost_spark.CatBoostClassifier(featuresCol='features', labelCol='label', evalMetric='AUC).setScalePosWeight(3.0)

catboost version: catboost-spark_3.2_2.12:1.2 (maven) - spark 3.2.1 and scala 2.12
Operating System: EMR cluster
CPU: master node of m4.2xlarge and executor nodes of r4.xlarge (4 and 8V cores and 32 GiB memory)
GPU: N/A