Precision and recall metrics doesn't work with scikit-learn==1.6.0 #655
Description
Description
The metric "precision" and "recall" doesn't work with the latest scikit-learn==1.6.0, which may have API changes.
At the time of writing this issue, the API changelog of scikit-learn hasn't updated, but it seems like precision_score() from sklearn.metrics has changed to returning float instead of its original type.
Other metrics may have similar issues, these two are what I've encountered.
Temporary Fix
Downgrade scikit-learn to 1.5.2.
pip install scikit-learn==1.5.2
Steps to Reproduce
- Install evaluate with pip (which will automatically install the latest
scikit-learn==1.6.0 - Run the following code
import evaluate
precision = evaluate.load("precision")
precision.compute(predictions=[0, 1], references=[0, 1])Error Message
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[24], line 1
----> 1 precision.compute(predictions=[0, 1], references=[0, 1])
File /opt/conda/lib/python3.10/site-packages/evaluate/module.py:467, in EvaluationModule.compute(self, predictions, references, **kwargs)
465 inputs = {input_name: self.data[input_name] for input_name in self._feature_names()}
466 with temp_seed(self.seed):
--> 467 output = self._compute(**inputs, **compute_kwargs)
469 if self.buf_writer is not None:
470 self.buf_writer = None
File ~/.cache/huggingface/modules/evaluate_modules/metrics/evaluate-metric--precision/4e7f439a346715f68500ce6f2be82bf3272abd3f20bdafd203a2c4f85b61dd5f/precision.py:145, in Precision._compute(self, predictions, references, labels, pos_label, average, sample_weight, zero_division)
126 def _compute(
127 self,
128 predictions,
(...)
134 zero_division="warn",
135 ):
136 score = precision_score(
137 references,
138 predictions,
(...)
143 zero_division=zero_division,
144 )
--> 145 return {"precision": float(score) if score.size == 1 else score}
AttributeError: 'float' object has no attribute 'size'