-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Open
Description
Describe the bug
It is possible to register custom features using datasets.features.features.register_feature (#6727)
However such features are not compatible with Features.encode_example/decode_example if they require special encoding / decoding logic because encode_nested_example / decode_nested_example checks whether the feature is in a fixed list of encodable types:
datasets/src/datasets/features/features.py
Line 1349 in 16a121d
elif isinstance(schema, (Audio, Image, ClassLabel, TranslationVariableLanguages, Value, _ArrayXD)): |
This prevents the extensibility of features to complex cases
Steps to reproduce the bug
class ListOfStrs:
def encode_example(self, value):
if isinstance(value, str):
return [str]
else:
return value
feats = Features(strlist=ListOfStrs())
assert feats.encode_example({"strlist": "a"})["strlist"] = feats["strlist"].encode_example("a")}
Expected behavior
Registered feature types should be encoded based on some property of the feature (e.g. requires_encoding)?
Environment info
3.0.2
Metadata
Metadata
Assignees
Labels
No labels