Loading SUN397 crashed #2889
Description
/!\ IF YOU WANT PEOPLE TO HELP YOU, PLEASE GIVE AS MUCH DETAIL AS POSSIBLE, INCLUDING THE FULL STACKTRACE AND CODE SNIPPET
Hi there i'am trying to load the SUN397 Dataset to make a short image Classification NN. And when loading the Dataset I am getting an Error. When Tensorflow tries to load around the 55910-55920 Example, the Error: ValueError: Cannot take the length of shape with unknown rank Raises.
Windows 10 Pro
Python Version: 3.8.6
tfds Version: 4.1.0
tfds-nightly Version:
tensorflow Version: 2.4.0
Here is my Code, that I use to load the sun397 Dataset
batch_size = 128
(train_ds, val_ds), info = tfds.load("sun397", split=["train[:55900]", "validation"], as_supervised=True, with_info=True)
Model
train_ds = train_ds.map(lambda img, label: (tf.image.resize(img, [img_width, img_height]) / 255.0, label)).shuffle(1024).batch(batch_size)
val_ds = val_ds.map(lambda img, label: (tf.image.resize(img, [img_width, img_height]) / 255.0, label)).batch(batch_size)
Here are my Logs:
Generating train examples...: 55816 examples [16:46, 113.83 examples/s]
Generating train examples...: 55830 examples [16:46, 117.48 examples/s]
Generating train examples...: 55843 examples [16:46, 117.46 examples/s]
Generating train examples...: 55856 examples [16:46, 81.19 examples/s]
Generating train examples...: 55866 examples [16:46, 53.75 examples/s]
Generating train examples...: 55874 examples [16:47, 52.12 examples/s]
Generating train examples...: 55881 examples [16:47, 42.26 examples/s]
Generating train examples...: 55887 examples [16:47, 46.28 examples/s]
Generating train examples...: 55893 examples [16:47, 44.83 examples/s]
Generating train examples...: 55907 examples [16:47, 56.20 examples/s]
Generating train examples...: 55915 examples [16:48, 39.78 examples/s]
Generating train examples...: 55922 examples [16:48, 44.33 examples/s]WARNING:absl:Image /t/track/outdoor/sun_aophkoiosslinihb.jpg could not be decoded by OpenCV, falling back to TF
CRITICAL:absl:Image /t/track/outdoor/sun_aophkoiosslinihb.jpg could not be decoded by Tensorflow
Traceback (most recent call last):
File "C:/Users/Jann/OneDrive - UMB AG/Schule/IT/Semester 3/122/Projekt/Code/Test/TestModel.py", line 31, in <module>
(train_ds, val_ds), info = tfds.load("sun397", split=["train[:55900]", "validation"], as_supervised=True, with_info=True)
File "C:\Users\Jann\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow_datasets\core\load.py", line 328, in load
dbuilder.download_and_prepare(**download_and_prepare_kwargs)
File "C:\Users\Jann\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow_datasets\core\dataset_builder.py", line 432, in download_and_prepare
self._download_and_prepare(
File "C:\Users\Jann\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow_datasets\core\dataset_builder.py", line 1158, in _download_and_prepare
split_info_futures = [
File "C:\Users\Jann\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow_datasets\core\dataset_builder.py", line 1159, in <listcomp>
split_builder.submit_split_generation( # pylint: disable=g-complex-comprehension
File "C:\Users\Jann\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow_datasets\core\split_builder.py", line 295, in submit_split_generation
return self._build_from_generator(**build_kwargs)
File "C:\Users\Jann\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow_datasets\core\split_builder.py", line 354, in _build_from_generator
for key, example in utils.tqdm(
File "C:\Users\Jann\AppData\Local\Programs\Python\Python38\lib\site-packages\tqdm\std.py", line 1167, in __iter__
for obj in iterable:
File "C:\Users\Jann\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow_datasets\image_classification\sun.py", line 283, in _generate_examples
image = _process_image_file(
File "C:\Users\Jann\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow_datasets\image_classification\sun.py", line 123, in _process_image_file
image = _decode_image(fobj, session, filename=filename)
File "C:\Users\Jann\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow_datasets\image_classification\sun.py", line 104, in _decode_image
if len(image.shape) == 4: # rank=4 -> rank=3
File "C:\Users\Jann\AppData\Local\Programs\Python\Python38\lib\site-packages\tensorflow\python\framework\tensor_shape.py", line 848, in __len__
raise ValueError("Cannot take the length of shape with unknown rank.")
ValueError: Cannot take the length of shape with unknown rank.
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: iCCP: known incorrect sRGB profile
libpng warning: Ignoring invalid time value
Process finished with exit code 1
Thanks for anyone, who has an idea whats happening. I was trying a lot and reading through the Documentation but dindnt find anything so thanks a lot.