Skip to content

Keras & data.Dataset : "Your dataset iterator ran out of data" #25254

@andsteing

Description

@andsteing

System information

  • Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
  • OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Colab
  • Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: n/a
  • TensorFlow installed from (source or binary): Colab
  • TensorFlow version (use command below): 1.12
  • Python version: 3.6.7
  • Bazel version (if compiling from source): n/a
  • GCC/Compiler version (if compiling from source): n/a
  • CUDA/cuDNN version: n/a
  • GPU model and memory: n/a

Describe the current behavior
Keras model.fit() does not reset validation dataset iterator between epochs. Thus, when specifying validation_steps < validation_dataset_size / batch_size, then every evaluation will be performed on a different set of examples.

Describe the expected behavior
I would expect that model.fit() restarts from the beginning in the validation dataset after every epoch of training. This way the validation dataset could be used without .repeat() and the evaluation would be performed on the same set of examples.

Code to reproduce the issue
https://colab.research.google.com/drive/1UjKNbX38UC4EG6EPm6xLzQ1AmFV8HWe5

Other info / logs

WARNING:tensorflow:Your dataset iterator ran out of data interrupting testing. Make sure that your dataset can generate at least `steps` batches (in this case, 100 batches). You may need to use the repeat() function when building your dataset.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions