Use distributed training like the following snippet

  "distributed": {
    "cuda_devices": if num_gpus > 1 then std.range(0, num_gpus - 1) else 0,
  },

The include-package option does not work for the allennlp tune command. My custom dataset_reader is not found. Adding the fully qualified class name solves this issue, which basically means for some reason include-package doesn't seem to work.

It works perfectly fine if i use the allennlp train command.