Loading ERA5 data with CHUNK_CONTROL fails with TypeError #5864
Description
🐛 Bug Report
Loading ERA5 data in netCDF format within a CHUNK_CONTROL.set(time=-1) context leads to a TypeError: 'NoneType' object does not support item assignment. Any coordinate that is present in the file (time, level, latitude, longitude) will give this error regardless of the value specified. No error is raised if coordinates are given that are not present in the file.
The file can be downloaded here (select "Monthly averaged reanalysis", "Temperature", all levels, "2022", all months", "00:00", "Whole available region").
How To Reproduce
import iris
from iris.fileformats.netcdf.loader import CHUNK_CONTROL
print("iris version", iris.__version__)
print()
path = 'adaptor.mars.internal-1710689764.140916-27680-14-21cccc98-2ce5-4fb3-b77d-e5a55df40a17.nc'
# Works
cube = iris.load_cube(path)
print(cube)
print(cube.lazy_data().chunksize)
# Fails
with CHUNK_CONTROL.set(time=-1):
cube = iris.load_cube(path)gives
iris version 3.8.1
air_temperature / (K) (time: 12; pressure_level: 37; latitude: 721; longitude: 1440)
Dimension coordinates:
time x - - -
pressure_level - x - -
latitude - - x -
longitude - - - x
Attributes:
Conventions 'CF-1.6'
history '2024-03-17 15:36:09 GMT by grib_to_netcdf-2.25.1: /opt/ecmwf/mars-client/bin/grib_to_netcdf.bin ...'
(1, 12, 721, 1440)
Traceback (most recent call last):
File "/home/b/b309141/scripts/iris/chunk_control_era5.py", line 17, in <module>
cube = iris.load_cube(path)
^^^^^^^^^^^^^^^^^^^^
File "/work/bd0854/b309141/miniforge3/envs/esm/lib/python3.11/site-packages/iris/__init__.py", line 354, in load_cube
cubes = _load_collection(uris, constraints, callback).cubes()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/work/bd0854/b309141/miniforge3/envs/esm/lib/python3.11/site-packages/iris/__init__.py", line 294, in _load_collection
result = _CubeFilterCollection.from_cubes(cubes, constraints)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/work/bd0854/b309141/miniforge3/envs/esm/lib/python3.11/site-packages/iris/cube.py", line 97, in from_cubes
for cube in cubes:
File "/work/bd0854/b309141/miniforge3/envs/esm/lib/python3.11/site-packages/iris/__init__.py", line 275, in _generate_cubes
for cube in iris.io.load_files(part_names, callback, constraints):
File "/work/bd0854/b309141/miniforge3/envs/esm/lib/python3.11/site-packages/iris/io/__init__.py", line 219, in load_files
for cube in handling_format_spec.handler(fnames, callback, constraints):
File "/work/bd0854/b309141/miniforge3/envs/esm/lib/python3.11/site-packages/iris/fileformats/netcdf/loader.py", line 636, in load_cubes
cube = _load_cube(engine, cf, cf_var, cf.filename)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/work/bd0854/b309141/miniforge3/envs/esm/lib/python3.11/site-packages/iris/fileformats/netcdf/loader.py", line 322, in _load_cube
return _load_cube_inner(engine, cf, cf_var, filename)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/work/bd0854/b309141/miniforge3/envs/esm/lib/python3.11/site-packages/iris/fileformats/netcdf/loader.py", line 329, in _load_cube_inner
data = _get_cf_var_data(cf_var, filename)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/work/bd0854/b309141/miniforge3/envs/esm/lib/python3.11/site-packages/iris/fileformats/netcdf/loader.py", line 277, in _get_cf_var_data
chunks[i_dim] = cf_var.shape[i_dim]
~~~~~~^^^^^^^
TypeError: 'NoneType' object does not support item assignmentExpected behaviour
No error. The file should be loaded with a chunksize of (12, ...).
Environment
- OS & Version: Red Hat Enterprise Linux 8.7 (Ootpa)
- Iris Version: 3.8.1