This is a request from the NOAA/EMC Aerosols group. The regrid_data_plane tool was enhanced to process GOES-16 AOD data. During a meeting at NCWCP on 9/11/2019, Ho-Chun Huang requested that regrid_data_plane be enhanced to also process data from GOES-16 ADP files.

See a description of the GOES-16 ADP data format attached.
GOES-16_ADP_readme_idl_code_20180328.docx

Also see and excerpt from the ADP user's guide attached.
GOES-16 ADP Users Guide extract.pdf

The ADP files are very similar to the AOD files, but slightly different. The relevant data variables are named Aerosol, Dust, and Smoke. Each of these fields contains 0's or 1's as a mask to define whether Aerosol, Dust, or Smoke is present at that pixel. In addition, the DQF variable contains quality control flags. However, the QC info is stored in the 8 bits of the DQF values as described in the document above.

When processing Smoke or Dust, parse the DCF bits to construct a QC value of 0, 1, or 2, for high, medium, and low quality, respectively. Use the user-specified -qc flag command line option to determine which pixels should be used. When doing the regridding step, determine how many of the pixels inside each grid box meet the -qc flag setting. And then report the output value for that grid box as the average value of the good data pixel values. The result should be a number between 0 and 1 indicating the percentage of the pixels in that grid box containing smoke or dust.

Ultimately, Ho-Chun plans to process both the ADP files to compute fields for smoke percentage and dust percentage. Then he'll use those percentages to define masks when computing stats in Grid-Stat for the GOES-16 AOD values. For example, he can compute the RMSE for AOD for grid boxes that are at least 50% smoke.

Even though Ho-Chun has no planned usage for the Aerosol field, we may as well include that as an optional output.

Here's a description of how the -valid_thresh command line option of regrid_data_plane should be used. Let's say one target output grid box has 100 GOES-16 pixels inside of it. The -valid_thresh option defines the ratio (between 0 and 1) of those 100 pixels which must meet the requested -qc flag information in order to get an output value for that grid box.

Sample data can be found on WCOSS, here:
The daily ADP data file can be found in (phase II)
/gpfs/gd3/emc/meso/noscrub/Ho-Chun.Huang/GOES16_AOD/ADP/yyyymmdd

module load NetCDF/4.2/serial

You can use ncdump to read the header, I believe we will read Smoke, Dust, and DQF (for high, medium, and low confidence of Smoke and Dust mask)

ncdump -h OR_ABI-L2-ADPC-M6_G16_s20192472341118_e20192472343491_c20192472344453.nc | more