Enhance the DataLine::get_item() error message to print the filename for which the parsing error occurred. #1429
Description
Describe the Enhancement
The original issue Dan found was a bit of red herring, caused by using a wildcard in the "-lookin" option. See below for details.
The real underlying problem is that tc_gen does not print a log message listing the input file it's going to read before actually reading it. Dan used "-lookin /home/dadriaan/projects/jtwcatcf/data/obs/*" and that directory contained some valid ATCF files ending in ".dat" and some gzipped files. The * wildcard was expanded by the shell, and tc_gen interpreted that as a list of explicit file names to be read. The error occurs when attempting to read gzipped files as ATCF.
The task is to review the MET-TC tools (tc_gen, tc_rmw, tc_pairs, and perhaps tc_stat) and make sure they list the file to be read before actually reading it.
Best Track ATCF files from JTWC seem to have lots of extra columns in them (sometimes?), which is apparently unexpected to tc_gen. This causes tc_gen to fail when attempting to read them and report an error even though all the information tc_gen needs is available and in the typical locations per ATCF best track file format guidelines. This extra information may or may not be needed, I do not know, but for my case I did not need them and was able to get tc_gen to run by cutting off all columns beyond column 30. It would be nice to support reading these files and not imposing a hard limit on the number of columns in the ATCF file.
Error reported:
DEBUG 4: [File 24 of 763] Found 1 BEST genesis events, from 1 tracks, from 21 input lines, from file "/home/dadriaan/projects/jtwcatcf/data/obs/bio082018.dat".
ERROR :
ERROR : DataLine::get_item(int) -> range check error
ERROR :
Command/environment used on kiowa:
/home/dadriaan/projects/jtwc/cmd
JTWC Best Track data source:
https://www.metoc.navy.mil/jtwc/jtwc.html?best-tracks
Time Estimate
1 day or less
Relevant Deadlines
NONE
Funding Source
Define the source of funding and account keys here or state NONE.
Define the Metadata
Assignee
- Select engineer(s) or no engineer required: John Halley-Gotway
- Select scientist(s) or no scientist required: Jonathan Vigh for ATCF expertise, Dan Adriaansen for tc_gen review
Labels
- Select component(s)
- Select priority
- Select requestor(s)
Projects and Milestone
- Review projects and select relevant Repository and Organization ones
- Select milestone
Define Related Issue(s)
Consider the impact to the other METplus components.
Enhancement Checklist
See the METplus Workflow for details.
- Complete the issue definition above.
- Fork this repository or create a branch of develop.
Branch name:feature_<Issue Number>_<Description> - Complete the development and test your changes.
- Add/update unit tests.
- Add/update documentation.
- Push local changes to GitHub.
- Submit a pull request to merge into develop.
Pull request:feature <Issue Number> <Description> - Iterate until the reviewer(s) accept and merge your changes.
- Delete your fork or branch.
- Close this issue.