I know this issue has been discussed before and the view of the jq team is that this should be in an external tool. I've written a poor external tool to show that some of this can be done externally, but it runs into limits that I don't believe can be overcome as an external tool (without re-implementing all of jq which isn't really an external tool then). So, I'm opening a new issue to specifically discuss:

• types of comment support
• the pros/cons of an external tool
• an example of an external tool that you can find at https://github.com/mewalig/json-comments
• limitations of an external tool and potential solutions

The two reasons I wrote the external tool were a) to provide a quick and dirty hack for limited use cases and b) to explore the limits of what can be done using an external tool.

I understand there are differing views on whether JSON should support comments at all. This discussion is targeting use cases where we have already decided that having comments in JSON would be useful and are now evaluating what options are available to support them.

There are many characteristics and flavors of comment support, including:

• comment syntax: C-style, C++-style, other. If we can support one, it should be trivial to support others
• comment placement: separate line, same line, etc. I think the main question here is, given an original ugly format that contains comments, where to place the comments when the format is prettied
• comment existence: if the json is filtered, whether / how comments should be output

Regarding comment placement and existence, I would propose that the comment is "attached" to its immediately-preceding token in the input JSON. If/when that corresponding token is output, the comment is output with it. This would apply when the JSON output is filtered as well.

As for pros/cons, I think the most important "pro" is to keep jq focused strictly to the JSON spec (and therefore keep the code base etc streamlined), and the most important "cons" are:

• it would be difficult (maybe impossible) to use jq as a filtering tool and still properly handle comments
• jq is very popular already, but even so, I suspect that adding comment-parsing capabilities would make it significantly more compelling for many of its existing and would-be users

I have posted a partial proof-of-concept tool at https://github.com/mewalig/json-comments. All it does is strip comments out, save them to a file, and then add them back in. In between those steps, jq can be used to pretty-print the output, so the end result can be pretty-printed output with comments. Because it is proof-of-concept, it only supports ASCII and C++-style comments, but those limitations are easy enough to remove and what I'd classify as trivial.

The most important limitation is that it only adds comments back into equivalent JSON. In other words, it can't be used with a jq filter other than e.g. "."-- obviously this is a big limitation.

Which brings us to the final topic: limitations of an external tool and potential solutions. Some potential solutions:

• build filtering capabilities into the external tool. This is a bad idea as it would just recreate what jq already does so well.
• create some "add-on" functionality for jq to work together with the external tool. For example, jq and or the external tool could create some data files or communication channels that jq and/or tool use that ultimately result in some filtered output that contains comments. The problem with this approach is that it's likely to be more work than building into jq, because there would need the additional work of exchanging data between jq and the external program.
• build the capabilities entirely into jq. I actually don't think this would be that hard. It could work as follows:
  • struct jv is augmented to support comment information associated with that value
  • whenever a comment is read from input, it is saved with most recently-read jv structure
  • whenever that jv structure is output, the comment is also output. to keep it simple, the output could be C-style so that it can come in the middle of a line and e.g. before any trailing comma

It has been proposed in other discussions to use an external tool, but I don't see how that could possibly work (or be less work) given the above issues, unless we are content to give up the combination of filtering + comment support.

Thoughts?