-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Document the potential security issues when using -r
/-0
.
#2350
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Also update the older versions of the manual.
Also update the older versions of the manual. Reported-by: @pcworld Reported-in: jqlang#1271 (comment)
133c88a
to
13d29af
Compare
Reported-by: @pcworld Reported-in: jqlang#1271 (comment)
13d29af
to
f65a269
Compare
Hmm, the AppVeyor failure seems unrelated to the changes in this PR. |
Please note a **potential security issue** when using this option. | ||
Please note that if the selected data of the input JSON contains | ||
newline characters then processing of jq output will incorrectly | ||
split a single item containing an newline character into two items. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Frankly, I'm not convinced this is a security issue, for example, when CSV output is generated with @csv
, multi-line output works correctly, because CSV handles multi-line strings correctly. Similar observations can be made about @html, etc.
Raw strings only have as much structure as is ensured in their construction, and there's nothing jq
-specific here. If one wants structured output that won't confuse some downstream parser, one has to ensure the right encoding is used, which may or may be JSON.
Thus, if one wants to use jq
as an awk
or sed
replacement working with line-oriented data, one can do that:
$ printf -- 'Do it with %s!\n' "sed" |
jq -Rr 'sub("\\bsed\\b"; "jq"; "g")'
Do it with jq
$ cat /etc/passwd | # Averaging uids from /etc/passwd is of course nonsense!
jq -Rrn 'reduce (inputs | [limit(3;splits(":"))] | .[2] | tonumber) as $n ([0, 0]; [.[0]+$n, .[1] + 1])
| "The average uid is: \(.[0]*1000/.[1] | rint | ./1000)"'
The average uid is: 1715.038
So what we have here is that when the output is not JSON it needs to be appropriately encoded for the syntax expected by the consumer. I don't think this really deserves any sort of security warning.
Even with JSON output, if the elements encoded as JSON are sloppily constructed, they may already mediate some sort of "injection" attack.
Its not a security issue in jq itself, but in the ways that the jq raw
LF or NUL delimited output feature can be used by other programs.
Usually when there are such features with sharp edges in programming
languages and tools, those issues are documented so that people can
avoid them. Without this warning, folks using these features might not
even know that JSON can contain LF or especially NUL characters and so
they might write commands that rely on LF/NUL not being in the input
and introduce either breakage or security issues when they are. This is
a bigger problem with NUL than LF because people generally know about
the LF issues and then reach for NUL delimiters to solve them, but with
jq NUL has the same problem since JSON can contain NUL.
Please see the discussion in #1271 for other potential ways to solve this.
…--
bye,
pabs
https://bonedaddy.net/pabs3/
|
It is clear that this particular PR isn't acceptable, so closing. |
`--nul-output` / `-0` was removed RE: jqlang#1271 and jqlang#2350
Also document how
-r
does separators.Reported-by: @pcworld
Reported-in: #1271 (comment)