"Fossies" - the Fresh Open Source Software Archive  

Source code changes of the file "docs/content/3.manual/manual.yml" between
jq-1.5.tar.gz and jq-1.6.tar.gz

About: jq is a lightweight and flexible command-line JSON processor.

manual.yml  (jq-1.5):manual.yml  (jq-1.6)
--- ---
headline: jq Manual (development version) headline: jq Manual (development version)
history: | history: |
*For released versions, see [jq 1.4](/jq/manual/v1.4) or *For released versions, see [jq 1.6](/jq/manual/v1.6),
[jq 1.3](/jq/manual/v1.3).* [jq 1.5](/jq/manual/v1.5), [jq 1.4](/jq/manual/v1.4)
or [jq 1.3](/jq/manual/v1.3).*
body: | body: |
A jq program is a "filter": it takes an input, and produces an A jq program is a "filter": it takes an input, and produces an
output. There are a lot of builtin filters for extracting a output. There are a lot of builtin filters for extracting a
particular field of an object, or converting a number to a string, particular field of an object, or converting a number to a string,
or various other standard tasks. or various other standard tasks.
Filters can be combined in various ways - you can pipe the output of Filters can be combined in various ways - you can pipe the output of
one filter into another filter, or collect the output of a filter one filter into another filter, or collect the output of a filter
skipping to change at line 110 skipping to change at line 111
Output the jq version and exit with zero. Output the jq version and exit with zero.
* `--seq`: * `--seq`:
Use the `application/json-seq` MIME type scheme for separating Use the `application/json-seq` MIME type scheme for separating
JSON texts in jq's input and output. This means that an ASCII JSON texts in jq's input and output. This means that an ASCII
RS (record separator) character is printed before each value on RS (record separator) character is printed before each value on
output and an ASCII LF (line feed) is printed after every output and an ASCII LF (line feed) is printed after every
output. Input JSON texts that fail to parse are ignored (but output. Input JSON texts that fail to parse are ignored (but
warned about), discarding all subsequent input until the next warned about), discarding all subsequent input until the next
RS. This more also parses the output of jq without the `--seq` RS. This mode also parses the output of jq without the `--seq`
option. option.
* `--stream`: * `--stream`:
Parse the input in streaming fashion, outputing arrays of path Parse the input in streaming fashion, outputing arrays of path
and leaf values (scalars and empty arrays or empty objects). and leaf values (scalars and empty arrays or empty objects).
For example, `"a"` becomes `[[],"a"]`, and `[[],"a",["b"]]` For example, `"a"` becomes `[[],"a"]`, and `[[],"a",["b"]]`
becomes `[[0],[]]`, `[[1],"a"]`, and `[[1,0],"b"]`. becomes `[[0],[]]`, `[[1],"a"]`, and `[[1,0],"b"]`.
This is useful for processing very large inputs. Use this in This is useful for processing very large inputs. Use this in
skipping to change at line 163 skipping to change at line 164
* `--indent n`: * `--indent n`:
Use the given number of spaces (no more than 8) for indentation. Use the given number of spaces (no more than 8) for indentation.
* `--color-output` / `-C` and `--monochrome-output` / `-M`: * `--color-output` / `-C` and `--monochrome-output` / `-M`:
By default, jq outputs colored JSON if writing to a By default, jq outputs colored JSON if writing to a
terminal. You can force it to produce color even if writing to terminal. You can force it to produce color even if writing to
a pipe or a file using `-C`, and disable color with `-M`. a pipe or a file using `-C`, and disable color with `-M`.
Colors can be configured with the `JQ_COLORS` environment
variable (see below).
* `--ascii-output` / `-a`: * `--ascii-output` / `-a`:
jq usually outputs non-ASCII Unicode codepoints as UTF-8, even jq usually outputs non-ASCII Unicode codepoints as UTF-8, even
if the input specified them as escape sequences (like if the input specified them as escape sequences (like
"\u03bc"). Using this option, you can force jq to produce pure "\u03bc"). Using this option, you can force jq to produce pure
ASCII output with every non-ASCII character replaced with the ASCII output with every non-ASCII character replaced with the
equivalent escape sequence. equivalent escape sequence.
* `--unbuffered` * `--unbuffered`
skipping to change at line 212 skipping to change at line 216
* `-e` / `--exit-status`: * `-e` / `--exit-status`:
Sets the exit status of jq to 0 if the last output values was Sets the exit status of jq to 0 if the last output values was
neither `false` nor `null`, 1 if the last output value was neither `false` nor `null`, 1 if the last output value was
either `false` or `null`, or 4 if no valid result was ever either `false` or `null`, or 4 if no valid result was ever
produced. Normally jq exits with 2 if there was any usage produced. Normally jq exits with 2 if there was any usage
problem or system error, 3 if there was a jq program compile problem or system error, 3 if there was a jq program compile
error, or 0 if the jq program ran. error, or 0 if the jq program ran.
Another way to set the exit status is with the `halt_error`
builtin function.
* `--arg name value`: * `--arg name value`:
This option passes a value to the jq program as a predefined This option passes a value to the jq program as a predefined
variable. If you run jq with `--arg foo bar`, then `$foo` is variable. If you run jq with `--arg foo bar`, then `$foo` is
available in the program and has the value `"bar"`. Note that available in the program and has the value `"bar"`. Note that
`value` will be treated as a string, so `--arg foo 123` will `value` will be treated as a string, so `--arg foo 123` will
bind `$foo` to `"123"`. bind `$foo` to `"123"`.
Named arguments are also available to the jq program as
`$ARGS.named`.
* `--argjson name JSON-text`: * `--argjson name JSON-text`:
This option passes a JSON-encoded value to the jq program as a This option passes a JSON-encoded value to the jq program as a
predefined variable. If you run jq with `--argjson foo 123`, then predefined variable. If you run jq with `--argjson foo 123`, then
`$foo` is available in the program and has the value `123`. `$foo` is available in the program and has the value `123`.
* `--slurpfile variable-name filename`: * `--slurpfile variable-name filename`:
This option reads all the JSON texts in the named file and binds This option reads all the JSON texts in the named file and binds
an array of the parsed JSON values to the given global variable. an array of the parsed JSON values to the given global variable.
If you run jq with `--argfile foo bar`, then `$foo` is available If you run jq with `--slurpfile foo bar`, then `$foo` is available
in the program and has an array whose elements correspond to the in the program and has an array whose elements correspond to the
texts in the file named `bar`. texts in the file named `bar`.
* `--rawfile variable-name filename`:
This option reads in the named file and binds its contents to the given
global variable. If you run jq with `--rawfile foo bar`, then `$foo` is
available in the program and has a string whose contents are to the texs
in the file named `bar`.
* `--argfile variable-name filename`: * `--argfile variable-name filename`:
Do not use. Use `--slurpfile` instead. Do not use. Use `--slurpfile` instead.
(This option is like `--slurpfile`, but when the file has just (This option is like `--slurpfile`, but when the file has just
one text, then that is used, else an array of texts is used as one text, then that is used, else an array of texts is used as
in `--slurpfile`.) in `--slurpfile`.)
* `--args`:
Remaining arguments are positional string arguments. These are
available to the jq program as `$ARGS.positional[]`.
* `--jsonargs`:
Remaining arguments are positional JSON text arguments. These
are available to the jq program as `$ARGS.positional[]`.
* `--run-tests [filename]`: * `--run-tests [filename]`:
Runs the tests in the given file or standard input. This must Runs the tests in the given file or standard input. This must
be the last option given and does not honor all preceding be the last option given and does not honor all preceding
options. The input consists of comment lines, empty lines, and options. The input consists of comment lines, empty lines, and
program lines followed by one input line, as many lines of program lines followed by one input line, as many lines of
output as are expected (one per output), and a terminating empty output as are expected (one per output), and a terminating empty
line. Compilation failure tests start with a line containing line. Compilation failure tests start with a line containing
only "%%FAIL", then a line containing the program to compile, only "%%FAIL", then a line containing the program to compile,
then a line containing an error message to compare to the then a line containing an error message to compare to the
actual. actual.
Be warned that this option can change backwards-incompatibly. Be warned that this option can change backwards-incompatibly.
- title: Basic filters - title: Basic filters
entries: entries:
- title: "`.`" - title: "Identity: `.`"
body: | body: |
The absolute simplest (and least interesting) filter The absolute simplest filter is `.` . This is a filter that
is `.`. This is a filter that takes its input and takes its input and produces it unchanged as output. That is,
produces it unchanged as output. this is the identity operator.
Since jq by default pretty-prints all output, this trivial Since jq by default pretty-prints all output, this trivial
program can be a useful way of formatting JSON output from, program can be a useful way of formatting JSON output from,
say, `curl`. say, `curl`.
examples: examples:
- program: '.' - program: '.'
input: '"Hello, world!"' input: '"Hello, world!"'
output: ['"Hello, world!"'] output: ['"Hello, world!"']
- title: "`.foo`, `.foo.bar`" - title: "Object Identifier-Index: `.foo`, `.foo.bar`"
body: | body: |
The simplest *useful* filter is `.foo`. When given a The simplest *useful* filter is `.foo`. When given a
JSON object (aka dictionary or hash) as input, it produces JSON object (aka dictionary or hash) as input, it produces
the value at the key "foo", or null if there's none present. the value at the key "foo", or null if there's none present.
A filter of the form `.foo.bar` is equivalent to `.foo|.bar`.
This syntax only works for simple, identifier-like keys, that
is, keys that are all made of alphanumeric characters and
underscore, and which do not start with a digit.
If the key contains special characters, you need to surround If the key contains special characters, you need to surround
it with double quotes like this: `."foo$"`. it with double quotes like this: `."foo$"`, or else `.["foo$"]`.
A filter of the form `.foo.bar` is equivalent to `.foo|.bar`. For example `.["foo::bar"]` and `.["foo.bar"]` work while
`.foo::bar` does not, and `.foo.bar` means `.["foo"].["bar"]`.
examples: examples:
- program: '.foo' - program: '.foo'
input: '{"foo": 42, "bar": "less interesting data"}' input: '{"foo": 42, "bar": "less interesting data"}'
output: [42] output: [42]
- program: '.foo' - program: '.foo'
input: '{"notfoo": true, "alsonotfoo": false}' input: '{"notfoo": true, "alsonotfoo": false}'
output: ['null'] output: ['null']
- program: '.["foo"]' - program: '.["foo"]'
input: '{"foo": 42}' input: '{"foo": 42}'
output: [42] output: [42]
- title: "`.foo?`" - title: "Optional Object Identifier-Index: `.foo?`"
body: | body: |
Just like `.foo`, but does not output even an error when `.` Just like `.foo`, but does not output even an error when `.`
is not an array or an object. is not an array or an object.
examples: examples:
- program: '.foo?' - program: '.foo?'
input: '{"foo": 42, "bar": "less interesting data"}' input: '{"foo": 42, "bar": "less interesting data"}'
output: [42] output: [42]
- program: '.foo?' - program: '.foo?'
input: '{"notfoo": true, "alsonotfoo": false}' input: '{"notfoo": true, "alsonotfoo": false}'
output: ['null'] output: ['null']
- program: '.["foo"]?' - program: '.["foo"]?'
input: '{"foo": 42}' input: '{"foo": 42}'
output: [42] output: [42]
- program: '[.foo?]' - program: '[.foo?]'
input: '[1,2]' input: '[1,2]'
output: ['[]'] output: ['[]']
- title: "`.[<string>]`, `.[2]`, `.[10:15]`" - title: "Generic Object Index: `.[<string>]`"
body: | body: |
You can also look up fields of an object using syntax like You can also look up fields of an object using syntax like
`.["foo"]` (.foo above is a shorthand version of this). This `.["foo"]` (.foo above is a shorthand version of this, but
one works for arrays as well, if the key is an only for identifier-like strings).
integer. Arrays are zero-based (like javascript), so `.[2]`
returns the third element of the array.
The `.[10:15]` syntax can be used to return a subarray of an - title: "Array Index: `.[2]`"
array or substring of a string. The array returned by body: |
`.[10:15]` will be of length 5, containing the elements from
index 10 (inclusive) to index 15 (exclusive). Either index may
be negative (in which case it counts backwards from the end of
the array), or omitted (in which case it refers to the start
or end of the array).
The `.[2]` syntax can be used to return the element at the When the index value is an integer, `.[<value>]` can index
given index. Negative indices are allowed, with -1 referring arrays. Arrays are zero-based, so `.[2]` returns the third
to the last element, -2 referring to the next to last element, element.
and so on.
The `.foo` syntax only works for simply keys i.e. keys that
are all alphanumeric characters. `.[<string>]` works with
keys that contain special characters such as colons and dots.
For example `.["foo::bar"]` and `.["foo.bar"]` work while
`.foo::bar` and `.foo.bar` would not.
The `?` "operator" can also be used with the slice operator, Negative indices are allowed, with -1 referring to the last
as in `.[10:15]?`, which outputs values where the inputs are element, -2 referring to the next to last element, and so on.
slice-able.
examples: examples:
- program: '.[0]' - program: '.[0]'
input: '[{"name":"JSON", "good":true}, {"name":"XML", "good":false}] ' input: '[{"name":"JSON", "good":true}, {"name":"XML", "good":false}] '
output: ['{"name":"JSON", "good":true}'] output: ['{"name":"JSON", "good":true}']
- program: '.[2]' - program: '.[2]'
input: '[{"name":"JSON", "good":true}, {"name":"XML", "good":false}] ' input: '[{"name":"JSON", "good":true}, {"name":"XML", "good":false}] '
output: ['null'] output: ['null']
- program: '.[-2]'
input: '[1,2,3]'
output: ['2']
- title: "Array/String Slice: `.[10:15]`"
body: |
The `.[10:15]` syntax can be used to return a subarray of an
array or substring of a string. The array returned by
`.[10:15]` will be of length 5, containing the elements from
index 10 (inclusive) to index 15 (exclusive). Either index may
be negative (in which case it counts backwards from the end of
the array), or omitted (in which case it refers to the start
or end of the array).
examples:
- program: '.[2:4]' - program: '.[2:4]'
input: '["a","b","c","d","e"]' input: '["a","b","c","d","e"]'
output: ['["c", "d"]'] output: ['["c", "d"]']
- program: '.[2:4]' - program: '.[2:4]'
input: '"abcdefghi"' input: '"abcdefghi"'
output: ['"cd"'] output: ['"cd"']
- program: '.[:3]' - program: '.[:3]'
input: '["a","b","c","d","e"]' input: '["a","b","c","d","e"]'
output: ['["a", "b", "c"]'] output: ['["a", "b", "c"]']
- program: '.[-2:]' - program: '.[-2:]'
input: '["a","b","c","d","e"]' input: '["a","b","c","d","e"]'
output: ['["d", "e"]'] output: ['["d", "e"]']
- program: '.[-2]' - title: "Array/Object Value Iterator: `.[]`"
input: '[1,2,3]'
output: ['2']
- title: "`.[]`"
body: | body: |
If you use the `.[index]` syntax, but omit the index If you use the `.[index]` syntax, but omit the index
entirely, it will return *all* of the elements of an entirely, it will return *all* of the elements of an
array. Running `.[]` with the input `[1,2,3]` will produce the array. Running `.[]` with the input `[1,2,3]` will produce the
numbers as three separate results, rather than as a single numbers as three separate results, rather than as a single
array. array.
You can also use this on an object, and it will return all You can also use this on an object, and it will return all
the values of the object. the values of the object.
skipping to change at line 411 skipping to change at line 442
- program: '.[]' - program: '.[]'
input: '{"a": 1, "b": 1}' input: '{"a": 1, "b": 1}'
output: ['1', '1'] output: ['1', '1']
- title: "`.[]?`" - title: "`.[]?`"
body: | body: |
Like `.[]`, but no errors will be output if . is not an array Like `.[]`, but no errors will be output if . is not an array
or object. or object.
- title: "`,`" - title: "Comma: `,`"
body: | body: |
If two filters are separated by a comma, then the If two filters are separated by a comma, then the
input will be fed into both and there will be multiple same input will be fed into both and the two filters' output
outputs: first, all of the outputs produced by the left value streams will be concatenated in order: first, all of the
expression, and then all of the outputs produced by the outputs produced by the left expression, and then all of the
right. For instance, filter `.foo, .bar`, produces outputs produced by the right. For instance, filter `.foo,
both the "foo" fields and "bar" fields as separate outputs. .bar`, produces both the "foo" fields and "bar" fields as
separate outputs.
examples: examples:
- program: '.foo, .bar' - program: '.foo, .bar'
input: '{"foo": 42, "bar": "something else", "baz": true}' input: '{"foo": 42, "bar": "something else", "baz": true}'
output: ['42', '"something else"'] output: ['42', '"something else"']
- program: ".user, .projects[]" - program: ".user, .projects[]"
input: '{"user":"stedolan", "projects": ["jq", "wikiflow"]}' input: '{"user":"stedolan", "projects": ["jq", "wikiflow"]}'
output: ['"stedolan"', '"jq"', '"wikiflow"'] output: ['"stedolan"', '"jq"', '"wikiflow"']
- program: '.[4,2]' - program: '.[4,2]'
input: '["a","b","c","d","e"]' input: '["a","b","c","d","e"]'
output: ['"e"', '"c"'] output: ['"e"', '"c"']
- title: "`|`" - title: "Pipe: `|`"
body: | body: |
The | operator combines two filters by feeding the output(s) of The | operator combines two filters by feeding the output(s) of
the one on the left into the input of the one on the right. It's the one on the left into the input of the one on the right. It's
pretty much the same as the Unix shell's pipe, if you're used to pretty much the same as the Unix shell's pipe, if you're used to
that. that.
If the one on the left produces multiple results, the one on If the one on the left produces multiple results, the one on
the right will be run for each of those results. So, the the right will be run for each of those results. So, the
expression `.[] | .foo` retrieves the "foo" field of each expression `.[] | .foo` retrieves the "foo" field of each
element of the input array. element of the input array.
Note that `.a.b.c` is the same as `.a | .b | .c`.
Note too that `.` is the input value at the particular stage
in a "pipeline", specifically: where the `.` expression appears.
Thus `.a | . | .b` is the same as `.a.b`, as the `.` in the
middle refers to whatever value `.a` produced.
examples: examples:
- program: '.[] | .name' - program: '.[] | .name'
input: '[{"name":"JSON", "good":true}, {"name":"XML", "good":false}] ' input: '[{"name":"JSON", "good":true}, {"name":"XML", "good":false}] '
output: ['"JSON"', '"XML"'] output: ['"JSON"', '"XML"']
- title: "Parenthesis"
body: |
Parenthesis work as a grouping operator just as in any typical
programming language.
examples:
- program: '(. + 2) * 5'
input: '1'
output: [15]
- title: Types and Values - title: Types and Values
body: | body: |
jq supports the same set of datatypes as JSON - numbers, jq supports the same set of datatypes as JSON - numbers,
strings, booleans, arrays, objects (which in JSON-speak are strings, booleans, arrays, objects (which in JSON-speak are
hashes with only string keys), and "null". hashes with only string keys), and "null".
Booleans, null, strings and numbers are written the same way as Booleans, null, strings and numbers are written the same way as
in javascript. Just like everything else in jq, these simple in javascript. Just like everything else in jq, these simple
values take an input and produce an output - `42` is a valid jq values take an input and produce an output - `42` is a valid jq
expression that takes an input, ignores it, and returns 42 expression that takes an input, ignores it, and returns 42
instead. instead.
entries: entries:
- title: Array construction - `[]` - title: "Array construction: `[]`"
body: | body: |
As in JSON, `[]` is used to construct arrays, as in As in JSON, `[]` is used to construct arrays, as in
`[1,2,3]`. The elements of the arrays can be any jq `[1,2,3]`. The elements of the arrays can be any jq
expression. All of the results produced by all of the expression, including a pipeline. All of the results produced
expressions are collected into one big array. You can use it by all of the expressions are collected into one big array.
to construct an array out of a known quantity of values (as You can use it to construct an array out of a known quantity
in `[.foo, .bar, .baz]`) or to "collect" all the results of a of values (as in `[.foo, .bar, .baz]`) or to "collect" all the
filter into an array (as in `[.items[].name]`) results of a filter into an array (as in `[.items[].name]`)
Once you understand the "," operator, you can look at jq's array Once you understand the "," operator, you can look at jq's array
syntax in a different light: the expression `[1,2,3]` is not using a syntax in a different light: the expression `[1,2,3]` is not using a
built-in syntax for comma-separated arrays, but is instead applying built-in syntax for comma-separated arrays, but is instead applying
the `[]` operator (collect results) to the expression 1,2,3 (which the `[]` operator (collect results) to the expression 1,2,3 (which
produces three different results). produces three different results).
If you have a filter `X` that produces four results, If you have a filter `X` that produces four results,
then the expression `[X]` will produce a single result, an then the expression `[X]` will produce a single result, an
array of four elements. array of four elements.
examples: examples:
- program: "[.user, .projects[]]" - program: "[.user, .projects[]]"
input: '{"user":"stedolan", "projects": ["jq", "wikiflow"]}' input: '{"user":"stedolan", "projects": ["jq", "wikiflow"]}'
output: ['["stedolan", "jq", "wikiflow"]'] output: ['["stedolan", "jq", "wikiflow"]']
- title: Objects - `{}` - program: "[ .[] | . * 2]"
input: '[1, 2, 3]'
output: ['[2, 4, 6]']
- title: "Object Construction: `{}`"
body: | body: |
Like JSON, `{}` is for constructing objects (aka Like JSON, `{}` is for constructing objects (aka
dictionaries or hashes), as in: `{"a": 42, "b": 17}`. dictionaries or hashes), as in: `{"a": 42, "b": 17}`.
If the keys are "sensible" (all alphabetic characters), then If the keys are "identifier-like", then the quotes can be left
the quotes can be left off. The value can be any expression off, as in `{a:42, b:17}`. Keys generated by expressions need
(although you may need to wrap it in parentheses if it's a to be parenthesized, e.g., `{("a"+"b"):59}`.
complicated one), which gets applied to the {} expression's
input (remember, all filters have an input and an The value can be any expression (although you may need to
output). wrap it in parentheses if it's a complicated one), which gets
applied to the {} expression's input (remember, all filters
have an input and an output).
{foo: .bar} {foo: .bar}
will produce the JSON object `{"foo": 42}` if given the JSON will produce the JSON object `{"foo": 42}` if given the JSON
object `{"bar":42, "baz":43}`. You can use this to select object `{"bar":42, "baz":43}` as its input. You can use this
particular fields of an object: if the input is an object to select particular fields of an object: if the input is an
with "user", "title", "id", and "content" fields and you object with "user", "title", "id", and "content" fields and
just want "user" and "title", you can write you just want "user" and "title", you can write
{user: .user, title: .title} {user: .user, title: .title}
Because that's so common, there's a shortcut syntax: `{user, title}`. Because that is so common, there's a shortcut syntax for it:
`{user, title}`.
If one of the expressions produces multiple results, If one of the expressions produces multiple results,
multiple dictionaries will be produced. If the input's multiple dictionaries will be produced. If the input's
{"user":"stedolan","titles":["JQ Primer", "More JQ"]} {"user":"stedolan","titles":["JQ Primer", "More JQ"]}
then the expression then the expression
{user, title: .titles[]} {user, title: .titles[]}
skipping to change at line 548 skipping to change at line 606
examples: examples:
- program: '{user, title: .titles[]}' - program: '{user, title: .titles[]}'
input: '{"user":"stedolan","titles":["JQ Primer", "More JQ"]}' input: '{"user":"stedolan","titles":["JQ Primer", "More JQ"]}'
output: output:
- '{"user":"stedolan", "title": "JQ Primer"}' - '{"user":"stedolan", "title": "JQ Primer"}'
- '{"user":"stedolan", "title": "More JQ"}' - '{"user":"stedolan", "title": "More JQ"}'
- program: '{(.user): .titles}' - program: '{(.user): .titles}'
input: '{"user":"stedolan","titles":["JQ Primer", "More JQ"]}' input: '{"user":"stedolan","titles":["JQ Primer", "More JQ"]}'
output: ['{"stedolan": ["JQ Primer", "More JQ"]}'] output: ['{"stedolan": ["JQ Primer", "More JQ"]}']
- title: "Recursive Descent: `..`"
body: |
Recursively descends `.`, producing every value. This is the
same as the zero-argument `recurse` builtin (see below). This
is intended to resemble the XPath `//` operator. Note that
`..a` does not work; use `..|.a` instead. In the example
below we use `..|.a?` to find all the values of object keys
"a" in any object found "below" `.`.
This is particularly useful in conjunction with `path(EXP)`
(also see below) and the `?` operator.
examples:
- program: '..|.a?'
input: '[[{"a":1}]]'
output: ['1']
- title: Builtin operators and functions - title: Builtin operators and functions
body: | body: |
Some jq operator (for instance, `+`) do different things Some jq operator (for instance, `+`) do different things
depending on the type of their arguments (arrays, numbers, depending on the type of their arguments (arrays, numbers,
etc.). However, jq never does implicit type conversions. If you etc.). However, jq never does implicit type conversions. If you
try to add a string to an object you'll get an error message and try to add a string to an object you'll get an error message and
no result. no result.
entries: entries:
- title: Addition - `+` - title: "Addition: `+`"
body: | body: |
The operator `+` takes two filters, applies them both The operator `+` takes two filters, applies them both
to the same input, and adds the results together. What to the same input, and adds the results together. What
"adding" means depends on the types involved: "adding" means depends on the types involved:
- **Numbers** are added by normal arithmetic. - **Numbers** are added by normal arithmetic.
- **Arrays** are added by being concatenated into a larger array. - **Arrays** are added by being concatenated into a larger array.
skipping to change at line 597 skipping to change at line 673
- program: '.a + null' - program: '.a + null'
input: '{"a": 1}' input: '{"a": 1}'
output: ['1'] output: ['1']
- program: '.a + 1' - program: '.a + 1'
input: '{}' input: '{}'
output: ['1'] output: ['1']
- program: '{a: 1} + {b: 2} + {c: 3} + {a: 42}' - program: '{a: 1} + {b: 2} + {c: 3} + {a: 42}'
input: 'null' input: 'null'
output: ['{"a": 42, "b": 2, "c": 3}'] output: ['{"a": 42, "b": 2, "c": 3}']
- title: Subtraction - `-` - title: "Subtraction: `-`"
body: | body: |
As well as normal arithmetic subtraction on numbers, the `-` As well as normal arithmetic subtraction on numbers, the `-`
operator can be used on arrays to remove all occurrences of operator can be used on arrays to remove all occurrences of
the second array's elements from the first array. the second array's elements from the first array.
examples: examples:
- program: '4 - .a' - program: '4 - .a'
input: '{"a":3}' input: '{"a":3}'
output: ['1'] output: ['1']
- program: . - ["xml", "yaml"] - program: . - ["xml", "yaml"]
input: '["xml", "yaml", "json"]' input: '["xml", "yaml", "json"]'
output: ['["json"]'] output: ['["json"]']
- title: Multiplication, division, modulo - `*`, `/`, and `%` - title: "Multiplication, division, modulo: `*`, `/`, and `%`"
body: | body: |
These infix operators behave as expected when given two numbers. These infix operators behave as expected when given two numbers.
Division by zero raises an error. `x % y` computes x modulo y. Division by zero raises an error. `x % y` computes x modulo y.
Multiplying a string by a number produces the concatenation of Multiplying a string by a number produces the concatenation of
that string that many times. `"x" * 0` produces **null**. that string that many times. `"x" * 0` produces **null**.
Dividing a string by another splits the first using the second Dividing a string by another splits the first using the second
as separators. as separators.
skipping to change at line 664 skipping to change at line 740
- The length of an **object** is the number of key-value pairs. - The length of an **object** is the number of key-value pairs.
- The length of **null** is zero. - The length of **null** is zero.
examples: examples:
- program: '.[] | length' - program: '.[] | length'
input: '[[1,2], "string", {"a":2}, null]' input: '[[1,2], "string", {"a":2}, null]'
output: [2, 6, 1, 0] output: [2, 6, 1, 0]
- title: "`utf8bytelength`"
body: |
The builtin function `utf8bytelength` outputs the number of
bytes used to encode a string in UTF-8.
examples:
- program: 'utf8bytelength'
input: '"\u03bc"'
output: [2]
- title: "`keys`, `keys_unsorted`" - title: "`keys`, `keys_unsorted`"
body: | body: |
The builtin function `keys`, when given an object, returns The builtin function `keys`, when given an object, returns
its keys in an array. its keys in an array.
The keys are sorted "alphabetically", by unicode codepoint The keys are sorted "alphabetically", by unicode codepoint
order. This is not an order that makes particular sense in order. This is not an order that makes particular sense in
any particular language, but you can count on it being the any particular language, but you can count on it being the
same for any two objects with the same set of keys, same for any two objects with the same set of keys,
skipping to change at line 713 skipping to change at line 800
- program: 'map(has("foo"))' - program: 'map(has("foo"))'
input: '[{"foo": 42}, {}]' input: '[{"foo": 42}, {}]'
output: ['[true, false]'] output: ['[true, false]']
- program: 'map(has(2))' - program: 'map(has(2))'
input: '[[0,1], ["a","b","c"]]' input: '[[0,1], ["a","b","c"]]'
output: ['[false, true]'] output: ['[false, true]']
- title: "`in`" - title: "`in`"
body: | body: |
The builtin function `in` returns the input key is in the The builtin function `in` returns whether or not the input key is in t he
given object, or the input index corresponds to an element given object, or the input index corresponds to an element
in the given array. It is, essentially, an inversed version in the given array. It is, essentially, an inversed version
of `has`. of `has`.
examples: examples:
- program: '.[] | in({"foo": 42})' - program: '.[] | in({"foo": 42})'
input: '["foo", "bar"]' input: '["foo", "bar"]'
output: ['true', 'false'] output: ['true', 'false']
- program: 'map(in([0,1]))' - program: 'map(in([0,1]))'
input: '[2, 0]' input: '[2, 0]'
output: ['[false, true]'] output: ['[false, true]']
- title: "`map(x)`, `map_values(x)`"
body: |
For any filter `x`, `map(x)` will run that filter for each
element of the input array, and return the outputs in a new
array. `map(.+1)` will increment each element of an array of numbers.
Similarly, `map_values(x)` will run that filter for each element,
but it will return an object when an object is passed.
`map(x)` is equivalent to `[.[] | x]`. In fact, this is how
it's defined. Similarly, `map_values(x)` is defined as `.[] |= x`.
examples:
- program: 'map(.+1)'
input: '[1,2,3]'
output: ['[2,3,4]']
- program: 'map_values(.+1)'
input: '{"a": 1, "b": 2, "c": 3}'
output: ['{"a": 2, "b": 3, "c": 4}']
- title: "`path(path_expression)`" - title: "`path(path_expression)`"
body: | body: |
Outputs array representations of the given path expression Outputs array representations of the given path expression
in `.`. The outputs are arrays of strings (keys in objects0 in `.`. The outputs are arrays of strings (object keys)
and/or numbers (array indices. and/or numbers (array indices).
Path expressions are jq expressions like `.a`, but also `.[]`. Path expressions are jq expressions like `.a`, but also `.[]`.
There are two types of path expressions: ones that can match There are two types of path expressions: ones that can match
exactly, and ones that cannot. For example, `.a.b.c` is an exactly, and ones that cannot. For example, `.a.b.c` is an
exact match path expression, while `.a[].b` is not. exact match path expression, while `.a[].b` is not.
`path(exact_path_expression)` will produce the array `path(exact_path_expression)` will produce the array
representation of the path expression even if it does not representation of the path expression even if it does not
exist in `.`, if `.` is `null` or an array or an object. exist in `.`, if `.` is `null` or an array or an object.
skipping to change at line 772 skipping to change at line 881
value from an object. value from an object.
examples: examples:
- program: 'del(.foo)' - program: 'del(.foo)'
input: '{"foo": 42, "bar": 9001, "baz": 42}' input: '{"foo": 42, "bar": 9001, "baz": 42}'
output: ['{"bar": 9001, "baz": 42}'] output: ['{"bar": 9001, "baz": 42}']
- program: 'del(.[1, 2])' - program: 'del(.[1, 2])'
input: '["foo", "bar", "baz"]' input: '["foo", "bar", "baz"]'
output: ['["foo"]'] output: ['["foo"]']
- title: "`getpath(PATHS)`"
body: |
The builtin function `getpath` outputs the values in `.` found
at each path in `PATHS`.
examples:
- program: 'getpath(["a","b"])'
input: 'null'
output: ['null']
- program: '[getpath(["a","b"], ["a","c"])]'
input: '{"a":{"b":0, "c":1}}'
output: ['[0, 1]']
- title: "`setpath(PATHS; VALUE)`"
body: |
The builtin function `setpath` sets the `PATHS` in `.` to `VALUE`.
examples:
- program: 'setpath(["a","b"]; 1)'
input: 'null'
output: ['{"a": {"b": 1}}']
- program: 'setpath(["a","b"]; 1)'
input: '{"a":{"b":0}}'
output: ['{"a": {"b": 1}}']
- program: 'setpath([0,"a"]; 1)'
input: 'null'
output: ['[{"a":1}]']
- title: "`delpaths(PATHS)`"
body: |
The builtin function `delpaths` sets the `PATHS` in `.`.
`PATHS` must be an array of paths, where each path is an array
of strings and numbers.
examples:
- program: 'delpaths([["a","b"]])'
input: '{"a":{"b":1},"x":{"y":2}}'
output: ['{"a":{},"x":{"y":2}}']
- title: "`to_entries`, `from_entries`, `with_entries`" - title: "`to_entries`, `from_entries`, `with_entries`"
body: | body: |
These functions convert between an object and an array of These functions convert between an object and an array of
key-value pairs. If `to_entries` is passed an object, then key-value pairs. If `to_entries` is passed an object, then
for each `k: v` entry in the input, the output array for each `k: v` entry in the input, the output array
includes `{"key": k, "value": v}`. includes `{"key": k, "value": v}`.
`from_entries` does the opposite conversion, and `from_entries` does the opposite conversion, and
`with_entries(foo)` is a shorthand for `to_entries | `with_entries(foo)` is a shorthand for `to_entries |
map(foo) | from_entries`, useful for doing some operation to map(foo) | from_entries`, useful for doing some operation to
all keys and values of an object. `from_entries` accepts key, Key, all keys and values of an object. `from_entries` accepts key, Key,
Name, value and Value as keys. name, Name, value and Value as keys.
examples: examples:
- program: 'to_entries' - program: 'to_entries'
input: '{"a": 1, "b": 2}' input: '{"a": 1, "b": 2}'
output: ['[{"key":"a", "value":1}, {"key":"b", "value":2}]'] output: ['[{"key":"a", "value":1}, {"key":"b", "value":2}]']
- program: 'from_entries' - program: 'from_entries'
input: '[{"key":"a", "value":1}, {"key":"b", "value":2}]' input: '[{"key":"a", "value":1}, {"key":"b", "value":2}]'
output: ['{"a": 1, "b": 2}'] output: ['{"a": 1, "b": 2}']
- program: 'with_entries(.key |= "KEY_" + .)' - program: 'with_entries(.key |= "KEY_" + .)'
input: '{"a": 1, "b": 2}' input: '{"a": 1, "b": 2}'
skipping to change at line 848 skipping to change at line 999
output: [1, 2] output: [1, 2]
- program: '[1,2,empty,3]' - program: '[1,2,empty,3]'
input: 'null' input: 'null'
output: ['[1,2,3]'] output: ['[1,2,3]']
- title: "`error(message)`" - title: "`error(message)`"
body: | body: |
Produces an error, just like `.a` applied to values other than Produces an error, just like `.a` applied to values other than
null and objects would, but with the given message as the null and objects would, but with the given message as the
error's value. error's value. Errors can be caught with try/catch; see below.
- title: "`halt`"
body: |
Stops the jq program with no further outputs. jq will exit
with exit status `0`.
- title: "`halt_error`, `halt_error(exit_code)`"
body: |
Stops the jq program with no further outputs. The input will
be printed on `stderr` as raw output (i.e., strings will not
have double quotes) with no decoration, not even a newline.
The given `exit_code` (defaulting to `5`) will be jq's exit
status.
For example, `"Error: somthing went wrong\n"|halt_error(1)`.
- title: "`$__loc__`" - title: "`$__loc__`"
body: | body: |
Produces an object with a "file" key and a "line" key, with Produces an object with a "file" key and a "line" key, with
the filename and line number where `$__loc__` occurs, as the filename and line number where `$__loc__` occurs, as
values. values.
examples: examples:
- program: 'try error("\($__loc__)") catch .' - program: 'try error("\($__loc__)") catch .'
input: 'null' input: 'null'
output: ['"{\"file\":\"<top-level>\",\"line\":1}"'] output: ['"{\"file\":\"<top-level>\",\"line\":1}"']
- title: "`map(x)`, `map_values(x)`"
body: |
For any filter `x`, `map(x)` will run that filter for each
element of the input array, and produce the outputs a new
array. `map(.+1)` will increment each element of an array of numbers.
Similarly, `map_values(x)` will run that filter for each element,
but it will return an object when an object is passed.
`map(x)` is equivalent to `[.[] | x]`. In fact, this is how
it's defined. Similarly, `map_values(x)` is defined as `.[] |= x`.
examples:
- program: 'map(.+1)'
input: '[1,2,3]'
output: ['[2,3,4]']
- program: 'map_values(.+1)'
input: '{"a": 1, "b": 2, "c": 3}'
output: ['{"a": 2, "b": 3, "c": 4}']
- title: "`paths`, `paths(node_filter)`, `leaf_paths`" - title: "`paths`, `paths(node_filter)`, `leaf_paths`"
body: | body: |
`paths` outputs the paths to all the elements in its input `paths` outputs the paths to all the elements in its input
(except it does not output the empty list, representing . (except it does not output the empty list, representing .
itself). itself).
`paths(f)` outputs the paths to any values for which `f` is true. `paths(f)` outputs the paths to any values for which `f` is true.
That is, `paths(numbers)` outputs the paths to all numeric That is, `paths(numbers)` outputs the paths to all numeric
values. values.
skipping to change at line 932 skipping to change at line 1079
input: '[1, 2, 3]' input: '[1, 2, 3]'
output: [6] output: [6]
- program: add - program: add
input: '[]' input: '[]'
output: ["null"] output: ["null"]
- title: "`any`, `any(condition)`, `any(generator; condition)`" - title: "`any`, `any(condition)`, `any(generator; condition)`"
body: | body: |
The filter `any` takes as input an array of boolean values, The filter `any` takes as input an array of boolean values,
and produces `true` as output if any of the the elements of and produces `true` as output if any of the elements of
the array is `true`. the array are `true`.
If the input is an empty array, `any` returns `false`. If the input is an empty array, `any` returns `false`.
The `any(condition)` form applies the given condition to the The `any(condition)` form applies the given condition to the
elements of the input array. elements of the input array.
The `any(generator; condition)` form applies the given The `any(generator; condition)` form applies the given
condition to all the outputs of the given generator. condition to all the outputs of the given generator.
examples: examples:
skipping to change at line 958 skipping to change at line 1105
input: '[false, false]' input: '[false, false]'
output: ["false"] output: ["false"]
- program: any - program: any
input: '[]' input: '[]'
output: ["false"] output: ["false"]
- title: "`all`, `all(condition)`, `all(generator; condition)`" - title: "`all`, `all(condition)`, `all(generator; condition)`"
body: | body: |
The filter `all` takes as input an array of boolean values, The filter `all` takes as input an array of boolean values,
and produces `true` as output if all of the the elements of and produces `true` as output if all of the elements of
the array are `true`. the array are `true`.
The `all(condition)` form applies the given condition to the The `all(condition)` form applies the given condition to the
elements of the input array. elements of the input array.
The `all(generator; condition)` form applies the given The `all(generator; condition)` form applies the given
condition to all the outputs of the given generator. condition to all the outputs of the given generator.
If the input is an empty array, `all` returns `true`. If the input is an empty array, `all` returns `true`.
skipping to change at line 980 skipping to change at line 1127
- program: all - program: all
input: '[true, false]' input: '[true, false]'
output: ["false"] output: ["false"]
- program: all - program: all
input: '[true, true]' input: '[true, true]'
output: ["true"] output: ["true"]
- program: all - program: all
input: '[]' input: '[]'
output: ["true"] output: ["true"]
- title: "\\[Requires 1.5\\] `flatten`, `flatten(depth)`" - title: "`flatten`, `flatten(depth)`"
body: | body: |
The filter `flatten` takes as input an array of nested arrays, The filter `flatten` takes as input an array of nested arrays,
and produces a flat array in which all arrays inside the original and produces a flat array in which all arrays inside the original
array have been recursively replaced by their values. You can pass array have been recursively replaced by their values. You can pass
an argument to it to specify how many levels of nesting to flatten. an argument to it to specify how many levels of nesting to flatten.
`flatten(2)` is like `flatten`, but going only up to two `flatten(2)` is like `flatten`, but going only up to two
levels deep. levels deep.
skipping to change at line 1386 skipping to change at line 1533
- title: "`implode`" - title: "`implode`"
body: | body: |
The inverse of explode. The inverse of explode.
examples: examples:
- program: 'implode' - program: 'implode'
input: '[65, 66, 67]' input: '[65, 66, 67]'
output: ['"ABC"'] output: ['"ABC"']
- title: "`split`" - title: "`split(str)`"
body: | body: |
Splits an input string on the separator argument. Splits an input string on the separator argument.
examples: examples:
- program: 'split(", ")' - program: 'split(", ")'
input: '"a, b,c,d, e, "' input: '"a, b,c,d, e, "'
output: ['["a","b,c,d","e",""]'] output: ['["a","b,c,d","e",""]']
- title: "`join(str)`" - title: "`join(str)`"
body: | body: |
Joins the array of elements given as input, using the Joins the array of elements given as input, using the
argument as separator. It is the inverse of `split`: that is, argument as separator. It is the inverse of `split`: that is,
running `split("foo") | join("foo")` over any input string running `split("foo") | join("foo")` over any input string
returns said input string. returns said input string.
Numbers and booleans in the input are converted to strings.
Null values are treated as empty strings. Arrays and objects
in the input are not supported.
examples: examples:
- program: 'join(", ")' - program: 'join(", ")'
input: '["a","b,c,d","e"]' input: '["a","b,c,d","e"]'
output: ['"a, b,c,d, e"'] output: ['"a, b,c,d, e"']
- program: 'join(" ")'
input: '["a",1,2.3,true,null,false]'
output: ['"a 1 2.3 true false"']
- title: "`ascii_downcase`, `ascii_upcase`" - title: "`ascii_downcase`, `ascii_upcase`"
body: | body: |
Emit a copy of the input string with its alphabetic characters (a-z an d A-Z) Emit a copy of the input string with its alphabetic characters (a-z an d A-Z)
converted to the specified case. converted to the specified case.
example: example:
- program: 'ascii_upcase' - program: 'ascii_upcase'
input: '"useful but not for é"' input: '"useful but not for é"'
skipping to change at line 1520 skipping to change at line 1674
- '[1]' - '[1]'
- '1' - '1'
- program: 'recurse(. * .; . < 20)' - program: 'recurse(. * .; . < 20)'
input: 2 input: 2
output: output:
- 2 - 2
- 4 - 4
- 16 - 16
- title: "`..`" - title: "`walk(f)`"
body: | body: |
Short-hand for `recurse` without arguments. This is intended The `walk(f)` function applies f recursively to every
to resemble the XPath `//` operator. Note that `..a` does not component of the input entity. When an array is
work; use `..|a` instead. In the example below we use encountered, f is first applied to its elements and then to
`..|.a?` to find all the values of object keys "a" in any the array itself; when an object is encountered, f is first
object found "below" `.`. applied to all the values and then to the object. In
practice, f will usually test the type of its input, as
illustrated in the following examples. The first example
highlights the usefulness of processing the elements of an
array of arrays before processing the array itself. The second
example shows how all the keys of all the objects within the
input can be considered for alteration.
examples: examples:
- program: '..|.a?' - program: 'walk(if type == "array" then sort else . end)'
input: '[[{"a":1}]]' input: '[[4, 1, 7], [8, 5, 2], [3, 6, 9]]'
output: ['1'] output:
- '[[1,4,7],[2,5,8],[3,6,9]]'
- title: "`env`" - program: 'walk( if type == "object" then with_entries( .key |= sub(
"^_+"; "") ) else . end )'
input: '[ { "_a": { "__b": 2 } } ]'
output:
- '[{"a":{"b":2}}]'
- title: "`$ENV`, `env`"
body: | body: |
Outputs an object representing jq's environment. `$ENV` is an object representing the environment variables as
set when the jq program started.
`env` outputs an object representing jq's current environment.
At the moment there is no builtin for setting environment
variables.
examples: examples:
- program: '$ENV.PAGER'
input: 'null'
output: ['"less"']
- program: 'env.PAGER' - program: 'env.PAGER'
input: 'null' input: 'null'
output: ['"less"'] output: ['"less"']
- title: "`transpose`" - title: "`transpose`"
body: | body: |
Transpose a possibly jagged matrix (an array of arrays). Transpose a possibly jagged matrix (an array of arrays).
Rows are padded with nulls so the result is always rectangular. Rows are padded with nulls so the result is always rectangular.
skipping to change at line 1662 skipping to change at line 1838
* `@sh`: * `@sh`:
The input is escaped suitable for use in a command-line The input is escaped suitable for use in a command-line
for a POSIX shell. If the input is an array, the output for a POSIX shell. If the input is an array, the output
will be a series of space-separated strings. will be a series of space-separated strings.
* `@base64`: * `@base64`:
The input is converted to base64 as specified by RFC 4648. The input is converted to base64 as specified by RFC 4648.
* `@base64d`:
The inverse of `@base64`, input is decoded as specified by RFC 4648.
Note\: If the decoded string is not UTF-8, the results are undefined
.
This syntax can be combined with string interpolation in a This syntax can be combined with string interpolation in a
useful way. You can follow a `@foo` token with a string useful way. You can follow a `@foo` token with a string
literal. The contents of the string literal will *not* be literal. The contents of the string literal will *not* be
escaped. However, all interpolations made inside that string escaped. However, all interpolations made inside that string
literal will be escaped. For instance, literal will be escaped. For instance,
@uri "https://www.google.com/search?q=\(.search)" @uri "https://www.google.com/search?q=\(.search)"
will produce the following output for the input will produce the following output for the input
`{"search":"what is jq?"}`: `{"search":"what is jq?"}`:
skipping to change at line 1691 skipping to change at line 1872
output: ['"This works if x &lt; y"'] output: ['"This works if x &lt; y"']
# - program: '@html "<span>Anonymous said: \(.)</span>"' # - program: '@html "<span>Anonymous said: \(.)</span>"'
# input: '"<script>alert(\"lol hax\");</script>"' # input: '"<script>alert(\"lol hax\");</script>"'
# output: ["<span>Anonymous said: &lt;script&gt;alert(&quot;lol hax&q uot;);&lt;/script&gt;</span>"] # output: ["<span>Anonymous said: &lt;script&gt;alert(&quot;lol hax&q uot;);&lt;/script&gt;</span>"]
- program: '@sh "echo \(.)"' - program: '@sh "echo \(.)"'
input: "\"O'Hara's Ale\"" input: "\"O'Hara's Ale\""
output: ["\"echo 'O'\\\\''Hara'\\\\''s Ale'\""] output: ["\"echo 'O'\\\\''Hara'\\\\''s Ale'\""]
- program: '@base64'
input: '"This is a message"'
output: ['"VGhpcyBpcyBhIG1lc3NhZ2U="']
- program: '@base64d'
input: '"VGhpcyBpcyBhIG1lc3NhZ2U="'
output: ['"This is a message"']
- title: "Dates" - title: "Dates"
body: | body: |
jq provides some basic date handling functionality, with some jq provides some basic date handling functionality, with some
high-level and low-level builtins. In all cases these high-level and low-level builtins. In all cases these
builtins deal exclusively with time in UTC. builtins deal exclusively with time in UTC.
The `fromdateiso8601` builtin parses datetimes in the ISO 8601 The `fromdateiso8601` builtin parses datetimes in the ISO 8601
format to a number of seconds since the Unix epoch format to a number of seconds since the Unix epoch
(1970-01-01T00:00:00Z). The `todateiso8601` builtin does the (1970-01-01T00:00:00Z). The `todateiso8601` builtin does the
skipping to change at line 1714 skipping to change at line 1903
`fromdate` only supports ISO 8601 datetime strings, but in the `fromdate` only supports ISO 8601 datetime strings, but in the
future it will attempt to parse datetime strings in more future it will attempt to parse datetime strings in more
formats. formats.
The `todate` builtin is an alias for `todateiso8601`. The `todate` builtin is an alias for `todateiso8601`.
The `now` builtin outputs the current time, in seconds since The `now` builtin outputs the current time, in seconds since
the Unix epoch. the Unix epoch.
Low-level jq interfaces to the C-library time functions are Low-level jq interfaces to the C-library time functions are
also provided: `strptime`, `strftime`, `mktime`, and `gmtime`. also provided: `strptime`, `strftime`, `strflocaltime`,
Refer to your host operating system's documentation for the `mktime`, `gmtime`, and `localtime`. Refer to your host
format strings used by `strptime` and `strftime`. Note: these operating system's documentation for the format strings used
are not necessarily stable interfaces in jq, particularly as by `strptime` and `strftime`. Note: these are not necessarily
to their localization functionality. stable interfaces in jq, particularly as to their localization
functionality.
The `gmtime` builtin consumes a number of seconds since the The `gmtime` builtin consumes a number of seconds since the
Unix epoch and outputs a "broken down time" representation of Unix epoch and outputs a "broken down time" representation of
time as an array of numbers representing (in this order): the Greenwhich Meridian time as an array of numbers representing
year, the month (zero-based), the day of the month, the hour (in this order): the year, the month (zero-based), the day of
of the day, the minute of the hour, the second of the minute, the month (one-based), the hour of the day, the minute of the
the day of the week, and the day of the year -- all one-based hour, the second of the minute, the day of the week, and the
unless otherwise stated. day of the year -- all one-based unless otherwise stated. The
day of the week number may be wrong on some systems for dates
before March 1st 1900, or after December 31 2099.
The `localtime` builtin works like the `gmtime` builtin, but
using the local timezone setting.
The `mktime` builtin consumes "broken down time" The `mktime` builtin consumes "broken down time"
representations of time output by `gmtime` and `strptime`. representations of time output by `gmtime` and `strptime`.
The `strptime(fmt)` builtin parses input strings matching the The `strptime(fmt)` builtin parses input strings matching the
`fmt` argument. The output is in the "broken down time" `fmt` argument. The output is in the "broken down time"
representation consumed by `gmtime` and output by `mktime`. representation consumed by `gmtime` and output by `mktime`.
The `strftime(fmt)` builtin formats a time with the given The `strftime(fmt)` builtin formats a time (GMT) with the
format. given format. The `strflocaltime` does the same, but using
the local timezone setting.
The format strings for `strptime` and `strftime` are described The format strings for `strptime` and `strftime` are described
in typical C library documentation. The format string for ISO in typical C library documentation. The format string for ISO
8601 datetime is `"%Y-%m-%dT%H:%M:%SZ"`. 8601 datetime is `"%Y-%m-%dT%H:%M:%SZ"`.
jq may not support some or all of this date functionality on jq may not support some or all of this date functionality on
some systems. some systems. In particular, the `%u` and `%j` specifiers for
`strptime(fmt)` are not supported on macOS.
examples: examples:
- program: 'fromdate' - program: 'fromdate'
input: '"2015-03-05T23:51:47Z"' input: '"2015-03-05T23:51:47Z"'
output: ['1425599507'] output: ['1425599507']
- program: 'strptime("%Y-%m-%dT%H:%M:%SZ")' - program: 'strptime("%Y-%m-%dT%H:%M:%SZ")'
input: '"2015-03-05T23:51:47Z"' input: '"2015-03-05T23:51:47Z"'
output: ['[2015,2,5,23,51,47,4,63]'] output: ['[2015,2,5,23,51,47,4,63]']
- program: 'strptime("%Y-%m-%dT%H:%M:%SZ")|mktime' - program: 'strptime("%Y-%m-%dT%H:%M:%SZ")|mktime'
input: '"2015-03-05T23:51:47Z"' input: '"2015-03-05T23:51:47Z"'
output: ['1425599507'] output: ['1425599507']
- title: "SQL-Style Operators"
body: |
jq provides a few SQL-style operators.
* INDEX(stream; index_expression):
This builtin produces an object whose keys are computed by
the given index expression applied to each value from the
given stream.
* JOIN($idx; stream; idx_expr; join_expr):
This builtin joins the values from the given stream to the
given index. The index's keys are computed by applying the
given index expression to each value from the given stream.
An array of the value in the stream and the corresponding
value from the index is fed to the given join expression to
produce each result.
* JOIN($idx; stream; idx_expr):
Same as `JOIN($idx; stream; idx_expr; .)`.
* JOIN($idx; idx_expr):
This builtin joins the input `.` to the given index, applying
the given index expression to `.` to compute the index key.
The join operation is as described above.
* IN(s):
This builtin outputs `true` if `.` appears in the given
stream, otherwise it outputs `false`.
* IN(source; s):
This builtin outputs `true` if any value in the source stream
appears in the second stream, otherwise it outputs `false`.
- title: "`builtins`"
body: |
Returns a list of all builtin functions in the format `name/arity`.
Since functions with the same name but different arities are considere
d
separate functions, `all/0`, `all/1`, and `all/2` would all be present
in the list.
- title: Conditionals and Comparisons - title: Conditionals and Comparisons
entries: entries:
- title: "`==`, `!=`" - title: "`==`, `!=`"
body: | body: |
The expression 'a == b' will produce 'true' if the result of a and b The expression 'a == b' will produce 'true' if the result of a and b
are equal (that is, if they represent equivalent JSON documents) and are equal (that is, if they represent equivalent JSON documents) and
'false' otherwise. In particular, strings are never considered equal 'false' otherwise. In particular, strings are never considered equal
to numbers. If you're coming from Javascript, jq's == is like to numbers. If you're coming from Javascript, jq's == is like
Javascript's === - considering values equal only when they have the Javascript's === - considering values equal only when they have the
skipping to change at line 1792 skipping to change at line 2037
as `C` otherwise. as `C` otherwise.
Checking for false or null is a simpler notion of Checking for false or null is a simpler notion of
"truthiness" than is found in Javascript or Python, but it "truthiness" than is found in Javascript or Python, but it
means that you'll sometimes have to be more explicit about means that you'll sometimes have to be more explicit about
the condition you want: you can't test whether, e.g. a the condition you want: you can't test whether, e.g. a
string is empty using `if .name then A else B end`, you'll string is empty using `if .name then A else B end`, you'll
need something more like `if (.name | length) > 0 then A else need something more like `if (.name | length) > 0 then A else
B end` instead. B end` instead.
If the condition A produces multiple results, it is If the condition `A` produces multiple results, then `B` is evaluated
considered "true" if any of those results is not false or once for each result that is not false or null, and `C` is evaluated
null. If it produces zero results, it's considered false. once for each false or null.
More cases can be added to an if using `elif A then B` syntax. More cases can be added to an if using `elif A then B` syntax.
examples: examples:
- program: |- - program: |-
if . == 0 then if . == 0 then
"zero" "zero"
elif . == 1 then elif . == 1 then
"one" "one"
else else
skipping to change at line 1864 skipping to change at line 2109
# - program: '(true, false) and (true, false)' # - program: '(true, false) and (true, false)'
# input: 'null' # input: 'null'
# output: ['true', 'false', 'false', 'false'] # output: ['true', 'false', 'false', 'false']
- program: '(true, true) and (true, false)' - program: '(true, true) and (true, false)'
input: 'null' input: 'null'
output: ['true', 'false', 'true', 'false'] output: ['true', 'false', 'true', 'false']
- program: '[true, false | not]' - program: '[true, false | not]'
input: 'null' input: 'null'
output: ['[false, true]'] output: ['[false, true]']
- title: Alternative operator - `//` - title: "Alternative operator: `//`"
body: | body: |
A filter of the form `a // b` produces the same A filter of the form `a // b` produces the same
results as `a`, if `a` produces results other than `false` results as `a`, if `a` produces results other than `false`
and `null`. Otherwise, `a // b` produces the same results as `b`. and `null`. Otherwise, `a // b` produces the same results as `b`.
This is useful for providing defaults: `.foo // 1` will This is useful for providing defaults: `.foo // 1` will
evaluate to `1` if there's no `.foo` element in the evaluate to `1` if there's no `.foo` element in the
input. It's similar to how `or` is sometimes used in Python input. It's similar to how `or` is sometimes used in Python
(jq's `or` operator is reserved for strictly Boolean (jq's `or` operator is reserved for strictly Boolean
skipping to change at line 1941 skipping to change at line 2186
To break out of a `reduce`, for example: To break out of a `reduce`, for example:
label $out | reduce .[] as $item (null; if .==false then break $ou t else ... end) label $out | reduce .[] as $item (null; if .==false then break $ou t else ... end)
The following jq program produces a syntax error: The following jq program produces a syntax error:
break $out break $out
because no label `$out` is visible. because no label `$out` is visible.
- title: "`?` operator" - title: "Error Suppression / Optional Operator: `?`"
body: | body: |
The `?` operator, used as `EXP?`, is shorthand for `try EXP`. The `?` operator, used as `EXP?`, is shorthand for `try EXP`.
examples: examples:
- program: '[.[]|(.a)?]' - program: '[.[]|(.a)?]'
input: '[{}, true, {"a":1}]' input: '[{}, true, {"a":1}]'
output: ['[null, 1]'] output: ['[null, 1]']
- title: Regular expressions (PCRE) - title: Regular expressions (PCRE)
skipping to change at line 1993 skipping to change at line 2238
* test( "a\\sb", "x" ). * test( "a\\sb", "x" ).
Note that certain flags may also be specified within REGEX, e.g. Note that certain flags may also be specified within REGEX, e.g.
* jq -n '("test", "TEst", "teST", "TEST") | test( "(?i)te(?-i)st" )' * jq -n '("test", "TEst", "teST", "TEST") | test( "(?i)te(?-i)st" )'
evaluates to: true, true, false, false. evaluates to: true, true, false, false.
entries: entries:
- title: "\\[Requires 1.5\\] `test(val)`, `test(regex; flags)`" - title: "`test(val)`, `test(regex; flags)`"
body: | body: |
Like `match`, but does not return match objects, only `true` or `false ` Like `match`, but does not return match objects, only `true` or `false `
for whether or not the regex matches the input. for whether or not the regex matches the input.
examples: examples:
- program: 'test("foo")' - program: 'test("foo")'
input: '"foo"' input: '"foo"'
output: ['true'] output: ['true']
- program: '.[] | test("a b c # spaces are ignored"; "ix")' - program: '.[] | test("a b c # spaces are ignored"; "ix")'
input: '["xabcd", "ABC"]' input: '["xabcd", "ABC"]'
output: ['true', 'true'] output: ['true', 'true']
- title: "\\[Requires 1.5\\] `match(val)`, `match(regex; flags)`" - title: "`match(val)`, `match(regex; flags)`"
body: | body: |
**match** outputs an object for each match it finds. Matches have **match** outputs an object for each match it finds. Matches have
the following fields: the following fields:
* `offset` - offset in UTF-8 codepoints from the beginning of the inpu t * `offset` - offset in UTF-8 codepoints from the beginning of the inpu t
* `length` - length in UTF-8 codepoints of the match * `length` - length in UTF-8 codepoints of the match
* `string` - the string that it matched * `string` - the string that it matched
* `captures` - an array of objects representing capturing groups. * `captures` - an array of objects representing capturing groups.
skipping to change at line 2051 skipping to change at line 2296
- program: 'match("foo (?<bar123>bar)? foo"; "ig")' - program: 'match("foo (?<bar123>bar)? foo"; "ig")'
input: '"foo bar foo foo foo"' input: '"foo bar foo foo foo"'
output: output:
- '{"offset": 0, "length": 11, "string": "foo bar foo", "captures": [{"offset": 4, "length": 3, "string": "bar", "name": "bar123"}]}' - '{"offset": 0, "length": 11, "string": "foo bar foo", "captures": [{"offset": 4, "length": 3, "string": "bar", "name": "bar123"}]}'
- '{"offset": 12, "length": 8, "string": "foo foo", "captures": [{ "offset": -1, "length": 0, "string": null, "name": "bar123"}]}' - '{"offset": 12, "length": 8, "string": "foo foo", "captures": [{ "offset": -1, "length": 0, "string": null, "name": "bar123"}]}'
- program: '[ match("."; "g")] | length' - program: '[ match("."; "g")] | length'
input: '"abc"' input: '"abc"'
output: [3] output: [3]
- title: "\\[Requires 1.5\\] `capture(val)`, `capture(regex; flags)`" - title: "`capture(val)`, `capture(regex; flags)`"
body: | body: |
Collects the named captures in a JSON object, with the name Collects the named captures in a JSON object, with the name
of each capture as the key, and the matched string as the of each capture as the key, and the matched string as the
corresponding value. corresponding value.
examples: examples:
- program: 'capture("(?<a>[a-z]+)-(?<n>[0-9]+)")' - program: 'capture("(?<a>[a-z]+)-(?<n>[0-9]+)")'
input: '"xyzzy-14"' input: '"xyzzy-14"'
output: ['{ "a": "xyzzy", "n": "14" }'] output: ['{ "a": "xyzzy", "n": "14" }']
- title: "\\[Requires 1.5\\] `scan(regex)`, `scan(regex; flags)`" - title: "`scan(regex)`, `scan(regex; flags)`"
body: | body: |
Emit a stream of the non-overlapping substrings of the input Emit a stream of the non-overlapping substrings of the input
that match the regex in accordance with the flags, if any that match the regex in accordance with the flags, if any
have been specified. If there is no match, the stream is empty. have been specified. If there is no match, the stream is empty.
To capture all the matches for each input string, use the idiom To capture all the matches for each input string, use the idiom
`[ expr ]`, e.g. `[ scan(regex) ]`. `[ expr ]`, e.g. `[ scan(regex) ]`.
example: example:
- program: 'scan("c")' - program: 'scan("c")'
skipping to change at line 2091 skipping to change at line 2336
- title: "`split(regex; flags)`" - title: "`split(regex; flags)`"
body: | body: |
For backwards compatibility, `split` splits on a string, not a regex. For backwards compatibility, `split` splits on a string, not a regex.
example: example:
- program: 'split(", *"; null)' - program: 'split(", *"; null)'
input: '"ab,cd, ef"' input: '"ab,cd, ef"'
output: ['"ab","cd","ef"'] output: ['"ab","cd","ef"']
- title: "\\[Requires 1.5\\] `splits(regex)`, `splits(regex; flags)`" - title: "`splits(regex)`, `splits(regex; flags)`"
body: | body: |
These provide the same results as their `split` counterparts, These provide the same results as their `split` counterparts,
but as a stream instead of an array. but as a stream instead of an array.
example: example:
- program: 'splits(", *")' - program: 'splits(", *")'
input: '("ab,cd", "ef, gh")' input: '("ab,cd", "ef, gh")'
output: ['"ab"', '"cd"', '"ef"', '"gh"'] output: ['"ab"', '"cd"', '"ef"', '"gh"']
- title: "\\[Requires 1.5\\] `sub(regex; tostring)` `sub(regex; string; fl ags)`" - title: "`sub(regex; tostring)` `sub(regex; string; flags)`"
body: | body: |
Emit the string obtained by replacing the first match of regex in the Emit the string obtained by replacing the first match of regex in the
input string with `tostring`, after interpolation. `tostring` should input string with `tostring`, after interpolation. `tostring` should
be a jq string, and may contain references to named captures. The be a jq string, and may contain references to named captures. The
named captures are, in effect, presented as a JSON object (as named captures are, in effect, presented as a JSON object (as
constructed by `capture`) to `tostring`, so a reference to a captured constructed by `capture`) to `tostring`, so a reference to a captured
variable named "x" would take the form: "\(.x)". variable named "x" would take the form: "\(.x)".
example: example:
- program: 'sub("^[^a-z]*(?<x>[a-z]*).*")' - program: 'sub("^[^a-z]*(?<x>[a-z]*).*")'
input: '"123abc456"' input: '"123abc456"'
output: '"ZabcZabc"' output: '"ZabcZabc"'
- title: "\\[Requires 1.5\\] `gsub(regex; string)`, `gsub(regex; string; f lags)`" - title: "`gsub(regex; string)`, `gsub(regex; string; flags)`"
body: | body: |
`gsub` is like `sub` but all the non-overlapping occurrences of the re gex are `gsub` is like `sub` but all the non-overlapping occurrences of the re gex are
replaced by the string, after interpolation. replaced by the string, after interpolation.
example: example:
- program: 'gsub("(?<x>.)[^a]*"; "+\(.x)-")' - program: 'gsub("(?<x>.)[^a]*"; "+\(.x)-")'
input: '"Abcabc"' input: '"Abcabc"'
output: '"+A-+a-"' output: '"+A-+a-"'
skipping to change at line 2159 skipping to change at line 2404
It may not be obvious at first, but jq is all about generators It may not be obvious at first, but jq is all about generators
(yes, as often found in other languages). Some utilities are (yes, as often found in other languages). Some utilities are
provided to help deal with generators. provided to help deal with generators.
Some minimal I/O support (besides reading JSON from standard Some minimal I/O support (besides reading JSON from standard
input, and writing JSON to standard output) is available. input, and writing JSON to standard output) is available.
Finally, there is a module/library system. Finally, there is a module/library system.
entries: entries:
- title: Variables - title: "Variable / Symbolic Binding Operator: `... as $identifier | ...` "
body: | body: |
In jq, all filters have an input and an output, so manual In jq, all filters have an input and an output, so manual
plumbing is not necessary to pass a value from one part of a program plumbing is not necessary to pass a value from one part of a program
to the next. Many expressions, for instance `a + b`, pass their input to the next. Many expressions, for instance `a + b`, pass their input
to two distinct subexpressions (here `a` and `b` are both passed the to two distinct subexpressions (here `a` and `b` are both passed the
same input), so variables aren't usually necessary in order to use a same input), so variables aren't usually necessary in order to use a
value twice. value twice.
For instance, calculating the average value of an array of numbers For instance, calculating the average value of an array of numbers
skipping to change at line 2258 skipping to change at line 2503
- program: '. as $i|[(.*2|. as $i| $i), $i]' - program: '. as $i|[(.*2|. as $i| $i), $i]'
input: '5' input: '5'
output: ['[10,5]'] output: ['[10,5]']
- program: '. as [$a, $b, {c: $c}] | $a + $b + $c' - program: '. as [$a, $b, {c: $c}] | $a + $b + $c'
input: '[2, 3, {"c": 4, "d": 5}]' input: '[2, 3, {"c": 4, "d": 5}]'
output: ['9'] output: ['9']
- program: '.[] as [$a, $b] | {a: $a, b: $b}' - program: '.[] as [$a, $b] | {a: $a, b: $b}'
input: '[[0], [0, 1], [2, 1, 0]]' input: '[[0], [0, 1], [2, 1, 0]]'
output: ['{"a":0,"b":null}', '{"a":0,"b":1}', '{"a":2,"b":1}'] output: ['{"a":0,"b":null}', '{"a":0,"b":1}', '{"a":2,"b":1}']
- title: 'Destructuring Alternative Operator: `?//`'
body: |
The destructuring alternative operator provides a concise mechanism
for destructuring an input that can take one of several forms.
Suppose we have an API that returns a list of resources and events
associated with them, and we want to get the user_id and timestamp of
the first event for each resource. The API (having been clumsily
converted from XML) will only wrap the events in an array if the resou
rce
has multiple events:
{"resources": [{"id": 1, "kind": "widget", "events": {"action": "c
reate", "user_id": 1, "ts": 13}},
{"id": 2, "kind": "widget", "events": [{"action": "
create", "user_id": 1, "ts": 14}, {"action": "destroy", "user_id": 1, "ts": 15}]
}]}
We can use the destructuring alternative operator to handle this struc
tural change simply:
.resources[] as {$id, $kind, events: {$user_id, $ts}} ?// {$id, $k
ind, events: [{$user_id, $ts}]} | {$user_id, $kind, $id, $ts}
Or, if we aren't sure if the input is an array of values or an object:
.[] as [$id, $kind, $user_id, $ts] ?// {$id, $kind, $user_id, $ts}
| ...
Each alternative need not define all of the same variables, but all na
med
variables will be available to the subsequent expression. Variables no
t
matched in the alternative that succeeded will be `null`:
.resources[] as {$id, $kind, events: {$user_id, $ts}} ?// {$id, $k
ind, events: [{$first_user_id, $first_ts}]} | {$user_id, $first_user_id, $kind,
$id, $ts, $first_ts}
Additionally, if the subsequent expression returns an error, the
alternative operator will attempt to try the next binding. Errors
that occur during the final alternative are passed through.
[[3]] | .[] as [$a] ?// [$b] | if $a != null then error("err: \($a
)") else {$a,$b} end
examples:
- program: '.[] as {$a, $b, c: {$d, $e}} ?// {$a, $b, c: [{$d, $e}]} |
{$a, $b, $d, $e}'
input: '[{"a": 1, "b": 2, "c": {"d": 3, "e": 4}}, {"a": 1, "b": 2, "
c": [{"d": 3, "e": 4}]}]'
output: ['{"a":1,"b":2,"d":3,"e":4}', '{"a":1,"b":2,"d":3,"e":4}']
- program: '.[] as {$a, $b, c: {$d}} ?// {$a, $b, c: [{$e}]} | {$a, $b
, $d, $e}'
input: '[{"a": 1, "b": 2, "c": {"d": 3, "e": 4}}, {"a": 1, "b": 2, "
c": [{"d": 3, "e": 4}]}]'
output: ['{"a":1,"b":2,"d":3,"e":null}', '{"a":1,"b":2,"d":null,"e":
4}']
- program: '.[] as [$a] ?// [$b] | if $a != null then error("err: \($a
)") else {$a,$b} end'
input: '[[3]]'
output: ['{"a":null,"b":3}']
- title: 'Defining Functions' - title: 'Defining Functions'
body: | body: |
You can give a filter a name using "def" syntax: You can give a filter a name using "def" syntax:
def increment: . + 1; def increment: . + 1;
From then on, `increment` is usable as a filter just like a From then on, `increment` is usable as a filter just like a
builtin function (in fact, this is how some of the builtins builtin function (in fact, this is how many of the builtins
are defined). A function may take arguments: are defined). A function may take arguments:
def map(f): [.[] | f]; def map(f): [.[] | f];
Arguments are passed as filters, not as values. The Arguments are passed as _filters_ (functions with no
same argument may be referenced multiple times with arguments), _not_ as values. The same argument may be
different inputs (here `f` is run for each element of the referenced multiple times with different inputs (here `f` is
input array). Arguments to a function work more like run for each element of the input array). Arguments to a
callbacks than like value arguments. This is important to function work more like callbacks than like value arguments.
understand. Consider: This is important to understand. Consider:
def foo(f): f|f; def foo(f): f|f;
5|foo(.*2) 5|foo(.*2)
The result will be 20 because `f` is `.*2`, and during the The result will be 20 because `f` is `.*2`, and during the
first invocation of `f` `.` will be 5, and the second time it first invocation of `f` `.` will be 5, and the second time it
will be 10 (5 * 2), so the result will be 20. Function will be 10 (5 * 2), so the result will be 20. Function
arguments are filters, and filters expect an input when arguments are filters, and filters expect an input when
invoked. invoked.
If you want the value-argument behaviour for defining simple If you want the value-argument behaviour for defining simple
functions, you can just use a variable: functions, you can just use a variable:
def addvalue(f): f as $f | map(. + $f); def addvalue(f): f as $f | map(. + $f);
Or use the short-hand: Or use the short-hand:
def addvalue($f): ...; def addvalue($f): ...;
With either definition, `addvalue(.foo)` will add the current With either definition, `addvalue(.foo)` will add the current
input's `.foo` field to each element of the array. input's `.foo` field to each element of the array. Do note
that calling `addvalue(.[])` will cause the `map(. + $f)` part
to be evaluated once per value in the value of `.` at the call
site.
Multiple definitions using the same function name are allowed. Multiple definitions using the same function name are allowed.
Each re-definition replaces the previous one for the same Each re-definition replaces the previous one for the same
number of function arguments, but only for references from number of function arguments, but only for references from
functions (or main program) subsequent to the re-definition. functions (or main program) subsequent to the re-definition.
See also the section below on scoping.
examples: examples:
- program: 'def addvalue(f): . + [f]; map(addvalue(.[0]))' - program: 'def addvalue(f): . + [f]; map(addvalue(.[0]))'
input: '[[1,2],[10,20]]' input: '[[1,2],[10,20]]'
output: ['[[1,2,1], [10,20,10]]'] output: ['[[1,2,1], [10,20,10]]']
- program: 'def addvalue(f): f as $x | map(. + $x); addvalue(.[0])' - program: 'def addvalue(f): f as $x | map(. + $x); addvalue(.[0])'
input: '[[1,2],[10,20]]' input: '[[1,2],[10,20]]'
output: ['[[1,2,1,2], [10,20,1,2]]'] output: ['[[1,2,1,2], [10,20,1,2]]']
- title: 'Scoping'
body: |
There are two types of symbols in jq: value bindings (a.k.a.,
"variables"), and functions. Both are scoped lexically,
with expressions being able to refer only to symbols that
have been defined "to the left" of them. The only exception
to this rule is that functions can refer to themselves so as
to be able to create recursive functions.
For example, in the following expression there is a binding
which is visible "to the right" of it, `... | .*3 as
$times_three | [. + $times_three] | ...`, but not "to the
left". Consider this expression now, `... | (.*3 as
$times_three | [.+ $times_three]) | ...`: here the binding
`$times_three` is _not_ visible past the closing parenthesis.
- title: Reduce - title: Reduce
body: | body: |
The `reduce` syntax in jq allows you to combine all of the The `reduce` syntax in jq allows you to combine all of the
results of an expression by accumulating them into a single results of an expression by accumulating them into a single
answer. As an example, we'll pass `[3,2,1]` to this expression: answer. As an example, we'll pass `[3,2,1]` to this expression:
reduce .[] as $item (0; . + $item) reduce .[] as $item (0; . + $item)
For each result that `.[]` produces, `. + $item` is run to For each result that `.[]` produces, `. + $item` is run to
skipping to change at line 2335 skipping to change at line 2647
0 | (3 as $item | . + $item) | 0 | (3 as $item | . + $item) |
(2 as $item | . + $item) | (2 as $item | . + $item) |
(1 as $item | . + $item) (1 as $item | . + $item)
examples: examples:
- program: 'reduce .[] as $item (0; . + $item)' - program: 'reduce .[] as $item (0; . + $item)'
input: '[10,2,5,3]' input: '[10,2,5,3]'
output: ['20'] output: ['20']
- title: "`isempty(exp)`"
body: |
Returns true if `exp` produces no outputs, false otherwise.
examples:
- program: 'isempty(empty)'
input: 'null'
output: ['true']
- title: "`limit(n; exp)`" - title: "`limit(n; exp)`"
body: | body: |
The `limit` function extracts up to `n` outputs from `exp`. The `limit` function extracts up to `n` outputs from `exp`.
examples: examples:
- program: '[limit(3;.[])]' - program: '[limit(3;.[])]'
input: '[0,1,2,3,4,5,6,7,8,9]' input: '[0,1,2,3,4,5,6,7,8,9]'
output: ['[0,1,2]'] output: ['[0,1,2]']
skipping to change at line 2486 skipping to change at line 2808
body: | body: |
jq currently only has IEEE754 double-precision (64-bit) floating jq currently only has IEEE754 double-precision (64-bit) floating
point number support. point number support.
Besides simple arithmetic operators such as `+`, jq also has most Besides simple arithmetic operators such as `+`, jq also has most
standard math functions from the C math library. C math functions standard math functions from the C math library. C math functions
that take a single input argument (e.g., `sin()`) are available as that take a single input argument (e.g., `sin()`) are available as
zero-argument jq functions. C math functions that take two input zero-argument jq functions. C math functions that take two input
arguments (e.g., `pow()`) are available as two-argument jq arguments (e.g., `pow()`) are available as two-argument jq
functions that ignore `.`. functions that ignore `.`. C math functions that take three input
arguments are available as three-argument jq functions that ignore
`.`.
Availability of standard math functions depends on the Availability of standard math functions depends on the
availability of the corresponding math functions in your operating availability of the corresponding math functions in your operating
system and C math library. Unavailable math functions will be system and C math library. Unavailable math functions will be
defined but will raise an error. defined but will raise an error.
One-input C math functions: `acos` `acosh` `asin` `asinh` `atan`
`atanh` `cbrt` `ceil` `cos` `cosh` `erf` `erfc` `exp` `exp10`
`exp2` `expm1` `fabs` `floor` `gamma` `j0` `j1` `lgamma` `log`
`log10` `log1p` `log2` `logb` `nearbyint` `pow10` `rint` `round`
`significand` `sin` `sinh` `sqrt` `tan` `tanh` `tgamma` `trunc`
`y0` `y1`.
Two-input C math functions: `atan2` `copysign` `drem` `fdim`
`fmax` `fmin` `fmod` `frexp` `hypot` `jn` `ldexp` `modf`
`nextafter` `nexttoward` `pow` `remainder` `scalb` `scalbln` `yn`.
Three-input C math functions: `fma`.
See your system's manual for more information on each of these.
- title: 'I/O' - title: 'I/O'
body: | body: |
At this time jq has minimal support for I/O, mostly in the At this time jq has minimal support for I/O, mostly in the
form of control over when inputs are read. Two builtins functions form of control over when inputs are read. Two builtins functions
are provided for this, `input` and `inputs`, that read from the are provided for this, `input` and `inputs`, that read from the
same sources (e.g., `stdin`, files named on the command-line) as same sources (e.g., `stdin`, files named on the command-line) as
jq itself. These two builtins, and jq's own reading actions, can jq itself. These two builtins, and jq's own reading actions, can
be interleaved with each other. be interleaved with each other.
One builtin provides minimal output capabilities, `debug`. Two builtins provide minimal output capabilities, `debug`, and
(Recall that a jq program's output values are always output as `stderr`. (Recall that a jq program's output values are always
JSON texts on `stdout`.) The `debug` builtin can have output as JSON texts on `stdout`.) The `debug` builtin can have
application-specific behavior, such as for executables that use application-specific behavior, such as for executables that use
the libjq C API but aren't the jq executable itself. the libjq C API but aren't the jq executable itself. The `stderr`
builtin outputs its input in raw mode to stder with no additional
decoration, not even a newline.
Most jq builtins are referentially transparent, and yield constant
and repeatable value streams when applied to constant inputs.
This is not true of I/O builtins.
entries: entries:
- title: "`input`" - title: "`input`"
body: | body: |
Outputs one new input. Outputs one new input.
- title: "`inputs`" - title: "`inputs`"
body: | body: |
skipping to change at line 2531 skipping to change at line 2876
inputs. inputs.
- title: "`debug`" - title: "`debug`"
body: | body: |
Causes a debug message based on the input value to be Causes a debug message based on the input value to be
produced. The jq executable wraps the input value with produced. The jq executable wraps the input value with
`["DEBUG:", <input-value>]` and prints that and a newline on `["DEBUG:", <input-value>]` and prints that and a newline on
stderr, compactly. This may change in the future. stderr, compactly. This may change in the future.
- title: "`stderr`"
body: |
Prints its input in raw and compact mode to stderr with no
additional decoration, not even a newline.
- title: "`input_filename`" - title: "`input_filename`"
body: | body: |
Returns the name of the file whose input is currently being Returns the name of the file whose input is currently being
filtered. Note that this will not work well unless jq is filtered. Note that this will not work well unless jq is
running in a UTF-8 locale. running in a UTF-8 locale.
- title: "`input_line_number`" - title: "`input_line_number`"
body: | body: |
skipping to change at line 2557 skipping to change at line 2908
fashion, allowing jq programs to start processing large JSON texts fashion, allowing jq programs to start processing large JSON texts
immediately rather than after the parse completes. If you have a immediately rather than after the parse completes. If you have a
single JSON text that is 1GB in size, streaming it will allow you single JSON text that is 1GB in size, streaming it will allow you
to process it much more quickly. to process it much more quickly.
However, streaming isn't easy to deal with as the jq program will However, streaming isn't easy to deal with as the jq program will
have `[<path>, <leaf-value>]` (and a few other forms) as inputs. have `[<path>, <leaf-value>]` (and a few other forms) as inputs.
Several builtins are provided to make handling streams easier. Several builtins are provided to make handling streams easier.
The examples below use the the streamed form of `[0,[1]]`, which The examples below use the streamed form of `[0,[1]]`, which is
is `[[0],0],[[1,0],1],[[1,0]],[[1]]`. `[[0],0],[[1,0],1],[[1,0]],[[1]]`.
Streaming forms include `[<path>, <leaf-value>]` (to indicate any Streaming forms include `[<path>, <leaf-value>]` (to indicate any
scalar value, empty array, or empty object), and `[<path>]` (to scalar value, empty array, or empty object), and `[<path>]` (to
indicate the end of an array or object). Future versions of jq indicate the end of an array or object). Future versions of jq
run with `--stream` and `-seq` may output additional forms such as run with `--stream` and `-seq` may output additional forms such as
`["error message"]` when an input text fails to parse. `["error message"]` when an input text fails to parse.
entries: entries:
- title: "`truncate_stream(stream_expression)`" - title: "`truncate_stream(stream_expression)`"
body: | body: |
skipping to change at line 2602 skipping to change at line 2953
The `tostream` builtin outputs the streamed form of its input. The `tostream` builtin outputs the streamed form of its input.
examples: examples:
- program: '. as $dot|fromstream($dot|tostream)|.==$dot' - program: '. as $dot|fromstream($dot|tostream)|.==$dot'
input: '[0,[1,{"a":1},{"b":2}]]' input: '[0,[1,{"a":1},{"b":2}]]'
output: ['true'] output: ['true']
- title: Assignment - title: Assignment
body: | body: |
Assignment works a little differently in jq than in most Assignment works a little differently in jq than in most
programming languages. jq doesn't distinguish between references programming languages. jq doesn't distinguish between references
to and copies of something - two objects or arrays are either to and copies of something - two objects or arrays are either
equal or not equal, without any further notion of being "the equal or not equal, without any further notion of being "the
same object" or "not the same object". same object" or "not the same object".
If an object has two fields which are arrays, `.foo` and `.bar`, If an object has two fields which are arrays, `.foo` and `.bar`,
and you append something to `.foo`, then `.bar` will not get and you append something to `.foo`, then `.bar` will not get
bigger. Even if you've just set `.bar = .foo`. If you're used to bigger, even if you've previously set `.bar = .foo`. If you're
programming in languages like Python, Java, Ruby, Javascript, used to programming in languages like Python, Java, Ruby,
etc. then you can think of it as though jq does a full deep copy Javascript, etc. then you can think of it as though jq does a full
of every object before it does the assignment (for performance, deep copy of every object before it does the assignment (for
it doesn't actually do that, but that's the general idea). performance it doesn't actually do that, but that's the general
idea).
This means that it's impossible to build circular values in jq
(such as an array whose first element is itself). This is quite
intentional, and ensures that anything a jq program can produce
can be represented in JSON.
All the assignment operators in jq have path expressions on the All the assignment operators in jq have path expressions on the
left-hand side. left-hand side (LHS). The right-hand side (RHS) provides values
to set to the paths named by the LHS path expressions.
entries: Values in jq are always immutable. Internally, assignment works
- title: "`=`" by using a reduction to compute new, replacement values for `.` that
body: | have had all the desired assignments applied to `.`, then
outputting the modified value. This might be made clear by this
The filter `.foo = 1` will take as input an object example: `{a:{b:{c:1}}} | (.a.b|=3), .`. This will output
and produce as output an object with the "foo" field set to `{"a":{"b":3}}` and `{"a":{"b":{"c":1}}}` because the last
1. There is no notion of "modifying" or "changing" something sub-expression, `.`, sees the original value, not the modified
in jq - all jq values are immutable. For instance, value.
.foo = .bar | .foo.baz = 1 Most users will want to use modification assignment operators,
such as `|=` or `+=`, rather than `=`.
will not have the side-effect of setting .bar.baz to be set
to 1, as the similar-looking program in Javascript, Python, Note that the LHS of assignment operators refers to a value in
Ruby or other languages would. Unlike these languages (but `.`. Thus `$var.foo = 1` won't work as expected (`$var.foo` is
like Haskell and some other functional languages), there is not a valid or useful path expression in `.`); use `$var | .foo =
no notion of two arrays or objects being "the same array" or 1` instead.
"the same object". They can be equal, or not equal, but if
we change one of them in no circumstances will the other
change behind our backs.
This means that it's impossible to build circular values in
jq (such as an array whose first element is itself). This is
quite intentional, and ensures that anything a jq program
can produce can be represented in JSON.
Note that the left-hand side of '=' refers to a value in `.`.
Thus `$var.foo = 1` won't work as expected (`$var.foo` is not
a valid or useful path expression in `.`); use `$var | .foo =
1` instead.
If the right-hand side of '=' produces multiple values, then
for each such value jq will set the paths on the left-hand
side to the value and then it will output the modified `.`.
For example, `(.a,.b)=range(2)` outputs `{"a":0,"b":0}`, then
`{"a":1,"b":1}`. The "update" assignment forms (see below) do
not do this.
Note too that `.a,.b=0` does not set `.a` and `.b`, but Note too that `.a,.b=0` does not set `.a` and `.b`, but
`(.a,.b)=0` sets both. `(.a,.b)=0` sets both.
- title: "`|=`" entries:
- title: "Update-assignment: `|=`"
body: | body: |
As well as the assignment operator '=', jq provides the "update" This is the "update" operator '|='. It takes a filter on the
operator '|=', which takes a filter on the right-hand side and right-hand side and works out the new value for the property
works out the new value for the property of `.` being assigned of `.` being assigned to by running the old value through this
to by running the old value through this expression. For expression. For instance, (.foo, .bar) |= .+1 will build an
instance, .foo |= .+1 will build an object with the "foo" object with the "foo" field set to the input's "foo" plus 1,
field set to the input's "foo" plus 1. and the "bar" field set to the input's "bar" plus 1.
This example should show the difference between '=' and '|=':
Provide input '{"a": {"b": 10}, "b": 20}' to the programs:
.a = .b
.a |= .b
The former will set the "a" field of the input to the "b" field of the
input, and produce the output {"a": 20}. The latter will set the "a"
field of the input to the "a" field's "b" field, producing {"a": 10}.
The left-hand side can be any general path expression; see `path()`. The left-hand side can be any general path expression; see `path()`.
Note that the left-hand side of '|=' refers to a value in `.`. Note that the left-hand side of '|=' refers to a value in `.`.
Thus `$var.foo |= . + 1` won't work as expected (`$var.foo` is Thus `$var.foo |= . + 1` won't work as expected (`$var.foo` is
not a valid or useful path expression in `.`); use `$var | not a valid or useful path expression in `.`); use `$var |
.foo |= . + 1` instead. .foo |= . + 1` instead.
If the right-hand side outputs multiple values, only the last If the right-hand side outputs no values (i.e., `empty`), then
one will be used. the left-hand side path will be deleted, as with `del(path)`.
examples: If the right-hand side outputs multiple values, only the first
one will be used (COMPATIBILITY NOTE: in jq 1.5 and earlier
releases, it used to be that only the last one was used).
examples:
- program: '(..|select(type=="boolean")) |= if . then 1 else 0 end' - program: '(..|select(type=="boolean")) |= if . then 1 else 0 end'
input: '[true,false,[5,true,[true,[false]],false]]' input: '[true,false,[5,true,[true,[false]],false]]'
output: ['[1,0,[5,1,[1,[0]],0]]'] output: ['[1,0,[5,1,[1,[0]],0]]']
- title: "`+=`, `-=`, `*=`, `/=`, `%=`, `//=`" - title: "Arithmetic update-assignment: `+=`, `-=`, `*=`, `/=`, `%=`, `//= `"
body: | body: |
jq has a few operators of the form `a op= b`, which are all jq has a few operators of the form `a op= b`, which are all
equivalent to `a |= . op b`. So, `+= 1` can be used to increment value equivalent to `a |= . op b`. So, `+= 1` can be used to
s. increment values, being the same as `|= . + 1`.
examples: examples:
- program: .foo += 1 - program: .foo += 1
input: '{"foo": 42}' input: '{"foo": 42}'
output: ['{"foo": 43}'] output: ['{"foo": 43}']
- title: "Plain assignment: `=`"
body: |
This is the plain assignment operator. Unlike the others, the
input to the right-hand-side (RHS) is the same as the input to
the left-hand-side (LHS) rather than the value at the LHS
path, and all values output by the RHS will be used (as shown
below).
If the RHS of '=' produces multiple values, then for each such
value jq will set the paths on the left-hand side to the value
and then it will output the modified `.`. For example,
`(.a,.b)=range(2)` outputs `{"a":0,"b":0}`, then
`{"a":1,"b":1}`. The "update" assignment forms (see above) do
not do this.
This example should show the difference between '=' and '|=':
Provide input '{"a": {"b": 10}, "b": 20}' to the programs:
.a = .b
.a |= .b
The former will set the "a" field of the input to the "b"
field of the input, and produce the output {"a": 20, "b": 20}.
The latter will set the "a" field of the input to the "a"
field's "b" field, producing {"a": 10, "b": 20}.
Another example of the difference between '=' and '|=':
null|(.a,.b)=range(3)
outputs '{"a":0,"b":0}', '{"a":1,"b":1}', and '{"a":2,"b":2}',
while
null|(.a,.b)|=range(3)
outputs just '{"a":0,"b":0}'.
- title: Complex assignments - title: Complex assignments
body: | body: |
Lots more things are allowed on the left-hand side of a jq assignment Lots more things are allowed on the left-hand side of a jq assignment
than in most languages. We've already seen simple field accesses on than in most languages. We've already seen simple field accesses on
the left hand side, and it's no surprise that array accesses work just the left hand side, and it's no surprise that array accesses work just
as well: as well:
.posts[0].title = "JQ Manual" .posts[0].title = "JQ Manual"
What may come as a surprise is that the expression on the left may What may come as a surprise is that the expression on the left may
skipping to change at line 2867 skipping to change at line 3238
- title: "`modulemeta`" - title: "`modulemeta`"
body: | body: |
Takes a module name as input and outputs the module's metadata Takes a module name as input and outputs the module's metadata
as an object, with the module's imports (including metadata) as an object, with the module's imports (including metadata)
as an array value for the "deps" key. as an array value for the "deps" key.
Programs can use this to query a module's metadata, which they Programs can use this to query a module's metadata, which they
could then use to, for example, search for, download, and could then use to, for example, search for, download, and
install missing dependencies. install missing dependencies.
- title: Colors
body: |
To configure alternative colors just set the `JQ_COLORS`
environment variable to colon-delimited list of partial terminal
escape sequences like `"1;31"`, in this order:
- color for `null`
- color for `false`
- color for `true`
- color for numbers
- color for strings
- color for arrays
- color for objects
The default color scheme is the same as setting
`"JQ_COLORS=1;30:0;39:0;39:0;39:0;32:1;39:1;39"`.
This is not a manual for VT100/ANSI escapes. However, each of
these color specifications should consist of two numbers separated
by a semi-colon, where the first number is one of these:
- 1 (bright)
- 2 (dim)
- 4 (underscore)
- 5 (blink)
- 7 (reverse)
- 8 (hidden)
and the second is one of these:
- 30 (black)
- 31 (red)
- 32 (green)
- 33 (yellow)
- 34 (blue)
- 35 (magenta)
- 36 (cyan)
- 37 (white)
 End of changes. 103 change blocks. 
223 lines changed or deleted 614 lines changed or added

Home  |  About  |  Features  |  All  |  Newest  |  Dox  |  Diffs  |  RSS Feeds  |  Screenshots  |  Comments  |  Imprint  |  Privacy  |  HTTP(S)