Write output to a file after piped to jq

supersize picture supersize · Feb 24, 2018 · Viewed 44.5k times · Source

How can I write a piped output that went through jq to a file in shell

Example:

curl api.example.com | jq > call.txt

won't work. Neither does

(curl api.example.com | jq) > call.txt

Help!

Edit: So doing curl api.example.com > call.txt works just fine. So it has to do with piping it to jq

Answer

Attie picture Attie · Feb 24, 2018

Just calling jq without a filter will throw errors if stdout isn't a terminal

$ curl https://jsonplaceholder.typicode.com/posts/1 | jq > test.txt
jq - commandline JSON processor [version 1.5-1-a5b5cbe]
Usage: jq [options] <jq filter> [file...]

        jq is a tool for processing JSON inputs, applying the
        given filter to its JSON text inputs and producing the
[...]

Try jq '.' (i.e: pretty-print the input JSON):

$ curl https://jsonplaceholder.typicode.com/posts/1 | jq '.' > test.txt
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   292  100   292    0     0   1698      0 --:--:-- --:--:-- --:--:--  1707

Note that the filter is not really optional:

From man jq:

JQ(1)                                                                                JQ(1)

NAME
       jq - Command-line JSON processor

SYNOPSIS
       jq [options...] filter [files...]

According to the tip of the master branch... your described (and my observed) behaviour is not expected...

Older versions of jq have the following: (here)

if (!program && isatty(STDOUT_FILENO) && !isatty(STDIN_FILENO))
  program = ".";

i.e: use a default filter if stdout is a TTY, and stdin is not a TTY.

This behaviour appears to be corrected in commit 5fe05367, with the following snippet of code:

if (!program && (!isatty(STDOUT_FILENO) || !isatty(STDIN_FILENO)))
  program = ".";