0

The command find . -name 'reddit.*.flattened.json' | jq '.' gives the error parse error: Invalid numeric literal at line 2, column 0.

Motivated by this question, I tried jq --arg.

But, find . -name 'reddit.*.flattened.json' | jq --arg filename $1 '. $filename' gives the same error.

Output of find query:

./reddit.opiates.flattened.json
./reddit.OurOverUsedVeins.flattened.json
./reddit.microdosing.flattened.json
./reddit.naloxone.flattened.json
./reddit.suboxone.flattened.json
./reddit.heroin.flattened.json
./reddit.OpiatesRecovery.flattened.json
./reddit.fentanyl.flattened.json

I can load each file with jq by typing jq '.' FILENAME so there is no error in the files.

mac389
  • 3,004
  • 5
  • 38
  • 62
  • 1
    Are you looking for `xargs`? `… | xargs jq '.'` – pmf Mar 16 '23 at 14:35
  • 1
    `find ... -exec jq . {} +`? – Fravadona Mar 16 '23 at 14:35
  • 1
    If all those files are in a single dir (no depth needed), why not use a the 'search' directly `jq 'filter' reddit.*.flattened.json` – 0stone0 Mar 16 '23 at 14:40
  • Originally I used xargs and it gave them error. But then I realized I typed `find ... | xargs jq` rather than `find ... | xargs jq '.'`. The latter command works. – mac389 Mar 16 '23 at 14:40
  • @0stone0 nice alternate solution! – mac389 Mar 16 '23 at 14:40
  • If you would see this as a valid answer @mac389, please let me know so I can post it. – 0stone0 Mar 16 '23 at 15:29
  • 2
    Note that your solution above is a bit buggy. `find ... -exec jq . {} +` or `find ... -print0 | xargs -0 jq .` is better. Note the `-0` on the xargs side and `-print0` on the find side, it's important to handle unusual filenames, which xargs fails at by default. – Charles Duffy Mar 16 '23 at 15:32

2 Answers2

4

Let find do it, instead of having the shell stick its nose in the middle.

find . -name 'reddit.*.flattened.json' -exec jq '.' {} \;

This avoids issues with names that contain any whitespace (and especially names that contain newline characters) or glob metacharacters.

(For most filters and combinations of arguments, you can safely reduce the number of times jq is called by letting it read from multiple files per invocation with

find ... -exec jq '.' {} +

)

chepner
  • 497,756
  • 71
  • 530
  • 681
0

Since you say you

can load each file with jq by typing jq '.' FILENAME

you could use the anti-pattern:

jq . $(find ……)

However you would have to be a bit more ingenious to handle more tricky filenames. See e.g. solutions proposed in comments by @CharlesDuffy, How do I store the output from a find command in an array? + bash, etc etc.

peak
  • 105,803
  • 17
  • 152
  • 177
  • boo, hiss re: showcasing a major antipattern (see [BashPitfalls #1](https://mywiki.wooledge.org/BashPitfalls#for_f_in_.24.28ls_.2A.mp3.29)) – Charles Duffy Mar 16 '23 at 15:33
  • (really do feel entirely free to fold any suggestions I make in the comments or elsewhere into the answer itself) – Charles Duffy Mar 16 '23 at 15:43
  • Good to know. Likewise, feel free to improve my responses. In this case I was mainly trying to point out the muddle. – peak Mar 16 '23 at 15:49