Shell - echo some command (curl) vs type the command in the terminal

Question

Why am I not getting the same result when I do :
1. Go to my terminal and enter

curl -I www.httpbin.org

Result :

HTTP/1.1 200 OK
Connection: keep-alive
Server: meinheld/0.6.1
Date: Wed, 20 Dec 2017 19:20:56 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 13011
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
X-Powered-By: Flask
X-Processed-Time: 0.00580310821533
Via: 1.1 vegur

2. Create a file APIConnection.sh
Containing :

#! /bin/bash

api_url_1="www.httpbin.org"

echo `curl -I $api_url_1`

Then go to my terminal and execute the file : ./APIConnection.sh Result :

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                             Dload  Upload   Total   Spent    Left  Speed
  0 13011    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  Via: 1.1 vegurme: 0.0147929191589 true

Remove the `echo` from the script and you will get the same result — Zlemini, Dec 20 '17 at 20:25
@Zlemini, ...not *quite* the same -- whether stdout is to the TTY matters. — Charles Duffy, Dec 20 '17 at 20:51
@NicolasN, ...there are arguably two separate questions in here: One, why `echo $(...)` puts all the output from `...` on one line -- this one is absolutely a FAQ; covered in [BashPitfalls #14](http://mywiki.wooledge.org/BashPitfalls#echo_.24foo), and also by running your code through http://shellcheck.net/. Two, why the percentage/status bar is only present in one case and not the other. That's a `curl`-specific question (though several other programs also make distinctions in their default output based on `isatty()` results, and thus will have their own variances in similar conditions). — Charles Duffy, Dec 20 '17 at 20:52

score 3 · Accepted Answer · edited Jun 20 '20 at 09:12

The reason for which a progress bar is only displayed when running curl between backticks is explained in man curl:

PROGRESS METER

   curl normally displays a progress meter during operations,
   indicating the amount of transferred data, transfer speeds
   and estimated time left, etc. The progress meter displays
   number of bytes and the speeds are in bytes per second. The
   suffixes (k, M, G, T, P) are 1024 based. For example 1k is
   1024 bytes. 1M is 1048576 bytes.

   curl displays this data to the terminal by default, so if you
   invoke curl to do an operation and it is about to write data
   to the terminal, it disables the progress meter as otherwise
   it would mess up the output mixing progress meter and
   response data.

If you want to turn off the progress meter explicitly, this can be done with --silent.

`` is legacy syntax for command substitution in bash and other POSIX-compliant shells. This means that output is captured by the shell, not written directly to the terminal. Because curl cannot detect that it would later be written to the terminal by echo, it does not disable the progress meter as described above.

If you really do want to use a command substitution here (instead of just having your script run curl -I "$api_url_1" without any command substitution or echo), put quotes around your command substitution to avoid string-splitting and glob expansion.

It's also advisable to use the modern command substitution syntax, $(), so each substitution has its own quoting context:

echo "$(curl --silent -I "$api_url_1")"

score 0 · Answer 2 · answered Dec 20 '17 at 20:05

0

executing curl -I www.httpbin.org gives you the result of the standard output while executing echo `curl -I $api_url_1` gives you the result of the standard error. To see the differences execute the following command and then look at the content of created se.txt and so.txt files:

 curl -I www.httpbin.org 1>so.txt 2>se.txt

look at about stdin, stdout and stderr for more information.

answered Dec 20 '17 at 20:05

ayyoob imani

639
7
16

1

It's untrue to say that the version with `echo` and a command substitution emits only stderr -- the stdout is written into the FIFO, string-split and glob-expanded, and then passed to `echo` as arguments. Some of it may be overwritten by the progress bar on `stderr` moving the cursor back to the end of the line and thus not rendered, but *some* content from stdout is absolutely present (and visible in the OP's stated output). – Charles Duffy Dec 20 '17 at 20:55

Shell - echo some command (curl) vs type the command in the terminal

2 Answers2