4

When making a curl POST call with something like ... | curl -XPOST --data @- https://example.com/ can I also url-encode some url params on the command line (using curl itself, not bash funcs) without having to encode them myself and put them in the url argument to curl?

So I want the POST Body to not be urlencoded, but the url params to be url encoded within the same curl command.

So for example:

echo '{"username":"a","password": [1,2]}' | curl --header "Content-Type: application/json" --request POST -d@- --data-urlencode "filter=." http://localhost:8080

Doesn't seem to do what I want, and I have postfix my https://localhost:8080/ url with ?filter... which I would rather avoid for cases when the param needs a lot of encoding.

Kyle Brandt
  • 26,938
  • 37
  • 124
  • 165

1 Answers1

6

You can use --data-urlencode option:

... | curl -s -X POST --data @- --data-urlencode "filter=ab&123" <url>

As per man curl:

--data-urlencode <data>
(HTTP) This posts data, similar to the other -d, --data options with the exception that this performs URL-encoding.

anubhava
  • 761,203
  • 64
  • 569
  • 643
  • 1
    The thing is that I don't want to encode the data, only some url params. So I want both behaviors at once – Kyle Brandt Sep 14 '18 at 15:28
  • I think that is the inverse of what I am after, edited my question some. JSON will be coming over stdin, and then I don't want to have escape the url params myself. – Kyle Brandt Sep 14 '18 at 15:40
  • 1
    `--data-urlencode` takes care of encoding your data. Test with these 2 options: `--data-urlencode "filter=ab&123"` and then `--data "filter=ab&123"` and you will note difference. `ab&123` is correctly encoded as `filter=ab%26123` only when you use `-data-urlencode` but not when you use `--data` – anubhava Sep 14 '18 at 17:18
  • I get the difference between data and data-urlencode in that respect, but is there a anyway to encode url params but not encode post data? – Kyle Brandt Sep 14 '18 at 17:54
  • See my updated command. `--data @-` doesn't encode your POST body and `--data-urlencode "filter=ab&123"` encodes whatever additional data you want to encode. – anubhava Sep 14 '18 at 17:57
  • That doesn't seem to work for me.... I believe that matches what is in my question more or less. Have you confirmed the behavior or is that the way you expect to work based on the man pages? – Kyle Brandt Sep 14 '18 at 18:04
  • I never post any answer without verifying myself. It might be some misunderstanding between requirement and solution. Here is output of what I am seeing: https://pastebin.com/8q0GmPe4 Note difference of `--data` and `--data-urlencode` in output from URL where I am just dumping all the input using `var_dump( file_get_contents('php://input') );` – anubhava Sep 14 '18 at 18:30
  • 1
    "I never post any answer without verifying myself" I apologize - I should expect as much from such a high rep user - just trying to wrap my head around what I'm see and wanted to be sure - I should have worded that more carefully. I greatly appreciate this help! – Kyle Brandt Sep 14 '18 at 18:57
  • 1
    So going off your pastebin, I think what I would be hoping to see would be `'POST /post.php?filter=.user%23name HTTP/1.1` and then the JSON is the post body. – Kyle Brandt Sep 14 '18 at 19:03
  • 1
    Maybe I'm confused because of how --data-urlencode behaves with a GET? (with the `-G` option), something like `curl --data-urlencode 'filter=.' -G http://localhost:8080 -vvv` makes `GET /?filter=. HTTP/1.1` – Kyle Brandt Sep 14 '18 at 19:05
  • 1
    But using `-G` option will also try to convert your POST body as query string that you definitely don't want. – anubhava Sep 14 '18 at 19:30
  • 1
    So sounds like what I want would be a feature request to curl. – Kyle Brandt Sep 14 '18 at 19:56