0

For some reason a third party API is accepting my request.post request but not my https.request one. The returned error from the latter is that the XML is invalid or malformed. I would prefer to use https and not request as it adds unnecessary overhead.

How can i debug the 2 functions so that I can match my https.request output to the request.post one?

request.post

request.post({url: 'https://api.example.com/post.php', form: { xml: full_xml_data }}, function(err, resp, xml){
  // request is successful
});

https.request

let options = {
  "hostname": api.example.com,
  "port": 443,
  "path": '/post.php',
  "method": "POST",
  "headers": {
    "Content-Type": "application/x-www-form-urlencoded",
    "Content-Length": full_xml_data.length
  }
};
let req = https.request(options, (res) => {
  let xml="";
  res.on("data", function(data){ xml+=data; });
  res.on("end", function(){
    // request is not successful, respons from API is that xml is invalid/malformed
  });
});
req.on("error", (err) => {
  // error handler
});

let post_data = querystring.stringify({"xml":full_xml_data});
// i have also tried: let post_data = 'xml='+encodeURIComponent(full_xml_data);
req.write(post_data);
req.end();

Any ideas on how to resolve this would be much appreciated! Thomas

Thomas Smart
  • 135
  • 1
  • 9
  • Most likely guess is something in here: `let post_data = querystring.stringify({"xml":full_xml_data});` that's different. You could compare the two requests to see exactly what is being sent in both cases. You can either use a network sniffer to see the results of both requests or you can make your own dummy Express web server and post both to that and have it output exactly what is being received. This just requires some debugging to see what's different. – jfriend00 Jul 17 '18 at 02:28
  • Are you sure that `querystring.stringify()` makes `application/x-www-form-urlencoded`? Might want to check here: [convert Javascript object data to x-www-form-urlencoded](https://stackoverflow.com/a/39787203/816620). – jfriend00 Jul 17 '18 at 02:39
  • @jfriend00 that link seems to indicate that querystring does that, the other part of the link reflects the "i also tried" attempt noted in my post. a network sniffer or similar will be my "plan b" if no-one has a direct answer here. – Thomas Smart Jul 17 '18 at 02:54
  • @jfriend00 :facepalm: that was it..... many thanks. please add as answer so i can select it. – Thomas Smart Jul 17 '18 at 03:22
  • Per your request, I made an answer below... – jfriend00 Jul 17 '18 at 03:30

1 Answers1

1

One thing I notice is that in the second request you are setting the Content-Length to the raw data length, not to the encoded data length. It needs to be the length of the data you are actually sending. The documentation for the http module suggest that you should use: Buffer.byteLength(encoded_data) for getting the length like this: 

let post_data = querystring.stringify({"xml":full_xml_data});

let options = {
  "hostname": api.example.com,
  "port": 443,
  "path": '/post.php',
  "method": "POST",
  "headers": {
    "Content-Type": "application/x-www-form-urlencoded",
    "Content-Length": Buffer.byteLength(encoded_data)    // put length of encoded data here
  }
};
let req = https.request(options, (res) => {
  let xml="";
  res.on("data", function(data){ xml+=data; });
  res.on("end", function(){
    // request is not successful, respons from API is that xml is invalid/malformed
  });
});
req.on("error", (err) => {
  // error handler
});

req.write(post_data);
req.end();
jfriend00
  • 683,504
  • 96
  • 985
  • 979