Posting multipart form with inputstream rather than a file in Java

Question

I would like to post CSV formatted data to an external URL. The URL requires a multipart form containing a string and a file. It seems to me that this requires saving a file to disk before I can call the URL. Is it possible to use this URL (It is the US Census Bureau's geocoding service) without saving a file to disk? For example, can I use a ByteArrayInputStream that only lives temporarily in memory? Below is my initial code that involves saving a file to disk.

        String benchmark = "9";
        CloseableHttpClient httpClient = HttpClients.createDefault();
        HttpPost uploadFile = new HttpPost("https://geocoding.geo.census.gov/geocoder/locations/addressbatch");
        MultipartEntityBuilder builder2 = MultipartEntityBuilder.create();
        builder2.addTextBody("benchmark", benchmark, ContentType.TEXT_PLAIN);
        File f = new File("UscbGeocodingInput.csv");
        FileInputStream is = new FileInputStream(f);

        builder2.addBinaryBody(
                "addressFile",
                is,
                ContentType.APPLICATION_OCTET_STREAM,
                f.getName());

        HttpEntity multipart = builder2.build();
        uploadFile.setEntity(multipart);
        CloseableHttpResponse response = httpClient.execute(uploadFile);

I don't want to save a file to disk because it only needs to exist temporarily until we call the external URL. I tried to use a byte array input stream in instead of a FileInputStream, but the request then fails this way with a 400 error.

Refer in here https://stackoverflow.com/questions/18381928/how-to-convert-byte-array-to-multipartfile — Louis Nguyen, Nov 07 '19 at 07:12
By referring to that post, are you suggesting I use java.io.tmpdir to temporarily store the file? So there is no way to use this service without creating at least a temporary file on disk? — GNG, Nov 07 '19 at 08:08
I had this exact use case tonight, thanks for posting this!!! :) — John, Sep 10 '20 at 02:09

score 1 · Answer 1 · answered Nov 07 '19 at 23:03

This worked... Turns out the US Census Bureau's geocoding API looks for a filename ending in '.csv'. Even though I'm passing in a byte array rather than a file, you still need to give a name to that input stream.

        String benchmark = "9";
        CloseableHttpClient httpClient = HttpClients.createDefault();
        HttpPost uploadFile = new HttpPost("https://geocoding.geo.census.gov/geocoder/locations/addressbatch");
        MultipartEntityBuilder builder2 = MultipartEntityBuilder.create();
        builder2.addTextBody("benchmark", benchmark, ContentType.TEXT_PLAIN);
        byte[] addressFileAsBytes = baos.toByteArray();
//      AnalyzeInputStream(is);

        builder2.addBinaryBody(
                "addressFile",
                addressFileAsBytes,
                ContentType.APPLICATION_OCTET_STREAM,
                "anyStringThatEndsInDotCSV.csv");
        HttpEntity multipart = builder2.build();
        uploadFile.setEntity(multipart);
        CloseableHttpResponse response = httpClient.execute(uploadFile);
        HttpEntity responseEntity = response.getEntity();

score 0 · Answer 2 · answered Nov 07 '19 at 07:58

0

Can I use a ByteArrayInputStream?

Yes, but why would you do that when there is an overload that takes the byte[] directly?

addBinaryBody(String name, byte[] b, ContentType contentType, String filename)

answered Nov 07 '19 at 07:58

Andreas

154,647
11
152
247

Does this overloaded function bypass the need to create a File? – GNG Nov 07 '19 at 08:16
I have tried ````byte[] ba = baos.toByteArray(); builder2.addBinaryBody( "addressFile", ba, ContentType.APPLICATION_OCTET_STREAM, "someFileName"); ```` and ````byte[] ba = baos.toByteArray(); builder2.addBinaryBody( "addressFile", ba); ```` where ````ByteArrayOutputStream baos = new ByteArrayOutputStream();```` and has been written to. Both resulted in bad request (400) responses – GNG Nov 07 '19 at 08:27
@GNG Yes, the overloaded *method* bypasses the need to create a File. --- If using `byte[]` fails while using `FileInputStream` works, then the `byte[]` does not have the same content as the file. Did you debug your code and check the content of the `byte[]`? Start by comparing the length of the byte array to the size of the file. – Andreas Nov 07 '19 at 08:53
They are both 38 bytes. One odd trait is that if I copy the bytes into https://onlineutf8tools.com/convert-bytes-to-utf8, the result is missing the first character. If I enter a space at the beginning of the byte array, the first character appears. But this might be a bug in the online tool rather than in the byte array. – GNG Nov 07 '19 at 17:06
````[49, 44, 97, 102, 100, 115, 103, 97, 110, 118, 105, 100, 108, 110, 97, 115, 99, 108, 44, 34, 83, 97, 110, 32, 68, 105, 101, 103, 111, 34, 44, 44, 57, 50, 49, 48, 57, 10] ```` vs ````[ 49, 44, 97, 102, 100, 115, 103, 97, 110, 118, 105, 100, 108, 110, 97, 115, 99, 108, 44, 34, 83, 97, 110, 32, 68, 105, 101, 103, 111, 34, 44, 44, 57, 50, 49, 48, 57, 10]```` – GNG Nov 07 '19 at 17:06

Posting multipart form with inputstream rather than a file in Java

2 Answers2