4

I am trying to programatically download mp3 files from this rss feed. When I open an url such as: http://menlochurch.podbean.com/mf/feed/5gv2gb/170219_jortberg.mp3 it redirects to an url like: http://s62.podbean.com/pb/67f34563539acbe87b9566ecc5738d57/58aeff8e/data4/fs145/948579/uploads/170219_jortberg.mp3

If I curl the first url it downloads an empty file.

If I curl -L the first url it correctly downloads the file.

If I curl the second url it correctly downloads the file.

If I slurp the first url in Clojure it downloads something that looks like an mp3 (is big and has no legible text) but is unplayable and not the same as the output from curl -L.

If I slurp the second url in Clojure it also downloads something that looks like an mp3 (is big and has no legible text) but is unplayable and not the same as the output from curl -L and is identical to the output from the previous slurp.

➜ ~ git:(master) ✗ ls -l *.mp3 -rw-r--r-- 1 adam adam 38038533 Feb 23 08:32 curl-url1.mp3 -rw-r--r-- 1 adam adam 38038533 Feb 23 08:37 curl-url2.mp3 -rw-r--r-- 1 adam adam 0 Feb 23 08:45 curl-without-L.mp3 -rw-r--r-- 1 adam adam 67144297 Feb 23 08:31 slurp-url1.mp3 -rw-r--r-- 1 adam adam 67144297 Feb 23 08:31 slurp-url2.mp3

As you can see whatever I'm slurping, is almost twice as big as what I'm curling.

I don't know what to make of this, am I using slurp in an inappropriate context? Any advice would be appreciated!

Alex Miller
  • 69,183
  • 25
  • 122
  • 167
Adam Lee
  • 1,784
  • 1
  • 11
  • 15

2 Answers2

4

slurp is for strings so won't work with binary data.

user=> (doc slurp)
-------------------------
clojure.core/slurp
([f & opts])
  Opens a reader on f and reads all its contents, returning a string.
  See clojure.java.io/reader for a complete list of supported arguments.
Alex Miller
  • 69,183
  • 25
  • 122
  • 167
3

While slurping binary data is not possible, the following function will do something similar to slurp but accepts binary streams:

(defn copy-uri-to-file [uri file] (with-open [in (clojure.java.io/input-stream uri) out (clojure.java.io/output-stream file)] (clojure.java.io/copy in out)))

For this particular example the mp3 file can be downloaded like this:

(copy-uri-to-file "http://menlochurch.podbean.com/mf/feed/5gv2gb/170219_jortberg.mp3" "foo.mp3")

Adam Lee
  • 1,784
  • 1
  • 11
  • 15