Wrap a download function to read as data.frame

Question

I have a function:

session$files$download(path = "/user/ExampleData/my_file.csv",filename ="my_file.csv", overwrite = TRUE)

which downloads files from a remote system using RESTful api. Instead of downloading the file onto my local machine or when using HPC I just want to send the contents into another function like a read_csv. The function says it can take additional arguments:

download( path = , filename = , ...)

Any ideas of a function I can add or wrap with? I've seen the stdout() function but I either don't understand it or its not what I need at all.

Heres the complete function from the developer file:

download = function(destSystemId, path, filename, overwrite=FALSE, ...){
  args <- list(...)
  queryParams <- list()
  headerParams <- character()

  if (missing(filename)) {
    filename <- basename(path)
  }

  if (missing(destSystemId)) {
    urlPath <- "/files/v2/media/{filePath}"
  }
  else {
    urlPath <- "/files/v2/media/system/{systemId}/{filePath}"
    urlPath <- gsub(paste0("\\{", "systemId", "\\}"), `destSystemId`, urlPath)
  }

  if (!missing(`path`)) {
    urlPath <- gsub(paste0("\\{", "filePath", "\\}"), `path`, urlPath)
  }

  resp <- private$apiClient$callApi(url = paste0(private$apiClient$basePath, urlPath),
                             method = "GET",
                             queryParams = queryParams,
                             headerParams = headerParams,
                             body = body,
                             httr::progress(),
                             httr::write_disk(filename, overwrite=overwrite),
                             ...)

  if (httr::status_code(resp) >= 200 && httr::status_code(resp) <= 299) {
    normalizePath(filename)
  } else if (httr::status_code(resp) >= 400 && httr::status_code(resp) <= 499) {
    logger.warn(jsonlite::toJSON(httr::content(resp), auto_unbox=TRUE, null="null", na="null"))
    httr::content(resp)
  } else if (httr::status_code(resp) >= 500 && httr::status_code(resp) <= 599) {
    logger.warn(jsonlite::toJSON(httr::content(resp), auto_unbox=TRUE, null="null", na="null"))
    httr::content(resp)
  }
}

If you want to stream data into other R function, check out the `?connections` help page. Connections are the mechanism that R uses to abstract data.reading. Maybe use `write_memory` rather than `write_disk`. It's still a bit unclear to me what you want to happen and without a proper [reproducible example](https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) it's difficult to test any possible solutions. — MrFlick, Feb 13 '18 at 14:54

score 1 · Answer 1 · answered Feb 13 '18 at 15:50

1

@MrFlick that put me on the right track:

myfile <- GET("https://raw.githubusercontent.com/dhbrand/rAgave/master/morphLabels.csv")

body <- content(myfile, as = "parsed", encoding = "UTF-8", type = "text/csv")

Returns the file as a data frame.

answered Feb 13 '18 at 15:50

dhbrand

162
3
16

Wrap a download function to read as data.frame

1 Answers1