no such file or directory (error in R)

Question

I want to import and bind in one data frame a set of .csv files. I create a list of files and then a for loop to read each of the files and bind them together. However, at the second file I receive an error saying that such a file or directory is missing. However it is indeed present in the list of files I create. Any recommendations on the issue, please? Thank you.

file.names <- list.files(path = pathname, pattern = " trial.csv")

for(i in seq(file.names)){
  file1 <- read.csv(file.names[i], header = TRUE, stringsAsFactors = FALSE)
  file1 <- t(file)
  file1 <- data.frame(file, stringsAsFactors = FALSE)
  colnames(file1) = file1[1,]
  print(file1)
}

Maybe you should set your working directory to `pathname`? Or use `full.names = T` in `list.files`? — Gregor Thomas, Nov 28 '17 at 14:57

score 2 · Accepted Answer · answered Nov 28 '17 at 15:00

By default, list.files() returns file names without the directory details. When importing files you typically want to set full.names = TRUE to return absolute paths (including directory names).

A good code pattern is as follows.

library(assertive.files)
files_to_import <- list.files(some_path, full.names = TRUE)
assert_all_are_existing_files(files_to_import)
data <- lapply(files_to_import, read.csv)

score 0 · Answer 2 · answered Nov 28 '17 at 16:02

I want to import and bind in one data frame a set of .csv files.

@richie-cotton provided an answer that fixed your path issue, but I want to address the highlighted part of your question.

Richie Cotton's code:

library(assertive.files)
files_to_import <- list.files(some_path, full.names = TRUE)
assert_all_are_existing_files(files_to_import)
data <- lapply(files_to_import, read.csv)

Will end in data, a list of data.frames. You want these to be one data.frame. Take a look at this answer to a similar question: https://stackoverflow.com/a/18489830/5333994

My suggestion would be to end with using rbindlist on the resulting list of data.frame.

library(data.table)
data <- rbindlist(data)

In fact, if you're going to use data.table you can replace read.csv with fread which will likely speed things up.

library(data.table)
library(assertive.files)
files_to_import <- list.files(some_path, full.names = TRUE)
assert_all_are_existing_files(files_to_import)
data <- rbindlist(lapply(files_to_import, fread))

If you want to end in a data.frame rather than data.table simply use setDF.

setDF(data)

If you want to avoid data.table all together, running do.call is probably your best option.

data <- do.call("rbind", data)

Note from the answer I linked above that this is likely to be a much slower operation than using rbindlist, depending on the number of files you have.

no such file or directory (error in R)

2 Answers2