0

I'm using the following query and trying to write the results to a .csv file: 

.libPaths("G:/R/R-3.6.1/library")
library(mongolite)
library(data.table)
library(dplyr)


###Path to list of search terms

#------------------------------------------------------------------------------------------------------------#
#Input the words you want to search in a text file, and call the text file below in place of ".txt"
#-------------------------------------------------------------------------------------------------------------#

searchStrings = readLines("//int/elm/Work/Text Analytics/Opportunities/New Folder/terms.txt")

searchCounts = data.table(searchString = searchStrings, count = 0)


db <- mongo(collection = "2020Emails", db = "textAnalytics", verbose = TRUE,
            url = "mongodb://user:m0ng0b0ng0@interactionsprojection:7999/textAnalytics")

firstQuery = TRUE

#Update this JSON if if you want different fields returned
fieldsjson = '{"DocEid" : true,
"RequestType" : true,
"FromEmailAddress" : true,
"ReceiptTime" : true,
"RawBodyText" : true,
"CMF" : true,
"ReplyTo" : true,
"InboundMode" : true,
"PartNumbers" : true,
"_id": false}'

for (term in searchStrings) {

  queryString = paste0('{"$text" : { "$search" : "\\"',term,'\\"" }}')

  if (firstQuery == TRUE) {
    results = NULL
    results = data.table(db$find(query = queryString, fields = fieldsjson))
    if (nrow(results) > 0) {
      results[, searchString := term]
      searchCounts[searchString == term, count := nrow(results)]
      firstQuery = FALSE
    }
  } else {
    tempdt = data.table(db$find(query = queryString, fields = fieldsjson))
    if (nrow(tempdt) > 0) {
      tempdt[, searchString := term]
      results = rbind(results,tempdt, fill= TRUE)
      searchCounts[searchString == term, count := nrow(tempdt)]
    }
  } 

}

View(results)

#------------------------------------------------------------------------------#
# Specify the location and new file name where the results will be stored below
#------------------------------------------------------------------------------#
fwrite(results, "C:/Results/terms.csv")

I am getting the following error when trying to write this to a .csv. file.

Error in fwrite(results, "C:/Results/terms.csv") : Row 1 of list column is type 'list' - not yet implemented. fwrite() can write list columns containing items which are atomic vectors of type logical, integer, integer64, double, complex and character.

Michael
  • 19
  • 3
  • Remove the column which is of type "list" in results. and then try. – nikn8 Apr 20 '20 at 15:47
  • 1
    Please run `dput(results)` and include the output in your question, so we have something to copy-paste in order to reproduce the issue. If it's large, call `head` first. It's also fine to replace it with fake data -- just be sure it still produces the same error. More tips here: https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example – Tim Goodman Apr 20 '20 at 15:48
  • 1
    @Neel `fwrite` can handle list columns: `fwrite(data.table(1:2, list(1:3, 4:5)))`. @Michael the problem is nested lists. You can confirm the error message is right by looking at `sapply(results, typeof)`, and look for the `list` column. Say it's column 4. Then look at `table(sapply(results[[4]], typeof))` and you'll see that some rows are nested -- list of lists -- `fwrite` can't handle this. You can try to apply something like `toString` to such columns to convert them to character first before writing. OTOH, nested data structures are not perfect for CSV format -- you might consider JSON – MichaelChirico Apr 20 '20 at 15:52
  • @MichaelChirico Thanks for sharing the info. I just judged from the error and shared the info. I guess the error message can be made more smarter. – nikn8 Apr 20 '20 at 15:58

0 Answers0