6

I have a data frame appt which is 91.2MB in size, comprising 29255 observation of 51 variables.

When I tried to check its end with tail(appt), I get the error

Error: C stack usage 20212630 is too close to the limit

I have no clue what to do to trouble shoot this. Any suggestions on what I can do?

As additional info, I have in memory at the same time a few other variables of almost comparable size, including a 90.2MB character vector and a 42.3MB data frame of 77405 obs. x 60 variables. Calling tail on these two other variables did not trigger any error.

Edit: I have narrowed down that the error occurs only when the last row is accessed. i.e. appt[29254, ] is fine, appt[29255, ] throws the error.

Ricky
  • 4,616
  • 6
  • 42
  • 72
  • [try this.][1] I don"t know much but i [1]: http://stackoverflow.com/questions/14719349/error-c-stack-usage-is-too-close-to-the-limit – Hemant Rupani Jan 19 '15 at 07:41
  • Thanks, but unfortunately for the two answers in that question, one refers to recursion (which I have none of here), and the other is for Unix (I'm on Win 7 64-bit). – Ricky Jan 19 '15 at 07:42
  • It's an unusual error for R; does this occur in a session with only base packages loaded (e.g., `save()` the object in one session, the `load()` in another)? Is there a way to easily simulate the data and still reproduce the error? And what is the output of `sessionInfo()`? – Martin Morgan Jan 19 '15 at 07:59
  • We can't do much without a reproducible example. What is the content of that last line? – Roland Jan 19 '15 at 08:48
  • I know, and I too wonder how to reproduce that last line, since I got the error trying to retrieve it. – Ricky Jan 19 '15 at 08:59
  • I know this is a bit late but I get this same error when loading a cnv file as a dataframe. The file is a matrix of observations 57,000 long by 160. If I make the df smaller by reducing the number of observations then it loads with no problem. It seems to literally be a size issue. Having said that I still need a solution - did you ever manage to find a solution Ricky? – user3062260 Feb 24 '15 at 21:19
  • Unfortunately no. I resorted to reading a smaller chunk of data in the meantime. However, my project is still not completed and I likely will have to go back to handling the big data, so I'm still hoping there will be a smarter answer. – Ricky Feb 25 '15 at 06:57
  • I have had the same problem, with a large tsv file. I used read.table instead of read.csv, and I got some cells with unsplitted content (rows). For example an cell nchar of which was 11747135 (almost the content of the full file). In my experience read.table has problem with tsv files containing commas. With using read.csv it worked without problem. I can send reproducible example if it is interesting. – charisz Jan 27 '16 at 07:31

1 Answers1

0

I had exactly the same error but managed to solve it by disabling quoting when reading in the dataframe. For me the error showed up when trying tail(df) as well.

Turns out one of the lines in my text file had the \ character. You could run into the same problem if your file has a " or ' somewhere, as read.table considers this a quote character in the default case.

Add the option quote="" to disable quoting alltogether. Example: read.table(filepath, quote="")

If this doesn't solve your issue, check out some of the other options in ?read.table (such as allowEscapes, sep, ...) as they might be causing your error.

GR4
  • 203
  • 1
  • 7