Plotting sublist parameters efficiently using a loop in R

Question

I'm having trouble to make a loop and I have no idea what is going wrong. My data contains list within a list. I have >50 primary list i.e. [[i]]. Each of the [[i]] contains 20 `i`` (=sublist). A subset of my data looks like

>data
[[1]]$`1`
           X       Y  Height_m kt_Result   
1   253641.0 2630050        90       560  
74  253845.7 2630552        90       270
156 254353.6 2630195       130         0
171 254554.9 2630220       170       390
173 254565.9 2630323       120       304

[[1]]$`2`
           X       Y  Height_m kt_Result   
5   253641.0 2630050        50       860  
77  253845.7 2630552        20       370
159 254353.6 2630195       190        20
177 254554.9 2630220       140       310
200 254565.9 2630323       100       804

... ...

[[2]]$`1`
           X       Y  Height_m kt_Result   
4   253641.0 2630050        10       960  
78  253845.7 2630552        20       220
150 254353.6 2630195       330         5
377 254554.9 2630220       670       340
100 254565.9 2630323       220       314

... ...

When I want to plot each of the sublist with different colors in one plot, it doesn't work

#blank plot (dfs is a different data frame with the same data)
plot(dfs[[1]]$kt_Result, dfs[[1]]$Height_m, type='n')

#plot sublist
lapply(1:length(data[[1]]), function(i) 
  points(data[[1]]$`i`$kt_Result, data[[1]]$`i`$Height_m,
         ylim=rev(c(0, max(data[[1]]$`i`$Height_m))),
         xlim= c(min(data[[1]]$`i`$kt_Result, na.rm=TRUE), 
max(data[[1]]$`i`$kt_Result, na.rm=TRUE)), lwd=2, type='b',col=i))

I get warnings without any plot

There were 50 or more warnings (use warnings() to see the first 50)
> warnings()
Warning messages:
1: In max(data[[1]]$i$Height_m) : no non-missing arguments to max; returning -Inf
2: In min(data[[1]]$i$kt_Result, na.rm = TRUE) :
  no non-missing arguments to min; returning Inf
3: In max(data[[1]]$i$kt_Result, na.rm = TRUE) :
  no non-missing arguments to max; returning -Inf
... ...

When I plot without any x-/y- limits, it gives no warning or plot, just shows NULL in workspace!

lapply(1:length(data[[1]]), function(i) 
  points(data[[1]]$`i`$kt_Result, data[[1]]$`i`$Height_m,
         lwd=2, type='p',col=i))

[[1]]
NULL

[[2]]
NULL

...

[[20]]
NULL

However, when I plot the data one by one, it works, but this is an impractical way to deal with such a large dataset

plot(dfs[[1]]$kt_Result, dfs[[1]]$Height_m, type='n')

points(data[[1]]$`1`$kt_Result, data[[1]]$`1`$Height_m, col='red')
points(data[[1]]$`2`$kt_Result, data[[1]]$`2`$Height_m, col='green')
... ...
points(data[[1]]$`19`$kt_Result, data[[1]]$`19`$Height_m, col='cyan')
points(data[[1]]$`20`$kt_Result, data[[1]]$`20`$Height_m, col='blue')

Any idea why this simple loop is not working?

This is probably one of the few place were a `for` loop makes sense in R, I would try that. — Paul Hiemstra, Dec 31 '13 at 06:55
You are expecting something to work that will not, by design work. `$` *never* evaluates it arguments. [**See this answer**](http://stackoverflow.com/a/18228613/1478381) for more information and use `[,i]` rather than `$`. — Simon O'Hanlon, Dec 31 '13 at 08:31
`reshape2::melt(data)` will turn your data into a long format data.frame ideally suited for ggplot2 or lattice. — baptiste, Dec 31 '13 at 14:54
@baptiste, great idea, didn't realize `melt` works with nested lists. You do need to do this though: `melt(data, id.vars=c("height", "weight"))`. I'll update my code below. — BrodieG, Dec 31 '13 at 16:34
Also @ToNoY, there is stats::reshape, but it doesn't work with lists. If you really want base R the original code I had was base R only (other than ggplot, of course). — BrodieG, Dec 31 '13 at 16:46

BrodieG · Accepted Answer · 2013-12-31T17:03:29.853

@SimonO10 highlights the problem in your code, but beyond that, have you considered ggplot? It's really designed to do this type of stuff easily. The main problem you'll run into is you need to convert your data to long format. But once you've done that the plotting is almost trivial:

# Use ggplot to plot

ggplot(
  subset(data, L1==1),  # just use `data` here if you want all plots
  aes(x=height, y=weight, color=paste(L1, L2, sep="-"))
) + geom_point() + scale_color_discrete(name="id-sub.id") + scale_y_reverse()

grouped scatter plot

with facet_wrap if you want stuff to show up in different graphs (note the data changed b/c I didn't set the random seed):

ggplot(
  subset(data, L1==1),  # just use `data` here if you want all plots
  aes(x=height, y=weight, color=paste(L1, L2, sep="-"))
) + geom_point() + scale_color_discrete(name="id-sub.id") +
facet_wrap( ~ L2)

facetted by sub.id

In order to get your data into long format, I started with some data in your structure (see end of post for how I did it):

data
# [[1]]
# [[1]]$`1`
#        height     weight
# 1  0.79199970 0.19434040
# 2  0.83137244 0.41325506
# ...
# 
# [[1]]$`2`
#         height     weight
# 1  0.099096870 0.64563244
# 2  0.736456033 0.06103266
# ...
# 
# [[2]]
# [[2]]$`1`
#       height      weight
# 1  0.2622071 0.176313366
# 2  0.5747873 0.887846513
# ...

then converted to long format as @baptiste suggested (note numbers don't match exactly b/c I regenerated random data)

# Convert to long format

library(reshape2)
data <- melt(data, id.vars=c("height", "weight"))
#         height     weight L2 L1
# 1   0.55637070 0.50990818  1  1
# 2   0.59839293 0.91242349  1  1
# ...
# 11  0.39170638 0.86185414  2  1
# 12  0.69356092 0.03145715  2  1
# ...
# 21  0.67580737 0.55668117  1  2
# 22  0.01335459 0.29615540  1  2
# ...

and the code to generate data:

data <- 
  replicate(5, simplify=F,
    replicate(2, simplify=F,
      data.frame(height=runif(10), weight=runif(10))
  ) )
data <- lapply( data, function(x) { names(x) <- seq_along(x); x } )

Thanks, but you plotted everything (1-1, 1-2, 2-1, 2-2, etc.). How would you plot only 1-1 & 1-2? sorry I'm not familiar with ggplot. — ToNoY, Dec 31 '13 at 15:55
just subset data. Instead of passing `data` to ggplot, pass `subset(data, id==1)`. — BrodieG, Dec 31 '13 at 16:17
thanks for the update. well ... may be I'm asking too much, but how would you make a loop for each subset of id=L1?? As I told there are more than 50 ids i.e. the loop will create >50 plots ... — ToNoY, Dec 31 '13 at 16:54
If you want them to all show up at the same time, you can use `facet_wrap()`, but that may be too much if you want 50 graphs. Otherwise you can just put the `ggplot` statement in a for loop and pass the loop index value to the `subset` expression. I'll add a facet_wrap example. — BrodieG, Dec 31 '13 at 17:00

Plotting sublist parameters efficiently using a loop in R

1 Answers1