6

I'm using a logistic exposure to calculate hatching success for bird nests. My data set is quite extensive and I have ~2,000 nests, each with a unique ID ("ClutchID). I need to calculate the number of days a given nest was exposed ("Exposure"), or more simply, the difference between the 1st and last day. I used the following code:

HS_Hatch$Exposure=NA    
for(i in 2:nrow(HS_Hatch)){HS_Hatch$Exposure[i]=HS_Hatch$DateVisit[i]- HS_Hatch$DateVisit[i-1]}

where HS_Hatch is my dataset and DateVisit is the actual date. The only problem is R is calculating an exposure value for the 1st date (which doesn't make sense).

What I really need is to calculate the difference between the 1st and last date for a given clutch. I've also looked into the following:

Exposure=ddply(HS_Hatch, "ClutchID", summarize, 
                     orderfrequency = as.numeric(diff.Date(DateVisit)))


df %>%
  mutate(Exposure =  as.Date(HS_Hatch$DateVisit, "%Y-%m-%d")) %>%
  group_by(ClutchID) %>%
  arrange(Exposure) %>%
  mutate(lag=lag(DateVisit), difference=DateVisit-lag)

I'm still learning R so any help would be greatly appreciated.

Edit: Below is a sample of the data I'm using

HS_Hatch <- structure(list(ClutchID = c(1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 
                                        2L, 3L, 3L, 3L, 4L, 4L, 4L, 4L, 4L, 4L, 5L, 5L, 5L, 5L, 5L, 5L
), DateVisit = c("3/15/2012", "3/18/2012", "3/20/2012", "4/1/2012", 
                 "4/3/2012", "3/18/2012", "3/20/2012", "3/22/2012", "4/3/2012", 
                 "4/4/2012", "3/22/2012", "4/3/2012", "4/4/2012", "3/18/2012", 
                 "3/20/2012", "3/22/2012", "4/2/2012", "4/3/2012", "4/4/2012", 
                 "3/20/2012", "3/22/2012", "3/25/2012", "3/27/2012", "4/4/2012", 
                 "4/5/2012"), Year = c(2012L, 2012L, 2012L, 2012L, 2012L, 2012L, 
                                       2012L, 2012L, 2012L, 2012L, 2012L, 2012L, 2012L, 2012L, 2012L, 
                                       2012L, 2012L, 2012L, 2012L, 2012L, 2012L, 2012L, 2012L, 2012L, 
                                       2012L), Survive = c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
                                                           1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L)), class = c("tbl_df", 
                                                                                                                               "tbl", "data.frame"), row.names = c(NA, -25L), .Names = c("ClutchID", 
                                                                                                                                                                                         "DateVisit", "Year", "Survive"), spec = structure(list(cols = structure(list(
                                                                                                                                                                                             ClutchID = structure(list(), class = c("collector_integer", 
                                                                                                                                                                                                                                    "collector")), DateVisit = structure(list(), class = c("collector_character", 
                                                                                                                                                                                                                                                                                           "collector")), Year = structure(list(), class = c("collector_integer", 
                                                                                                                                                                                                                                                                                                                                             "collector")), Survive = structure(list(), class = c("collector_integer", 
                                                                                                                                                                                                                                                                                                                                                                                                  "collector"))), .Names = c("ClutchID", "DateVisit", "Year", 
                                                                                                                                                                                                                                                                                                                                                                                                                             "Survive")), default = structure(list(), class = c("collector_guess", 
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                "collector"))), .Names = c("cols", "default"), class = "col_spec"))
rosscova
  • 5,430
  • 1
  • 22
  • 35
birdnerd
  • 83
  • 1
  • 1
  • 5
  • 3
    Welcome to Stack Overflow! Can you please include data that will provide us with a [reproducible example](http://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example) ? – Ben Bolker Nov 13 '16 at 03:27
  • 3
    maybe `summarise(exposure=diff(range(DateVisit)))` ? – Ben Bolker Nov 13 '16 at 03:28
  • 1
    what @BenBolker said, just to add that his `summarise` line should go after your `group_by` line. And depending on the class of `DateVisit` you can either leave off your first `mutate` line, or change the `summarise` line to reference `Exposure` instead of `DateVisit`. – rosscova Nov 13 '16 at 03:57
  • Please, [edit] your question and add the data _properly formatted_ there, not in a comment and not as a link. Using `dput` is preferred. Thank you. – Uwe Nov 13 '16 at 03:59
  • @rosscova I modified the code per your Ben Boke suggestions, but I'm getting the following error message: "Error in UseMethod("mutate") : no applicable method for 'mutate' applied to an object of class "function" I tried removing the 1st mutate function but then I get this message: " no applicable method for 'regroup' applied to an object of class "Date" Like I mentioned I'm not super familiar with R and coding so I don't understand these error messages – birdnerd Nov 13 '16 at 04:15
  • First, please heed the `dput` suggestion above. Have you removed the last few lines of code in your question off? I'll leave putting all this together in an answer to @BenBolker if he wants to. – rosscova Nov 13 '16 at 04:24
  • @rosscova 1)I'm sorry, I don't understand how to properly format data with dput. Where can I find out more about that? 2)Which lines of code are you referring to? – birdnerd Nov 13 '16 at 04:31
  • One of your proximal problems is that `plyr` and `dplyr` both have `mutate` functions. Try specifying `dplyr::mutate`. – Ben Bolker Nov 13 '16 at 05:06
  • @BenBolker good point, except I think that the problematic mask might be with `summarise` should your first comment be implemented – rosscova Nov 13 '16 at 05:32

2 Answers2

8

Collecting some of the comments...

Load dplyr

We need only the dplyr package for this problem. If we load other packages, e.g. plyr, it can cause conflicts if both packages have functions with the same name. Let's load only dplyr.

library(dplyr)

In the future, you may wish to load tidyverse instead -- it includes dplyr and other related packages, for graphics, etc.

Converting dates

Let's convert the DateVisit variable from character strings to something R can interpret as a date. Once we do this, it allows R to calculate differences in days by subtracting two dates from each other.

HS_Hatch <- HS_Hatch %>%
 mutate(date_visit = as.Date(DateVisit, "%m/%d/%Y"))

The date format %m/%d/%Y is different from your original code. This date format needs to match how dates look in your data. DateVisit has dates as month/day/year, so we use %m/%d/%Y.

Also, you don't need to specify the dataset for DateVisit inside mutate, as in HS_Hatch$DateVisit, because it's already looking in HS_Hatch. The code HS_Hatch %>% ... says 'use HS_Hatch for the following steps'.

Calculating exposures

To calculate exposure, we need to find the first date, last date, and then the difference between the two, for each set of rows by ClutchID. We use summarize, which collapses the data to one row per ClutchID.

exposure <- HS_Hatch %>% 
    group_by(ClutchID) %>%
    summarize(first_visit = min(date_visit), 
              last_visit = max(date_visit), 
              exposure = last_visit - first_visit)

first_visit = min(date_visit) will find the minimum date_visit for each ClutchID separately, since we are using group_by(ClutchID).

exposure = last_visit - first_visit takes the newly-calculated first_visit and last_visit and finds the difference in days.

This creates the following result:

  ClutchID first_visit last_visit exposure
     <int>      <date>     <date>    <dbl>
1        1  2012-03-15 2012-04-03       19
2        2  2012-03-18 2012-04-04       17
3        3  2012-03-22 2012-04-04       13
4        4  2012-03-18 2012-04-04       17
5        5  2012-03-20 2012-04-05       16

If you want to keep all the original rows, you can use mutate in place of summarize.

David Kretch
  • 390
  • 1
  • 7
  • Thank you so much! I've been tearing my hair out trying to figure this out. As someone who is still learning R coding, I appreciate you explaining the code step-by-step. I was able to understand it and follow along. – birdnerd Nov 13 '16 at 18:28
  • Click "accept" if the answer solves your problem. This will help saying thank you to the responder, and will help others find the solution in the future. – rosscova Nov 13 '16 at 20:44
  • I am facing a similar problem. Upon replicating this example, for exposure, I only get returned one line. The grouping factor is ignored. Do you experience the same issue? Or is it because I'm using a newer version perhaps? – FlyingDutch Jun 28 '18 at 14:54
  • I'm not sure. I was able to replicate it with R 3.4.3 and dplyr 0.7.4. What does your one row result look like? – David Kretch Jun 30 '18 at 00:34
1

Here is a similar solutions if you look for a difftime results in days, from a vector date, without NA values produce in the new column, and if you expect to group by several conditions/groups.

make sure that your vector of date as been converting in the good format as previously explained.

dat2 <- dat %>% 
select(group1, group2, date) %>% 
arrange(group1, group2, date) %>% 
group_by(group1, group2) %>% 
mutate(diff_date = c(0,diff(date)))
VincentP
  • 89
  • 10