Join dataframes by multiple columns with grep

Question

I want to do a full join with two dataframes based on 2 columns where 1 column contains the string found in the other. Below are my two dataframes:

date<-as.Date(c('2010-11-1','2008-3-25','2007-3-14'))
site<-c("abcejams.com", "reitimes.com", "posehbc")
desc1<-c("alpha", "beta", "gamma"
df1<-data.frame(date, site, desc1)
df1

        date         site    desc1
1 2010-11-01 abcejams.com    alpha
2 2008-03-25 reitimes.com     beta
3 2007-03-14      posehbc    gamma

date2<-as.Date(c('2010-11-1','2008-3-25','2007-3-14', '2018-2-9'))
site2<-c("jams", "time", "pose", "abce")
metric2<-c(1,2,3,4)
metric3<-c(10,20,30,40)
df2<-data.frame(date2, site2, metric2, metric3)
df2

       date2 site2 metric2 metric3
1 2010-11-01  jams       1      10
2 2008-03-25  time       2      20
3 2007-03-14  pose       3      30
4 2018-02-09  abce       4      40

I want to join this by Date AND Site based on site2 being in site by date. This is how you would normally do it without the grep portion.

finaldf<-full_join(df1, df2, by = c("date"="date2", "site" = "site2"))

There is a way to do this with sqldf but the only option is a left join and not a full join:

test<-sqldf("df1.*, df2.metric2, 
df2.metric3 
        from df1 
        left join df2 
        on 
        instr(df1.site,  df2.site2)
        and 
        df1.date=df2.date2")

The goal is to have the final output look like this:

        date         site     desc1     metric2    metric3
1 2010-11-01 abcejams.com     alpha           1         10 
2 2008-03-25 reitimes.com      beta           2         20
3 2007-03-14      posehbc     gamma           3         30
4 2018-02-09         abce        NA           4         40

Anyone have any experience with this?

Jake Kaupp · Accepted Answer · 2018-02-21T19:05:40.197

You can use the fuzzyjoin package and use a regex_full_join. I don't believe it's on CRAN right now, so check the github page to install it.

library(fuzzyjoin)

date <- as.Date(c('2010-11-1', '2008-3-25', '2007-3-14'))
site <- c("abcejams.com", "reitimes.com", "posehbc")

df1 <- data.frame(date, site, stringsAsFactors = FALSE)

date2 <- as.Date(c('2010-11-1', '2008-3-25', '2007-3-14', '2018-2-9'))
site2 <- c("jams", "time", "pose", "abce")
metric2 <- c(1, 2, 3, 4)
metric3 <- c(10, 20, 30, 40)

df2 <- data.frame(date2, site2, metric2, metric3, stringsAsFactors = FALSE)

regex_full_join(df1, df2, by = c("site" = "site2", "date" = "date2"))


            date         site      date2 site2 metric2 metric3
1 2010-11-01 abcejams.com 2010-11-01  jams       1      10
2 2008-03-25 reitimes.com 2008-03-25  time       2      20
3 2007-03-14      posehbc 2007-03-14  pose       3      30
4       <NA>         <NA> 2018-02-09  abce       4      40

Thanks, I got an error though that says: `Error: Can only modify plain character vectors.` Did you not get the same error I'm assuming? — nak5120, Feb 24 '18 at 20:13
Note the change of `stringsAsFactors = FALSE` in the data frame creation. — Jake Kaupp, Feb 24 '18 at 20:15

CPak · Answer 2 · 2018-02-21T19:35:54.730

# original data
date<-as.Date(c('2010-11-1','2008-3-25','2007-3-14'))
site<-c("abcejams.com", "reitimes.com", "posehbc")
desc1<-c("alpha", "beta", "gamma")
df1<-data.frame(date, site, desc1)

date2<-as.Date(c('2010-11-1','2008-3-25','2007-3-14', '2018-2-9'))
site2<-c("jams", "time", "pose", "abce")
metric2<-c(1,2,3,4)
metric3<-c(10,20,30,40)
df2<-data.frame(date2, site2, metric2, metric3)

I wrote a function that defines groups based on both conditions being TRUE.

library(tidyverse)
library(stringr)

make_groups <- function(bicond) {
    lapply(apply(bicond, 1, function(i) which(i == 1)), function(i) if (length(i)==0) { NA } else { i })
}

custom_join <- function(df1, df2) {
    cond1 <- outer(df2$date2, df1$date, "==")
    cond2 <- outer(as.character(df2$site2), as.character(df1$site), function(i, j) str_detect(j, i))
    bicond <- cond1 * cond2
    data1 <- df1 %>% mutate(G = row_number())
    data2 <- df2 %>% mutate(G = make_groups(bicond)) %>% unnest(G)
    full_join(data2, data1, by=c("G" = "G", "date2" = "date")) %>% select(-G)
}

custom_join(df1, df2)

       # date2 site2 metric2 metric3         site desc1
# 1 2010-11-01  jams       1      10 abcejams.com alpha
# 2 2008-03-25  time       2      20 reitimes.com  beta
# 3 2007-03-14  pose       3      30      posehbc gamma
# 4 2018-02-09  abce       4      40         <NA>  <NA>

Another example

# new data
date<-as.Date(c('2010-11-1','2008-3-25','2007-3-14','2007-3-14'))
site<-c("abcejams.com", "reitimes.com", "posehbc", "poseur")
desc1<-c("alpha", "beta", "gamma", "epsilon")
df1<-data.frame(date, site, desc1)

date2<-as.Date(c('2010-11-1','2008-3-25','2007-3-14', '2007-2-9'))
site2<-c("jams", "time", "pose", "abce")
metric2<-c(1,2,3,4)
metric3<-c(10,20,30,40)
df2<-data.frame(date2, site2, metric2, metric3)

custom_join(df1, df2)

       # date2 site2 metric2 metric3         site   desc1
# 1 2010-11-01  jams       1      10 abcejams.com   alpha
# 2 2008-03-25  time       2      20 reitimes.com    beta
# 3 2007-03-14  pose       3      30      posehbc   gamma
# 4 2007-03-14  pose       3      30       poseur epsilon
# 5 2007-02-09  abce       4      40         <NA>    <NA>

Join dataframes by multiple columns with grep

2 Answers2