egen rowtotal (Stata) equivalent in R?

Question

I have data that look like this

bankname    bankid  year    deposit dep_cert capital    surplus
Bank A         1    1881    244789  7250       20218    29513
Bank B         2    1881    195755  10243     185151    NA 
Bank C         3    1881    107736  13357     177612    NA
Bank D         4    1881    170600  NA         20000    NA
Bank E         5    1881    320000  351266    314012    NA

This is the code to replicate the data.

bankname <- c("Bank A","Bank B","Bank C","Bank D","Bank E")
bankid <- c( 1, 2,  3,  4,  5)
year<- c( 1881, 1881,   1881,   1881,   1881)
deposit  <- c(244789,    195755, 107736, 170600, 32000000)
dep_cert<-c(7250,10243,13357,NA,351266)
capital<-c(20218,185151,177612,20000,314012)
surplus<-c(29513,NA,NA,NA,NA)
bankdata<-data.frame(bankname, bankid,year,deposit, dep_cert, capital, surplus)

I want to create a new column called liability as the sum of deposit, dep_cert, capital, and surplus. This means that the data would look like this.

bankname    bankid  year    deposit dep_cert capital    surplus liability
Bank A         1    1881    244789  7250       20218    29513   301770 
Bank B         2    1881    195755  10243     185151    NA      391149
Bank C         3    1881    107736  13357     177612    NA      298705
Bank D         4    1881    170600  NA         20000    NA      190600
Bank E         5    1881    320000  351266    314012    NA      32665278

However, when I used the sum command in R, I got NAs due to missing values. In Stata, I would do

egen liability = rowtotal(deposit, dep_cert,capital, surplus)

What would be the equivalent code in R?

Also, my second question is, to replace all the missing values (NAs) with the number 0 in the data, in Stata, I would do

foreach x of varlist deposit dep_cert capital surplus {
    replace `x'=0 if missing(`x')
}

What would be the equivalent code in R?

Answer 1

The equivalent would be rowSums in this case:

rowSums(bankdata[c("deposit", "dep_cert", "capital", "surplus")], na.rm = TRUE)
# [1]   301770   391149   298705   190600 32665278

The main thing that you're missing is the na.rm = TRUE argument.

To add it to your data.frame, you would do:

bankdata$liability <- rowSums(bankdata[c("deposit", "dep_cert", 
                                         "capital", "surplus")], 
                              na.rm = TRUE)

---

To replace NA values with "0" in the same columns, you can do:

## save some typing
cols <- c("deposit", "dep_cert", "capital", "surplus")

bankdata[cols][is.na(bankdata[cols])] <- 0
bankdata
#   bankname bankid year  deposit dep_cert capital surplus
# 1   Bank A      1 1881   244789     7250   20218   29513
# 2   Bank B      2 1881   195755    10243  185151       0
# 3   Bank C      3 1881   107736    13357  177612       0
# 4   Bank D      4 1881   170600        0   20000       0
# 5   Bank E      5 1881 32000000   351266  314012       0

Answer 2

Not a full answer, but is too long to be a comment:

Your Stata code as originally stated

foreach `x' of varlist deposit dep_cert capital surplus {
    replace `x'=0 if missing(`x')
}

(1) would not work (2) is a bad idea any way.

This would work

foreach x of varlist deposit dep_cert capital surplus {
    replace `x' = 0 if missing(`x')
}

and this would work too, and is more concise,

foreach x in deposit dep_cert capital surplus {
    replace `x' = 0 if missing(`x')
}

but nevertheless overwriting missings with zeros in your raw data is likely to lead to loss of information and is a blow to the integrity of your data. As by default egen ignores missings when calculating row totals, it is not needed any way in this context.

Answer 3

Using data.table

library(data.table)
nm1 <- c("deposit", "dep_cert", "capital", "surplus")

setDT(bankdata)[,liabiliy:=Reduce(`+`,
           lapply(.SD, function(x) replace(x, is.na(x), 0))),.SDcols=nm1]


 bankdata
 #   bankname bankid year  deposit dep_cert capital surplus liabiliy
 #1:   Bank A      1 1881   244789     7250   20218   29513   301770
 #2:   Bank B      2 1881   195755    10243  185151      NA   391149
 #3:   Bank C      3 1881   107736    13357  177612      NA   298705
 #4:   Bank D      4 1881   170600       NA   20000      NA   190600
 #5:   Bank E      5 1881 32000000   351266  314012      NA 32665278

To replace NA with 0 and do the rowSums

 setDT(bankdata)[, (nm1):=lapply(.SD, function(x) 
       replace(x, is.na(x), 0)), .SDcols=nm1][,
             liability:=Reduce(`+`, .SD), .SDcols=nm1]

 bankdata
 #   bankname bankid year  deposit dep_cert capital surplus liability
 #1:   Bank A      1 1881   244789     7250   20218   29513    301770
 #2:   Bank B      2 1881   195755    10243  185151       0    391149
 #3:   Bank C      3 1881   107736    13357  177612       0    298705
 #4:   Bank D      4 1881   170600        0   20000       0    190600
 #5:   Bank E      5 1881 32000000   351266  314012       0  32665278

Benchmarks

 bankdata1 <- bankdata[rep(1:nrow(bankdata), 1e5),]
 row.names(bankdata1) <- 1:nrow(bankdata1)

 f1 <- function() {rowSums(bankdata1[c("deposit", "dep_cert", 
                                     "capital", "surplus")], 
                          na.rm = TRUE)

              }



 f2 <- function() {nm1 <- c("deposit", "dep_cert", "capital", "surplus")
                 DT <- data.table(bankdata1, key=c('bankname', 'bankid', 'year'))  
                 DT[, liabiliy:=Reduce(`+`,
                     lapply(.SD, function(x) replace(x, is.na(x), 0))),.SDcols=nm1]
              }

 library(microbenchmark)
 microbenchmark(f1(), f2(), unit="relative")
 #   Unit: relative
 #expr      min       lq   median       uq      max neval
 #f1() 1.558999 1.355819 1.457036 1.426796 1.525313   100
 #f2() 1.000000 1.000000 1.000000 1.000000 1.000000   100

Answer 4

For both tasks, you can also use mutate from the dplyr package.

library(dplyr)
vars <- c("deposit", "dep_cert", "capital", "surplus")

Calculating the row sum/total

As explained by A Handcart And Mohair in their answer, you can use rowSums together with na.rm = TRUE:

bankdata = bankdata %>%
    mutate(liability = rowSums(.[vars], na.rm = TRUE))

Setting NA's to 0

I would also advise you against doing this (see Nick Cox's comment), but if you need to, you can use mutate_ together with replace (see also this answer on SO).

var_fun <- paste("replace(", vars, ", is.na(", vars, "), 0)", sep="")

bankdata = bankdata %>%
    mutate_(.dots = setNames(var_fun, eval(vars)))

setNames creates a vector containing the name of the variable and the function to generate it. You need to use the underscore variant of mutate_ here in order to use quoted variable names. The technique is explained in more detail in the vignette on NSE (non-standard evaluation).