11

I'm working on a project involving cleaning a list of data on college majors. I find that a lot are misspelled, so I was looking to use the function gsub() to replace the misspelled ones with its correct spelling. For example, say 'biolgy' is misspelled in a list of majors called Major. How can I get R to detect the misspelling and replace it with its correct spelling? I've tried gsub('biol', 'Biology', Major) but that only replaces the first four letters in 'biolgy'. If I do gsub('biolgy', 'Biology', Major), it works for that case alone, but that doesn't detect other forms of misspellings of 'biology'.

Thank you!

Alan
  • 113
  • 1
  • 1
  • 5

5 Answers5

13

You should either define some nifty regular expression, or use agrep from base package. stringr package is another option, I know that people use it, but I'm a very huge fan of regular expressions, so it's a no-no for me.

Anyway, agrep should do the trick:

agrep("biol", "biology")
[1] 1
agrep("biolgy", "biology")
[1] 1

EDIT:

You should also use ignore.case = TRUE, but be prepared to do some bookkeeping "by hand"...

aL3xa
  • 35,415
  • 18
  • 79
  • 112
  • 1
    Thanks for your response, I played around with agrep just now. I find that it simply returns an integer (which I'm guessing corresponds to the number of strings changed), but does it actually execute the change? So for example, say I had animals = c("mouse", "dog", "cat"). If I wanted to replace mouse with, say "cheese", then can I use agrep("mou", "cheese", animals)? It returns an integer(0) when I do this. Thank you!! – Alan Mar 14 '11 at 18:57
  • 1
    `agrep` returns vector indices, so you can easily assign a value by using subscripts: `animals[agrep("mou", animals)] <- "cheese"` – aL3xa Mar 14 '11 at 19:10
  • 1
    Use `value = TRUE` to return the near-matches, rather than just the indicies. – Richie Cotton Jul 14 '11 at 09:31
2

You can set up a vector of all the possible misspellings and then do a loop over a gsub call. Something like:

biologySp = c("biolgy","biologee","bologee","bugs")

for(sp in biologySp){
  Major = gsub(sp,"Biology",Major)
}

If you want to do something smarter, see if there's any fuzzy matching packages on CRAN, or something that uses 'soundex' matching....

The wikipedia page on approx. string matching might be useful, and try searching R-help for some of the key terms.

http://en.wikipedia.org/wiki/Approximate_string_matching

Spacedman
  • 92,590
  • 12
  • 140
  • 224
  • There's already fuzzy matching in `base` package: `agrep` function does that. See my answer below. – aL3xa Mar 14 '11 at 18:49
2

You could first match the majors against a list of available majors, any not matching would then be the likely missspellings. Then use the agrep function to match these against the known majors again (agrep does approximate matching, so if it is similar to a correct value then you will get a match).

Greg Snow
  • 48,497
  • 6
  • 83
  • 110
  • And for that for the remaning majors, something like `replace(remainingMajor,agrep("biology",remainingMajor),"biology")` should do it. – hatmatrix Mar 14 '11 at 18:50
  • (but check before with `remainingMajor[agrep("biology",remainingMajor)]` to see what you'll be replacing) – hatmatrix Mar 14 '11 at 18:51
0

example 1a) perl/linux regex: 's/oldstring/newstring/'

example 1b) R equivalent of 1a: srcstring=sub(oldstring, newstring, srcstring)

example 2a) perl/linux regex: 's/oldstring//'

example 2b) R equivalent of 2a: srcstring=sub(oldstring, "", srcstring)

lennon310
  • 12,503
  • 11
  • 43
  • 61
cspoleta
  • 1
0

The vwr package has methods for string matching:

http://ftp.heanet.ie/mirrors/cran.r-project.org/web/packages/vwr/index.html

so your best bet might be to use the string with the minimum Levenshtein distance from the possible subject strings:

> levenshtein.distance("physcs",c("biology","physics","geography"))
  biology   physics geography 
        7         1         9

If you get identical minima then flip a coin:

> levenshtein.distance("biolsics",c("biology","physics","geography"))
  biology   physics geography 
        4         4         8
Spacedman
  • 92,590
  • 12
  • 140
  • 224