21

I'm trying to train an NER model using spaCy to identify locations, (person) names, and organisations. I'm trying to understand how spaCy recognises entities in text and I've not been able to find an answer. From this issue on Github and this example, it appears that spaCy uses a number of features present in the text such as POS tags, prefixes, suffixes, and other character and word-based features in the text to train an Averaged Perceptron.

However, nowhere in the code does it appear that spaCy uses the GLoVe embeddings (although each word in the sentence/document appears to have them, if present in the GLoVe corpus).

My questions are -

  1. Are these used in the NER system now?
  2. If I were to switch out the word vectors to a different set, should I expect performance to change in a meaningful way?
  3. Where in the code can I find out how (if it all) spaCy is using the word vectors?

I've tried looking through the Cython code, but I'm not able to understand whether the labelling system uses word embeddings.

Navaneethan Santhanam
  • 1,707
  • 2
  • 13
  • 17
  • 1
    Did you find out anything? I'd love the same information. – macarthy Oct 09 '17 at 07:54
  • Sadly, no - I wasn't able to and eventually gave up the search. I used MITIE instead - https://github.com/mit-nlp/MITIE/. – Navaneethan Santhanam Oct 09 '17 at 11:10
  • See an answer on the internals of the spaCy NER [here](https://stackoverflow.com/questions/60381170/which-deep-learning-algorithm-does-spacy-uses-when-we-train-custom-model/60394246#60394246). – Omri374 Aug 17 '20 at 08:12

1 Answers1

21

spaCy does use word embeddings for its NER model, which is a multilayer CNN. There's a quite a nice video that Matthew Honnibal, the creator of spaCy made, about how its NER works here. All three English models use GloVe vectors trained on Common Crawl, but the smaller models "prune" the number of vectors by having similar words mapped to the same vector link.

It's quite doable to add custom vectors. There's an overview of the process in the spaCy docs, plus some example code on Github.

ahalt
  • 601
  • 4
  • 10
  • Four years member and never finished the tour ;-) – ZF007 Jan 30 '18 at 21:18
  • More information (derived from the youtube video) can be found [here](https://stackoverflow.com/questions/60381170/which-deep-learning-algorithm-does-spacy-uses-when-we-train-custom-model/60394246#60394246) – Omri374 Aug 17 '20 at 08:12