Problems while installing StanfordCoreNLP in OSX: How to set up the environment variables for jupyter/pycharm?

Question

I downloaded and installed the StanfordCoreNLP as follows:

$ cd
$ wget http://nlp.stanford.edu/software/stanford-ner-2015-12-09.zip
$ unzip stanford-ner-2015-12-09.zip
$ ls stanford-ner-2015-12-09/
build.xml    LICENSE.txt   ner-gui.bat      ner.sh                 sample.ner.txt     stanford-ner-3.6.0.jar          stanford-ner.jar
classifiers  ner.bat       ner-gui.command  README.txt             sample.txt         stanford-ner-3.6.0-javadoc.jar
lib          NERDemo.java  ner-gui.sh       sample-conll-file.txt  sample-w-time.txt  stanford-ner-3.6.0-sources.jar
$ export STANFORDTOOLSDIR=$HOME
$ export CLASSPATH=$STANFORDTOOLSDIR/stanford-ner-2015-12-09/stanford-ner.jar
$ export STANFORD_MODELS=$STANFORDTOOLSDIR/stanford-ner-2015-12-09/classifiers

Then let's try if it actually worked:

$ python3
>>>import nltk
>>> nltk.__version__
'3.1'
>>> from nltk.tag import StanfordNERTagger
>>> st = StanfordNERTagger('english.all.3class.distsim.crf.ser.gz')
>>> st.tag('Rami Eid is studying at Stony Brook University in NY'.split())

[('Rami', 'PERSON'), ('Eid', 'PERSON'), ('is', 'O'), ('studying', 'O'), ('at', 'O'), ('Stony', 'ORGANIZATION'), ('Brook', 'ORGANIZATION'), ('University', 'ORGANIZATION'), ('in', 'O'), ('NY', 'O')]

So, upon here everything works perfect. The problem is when I close the terminal and start working in the jupyter notebook (or pycharm):

In:

from nltk.tag import StanfordNERTagger
st = StanfordNERTagger('english.all.3class.distsim.crf.ser.gz')
st.tag('Rami Eid is studying at Stony Brook University in NY'.split())

Out:

---------------------------------------------------------------------------
LookupError                               Traceback (most recent call last)
<ipython-input-2-118c7edca797> in <module>()
      1 from nltk.tag import StanfordNERTagger
----> 2 st = StanfordNERTagger('english.all.3class.distsim.crf.ser.gz')
      3 st.tag('Rami Eid is studying at Stony Brook University in NY'.split())

/usr/local/lib/python3.5/site-packages/nltk/tag/stanford.py in __init__(self, *args, **kwargs)
    166 
    167     def __init__(self, *args, **kwargs):
--> 168         super(StanfordNERTagger, self).__init__(*args, **kwargs)
    169 
    170     @property

/usr/local/lib/python3.5/site-packages/nltk/tag/stanford.py in __init__(self, model_filename, path_to_jar, encoding, verbose, java_options)
     51                 self._JAR, path_to_jar,
     52                 searchpath=(), url=_stanford_url,
---> 53                 verbose=verbose)
     54 
     55         self._stanford_model = find_file(model_filename,

/usr/local/lib/python3.5/site-packages/nltk/__init__.py in find_jar(name_pattern, path_to_jar, env_vars, searchpath, url, verbose, is_regex)
    717         searchpath=(), url=None, verbose=True, is_regex=False):
    718     return next(find_jar_iter(name_pattern, path_to_jar, env_vars,
--> 719                          searchpath, url, verbose, is_regex))
    720 
    721 

/usr/local/lib/python3.5/site-packages/nltk/__init__.py in find_jar_iter(name_pattern, path_to_jar, env_vars, searchpath, url, verbose, is_regex)
    712                     (name_pattern, url))
    713         div = '='*75
--> 714         raise LookupError('\n\n%s\n%s\n%s' % (div, msg, div))
    715 
    716 def find_jar(name_pattern, path_to_jar=None, env_vars=(),

LookupError: 

===========================================================================
  NLTK was unable to find stanford-ner.jar! Set the CLASSPATH
  environment variable.

  For more information, on stanford-ner.jar, see:
    <http://nlp.stanford.edu/software>
===========================================================================

So, my question is how to set the environment variables correctly?, I also tried to add this to my bash profile:

#StanfordCORENLP
export STANFORDTOOLSDIR=$HOME
export CLASSPATH=$STANFORDTOOLSDIR/stanford-ner-2015-12-09/stanford-ner.jar
export STANFORD_MODELS=$STANFORDTOOLSDIR/stanford-ner-2015-12-09/classifiers

Update

Based on @alexis answer, this is what I tried:

user@MacBook-Pro:~$ % jupyter notebook /Users/user/Jupyter/Project1/Version1 &
-bash: bg: %: no such job
-bash: bg: jupyter: no such job
-bash: bg: notebook: no such job
-bash: bg: /Users/user/Jupyter/Project1/Version1: no such job

And this:

user@MacBook-Pro-de-User:~$ sudo mate /etc/launchd.conf
Password:
path::entries: scandir("/Applications/TextMate.app/Contents/Resources/PrivilegedTool"): Not a directory

Then:

user@MacBook-Pro-de-User:~$ sudo mate /etc/launchd.conf
Password:
path::entries: scandir("/Applications/TextMate.app/Contents/Resources/PrivilegedTool"): Not a directory

Then in the text editor:

#StanfordCORENLP
setenv export STANFORDTOOLSDIR=$HOME
setenv export CLASSPATH=$STANFORDTOOLSDIR/stanford-ner-2015-12-09/stanford-ner.jar
setenv export STANFORD_MODELS=$STANFORDTOOLSDIR/stanford-ner-2015-12-09/classifiers

Update2

I tried the following:

user@MacBook-Pro-de-User:~$ jupyter notebook /Users/user/Jupyter/Project1/Version1 & /Users/user/stanford-ner-2015-12-09

Then:

---------------------------------------------------------------------------
LookupError                               Traceback (most recent call last)
<ipython-input-1-215fd2418c10> in <module>()
      1 from nltk.tag import StanfordNERTagger
----> 2 st = StanfordNERTagger('english.all.3class.distsim.crf.ser.gz')
      3 #st = StanfordNERTagger('/home/me/stanford/stanford-postagger-full-2015-04-20/classifier/tagger.ser.gz',\
      4                            #'/home/me/stanford/stanford-spanish-corenlp-2015-01-08-models.jar')
      5 st.tag('Rami Eid is studying at Stony Brook University in NY'.split())

/usr/local/lib/python3.5/site-packages/nltk/tag/stanford.py in __init__(self, *args, **kwargs)
    166 
    167     def __init__(self, *args, **kwargs):
--> 168         super(StanfordNERTagger, self).__init__(*args, **kwargs)
    169 
    170     @property

/usr/local/lib/python3.5/site-packages/nltk/tag/stanford.py in __init__(self, model_filename, path_to_jar, encoding, verbose, java_options)
     51                 self._JAR, path_to_jar,
     52                 searchpath=(), url=_stanford_url,
---> 53                 verbose=verbose)
     54 
     55         self._stanford_model = find_file(model_filename,

/usr/local/lib/python3.5/site-packages/nltk/__init__.py in find_jar(name_pattern, path_to_jar, env_vars, searchpath, url, verbose, is_regex)
    717         searchpath=(), url=None, verbose=True, is_regex=False):
    718     return next(find_jar_iter(name_pattern, path_to_jar, env_vars,
--> 719                          searchpath, url, verbose, is_regex))
    720 
    721 

/usr/local/lib/python3.5/site-packages/nltk/__init__.py in find_jar_iter(name_pattern, path_to_jar, env_vars, searchpath, url, verbose, is_regex)
    712                     (name_pattern, url))
    713         div = '='*75
--> 714         raise LookupError('\n\n%s\n%s\n%s' % (div, msg, div))
    715 
    716 def find_jar(name_pattern, path_to_jar=None, env_vars=(),

LookupError: 

===========================================================================
  NLTK was unable to find stanford-ner.jar! Set the CLASSPATH
  environment variable.

  For more information, on stanford-ner.jar, see:
    <http://nlp.stanford.edu/software>
===========================================================================

You didn't understand my answer: The `%` represents the bash prompt. Try `jupyter notebook ` — alexis, Oct 13 '16 at 14:58
What's the extra argument `/Users/user/stanford-ner-2015-12-09` doing after the jupyter command? Is your CLASSPATH still set correctly? Does your script rely on any relative paths? Check `os.environ` from within the notebook to make sure it is set correctly. Try launching the notebook with `jupyter notebook .`, from a folder where commandline python3 invocation worked (and ensure it still works). — alexis, Oct 13 '16 at 16:52

score 2 · Answer 1 · edited May 23 '17 at 10:32

Look like you're starting the notebook server from the GUI. The problem is that in OS X, the GUI does not inherit the environment of the shell. You can immediately get it to work by starting the notebook server from the terminal:

jupyter notebook <path-to-notebook-folder> &

This should let you run with the same environment that your successful python3 invocation used.

There are ways to modify the environment of applications launched by the GUI; if you don't mind setting CLASSPATH system-wide, you can set it in /etc/launchd.conf as described here. (Log out or restart your system after editing launchd.conf). You could also try this (not sure if it'll still work), or google around for other alternatives.

I updated the status of the question alexis, thanks for the help!. — john doe, Oct 13 '16 at 13:11

Problems while installing StanfordCoreNLP in OSX: How to set up the environment variables for jupyter/pycharm?

1 Answers1