0

I try to import a Python package from github. I am working in Google Colab.

The repository is at the following url https://github.com/microsoft/nlp-recipes/tree/master/utils_nlp.

So I use the following code

!pip install --upgrade 
!pip install -q git+git://github.com/microsoft/nlp-recipes/tree/master/utils_nlp
from utils_nlp import *

I tried as well

!pip install -q git+https://github.com/microsoft/nlp-recipes/tree/master/utils_nlp

I saw other (working) examples where the url ends in .git :

!pip install -q git+https://github.com/huggingface/transformers.git

so I tried in turn

!pip install -q git+https://github.com/microsoft/nlp-recipes/tree/master/utils_nlp.git

But I also noticed that https://github.com/microsoft/nlp-recipes/tree/master/utils_nlp.git loads to an error page while https://github.com/huggingface/transformers.git load to https://github.com/huggingface/transformers which surprises me.

How do we load the Python package here ?

EDIT

I am using as suggested

pip install git+https://github.com/microsoft/nlp-recipes.git

then

from utils_nlp import *

works but it doesn't successfully import subfolders, it fails when I do

from utils_nlp.models.transformers.abstractive_summarization_bertsum \
      import BertSumAbs, BertSumAbsProcessor

whereas utils_nlp does contain a folder models which in turn contains transformers

The error stack is then the following

/usr/local/lib/python3.7/dist-packages/utils_nlp/models/transformers/abstractive_summarization_bertsum.py in <module>()
     15 from torch.utils.data.distributed import DistributedSampler
     16 from tqdm import tqdm
---> 17 from transformers import AutoTokenizer, BertModel
     18 
     19 from utils_nlp.common.pytorch_utils import (

ModuleNotFoundError: No module named 'transformers'

So strangely code in utils_nlp.models.transformers.abstractive_summarization_bertsum doesn't resolve the dependency to transformers

kiriloff
  • 25,609
  • 37
  • 148
  • 229

1 Answers1

1

This is the correct way to install it:

pip install git+https://github.com/microsoft/nlp-recipes.git

You can't install a module, only a package can be installed. After the installation you can go on with the rest of your code

from utils_nlp import *
...

As you can read in the setup guide (which you should definitely read when installing a package):

The pip installation does not install any of the necessary package dependencies, it is expected that conda will be used as shown above to setup the environment for the utilities being used.

This explains your error: the transformers package is not installed automatically, you need to install it on your own (simply use pip install transformers). This is also confirmed in the setup.py file: the are no "install_requires" dependencies.

Riccardo Bucco
  • 13,980
  • 4
  • 22
  • 50