8

I installed pipenv and then used pipenv install beautifulsoup4. My understanding is that this should have created a pipfile and a virtual env. So I started up pipenv shell. My pipfile is there, with Beautiful Soup there. Next thing I tried to do was pipenv install selenium. I wrote this really short script:

from bs4 import BeautifulSoup
from selenium import webdriver

driver = webdriver.Firefox()
profile = 'https://www.linkedin.com/in/user-profile-name'

driver.get(profile)

html = driver.page_source

soup = BeautifulSoup(html)

print(soup)

I tried running it and got this error:

Traceback (most recent call last):
  File "LiScrape.py", line 2, in <module>
    from selenium import webdriver
ModuleNotFoundError: No module named 'selenium'

I tried running python3 in the shell and just doing import selenium to see if it would let me check the version. Again, I got the ModuleNotFoundError.

What am I doing wrong with selenium that I didn't do wrong with beautiful soup??

Rob Bednark
  • 25,981
  • 23
  • 80
  • 125
bkula
  • 541
  • 3
  • 10
  • 22

1 Answers1

13

You just need to activate the virtual environment created by pipenv either by:

$ pipenv run python foo.py

or:

$ pipenv shell
> python foo.py

The whole process for reference:

$ pipenv --python 3.6.4 install beautifulsoup4 selenium
$ echo "import bs4 ; import selenium" > foo.py
$ pipenv run python foo.py

Or whatever version of Python you prefer.

(You should see no errors.)

This works for me.

Taylor D. Edmiston
  • 12,088
  • 6
  • 56
  • 76
  • It worked! Mostly... I got an error from selenium about 'geckodriver' executable needing to be in PATH, but your solution fixed my problem. – bkula Mar 23 '18 at 01:41
  • Awesome! This should help if you haven't resolved the PATH error already. https://stackoverflow.com/questions/40208051/selenium-using-python-geckodriver-executable-needs-to-be-in-path – Taylor D. Edmiston Mar 23 '18 at 03:38