2

I am attempting to move over my web scraping program from using the requests library to the requests-html library to allow me to render the javascript on webpages. On the import of the module this error is thrown:

Traceback (most recent call last):
  File "backend2.py", line 2, in <module>
    import requests_html
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests_html.py", line 9, in <module>
    import pyppeteer
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/__init__.py", line 30, in <module>
    from pyppeteer.launcher import connect, launch, executablePath  # noqa: E402
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/launcher.py", line 24, in <module>
    from pyppeteer.browser import Browser
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/browser.py", line 15, in <module>
    from pyppeteer.page import Page
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/page.py", line 20, in <module>
    from pyppeteer.coverage import Coverage
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/coverage.py", line 15, in <module>
    from pyppeteer.util import merge_dict
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/util.py", line 10, in <module>
    from pyppeteer.chromium_downloader import check_chromium, chromium_executable
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/pyppeteer/chromium_downloader.py", line 15, in <module>
    from tqdm import tqdm
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tqdm/__init__.py", line 1, in <module>
    from ._tqdm import tqdm
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tqdm/_tqdm.py", line 75, in <module>
    mp_lock = mp.RLock()  # multiprocessing lock
AttributeError: module 'multiprocessing' has no attribute 'RLock'

Any help is appreciated!

S. Allen
  • 171
  • 1
  • 10

1 Answers1

1

You are using Python 3.7, but the github of requests-html states that only Python 3.6 is supported (bottom of page). I get all kinds of horrible errors when I try to use Python 3.7 but 3.6 works fine. So, seems strange I know, but please try using 3.6

Even so, as @georgexsh states, there should still be a RLock available for import from multiprocessing in 3.7 so it looks like your error may actually be that one of your code files is called multiprocessing.py or you have your own package called multiprocessing (so you are importing that instead of the module you wanted).

Rob Bricheno
  • 4,467
  • 15
  • 29
  • I think that may have helped a bit (using 3.6.6) though now the program is failing the import because `from multiprocessor import simpleQueue` throws an `ImportError: cannot import name SimpleQueue`. (I'll probably end up posting another question relating to this) – S. Allen Nov 27 '18 at 12:56
  • @S.Allen That should be `from multiprocessing import SimpleQueue` – Rob Bricheno Nov 27 '18 at 13:06
  • Sorry, that was meant to me `multiprocessing`, I should have ctrl-c/v. I also tried reinstalling multiprocessing to no avail. – S. Allen Nov 27 '18 at 13:12
  • 1
    @S.Allen I am really suspicious that you have another module called multiprocessing somewhere in your path. What is the output if you follow the instructions here https://stackoverflow.com/a/248862/9794932 using `multiprocessing` as `a_module` ? – Rob Bricheno Nov 27 '18 at 13:17
  • I actually did which solved the problem. Somehow I managed to avoid scrolling down in the current working directory where there was a `multiprocessing.py` file, thanks. – S. Allen Nov 28 '18 at 01:46