3

I am getting an error when trying to run a example .py from the serpscrap package.

I am on an iPhoneX in Pythonista.

Any help would be greatly appreciated.

Here is the traceback

Traceback (most recent call last):
  File "/private/var/mobile/Containers/Shared/AppGroup/C27C5B11-1804-47B9-AF18-7287A8337C55/Pythonista3/Documents/testing/serpscrapetest.py", line 3, in <module>
    import serpscrap
  File "/private/var/mobile/Containers/Shared/AppGroup/C27C5B11-1804-47B9-AF18-7287A8337C55/Pythonista3/Documents/site-packages-3/serpscrap/__init__.py", line 5, in <module>
    from serpscrap.serpscrap import SerpScrap
  File "/private/var/mobile/Containers/Shared/AppGroup/C27C5B11-1804-47B9-AF18-7287A8337C55/Pythonista3/Documents/site-packages-3/serpscrap/serpscrap.py", line 11, in <module>
    from scrapcore.core import Core
  File "/private/var/mobile/Containers/Shared/AppGroup/C27C5B11-1804-47B9-AF18-7287A8337C55/Pythonista3/Documents/site-packages-3/scrapcore/core.py", line 8, in <module>
    from scrapcore.cachemanager import CacheManager
  File "/private/var/mobile/Containers/Shared/AppGroup/C27C5B11-1804-47B9-AF18-7287A8337C55/Pythonista3/Documents/site-packages-3/scrapcore/cachemanager.py", line 11, in <module>
    from scrapcore.parsing import Parsing
  File "/private/var/mobile/Containers/Shared/AppGroup/C27C5B11-1804-47B9-AF18-7287A8337C55/Pythonista3/Documents/site-packages-3/scrapcore/parsing.py", line 6, in <module>
    from scrapcore.parser.google_parser import GoogleParser
  File "/private/var/mobile/Containers/Shared/AppGroup/C27C5B11-1804-47B9-AF18-7287A8337C55/Pythonista3/Documents/site-packages-3/scrapcore/parser/google_parser.py", line 7, in <module>
    from scrapcore.parser.parser import Parser
  File "/private/var/mobile/Containers/Shared/AppGroup/C27C5B11-1804-47B9-AF18-7287A8337C55/Pythonista3/Documents/site-packages-3/scrapcore/parser/parser.py", line 7, in <module>
    import lxml.html
  File "/private/var/mobile/Containers/Shared/AppGroup/C27C5B11-1804-47B9-AF18-7287A8337C55/Pythonista3/Documents/site-packages-3/lxml/html/__init__.py", line 54, in <module>
    from .. import etree
ImportError: cannot import name 'etree'
Andrej Kesely
  • 168,389
  • 15
  • 48
  • 91
Jake Walter
  • 31
  • 1
  • 4

2 Answers2

2

SerpScrap doesn't work on iOS at the moment, because of the lxml dependency. I will update the docs to clearify this point.

But maybe you take a look on https://github.com/pybee/Python-Apple-support . I'm not familiar with iOS, maybe someone can check if this can help to solve this issue. Otherwise feel free to open an issue on the serpscrap github page.

Niklas Rosencrantz
  • 25,640
  • 75
  • 229
  • 424
ecoron
  • 36
  • 1
1

serpscrap evidently requires lxml, even though for some reason it only documents that requirement for Windows, rather than for all platforms.

You can't install packages that need C extension modules, like lxml, in Pythonista. You seem to have somehow gotten the pure-Python part of lxml installed,1 but that won't do any good without the C extension modules.

There's an open issue, #245, to add lxml as a pre-installed package with Pythonista, which would probably fix your problem.

So, your options are:

  • Help Pythonista get lxml building so they can close that bug and add lxml to the next version.
  • Modify serpscrap to work without lxml (if it's just using lxml.etree, it's possible that it would work with the stdlib ETree implementation).
  • File a feature request with serpscrap and hope someone else does it.
  • Switch to a different scraper that doesn't require lxml.

1. I'm not sure how that could happen; the install should just fail. But maybe serpscrap does some weird thing that happens to work on Linux and macOS but not iOS, and that's why they only list lxml as a dependency on Windows in the first place?

abarnert
  • 354,177
  • 51
  • 601
  • 671
  • @JakeWalter No, I don't. – abarnert Aug 03 '18 at 20:29
  • I installed lxml with pip – Jake Walter Aug 03 '18 at 20:37
  • Pythonista comes with examples using c so I don’t understand why you are saying that it can’t use c ? – Jake Walter Aug 04 '18 at 13:44
  • @JakeWalter Which examples? The docs say C extensions don’t work. There are open issues asking for specific modules to be added because C extensions don’t work. I’m not making this up to make your life harder, and arguing with me isn’t going to make your code start working. – abarnert Aug 04 '18 at 20:29
  • I apologize, I am not trying to argue I am just a little confused. This link says you can build c Extensions in Pythonista http://omz-software.com/pythonista/docs/extending/building.html – Jake Walter Aug 05 '18 at 14:47
  • https://photos.app.goo.gl/9QNKvjPuZw9DxZf89 here is a photo showing the objC examples folder – Jake Walter Aug 05 '18 at 14:49
  • You know what, I think I understand what you are saying now. Because etree.pyx right? – Jake Walter Aug 05 '18 at 14:52
  • http://omz-software.com/pythonista/docs/ios/index.html says objective-C API is installed – Jake Walter Aug 05 '18 at 15:52
  • @JakeWalter The extending docs you're linking to are a copy of the CPython docs. You _can_ actually use this to build extensions for iOS Python, but (a) you can't do that in Pythonista, or on your iOS device at all, only on your Mac, and (b) you can't install them to Pythonista, only to build your own app linking the Pythonista frameworks to embed Python and the extension into your own iOS app. – abarnert Aug 05 '18 at 18:26
  • Meanwhile, pyobjc has nothing to do with building native extensions; it's a way to use Cocoa Touch ObjC libraries _without_ writing and compiling native extensions, from pure Python. – abarnert Aug 05 '18 at 18:27
  • `.pyx` files are just a special case of this problem. They're Cython files, which Cython can compile to C source code—and that part could work on Pythonista—but that C source needs to be compiled to native code and built into an extension, and that part _can't_ work on Pythonista. – abarnert Aug 05 '18 at 18:29
  • I just want to say thank you so much. thank you for your time an thank you for teaching me so much. i now understand. – Jake Walter Aug 05 '18 at 21:25
  • 1
    c extensions wont work in pythonista. pythonista has a bridge to the ios Objective C runtime, but that is not fresh compiled c code -- the runtime is written in c, but operates based on messages, which we can send, and bridge functions back to python code. – JonB Oct 11 '18 at 16:00