pylint 1.4 reports E1101(no-member) on all C extensions

Question

We've been long-time fans of pylint. Its static analysis has become a critical part of all our python projects and has saved tons of time chasing obscure bugs. But after upgrading from 1.3 -> 1.4, almost all compiled c extensions result in E1101(no-member) errors.

Projects that previously run perfectly clean through pylint 1.3 now complain about almost every C extension member with E1101. We've been forced to disable E1101 errors, but this materially detracts from the usefulness of pylint.

For example, this perfectly valid use of the lxml package

r"""valid.py: demonstrate pylint 1.4 error"""
from lxml import etree
print etree.Element('mydoc')

Run this through pylint, and it reports:

$ pylint -rn valid.py
No config file found, using default configuration
************* Module valid
E:  3, 6: Module 'lxml.etree' has no 'Element' member (no-member)

But it is perfectly valid:

$ python valid.py
<Element mydoc at 7fddf67b1ba8>

Here's where it gets really weird. A very small handful of C extensions seem to work just fine through pylint, e.g.:

r"""valid2.py: this one works fine"""
import sqlite3
print sqlite3.version

$ pylint -rn valid2.py
No config file found, using default configuration

My question is, has anyone else witnessed this? And if so, would you be willing to share your workaround/solution?

We've experimented with trying to create plugins to suppress these warnings (http://docs.pylint.org/plugins.html#enter-plugin), but we're having difficulty making heads or tails of the docs -- and the astroid base class is uber-complex and has defied our attempts to grok it.

For real bonus points (and our eternal gratitude) we'd love to understand what changed in pylint. We'd be happy to fix the code (or at least publish a best practice document for C extension authors) that would satisfy pylint.

Platform details

$ pylint --version
No config file found, using default configuration
pylint 1.4.0,
astroid 1.3.2, common 0.63.2
Python 2.7.5 (default, Jul  1 2013, 18:09:11)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)]

score 75 · Accepted Answer · answered Feb 10 '15 at 21:25

75

Shortly after posting my question, I found the answer. The change was in fact done on purpose as a security measure. Pylint imports modules to effectively identify valid methods and attributes. It was decided that importing c extensions that are not part of the python stdlib is a security risk and could introduce malicious code.

This was done in the release of Astroid 1.3.1 https://mail.python.org/pipermail/code-quality/2014-November/000394.html

Only C extensions from trusted sources (the standard library) are loaded into the examining Python process to build an AST from the live module.

There are four solutions if you want to use pylint on projects that import non-stdlib c extensions.

1) Disable safety using the --unsafe-load-any-extension=y command line option. This feature is undocumented and classified as a hidden option (https://mail.python.org/pipermail/code-quality/2014-November/000439.html).

2) Disable safety using the pylint.rc setting unsafe-load-any-extensions=yes. This is recommended over option 1 and includes full documentation in the default pylint.rc file (created with --generate-rcfile).

3) Specifically list packages or modules names that you trust to be loaded by pylint in the pylint.rc file using the extension-pkg-whitelist= option.

4) Create a plugin to manipulate the AST (I have no idea how to effect this -- but it's regularly discussed on on the pylint mailing list).

We opted for Option 3. We added the following line to our project pylint.rc file:

extension-pkg-whitelist=lxml

answered Feb 10 '15 at 21:25

user590028

11,364
3
40
57

Thank you very much. I have to note, however, that option 3 unfortunately doesn't seem to work with some STL packages, e.g., `multiprocessing`. – 4ae1e1 Jun 10 '15 at 07:59
For #4, this documentation seems to directly address how to fix false positive `no-member`s - http://docs.pylint.org/plugins.html – Spain Train Jul 22 '15 at 02:34
Thanks for pointing that out. Updated link: https://pylint.readthedocs.io/en/latest/plugins.html – Spain Train Jul 11 '16 at 18:31
@SpainTrain sorry but link is dead again. – ibizaman Aug 30 '16 at 16:50
https://docs.pylint.org/en/1.6.0/plugins.html I think they no longer do the `latest` redirection. Thanks again for pointing out (again X-D). – Spain Train Aug 31 '16 at 18:49
Here's the latest link: https://docs.pylint.org/en/1.8/how_tos/transform_plugins.html – Fletch Feb 15 '18 at 06:01
Even better link to latest docs: https://pylint.readthedocs.io/en/latest/how_tos/transform_plugins.html – Fletch Feb 15 '18 at 06:07
See also: [How do I create a pylintrc file](https://stackoverflow.com/q/22448731/3357935) – Stevoisiak Mar 28 '18 at 21:08
2

It's now called `extension-pkg-allow-list` – Luciano Jul 19 '22 at 18:41

score 8 · Answer 2 · answered Feb 17 '15 at 23:58

8

@user590028, thanks a lot for your answer! I just ran into this same problem with the libraries win32api, win32evtlog, win32file, win32gui, and win32process, and your solution worked.

I used another method I think is worth posting here, which is to call pylint and pass the whitelisted packages as a parameter:

pylint --extension-pkg-whitelist=win32api,win32evtlog,win32file,win32gui,win32process myfile.py

answered Feb 17 '15 at 23:58

twasbrillig

17,084
9
43
67

1

Your quite welcome. fyi -- the `extension-pkg-whitelist` was option 3, and the same one we opted for. – user590028 Feb 18 '15 at 00:56
2

I just want to make sure people know you can pass it as a command-line parameter to pylint as well. – twasbrillig Feb 18 '15 at 05:31

score 5 · Answer 3 · answered Sep 10 '18 at 14:39

5

For those of you using VS Code, it's a bit tricky to find where to put the command as I couldn't find my executable.

In VS Code;

click on File > Preferences > Settings.
Scroll down to "Python Configurations" in the left window
scroll down to "Python Linting: Mypy Args" in the right window
click on "Edit in settings.json" link
edit the json to include: "--extension-pkg-whitelist="

I had to do all this because PyLint isn't executable from my Windows command line...

answered Sep 10 '18 at 14:39

BlakcRece

88
1
5

I was able to use the command line argument in VS Code for pylint by editing the user settings file to include this line: ```"python.linting.pylintArgs": ["--unsafe-load-any-extension=y"],``` – John Forbes Sep 12 '18 at 00:28

Fritz Lim · Answer 4 · 2019-02-25T12:42:13.973

1

If you're using VS Code for Mac, this is what you need to do in order to edit the settings.json file:

Click on Code (i.e. the Visual Studio Code tab which is on the left of the 'File' tab) -> Preferences - > Settings
Scroll down to Extensions and click on Python in the list.
Click on any of the Edit in settings.json links. This opens up settings.json for editing.
Add the line "python.linting.pylintArgs": ["----extension-pkg-whitelist=1xml"].

edited Feb 25 '19 at 12:42

answered Feb 25 '19 at 12:30

Fritz Lim

2,089
2
19
22

here, it goes to `{… "pylint.args": […, --extension-pkg-allow-list=posix_ipc ]}` – Alex Cohn Mar 27 '23 at 11:36

pylint 1.4 reports E1101(no-member) on all C extensions

4 Answers4

Linked