328

I just switched to PyCharm and I am very happy about all the warnings and hints it provides me to improve my code. Except for this one which I don't understand:

This inspection detects shadowing names defined in outer scopes.

I know it is bad practice to access variable from the outer scope, but what is the problem with shadowing the outer scope?

Here is one example, where PyCharm gives me the warning message:

data = [4, 5, 6]

def print_data(data): # <-- Warning: "Shadows 'data' from outer scope
    print data

print_data(data)
TylerH
  • 20,799
  • 66
  • 75
  • 101
Framester
  • 33,341
  • 51
  • 130
  • 192
  • 1
    Also I searched for the string "This inspection detects..." but found nothing in the pycharm online help: http://www.jetbrains.com/pycharm/webhelp/getting-help.html – Framester Nov 21 '13 at 15:39
  • 3
    To turn off this message in PyCharm: **++s** (settings), **Editor**, **Inspections**, "**Shadowing names from outer scopes**". Uncheck. – ChaimG Jan 23 '19 at 17:33
  • I got the same warning and see nothing wrong with the code. The function is using the local parameter "data", not the global "data". Of course if you rename the parameter, you'll have to rename its occurrences in the function scope. Not sure what the author of the plugin thinks. It's like he's never used IDE's Code Refactoring. – emeraldhieu Jan 12 '23 at 07:28
  • As of PyCharm 2023, the path to turn of the Shadows warning is ***File -> Settings -> Editor -> Inspections -> Shadowing names from outer scopes*** – user3785010 Aug 03 '23 at 17:42

11 Answers11

315

There isn't any big deal in your above snippet, but imagine a function with a few more arguments and quite a few more lines of code. Then you decide to rename your data argument as yadda, but miss one of the places it is used in the function's body... Now data refers to the global, and you start having weird behaviour - where you would have a much more obvious NameError if you didn't have a global name data.

Also remember that in Python everything is an object (including modules, classes and functions), so there's no distinct namespaces for functions, modules or classes. Another scenario is that you import function foo at the top of your module, and use it somewhere in your function body. Then you add a new argument to your function and named it - bad luck - foo.

Finally, built-in functions and types also live in the same namespace and can be shadowed the same way.

None of this is much of a problem if you have short functions, good naming and a decent unit test coverage, but well, sometimes you have to maintain less than perfect code and being warned about such possible issues might help.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
bruno desthuilliers
  • 75,974
  • 6
  • 88
  • 118
  • 43
    Fortunately PyCharm (as used by the OP) has a very nice rename operation that renames the variable everywhere it is used in the same scope, which makes renaming errors less likely. – wojtow Apr 06 '17 at 00:29
  • 3
    In addition to PyCharm's renaming operation I would love to have special syntax highlights for variables that refer to outer scope. These two should render this time consuming shadow resolution game irrelevant. – Leo Jul 25 '19 at 13:46
  • Side note: You can use the `nonlocal` keyword to make outer score referring (like in closures) explicit. Note that this is different from shadowing, as it explicitly does not shadow variables from outside. – Felix D. Oct 07 '19 at 08:21
  • I believe this is not the right answer and does not propose a solution. I believe this should be the answer: https://stackoverflow.com/a/40008745/2424587 – Hanan Shteingart Jan 25 '22 at 15:50
  • @HananShteingart I already commented on why what you believe should be the answer is not, and it's also part of my own answer. – bruno desthuilliers Jan 27 '22 at 17:59
261

The currently most up-voted and accepted answer and most answers here miss the point.

It doesn't matter how long your function is, or how you name your variable descriptively (to hopefully minimize the chance of potential name collision).

The fact that your function's local variable or its parameter happens to share a name in the global scope is completely irrelevant. And in fact, no matter how carefully you choose you local variable name, your function can never foresee "whether my cool name yadda will also be used as a global variable in future?". The solution? Simply don't worry about that! The correct mindset is to design your function to consume input from and only from its parameters in signature. That way you don't need to care what is (or will be) in global scope, and then shadowing becomes not an issue at all.

In other words, the shadowing problem only matters when your function need to use the same name local variable and the global variable. But you should avoid such design in the first place. The OP's code does not really have such design problem. It is just that PyCharm is not smart enough and it gives out a warning just in case. So, just to make PyCharm happy, and also make our code clean, see this solution quoting from silyevsk's answer to remove the global variable completely.

def print_data(data):
    print data

def main():
    data = [4, 5, 6]
    print_data(data)

main()

This is the proper way to "solve" this problem, by fixing/removing your global thing, not adjusting your current local function.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
RayLuo
  • 17,257
  • 6
  • 88
  • 73
  • 19
    Well, sure, in a perfect world, you enver make a typo, or forget one of your search-replace when you change the parameter, but mistakes happens and that's what PyCharm is saying - "Warning - nothing is technically in error, but this could easily become a problem" – dwanderson Feb 22 '17 at 02:19
  • 1
    @dwanderson The situation you mentioned is nothing new, it is clearly described in the currently chosen answer. However, the point I try to make, is that we should avoid global variable, not avoid shadowing global variable. The latter misses the point. Get it? Got it? – RayLuo Feb 22 '17 at 04:44
  • 7
    I wholefully agree on the fact that functions should be as "pure" as possible but you totally miss the two important points: there's no way to restrict Python from looking up a name in the enclosing scopes if it's not locally defined, and _everything_ (modules, functions, classes etc) is an object and lives in the same namespace as any other "variable". In your above snippet, `print_data` IS a global variable. Think about it... – bruno desthuilliers Mar 17 '17 at 09:07
  • I know referencing non-local variables is a feature in Python, but for those who want to program in this answer's way (like me), it would never be a good idea to reference outer scopes. Therefore, the warning should not be 'This name shadows that name' for the same names in different scopes, but the warning could be issued for any variable referencing an outer scope. It's a warning, not an error, so still perfectly legal and valid at times. Or am I missing a different point? – Erik Feb 15 '18 at 06:33
  • 2
    I ended up on this thread because I'm using functions defined in functions, to make the outer function more readable without cluttering the global namespace or heavy-handedly using separate files. This example here doesn't apply to that general case, of non-local non-global variables being shadowed. – micseydel Jul 02 '18 at 22:39
  • 3
    Agree. The problem here is Python scoping. Non explicit access to objects outside the current scope is asking for trouble. Who would want that! A shame because otherwise Python is a pretty well thought out language (not withstanding a similar ambiguity in module naming). – CodeCabbie May 12 '20 at 10:21
  • Neat, but: I like to debug my scripts by running it in Python Console within Pycharm, allowing me to access the variables from the global scope after the script terminates. After shifting all into function main(), the variables won't be readily available anymore. Any simple solution to that? – FlorianH May 27 '20 at 07:29
  • 2
    @florianH , I do not use PyCharm, perhaps you can somehow set a breakpoint at the end of main()? – RayLuo Jun 05 '20 at 16:13
  • 2
    @RayLuo, you don't know how much your small hint will help me throughout the coming years of my Py coding. Embarrassed & happy :-). Note to self (and if any others..): **Breakpoint & Debugger** to live interact with code at any point of interest; never again annoyingly write tons of data printing commands to check what's going on in the code. If others cannot get the Pycharm prompt to open: after debugger process connected, find the colorful Pycharm cross at the very bottom right of the debugger window (may first be hidden --> click the tiny arrows >>) – FlorianH Jun 06 '20 at 19:58
  • @florianH , i put a pass row at the end and then put a breakpoint on the pass line. Pass does not do anything. – Remko van Hierden Nov 29 '20 at 09:21
  • I find it irritating that local variable names shadow _modules_ that I import or from which I import, like `from settings import Settings` followed by a function with `settings = Settings()`... I can't 'improve' my code by removing the import. Based on this answer (which I tend to agree with) the only option is to disable the pycharm warning – Rhubarb Aug 29 '23 at 20:43
61

A good workaround in some cases may be to move the variables and code to another function:

def print_data(data):
    print data

def main():
    data = [4, 5, 6]
    print_data(data)

main()
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
silyevsk
  • 4,021
  • 3
  • 31
  • 30
6

I like to see a green tick in the top right corner in PyCharm. I append the variable names with an underscore just to clear this warning so I can focus on the important warnings.

data = [4, 5, 6]

def print_data(data_):
    print(data_)

print_data(data)
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Baz
  • 12,713
  • 38
  • 145
  • 268
5

It depends how long the function is. The longer the function, the greater the chance that someone modifying it in future will write data thinking that it means the global. In fact, it means the local, but because the function is so long, it's not obvious to them that there exists a local with that name.

For your example function, I think that shadowing the global is not bad at all.

Steve Jessop
  • 273,490
  • 39
  • 460
  • 699
3

It looks like it is 100% a pytest code pattern.

See:

pytest fixtures: explicit, modular, scalable

I had the same problem with it, and this is why I found this post ;)

# ./tests/test_twitter1.py
import os
import pytest

from mylib import db
# ...

@pytest.fixture
def twitter():
    twitter_ = db.Twitter()
    twitter_._debug = True
    return twitter_

@pytest.mark.parametrize("query,expected", [
    ("BANCO PROVINCIAL", 8),
    ("name", 6),
    ("castlabs", 42),
])
def test_search(twitter: db.Twitter, query: str, expected: int):

    for query in queries:
        res = twitter.search(query)
        print(res)
        assert res

And it will warn with This inspection detects shadowing names defined in outer scopes.

To fix that, just move your twitter fixture into ./tests/conftest.py

# ./tests/conftest.py
import pytest

from syntropy import db


@pytest.fixture
def twitter():
    twitter_ = db.Twitter()
    twitter_._debug = True
    return twitter_

And remove the twitter fixture, like in ./tests/test_twitter2.py:

# ./tests/test_twitter2.py
import os
import pytest

from mylib import db
# ...

@pytest.mark.parametrize("query,expected", [
    ("BANCO PROVINCIAL", 8),
    ("name", 6),
    ("castlabs", 42),
])
def test_search(twitter: db.Twitter, query: str, expected: int):

    for query in queries:
        res = twitter.search(query)
        print(res)
        assert res

This will be make happy for QA, PyCharm and everyone.

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Andrei.Danciuc
  • 1,000
  • 10
  • 24
2
data = [4, 5, 6] # Your global variable

def print_data(data): # <-- Pass in a parameter called "data"
    print data  # <-- Note: You can access global variable inside your function, BUT for now, which is which? the parameter or the global variable? Confused, huh?

print_data(data)
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
JoeC
  • 1,850
  • 14
  • 12
  • 60
    I for one am not confused. It's pretty obviously the parameter. –  Nov 21 '13 at 15:42
  • 2
    @delnan You may not be confused in this trivial example, but what if other functions defined nearby used the global `data`, all deep within a few hundred lines of code? – John Colanduoni Nov 21 '13 at 15:48
  • 15
    @HevyLight I don't need to look at other functions nearby. I look at this function only and can see that `data` is a local name in *this* function, so I don't even bother checking/remembering whether a global of the same name *exists*, let alone what it contains. –  Nov 21 '13 at 15:50
  • 4
    I don't think this reasoning is valid, solely because to use a global, you would need to define "global data" inside of the function. Otherwise, the global is not accessible. – CodyF Feb 15 '15 at 19:38
  • 3
    @CodyF `False` - if you don't define, but just try to use `data`, it looks up through scopes until it finds one, so it *does* find the global `data`. `data = [1, 2, 3]; def foo(): print(data); foo()` – dwanderson Feb 22 '17 at 02:21
2

I think this rule doesn't help much. I simply disabled it by going to Settings -> Editor -> Inspections and then checking off this rule: Shadowing names from outer scope

wjandrea
  • 28,235
  • 9
  • 60
  • 81
David Lasry
  • 829
  • 3
  • 12
  • 31
1

Do this:

data = [4, 5, 6]

def print_data():
    global data
    print(data)

print_data()
0

To ignore the warning, as Chistopher said in a comment, you can comment above it

# noinspection PyShadowingNames
Joshua Wolff
  • 2,687
  • 1
  • 25
  • 42
  • Ignoring a warning just because it's annoying doesn't really answer the OPs question, which is really - what is the issue causing this, not simply "i don't like seeing a warning" – Michael Tuchman Aug 02 '23 at 04:03
  • @MichaelTuchman Here's how StackOverflow works: If the OP's question was already given a sufficient answer, such as in this case, it's useful to post answers that other people might find useful. – Joshua Wolff Aug 02 '23 at 15:38
  • thank you for sharing your point of view. Mine is that showing people how to ignore warnings rather than face the issue causing them is rarely useful. But you do you. – Michael Tuchman Aug 03 '23 at 00:48
  • 1
    @MichaelTuchman It's useful because PyCharm, as good as it is, has several bugs where it mistakenly shows a warning. Unless you want to suppress all warnings of that category, the only solution is to ignore it per line, or per class, etc.. So it's not only obviously useful, I use it every day because it's necessary. I've also posted similar answers of such a nature and have received many upvotes (clearly useful). – Joshua Wolff Aug 03 '23 at 21:10
  • I'll accept that. I removed the negative vote. However, I do see way too much of this "how to ignore it" genre though. @Joshua Wolff – Michael Tuchman Aug 04 '23 at 01:35
  • Ok thank you, you didn't have to @MichaelTuchman – Joshua Wolff Aug 04 '23 at 04:14
0

It may also be a subtle warning that your function is being defined as in internal function to another, and you did not intend this. Perhaps it's because of bad indenting, which is in turn caused by a lack of caffeine. In other words, you could see this message as reading " your scope is not what you think it is, young Padawan"

Michael Tuchman
  • 332
  • 3
  • 12