Is this a name-value binding issue during module load time by python interpreter?

Question

I wrote a small program in sample.py file,

a=6
def f1():
    a=3
    return f2()

#def f2(): commented intentionally
#   b = a*a
#   return b

and loaded as __main__ module using command

>>>python -i sample.py

But i see that interpreter does not check the binding of f2 during loading of module.

Interpreter only realises that f2 name is not binded to its value when i call

>>>f1()

Another example, where interpreter checks the binding of name big1 , while loading the file as __main__ module using command >>>python -i sample.py throws an error saying big1 is not defined

big, small =  3, -4
while big > 2 or small < -2:
    big, small = -small - 1, -big + 1
    print(big1)

My question is:

1) Why python interpreter does not take an effort to check whether all names are bind during loading of sample.py file as __main__ module and defer to using that name f1()?

2) Do you think large projects will create complexity, Because the test cases written must make sure each name is used for testing before it goes for production?

possible duplicate of [Python: Name resolution; order of function def's](http://stackoverflow.com/questions/4937532/python-name-resolution-order-of-function-defs) — dawg, Apr 02 '14 at 15:03
dawg your existing query just telling you that python check name binding while calling it, which i already mentioned in query. But my question is why ptyhon interp does like this and impact on large projects — overexchange, Apr 02 '14 at 15:06
Since you can bind names inside `if` blocks, Python can't know whether a particular name will be bound at some point in the code, unless it evaluates all possible combinations of every conditional beforehand. This quickly becomes intractable for programs that aren't trivially small. — Kevin, Apr 02 '14 at 15:12
dawg i would request you to remove that heading above my query. Please let me know the reason why it is a duplicate query, here the discussion on the impact of large projects? — overexchange, Apr 02 '14 at 15:18
@alecbenzer, I suppose so. We know that `if some_function_that_loops_forever(): f2()` won't ever crash even if f2 isn't bound to anything. But the interpreter would have to solve the halting problem if it wanted to tell you that before you ran the program. — Kevin, Apr 02 '14 at 15:40

alecbz · Answer 1 · 2014-04-09T19:09:36.583

4

Consider this example:

def f1():
   name = raw_input()
   exec "def {}(): print 'I am a function'".format(name)
   f2()

If a user enters "f2" when prompted, then f2() will execute correctly, but not otherwise.

This is a perfectly legal python program, and the line f2() may or may not execute correctly, dependent completely on user input. So python cannot determine at module load time whether execution of this code will result in a name error or not.

Addressing some additional confusion:

It might seem that python does sometimes do static name-checking. Ie, if we have file1.py:

def foo():
    print x1

and you do python file1.py, no error will be printed. The function foo has been defined, but not run. We could also do python -i file1.py, which will open up an interpreter. If in the interpreter we then type foo(), now the code will run, and we'll get the lookup error for x1.

Consider a second file file2.py:

print x1

Here, there is no function being defined. We're simply running a print statement at the top-level. Doing python file2.py will cause a lookup error for x1, since running file2.py constitutes actually running that print statement, as opposed to only defining a function that would run the print statement when called.

So it's not that python sometimes does static name-checking -- python will only throw a name error when code involving that name is actually run. It's just that when code gets run depends on where it is (in a function vs being top-level).

edited Apr 09 '14 at 19:09

answered Apr 02 '14 at 15:46

alecbz

6,292
4
30
50

Code like this is horrifying. – Eric Wilson Apr 02 '14 at 15:49
@EricWilson Sure is, but if python wants to keep stuff like `exec` around, it means static-checking of modules in the way OP suggests is impossible. Also, I can imagine this having uses. I imagine [patching](https://docs.python.org/3/library/unittest.mock.html#the-patchers) might do something similar, for instance. – alecbz Apr 02 '14 at 15:51
@alecbenzer DO you think i need to every function like this when i write my application? as you mentioned in your proposed answer? – overexchange Apr 03 '14 at 05:03
@Sham Sorry, not sure what you're asking? Do you need to *what* every function like this? I was just giving an example of legitimate python code where static checking isn't possible. – alecbz Apr 03 '14 at 19:20
@alecbenzer static checking can be possible, if declaration of symbol is done, before using that symbol. Why python doesn't enforce that? f2 should be declared/defined before using it. what advantage does python give by relaxing this rule? Above problem is a serious problem, because i would be uncomfortable thinking of big projects in python. – overexchange Apr 04 '14 at 00:49
1

@Sham Static checking would not be possible. The example in my answer might work even though there is no declaration of `f2` before its use (at least not one that python is aware of when it loads the module). So python would be incorrect in rejecting such a program. – alecbz Apr 04 '14 at 03:18
@alecbenzer it would take more time for me to understand this syntax, i would not encourage somebody to write such bad readable code---exec "def {}(): print 'I am a function'".format(name) – overexchange Apr 04 '14 at 05:39
@Sham I wouldn't either, but the fact that it and other things like it are possible means that python can't do at-module-load static checking. – alecbz Apr 04 '14 at 14:30
@alecbenzer Is python author not considering this as an issue? – overexchange Apr 04 '14 at 16:54
@Sham I doubt that it hasn't been brought to the attention of someone in the dev community before. There's potentially a trade-off between the benefits that this kind of thing can offer (eg, patching, like I mentioned above) and its downsides. Python has chosen to go with the benefits and accept the downside. If you don't think that was a good trade, a language other than python may be more appropriate for you. – alecbz Apr 04 '14 at 18:12
@alecbenzer Agreed. But i have added another example in my query, where interpreter throws an error saying PS C:\mystuff> python -i .\sample.py Traceback (most recent call last): File ".\sample.py", line 6, in print(big1) NameError: name 'big1' is not defined >>>----------so some places interpreter is checking the name binding value during load time. – overexchange Apr 07 '14 at 04:49
@alecbenzer can u add your statement as part of the answer? 'it hasn't been brought to the attention of someone in the dev community before'---currently it is part of your comment – overexchange Apr 07 '14 at 04:52
@Sham In your second example, the code is actually being *executed*, as its in the top-level of the module. This is why you get the error immediately. In your first example the code was in a function. Also I said "*I doubt that it hasn't* been brought to the attention..." Ie, I would think it *has* been brought to someone's attention. – alecbz Apr 07 '14 at 07:00
@alecbenzer when u say top level code is executed, do u mean, when i say 'python -i sample.py' this file is first compiled to bytecode and loaded as module in RAM and then top level code starts executing ? Is that what you are saying? – overexchange Apr 07 '14 at 07:51
@Sham More or less, yes. If you include a top-level `print`, you should see it execute when you import the module. – alecbz Apr 07 '14 at 17:17
@alecbenzer as you mentioned exec will resolve this problem. What does exec do for this given line in your answer? 'exec "def {}(): print 'I am a function'".format(name)' How exec is different from importing a method from a module/ – overexchange Apr 08 '14 at 07:33
@Sham I'm not sure what problem you're referring to that exec resolves. exec is not really different from importing a module. In each case the interpreter will "run" the code you give it. However, "running" a `def` block simply causes a new function to be defined -- it doesn't run the _body_ of that function. – alecbz Apr 08 '14 at 19:57
@alecbenzer if def block does not run the body of the function during loading of module, how exec will get interpreted during loading of module and raise error? – overexchange Apr 09 '14 at 02:34
1

@Sham It won't. I think you're misunderstanding my answer. I'm not explaining how you can cause name errors to happen at module load time -- I'm giving you an example of a legal python program for which module load time name checking is _not possible_. – alecbz Apr 09 '14 at 19:07

Is this a name-value binding issue during module load time by python interpreter?

1 Answers1