Variables from function to current file

Question

1. Summary

I can't find, how I can add variables from YAML files to Python files without duplicates.

---

2. Purpose

I use Pelican — static sites generator. It use .py files for configuration. Problems:

1. I can't reuse variables from .py for JavaScript
2. import * antipattern, that use even in official Pelican blog

I try move configuration to YAML files → I get problem of this question.

---

3. MCVE

3.1. Files

• Live demo on Repl.it

• main.py:

"""First Python file."""
# [INFO] Using ruamel.yaml — superset of PyYAML:
# https://stackoverflow.com/a/38922434/5951529
import ruamel.yaml as yaml

SETTINGS_FILES = ["kira.yaml", "kristina.yaml"]

for setting_file in SETTINGS_FILES:
    VARIABLES = yaml.safe_load(open(setting_file))
    # [INFO] Convert Python dictionary to variables:
    # https://stackoverflow.com/a/36059129/5951529
    locals().update(VARIABLES)

# [INFO] View all variables:
# https://stackoverflow.com/a/633134/5951529
print(dir())

• publishconf.py:

"""Second Python file."""
import ruamel.yaml as yaml

# [NOTE] Another value in list
SETTINGS_FILES = ["kira.yaml", "katya.yaml"]

for setting_file in SETTINGS_FILES:
    VARIABLES = yaml.load(open(setting_file))
    locals().update(VARIABLES)


print(dir())

• kira.yaml:

DECISION: Saint Petersburg

• kristina.yaml:

SPAIN: Marbella

• katya.yaml:

BURIED: Novoshakhtinsk

3.2. Expected behavior

• DECISION and SPAIN variables in main.py:

$ python main.py
['DECISION', 'SETTINGS_FILES', 'SPAIN', 'VARIABLES', '__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '__warningregistry__', 'setting_file', 'yaml']

• DECISION and BURIED variables in publishconf.py:

$ python publishconf.py
['BURIED', 'DECISION', 'SETTINGS_FILES', 'VARIABLES', '__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '__warningregistry__', 'setting_file', 'yaml']

3.3. Problem

Duplicate loop in main.py and publishconf.py:

for setting_file in SETTINGS_FILES:
    VARIABLES = yaml.load(open(setting_file))
    locals().update(VARIABLES)

Can I not use duplicates?

---

4. Not helped

4.1. Configuration file

• Live demo on Repl.it

• config.py:

"""Config Python file."""
# [INFO] Using ruamel.yaml — superset of PyYAML:
# https://stackoverflow.com/a/38922434/5951529
import ruamel.yaml as yaml

MAIN_CONFIG = ["kira.yaml", "kristina.yaml"]

PUBLISHCONF_CONFIG = ["kira.yaml", "katya.yaml"]


def kirafunction(pelicanplugins):
    """Function for both Python files."""
    for setting_file in pelicanplugins:
        # [INFO] Convert Python dictionary to variables:
        # https://stackoverflow.com/a/36059129/5951529
        variables = yaml.safe_load(open(setting_file))
        globals().update(variables)


def main_function():
    """For main.py."""
    kirafunction(MAIN_CONFIG)


def publishconf_function():
    """For publishconf.py."""
    kirafunction(PUBLISHCONF_CONFIG)

• main.py:

"""First Python file."""
import sys

from config import main_function

sys.path.append(".")


main_function()

# [INFO] View all variables:
# https://stackoverflow.com/a/633134/5951529
print(dir())

• publishconf.py:

"""Second Python file."""
import sys

from config import publishconf_function

sys.path.append(".")


publishconf_function()

print(dir())

Variables from main_function and publishconf_function doesn't share across files:

$ python main.py
['__annotations__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'main_function', 'signal', 'sys']

4.2. Another attempts

1. Wrap loop to function as in this example:

   def kirafunction():
       """Docstring."""
       for setting_file in SETTINGS_FILES:
           VARIABLES = yaml.safe_load(open(setting_file))
           locals().update(VARIABLES)
   
   
   kirafunction()
   

2. Using global keyword

3. “I think editing locals() like that is generally a bad idea. If you think globals() is a better alternative, think it twice!”

4. Search in Stack Overflow questions

Answer 1

I would avoid any update to what locals returns, because documentation explicitely states:

Note The contents of this dictionary should not be modified; changes may not affect the values of local and free variables used by the interpreter.

The globals function is a dictionnary simply containing the attributes of a module, and the mapping returned by globals is indeed writable.

So if this exists in one Python source:

def kirafunction(map,settings):
  # [NOTE] Another value in list
  for setting_file in settings:
    VARIABLES = yaml.load(open(setting_file))
    map.update(VARIABLES)

This can be used from any other Python source after importing the above function:

kirafunction(globals(), settings)

and will import the variables in the globals dictionnary of the calling module. And will be highly non-pythonic...

A slightly more Pythonic way, would be to dedicate one Python module to hold both the code loading the yaml files and the new variables:

loader.py:

import ruamel.yaml as yaml

SETTINGS_FILES = ["kira.yaml", "kristina.yaml"]

for setting_file in SETTINGS_FILES:
    VARIABLES = yaml.safe_load(open(setting_file))
    # [INFO] Convert Python dictionary to variables:
    # https://stackoverflow.com/a/36059129/5951529
    globals().update(VARIABLES)

Then from any other Python module you can use:

...
import loader      # assuming that it is in sys.path...
...
print(loader.DECISION)
print(dir(loader))

But it is still uncommon and would require comments to explain the rationale for it.

---

After reading the Pelican config example from you comment, I assume that what you need is a way to import in different scripts a bunch of variables declared in yaml files. In that case I would put the code loading the variables in one module, and update the globals() dictionnary in the other modules:

loader.py:

import ruamel.yaml as yaml

MAIN_CONFIG = ["kira.yaml", "kristina.yaml"]

PUBLISHCONF_CONFIG = ["kira.yaml", "katya.yaml"]


def kirafunction(pelicanplugins):
    """Function for both Python files."""
    variables = {}
    for setting_file in pelicanplugins:
        # [INFO] Convert Python dictionary to variables:
        # https://stackoverflow.com/a/36059129/5951529
        variables.update(yaml.safe_load(open(setting_file)))
    return variables

Then for example in publishconf.py you would use:

from loader import kirafunction, PUBLISHCONF_CONFIG as pelican_config

# other Python code...

# import variables from the yaml files defined in PUBLISHCONF_CONFIG
#  because Pelican expects them as plain Python module variables
globals().update(kirafunction(pelican_config))

Again, updating globals() is probably appropriate in this use case, but is generally frowned upon, hence the comment.