0

I'm trying to run a jupyter notebook file called Seg_pivot.ipynb from main.ipynb. The main.ipynb has one line of code: %run "fi/Seg_pivot.ipynb"

The current directory structure looks like the following: enter image description here

The Seg_pivot.ipynb runs a module in the MyMacro, used to style the files in the result folder. When I run the Seg_pivot.ipynb everything works. But when I run the main.ipynb, it gives a FileNotFound error: No such file: 'mymacro.xlsm'

In the Seg_pivot.ipynb this is how I call the macro:

import xlwings as xw
wb = xw.Book("MyMacro.xlsm")
s_macro = wb.macro("Module2.styleF")
s_macro()
M J
  • 379
  • 2
  • 8
  • Are you setting the working directory inside `Seg_pivot.ipynb` using the magic command `%cd` so that it works in both places? Without seeing more code, I'd guess you are running `Seg_pivot.ipynb` as itself and seeing it work because the working directory then is alongside `Seg_pivot.ipynb` so the running code sees the macro as that is with it. When you run it from `main.ipynb`, the directory where `main.ipynb` resides is the working directory by default. The macro isn't there. – Wayne Mar 13 '23 at 20:51
  • No, I'm not using that magic command. Can you show me how to use it? – M J Mar 13 '23 at 20:53

1 Answers1

1

I think putting inside your notebook Seg_pivot.ipynb near the top, the following may help:

import pathlib
%cd {pathlib.Path(__file__).parent.resolve()}

That will always set the working directory to where Seg_pivot.ipynb is in your file hierarchy when it runs.

Getting he path to the notebook file something is running it is based on here. The magic %cd command is documented here.

In your case, you may wish to reset the current working directory back in main.ipynb, too, i.e., after it runs the Seg_pivot.ipynb notebook if main.ipynb involves any paths after that.

Wayne
  • 6,607
  • 8
  • 36
  • 93
  • And it still allows `Seg_pivot.ipynb` to work when you run it directly? I was using a simple test but just was wondering if there was more complexity. – Wayne Mar 13 '23 at 21:00
  • It is working somehow, I get an output saying: ```[WinError 2] The system cannot find the file specified; '{pathlib.Path(__file__).parent.resolve()}'``` – M J Mar 13 '23 at 21:03
  • 1
    Oh so, it still works **both ways** but you just need to ignore that? Probably easier to just leave it then unless you are doing some fancy produciton thing? I think the cd fails when you run `Seg_pivot.ipynb` directly, but because current working directory will be already correct, not an issue. We could add a conditional but probably easier just to ignore the error if it continues running. – Wayne Mar 13 '23 at 21:12
  • After I run the ```main.ipynb``` code, it sets the working directory to the 'fi' folder where the ```Seg_pivot.ipynb``` file is stored. How to return to the directory where the ```main.ipynb``` stored once ```Seg_pivot.ipynb``` is executed? – M J Mar 13 '23 at 21:30
  • I tried adding the following at the end of the ```Seg_pivot.ipynb``` script: ```%cd {pathlib.Path(__file__).parent}``` . Refrence: https://stackoverflow.com/questions/30218802/get-parent-of-current-directory-from-python-script – M J Mar 13 '23 at 22:08
  • Adding the following at the end of the ```Seg_pivot.ipynb``` script works: ```%cd {pathlib.Path().resolve().parent}``` . Reference: https://stackoverflow.com/questions/30218802/get-parent-of-current-directory-from-python-script – M J Mar 13 '23 at 22:15
  • Yes, your last three comments refer to why I said, "In your case, you may wish to reset the current working directory back in `main.ipynb`, too". I would suggest it makes more sense to be a step in `main.ipynb` after it runs the other notebook from `main.ipynb`. But if it works in `Seg_pivot.ipynb`, that is fine. As long as a few months from now, you can tell how to run both notebooks clearly, it shouldn't be a big deal. There's other ways to do it that would be more thorough, such as add a flag that you send to `Seg_pivot.ipynb` when calling it from `main.ipynb`, and act aprropriately. – Wayne Mar 14 '23 at 13:49
  • Alternatively, if what `Seg_pivot.ipynb` does could be better modularized, there's ways restructure it to import it using [importnb](https://discourse.jupyter.org/t/package-for-importing-jupyter-notebooks-as-modules/12923/2?u=fomightez). Or restructure the main things it does to a script that both `main.ipynb` and `Seg_pivot.ipyn` can use. The main point is that now you see what the original issue was and that there's ways to address it. – Wayne Mar 14 '23 at 13:53
  • Extending on the above. I wanted to pass/import a string variable from the parent notebook ```main.ipynb``` to the child notebook ```Seg_pivot.ipynb```. How to do that? – M J Mar 15 '23 at 21:50
  • If I recall correctly, when you execute `%run "fi/Seg_pivot.ipynb"` in `main.ipynb`, it is running the notebook in the namespace of `main.ipynb`. I think I'm right and in fact that is why the `%cd` was needed. (When running scripts you actually need the specific `-i` flag for `interactively` to run the script in the current namespace instead of an empty one.) So when you execute that run command to run `Seg_pivot.ipynb` anything you already defined earlier in `main.ipynb` is useable by `Seg_pivot.ipynb`. You can easily test this separately from your complex notebooks by making two really .... – Wayne Mar 16 '23 at 02:19
  • simple ones that parallel yours and test if you can print a variable define from in the invoking one in the one you call with `%run`. So the hard part is already handled. But how do you keep `Seg_pivot.ipynb` working alone becomes the question, right? Because if I recall you also like to be able to run `Seg_pivot.ipynb` separately on its own? So inside of `Seg_pivot.ipynb` you put a `try ... except` block where you check if the variable you'll sometimes be passing in from `main` is of the type you expect it to be and you set the `except` to set a default dummy value if there's a... – Wayne Mar 16 '23 at 02:25
  • `NameError`. For example, set the variable to a strange string, like "iDIDnotComeFromMAiN". And you add an `if`conditional in `Seg_pivot.ipynb` to do the thing it should do when it gets something from `main`. Specifically, the pseudocode would be something like `if != "iDIDnotComeFromMAiN", then do the thing it should do when passed in and either handle everything else with an `else` clause or just don't have anything else besides the thing to do when it comes from `main.ipynb`. If it was only one thing to do, you can build the entire thing in the `try...except`. ... – Wayne Mar 16 '23 at 02:33
  • Setting a flag using the `try .. except` is just easier, can be made to be more explicit as to what is happening, and gives you more flexibility to do something different again later in the notebook, depending on whether the variable is passed in or not from `main.ipynb` – Wayne Mar 16 '23 at 02:35
  • What I said in my first comment about passing things is the long way of saying [this answer](https://stackoverflow.com/a/64697171/8508004). And there are other ways if you had more complexity involved, such as [this](https://felixchenier.uqam.ca/a-better-way-to-run-a-jupyter-notebook-with-arguments/) or use [Parpermill](https://github.com/nteract/papermill#) to execute the other notebook from `main.ipynb`. In fact, because Papermill allows you to set a default and then overwrite it when injecting, it is a more civilized way to handle what I was getting at with the `try..except` in `Seg_pivot`. – Wayne Mar 16 '23 at 02:57
  • 1
    Thanks, Wayne. This has been very helpful! – M J Mar 16 '23 at 17:54
  • 1
    [This](https://stackoverflow.com/a/75803877/8508004) may be helpful, too. – Wayne Mar 21 '23 at 17:54