1

I want to execute a command in cmd to run Matlab in -nodesktop mode (so without gui). The Matlab program that I will run will create a .txt file that later in the same script pandas is going to parse. But on my Windows 10 (on Linux it works), pandas doesn't wait for the command to finish and tries to parse an empty file which results in this error:

pandas.errors.EmptyDataError: No columns to parse from file

This is the command that I run (with a couple (correct) function calls later in Matlab:

matlab -nodesktop -r

The whole string of commands is then run like this:

os.system(COMMAND_START)

A few lines later I try to parse the file with pandas, but it doesn't wait for the os.system() to finish, so right after the Matlab command starts (and it takes quite a long time for it to finish) pandas wants to parse an empty file. How can I make my script wait for the os.system() to finish?

df = pd.read_csv("stabs.txt", header=None)
STABS_KG = df[0].to_list()
STABS_1_KG = df[1].to_list()
John
  • 197
  • 1
  • 3
  • 14
  • 1
    use module `subprocess` instead of `os.system` – furas May 23 '20 at 10:26
  • I tried but I think I dont know how to correctly. Could you give me an example? – John May 23 '20 at 10:26
  • did you check if this file is really created ? Maybe matlab doesn't create empty file or it create file in different folder and pandas reads different file (which is always empty). – furas May 23 '20 at 10:29
  • In my python script first I delete all files with the same name (it makes sense for my program to do so) then I create (in python) an empty `.txt` with that exact name and close it. Then I run my `Matlab` script which will append data to the file, but that will take time as my system has to start up `Matlab` which takes time. So the pandas part run right after me calling `Matlab` from my script and for that pandas command the `.txt` file is still empty! – John May 23 '20 at 10:34
  • using Google I found on Stackoverflow [wait process until all subprocess finish](https://stackoverflow.com/questions/15107714/wait-process-until-all-subprocess-finish) – furas May 23 '20 at 10:36
  • why do you create empty file in `Python`? Maybe `Matlab` doesn't use this file and finally you have only empty file. To make sure `Matlab` should create this file (not `Python`) and you should use `/full/path/to/file.txt` in `Matlab` and in `Python` – furas May 23 '20 at 10:43
  • It does and it uses this file. The problem is that the python script doesn't wait and it takes a while for matlab to start and execute its functions, but for python to jump to the next line where I want to run padnas its just a split second – John May 23 '20 at 10:48

1 Answers1

0

If you don't want to complicate with the subprocess module and you have an estimate for the time it takes to finish, you could simply add a sleep(seconds) after the call:

os.system(COMMAND_START)
sleep(2) -> wait 2 seconds

You can also use the subprocess module:

import subprocess
process = subprocess.Popen(['COMMAND_START'])
exitCode=process.wait()
CanciuCostin
  • 1,773
  • 1
  • 10
  • 25
  • When I run: process = subprocess.Popen([COMMAND_START]) exitCode=process.wait() df = pd.read_csv("stabs.txt", header=None) STABS_KG = df[0].to_list() STABS_1_KG = df[1].to_list() – John May 23 '20 at 10:36
  • It gives me this error: Traceback (most recent call last): File "main.py", line 113, in process = subprocess.Popen([COMMAND_START]) File "C:\Program Files\Python38\lib\subprocess.py", line 854, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Program Files\Python38\lib\subprocess.py", line 1307, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, FileNotFoundError: [WinError 2] The system cannot find the file specified – John May 23 '20 at 10:37
  • 3
    @John you have to put your command as list and you could also use full path to `matlab` - ie `Popen(["c:\\full\\path\\to\\matlab", "-nodesktop", "-r"],...)` – furas May 23 '20 at 10:39
  • 1
    @John you have to replace COMMAND_START with your actual command arguments. – CanciuCostin May 23 '20 at 10:42
  • Okay I will try this in a few minutes – John May 23 '20 at 10:49
  • Running this: `import subprocess matlab_path = "\"C:\\Program Files\\MATLAB\\R2018a\\bin\\matlab.exe\"" matlab_function = "\"nyquist_plot_with_k("+str(repr([1]))+","+str( repr([1, 2, 3]))+"," + "\'"+"a"+"\');exit;\"" process = subprocess.Popen([matlab_path, " -nodesktop", " -r ", matlab_function ]) exitCode = process.wait()` – John May 23 '20 at 11:01
  • Gives me this error: `Traceback (most recent call last): File "test.py", line 5, in process = subprocess.Popen([matlab_path, " -nodesktop", " -r ", matlab_function ]) File "C:\Program Files\Python38\lib\subprocess.py", line 854, in __init__ self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Program Files\Python38\lib\subprocess.py", line 1307, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, PermissionError: [WinError 5] Access is denied ` – John May 23 '20 at 11:01
  • I think you should remove the double quotes inside your paths – CanciuCostin May 23 '20 at 11:07
  • I run this: `import subprocess matlab_path = "C:\\Program Files\\MATLAB\\R2018a\\bin\\matlab.exe" matlab_function = "\"nyquist_plot_with_k("+str(repr([1]))+","+str( repr([1, 2, 3]))+"," + "\'"+"a"+"\');exit;\"" process = subprocess.Popen([matlab_path, " -nodesktop", " -r ", matlab_function ]) exitCode = process.wait() print("FINISH")` And not only it prints out `FINISH` before matlab even starts (so padnas would try to parse that file at this point and crash because matlab would have to to write it) but also matlab doesn't start in `-nodesktop` mode :( – John May 23 '20 at 11:16