0

I want in my makefile to retrieve all textfiles (at the moment i just got one) from a certain subfolder and call in a loop a specific python script with each text file as an input parameter.

This is the code i currently have:

run_analysis:
    @echo "Get text files"
    txt_files=$(wildcard ./input/*.txt)
    @echo "Current text files are:"
    @echo $(txt_files)
    for txt_file in $(txt_files); do \
        @echo "Iteration" \
        @echo $(txt_file ) \
        python ./scripts/my_test_script.py $(txt_file ) ; \
    done

It seems the wildcard results are not stored in the variable.

My output looks the following: 

Get text files
txt_files=./input/test_text_1.txt
Current text files are:

for txt_file in ; do \
    @echo "Iteration" \
    @echo  \
    python ./scripts/my_test_script.py  ; \
done
Stu Pitt
  • 45
  • 1
  • 8

1 Answers1

0

Each line in a Makefile recipe is executed in a separate shell instance by default.

Saving the files in a variable doesn't appear to serve any useful purpose anyway. Just inline the wildcard.

run_analysis:
    for txt_file in ./input/*.txt; do \
        python ./scripts/my_test_script.py "$$txt_file"; \
    done

(Notice also how txt_file is a shell variable, not a Make variable.)

Better yet, change your Python script so it accepts a list of input files.

run_analysis:
    python ./scripts/my_test_script.py ./input/*.txt

Maybe add incessant chatter with logging.debug() inside the Python script if you want to see exactly what it's doing. Unlike hard-coded echo, logging can easily be turned off once you are confident that your code works.

tripleee
  • 175,061
  • 34
  • 275
  • 318
  • I tried your suggested answer and my python script is executing. However, the passed parameter is empty. Since the input parameter is empty my check of `if passed_file.split(".")[-1] != "txt"` evalutes to true. EDIT: Hold on a second i might have a typo in my makefile....will update shortly. – Stu Pitt Feb 08 '19 at 13:55
  • Thanks. Your solution works fine, but can you elaborate on "$$txt_file"? I assume since it is a shell variable i need to escape the $ with another $ but why the quotation marks? – Stu Pitt Feb 08 '19 at 14:22
  • We basically always quote shell variables which contain file names. https://stackoverflow.com/questions/10067266/when-to-wrap-quotes-around-a-shell-variable – tripleee Feb 08 '19 at 14:26