0

I have a script called test.py that I am running in a step function. The code for this script is below:

def install(package):
    subprocess.call(['sudo', sys.executable, '-m', 'pip', 'install', package])

install('boto3')
install('Cython') # dependent package of pandas
install('numpy')  # dependent package of pandas
install('python-dateutil') # dependent package of pandas
install('pytz') # dependent package of pandas
install('pandas') # dependent package of pandas
import boto3
import pandas

When I run the state machine I get the following error:

  "Id": "s-BFGSEXELRRZZW",
      "Name": "Test",
      "Status": {
        "State": "CANCELLED",
        "StateChangeReason": {
          "Message": "Job terminated"
        },
        "Timeline": {
          "CreationDateTime": 1628382090869,
          "StartDateTime": 1628382113695
        }
      }
    }

I suspect the install('pandas') is causing the error. Is there some other way I should install pandas if I want to use this methodology? Should the order of installing the dependent packages be changed?

timeinception23
  • 49
  • 5
  • 1
    I suggest you get the pip install logs (from the subprocess call) and get the *actual* error messages. At this point, suspecting "`install('pandas')`" is basically just a guess, and there might be better or more information from the pip logs. – Gino Mempin Aug 10 '21 at 01:43
  • @GinoMempin: There are no logs for this error message in the S3 bucket where I am storing the logs. How would you get the pip install logs from the subprocess call? – timeinception23 Aug 10 '21 at 01:45
  • 1
    [Retrieving the output of subprocess.call()](https://stackoverflow.com/q/1996518/2745495) – Gino Mempin Aug 10 '21 at 01:46
  • 1
    Also, try running that piece of code on your local environment and check if it works. – Gino Mempin Aug 10 '21 at 01:48
  • @GinoMempin: It does work in my local environment. Also if I ran that code in the link you mentioned, would I be able to see the output in the step function? – timeinception23 Aug 10 '21 at 01:50

0 Answers0