How change a os.popen command to subprocess command in python

Question

I know that os.popen is deprecated now. So which is the easiest way to convert a os.popen command to subprocess.

cmd_i = os.popen('''sed -n /%s/,/%s/p %s | grep "Invalid Credentials"''' % (st_time, en_time, fileName))

Answer 1

The subprocess code will be roughly equivalent.

output = subprocess.check_output(
    '''sed -n /%s/,/%s/p %s | grep "Invalid Credentials"''' % (st_time, en_time, fileName),
    shell=True)

The output is captured into a variable, rather than spilled out onto standard output outside of Python's control. If all you want is to print it, go ahead and do that.

This will throw an exception if grep doesn't find anything. You can trap it with except if you want to handle this corner case one way or another.

However, you could easily replace all of this with native Python code.

with open(fileName, 'r') as input:
    between = False
    output = []
    for line in input:
        if st_time in line:
            between = True
        if between and 'Invalid Credentials' in line:
            output.append(line)
        if en_time in line:
            between = False

The output in this case is a list. Every captured line still contains its terminating newline. (Easy to fix if that's not what you want, of course.)

As an optimization, you could break when you see en_time (though then the script won't be exactly equivalent to the sed script).

This should also be easy to adapt to a scenario where st_time doesn't occur exactly in the log file. You will need to parse the time stamp on each line, and then simply start processing when the parsed date stamp is equal to or bigger than the (parsed) st_time value. Similarly, parse en_time and exit when you see log entries with time stamps equal to or above this value.