How can I pipe a Python process' output in Python?

Question

I'm writing a program that downloads videos from YouTube, using youtube-dl.

I used to call youtube-dl with subprocess:

import subprocess

p = subprocess.Popen([command], \
    stdout=subprocess.PIPE, \
    stderr=subprocess.STDOUT, \
    universal_newlines = True)

Then, I would read the process' output by calling:

for line in iter(p.stdout.readline, ""):
    hide_some_stuff_using_regex()
    show_some_stuff_using_regex()

However, I prefer using youtube-dl as a Python class. So I'm now doing this:

from youtube_dl import YoutubeDL as youtube_dl

options = {"restrictfilenames": True, \
           "progress_with_newline": True}

ydl = youtube_dl(options)
ydl.download([url])

The code works, but I'm having a hard time finding out, how to pipe youtube-dl's output. Note that I want to use parts of youtube-dl's output to print in real-time, so redirecting sys.stdout to a custom output stream will not work, as I still need sys.stdout to print.

Can you help me?

score 4 · Accepted Answer · answered Feb 03 '15 at 12:25

4

Specifically for youtube-dl, you can set a logger object, like in the advanced example in the documentation:

from youtube_dl import YoutubeDL


class MyLogger(object):
    def debug(self, msg):
        print('debug information: %r' % msg)

    def warning(self, msg):
        print('warning: %r' % msg)

    def error(self, msg):
        print('error: %r' % msg)


options = {
    "restrictfilenames": True,
    "progress_with_newline": True,
    "logger": MyLogger(),
}

url = 'http://www.youtube.com/watch?v=BaW_jenozKc'
with YoutubeDL(options) as ydl:
    ydl.download([url])

answered Feb 03 '15 at 12:25

Beautiful. Great to be able to be in contact with the developer himself! – Exeleration-G Feb 03 '15 at 19:01
What about getting ffmpeg info? I'd like to grab the the `frame= 1766 fps=157 q=-1.0 Lsize= 10152kB time=00:01:13.66 bitrate=1128.9kbits/s speed=6.54x` line – LukeSavefrogs May 15 '20 at 08:21

score 1 · Answer 2 · edited May 23 '17 at 11:53

1

You can try to redirect sys.stdout to your own output stream
see: https://stackoverflow.com/a/1218951/2134702

To quote the linked answer:

from cStringIO import StringIO
import sys

old_stdout = sys.stdout
sys.stdout = mystdout = StringIO()

# blah blah lots of code ...

sys.stdout = old_stdout

# examine mystdout.getvalue()

And if you want to output to stdout during the redirection, instead of print use old_stdout.write()

edited May 23 '17 at 11:53

Community

1
1

answered Feb 03 '15 at 11:16

Eran

2,324
3
22
27

1

This should be a comment, not an answer. – Feb 03 '15 at 11:16
I thought about that, but from the moment I redirect `sys.stdout` to an output stream, I'm not able to use print anymore, am I? The print statements would be redirected to the output stream and not get printed on my screen. That's a problem, because I want to print some parts of youtube-dl's output on my screen in real-time. – Exeleration-G Feb 03 '15 at 11:33
@Exeleration-G, if you save the stdout stream (as shown in the linked answer), you can write to it (instead of using print command), and it would be printed to the main screen in real time. – Eran Feb 03 '15 at 11:55
@Eran: Of course! I didn't think of using stuff like `old_stdout.write()`! – Exeleration-G Feb 03 '15 at 12:24

How can I pipe a Python process' output in Python?

2 Answers2