How to parse results from a subprocess

Question

I'm trying to pass a string in Python that was obtained from netstat to awk for use in building a new object.

Any clue why this isn't working? Please excuse the horrible coding, I just started using Python today and trying to learn how to use the language.

class NetstatGenerator():

    def __init__(self):
    import subprocess
    self.results = subprocess.Popen("netstat -anbp tcp", shell=True, stdout=subprocess.PIPE).stdout.read()
    self.list = []
    self.parse_results()

def parse_results(self):
    lines = self.results.splitlines(True)
    for line in lines:
        if not str(line).startswith("tcp"):
            print("Skipping")
            continue

        line_data = line.split(" ")
        self.list.append(NetstatData(self.line_data[0], self.line_data[15], self.line_data[18], self.line_data[23]))

def get_results(self):
    return self.list

class NetstatData():

    def __init__(self, protocol, local_address, foreign_address, state):
        self.protocol = protocol
        self.local_address = local_address
        self.foreign_address = foreign_address
        self.state = state

    def get_protocol(self):
        return str(self.protocol)

    def get_local_address(self):
        return str(self.local_address)

    def get_foreign_address(self):
        return str(self.foreign_address)

    def get_state(self):
        return str(self.state)

data = NetstatGenerator()

I'd cut the task in halves. One half is running a subprocess; apparently you figured it. The other is parsing. I'd copy-paste some `netstat` output to my code and make it parse it first, for quicker turnaround. Note: what `awk` does for you `line.split()` would do right inside Python. — 9000, Dec 22 '15 at 19:40
@9000 thanks, trying to rewrite that section now using `line.split()` — nullByteMe, Dec 22 '15 at 19:43
First, you are using `open()` to open the result of `netstat -anbp tcp` as a filename. Second, you placed `continue` inside `with`, which is not allowed. — vrs, Dec 22 '15 at 19:54
@vrs addressing your first point, how should I execute the command? All of my searching seemed to indicate I use `open()` — nullByteMe, Dec 22 '15 at 19:55
@vrs I also modified what I'm doing which hopefully is more clear — nullByteMe, Dec 22 '15 at 19:57
unrelated: don't use `str()` until you learn about bytes, Unicode strings and `.encode()`/`.decode()` methods. Every `str()` in your code is a potential bug. — jfs, Dec 22 '15 at 20:08
@J.F.Sebastian I took your suggestion and read up on Unicode strings and `.encode()` and `.decode()` in relation to Python. Prior to this, I was already familiar with Unicode, ascii, and various other character sets being represented as bytes, but not how to use them in the context of Python. From what I gathered, it seems like I use `str()` only when presenting human-readable data to the end user, and unicode or any other appropriate encoding when necessary? — nullByteMe, Dec 23 '15 at 20:23
no, don't call `str()` on a string data at all (it may appear to work in some cases but at best it is a noop (on Unicode strings on Python 3 or on bytestrings on Python 2), at worst it corrupts your data silently (if called on `bytes` objects on Python 3 or if someone hacked `sys.getdefaultencoding()` to return non "ascii" value on Python 2), typically it works with ascii-only data in your test and it fails in production on non-ascii data (if called on Unicode strings on Python 2). Use `bytestring = unicode_string.encode(enc)`, `unicode_string = bytestring.decode(enc)` when necessary instead. — jfs, Dec 23 '15 at 21:00
human-readable text should be represented as Unicode in your program. Use bytes only if external API requires it (on the boundaries of your application). — jfs, Dec 23 '15 at 21:06

9000 · Accepted Answer · 2015-12-22T20:07:59.823

2

Sorry, netstat does not support -b on Linux, and I don't have a BSD box lying around.

Let's assume you have a list of lines, called netstat_output, with items like this:

tcp 0 0 127.0.0.1:9557 127.0.0.1:56252 ESTABLISHED -

To parse a single line, you split() it and pick elements at indexes 0, 3, 4, 5.

To store the items, you don't need to define a boilerplate holding class; namedtuple does what you want:

from collections import namedtuple

NetstatInfo = namedtuple('NetstatInfo', 
    ['protocol', 'local_address', 'remote_address', 'state'])

Now you can parse a line:

def parseLine(line):
  fields = line.split()
  if len(fields) == 7 and fields[0] in ('tcp', 'udp'):  
    # alter this condition to taste; 
    # remember that netstat injects column headers.
    # consider other checks, too.
    return NetstatInfo(fields[0], fields[3], fields[4], fields[5])
  # otherwise, this function implicitly returns None

Now something like this must be possible:

result = []

for line in subprocess.Popen(...):
  item = parseLine(line)
    if line:  # parsed successfully
      result.append(line)

# now result is what you wanted; e.g. access result[0].remote_address

edited Dec 22 '15 at 20:07

answered Dec 22 '15 at 19:57

9000

39,899
9
66
104

This is a fantastic explanation. Thank you! This language is very different from Java, which is the language that I'm coming from. – nullByteMe Dec 22 '15 at 20:01
1

You're welcome. You might find some useful nuggets [here](http://programmers.stackexchange.com/a/47529/11732), too. – 9000 Dec 22 '15 at 20:02
3

@free_mind here's a [correct way to read output from a subprocess line by line](http://stackoverflow.com/a/17698359/4279). Though in your case, you could use `lines = subprocess.check_output("netstat -anbp tcp".split(), universal_newlines=True).splitlines()` – jfs Dec 22 '15 at 20:11
@J.F.Sebastian thanks, that's part of the reason why I was using `str()`, I didn't understand how to handle the output, though I do understand bytes and unicode conceptually. – nullByteMe Dec 22 '15 at 20:16

How to parse results from a subprocess

1 Answers1