Equivalent to grep, but not using open()

Question

I have the following script (see below) which is taken stdin and manipulating into some simple files.

# Import Modules for script
import os, sys, fileinput, platform, subprocess

# Global variables
hostsFile = "hosts.txt"
hostsLookFile = "hosts.csv"

# Determine platform
plat = platform.system()

if plat == "Windows":

# Define Variables based on Windows and process
    currentDir = os.getcwd()
    hostsFileLoc = currentDir + "\\" + hostsFile
    hostsLookFileLoc = currentDir + "\\" + hostsLookFile
    ipAddress = sys.argv[1]
    hostName = sys.argv[2]
    hostPlatform = sys.argv[3]
    hostModel = sys.argv[4]

    # Add ipAddress to the hosts file for python to process 
    with open(hostsFileLoc,"a") as hostsFilePython:
        for line in open(hostsFilePython, "r"):
            if ipAddress in line:
                print "ipAddress \(" + ipAddress + "\) already present in hosts file"
            else:
                print "Adding ipAddress: " + ipAddress + " to file"
                hostsFilePython.write(ipAddress + "\n")

    # Add all details to the lookup file for displaying on-screen and added value
    with open(hostsLookFileLoc,"a") as hostsLookFileCSV:
        for line in open(hostsLookFileCSV, "r"):
            if ipAddress in line:
                print "ipAddress \(" + ipAddress + "\) already present in lookup file"
            else:
                print "Adding details: " + ipAddress + "," + hostName + "," + hostPlatform + "," + hostModel + " to file"
                hostsLookFileCSV.write(ipAddress + "," + hostName + "," + hostPlatform + "," + hostModel + "\n")

if plat == "Linux":

# Define Variables based on Linux and process
    currentDir = os.getcwd()
    hostsFileLoc = currentDir + "/" + hostsFile
    hostsLookFileLoc = currentDir + "/" + hostsLookFile
    ipAddress = sys.argv[1]
    hostName = sys.argv[2]
    hostPlatform = sys.argv[3]
    hostModel = sys.argv[4]

    # Add ipAddress to the hosts file for python to process 
    with open(hostsFileLoc,"a") as hostsFilePython:
        print "Adding ipAddress: " + ipAddress + " to file"
        hostsFilePython.write(ipAddress + "\n")

    # Add all details to the lookup file for displaying on-screen and added value
    with open(hostsLookFileLoc,"a") as hostsLookFileCSV:
        print "Adding details: " + ipAddress + "," + hostName + "," + hostPlatform + "," + hostModel + " to file"
        hostsLookFileCSV.write(ipAddress + "," + hostName + "," + hostPlatform + "," + hostModel + "\n")

This code obviously does not work, because the for line in open(hostsFilePython, "r"): syntax is wrong... I can not use a current file object with "open()". However this is want I want to achieve, how can I do this?

Are you aware that forward slashes work on Windows as path separators, and that you should be using `os.path.join()` anyway? Think DRY. (Also, you almost certainly want to use the `csv` module, even if you probably won't see a comma embedded in a hostname to break your method) — Wooble, Jun 14 '12 at 12:48

score 2 · Answer 1 · edited May 23 '17 at 12:03

2

You want to open your file using the a+ mode so that you can both read and write, then simply use the existing file object (hostsFilePython).

However, this still won't work as you can only iterate over a file once before it is exhausted.

It's worth noting that this isn't very efficient. The better plan is to read the data into a set, update the set with your new values, then write the set to the file. (As pointed out in the comments, sets don't preserve duplicates (good for your purposes), and order, which may or may not work for you. If not, then you might need to use a list, which will be less efficient).

edited May 23 '17 at 12:03

Community

1
1

answered Jun 14 '12 at 12:40

Gareth Latty

86,389
17
178
183

I have some doubts the mixed reads and writes in the original code will lead to a useful result. – Sven Marnach Jun 14 '12 at 12:42
@SvenMarnach Slowly making my way there XD. – Gareth Latty Jun 14 '12 at 12:46
(I didn't spend too much time trying to figure out the OP's code), but reading the data into a set will not preserve order -- which for a grep-like utility seems important. – mgilson Jun 14 '12 at 12:46
@mgilson That is true. In that case, a list may be needed to preserve order. It depends whether or not order matters here, with could go either way. I have updated to make the downsides of a set more clear. – Gareth Latty Jun 14 '12 at 12:47
So I'm a little confused over this list comment, is this a function? – MHibbin Jun 14 '12 at 13:10
I get the following when I use the a+ way you described Traceback (most recent call last): File "addNew.py", line 33, in for line in open(hostsFilePython): TypeError: coercing to Unicode: need string or buffer, file found – MHibbin Jun 14 '12 at 13:10
@MHibbin You need to loop over the file, it's already been opened, why are you trying to open it again? – Gareth Latty Jun 14 '12 at 14:11

Burhan Khalid · Answer 2 · 2012-06-14T12:50:31.500

0

with open(hostsFileLoc) as hostsFilePython:
    lines = hostsFilePython.readlines()

for filename in lines:
    with open(hostsFileLoc, 'a') as hostFilePython:
        with open(filename) as hostsFile:
            for line in hostsFile.readlines():
                if ipAddress in line:
                   print "ipAddress \(" + ipAddress + "\) already present in hosts file"
                else:
                   print "Adding ipAddress: " + ipAddress + " to file"
                   hostsFilePython.write(ipAddress + "\n")

The default mode is read, so you don't need to pass in r explicitly.

edited Jun 14 '12 at 12:50

answered Jun 14 '12 at 12:44

Burhan Khalid

169,990
18
245
284

1

This won't work. You are opening `hostsFilePython` in `a` mode and then trying to read. – Gareth Latty Jun 14 '12 at 12:45
You could change the mode to `'a+'` – mgilson Jun 14 '12 at 12:48

Equivalent to grep, but not using open()

2 Answers2