0

I am learning python and writing a small script. I need my output to be in a file and also print the output on the screen as well.I tried various methods like stdout=subprocess.PIPEi could not figure out outputting to both.please forgive me for this silly question`

#!/usr/bin/python
import os
import subprocess
with  open('/root/test/testfile') as f , open('aaa.txt',"w") as final:
    content =  f.read().splitlines()
    for x in content:
            result= subprocess.Popen(['grep',x,'/news/today/night/logs'],stdout=final)
evanooruvan
  • 25
  • 1
  • 6
  • I think this is a duplicate. Let me know if it doesn't meet your needs. – tdelaney Oct 23 '20 at 05:43
  • @tdelaney i am unable to understand that solution comparing with mine question can you please assist me with the proc wait and sys command. – evanooruvan Oct 23 '20 at 05:57
  • What are the lines in testfile? Some words to find in logs, or regular expressions? – tdelaney Oct 23 '20 at 06:06
  • I don't think you need a subprocess at all. Just have python find the lines. That's kinda what the existing answer does but it seems to have the files mixed up. i'll post my attempt. – tdelaney Oct 23 '20 at 06:07
  • @tdelaney thank you would be great if you could share your solution , i created something similar to your but i am getting only one grep output. – evanooruvan Oct 23 '20 at 06:12
  • @tdelaney testfile has servernames it would search in this /news/today/night/logs and grep the output of those servernames in test file – evanooruvan Oct 23 '20 at 06:14

2 Answers2

1

It looks like you are only using subprocess to run grep, but python can do grep-like matches of strings too. This program will read the lines in "testfile" and then write out lines from "log" that contain the testfile line. All of the log matches from the first line in "testfile" will be above the matches for the second line, etc... And log lines that match multiple testfile lines will output multiple times.

This code assumes that you are not matching a regular expression.

#!/usr/bin/python

# assuming logs is not too big, you could read it into memory once
# with open('/news/today/night/logs') as logfile:
#     logs = list(logfile)
    
with  open('/root/test/testfile') as f , open('aaa.txt',"w") as final:
    for wanted in f:
        wanted = wanted.strip()
        with open('/news/today/night/logs') as logs:
            for log in logs:
                if wanted in log:
                    print(log, end='')
                    final.write(log)
tdelaney
  • 73,364
  • 6
  • 83
  • 116
  • wow did not realize i could write this way... is there any drawback of using my code vs yours and can you help me print stdout in my code. Thanks a lot – evanooruvan Oct 23 '20 at 06:52
  • My answer that I originally linked shows you how to do that. Make your subprocess stdout a pipe, then have a loop read the pipe and write to multiple places. https://stackoverflow.com/questions/15535240/how-to-write-to-stdout-and-to-log-file-simultaneously-with-popen – tdelaney Oct 23 '20 at 06:58
  • thank you , but as you said i cannot open the logs(open('/news/today/night/logs') as logs) in a memory as it will system/application logs which can be big i'll give your other method a try and will update you :) – evanooruvan Oct 23 '20 at 13:47
0

Try this:

#!/usr/bin/python
import os

with open('/root/test/testfile') as f, open('/root/test/aaa.txt',"a") as final:
    content = f.read().splitlines()
    for line in content:
        if "/news/today/night/logs" in line:
            print(line)
            final.write(line)

I made your aaa.txt file append rather than write.

Cole Arseneau
  • 104
  • 6