File operations - filtering data from txt file in python

Question

This code works fine but is too big, i would like to know if there is any other way to write this code to make it shorter.

 import openpyxl as excel
PATH = "/home/Fathima/workspace/training/"
ACCESS_LIST = []
def READ_CONFIG():
    FROM_ZONE = ""
    TO_ZONE = ""
    POLICY_NAME = ""
    SOURCE_ADDR = ""
    DESTINATION_ADDR = ""
    PORT = ""
    global PATH
    global ACCESS_LIST
    count = 0
    CONFIG_PATH=PATH+"hofwdcn05dcn_20210216(2).txt"
    fh = open(CONFIG_PATH, 'r')
    CONFIG_LINES=fh.readlines()
    config_lines_cnt = len(CONFIG_LINES)
    while count < config_lines_cnt:
        line = CONFIG_LINES[count].strip()
        if len(line) > 0:
            line_to_array = line.split(' ')
            if line.startswith('from-zone '):
                FROM_ZONE = line_to_array[1]
                TO_ZONE = line_to_array[3]
            elif line.startswith('policy '):
                POLICY_NAME = line_to_array[1]
            elif line.startswith('source-address '):
                SOURCE_ADDR = line_to_array[1].replace(";", "")
            elif line.startswith('destination-address '):
                DESTINATION_ADDR = line_to_array[1].replace(";", "")
            elif line.startswith('application '):
                PORT = line_to_array[1].replace(";", "")
            elif line.startswith('then {'):
                count = count+1
                line = CONFIG_LINES[count].strip()
                if line == "permit;":
                    dummy = { 'FROM_ZONE' : FROM_ZONE,'TO_ZONE' : TO_ZONE,'POLICY_NAME' : POLICY_NAME,'SOURCE_ADDR' : SOURCE_ADDR,'DESTINATION_ADDR' : DESTINATION_ADDR,'PORT' : PORT}
                    ACCESS_LIST.append(dummy)
                    FROM_ZONE = ""
                    TO_ZONE = ""
                    POLICY_NAME = ""
                    SOURCE_ADDR = ""
                    DESTINATION_ADDR = ""
                    PORT = ""
        count +=1
#MAIN PROGRAM STARTS FROM HERE
READ_CONFIG()
print(ACCESS_LIST)

Here i have a huge file and need the output appearing as below format

[{ from-zone: to-zone: policy: source-address: destination-address: application: },{ from-zone: to-zone: policy: source-address: destination-address: application: }]

You're missing indentations for all blocks i.e. functions, while loops, conditionals. this makes it difficult to follow the program flow. — DarrylG, Mar 15 '21 at 12:31
I have edited the code, please check now, Thanks in advance @DarrylG — Tahera, Mar 15 '21 at 12:46

DarrylG · Accepted Answer · 2021-03-15T13:47:16.493

There is a separate related site for a review of working code i.e. StackExchange Code Review

That said, below is a more Pythonic code flow. I didn't change conditionals since they are easy to follow.

Main Changes

Eliminate globals (discouraged--only for special needs)
Use file context manager (i.e. use 'with block' on file open)
Iterate through file rather than read the entire file (allows processing arbitrary file size)
Use Python variable and function naming convention i.e. PEP 8
Remove import openpyxl (unused)

Code

def read_config(path):

    from_zone, to_zone, policy_name, source_addr, destination_addr, port = [''] * 6
    access_list = []

    with open(path + "hofwdcn05dcn_20210216(2).txt", 'r') as fh:
        for line in fh:
            line = line.strip()
            if line:
                line_to_array = line.split(' ')

                if line.startswith('from-zone '):
                    from_zone = line_to_array[1]
                    to_zone = line_to_array[3]
                elif line.startswith('policy '):
                    policy_name = line_to_array[1]
                elif line.startswith('source-address '):
                    source_addr = line_to_array[1].replace(";", "")
                elif line.startswith('destination-address '):
                    destination_addr = line_to_array[1].replace(";", "")
                elif line.startswith('application '):
                    port = line_to_array[1].replace(";", "")
                elif line.startswith('then {'):
                    line = next(fh).strip()  # Gets next line in file
                    if line == "permit;":
                        access_list.append({'FROM_ZONE': from_zone,
                                            'TO_ZONE': to_zone,
                                            'POLICY_NAME': policy_name,
                                            'SOURCE_ADDR': source_addr,
                                            'DESTINATION_ADDR': destination_addr,
                                            'PORT': port})

                        from_zone, to_zone, policy_name, source_addr, destination_addr, port = [''] * 6
    return access_list

access_list = read_config("/home/Fathima/workspace/training/")
print(access_list)

Done, i would like to know why are we multiplying with 6 in the code — Tahera, Mar 16 '21 at 10:29
@Tahera--those two instances are using [unpacking](https://stackoverflow.com/questions/55595010/what-does-it-mean-to-unpack-in-python) of a replicated list i.e. [replicating elements in list](https://stackoverflow.com/questions/20276138/replicating-elements-in-list). `[''] * 6` creates a list of 6 empty strings i.e. `['', '', '', '', '', '']`, whose elements are unpacked into the variables `from_zone, to_zone, ` etc. Does this help? — DarrylG, Mar 16 '21 at 12:13
There's similar type of solution I'm seeking for another txt file, https://stackoverflow.com/q/66655050/15398942 please check if you can help @DarryIG — Tahera, Mar 16 '21 at 12:53

File operations - filtering data from txt file in python

1 Answers1