Editing csv file to edit subnets so there is no overlap in IP ranges (ipaddress module in Python 3.3)

Question

I have been trying to create a csv file that will change all the subnets and IP address ranges so none overlap.

I have this file to start:

Zone Name, IPStart, IPStop,Range,Source
Group A,10.0.0.0,10.127.255.255,10.0.0.0/9,New List
Group A Sales,10.16.0.0,10.31.255.255,10.16.0.0/12,New List
Group A Marketing,10.62.0.0,10.62.255.255,10.62.0.0/16,New List
Group A Research,10.62.0.0,10.63.255.255,10.62.0.0/15,Old List
Group A Sales Primary routers,10.23.1.0,10.23.1.15,10.24.1.0/28,New List
Group A Sales Web Servers - Primary,10.18.0.0,10.18.0.255,10.18.0.0/24,New List
Group A Sales Web Servers,10.16.0.0,10.19.255.255,10.16.0.0/14,New List
Group B,10.128.0.0,10.255.255.255,10.128.0.0/9,Old List

And I want it to look like this (order of lines does not matter):

Zone Name, IPStart, IPStop,Range,Source
Group A,10.0.0.0,10.15.255.255,10.0.0.0/12,New List
Group A,10.32.0.0,10.47.255.255,10.32.0.0/12,New List
Group A,10.48.0.0,10.55.255.255,10.48.0.0/13,New List
Group A,10.56.0.0,10.59.255.255,10.56.0.0/14,New List
Group A,10.60.0.0,10.61.255.255,10.60.0.0/15,New List
Group A,10.64.0.0,10.127.255.255,10.64.0.0/10,New List
Group A Marketing,10.62.0.0,10.62.255.255,10.62.0.0/16,New List
Group A Research,10.63.0.0,10.63.255.255,10.63.0.0/16,Old List
Group A Sales,10.20.0.0,10.21.255.255,10.20.0.0/15,New List
Group A Sales,10.22.0.0,10.22.255.255,10.22.0.0/16,New List
Group A Sales,10.23.0.0,10.23.0.255,10.23.0.0/24,New List
Group A Sales,10.23.1.128,10.23.1.255,10.23.1.128/25,New List
Group A Sales,10.23.1.16,10.23.1.31,10.23.1.16/28,New List
Group A Sales,10.23.1.32,10.23.1.63,10.23.1.32/27,New List
Group A Sales,10.23.1.64,10.23.1.127,10.23.1.64/26,New List
Group A Sales,10.23.128.0,10.23.255.255,10.23.128.0/17,New List
Group A Sales,10.23.16.0,10.23.31.255,10.23.16.0/20,New List
Group A Sales,10.23.2.0,10.23.3.255,10.23.2.0/23,New List
Group A Sales,10.23.32.0,10.23.63.255,10.23.32.0/19,New List
Group A Sales,10.23.4.0,10.23.7.255,10.23.4.0/22,New List
Group A Sales,10.23.64.0,10.23.127.255,10.23.64.0/18,New List
Group A Sales,10.23.8.0,10.23.15.255,10.23.8.0/21,New List
Group A Sales,10.24.0.0,10.31.255.255,10.24.0.0/13,New List
Group A Sales,10.24.32.0,10.24.63.255,10.24.32.0/19,New List
Group A Sales Primary routers,10.23.1.0,10.23.1.15,10.24.1.0/28,New List
Group A Sales Web Servers,10.16.0.0,10.17.255.255,10.16.0.0/15,New List
Group A Sales Web Servers,10.19.0.0,10.19.255.255,10.19.0.0/16,New List
Group A Sales Web Servers,10.18.128.0,10.18.255.255,10.18.128.0/17,New List
Group A Sales Web Servers,10.18.64.0,10.18.127.255,10.18.64.0/18,New List
Group A Sales Web Servers,10.18.32.0,10.18.63.255,10.18.32.0/19,New List
Group A Sales Web Servers,10.18.16.0,10.18.31.255,10.18.16.0/20,New List
Group A Sales Web Servers,10.18.8.0,10.18.15.255,10.18.8.0/21,New List
Group A Sales Web Servers,10.18.4.0,10.18.7.255,10.18.4.0/22,New List
Group A Sales Web Servers,10.18.2.0,10.18.3.255,10.18.2.0/23,New List
Group A Sales Web Servers,10.18.1.0,10.18.1.255,10.18.1.0/24,New List
Group A Sales Web Servers - Primary,10.18.0.0,10.18.0.255,10.18.0.0/24,New List
Group B,10.128.0.0,10.255.255.255,10.128.0.0/9,Old List

I have been playing around with the ipaddress module in Python 3.3 using the network objects as described here http://docs.python.org/dev/library/ipaddress.

This is my code so far:

import ipaddress
import csv
from csv import DictReader, DictWriter

with open(r'file1.csv', newline='') as fin3,\
     open(r'file2.csv', newline='') as fin4,\
     open(r'file3.csv', 'w', newline='') as fout3:


    read3 = DictReader(fin3) # fin 3 and 4 are copies of the same file
    read4 = DictReader(fin4)

    writenum3 = DictWriter(fout3, fieldnames=read3.fieldnames) 
    writenum3.writeheader()

    for line3 in read3:
        line3['Range']=ipaddress.ip_network(line3['Range']) # Convert IP ranges to ip network object
        for line4 in read4:
            line4['Range']=ipaddress.ip_network(line4['Range']) # Convert IP ranges to ip network object
            if line3['Range'].netmask < line4['Range'].netmask: # To avoid "Not contained in" errors
                if line3['Range'].overlaps(line4['Range']): # Tests for IP overlap
                        for line4 in read4:
                            line4['Range']=ipaddress.ip_network(line4['Range'])
                            lst=list(line3['Range'].address_exclude(line4['Range'])) # List of subnets excluding line4 subnet
                            print (lst) # Temporary to view contents of lst
                            for val in lst:
                                line3['Range']=val # New range
                                line3[' IPStart']=val.network_address #New net address
                                line3[' IPStop']=val.broadcast_address #New broadcast address
                                writenum3.writerow(line3) # Write lines

With this code I am trying to get the new subnets with the correct names to write to file 3. Both file1 and file2 are the starting file in the beginning of the code (comparing it to itself, basically). There are two problems I am having with this code:

It works until it finds a subnet not contained in another subnet. Then it returns ValueError: 10.24.1.0/28 not contained in 10.20.0.0/14 (for example). I have tried to stop this from happening by using the if statements above to make sure that the lines overlap and that the IP subnet being excluded has a larger subnet mask than the one excluding but it is obviously not working and I have tried a lot of different things to try to make them work (putting them after all the for statements, etc.). If I could somehow get around this error and have it ignore that entry and keep comparing the rest, I think it could work.
When the subnets are written to file3 they always use the first Zone Name "Group A" no matter what the name is supposed to be.

I apologize for anything that may be obvious I am missing, I am still a beginner at programming.

Any help is appreciated.

-B0T

score 0 · Accepted Answer · answered Jul 31 '13 at 19:51

Both of my issues were solved by reordering the if statements and correctly resetting the iterator. There continues to be issues with the code, but my questions here have been answered.

This is my current code and it works better:

import ipaddress
import csv
from csv import DictReader, DictWriter
from itertools import filterfalse

with open(r'file1.csv', newline='') as fin3,\
     open(r'file2.csv', newline='') as fin4,\
     open(r'file3.csv', 'w', newline='') as fout3:


    read3 = DictReader(fin3) # fin 3 and 4 are copies of the same file
    read4 = DictReader(fin4)

    writenum3 = DictWriter(fout3, fieldnames=read3.fieldnames) 
    writenum3.writeheader()

    lst4=[] # To prevent duplicate lines


    for line3 in read3:
        line3['Range']=ipaddress.ip_network(line3['Range']) # Convert IP ranges to ip network object
        fin4.seek(0) # Reset iterator
        read4 = DictReader(fin4) # Read again to prevent errors
        for line4 in read4:
            line4['Range']=ipaddress.ip_network(line4['Range']) # Convert IP ranges to ip network object
            if line3 not in lst4: # To prevent duplicate lines
                if line3['Range'].overlaps(line4['Range']): # Tests for IP overlap
                    if line3['Range'].netmask < line4['Range'].netmask: # To avoid "Not contained in" errors
                        lst=list(line3['Range'].address_exclude(line4['Range'])) # List of subnets excluding line4 subnet
                        for val in lst:
                            line3['Range']=val # New range
                            line3[' IPStart']=val.network_address #New net address
                            line3[' IPStop']=val.broadcast_address #New broadcast address
                            lst4.append(line3)
                            writenum3.writerow(line3) # Write lines

                    else:
                        lst4.append(line3)
                        writenum3.writerow(line3) # Write lines
                else:
                    lst4.append(line3)
                    writenum3.writerow(line3) # Write lines

Editing csv file to edit subnets so there is no overlap in IP ranges (ipaddress module in Python 3.3)

1 Answers1

Linked