The output is unsorted and sorting on the second value is not possible. Is there special method to sort on the second value

Question

The output is unsorted, and sorting on the second column is not possible. Is there special method to sort on the second value.

This program takes a text and counts how many times a word is in a text

import string
with open("romeo.txt") as file:  # opens the file with text
    lst = []
    d = dict ()
    uniquewords = open('romeo_unique.txt', 'w')
    for line in file:
        words = line.split()
        for word in words:  # loops through all words
            word = word.translate(str.maketrans('', '', string.punctuation)).upper()  #removes the punctuations
            if word not in d:
                d[word] =1
            else:
                d[word] = d[word] +1

            if word not in lst:
                lst.append(word)    # append only this unique word to the list
                uniquewords.write(str(word) + '\n') # write the unique word to the file
print(d)

Answer 1

Dictionaries with default value

The code snippet:

d = dict()
...
if word not in d:
    d[word] =1
else:
    d[word] = d[word] +1

has become so common in python that a subclass of dict has been created to get rid of it. It goes by the name defaultdict and can be found in module collections.

Thus we can simplify your code snippet to:

from collections import defaultdict

d = defaultdict(int)
...
d[word] = d[word] + 1

No need for this manual if/else test; if word is not in the defaultdict, it will be added automatically with initial value 0.

Counters

Counting occurrences is also something that is frequently useful; so much so that there exists a subclass of dict called Counter in module collections. It will do all the hard work for you.

from collections import Counter
import string

with open('romeo.txt') as input_file:
    counts = Counter(word.translate(str.maketrans('', '', string.punctuation)).upper() for line in input_file for word in line.split())

with open('romeo_unique.txt', 'w') as output_file:
  for word in counts:
    output_file.write(word + '\n')

As far as I can tell from the documentation, Counters are not guaranteed to be ordered by number of occurrences by default; however:

• When I use them in the interactive python interpreter they are always printed in decreasing number of occurrences;
• they provide a method .most_common() which is guaranteed to return in decreasing number of occurrences.

Answer 2

In Python, standard dictionaries are an unsorted data type, but you can look here, assuming that with sorting your output you mean d

Answer 3

A couple of remarks first:

• You are not sorting explicitly (e.g. by using sorted) by a given property. Dictionaries might be considered to have a "natural" order by the alphanumeric value of the value part of each key-value pair and they might sort correctly when iterated (e.g. for printing), but it is better to explicitly sort a dict.
• You check the existence of a word in the lst variable, which is very slow since checking a list requires checking all entries until something is found (or not). It would be much better to check for existence in a dict.
• I'm assuming by "the second column" you mean the information for each word that counts the order in which the word first appeared.

With that I'd change the code to also record the word index of the first occurence of each word with, which then allows for sorting on exactly that.

Edit: Fixed the code. The sorting yielded by sorted sorts by key, not value. That's what I get for not testing code before posting an answer.

import string
from operator import itemgetter

with open("romeo.txt") as file: # opens the file with text
    first_occurence = {}
    uniqueness = {}
    word_index = 1
    uniquewords = open('romeo_unique.txt', 'w')

    for line in file:
        words = line.split()

        for word in words: # loops through all words
            word = word.translate(str.maketrans('', '', string.punctuation)).upper() #removes the punctuations

            if word not in uniqueness:
                uniqueness[word] = 1
            else:
                uniqueness[word] += 1

            if word not in first_occurence:
                first_occurence[word] = word_index
                uniquewords.write(str(word) + '\n') # write the unique word to the file

            word_index += 1

    print(sorted(uniqueness.items(), key=itemgetter(1)))
    print(sorted(first_occurence.items(), key=itemgetter(1)))