Read lines from a multiple txt files into variables

Question

I have multiple txt files in a directory where each txt file have few lines of hashes:

546516fdsfgbfdgbfdf

232321dfsfdsgfdgvfd

321656fdsfsffgfdgfd

I'm trying to read each line in all the txt files in the directory and then pass them to the "hash" variable And run the script every time anew in order to download the hash binary file.

import requests

f = open('*.txt')
    hash = f.read().splitlines()
    f.close()

    params = {'apikey': 'XXXXXXXXXXXXXX', 'file': (hash)}
response = requests.get('https://www.test.com/file/download', params=params)

downloaded_file = response.content

if response.status_code == 200:
    with open('/tmp/sample.bin', 'wb') as f:
        f.write(response.content)

Answer 1

If you're trying to just read the whole line into num, that will do the trick.

if response.status_code == 200:
    with open('/tmp/file.bin', 'wb') as f:
        num = f.read()

Answer 2

Had a similar problem, I needed to change a certain String in every file of a folder if appearing.

Important This is a rather older script, so there are some to do's. I would recommend to resolve those issues as an exercise to improve the code quality:

• Using a try and raise Block if an Error occurs.
• Define regular expressions object instead of defining them in the line, the algorithm works faster with this
• Applying List Comprehensions

Note I use regex101.com for creating my regular expressions.

import os, fnmatch, re

allLinesString = []

def findReplace(directory, filePattern):
    for path, dirs, files in os.walk(os.path.abspath(directory)):
        for filename in fnmatch.filter(files, filePattern):
            filepath = os.path.join(path, filename)
            global i
                with open(filepath, "r") as f:
                    for line in f:
                    # So lets decide here what to do when we go through each line of each file found in the directory
                        if "String" in line:
                            # Okay and here we write the String if "String" is in line to an array
                            allLinesString.append(line)
                            # And if you want you can even use a regex expression to manipulate the "String" which you just found, or delete certain characters what so ever
                            # Or write the entry to a varible called num in your case
                            allLinesString[i] = re.sub(r'^\s', '', allLinesString[i])

# In this case the function searches for all markdown files for the specific string
if __name__ == '__main__':
    findReplace("wantedDirectoryPath/", "*.md")

Answer 3

As I understand it, reading all hashes will solve your problem. Try this, and take a look at the comments, too.

import os
import requests

# If text files aren't in the same dir, edit path as you see fit
path = os.getcwd()

# Reads all text files in current directory.
all_files = [i for i in os.listdir(path) if i[-4:] == '.txt']
all_hashes = []

for file in all_files:
    with open(path+file,'r+') as f:
        lines = f.readlines()
    for line in lines:
        # Handling empty lines only here, add regex or more params to strip if needed
        if not line.strip('\n'):
            continue
        # All hashes are now stored in a list
        all_hashes.append(line.strip('\n'))

# Just iterate over the list and make requests
# You may want to add a delay with time.sleep() in the loop
for hash in all_hashes:
    params = {'apikey': 'XXXXXXXXXXXXXX', 'file': (hash)}
    response = requests.get('https://www.test.com/file/download', params=params)
    downloaded_file = response.content

    if response.status_code == 200:
        # you may want to use 'a' instead of 'wb' here?
        with open('/tmp/sample.bin', 'wb') as f:
            f.write(response.content)

Edit: Also take a look at this post which tells you how you can read all txt files from a directory, and at this one which shows how to read a file, line-by-line, into a list.