0

i am trying to add a progressbar to my script but i couldn't succeed because i think it is multi-threaded or maybe it should be added in a separate thread . i found plenty of solutions in stackoverflow , for example tqdm library but i couldn't implement it , also i think i have a confusion where exactly i have to implement the progress bar code to make it works.

this is my code :

# -*- coding: utf-8 -*
from __future__ import unicode_literals
# !/usr/bin/python
import  codecs
from multiprocessing.dummy import Pool

start_raw = "myfile"
threads = 10

with codecs.open(start_raw, mode='r', encoding='ascii', errors='ignore') as f:
    lists = f.read().splitlines()
    lists = list((lists))


def myfunction(x):
    try:
        print x

    except:
        pass


def Main():
    try:
        pp = Pool(int(threads))
        pr = pp.map(myfunction, lists)


    except:
        pass


if __name__ == '__main__':
    Main()

i have tried this solution https://stackoverflow.com/a/45276885/9746396 :

# -*- coding: utf-8 -*
from __future__ import unicode_literals
# !/usr/bin/python
import  codecs
from multiprocessing.dummy import Pool
import tqdm

start_raw = "myfile"
threads = 1

with codecs.open(start_raw, mode='r', encoding='ascii', errors='ignore') as f:
    lists = f.read().splitlines()
    lists = list((lists))


def myfunction(x):
    try:
        print (x)

    except:
        pass

def Main():
    try:
        pp = Pool(int(threads))
        pr = pp.map(myfunction, lists)


    except:
        pass


if __name__ == '__main__':
    with Pool(2) as p:
        r = list(tqdm.tqdm(p.imap(Main(), range(30)), total=30))

but code does not run and i get exception (TypeError: 'NoneType' object is not callable)

0%|                                                                                           | 0/30 [00:00<?, ?it/s]Traceback (most recent call last):
  File "file.py", line 35, in <module>
    r = list(tqdm.tqdm(p.imap(Main(), range(30)), total=30))
  File "C:\mypath\Python\Python38-32\lib\site-packages\tqdm\std.py", line 1118, in __iter__
    for obj in iterable:
  File "C:\mypath\Python\Python38-32\lib\multiprocessing\pool.py", line 865, in next
    raise value
  File "C:\mypath\Python\Python38-32\lib\multiprocessing\pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
TypeError: 'NoneType' object is not callable
  0%|                                                                                           | 0/30 [00:00<?, ?it/s]
Taib Adhibi
  • 67
  • 1
  • 9
  • Does this answer your question? [Multiprocessing : use tqdm to display a progress bar](https://stackoverflow.com/questions/41920124/multiprocessing-use-tqdm-to-display-a-progress-bar) – Armin Apr 13 '20 at 13:02
  • no sir , i have tryed to implement but almost all of the solutions not working – Taib Adhibi Apr 13 '20 at 13:21
  • Could you please be more specific, what exactly in the solution above fails for you? – Armin Apr 13 '20 at 15:33
  • i have posted an update about what i tried to implement – Taib Adhibi Apr 13 '20 at 17:32

1 Answers1

0

I presume you wanted to pass myfunction instead of Main to imap, consistently with the first example.

When you pass Main() to p.imap in r = list(tqdm.tqdm(p.imap(Main(), range(30)), total=30)), Python calls executes Main method and passes the return value as the first argument to imap.

You should remove the parentheses after Main as: p.imap in r = list(tqdm.tqdm(p.imap(Main, range(30)), total=30)).

Armin
  • 168
  • 1
  • 1
  • 15