add py3 multithreading example, fix py2 and multiprocessing #796
Description
tqdm works fine with multithreading on python3:
from time import sleep
from tqdm import tqdm
from concurrent.futures import ThreadPoolExecutor
def foo(a):
tqdm.write("Working on %d" % a)
sleep(0.1)
return a
tqdm.get_lock() # ensures locks exist
with ThreadPoolExecutor(max_workers=2) as executor:
list(tqdm(executor.map(foo, range(8)), total=8))Working on 0
Working on 1
Working on 2
Working on 3
Working on 4
Working on 5
Working on 6
Working on 7
100%|██████████| 8/8 [00:00<00:00, 20.05it/s]
[0, 1, 2, 3, 4, 5, 6, 7]however the py2 output is:
Working on 0
Working on 1
0%| | 0/8 [00:00<?, ?it/s]Working on 2
Working on 3
Working on 4
38%|███▊ | 3/8 [00:00<00:00, 15.12it/s]Working on 5
62%|██████▎ | 5/8 [00:00<00:00, 16.29it/s]Working on 6
Working on 7
100%|██████████| 8/8 [00:00<00:00, 19.27it/s]
[0, 1, 2, 3, 4, 5, 6, 7]
while there's still an issue (on both py2 and py3) with multiprocessing not sharing tqdm._instances:
from time import sleep
from tqdm import tqdm
from multiprocessing import Pool
def foo(a):
tqdm.write("Working on %d" % a)
sleep(0.1)
return a
tqdm.get_lock() # ensures locks exist
p = Pool(2)
list(tqdm(p.imap(foo, range(8)), total=8))
p.close()py3:
Working on 0
Working on 1
0%| | 0/8 [00:00<?, ?it/s]
Working on 2
Working on 3
Working on 4
Working on 5
38%|███▍ | 3/8 [00:00<00:00, 15.05it/s]
Working on 6
62%|██████▋ | 5/8 [00:00<00:00, 16.25it/s]
Working on 7
100%|██████████| 8/8 [00:00<00:00, 19.98it/s]py2:
Working on 0
Working on 1
0%| | 0/8 [00:00<?, ?it/s]Working on 2
Working on 3
12%|█▎ | 1/8 [00:00<00:00, 9.96it/s]Working on 4
Working on 5
38%|███▊ | 3/8 [00:00<00:00, 11.72it/s]Working on 6
Working on 7
100%|██████████| 8/8 [00:00<00:00, 19.28it/s]As of now it means that:
- py3 threading is fully supported
- py2 threading, py2/3 multiprocessing require things like
set_lockand evenposition- can this be fixed/made easier?
- documentation needs updating