-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Closed
Labels
duplicate 🗐Seen it beforeSeen it beforesynchronisation ⇶Multi-thread/processingMulti-thread/processing
Description
python 3.6.4 from Anaconda distribution
OS X 10.13.3
tqdm 4.11.2
I have the following code:
def populate(desc, events, ids):
remaining = len(ids)
for _ in tqdm(range(0, ((remaining + 999) // 1000)), desc=desc):
batch = ids[:1000]
ids = ids[1000:]
result = get_batch(batch)
events.insert_many(result)
ids is a list of strings, and events is a MongoDB collection; I'm doing some processing to get data for each ID and then populating the collection with the data and want a progress bar.
The above is called from this code:
if nthreads > 1:
threads = []
chunksize = (len(ids) + nthreads - 1) // nthreads
for i in range(0, nthreads):
t = threading.Thread(target=populate, args=(str(i), events, ids[(chunksize * i):(chunksize * (i + 1))]))
threads.append(t)
t.start()
else:
populate('-', events, ids)
It works fine if nthreads
is 1, but if it is greater than one I get:
Exception in thread Thread-4:
Traceback (most recent call last):
File "/Users/gwheeler/anaconda/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/Users/gwheeler/anaconda/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "backfill_edd.py", line 14, in populate
for _ in tqdm(range(0, ((remaining + 999) // 1000)), desc=desc):
File "/Users/gwheeler/anaconda/lib/python3.6/site-packages/tqdm/_tqdm.py", line 732, in __init__
self.pos = self._get_free_pos(self) if position is None else position
File "/Users/gwheeler/anaconda/lib/python3.6/site-packages/tqdm/_tqdm.py", line 391, in _get_free_pos
return max(inst.pos for inst in cls._instances
File "/Users/gwheeler/anaconda/lib/python3.6/site-packages/tqdm/_tqdm.py", line 392, in <genexpr>
if inst is not instance) + 1
AttributeError: 'tqdm' object has no attribute 'pos'
Metadata
Metadata
Assignees
Labels
duplicate 🗐Seen it beforeSeen it beforesynchronisation ⇶Multi-thread/processingMulti-thread/processing