Skip to content

Issues with Notebook + multiprocessing #1133

@andytaylor823

Description

@andytaylor823

I posted a question on StackOverflow with the same info, so feel free to refer to that as well: https://stackoverflow.com/questions/66372005/showing-tqdm-progress-bar-while-using-python-multiprocessing

I am trying to run N processes using Python's multiprocessing library inside a Jupyter notebook environment, and I would like to show N progress bars using tqdm, one bar for each process. The code outlined here (https://github.com/tqdm/tqdm#nested-progress-bars) does not work out-of-the-box for me in a notebook environment -- it appears this was written for a console or script environment.

Here is some sample code I would like to have work:

from tqdm import notebook
import time
def foo2(id):
    total = 100
    with notebook.tqdm(total=total, position=id) as pbar:
        for _ in range(0, total, 5):
            pbar.update(5)
            time.sleep(0.1)

plus

%%time
from multiprocessing import Pool
pool = Pool(5)
pool.map(foo2, range(5))
pool.close()
pool.join()

This does not show the progress bars as intended. However, running the same code with a ThreadPool worker set from multiprocessing is able to show the progress bars as intended.

%%time
from multiprocessing.pool import ThreadPool
pool = ThreadPool(5)
pool.map(foo2, range(5))
pool.close()
pool.join()

Is it possible to update tqdm such that it is possible to display several multiple independent progress bars simultaneously in a notebook environment?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions