@casperdcl I got some trace info from AWS lambda, as you said, it is caused by locks.

I guess AWS Lambda does not fully support multiprocessing in python.

According to the docs (https://aws.amazon.com/cn/blogs/compute/parallel-processing-in-python-with-aws-lambda/), it says "Due to the Lambda execution environment not having /dev/shm (shared memory for processes) support, you can’t use multiprocessing.Queue or multiprocessing.Pool."

Clearly, even Lock has been removed.

I think it's better for tqdm to not rely on multiprocessing. Users who using multiprocessing should deal with the disordered IO problem by themselves. tqdm should not do.

{
  "errorMessage": "[Errno 38] Function not implemented",
  "errorType": "OSError",
  "stackTrace": [
    [
      "/var/task/lambda_function.py",
      8,
      "lambda_handler",
      "from tqdm import tqdm"
    ],
    [
      "/var/task/tqdm/__init__.py",
      1,
      "<module>",
      "from ._tqdm import tqdm"
    ],
    [
      "/var/task/tqdm/_tqdm.py",
      53,
      "<module>",
      "mp_lock = mp.Lock()  # multiprocessing lock"
    ],
    [
      "/var/lang/lib/python3.6/multiprocessing/context.py",
      67,
      "Lock",
      "return Lock(ctx=self.get_context())"
    ],
    [
      "/var/lang/lib/python3.6/multiprocessing/synchronize.py",
      163,
      "__init__",
      "SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)"
    ],
    [
      "/var/lang/lib/python3.6/multiprocessing/synchronize.py",
      60,
      "__init__",
      "unlink_now)"
    ]
  ]
}

Uh oh!

tqdm no longer works on aws lambda #466

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions