Skip to content

Conversation

lrq3000
Copy link
Member

@lrq3000 lrq3000 commented Dec 29, 2015

Add asynchronized coroutine support in tqdm as proposed in #65.

Code thank's to @aplavin, from here and here.

This code wasn't tested out, can someone devise an example case? (I'm inexperienced with Python coroutines :S).

@lrq3000 lrq3000 added the help wanted 🙏 We need you (discussion or implementation) label Dec 29, 2015
@codecov-io
Copy link

Current coverage is 96.86%

Merging #91 into master will decrease coverage by -3.14% as of f0abb3d

@@            master     #91   diff @@
======================================
  Files            5       5       
  Stmts          239     255    +16
  Branches        49      49       
  Methods          0       0       
======================================
+ Hit            239     247     +8
  Partial          0       0       
- Missed           0       8     +8

Review entire Coverage Diff as of f0abb3d

Powered by Codecov. Updated on successful CI builds.

@sametmax
Copy link

import asyncio

from tqdm import tqdm

async def main():
    async for x in tqdm(range(10), miniters=0):
        do_stuff()

loop = asyncio.get_event_loop()
loo.run_until_complete(main())

How ever, there is much more to consider to unit test it:

  • aiter is only awailable on 3.5 and requires async/await, which are SyntaxErrors on earlier versions. So the test must be in a separate files, and ignored by the test runner for python 3.4-.

  • anext should probably do yield from self.iterable instead of next(self.iterable) in case the iterable itself is asynchronous.

  • if you want to unit test it, you'll need to start the eventloop as a setUp and stop it as a tearDown. Libs like pytest-asyncio do it automatically for you.

  • if you need to mock an awaitable, do:

    import asyncio
    
    class AMock(Mock):
    
        async def __call__(self, *args, **kwargs):
            result = super().__call__(self, *args, **kwargs)
            async def coroutine():
                return result
            return asyncio.ensure_future(coro())
    

If that works well, you can also make an async context manager : https://docs.python.org/3/reference/datamodel.html#asynchronous-context-managers

@lrq3000
Copy link
Member Author

lrq3000 commented Jan 21, 2016

Wow thank you very much sametmax, this sure helps a lot ! I'll train a bit
with toy examples following your guidelines before making a unit test, and
ill change next() for the more pythonic syntax in my PR :)
Le 21 Jan. 2016 18:44, "sametmax" notifications@github.com a écrit :

import asyncio

from tqdm import tqdm

async def main():
async for x in tqdm(range(10)):
print(x)

loop = asyncio.get_event_loop()
loo.run_until_complete()

How ever, there is much more to consider to unit test it:

  • aiter is only awailable on 3.5 and requires async/await, which are
    SyntaxErrors on earlier versions. So the test must be in a separate files,
    and ignored by the test runner for python 3.4-.

  • anext should probably do "yield from self.iterable" instead of
    next() in case the iterable itself is asyncronous.

  • if you want to unit test it, you'll need to start the eventloop as a
    setUp and stop it as a tearDown. Libs like pytest-asyncio do it

    automatically for you.

    if you need to mock an awaitable, do:

    import asyncio

    class AMock(Mock):

    async def call(self, _args, *_kwargs):
    result = super().call(self, _args, *_kwargs)
    async def coroutine():
    return result
    return asyncio.ensure_future(coro())


Reply to this email directly or view it on GitHub
#91 (comment).

@AlJohri
Copy link
Contributor

AlJohri commented Jul 6, 2019

Just ran into this issue myself. Bumping this PR.

TypeError: 'async for' requires an object with __aiter__ method, got tqdm

tqdm/_tqdm.py Outdated
@@ -409,6 +409,20 @@ def __exit__(self, *exc):
self.close()
return False

@_coroutine
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this one should not be a coroutine.

@AlJohri
Copy link
Contributor

AlJohri commented Jul 6, 2019

This works for me:

from tqdm.auto import tqdm

def __aiter__(self):
    return self

async def __anext__(self):
    try:
        self.update()
        return await self.iterable.__anext__()
    except StopIteration:
        self.close()
        raise StopAsyncIteration
tqdm.__aiter__ = __aiter__
tqdm.__anext__ = __anext__

################################################

async for row in tqdm(con.cursor('SELECT * FROM pageviews ORDER BY user_id')):
    pass

@AlJohri AlJohri mentioned this pull request Jul 7, 2019
@casperdcl
Copy link
Member

ah nice, I suppose I should take over. @AlJohri could you write a unit test?

@casperdcl casperdcl self-assigned this Jul 7, 2019
@casperdcl casperdcl added p3-enhancement 🔥 Much new such feature submodule ⊂ Periphery/subclasses labels Jul 7, 2019
@Midnighter Midnighter mentioned this pull request Feb 13, 2020
@malarinv
Copy link

@casperdcl will this be merged soon?

@casperdcl casperdcl removed the help wanted 🙏 We need you (discussion or implementation) label Jul 11, 2020
@casperdcl casperdcl added the to-review 🔍 Awaiting final confirmation label Jul 11, 2020
@casperdcl casperdcl added this to the v5.0.0 milestone Jul 11, 2020
Signed-off-by: Stephen L. <lrq3000@gmail.com>
@casperdcl casperdcl linked an issue Jul 11, 2020 that may be closed by this pull request
@lrq3000 lrq3000 requested a review from casperdcl as a code owner July 11, 2020 18:14
@codecov-commenter
Copy link

codecov-commenter commented Jul 11, 2020

Codecov Report

Merging #91 into master will decrease coverage by 0.70%.
The diff coverage is 26.66%.

@@            Coverage Diff             @@
##           master      #91      +/-   ##
==========================================
- Coverage   87.16%   86.46%   -0.71%     
==========================================
  Files          21       21              
  Lines        1278     1293      +15     
  Branches      217      218       +1     
==========================================
+ Hits         1114     1118       +4     
- Misses        143      154      +11     
  Partials       21       21              

casperdcl added a commit that referenced this pull request Jul 11, 2020
@casperdcl casperdcl mentioned this pull request Jul 11, 2020
16 tasks
casperdcl added a commit that referenced this pull request Jul 16, 2020
casperdcl added a commit that referenced this pull request Jul 16, 2020
@casperdcl casperdcl mentioned this pull request Jul 16, 2020
@casperdcl casperdcl closed this Jul 16, 2020
@casperdcl casperdcl deleted the async-coroutine branch July 16, 2020 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
p3-enhancement 🔥 Much new such feature submodule ⊂ Periphery/subclasses to-review 🔍 Awaiting final confirmation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Asynchronous tqdm
7 participants