0.5.x -- ENH: JOBLIB_MULTIPROCESSING env var to disable multiprocessing from the environment #6
Conversation
…he environment + adjusts the tests to use multiprocessing as it is known to joblib.parallel (thus might be disabled per above) Note: apparently multiprocessing should not be imported module wide in test_paralell.py
|
Darn (the doctest failure). This failure makes sens, as when there is no parallel processing going on, the output is indeed not what corresponds to the doctest. I am afraid that I will have to convert this to a unit test. Unless I can make the interface more homogeneous, which might be a good thing anyhow. I'll merge your pull request and I'll have a look at that. |
|
Also, I'd like to systematically run tests in both conditions. If I add a Makefile for my own personal convenience that does that, is it of any inconvenience for Debian packaging? |
|
no -- that is ok, I can handle it ;) thanks for asking On Mon, 11 Apr 2011, GaelVaroquaux wrote:
=------------------------------------------------------------------= |
Convenience for testing with and without multiprocessing
|
apparently because merge wasn't pushed to the joblib/joblib I just need to update this pull request -- that test failed within clean chroots (cowbuild chroots) across platforms. Otherwise I am ready for 0.5.1 ;) |
|
and 1 more -- finally decided to read the docs ;-) |
|
It wasn't pushed because it is not ready (a failing test, as you mentionned, I am working on it). Thanks for your update. I'll have a look when I find time (this evening probably). |
|
All right. Merged. I'll do a release next time I find time to breath (tomorrow night?). |
|
cool! if you like, you could buzz me few hours before release, then I
|
|
You mean after release, but before announcement? I can do that. I could buzz you before the release to PyPI, but I would need to tag before you can create the deb, right? |
yeap, or even upload to pypi (I think there should be no surprising =------------------------------------------------------------------= |
|
OK, I got stuck in public transport without Internet connection, so I did the release. I just pushed the tag to github. |
|
ok -- uploaded to NeuroDebian (should be already available) and to Debian
neurodebian@head2: ../../joblib/0.5.1/joblib_0.5.1-1~nd09.04+1_amd64.build:IOError: [Errno 2] No such file or directory: '/tmp/tmpK7jrzk/joblib/joblib/test/test_memory/f/4ab6c9a599584035286137201aa497c7/output.pkl'../../joblib/0.5.1/joblib_0.5.1-1 Cheers, On Tue, 12 Apr 2011, GaelVaroquaux wrote:
|
Setting to 0 results now only in doctests failure:
====================================================================== FAIL: Doctest: joblib.parallel.Parallel ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.6/doctest.py", line 2163, in runTest raise self.failureException(self.format_failure(new.getvalue())) AssertionError: Failed doctest test for joblib.parallel.Parallel File "/home/yoh/deb/gits/joblib/joblib/parallel.py", line 145, in Parallel ---------------------------------------------------------------------- File "/home/yoh/deb/gits/joblib/joblib/parallel.py", line 262, in joblib.parallel.Parallel Failed example: out = Parallel(n_jobs=2, verbose=100, pre_dispatch='1.5*n_jobs')( delayed(sqrt)(i) for i in producer()) #doctest: +ELLIPSIS Expected: Produced 0 Produced 1 Produced 2 [Parallel(n_jobs=2)]: Done 1 out of 3+ |elapsed: ...s remaining: ...s Produced 3 [Parallel(n_jobs=2)]: Done ... out of 4+ |elapsed: ...s remaining: ...s ... Got: Produced 0 [Parallel(n_jobs=2)]: Done job 1 | elapsed: 0.0s Produced 1 [Parallel(n_jobs=2)]: Done job 2 | elapsed: 0.0s Produced 2 [Parallel(n_jobs=2)]: Done job 3 | elapsed: 0.0s Produced 3 [Parallel(n_jobs=2)]: Done job 4 | elapsed: 0.0s Produced 4 [Parallel(n_jobs=2)]: Done job 5 | elapsed: 0.0s Produced 5 [Parallel(n_jobs=2)]: Done job 6 | elapsed: 0.0s >> raise self.failureException(self.format_failure(.getvalue()))