pickling breaks between joblib version 0.11 and 0.12.2 #747
Description
For my script below, will someone please tell me why the result of call_and_shelve() can be pickled in version 0.11, but not in version 0.12.2? Or tell me what I am doing wrong, or if this is a bug...
Thanks.
I have this little script
from joblib import Memory
import numpy as np
import pickle
cachedir = 'PATH_TO_CACHE'
memory = Memory(cachedir, verbose=0)
@memory.cache
def g(x):
print('A long-running calculation, with parameter %s' % x)
return np.hamming(x)
result = g.call_and_shelve(4)
print('\nMemorized result: {}'.format(result))
canned_pickle = pickle.dumps(result)
open_can = pickle.loads(canned_pickle)
print('Depickled: {}\n'.format(open_can))With joblib:0.11 everything works and I get this output:
Memorized result: MemorizedResult(cachedir="/Users/mpsommer/Documents/python/job_lib_cache/joblib", func="__main__--Users-mpsommer-Documents-python-job_lib/g-alias", argument_hash="f7c5defad10d1a7505df243840ad5209")
Depickled: MemorizedResult(cachedir="/Users/mpsommer/Documents/python/job_lib_cache/joblib", func="__main__--Users-mpsommer-Documents-python-job_lib/g-alias", argument_hash="f7c5defad10d1a7505df243840ad5209")With joblib:0.12.2 the pickling breaks and I get this output:
Memorized result: MemorizedResult(location="/Users/mpsommer/Documents/python/job_lib_cache/joblib", func="<function g at 0x104448aa0>", args_id="f7c5defad10d1a7505df243840ad5209")
Traceback (most recent call last):
File "job_lib.py", line 15, in <module>
canned_pickle = pickle.dumps(result)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 1374, in dumps
Pickler(file, protocol).dump(obj)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 331, in save
self.save_reduce(obj=obj, *rv)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 401, in save_reduce
save(args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 562, in save_tuple
save(element)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 286, in save
f(self, obj) # Call unbound method with explicit self
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/pickle.py", line 753, in save_global
(obj, module, name))
pickle.PicklingError: Can't pickle <function g at 0x104448aa0>: it's not the same object as __main__.g