Skip to content

Alignment issue when passing arrays though mmap #563

@aseyboldt

Description

@aseyboldt

Arrays that are transfered to subprocesses though mmap appear not always to be aligned:

import joblib
import numpy as np

def func(x):
    print(type(x))
    print(x.flags)
    print(x.ctypes.data / 8)

if __name__ == '__main__':
    # Make the array larger than 1Mbyte so that mmap is used
    n = 1024 ** 2 // 8 + 1
    x = np.random.randn(n)
    jobs = joblib.Parallel(2)
    print(x.nbytes)
    res = jobs([joblib.delayed(func)(x), joblib.delayed(func)(x)])

This returns

<class 'numpy.core.memmap.memmap'>
1048584
  C_CONTIGUOUS : True
  F_CONTIGUOUS : True
  OWNDATA : False
  WRITEABLE : False
  ALIGNED : False
  UPDATEIFCOPY : False
570054683.375

Note the last line, the address of the first element of the array is indeed not aligned.

I'm using version 0.11, on OSx sierra.

This came up in pymc-devs/pymc#2640, and seems to happen on several OSs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions