Skip to content

Support multiple subinterpreters #993

@JukkaL

Description

@JukkaL

Python 3.12 will (most likely) support multiple subinterpreters with separate GILs. This improves performance on multicore systems and will enable many exciting options for sharing state between subinterpreters (not discussed here). I think that it's essential that mypyc will fully support multiple interpreters.

PEP 630 explains things we need to do to support multiple subinterpreters. Here's a summary based on my understanding so far:

  • Move all global C-level state in the generated C to a per-subinterpreter heap-allocated struct. We generally can't use C globals or statics in the generated code.
  • Add an implicit first argument with a pointer to the per-interpreter state to all native functions (or at least the ones that may need to access the global state).
  • All wrapper functions need to find the global state pointer somehow and pass it to the wrapped native function. For example, it can be accessed via type objects.
  • Make all Python types heap types and access the type objects through the per-interpreter state struct or through PyObject pointers (not through C globals).

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureSupporting previously unsupported Python, new native types, new features, etc.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions