Skip to content

Compilation failure on MSVC when changing the default calling convention #403

@marzer

Description

@marzer

Hello,

I'm trying to use meshoptimizer in a Visual Studio project that overrides the default calling convention with __vectorcall (/Gv), and seeing compilation failures in src/meshoptimizer.h:

1>meshoptimizer.h(732,23): error C2440: 'initializing': cannot convert from 'overloaded-function' to 'void (__vectorcall *)(void *)'
1>meshoptimizer.h(732,23): error C2440: template <typename T> void (*meshopt_Allocator::StorageT<T>::deallocate)(void*) = operator delete;

This appears to be because the MSVC C++ libs explicitly mark operator new and operator delete as __cdecl, rather than implicitly picking up the calling convention from the compiler settings. Since your allocator/deallocate free-function pointers don't specify a calling convention, they get __vectorcall implicitly in this scenario, and thus become incompatible with operator new.

Explicitly changing the calling convention back to __cdecl for the translation unit allows it to compile OK, but then causes linker errors because the other translation units that include meshoptimizer.h are still set to __vectorcall. The only path I can see to using the library in its current form is to change the whole project back to __cdecl, but that comes at a runtime performance cost I'd like to avoid.

(I could also get around it by wrapping the library it in a separately-compiled library and forwarding all the calls through shims with explicit calling conventions, but that's clearly a big headache.)

Would you be amenable to adding a user-configurable #define for explicitly specifying the calling convention of alloc+dealloc function pointers? I'd be happy to prepare a PR for this myself.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions