Skip to content

Add support for AVX512 #3536

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Mar 30, 2025
Merged

Add support for AVX512 #3536

merged 15 commits into from
Mar 30, 2025

Conversation

torusrxxx
Copy link
Member

This pull request adds support for AVX-512.
Now, only commands to read & write AVX-512 registers. UI will come next.

@torusrxxx
Copy link
Member Author

@mrexodia I need your suggestions on API changes and compatibility. Currently managing AVX-512 register contexts uses all new APIs in TitanEngine, x32dbg, and x32bridge. There will also be breaking changes to trace files (#3136).

When Intel “AVX-10” comes out with new registers R16-R31, YMM16-YMM31 and XMM16-XMM31 breaking changes will again be required. Therefore it seems better to create different REGDUMP structures for different compatibility and needs (#3116, #3136 and AVX-10) and use a new API interface like bool DbgGetRegDumpEx(REGDUMP* regdump, size_t size);

@mrexodia
Copy link
Member

Yeah I agree with doing DbgGetRegDumpEx. Honestly I thought we were already kinda doing that. The _dbg_xxx APIs do not need to be backwards compatible, so we can just introduce a new size parameter there perhaps?

Also are you good with moving to Visual Studio 2022 and Qt6? I think for a while we will keep support for Qt 5.6 and VS2013 for a while, but eventually move to modern standards. Doing two snapshots will break all Qt plugins, but I think it's really time. The CMake changes should make this pretty transparent, we just have to keep writing C++11 for now if we want XP compatibility.

@torusrxxx
Copy link
Member Author

What do you think of API interface in TitanEngine (and you will need to copy that code into GleeBug)?

I'm already using Visual Studio 2022, just not Qt 6 yet. I don't want to have a Qt account.

@mrexodia
Copy link
Member

What do you think of API interface in TitanEngine (and you will need to copy that code into GleeBug)?

Yeah it looks good, nothing controversial in there I think! I will fix the CI and merge it as well as port the changes to GleeBug.

I'm already using Visual Studio 2022, just not Qt 6 yet. I don't want to have a Qt account.

I will make sure to publish the extracted Qt installations as 7z archives on GitHub and SourceForge. Any other place I should publish it to make them easy to download?

There will also be breaking changes to trace files

I think while we're at it, there was some issue with an 8-bit index being used for memory sizes and this was causing some overflow with instructions like FXSAVE: https://pvs-studio.com/en/blog/posts/cpp/1146/#ID906A1478C3. We have a mechanism to version the trace files afaik, so that should also be a safe addition.

This weekend I will try to move x64dbg to a more modern stack, let's pray Microsoft won't decide to break ABI compatibility with their next Visual Studio release and then we should be set for the foreseeable future.

@torusrxxx
Copy link
Member Author

torusrxxx commented Mar 16, 2025

CI won’t work until changes in TitanEngine are merged. Currently GetThreadContext is called multiple times in TitanEngine, for general purpose registers, for AVX and for AVX-512 separately. In fact it should be called only once, that still needs to be fixed. So I’m also going to add a DbgGetRegDumpEx-like API interface in TitanEngine.

Edit: I just canceled this plan, decided not to change current TitanEngine API. Not many places need AVX-512, so don’t need to update everywhere just to get everything together with AVX-512 registers.

@mrexodia
Copy link
Member

Yeah my plan is to merge the TitanEngine/GleeBug changes and then add them as submodules to the x64dbg repo directory. This way we do not have to keep shipping binaries for updates like this...

@torusrxxx
Copy link
Member Author

Note: when testing AVX-512 register modification, remember #693. The program has to use AVX-512 registers once before they can be modified. I’ll see whether it can be fixed in TitanEngine.

@torusrxxx
Copy link
Member Author

Now it's possible to view ZMM registers in the RegistersView!
Because there are so many XMM, YMM and ZMM registers, these registers are now combined. If it has a nonzero value in the AVX512 part, then only ZMM registers are shown, otherwise if it has a nonzero value in the AVX part, then only YMM registers are shown, otherwise, just XMM registers are shown.

@torusrxxx
Copy link
Member Author

Now it's mostly complete except tracing support, which will be worked on later. I hope we will get it ready soon with a news update!

@mrexodia mrexodia marked this pull request as ready for review March 30, 2025 11:49
@mrexodia mrexodia merged commit 3caa50e into x64dbg:development Mar 30, 2025
3 checks passed
@mrexodia
Copy link
Member

I updated the TitanEngine.dll files for all debug engine. At a later date I will start including them in the CMake project.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants