Undefined behaviour in telfhash causes different results in big-endian machines. #1874
Description
Describe the bug
The test case below produces a different result in big-endian machines. Part of the issue was in the ELF module itself, but it was fixed in 32ae80d. However, after that fix I discovered another issue in the implementation of telfhash.
Lines 263 to 270 in 32ae80d
The telfhash produced for that test case in big-endian machines is:
T174B021188204F00184540770331E0B111373086019509C464D0ACE88181266C09774FA
Notice that they are very similar, except for the third byte in the hash, where the nibbles are swapped:
T174B012..
T174B021..
The issue is related to this union
yara/libyara/tlshc/tlsh_impl.h
Lines 63 to 71 in 32ae80d
Apparently the order of fields q1ratio and q2ratio depend on the machine endianness, but the those fields are later read using the union's qb member, which causes a different value for qb. This looks like telfhash is relying on undefined behaviour of the C compiler.
Screenshots
If applicable, add screenshots to help explain your problem.
Please complete the following information:
- OS: All, depend on machine endianness.
- YARA version: 4.3.0