Skip to content

fix: enhance marshalUint64 to trim leading zeros and ensure one byte #513

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

wregen
Copy link
Contributor

@wregen wregen commented Jun 26, 2025

fix: enhance marshalUint64 to trim leading zeros and ensure at least one byte is returned

…one byte is returned

Signed-off-by: wregen <W.Regenczuk@gmail.com>
@Pieras2
Copy link

Pieras2 commented Jun 26, 2025

@soniah can I ask you to look at it?

@SuperQ
Copy link
Contributor

SuperQ commented Jun 26, 2025

What issue does this fix?

return []byte{0} // Always return at least one byte
}

if trimmed[0]&0x80 > 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this doing?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same functionality is implemented in marshalUint32 function

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strange, I wonder why it was implemented only for 32-bit.

@Pieras2
Copy link

Pieras2 commented Jun 26, 2025

What issue does this fix?

I’m currently out of office and shortly for now I can mention what I remember

  • unmarshall was not returning anything for zero value while snmp requires value even if zero

Tommorow I will ask my developer for technical justification of this change and some snmp spec refering to it.

With this change my snmp simulator with your library successfully discover my devices (represented by *.snmprec files)

@SuperQ
Copy link
Contributor

SuperQ commented Jun 26, 2025

I think I understand the first part, always returning a single empty byte. But I'm not sure about the second half.

@Pieras2
Copy link

Pieras2 commented Jun 26, 2025

I think I understand the first part, always returning a single empty byte. But I'm not sure about the second half.

I asked AI to explain as I cannot repeat exactly my college (I will ask him tommorow to confirm)

🔧 The Issue

In helper.go, the unmarshalInt64 implementation had a subtle bug: when decoding a value of zero, it was not returning 0 properly. Instead, due to how length or payload were interpreted, it returned an error or a nil value. The correct behavior—as per ASN.1 BER encoding—is to return 0 when the integer data is exactly zero.

✅ What the Fixed Version Does
1. Detects “zero-length” or empty payloads
If the BER length indicates zero (no payload), the code now returns the integer 0. That’s the correct representation for 0.
2. Properly parses byte payload
Reads each byte of the payload (1 to 8 bytes, inclusive), shifting bits and building the integer.
3. Handles sign extension correctly
If the first payload byte has its highest bit set (e.g. 0x80), the code ensures the integer is interpreted as a signed value, adjusting the result to negative as intended.
4. Supports 8-bit and up to 64-bit integers
The loop reads up to 8 bytes, so small values (like a single zero byte 0x00) or full 8-byte values (e.g. 0x00…80) work correctly.

@SuperQ
Copy link
Contributor

SuperQ commented Jun 26, 2025

I'm pretty sure the second half of this is unnecessary, as the binary.BigEndian function does the necessary bit handling.

If this is not correct, tests need to be updated in order to prove the change does the correct thing.

Comment on lines +338 to +339
if len(trimmed) == 0 {
return []byte{0} // Always return at least one byte
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should really add some tests for marshalUint64().

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by the way thank you for prompt response

@Pieras2
Copy link

Pieras2 commented Jun 26, 2025

I'm pretty sure the second half of this is unnecessary, as the binary.BigEndian function does the necessary bit handling.

If this is not correct, tests need to be updated in order to prove the change does the correct thing.

I will check tommorow if it works without this part however this won’t be a proof that it won’t break some other scenario. I will ask my college to comment that.

By the way … yes, tests are crutial :-) I’m a QA/QC engineer and write a bit some tests software.

@miko12215
Copy link

miko12215 commented Jun 27, 2025

Current implementation did not handle case properly when Counter64 value was 0 or 255(The MSB is negative):

func marshalUint64(v interface{}) []byte {
	bs := make([]byte, 8)
	source := v.(uint64)
	binary.BigEndian.PutUint64(bs, source) // will panic on failure
	// truncate leading zeros. Cleaner technique?
	return bytes.TrimLeft(bs, "\x00")
}

This implementation for 0 returns nil.

Problem with arraises in marshall.go when writing to buffer:

intBytes := marshalUint64(pdu.Value)
tmpBuf.WriteByte(byte(len(intBytes)))
tmpBuf.Write(intBytes)

the len(nil) is 0, but actuall value is sent so length byte is missing:
e.g.
[48 12 6 8 43 6 1 2 1 1 1 0 70 0]
[48 13 6 8 43 6 1 2 1 1 1 0 70 1 0]

Lets look into 0 and 255 case in new implementation:

  • 0:
value := uint64(0)

bs := make([]byte, 8)  // [0, 0, 0, 0, 0, 0, 0, 0]

binary.BigEndian.PutUint64(bs, 0)  // [0, 0, 0, 0, 0, 0, 0, 0]

trimmed := bytes.TrimLeft(bs, "\x00")  // []
// This DOES get trimmed! All zeros are removed.

if len(trimmed) == 0 {  // true, length is 0
    return []byte{0}
}

// Result: [0] ----- previously it would be nil
  • 255:
value := uint64(255)

bs := make([]byte, 8)  // [0, 0, 0, 0, 0, 0, 0, 0]

binary.BigEndian.PutUint64(bs, 255)  // [0, 0, 0, 0, 0, 0, 0, 255]

trimmed := bytes.TrimLeft(bs, "\x00")  // []
// This DOES get trimmed! All zeros are removed.

if len(trimmed) == 0 {  // true, length is 0
    return []byte{0}
}
//this condition now is true -> we prepend leading 0 so it is not treated as negative according to BER
if trimmed[0]&0x80 > 0 {
		trimmed = append([]byte{0}, trimmed...)
	}
	return trimmed

// Result: [0, 255] ----- previously it would be [255]

Function parseUint64 checks for leading zeros assumming the size can be up to 9 bytes.

@wdukar
Copy link

wdukar commented Jun 30, 2025

As mentioned earlier, the change works as follows:

  • Encode the value in big endian format (using PutUint64).
  • Remove leading zero bytes on the left side (TrimLeft)—these bytes do not provide any information and should be removed according to BER number encoding rules.
  • If all bytes were zero (it means input value is 0) the array is empty, just return 0.
  • If the remaining bytes would represent a negative value (i.e., the leftmost bit is 1), prepend an additional zero byte to ensure the value is interpreted as positive. This is necessary to adjust the sign, since Counter64 is stored as a 64-bit unsigned integer and should always be positive (see below). BER encoding uses endian encoding, but there is no special treatment for unsigned numbers. To ensure the number is positive, it must not start with the leftmost bit set to 1. This is why the final condition prepends zero if necessary.
Counter64 ::=
    [APPLICATION 6]
        IMPLICIT INTEGER (0..18446744073709551615)

@SuperQ SuperQ requested a review from TimRots June 30, 2025 07:24
TimRots
TimRots previously requested changes Jul 8, 2025
Copy link
Member

@TimRots TimRots left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are on the right way by fixing this bug, but need to iterate on added comments as we then also improve on stability.

As Ben mentioned we are missing tests.

@@ -331,9 +331,18 @@ func marshalInt32(value int) ([]byte, error) {
func marshalUint64(v interface{}) []byte {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we decide to touch this. Lets make clear why this function is here, and how it should work as per spec.
I would prefer to add a comment like this above the function definition.

// marshalUint64 encodes a uint64 according to BER rules (X.690) for SNMP Counter64 values.
// It trims leading zeros and prepends 0x00 if the first byte would otherwise be interpreted as negative.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed to add about 5 pending comments here ... My apologies for that, I'll try to update it within ~ 24h.

@TimRots TimRots dismissed their stale review July 9, 2025 15:57

My review comments can be fixed in another iteration.

@TimRots TimRots merged commit 2977369 into gosnmp:master Jul 9, 2025
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants