@chrisbra, will this work for mb_strncmp()?

/*
 * Compare two multibyte strings up to "n" characters.
 * Case-sensitive version of mb_strnicmp().
 */
int mb_strncmp(char_u *s1, char_u *s2, size_t n)
{
    int c1, c2;
    size_t i = 0;

    while (i < n)
    {
        c1 = mb_ptr2char_adv(&s1);
        c2 = mb_ptr2char_adv(&s2);

        if (c1 != c2)
            return c1 - c2;
        if (c1 == NUL)  // end of string
            break;
        ++i;
    }
    return 0;
}

It should work for utf8 and DBCS. But, the comment says:

 * Needed for Big5, Shift-JIS and UTF-8 encoding.  Other DBCS encodings can
 * probably use strnicmp(), because there are no ASCII characters in the...

So, the above version may not work for Big5, etc. An alternative approach is to implement a utf_strncmp() function (below), and use it in mb_strncmp() modeled after mb_strnicmp(), but without the MB_TOLOWER() step.

/*
 * Compare two UTF-8 strings up to "n" Unicode characters.
 * Case-sensitive.
 */
int utf_strncmp(char_u *s1, char_u *s2, size_t n)
{
    int c1, c2;
    size_t i = 0;

    while (i < n)
    {
        if (*s1 == NUL && *s2 == NUL)
            return 0;

        c1 = utf_ptr2char((char_u *)s1);
        c2 = utf_ptr2char((char_u *)s2);

        if (c1 != c2)
            return c1 - c2;

        s1 += utf_ptr2len(s1);
        s2 += utf_ptr2len(s2);
        ++i;
    }

    return 0;
}

Makes sense. I'll include your version then

I'm confused. How does STRNCMP() not handle multi-byte correctly and why is utf_strncmp() needed? And using this utf_strncmp() here is wrong, as lastpatlen is the number of bytes, not the number of characters.

I'm confused. How does STRNCMP() not handle multi-byte correctly and why is utf_strncmp() needed? And using this utf_strncmp() here is wrong, as lastpatlen is the number of bytes, not the number of characters.

I think I was not clear. I meant using utf_strncmp() inside mb_strncmp:

    int
mb_strncmp(char_u *s1, char_u *s2, size_t nn)
{
...
    if (enc_utf8)
    {
	return utf_strncmp(...);
    }
    else
    {
	return STRNCMP(...)
   }
    return 0;
}

I assumed non-utf8 multi-code chars are different from utf8 multi-code (from reading the comments) and that strncmp() will not work for them. I assumed that is why strncmp() was not used earlier. Let me verify this with some tests.

@zeertzjq

STRNCMP() and STRCMP() do not appear to handle multi-byte characters correctly. Here's an example:

STRNCMP("é", "é", 2)

This returns 0, indicating the strings are equal—but they are not. The first string is a single character "é" (U+00E9), while the second is composed of two code points: "e" (U+0065) followed by a combining acute accent (U+0301).

In digraph notation:

• The first string is entered as Ctrl-K ' e
• The second string as e Ctrl-V u 0301

Since STRNCMP() compares raw bytes, it happens that the first two bytes match—even though the second string consists of three bytes in total. STRCMP() also gives an incorrect result for similar reasons.

EDIT: strcmp()/strncmp() also does not handle DBCS (Shift-JIS, Big5) correctly

STRNCMP() and STRCMP() do not appear to handle multi-byte characters correctly. Here's an example:

STRNCMP("é", "é", 2)

This case is not relevant here, since lastpatlen is the full length of lastpat, so it'll not compare only a part of a multibyte character.

STRCMP() also gives an incorrect result for similar reasons.

Example?

STRCMP() does handle multi-byte characters correctly—there was a bug in my test. The actual issue lies with STRNCMP(), which compares a fixed number of bytes and can misinterpret multi-byte sequences.

The correct solution seems to be to use STRCMP() or STRNCMP() (since lastpatlen will not split multibyte char).

Apologies for the earlier confusion.

That's why I initially used STRNCMP. Because even if it does not handle multi-byte chars correctly, it would cause the search indicator to be updated additionally. Okay, I'll revert it then. Thanks both

Thanks.