ComboboxControl: Handle Unicode characters when matching values #70180
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
Mamaduka
commented
May 21, 2025
Comment on lines
+174
to
+180
| expect( normalizeTextString( '①' ) ).toBe( '1' ); | ||
| expect( normalizeTextString( 'Ⅸ' ) ).toBe( 'ix' ); | ||
| expect( normalizeTextString( 'MC²' ) ).toBe( 'mc2' ); | ||
| expect( normalizeTextString( 'ＰｌａｙＳｔａｔｉｏｎ ２' ) ).toBe( | ||
| 'playstation 2' | ||
| ); | ||
| expect( normalizeTextString( 'ＡＢＣ' ) ).toBe( 'abc' ); |
There was a problem hiding this comment.
It would be nice to add more cases here, and I'm open to suggestions.
There was a problem hiding this comment.
Maybe one for accent removal, which is part of the function? Like "Amélie".
There was a problem hiding this comment.
That's handled by the removeAccents library (props @tyxla), but I'm happy to add coverage.
There was a problem hiding this comment.
Yeah, we have a bunch of tests there, but having an extra test here can't hurt. I remember duplicating a bunch of WP core backend tests of remove_accents to ensure this works as expected.
There was a problem hiding this comment.
Thanks for confirmation, @tyxla!
|
cc @WordPress/gutenberg-components |
|
Size Change: +41 B (0%) Total Size: 1.85 MB
ℹ️ View Unchanged
|
|
Flaky tests detected in 5961dfd. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/15211583508
|
291c5e8 to
d817449
Compare
t-hamano
reviewed
May 23, 2025
There was a problem hiding this comment.
As far as I understand, this PR only affects the ComboboxControl component for now.
I tested it on Storybook with the following changes:
diff --git a/packages/components/src/combobox-control/stories/index.story.tsx b/packages/components/src/combobox-control/stories/index.story.tsx
index f033742a33..3021d1c437 100644
import type { ComboboxControlProps } from '../types';
const countries = [
+ { name: 'ＰｌａｙＳｔａｔｉｏｎ ２', code: 'NA' },
{ name: 'Afghanistan', code: 'AF' },
{ name: 'Åland Islands', code: 'AX' },
{ name: 'Albania', code: 'AL' },The filtering certainly works as expected, but the highlighting of matched characters doesn't seem to work properly.
We can search for full-width characters with half-width characters, but the Ｐｌａｙ should be highlighted instead of the Ｐｌａ.
The following change seems to solve the problem.
diff --git a/packages/components/src/form-token-field/suggestions-list.tsx b/packages/components/src/form-token-field/suggestions-list.tsx
index 1339e0cc18..23be6b7b5b 100644
--- a/packages/components/src/form-token-field/suggestions-list.tsx
+++ b/packages/components/src/form-token-field/suggestions-list.tsx
@@ -79,6 +79,7 @@ export function SuggestionsList<
const transformedSuggestion = displayTransform( suggestion );
const indexOfMatch = transformedSuggestion
+ .normalize( 'NFKC' )
.toLocaleLowerCase()
.indexOf( matchText );|
Good catch, @t-hamano! I'll update the Combobox component. |
|
Pushed changes based on feedback. I've also updated PR's title and description to reflect changes better. |
|
Sorry, I just found one more thing that we might need to fix: the In other words, I think this PR fixes the Storybook changes: diff --git a/packages/components/src/form-token-field/stories/index.story.tsx b/packages/components/src/form-token-field/stories/index.story.tsx
index 52daabe560..3d77d93177 100644
--- a/packages/components/src/form-token-field/stories/index.story.tsx
+++ b/packages/components/src/form-token-field/stories/index.story.tsx
@@ -35,6 +35,7 @@ const meta: Meta< typeof FormTokenField > = {
export default meta;
const continents = [
+ 'ＰｌａｙＳｔａｔｉｏｎ ２',
'Africa',
'America',
'Antarctica',Suggested changes: diff --git a/packages/components/src/form-token-field/index.tsx b/packages/components/src/form-token-field/index.tsx
index 987c75d769..42a45063ee 100644
--- a/packages/components/src/form-token-field/index.tsx
+++ b/packages/components/src/form-token-field/index.tsx
@@ -516,7 +516,10 @@ export function FormTokenField( props: FormTokenFieldProps ) {
match = match.toLocaleLowerCase();
_suggestions.forEach( ( suggestion ) => {
- const index = suggestion.toLocaleLowerCase().indexOf( match );
+ const index = suggestion
+ .normalize( 'NFKC' )
+ .toLocaleLowerCase()
+ .indexOf( match );
if ( normalizedValue.indexOf( suggestion ) === -1 ) {
if ( index === 0 ) {
startsWithMatch.push( suggestion );Beforebefore.mp4Afterafter.mp4 |
t-hamano
approved these changes
May 23, 2025
There was a problem hiding this comment.
Once this comment is addressed, we think I can ship this PR👍
|
Almost forgot. Thanks, everyone, for the help 🙇 |
chriszarate
pushed a commit
to chriszarate/gutenberg
that referenced
this pull request
Jul 1, 2025
…Press#70180) Co-authored-by: Mamaduka <mamaduka@git.wordpress.org> Co-authored-by: mirka <0mirka00@git.wordpress.org> Co-authored-by: tyxla <tyxla@git.wordpress.org> Co-authored-by: t-hamano <wildworks@git.wordpress.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What?
Related #70169.
PR improves unicode handling in the
normalizeTextStringhelper method, currently used byComboboxControlcomponentWhy?
See: #70169 (comment).
Testing Instructions
Confirm that new unit tests are correct and CI checks are passing.