Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modification to syntax.js parser RegEx to provide better Unicode support #4390

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

AndyKilmory
Copy link
Collaborator

@AndyKilmory AndyKilmory commented Jan 8, 2025

What does this change?

Currently if a user inputs unicode characters into the general search string (e.g Möbius) then after the user has viewed a matching image and returned to the grid view via 'Back to Search' (or performed a page refresh) the unicode characters are lost from the search bar content...

Screenshot 2025-01-08 at 10 23 28

This PR corrects this problem by modifying the parser RegEx that is used to analyse the query content to create the structured query including chips. After implementation the resulting search bar content retains the unicode characters...

Screenshot 2025-01-08 at 10 24 50

The characters are also retained within chips within the search bar.

How should a reviewer test this change?

Ensure that any unicode characters entered into the search bar are retained on page refresh.

Who should look at this?

Tested? Documented?

  • locally by committer
  • locally by Guardian reviewer
  • on the Guardian's TEST environment
  • relevant documentation added or amended (if needed)

@AndyKilmory AndyKilmory requested review from a team as code owners January 8, 2025 10:32
@paperboyo
Copy link
Contributor

This regex is pure evil: quotes disappear, chips to arbitrary metadata are broken, ligatures…

But thank you, it fixes diacritics at least. So (almost) fixes #1900.
🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants