CSV export email messages for pt_BR, zh_CN, zh_TW fall back to en_US, instead of using user language

[nagpai]

Email notifications for CSV exports for the following supported languages pt_BR, zh_CN, zh_TW fall back to English.

To Reproduce

1. Browse to WP Admin -> Users -> Profile
2. Change the profile language to any of the languages above.
3. Run a CSV export of transactions, payouts or disputes, making sure you have more than a single pagination to trigger a server export
4. Check the language of the email triggered, and the CSV contents. It should be in user language but is provided in US English.

Detailed observations here #10003 (review)

Additional context

• The server uses dash-separated locales e.g. pt-br, zh-cn, zh-TW
• WordPress core at the client side, uses locale in the format pt_BR, zh_CN etc.
• We have a utility function convert_to_server_locale() that roughly converts the WordPress core locale type to the server supported dash-separated locales.
• The CSV export POST request params incorrectly include the locale en_US. This is likely a bug that occurs when the request query params are prepared in convert_to_server_locale().
• The issue was found while creating the PR - CSV Exports: Simplify localization by setting export language to that in user profile settings #10003

[haszari]

Some context about these codes:

• There are ISO country codes – 2 chars, uppercase
• And ISO language codes – lower case, sometimes paired with a country code

We have a utility function convert_to_server_locale() that roughly converts the WordPress core locale type to the server supported dash-separated locales.

Another potential bug in that logic – what language would we use for emails to a site/merchant using Portuguese (Portugal)?

[Jinksi]

The server falls back to en_US if there is no matching locale found. This may be the reason for the inaccuracy, but we may need to investigate if there is any other cause.

I believe the bug causing this en_US fallback occurs before the request query params are prepared, in convert_to_server_locale().

[Jinksi]

what language would we use for emails to a site/merchant using Portuguese (Portugal)?

Transact-platform-server on WPCOM doesn't have complete Portuguese (Portugal) translations, so "English (US)" is used as a the default fallback.

It would be good to enumerate which translations we have available on server – filtering via translate.wordpress.com doesn't seem ideal. I know there is the following list in the client code, but is it accurate/maintained?

woocommerce-payments/includes/class-wc-payments-utils.php

Lines 1288 to 1305 in 5ba021b

	$supported = [
	'ar', // Arabic.
	'de', // German (Germany).
	'es', // Spanish (Spain).
	'fr', // French (France).
	'he', // Hebrew (Israel).
	'id', // Indonesian (Indonesia).
	'it', // Italian (Italy).
	'ja', // Japanese.
	'ko', // Korean.
	'nl', // Dutch (Netherlands).
	'pt-br', // Portuguese (Brazil).
	'ru', // Russian (Russia).
	'sv', // Swedish (Sweden).
	'tr', // Turkish (Turkey).
	'zh-cn', // Simplified, Singapore).
	'zh-tw', // Chinese Traditional (Taiwan).
	];

[haszari]

They may be the fallback language - English

Is this "sometimes" or does the bug consistently happen every time for these languages?

I'm assuming that it happens every time, in which case this is priority: high.

[Jinksi]

Is this "sometimes" or does the bug consistently happen every time for these languages?

In my testing using production transact-platform-server and Pressable, every time.

[haszari]

In my testing using production transact-platform-server and Pressable, every time.

Thanks for confirming! I strengthened the wording in reproduce steps 😁

[Jinksi]

We have a utility function convert_to_server_locale() that roughly converts the WordPress core locale type to the server supported dash-separated locales.

It looks like there's an opportunity to remove a bunch of complexity here.

1. convert_to_server_locale() is only used for the wcpaySettings.userLocale variable, so it might be better to move this logic client-side. userLocale is only used when preparing the locale query params for a request to download a CSV.
2. This WP Locale → WPCOM-compatible language code is available in the package @automattic/languages, replacing the need for a WooPayments-specific convert_to_server_locale().

Example language entry and relevant fields from @automattic/languages:

{
  "langSlug": "pt-br",
  "name": "Português do Brasil",
  "wpLocale": "pt_BR",
}

So, rather than maintaining our own mapping, we can get the langSlug based on the current user's wpLocale (already available via WP data getCurrentUser()) and send it as the locale query param when making CSV download POST requests.

[Jinksi]

So, rather than maintaining our own mapping, we can get the langSlug based on the current user's wpLocale (already available via WP data getCurrentUser()) and send it as the locale query param when making CSV download POST requests.

Note: I've decided to reduce the complexity of this solution and move the language code conversion logic to PHP, within the REST API controllers in #10197.

I have not found a PHP package/resource with the same list of WP Locale → WPCOM Language. However, in reducing the hardcoded list to only the three that are region-specific, the maintenance burden is reduced. This approach is common across A8C (GH search), of which I found Jetpack's to be the simplest, which I've adapted and used in #10197.