Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

logger: fix UnicodeEncodeError with non-utf8 stdout in frozen envs #617

Merged
merged 3 commits into from
Jan 10, 2025

Conversation

glowinthedark
Copy link
Contributor

@glowinthedark glowinthedark commented Jan 9, 2025

Fix UnicodeEncodeError when logged output contains non-ascii characters such as e.g. Diccionario de la lengua española...

Only reproducible with files like this one when running a frozen executable binary such as builds generated with nuitka which seem to have a dumb stdout that does not support utf-8.

Example stacktrace

[INFO] Found 1 mdd files with 54164 entries
[INFO] extracting links...
[INFO] extracting links done, sizeof(linksDict)=64
[INFO] wordCount = 38521
[INFO] detectLangsFromName: langNames = ['German', 'Latin', 'Spanish']
[ERROR] Exception in Tkinter callback:
Traceback (most recent call last):
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/ui/ui_tk.py", line 182, in CallWrapper__call__
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/ui/ui_tk.py", line 1452, in convert
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/glossary_v2.py", line 1267, in convert
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/glossary_v2.py", line 1216, in convertV2
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/glossary_v2.py", line 1186, in _convertPrepare
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/glossary_info.py", line 222, in detectLangsFromName
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/logging/__init__.py", line 1539, in info
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/logging/__init__.py", line 1684, in _log
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/logging/__init__.py", line 1700, in handle
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/logging/__init__.py", line 1762, in callHandlers
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/logging/__init__.py", line 1028, in handle
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/logger.py", line 221, in emit
UnicodeEncodeError: 'ascii' codec can't encode character '\xf1' in position 105: ordinal not in range(128)
Traceback (most recent call last):
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/ui/ui_tk.py", line 182, in CallWrapper__call__
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/ui/ui_tk.py", line 1452, in convert
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/glossary_v2.py", line 1267, in convert
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/glossary_v2.py", line 1216, in convertV2
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/glossary_v2.py", line 1186, in _convertPrepare
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/glossary_info.py", line 222, in detectLangsFromName
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/logging/__init__.py", line 1539, in info
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/logging/__init__.py", line 1684, in _log
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/logging/__init__.py", line 1700, in handle
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/logging/__init__.py", line 1762, in callHandlers
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/logging/__init__.py", line 1028, in handle
  File "/Applications/PyGlossaryTK.app/Contents/MacOS/pyglossary/logger.py", line 221, in emit
UnicodeEncodeError: 'ascii' codec can't encode character '\xf1' in position 105: ordinal not in range(128)

@glowinthedark
Copy link
Contributor Author

glowinthedark commented Jan 9, 2025

@ilius not quite sure why this test is failing: g_ebook_epub2_test.py — any clue?

when I open the generated epub file some pages contain errors...

@ilius
Copy link
Owner

ilius commented Jan 9, 2025

I updated master branch.
You can rebase.

@ilius ilius merged commit 4a6556a into ilius:master Jan 10, 2025
10 checks passed
@glowinthedark glowinthedark deleted the bugfix/log-unicode-error branch January 10, 2025 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants