Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZEO.asyncio: switch to async/await style #195

Closed
wants to merge 87 commits into from
Closed
Show file tree
Hide file tree
Changes from 86 commits
Commits
Show all changes
87 commits
Select commit Hold shift + click to select a range
b577b7a
fix problems releaved by the test suite
d-maurer Apr 2, 2022
dc698a7
make doctest happy
d-maurer Apr 2, 2022
8c46a0e
- minor cleanups
dataflake Apr 4, 2022
655b1e3
- remove pypy from tox configuration [ci skip]
dataflake Apr 4, 2022
aa7dda1
limit connection waiting (to reduce test time in case of failure)
d-maurer Apr 5, 2022
b089a75
Merge branch 'asyncio3.5+' of https://github.com/zopefoundation/ZEO i…
d-maurer Apr 5, 2022
d651533
Update src/ZEO/tests/ConnectionTests.py
d-maurer Apr 5, 2022
c644ba2
`uvloop` workaround
d-maurer Apr 5, 2022
2faae53
Merge branch 'asyncio3.5+' of https://github.com/zopefoundation/ZEO i…
d-maurer Apr 8, 2022
7f68329
remove `msgpack` version restriction (no longer necessary as Python 2…
d-maurer Apr 8, 2022
9187165
Merge branch 'master' into asyncio3.5+
dataflake Apr 8, 2022
a9d9692
- fix merge issue [ci skip]
dataflake Apr 8, 2022
33639b9
drop tests with some storages to save test time
d-maurer Apr 9, 2022
4cacde8
bring `tox.ini` in line with `test_requires` in `setup.py`
d-maurer Apr 11, 2022
62efb98
fix connection race condition
d-maurer Apr 11, 2022
73c7871
Merge branch 'asyncio3.5+' of https://github.com/zopefoundation/ZEO i…
d-maurer Apr 11, 2022
c523b85
the `asyncio` tests observe from an external thread activity controll…
d-maurer Apr 11, 2022
814cc77
fix "wait||ready-connected` race"
d-maurer Apr 12, 2022
1283aef
Test loop observation: switch from "wait(DELAY)" (prone to race condi…
d-maurer Apr 12, 2022
4e20020
make `flake8` happy
d-maurer Apr 12, 2022
937d29c
Merge branch 'master' into asyncio3.5+
dataflake Apr 12, 2022
e9360d7
make `prefetch` more reliable:
d-maurer Apr 14, 2022
65dc1b6
Merge branch 'asyncio3.5+' of https://github.com/zopefoundation/ZEO i…
d-maurer Apr 14, 2022
defe92f
optimize prefetch
d-maurer Apr 14, 2022
5059a2c
Python 3.10 kills a coroutine started with `run_coroutine_threadsafe`…
d-maurer Apr 14, 2022
a1e8e66
make `flake8` happy
d-maurer Apr 14, 2022
98cecdf
Python 3.10 has removed the `loop` parameter
d-maurer Apr 14, 2022
574cc35
improve `close`
d-maurer Apr 19, 2022
c46a0da
`zeo-fan-out` race condition hunting; documentation improvements
d-maurer Apr 20, 2022
bf84637
cleanup -- now that we have understood the race condition
d-maurer Apr 20, 2022
cc3a606
improve flow control test
d-maurer Apr 20, 2022
2c5c167
redesign client close logic (with `asyncio` `close` starts the closin…
d-maurer May 6, 2022
2e27b85
Avoid additional `ResourceWarning`s (2 reamin due to the weakness of …
d-maurer May 11, 2022
504a95c
fix problems for Python before 3.8
d-maurer May 12, 2022
ebc229b
try to fix https://github.com/zopefoundation/ZEO/pull/195#issuecommen…
d-maurer May 13, 2022
1f61e73
add further closing tests
d-maurer May 16, 2022
ff8f0f7
capture log message in `client-config.test`
d-maurer May 16, 2022
72d815d
- experiment trying to prevent duplicate workflow runs
dataflake May 16, 2022
bdb31ce
- try to fix syntax
dataflake May 16, 2022
7cea602
improve log capturing: do not consider the registration failure log e…
d-maurer May 16, 2022
9a85643
Merge branch 'asyncio3.5+' of https://github.com/zopefoundation/ZEO i…
d-maurer May 16, 2022
9a543bc
uvloop workaround to prevent `close` deadlock
d-maurer May 17, 2022
09a4ece
clean up `tests_require` in `setup.py`
d-maurer May 18, 2022
868f189
Drop Python2 support
d-maurer May 18, 2022
61818b4
Allow to specify the process type of ZEO server processes with the en…
d-maurer May 19, 2022
0c23b5b
replace `mock` by `unittest.mock`
d-maurer May 19, 2022
80f3928
fix `unittest.mock` insufficiency in Python 3.5
d-maurer May 20, 2022
b17cdc4
improve test `zeo-fan-out.test`
d-maurer May 21, 2022
1728747
drop Python 3.5 support
d-maurer May 21, 2022
b3313fd
remove 3.5 from `tox.ini` as well (it will no longer work)
d-maurer May 21, 2022
7b77b2b
log flowcontrol (as it might be responsible for the "request blob" ha…
d-maurer May 22, 2022
463e0a1
Drop use of six
navytux May 23, 2022
93b1a81
Drop use of mock egg
d-maurer May 23, 2022
a586ef4
drop Python 3.5 support
d-maurer May 21, 2022
d455858
Drop use of manuel
navytux May 23, 2022
8060b6d
No longer pin msgpack < 1
navytux May 23, 2022
7763458
Merge branch 'master' into y/py3.1
navytux May 23, 2022
ff75bca
Sync with master
navytux May 23, 2022
cac3639
Sync with PR#200 (y/py3)
navytux May 23, 2022
e244529
optimization: switch back to futures which directly call (rather tha…
d-maurer May 27, 2022
e1cac43
Merge branch 'asyncio3.5+' of https://github.com/zopefoundation/ZEO i…
d-maurer May 27, 2022
214e209
fix for Python 3.7-
d-maurer May 27, 2022
c82021c
Merge branch 'master' into y/py3.1
navytux May 31, 2022
39ee68f
Sync with PR#200 (y/py3)
navytux May 31, 2022
28eaee5
Restore server_sync functionality
navytux May 31, 2022
72e6237
Optimize `SizedMessageProtocol` regarding readability and efficiency:
d-maurer Jun 1, 2022
8c9fa52
Merge branch 'asyncio3.5+' of https://github.com/zopefoundation/ZEO i…
d-maurer Jun 1, 2022
1ffc106
break reference cycle introduced by `ClientStorage.sync`
d-maurer Jun 1, 2022
2f125ec
avoid (some) cyclic garbage in server implementation
d-maurer Jun 2, 2022
d432ff2
simplify (and slightly optimize) `TaskExecutor`
d-maurer Jun 6, 2022
a143d41
move cache access out of `ClientIO` into `Protocol` (to ensure that w…
d-maurer Jun 9, 2022
2d62007
`cythonize`d `SizedMessageProtocol` and future/task optimizations (`s…
d-maurer Jun 9, 2022
e370854
move cache access out of `ClientIO` into `Protocol` (to ensure that w…
d-maurer Jun 9, 2022
b137352
own `Future` implementation
d-maurer Jun 9, 2022
1a51f29
minor optimization
d-maurer Jun 9, 2022
b2b2486
minor optimization
d-maurer Jun 9, 2022
6fad76c
inline state constants -- for speed
d-maurer Jun 10, 2022
6ad732a
optimize `cython` implementation for `SizedMessageProtocol`
d-maurer Jun 10, 2022
58182b9
optimize asynchronous calls to the server:
d-maurer Jun 11, 2022
b77847f
simplification and cleanup
d-maurer Jun 13, 2022
2ecbd41
provide `cython` optimizations
d-maurer Jun 13, 2022
163e85d
update `CHANGES.rst`
d-maurer Jun 14, 2022
9bb7c08
Python 3.6 compatibility
d-maurer Jun 14, 2022
afae7a5
optimization: suggest thread switches at main <-> IO thread boundaries;
d-maurer Jun 16, 2022
ac97508
fix doctest
d-maurer Jun 16, 2022
d41902b
optimization: suggest thread switches via `sleep(1us)` (rather than `…
d-maurer Jun 16, 2022
30e271b
optimization: do not suggest a thread switch for asynchronous calls: …
d-maurer Jun 18, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 1 addition & 6 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,20 +21,14 @@ jobs:
config:
# [Python version, tox env]
- ["3.9", "lint"]
- ["2.7", "py27"]
- ["3.5", "py35"]
- ["3.6", "py36"]
- ["3.7", "py37"]
- ["3.8", "py38"]
- ["3.9", "py39"]
- ["3.10", "py310"]
- ["pypy-2.7", "pypy"]
- ["pypy-3.7", "pypy3"]
- ["3.9", "docs"]
- ["3.9", "coverage"]
- ["2.7", "py27-msgpack1"]
- ["2.7", "py27-zeo4"]
- ["2.7", "py27-zodbmaster"]
- ["3.7", "py37-msgpack1"]
- ["3.7", "py37-uvloop"]
- ["3.7", "py37-zodbmaster"]
Expand All @@ -47,6 +41,7 @@ jobs:
- ["pypy-3.7", "pypy3-msgpack1"]

runs-on: ${{ matrix.os }}-latest
if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name != github.event.pull_request.base.repo.full_name
name: ${{ matrix.config[1] }}
steps:
- uses: actions/checkout@v2
Expand Down
10 changes: 10 additions & 0 deletions .meta.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,16 @@ testenv-setenv = [
"zeo4: ZEO4_SERVER=1",
]

[flake8]
additional-config = [
"# E501 line too long",
"# F401 unused import",
"per-file-ignores =",
" src/ZEO/tests/test_cache.py: E501",
" src/ZEO/_compat.py: F401",
" src/ZEO/asyncio/compat.py: F401",
]

[coverage]
fail-under = 53

Expand Down
24 changes: 24 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,30 @@
Changelog
=========

6.0 (unreleased)
----------------

- Drop Python 2 and 3.5 support

- Reimplement and streamline the ``asyncio`` part of the ``ClientStorage``
implementation:
- switch from futures with explicit callbacks to `async/await` style
- use standard ``asyncio`` features to implement timeouts
- redesign the API of the class implementing the ZEO client protocol
- significantly improve source documentation
- add optional ``cython`` based optimization;
it speeds up reads but slows down writes.
To use it, install ``cython`` (and its dependencies) and
run ``cythonize -i *.pyx`` in ``src/ZEO/asyncio``.

- Drop credentials support: the corresponding ``ClientStorage.__init__``
parameters (i.e. ``credentials``, ``username``, ``password``)
are retained but ignored.
Note: the ZEO 5 server never supported credentials; the feature
has previously been retained for the use case
"ZEO 5 client with ZEO 4 server".


5.4.0 (unreleased)
------------------

Expand Down
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ recursive-include docs *.txt
recursive-include docs Makefile

recursive-include src *.py
recursive-include src *.pyx
include *.py
include COPYING
include log.ini
Expand Down
17 changes: 0 additions & 17 deletions asyncio-todo.rst

This file was deleted.

6 changes: 6 additions & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,12 @@ universal = 1

[flake8]
doctests = 1
# E501 line too long
# F401 unused import
per-file-ignores =
src/ZEO/tests/test_cache.py: E501
src/ZEO/_compat.py: F401
src/ZEO/asyncio/compat.py: F401

[check-manifest]
ignore =
Expand Down
16 changes: 3 additions & 13 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,10 @@
from setuptools import setup, find_packages
import os

version = '5.4.0.dev0'
version = '6.0.0.dev0'

install_requires = [
'ZODB >= 5.1.1',
'six',
'transaction >= 2.0.3',
'persistent >= 4.1.0',
'zc.lockfile',
Expand All @@ -35,21 +34,16 @@
'ZODB >= 5.5.1',
'ZopeUndo',
'zope.testing',
'manuel',
'transaction',
'mock',
'msgpack < 1',
'msgpack',
'zdaemon',
'zope.testrunner',
]

classifiers = """
Intended Audience :: Developers
License :: OSI Approved :: Zope Public License
Programming Language :: Python :: 2
Programming Language :: Python :: 2.7
Programming Language :: Python :: 3
Programming Language :: Python :: 3.5
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Expand Down Expand Up @@ -150,10 +144,6 @@ def emit(self, record):
'msgpack': [
'msgpack-python'
],
':python_version == "2.7"': [
'futures',
'trollius',
],
'docs': [
'Sphinx',
'repoze.sphinx.autointerface',
Expand All @@ -170,5 +160,5 @@ def emit(self, record):
zeo-nagios = ZEO.nagios:main
""",
include_package_data=True,
python_requires='>=2.7.9,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*',
python_requires='>=3.6',
)
Loading