Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
IStorage: Require lastTransaction() to invalidate DB before returning
Because if lastTransaction() returns tid, for which local database handle has not yet been updated with invalidations, it could lead to data corruption due to concurrency issues similar to zopefoundation#290: - DB refreshes a Connection for new transaction; - zstor.lastTransaction() is called to obtain database view for this connection. - objects in live-cache for this Connection are invalidated with invalidations that were queued through DB.invalidate() calls from storage. - if lastTransaction does not guarantee that all DB invalidations for transactions with ID ≤ returned tid have been completed, it can be that: incomplete set of objects are invalidated in live cache i.e. data corruption. This particular data corruption has been hit when working on core of ZODB and was not immediately noticed: zopefoundation#307 (review) this fact justifies the importance of explicitly stating what IStorage guarantees are / must be in the interface. This guarantee - already holds for FileStorage (no database mutations from outside of single process); - is already true for ZEO4 and ZEO5 zopefoundation#307 (review) zopefoundation#307 (comment) - holds for RelStorage because it implements IMVCCStorage natively; - is *not* currently true for NEO because NEO sets zstor.last_tid before calling DB.invalidate: https://lab.nexedi.com/nexedi/neoppod/blob/fc58c089/neo/client/handlers/master.py#L109-124 However NEO is willing to change and already prepared the fix to provide this guarantee because of data corruption scenario that can happen without it: zopefoundation#307 (comment) https://lab.nexedi.com/nexedi/neoppod/commit/5870c5d7 In other words all storages that, to my knowledge, are in current use are either already providing specified semantic, or will be shortly fixed to provide it. This way we can fix up the interface and make the semantic clear. /cc @jamadden, @vpelletier, @arnaud-fontaine, @jwolf083, @klawlf82, @gitzit, @jimfulton
- Loading branch information
9a1d29f
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/cc @gidzit (-> zopefoundation#313)