Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for unsigned integer values #99

Open
wants to merge 58 commits into
base: master
Choose a base branch
from

Conversation

bmatsuo
Copy link
Owner

@bmatsuo bmatsuo commented Feb 7, 2017

Closes #11

I am opening this now because it is close to done. But it is probably not done right now. It is a rather large change though. So I am letting it bake a here for a little bit.

The one thing that differs from the discussion in #11 is that the interface here is called Data, not Value. The word "value" was too overloaded already. Data is pretty precise, and shorter by one character. So I have opted to use it.

edit: The only unfortunate thing about the use of the word Data is its appearance near the beginning of the function index generated by go doc. This is fairly minor, but it does bug me with the number of conversion functions I had to define for working with the output of Txn.GetData and Cursor.GetData.

edit: I have actually stopped using an interface. The base code is the same and I can reintroduce an interface if needed in the future. But the API felt unwieldy and even without the new interfaces the API change is not negligible. But I think the result is quite clean now, and it's a few ns faster than the version that used interfaces.

bmatsuo added 30 commits March 7, 2016 16:54
The intial benchmark results are not impressive.  Writing a bunch of
size_t values doesn't appear to be significantly faster than encoding
uint64 values in big-endian. And, when optmized to eliminate allocations
big-endian byte slices have consistently better performance
characteristics.
It doesn't seem like a good idea to expose the String function because
there may be a StringValue implementation of Value in the future.  So
this type either needs to be implemented premptively as a stub or
omitted for the time being.
This will provide better grouping of topics in godoc.  Uint stuff will
all be together and Uintptr stuff will all be together.

The byte-related Value stuff won't really be together, but that is a
tragedy of evolution.  Maybe for lmdb-go 2.0 this part of the Value and
FixedPage interfaces can be made consistent for all data types.
Instead using a method named Put, the name Append is used to more
closely match what it is doing.  This also more transparently exposes
slice appending semantics and its quirks (which were already there).
There is some additional overhead processing integer Multi variants but
I couldn't trust users if I were to make the types raw byte slices.
Using the word Value so much was really confusing because LMDB is a
key-value store and the word "value" already has a reserved meaning.
This is where the Data interface is defined as well as the generic
BytesData type.  This is also where the FixedPage interface is defined
because it seems like the most appropriate place, though the generic
Multi type has been left if val.go, mostly for historical reasons.  It
may be moved later.
It's not really clear how those functions should behave and the data
conversion functions provided should provide more help overall.
@bmatsuo bmatsuo changed the title Data interface Support for unsigned integer data Feb 11, 2017
It turns out I have been using the official terminology backwards.  LMDB
associates keys with data.  Keys and data are both values (this was
a bit of a facepalm given the type names).
Fixes #103

This will effectively deprecate Cursor.PutMulti once merged.  It can
survive for a while longer.  But PutMultiple is a bit simpler if
somewhat slower.  The overhead of interfaces should be dominated by the
actual work of inserting items into the database in practice.
@bmatsuo bmatsuo changed the title Support for unsigned integer data Support for unsigned integer values Feb 12, 2017
@bmatsuo bmatsuo force-pushed the bmatsuo/value-interface branch from 6eeba8b to 9f15f38 Compare February 12, 2017 17:04
I am still working back from a spot where there were way too many
panics.  A new type, Stride, has been introduced with a method
Stride.Multiple which has the same signature as MultipleCUint and
MultipleCSizet.
These methods can be useful occasionally.  Examples were added to
demonstrate when slice syntax can be used instead.

Also, this change renames the receiver in CUintValue and CSizetValue
methods to be u and z resectively.  This will help drive the naming
convention that the Value* convertion functions adopted.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

lmdb: Support integer flags (MDB_INTEGERKEY/MDB_INTEGERDUP)
1 participant