Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added version.ignore config #628

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

yzia2000
Copy link

@yzia2000 yzia2000 commented Apr 16, 2022

Problem

Should be able to control if INTERNAL version is used for requests when key.ignore is set to false. The general scenario is duplicate records being indexed and the worst case scenario is out-of-order messages being indexed which is not as likely.

See #499 (comment)

Solution

Created version.ignore config.
true -> INTERNAL
false -> EXTERNAL

Does this solution apply anywhere else?
  • yes
  • no
If yes, where?

Test Strategy

Testing if INTERNAL version is used for version.ignore set to true and also check if any error is thrown with duplicate messages.

Testing done:
  • Unit tests
  • Integration tests
  • System tests
  • Manual tests

Release Plan

This feature is backwards compatible.

Merging to master

Rolling back might result in validation error for people that use version.ignore

@yzia2000 yzia2000 requested a review from a team as a code owner April 16, 2022 09:37
@CLAassistant
Copy link

CLAassistant commented Apr 16, 2022

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@yzia2000 yzia2000 changed the title feat: added version.ignore field Added version.ignore field Apr 16, 2022
@yzia2000 yzia2000 changed the title Added version.ignore field Added version.ignore config Apr 16, 2022
@yzia2000 yzia2000 force-pushed the feature/version-ignore branch from 553d81e to 41142a4 Compare April 16, 2022 10:21
With insert write.method and ignore.key set to true, kafka offsets are
used as external versions for indexed documents.

version.ignore would instead use internal version if set to true.
@yzia2000 yzia2000 force-pushed the feature/version-ignore branch from 41142a4 to d705421 Compare April 16, 2022 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants