Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-859635: Retry Strategy #671

Merged
merged 29 commits into from
Oct 27, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
1e5a951
Retry Strategy
sfc-gh-ext-simba-jy Oct 19, 2023
66fe71c
changed the Jitter to the original
sfc-gh-ext-simba-jy Oct 19, 2023
e3f6fb5
changed jitter
sfc-gh-ext-simba-jy Oct 19, 2023
a233abd
switched the point to add headers and make a function
sfc-gh-ext-simba-jy Oct 19, 2023
829edee
Merge branch 'master' into retrystrategy
sfc-gh-ext-simba-jy Oct 19, 2023
c2299fc
add testing cases
sfc-gh-ext-simba-jy Oct 20, 2023
11bc6a5
added testing for jitteredSleepTime and add comments
sfc-gh-ext-simba-jy Oct 20, 2023
43e8442
Removed spaces and formatted code
sfc-gh-ext-simba-jy Oct 20, 2023
e50573e
refactor isLoginRequest test
sfc-gh-ext-simba-jy Oct 20, 2023
15d7523
added max login time out, and refactored jittered time testing case
sfc-gh-ext-simba-jy Oct 23, 2023
aba19ae
Merge branch 'master' into retrystrategy
sfc-gh-ext-simba-jy Oct 23, 2023
9c7a7e4
edited the loginTimeout options
sfc-gh-ext-simba-jy Oct 23, 2023
1039fb8
add trailing comma in the default param
sfc-gh-ext-simba-jy Oct 23, 2023
3512bbc
fixed the format of jitter function
sfc-gh-ext-simba-jy Oct 23, 2023
1b4e12c
edited typos and fix some logics
sfc-gh-ext-simba-jy Oct 24, 2023
d6b82ca
added timeout in the ocsp login request test
sfc-gh-ext-simba-jy Oct 24, 2023
6f5df98
Merge branch 'master' into retrystrategy
sfc-gh-ext-simba-jy Oct 24, 2023
ab065b0
refactored chooseRandom code, and edited the comment
sfc-gh-ext-simba-jy Oct 24, 2023
0239e8f
Merge branch 'retrystrategy' of https://github.com/snowflakedb/snowfl…
sfc-gh-ext-simba-jy Oct 24, 2023
6dd5dcf
refactored chooseRandom function
sfc-gh-ext-simba-jy Oct 25, 2023
dc33cef
Changed the jitter rule, and comments
sfc-gh-ext-simba-jy Oct 26, 2023
9741f68
Merge branch 'master' into retrystrategy
sfc-gh-ext-simba-jy Oct 26, 2023
c7b7841
refactored jitter
sfc-gh-ext-simba-jy Oct 27, 2023
422854b
Merge branch 'retrystrategy' of https://github.com/snowflakedb/snowfl…
sfc-gh-ext-simba-jy Oct 27, 2023
717314d
removed unnecessary comments
sfc-gh-ext-simba-jy Oct 27, 2023
0688742
add getNextSleep testing
sfc-gh-ext-simba-jy Oct 27, 2023
7643c34
refactored loginTimeout in the connection config
sfc-gh-ext-simba-jy Oct 27, 2023
2d88fd0
removed console.log
sfc-gh-ext-simba-jy Oct 27, 2023
13b05b6
fixed typo
sfc-gh-ext-simba-jy Oct 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions lib/services/sf.js
Original file line number Diff line number Diff line change
Expand Up @@ -1180,9 +1180,8 @@ StateConnecting.prototype.continue = function ()
Date.now() : 'FIXEDTIMESTAMP';
const maxLoginRetries = connectionConfig.getRetrySfMaxLoginRetries();
const maxLoginTimeout = connectionConfig.getLoginTimeout();
const base = connectionConfig.getRetrySfStartingSleepTime();
let sleep = base;
let totalTimeout = base;
let sleep = connectionConfig.getRetrySfStartingSleepTime();;
let totalTimeout = sleep;
Logger.getInstance().debug("Total loginTimeout is for the retries = " + maxLoginTimeout);
const parent = this;
const requestCallback = function (err, body)
Expand Down Expand Up @@ -1217,7 +1216,7 @@ StateConnecting.prototype.continue = function ()
totalTimeout < maxLoginTimeout)
{
numRetries++;
const jitter = Util.jitteredSleepTime(numRetries, sleep, totalTimeout, base, maxLoginTimeout);
const jitter = Util.jitteredSleepTime(numRetries, sleep, totalTimeout, maxLoginTimeout);
sleep = jitter.sleep;
totalTimeout = jitter.totalTimeout;

Expand Down
9 changes: 5 additions & 4 deletions lib/util.js
Original file line number Diff line number Diff line change
Expand Up @@ -431,9 +431,10 @@ exports.nextSleepTime = function (
* @param {Number} maxLoginTimeout
* @returns {JSON} return next sleep Time and totalTime.
*/
exports.jitteredSleepTime = function (numofRetries, currentSleepTime, totalTimeout, base, maxLoginTimeout) {
const sleep = Math.min((maxLoginTimeout - totalTimeout), getNextSleepTime(numofRetries, base, currentSleepTime) );
exports.jitteredSleepTime = function (numofRetries, currentSleepTime, totalTimeout, maxLoginTimeout) {
const sleep = Math.min((maxLoginTimeout - totalTimeout), getNextSleepTime(numofRetries, currentSleepTime) );
totalTimeout += sleep
console.log(totalTimeout);
return {sleep, totalTimeout}
}

Expand All @@ -460,9 +461,9 @@ exports.chooseRandom = chooseRandom;
* @param {Number} currentSleepTime
* @returns {Boolean} return jitter.
*/
function getNextSleepTime (numofRetries, base, currentSleepTime) {
function getNextSleepTime (numofRetries, currentSleepTime) {
sfc-gh-ext-simba-jy marked this conversation as resolved.
Show resolved Hide resolved
const multiplicationFactor = chooseRandom(1, -1);
const nextSleep = (base * (2 ** (numofRetries - 1)));
const nextSleep = (2 ** (numofRetries));
const jitterAmount = 0.5 * currentSleepTime * multiplicationFactor;

return nextSleep + jitterAmount;
Expand Down
9 changes: 4 additions & 5 deletions test/unit/util_test.js
Original file line number Diff line number Diff line change
Expand Up @@ -603,17 +603,16 @@ describe('Util', function ()
];

const maxLoginTimeout = 300;
const base = 4;
let currentSleepTime = base;
let currentSleepTime = 4;
let retryCount = 1;
let totalTimeout = base;
let totalTimeout = currentSleepTime;
for (const response of errorCodes) {
retryCount++;
assert.strictEqual(Util.isRetryableHttpError(response,true), true);

const result = Util.jitteredSleepTime(retryCount, currentSleepTime, totalTimeout, base, maxLoginTimeout);
const result = Util.jitteredSleepTime(retryCount, currentSleepTime, totalTimeout, maxLoginTimeout);
const jitter = currentSleepTime / 2
const nextSleep = base * (2 ** (retryCount-1));
const nextSleep = 2 ** retryCount;
currentSleepTime = result.sleep;
totalTimeout = result.totalTimeout;

Expand Down