Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improvement/cldsrv 553 bump deps #5711

Open
wants to merge 21 commits into
base: development/9.0
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/setup-ci/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ runs:
mkdir -p /tmp/artifacts/${JOB_NAME}/;
- uses: actions/setup-node@v4
with:
node-version: '16'
node-version: '22'
cache: 'yarn'
- name: install typescript
shell: bash
Expand Down
70 changes: 35 additions & 35 deletions .github/workflows/tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ jobs:
uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '16'
node-version: '22'
cache: yarn
- name: install typescript
shell: bash
Expand Down Expand Up @@ -106,7 +106,7 @@ jobs:
uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '16'
node-version: '22'
cache: yarn
- name: install typescript
shell: bash
Expand Down Expand Up @@ -149,7 +149,7 @@ jobs:
if: always()

build:
runs-on: ubuntu-20.04
runs-on: ubuntu-latest
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

best to specify the version explicitely (24.04)

permissions:
contents: read
packages: write
Expand Down Expand Up @@ -361,38 +361,38 @@ jobs:
source: /tmp/artifacts
if: always()

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be skipped instead of removed:

Suggested change
utapi-v2-tests:
if: false

utapi-v2-tests:
runs-on: ubuntu-latest
needs: build
env:
ENABLE_UTAPI_V2: t
S3BACKEND: mem
BUCKET_DENY_FILTER: utapi-event-filter-deny-bucket
CLOUDSERVER_IMAGE: ghcr.io/${{ github.repository }}:${{ github.sha }}
MONGODB_IMAGE: ghcr.io/${{ github.repository }}/ci-mongodb:${{ github.sha }}
JOB_NAME: ${{ github.job }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup CI environment
uses: ./.github/actions/setup-ci
- name: Setup CI services
run: docker compose up -d
working-directory: .github/docker
- name: Run file utapi v2 tests
run: |-
set -ex -o pipefail;
bash wait_for_local_port.bash 8000 40
yarn run test_utapi_v2 | tee /tmp/artifacts/${{ github.job }}/tests.log
- name: Upload logs to artifacts
uses: scality/action-artifacts@v4
with:
method: upload
url: https://artifacts.scality.net
user: ${{ secrets.ARTIFACTS_USER }}
password: ${{ secrets.ARTIFACTS_PASSWORD }}
source: /tmp/artifacts
if: always()
# utapi-v2-tests:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better to skip the job than to comment it:

Suggested change
# utapi-v2-tests:
utapi-v2-tests:
if: false
[...]

# runs-on: ubuntu-latest
# needs: build
# env:
# ENABLE_UTAPI_V2: t
# S3BACKEND: mem
# BUCKET_DENY_FILTER: utapi-event-filter-deny-bucket
# CLOUDSERVER_IMAGE: ghcr.io/${{ github.repository }}:${{ github.sha }}
# MONGODB_IMAGE: ghcr.io/${{ github.repository }}/ci-mongodb:${{ github.sha }}
# JOB_NAME: ${{ github.job }}
# steps:
# - name: Checkout
# uses: actions/checkout@v4
# - name: Setup CI environment
# uses: ./.github/actions/setup-ci
# - name: Setup CI services
# run: docker compose up -d
# working-directory: .github/docker
# - name: Run file utapi v2 tests
# run: |-
# set -ex -o pipefail;
# bash wait_for_local_port.bash 8000 40
# yarn run test_utapi_v2 | tee /tmp/artifacts/${{ github.job }}/tests.log
# - name: Upload logs to artifacts
# uses: scality/action-artifacts@v4
# with:
# method: upload
# url: https://artifacts.scality.net
# user: ${{ secrets.ARTIFACTS_USER }}
# password: ${{ secrets.ARTIFACTS_PASSWORD }}
# source: /tmp/artifacts
# if: always()

quota-tests:
runs-on: ubuntu-latest
Expand Down
7 changes: 5 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
ARG NODE_VERSION=16.20.2-bookworm-slim
ARG NODE_VERSION=22.4.0-bookworm-slim

FROM node:${NODE_VERSION} AS builder

Expand All @@ -22,8 +22,11 @@ RUN apt-get update \
&& ssh-keyscan -H github.com > /root/ssh/known_hosts

ENV PYTHON=python3
RUN npm install -g \
node-gyp \
[email protected]
COPY package.json yarn.lock /usr/src/app/
RUN npm install [email protected] -g

RUN yarn install --production --ignore-optional --frozen-lockfile --ignore-engines --network-concurrency 1

################################################################################
Expand Down
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@
[![Docker Pulls][badgedocker]](https://hub.docker.com/r/zenko/cloudserver)
[![Docker Pulls][badgetwitter]](https://twitter.com/zenko)

## Build Status

![Public Build Status][badgepub]
![Private Build Status][badgepriv]

## Overview

CloudServer (formerly S3 Server) is an open-source Amazon S3-compatible
Expand Down
2 changes: 1 addition & 1 deletion bin/create_bucket_with_NFS_enabled.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/usr/bin/env node
'use strict'; // eslint-disable-line strict
'use strict';

require('../lib/nfs/utilities.js').createBucketWithNFSEnabled();
2 changes: 1 addition & 1 deletion bin/create_encrypted_bucket.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/bin/sh
// 2>/dev/null ; exec "$(which nodejs || which node)" "$0" "$@"
'use strict'; // eslint-disable-line strict
'use strict';

require('../lib/kms/utilities.js').createEncryptedBucket();
2 changes: 1 addition & 1 deletion bin/list_bucket_metrics.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/usr/bin/env node
'use strict'; // eslint-disable-line strict
'use strict';

require('../lib/utapi/utilities.js').listMetrics('buckets');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should keep this file, and just add early return (if needed) in listMetrics()

2 changes: 1 addition & 1 deletion bin/list_metrics.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/usr/bin/env node
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should keep this file, and just add early return (if needed) in listMetrics()

'use strict'; // eslint-disable-line strict
'use strict';

require('../lib/utapi/utilities.js').listMetrics();
2 changes: 1 addition & 1 deletion bin/metrics_server.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/usr/bin/env node
'use strict'; // eslint-disable-line strict
'use strict';

const {
startWSManagementClient,
Expand Down
2 changes: 1 addition & 1 deletion bin/search_bucket.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/sh
// 2>/dev/null ; exec "$(which nodejs 2>/dev/null || which node)" "$0" "$@"
'use strict'; // eslint-disable-line strict
'use strict';

const { auth } = require('arsenal');
const commander = require('commander');
Expand Down
2 changes: 1 addition & 1 deletion bin/secure_channel_proxy.js
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#!/usr/bin/env node
'use strict'; // eslint-disable-line strict
'use strict';

const {
startWSManagementClient,
Expand Down
12 changes: 6 additions & 6 deletions config.json
Original file line number Diff line number Diff line change
Expand Up @@ -67,27 +67,27 @@
"allowFrom": ["127.0.0.1/8", "::1"]
},
"metadataClient": {
"host": "localhost",
"host": "127.0.0.1",
"port": 9990
},
"dataClient": {
"host": "localhost",
"host": "127.0.0.1",
"port": 9991
},
"pfsClient": {
"host": "localhost",
"host": "127.0.0.1",
"port": 9992
},
"metadataDaemon": {
"bindAddress": "localhost",
"bindAddress": "127.0.0.1",
"port": 9990
},
"dataDaemon": {
"bindAddress": "localhost",
"bindAddress": "127.0.0.1",
"port": 9991
},
"pfsDaemon": {
"bindAddress": "localhost",
"bindAddress": "127.0.0.1",
"port": 9992
},
"recordLog": {
Expand Down
2 changes: 1 addition & 1 deletion dataserver.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
'use strict'; // eslint-disable-line strict
'use strict';

const arsenal = require('arsenal');
const { config } = require('./lib/Config.js');
Expand Down
39 changes: 29 additions & 10 deletions .eslintrc → eslint.config.mjs
Original file line number Diff line number Diff line change
@@ -1,9 +1,28 @@
{
"extends": "scality",
"plugins": [
"mocha"
],
"rules": {
import mocha from "eslint-plugin-mocha";
import path from "node:path";
import { fileURLToPath } from "node:url";
import js from "@eslint/js";
import { FlatCompat } from "@eslint/eslintrc";

const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const compat = new FlatCompat({
baseDirectory: __dirname,
recommendedConfig: js.configs.recommended,
allConfig: js.configs.all
});

export default [...compat.extends("scality"), {
plugins: {
mocha,
},

languageOptions: {
ecmaVersion: 2020,
sourceType: "script",
},

rules: {
"import/extensions": "off",
"lines-around-directive": "off",
"no-underscore-dangle": "off",
Expand Down Expand Up @@ -47,8 +66,8 @@
"no-multi-spaces": "off",
"quote-props": "off",
"mocha/no-exclusive-tests": "error",
"no-redeclare": ["error", { "builtinGlobals": false }],
},
"parserOptions": {
"ecmaVersion": 2020
}
}
}];


2 changes: 1 addition & 1 deletion index.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
'use strict'; // eslint-disable-line strict
'use strict';

require('werelogs').stderrUtils.catchAndTimestampStderr(
undefined,
Expand Down
6 changes: 3 additions & 3 deletions lib/Config.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,14 @@ const { isValidBucketName } = s3routes.routesUtils;
const validateAuthConfig = arsenalAuth.inMemory.validateAuthConfig;
const { buildAuthDataAccount } = require('./auth/in_memory/builder');
const validExternalBackends = require('../constants').externalBackends;
const {
azureAccountNameRegex,
const {
azureAccountNameRegex,
base64Regex,
allowedUtapiEventFilterFields,
allowedUtapiEventFilterStates,
supportedLifecycleRules,
} = require('../constants');
const { utapiVersion } = require('utapi');
const utapiVersion = 0; // require('utapi');
const { scaleMsPerDay } = s3middleware.objectUtils;

const constants = require('../constants');
Expand Down
3 changes: 0 additions & 3 deletions lib/api/api.js
Original file line number Diff line number Diff line change
Expand Up @@ -83,11 +83,9 @@ auth.setHandler(vault);
const api = {
callApiMethod(apiMethod, request, response, log, callback) {
// Attach the apiMethod method to the request, so it can used by monitoring in the server
// eslint-disable-next-line no-param-reassign
request.apiMethod = apiMethod;
// Array of end of API callbacks, used to perform some logic
// at the end of an API.
// eslint-disable-next-line no-param-reassign
request.finalizerHooks = [];

const actionLog = monitoringMap[apiMethod];
Expand Down Expand Up @@ -149,7 +147,6 @@ const api = {
// Extract all the _apiMethods and store them in an array
const apiMethods = requestContexts ? requestContexts.map(context => context._apiMethod) : [];
// Attach the names to the current request
// eslint-disable-next-line no-param-reassign
request.apiMethods = apiMethods;

function checkAuthResults(authResults) {
Expand Down
5 changes: 2 additions & 3 deletions lib/api/apiUtils/authorization/permissionChecks.js
Original file line number Diff line number Diff line change
Expand Up @@ -530,10 +530,10 @@ function checkIp(value) {
try {
try {
parseCIDR(values[i]);
} catch (err) {
} catch {
isValid(values[i]);
}
} catch (err) {
} catch {
return errString;
}

Expand Down Expand Up @@ -577,7 +577,6 @@ function validatePolicyConditions(policy) {
if (s.Condition) {
const conditionOperators = Object.keys(s.Condition);
// there can be multiple condition operations in the Condition enclosure
// eslint-disable-next-line no-restricted-syntax
for (const conditionOperator of conditionOperators) {
const conditionKey = Object.keys(s.Condition[conditionOperator])[0];
const conditionValue = s.Condition[conditionOperator][conditionKey];
Expand Down
7 changes: 0 additions & 7 deletions lib/api/apiUtils/authorization/prepareRequestContexts.js
Original file line number Diff line number Diff line change
Expand Up @@ -155,13 +155,6 @@ function prepareRequestContexts(apiMethod, request, sourceBucket,
generateRequestContext('objectPutTaggingVersion');
requestContexts.push(putObjectVersionRequestContext);
}
francoisferrand marked this conversation as resolved.
Show resolved Hide resolved
} else if (apiMethodAfterVersionCheck === 'objectPutCopyPart') {
const putObjectRequestContext =
generateRequestContext('objectPut');
requestContexts.push(putObjectRequestContext);
const getObjectRequestContext =
generateRequestContext('objectGet');
requestContexts.push(getObjectRequestContext);
} else if (apiMethodAfterVersionCheck === 'objectPut') {
// if put object with version
if (request.headers['x-scal-s3-version-id'] ||
Expand Down
1 change: 0 additions & 1 deletion lib/api/apiUtils/authorization/tagConditionKeys.js
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,6 @@ function updateRequestContextsWithTags(request, requestContexts, apiMethod, log,
return cb(err);
}
// FIXME introduced by CLDSRV-256, this syntax should be allowed by the linter
// eslint-disable-next-line no-restricted-syntax
for (const rc of requestContexts) {
rc.setNeedTagEval(true);
if (requestTagsQuery) {
Expand Down
2 changes: 0 additions & 2 deletions lib/api/apiUtils/bucket/parseWhere.js
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,6 @@ function parseWhere(root) {
const e1 = parseWhere(root[operator][0]);
const e2 = parseWhere(root[operator][1]);

// eslint-disable-next-line
return { '$and' : [
e1,
e2,
Expand All @@ -62,7 +61,6 @@ function parseWhere(root) {
const e1 = parseWhere(root[operator][0]);
const e2 = parseWhere(root[operator][1]);

// eslint-disable-next-line
return { '$or' : [
e1,
e2,
Expand Down
1 change: 0 additions & 1 deletion lib/api/apiUtils/object/coldStorage.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ function getAmzRestoreResHeader(objMD) {
if (objMD['x-amz-restore']) {
if (objMD['x-amz-restore']['expiry-date']) {
const utcDateTime = new Date(objMD['x-amz-restore']['expiry-date']).toUTCString();
// eslint-disable-next-line
return `ongoing-request="${objMD['x-amz-restore']['ongoing-request']}", expiry-date="${utcDateTime}"`;
}
}
Expand Down
2 changes: 0 additions & 2 deletions lib/api/apiUtils/object/createAndStoreObject.js
Original file line number Diff line number Diff line change
Expand Up @@ -193,9 +193,7 @@ function createAndStoreObject(bucketName, bucketMD, objectKey, objMD, authInfo,
metadataStoreParams.oldReplayId = objMD.uploadId;
}

/* eslint-disable camelcase */
const dontSkipBackend = externalBackends;
/* eslint-enable camelcase */

const mdOnlyHeader = request.headers['x-amz-meta-mdonly'];
const mdOnlySize = request.headers['x-amz-meta-size'];
Expand Down
Loading
Loading