Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

processEnv - initial module; #1796

Open
wants to merge 9 commits into
base: minor
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 10 additions & 54 deletions api/api.js
Original file line number Diff line number Diff line change
Expand Up @@ -41,46 +41,7 @@ The req object represents the HTTP request and has properties for the request qu
The res object represents the HTTP response that an [Express] app sends when it gets an HTTP request.
*/

/**
@global
@typedef {Object} env
The process.ENV object holds configuration provided to the node process from the launch environment. The environment configuration allows the provision of keys and secrets which must not be accessible from the client. All env properties are limited to string type.
@property {String} [DIR=''] The XYZ API path which concatenated with the domain for all requests.
@property {String} [DBS_=''] DBS_* values are the connections used to establish connections to pg servers with the [dbs]{@link module:/utils/dbs} module.
@property {String} [PORT='3000'] The port on which the express app listens to for requests.
@property {String} [COOKIE_TTL='36000'] The Time To Live for all cookies issued by the XYZ API.
@property {String} [TITLE='GEOLYTIX | XYZ'] The TITLE value is used to identify cookies and is provided to as a param to Application View templates.
@property {String} [LOGS] The LOGS string will split on comma to determine which requests send to the [LOGGER]{@link module:/utils/logger} module will be logged.
@property {String} [LOGGER] Required to configure the [LOGGER]{@link module:/utils/logger} module for a remote out.
@property {String} [PRIVATE] All requests to XYZ API require authentication. The PRIVATE value represents the ACL connection.
@property {String} [PUBLIC] General requests to XYZ API do require authentication. The PUBLIC value represents an ACL connection for optional authentication.
@property {String} [SECRET] A secret string is required to sign and [validate JWT]{@link module:/user/auth}.
@property {String} [USER_SESSION] The [auth module]{@link module:/user/auth} will store and check a session key if the USER_SESSION env is not undefined.
@property {String} [AUTH_EXPIRY] The [user/fromACL module]{@link module:/user/fromACL} can expiry user authorization if the AUTH_EXPIRY env is configured.
@property {String} [FAILED_ATTEMPTS='3'] The [user/fromACL module]{@link module:/user/fromACL} will expire user validation if failed login attempts exceed the FAILED_ATTEMPTS value.
@property {String} [PASSWORD_REGEXP='(?=.*?[A-Z])(?=.*?[a-z])(?=.*?[0-9])^.{10,}$'] The [user/register module]{@link module:/user/register} will apply PASSWORD_REGEXP value to check the complexity of provided user passwords.
@property {String} [STATEMENT_TIMEOUT] The [utils/dbs module]{@link module:/utils/dbs} will apply the STATEMENT_TIMEOUT to the query.client.
@property {String} [RETRY_LIMIT='3'] The [utils/dbs module]{@link module:/utils/dbs} will apply the RETRY_LIMIT to the query.client.
@property {String} [WORKSPACE_AGE] The [workspace/cache module]{@link module:/mod/workspace/cache} flashes the workspace cache after the WORKSPACE_AGE is reached.
@property {String} [CUSTOM_TEMPLATES] The [workspace/cache module]{@link module:/mod/workspace/cache} caches templates defined as a src in the CUSTOM_TEMPLATES env.
@property {String} [TRANSPORT] The [utils/mailer module]{@link module:/utils/mailer} requires a TRANSPORT env.
@property {String} [TRANSPORT_HOST] The [utils/mailer module]{@link module:/utils/mailer} requires a TRANSPORT env.
@property {String} [TRANSPORT_EMAIL] The [utils/mailer module]{@link module:/utils/mailer} requires a TRANSPORT env.
@property {String} [TRANSPORT_PASSWORD] The [utils/mailer module]{@link module:/utils/mailer} requires a TRANSPORT env.
@property {String} [TRANSPORT_PORT] The [utils/mailer module]{@link module:/utils/mailer} requires a TRANSPORT env.
@property {String} [USER_DOMAINS] The [user/register module]{@link module:/user/register} will limit the registration to user emails for domains provided in the comma seperated USER_DOMAINS env.
@property {String} [SRC_] SRC_* values will replace the key wildcard [*] in the stringified workspace.
@property {String} [KEY_CLOUDFRONT] A key [*.pem] file matching the KEY_CLOUDFRONT value is required for authentication requests in the [cloudfront]{@link module:/provider/cloudfront} provider module.
@property {String} [AWS_S3_CLIENT] A AWS_S3_CLIENT env is required to sign requests with the [s3]{@link module:/sign/s3} signer module.
@property {String} [CLOUDINARY_URL] A CLOUDINARY_URL env is required to sign requests with the [cloudinary]{@link module:/sign/cloudinary} signer module.
@property {String} [SAML_ENTITY_ID] Required authentication via [SAML]{@link module:/user/saml}.
@property {String} [SAML_LOGIN] Required authentication via [SAML]{@link module:/user/saml}.
@property {String} [SAML_SP_CRT] Required authentication via [SAML]{@link module:/user/saml}.
@property {String} [SAML_ACS] Required authentication via [SAML]{@link module:/user/saml}.
@property {String} [SAML_SSO] Required authentication via [SAML]{@link module:/user/saml}.
@property {String} [SAML_SLO] Required authentication via [SAML]{@link module:/user/saml}.
@property {String} [SAML_IDP_CRT] Required authentication via [SAML]{@link module:/user/saml}.
*/
const env = require('../mod/utils/processEnv.js');

const login = require('../mod/user/login')

Expand All @@ -102,12 +63,6 @@ const routes = {
workspace: require('../mod/workspace/_workspace'),
}

process.env.COOKIE_TTL ??= '36000'

process.env.TITLE ??= 'GEOLYTIX | XYZ'

process.env.DIR ??= ''

/**
@function api

Expand Down Expand Up @@ -140,8 +95,8 @@ All other requests will passed to the async validateRequestAuth method.
module.exports = function api(req, res) {

// redirect if dir is missing in url path.
if (process.env.DIR && req.url.length === 1) {
res.setHeader('location', `${process.env.DIR}`)
if (env.DIR && req.url.length === 1) {
res.setHeader('location', `${env.DIR}`)
return res.status(302).send()
}

Expand All @@ -165,12 +120,12 @@ module.exports = function api(req, res) {
if (req.params.logout) {

// Remove cookie.
res.setHeader('Set-Cookie', `${process.env.TITLE}=null;HttpOnly;Max-Age=0;Path=${process.env.DIR || '/'}`)
res.setHeader('Set-Cookie', `${env.TITLE}=null;HttpOnly;Max-Age=0;Path=${env.DIR || '/'}`)

const msg = req.params.msg ? `?msg=${req.params.msg}` : '';

// Set location to the domain path.
res.setHeader('location', `${process.env.DIR || '/'}${msg}`)
res.setHeader('location', `${env.DIR || '/'}${msg}`)

return res.status(302).send()
}
Expand Down Expand Up @@ -226,7 +181,7 @@ async function validateRequestAuth(req, res) {
}

// Remove cookie.
res.setHeader('Set-Cookie', `${process.env.TITLE}=null;HttpOnly;Max-Age=0;Path=${process.env.DIR || '/'};SameSite=Strict${!req.headers.host.includes('localhost') && ';Secure' || ''}`)
res.setHeader('Set-Cookie', `${env.TITLE}=null;HttpOnly;Max-Age=0;Path=${env.DIR || '/'};SameSite=Strict${!req.headers.host.includes('localhost') && ';Secure' || ''}`)

// Set msg parameter for the login view.
// The msg provides information in regards to failed logins.
Expand All @@ -247,11 +202,12 @@ async function validateRequestAuth(req, res) {
}

// PRIVATE instances require user auth for all requests.
if (!req.params.user && process.env.PRIVATE) {
if (!req.params.user && env.PRIVATE) {

// Redirect to the SAML login.
if (process.env.SAML_LOGIN) {
res.setHeader('location', `${process.env.DIR}/saml/login`)
if (env.SAML_LOGIN) {
res.setHeader('location', `${env.DIR}/saml/login`)

return res.status(302).send()
}

Expand Down
47 changes: 25 additions & 22 deletions express.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,54 +10,57 @@
extensions: ['html']
}))

app.use(`${process.env.DIR || ''}/public`, express.static('public'))
const env = require('./mod/utils/processEnv.js');

app.use(process.env.DIR || '', express.static('public'))
app.use(`${env.DIR}/public`, express.static('public'))

app.use(`${process.env.DIR || ''}/tests`, express.static('tests'))
app.use(env.DIR, express.static('public'))

app.use(process.env.DIR || '', express.static('tests'))
app.use(`${env.DIR}/tests`, express.static('tests'))

app.use(env.DIR, express.static('tests'))

app.use(cookieParser())

Check failure

Code scanning / CodeQL

Missing CSRF middleware High

This cookie middleware is serving a
request handler
without CSRF protection.
This cookie middleware is serving a
request handler
without CSRF protection.
This cookie middleware is serving a
request handler
without CSRF protection.
This cookie middleware is serving a
request handler
without CSRF protection.
This cookie middleware is serving a
request handler
without CSRF protection.
This cookie middleware is serving a request handler without CSRF protection.

const api = require('./api/api')

app.get(`${process.env.DIR || ''}/api/provider/:provider?`, api)
app.get(`${env.DIR}/api/provider/:provider?`, api)
Dismissed Show dismissed Hide dismissed

app.post(`${env.DIR}/api/provider/:provider?`, express.json({ limit: '5mb' }), api)
Fixed Show fixed Hide fixed
Dismissed Show dismissed Hide dismissed

app.post(`${process.env.DIR || ''}/api/provider/:provider?`, express.json({ limit: '5mb' }), api)

app.get(`${process.env.DIR || ''}/api/sign/:signer?`, api)
app.get(`${env.DIR || ''}/api/sign/:signer?`, api)
Dismissed Show dismissed Hide dismissed


app.get(`${process.env.DIR || ''}/api/query/:template?`, api)
app.get(`${env.DIR}/api/query/:template?`, api)
Dismissed Show dismissed Hide dismissed

app.post(`${process.env.DIR || ''}/api/query/:template?`, express.json({ limit: '5mb' }), api)
app.post(`${env.DIR}/api/query/:template?`, express.json({ limit: '5mb' }), api)

Check failure

Code scanning / CodeQL

Missing rate limiting High

This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.


app.get(`${process.env.DIR || ''}/api/fetch/:template?`, api)
app.get(`${env.DIR}/api/fetch/:template?`, api)

Check failure

Code scanning / CodeQL

Missing rate limiting High

This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.

app.post(`${process.env.DIR || ''}/api/fetch/:template?`, express.json({ limit: '5mb' }), api)
app.post(`${env.DIR}/api/fetch/:template?`, express.json({ limit: '5mb' }), api)

Check failure

Code scanning / CodeQL

Missing rate limiting High

This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.


app.get(`${process.env.DIR || ''}/api/workspace/:key?`, api)
app.get(`${env.DIR}/api/workspace/:key?`, api)

Check failure

Code scanning / CodeQL

Missing rate limiting High

This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.


app.get(`${process.env.DIR || ''}/api/user/:method?/:key?`, api)
app.get(`${env.DIR}/api/user/:method?/:key?`, api)

Check failure

Code scanning / CodeQL

Missing rate limiting High

This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.

app.post(`${process.env.DIR || ''}/api/user/:method?`, [express.urlencoded({ extended: true }), express.json({ limit: '5mb' })], api)
app.post(`${env.DIR}/api/user/:method?`, [express.urlencoded({ extended: true }), express.json({ limit: '5mb' })], api)

Check failure

Code scanning / CodeQL

Missing rate limiting High

This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.

app.get(`${process.env.DIR || ''}/saml/metadata`, api)
app.get(`${env.DIR}/saml/metadata`, api)

Check failure

Code scanning / CodeQL

Missing rate limiting High

This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.

app.get(`${process.env.DIR || ''}/saml/logout`, api)
app.get(`${env.DIR}/saml/logout`, api)

Check failure

Code scanning / CodeQL

Missing rate limiting High

This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.
This route handler performs
authorization
, but is not rate-limited.

app.get(`${process.env.DIR || ''}/saml/login`, api)
app.get(`${env.DIR}/saml/login`, api)
Dismissed Show dismissed Hide dismissed

app.post(`${process.env.DIR || ''}/saml/acs`, express.urlencoded({ extended: true }), api)
app.post(`${env.DIR}/saml/acs`, express.urlencoded({ extended: true }), api)
Dismissed Show dismissed Hide dismissed

app.get(`${process.env.DIR || ''}/view/:template?`, api)
app.get(`${env.DIR}/view/:template?`, api)
Dismissed Show dismissed Hide dismissed

app.get(`${process.env.DIR || ''}/:locale?`, api)
app.get(`${env.DIR}/:locale?`, api)
Dismissed Show dismissed Hide dismissed

process.env.DIR && app.get(`/`, api)
app.get(`/`, api)
Dismissed Show dismissed Hide dismissed

app.listen(process.env.PORT || 3000)
app.listen(env.PORT)
2 changes: 1 addition & 1 deletion lib/utils/userIndexedDB.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
* - The database will not be created if there is a pre-existing DB.
* - The creation will trigger the `onupgradeneeded` event which checks whether the request `store` exists in the userIndexedDB.
*
* The `process.env.TITLE` will be added to the user object in the cookie module.
* The `env.TITLE` will be added to the user object in the cookie module.
* The `user.title` is required to generate a unique indexedDB for each user[email/instance[title]]
*
* All object stores use the key value as a keypath for object indicies.
Expand Down
3 changes: 3 additions & 0 deletions mod/provider/cloudfront.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ The cloudfront provider module exports a method to fetch resources from an AWS c
const cloudfront_signer = require('../sign/cloudfront');
const logger = require('../utils/logger')

const env = require('../utils/processEnv.js')
/**
@function cloudfront
@async
Expand Down Expand Up @@ -41,12 +42,14 @@ async function cloudfront(ref) {

const url = ref.params?.url || ref


const signedURL = await cloudfront_signer(url)

if(signedURL instanceof Error) {
return signedURL;
}


// Return signedURL only from request.
if (ref.params?.signedURL) {
return signedURL;
Expand Down
6 changes: 4 additions & 2 deletions mod/provider/file.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,14 @@ const { readFileSync } = require('fs')

const { join } = require('path')

const env = require('../utils/processEnv.js')

module.exports = async ref => {
try {

// Subtitutes {*} with process.env.SRC_* key values.
// Subtitutes {*} with env.SRC_* key values.
const path = (ref.params?.url || ref).replace(/{(?!{)(.*?)}/g,
matched => process.env[`SRC_${matched.replace(/(^{)|(}$)/g, '')}`])
matched => env[`SRC_${matched.replace(/(^{)|(}$)/g, '')}`])

const file = readFileSync(join(__dirname, `../../${path}`))

Expand Down
71 changes: 36 additions & 35 deletions mod/provider/s3.js
Original file line number Diff line number Diff line change
@@ -1,35 +1,36 @@
/**
### /provider/s3

The S3 provider module requires the [S3 signer]{@link module:/sign/s3} and will return a signed request URL.

@requires /sign/s3

@module /provider/s3
*/

const s3_signer = require('../sign/s3')

if (!s3_signer) {

module.exports = null
} else {

module.exports = s3_provider
}

/**
@function s3_provider

@description
The s3_provider method returns the s3_signer method.

@param {Object} req HTTP request.
@param {Object} res HTTP response.

@returns {Function} The s3_signer module method.
**/
async function s3_provider(req, res) {

return s3_signer(req, res)
}
/**
### /provider/s3

The S3 provider module requires the [S3 signer]{@link module:/sign/s3} and will return a signed request URL.

@requires /sign/s3

@module /provider/s3
*/

const s3_signer = require('../sign/s3')

if (!s3_signer) {

module.exports = null
} else {

module.exports = s3_provider
}

/**
@function s3_provider

@description
The s3_provider method returns the s3_signer method.

@param {Object} req HTTP request.
@param {Object} res HTTP response.

@returns {Function} The s3_signer module method.
**/
async function s3_provider(req, res) {

return s3_signer(req, res)
}

7 changes: 5 additions & 2 deletions mod/sign/cloudinary.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,13 @@ Exports the cloudinary signer method.

const { createHash } = require('crypto');

const env = require('../utils/processEnv.js')

/**
@function cloudinary
@async


@description
The cloudinary signer method signs requests for the cloudinary service.

Expand All @@ -34,14 +37,14 @@ A folder and public_id parameter for resources to be uploaded or destroyed are r
*/
module.exports = async function cloudinary(req, res) {

if (!process.env.CLOUDINARY_URL) return new Error('CLOUDINARY_URL not provided in process.env')
if (!env.CLOUDINARY_URL) return new Error('CLOUDINARY_URL not provided in env')

if (!req.params.folder) return new Error('A folder request param is required for the cloudinary signer.')

if (!req.params.public_id) return new Error('A public_id request param is required for the cloudinary signer.')

// Split CLOUDINARY_URL string into array of ['cloudinary', api_key, api_secret, cloud_name]
const cloudinary = process.env.CLOUDINARY_URL
const cloudinary = env.CLOUDINARY_URL
.replaceAll('://', '|')
.replaceAll(':', '|')
.replaceAll('@', '|')
Expand Down
8 changes: 5 additions & 3 deletions mod/user/acl.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@

The acl module provides access to the ACL table for all User API methods.

The module will split either the PRIVATE or PUBLIC process.env variables as an array of connection strings.
The module will split either the PRIVATE or PUBLIC env variables as an array of connection strings.

The module will export null if neither a PRIVATE or PUBLIC process.env are provided.
The module will export null if neither a PRIVATE or PUBLIC env are provided.

@requires pg

Expand All @@ -14,7 +14,9 @@ The module will export null if neither a PRIVATE or PUBLIC process.env are provi

const { Pool } = require('pg');

const connection = process.env.PRIVATE?.split('|') || process.env.PUBLIC?.split('|')
const env = require('../utils/processEnv.js')

const connection = env.PRIVATE?.split('|') || env.PUBLIC?.split('|')

// These variables can only be reassigned if the connection is an array.
let acl_table, acl_schema, pool;
Expand Down
Loading
Loading