Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug][Seatunnel-web] Support encyrption of datasource data #189

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

BilwaST
Copy link
Contributor

@BilwaST BilwaST commented Aug 20, 2024

Currently Datasource passwords are stored as it is in database and also is sent to seatunnel-engine in .profile file. All sensitive information like password should be encrypted. Also the API response should be encrypted for datasource password.

This patch handles the encryption of data using the encryption type available in seatunnel-engine. We can use base64 encryption type.

When we enable this encryption type, we need it configure it in job env like below:

env {
"shade.identifier" =base64
}

add shade.identifier to your env. you can refer to below doc for more understanding

https://seatunnel.apache.org/docs/2.3.2/connector-v2/Config-Encryption-Decryption/

@arshadmohammad
Copy link
Collaborator

I will review and test this PR in 1-2 days

datasourceConfig.replace(
Constants.PASSWORD,
ConfigShadeUtils.encryptOption(
datasourceEncryptionType, datasourceConfig.get(Constants.PASSWORD)));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

password is not a mandatory field, it can be empty also, then this code will throw NPE. Encrypt only if passord is not empty.

@arshadmohammad
Copy link
Collaborator

How this feature works? I see encryption only, I do not see any decryption. How it will be able to connnect to databases, it will fail.
Can you please create datasource, create a job, execute the job and attache the snapshot here.

@BilwaST
Copy link
Contributor Author

BilwaST commented Aug 25, 2024

How this feature works? I see encryption only, I do not see any decryption. How it will be able to connnect to databases, it will fail. Can you please create datasource, create a job, execute the job and attache the snapshot here.

Thank you for reviewing. Decryption is done on the engine side. There is a ConfigShadeUtils which takes care of the decryption of password while connecting to the datasource. Sure, i can share the details.

@arshadmohammad
Copy link
Collaborator

Decryption is done on the engine side.
We are addressing two components in this context: the datasource and the generated job configuration.

  1. In the Seatunnel-web, the datasource is used during job design via the UI. It's essential that it can establish a connection with the configured database. During this connection process, the encrypted password should be decrypted for successful connectivity.
  2. The generated job configuration file needs to include the 'datasourceEncryptType'. In the env section, it should specify something like 'shade.identifier = "base64"'.

@BilwaST BilwaST force-pushed the encryption branch 2 times, most recently from d3afefb to 0034129 Compare September 26, 2024 09:00
@BilwaST
Copy link
Contributor Author

BilwaST commented Oct 9, 2024

Hi @arshadmohammad
i have taken care of the scenarios you mentioned. Can you please review the updated patch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants