Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactoring harbor satellite and fixing test cases. #44

Merged
merged 10 commits into from
Sep 20, 2024

Conversation

Mehul-Kumar-27
Copy link
Collaborator

@Mehul-Kumar-27 Mehul-Kumar-27 commented Sep 8, 2024

Summary by CodeRabbit

  • New Features

    • Introduced a logging middleware for enhanced HTTP request logging.
    • Added configuration management capabilities for dynamic application settings.
    • Implemented profiling and metrics registration for performance monitoring.
    • Established a structured routing mechanism for improved route handling.
    • Added a /ping endpoint for connectivity verification in satellite operations.
    • Enhanced image handling with a new structured representation of images and repositories.
    • Introduced new environment variables for development configuration.
    • Enhanced flexibility in image transfers with support for secure and insecure connections.
  • Bug Fixes

    • Improved error handling during configuration loading.
  • Chores

    • Updated logging setup to allow configuration of log levels directly from context.
    • Streamlined application initialization and server setup processes for better maintainability.
    • Enhanced test clarity and maintainability with new constants for file paths and endpoints.
    • Improved documentation for running test cases using the dagger tool.
    • Added a new entry to the .gitignore file to keep the repository clean from unnecessary debug files.

@Mehul-Kumar-27 Mehul-Kumar-27 marked this pull request as draft September 8, 2024 10:46
@github-actions github-actions bot added the golang label Sep 8, 2024
Copy link

coderabbitai bot commented Sep 8, 2024

Walkthrough

The changes introduce a comprehensive restructuring of a Go-based server application, enhancing its configuration management, logging, routing, and profiling capabilities. New files and functions are added to modularize the application, improving the organization of server operations and route handling. The main entry point has been updated to streamline initialization and setup processes, ensuring a more maintainable and efficient architecture.

Changes

Files Change Summary
.gitignore Added entry to ignore debug binary file __debug_bin1949266242.
.env Introduced ENV=dev and USE_UNSECURE=true environment variables for development configuration.
config.toml Updated path for testing resources from an absolute URL to a local relative path.
internal/config/config.go Introduced Config struct and LoadConfig function for configuration management.
internal/replicate/replicate.go Enhanced BasicReplicator struct with new configuration parameters for image replication.
internal/store/http-fetch.go Updated RemoteImageList struct to include configuration fields for dynamic authentication.
internal/server/middleware.go Implemented logging middleware with LoggingMiddleware function.
internal/server/profiling.go Added MetricsRegistrar and DebugRegistrar for metrics and profiling routes.
internal/server/router.go Defined Router interface and DefaultRouter struct for HTTP routing mechanisms.
internal/server/router_group.go Created RouterGroup struct for organizing routes with shared middleware.
internal/server/server.go Established App struct for server lifecycle management and routing setup.
main.go Restructured application initialization, integrating new configuration and server setup methods.
test/e2e/satellite_test.go Enhanced tests with new constants and improved logic for interacting with the satellite service.
test/e2e/config.go Introduced configuration setup for end-to-end tests, including path constants and command-line flag handling.
internal/utils/utils.go Added utility functions for registry management, URL validation, and JSON parsing related to images.

Sequence Diagram(s)

sequenceDiagram
    participant Main
    participant Config
    participant Logger
    participant Router
    participant Server

    Main->>Config: LoadConfig()
    Config-->>Main: Config data
    Main->>Logger: SetupLogger()
    Logger-->>Main: Logger instance
    Main->>Router: NewDefaultRouter()
    Router-->>Main: Router instance
    Main->>Server: NewApp()
    Server-->>Main: App instance
    Main->>Server: SetupRoutes()
    Main->>Server: Start()
Loading

🐇 "In the garden where code does bloom,
New structures rise, dispelling gloom.
With routes and logs, we hop along,
A server's dance, a joyful song!
Configuration neat, like carrots in a row,
Let's celebrate this code we sow!" 🥕


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    -- I pushed a fix in commit <commit_id>, please review it.
    -- Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    -- @coderabbitai generate unit testing code for this file.
    -- @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    -- @coderabbitai generate interesting stats about this repository and render them as a table.
    -- @coderabbitai read src/utils.ts and generate unit testing code.
    -- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    -- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Tip

Early access features: enabled

We are currently testing the following features in early access:

  • OpenAI o1 for code reviews: OpenAI's new o1 model is being tested for code reviews. This model has advanced reasoning capabilities and can provide more nuanced feedback on your code.

Note:

  • You can enable or disable early access features from the CodeRabbit UI or by updating the CodeRabbit configuration file.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 02ba374 and 88e8588.

Files selected for processing (9)
  • .gitignore (1 hunks)
  • internal/config/config.go (1 hunks)
  • internal/server/middleware.go (1 hunks)
  • internal/server/profiling.go (1 hunks)
  • internal/server/router.go (1 hunks)
  • internal/server/router_group.go (1 hunks)
  • internal/server/server.go (1 hunks)
  • logger/logger.go (1 hunks)
  • main.go (2 hunks)
Files skipped from review due to trivial changes (1)
  • .gitignore
Additional comments not posted (16)
internal/server/profiling.go (4)

10-10: Struct Definition Approved

The MetricsRegistrar struct is appropriately defined for its intended use.


14-17: Route Registration Method Approved

The RegisterRoutes method for MetricsRegistrar is correctly implemented to handle Prometheus metrics.


12-12: Struct Definition Approved

The DebugRegistrar struct is appropriately defined for its intended use.


19-30: Route Registration Method Approved

The RegisterRoutes method for DebugRegistrar is correctly implemented, covering all essential pprof tools for debugging.

internal/server/server.go (7)

18-25: Struct Definition Approved

The App struct is well-structured and appropriately encapsulates all necessary components for comprehensive server management.


27-35: Constructor Method Approved

The NewApp method is correctly implemented, ensuring all components are properly initialized for the App instance.


38-42: Method Implementation Approved

The SetupRoutes method is efficiently implemented, correctly iterating over registrars to set up routes.


44-46: Method Implementation Approved

The ServeHTTP method correctly delegates the HTTP request handling to the router, maintaining clean separation of concerns.


47-49: Method Implementation Approved

The Start method is correctly implemented, handling the server start process efficiently.


51-53: Method Implementation Approved

The Shutdown method is correctly implemented, using the context for a graceful server shutdown.


55-68: Method Implementation Approved

The SetupServer method is correctly implemented, using errgroup to manage concurrent server operations effectively.

internal/server/router.go (4)

5-16: Well-defined Router interface.

The Router interface is well-documented and appropriately defines the necessary methods for a router, including route handling, middleware, and group management.


24-33: Well-structured DefaultRouter struct.

The DefaultRouter struct is well-organized, encapsulating the necessary components like http.ServeMux, middleware, and a root group. It effectively implements the Router interface.


35-40: Constructor function for DefaultRouter.

The NewDefaultRouter function correctly initializes a new DefaultRouter with a default http.ServeMux and a root group. This setup is efficient and follows best practices for initializing router structures.


43-45: Method implementations in DefaultRouter.

All methods (Handle, ServeHTTP, HandleFunc, Use, Group) are implemented correctly, providing clear and efficient functionality for routing and middleware management. Each method adheres to the interface specifications and utilizes Go's standard library effectively.

Also applies to: 48-50, 53-55, 58-60, 63-66

main.go (1)

41-42: Refactored initialization and error handling in main and run functions.

The main function correctly handles application errors by outputting them to stderr and exiting with a status code of 1, which is a standard practice for indicating errors in Unix-like systems.

The run function has been significantly refactored to include new configuration loading, logging setup, and server initialization. The use of config.LoadConfig, logger.SetupLogger, and server.NewDefaultRouter is appropriate and aligns with the new modular architecture. The error handling is straightforward and effective, directly returning errors when they occur, which simplifies the flow and improves readability.

The setup of routes and server using the app.SetupRoutes and app.SetupServer methods encapsulates functionality well, enhancing maintainability and clarity.

Also applies to: 48-51, 58-74

internal/server/middleware.go Outdated Show resolved Hide resolved
internal/config/config.go Outdated Show resolved Hide resolved
internal/server/router_group.go Show resolved Hide resolved
logger/logger.go Outdated Show resolved Hide resolved
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (3)
test/e2e/testdata/config.toml (2)

5-6: Ensure documentation for registry network configuration settings.

The own_registry_adr and own_registry_port settings are used in the codebase to configure the Zot registry's network address and port. However, there is no documentation found for these settings. Please ensure that these changes are documented appropriately to aid in understanding their purpose and usage.

  • Files Impacted:
    • main.go: Retrieves and uses these settings.
    • internal/config/config.go: Retrieves these settings.
Analysis chain

Confirm registry network configuration changes.

The own_registry_adr has been updated to 127.0.0.1, and a new setting own_registry_port has been added with the value 8585. Ensure that these changes are compatible with the network configurations and are documented appropriately.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the registry network configuration changes are compatible with the network configurations and are documented appropriately.

# Test: Search for the setting usage. Expect: Only occurrences of the new settings.
rg --type toml -A 5 $'own_registry_adr = "127.0.0.1"'
rg --type toml -A 5 $'own_registry_port = "8585"'

Length of output: 1325


Script:

#!/bin/bash
# Search for references to `own_registry_adr` and `own_registry_port` in the codebase to understand their usage.
rg 'own_registry_adr' -A 3
rg 'own_registry_port' -A 3

# Search for any documentation or comments related to these settings.
rg 'own_registry_adr' --type md -A 3
rg 'own_registry_port' --type md -A 3

Length of output: 1910


2-2: Update documentation to reflect the change in default registry behavior.

The bring_own_registry setting is actively used in the codebase to control whether an external registry is used. However, there are no references to this setting in the documentation files. It is recommended to update the documentation to clearly state the default behavior and any implications for users. This will ensure consistency and clarity for all stakeholders.

  • Code references found in main.go and internal/config/config.go.
  • No documentation references found.
Analysis chain

Confirm the change in default registry behavior.

The setting bring_own_registry has been changed from true to false, indicating a shift to using the built-in Zot registry by default. Ensure that this change is reflected in all dependent systems and documentation.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the change in default registry behavior is reflected in all dependent systems and documentation.

# Test: Search for the setting usage. Expect: Only occurrences of the new setting.
rg --type toml -A 5 $'bring_own_registry = false'

Length of output: 555


Script:

#!/bin/bash
# Description: Search for references to `bring_own_registry` in the codebase and documentation.

# Search for references in code files.
rg 'bring_own_registry' --type-not toml -A 5

# Search for references in documentation files.
rg 'bring_own_registry' --type md -A 5

Length of output: 771

main.go (1)

48-75: Run Function: Comprehensive Refactoring

The run function has been refactored to improve configuration loading, logging, and server initialization. The use of config.LoadConfig() centralizes configuration management, and the setup of server.App encapsulates server and route management effectively. This modular approach enhances maintainability and clarity.

Consider adding more detailed logging at each step of the initialization to aid in debugging and operational monitoring.

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 88e8588 and 7ed64c3.

Files selected for processing (4)
  • internal/satellite/routes.go (1 hunks)
  • main.go (2 hunks)
  • test/e2e/satellite_test.go (5 hunks)
  • test/e2e/testdata/config.toml (1 hunks)
Additional comments not posted (9)
test/e2e/testdata/config.toml (3)

9-9: Confirm the update to the registry URL.

The url_or_file setting has been updated to point to https://demo.goharbor.io/v2/myproject/album-server. Verify that this new URL is accessible and correctly configured in the testing environment.


15-15: Confirm the addition of the logging level setting.

The new setting log_level has been added with the value info, allowing for a standard level of verbosity in logs. Verify that this setting is correctly implemented across the application's logging configuration.

Verification successful

The log_level setting is correctly implemented.

The log_level setting is read from the configuration files and applied to the logging system in main.go using logger.SetupLogger(ctx, config.LogLevel). This confirms that the logging verbosity is controlled as intended. No further action is required.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the logging level setting is correctly implemented across the application's logging configuration.

# Test: Search for the setting usage. Expect: Only occurrences of the new setting.
rg --type toml -A 5 $'log_level = "info"'

Length of output: 511


Script:

#!/bin/bash
# Description: Search for code that reads the configuration files and applies the `log_level` setting to the logging system.

# Search for functions or methods that handle configuration parsing and logging setup.
rg 'log_level' -A 10

Length of output: 1032


Script:

#!/bin/bash
# Description: Search for the usage of the `LogLevel` field in the codebase to verify its application in logging setup.

# Search for where the `LogLevel` field is used in the codebase.
rg 'LogLevel' -A 10

Length of output: 1411


12-12: Confirm the addition of the Zot registry configuration path.

The new setting zotConfigPath specifies the path ./registry/config.json for the Zot registry's configuration file. Ensure that this path is valid and the file is accessible in the specified location.

Verification successful

The Zot registry configuration path is valid.

The zotConfigPath setting correctly points to an existing config.json file in the registry directory. No further action is required.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the Zot registry configuration path is valid and the file is accessible.

# Test: Search for the setting usage. Expect: Only occurrences of the new path.
rg --type toml -A 5 $'zotConfigPath = "./registry/config.json"'

Length of output: 533


Script:

#!/bin/bash
# Description: Check if the config.json file exists in the registry directory.

# Test: Verify the existence of config.json in the registry directory.
fd config.json --glob --full-path registry/config.json

Length of output: 164


Script:

#!/bin/bash
# Description: Check if the config.json file exists in the registry directory.

# Test: Verify the existence of config.json in the registry directory.
fd config.json --glob --search-path registry

Length of output: 67

test/e2e/satellite_test.go (2)

20-22: Review of Added Constants

The addition of relative_path, absolute_path, and satellite_ping_endpoint constants is a good practice for maintainability and clarity. Using constants for file paths and endpoints can help avoid hardcoding strings throughout the code, making future changes easier and less error-prone.

  • relative_path and absolute_path: Ensure that these paths are correctly used in file operations where paths are required.
  • satellite_ping_endpoint: This should be consistently used wherever the endpoint is needed in API calls.

142-142: Refactor and Enhancement in buildSatellite Function

The refactoring of the buildSatellite function to use the absolute_path constant for the configuration file path is a positive change that enhances code clarity and maintainability. Additionally, the introduction of a new service binding and the use of a wget command to ping the satellite service is a significant enhancement.

  • Configuration File Handling: Using absolute_path helps avoid hardcoding the path, which is a good practice.
  • Service Binding and Ping Test: The new logic to bind the satellite service and ping it to check its operational status is a robust addition to the test. It not only tests the build process but also the runtime behavior of the satellite service.

However, there are a few considerations:

  • Error Handling: Ensure that all potential errors from the wget command and other operations are handled appropriately.
  • Response Parsing: The parsing of the JSON response and the assertions based on it are crucial. Make sure the response structure is well-defined and matches the expected schema.

Consider adding more detailed error messages or logging to help diagnose issues during test failures.

Also applies to: 157-175

internal/satellite/routes.go (3)

10-16: Type Definitions: Well-structured

The SatelliteRegistrar and SatelliteResponse types are well-defined. The SatelliteResponse includes appropriate JSON tags for serialization, which is good practice for API responses.


18-21: Route Registration: Correct Implementation

The method RegisterRoutes correctly initializes a route group and assigns the Ping handler to the /ping endpoint. This setup is standard and follows good practices for modular route handling.


23-33: Ping Method: Proper Error Handling

The Ping method correctly sets the content type, writes a status code, and encodes the SatelliteResponse into JSON. The error handling within the JSON encoding is appropriate, ensuring that any encoding errors are caught and handled by sending an internal server error response.

main.go (1)

41-42: Main Function: Clean and Effective Error Handling

The main function is well-implemented, focusing solely on running the application and handling any errors that occur by logging to stderr and exiting. This separation of concerns is a good practice.

@github-actions github-actions bot added the documentation Improvements or additions to documentation label Sep 10, 2024
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (1)
ci/README.md (1)

64-83: Documentation changes look good, but please address the formatting issues.

The changes to the documentation are approved as they improve the clarity and structure of the README file. The addition of the new section for running test cases using dagger is particularly helpful.

However, please address the following formatting issues flagged by the static analysis tools:

  • Use consistent indentation for the unordered lists. The expected indentation is 2 spaces, but some lists use 4 spaces.
  • Replace "i.e" with "i.e." as the abbreviation requires two periods.
Tools
LanguageTool

[uncategorized] ~69-~69: The abbreviation “i.e.” (= that is) requires two periods.
Context: ... the release what kind of release it is i.e major, minor or path, The default value...

(I_E)

Markdownlint

67-67: Expected: 2; Actual: 4
Unordered list indentation

(MD007, ul-indent)


72-72: Expected: 2; Actual: 4
Unordered list indentation

(MD007, ul-indent)


77-77: Expected: 2; Actual: 4
Unordered list indentation

(MD007, ul-indent)


81-81: Expected: 2; Actual: 4
Unordered list indentation

(MD007, ul-indent)

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 7ed64c3 and 3807142.

Files selected for processing (3)
  • ci/README.md (1 hunks)
  • test/e2e/config.go (1 hunks)
  • test/e2e/satellite_test.go (6 hunks)
Additional context used
LanguageTool
ci/README.md

[uncategorized] ~69-~69: The abbreviation “i.e.” (= that is) requires two periods.
Context: ... the release what kind of release it is i.e major, minor or path, The default value...

(I_E)

Markdownlint
ci/README.md

67-67: Expected: 2; Actual: 4
Unordered list indentation

(MD007, ul-indent)


72-72: Expected: 2; Actual: 4
Unordered list indentation

(MD007, ul-indent)


77-77: Expected: 2; Actual: 4
Unordered list indentation

(MD007, ul-indent)


81-81: Expected: 2; Actual: 4
Unordered list indentation

(MD007, ul-indent)

Additional comments not posted (2)
test/e2e/config.go (1)

1-18: LGTM!

The code changes in this new file are approved:

  • The file is well-structured.
  • The constants are appropriately named and defined.
  • The flag variable is correctly initialized using the flag package.
test/e2e/satellite_test.go (1)

Line range hint 124-171: Great job enhancing the test!

The changes to the buildSatellite function are approved:

  • Using the PATH_TO_CONFIG variable improves the maintainability and clarity of the code.
  • The new service binding for the satellite container enhances the structure of the test.
  • Executing the wget command and parsing the JSON response ensures that the satellite service behaves as expected.
  • The assertions validate the response structure, including the success value, message, and status code.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Outside diff range and nitpick comments (1)
internal/images/get-images.go (1)

23-23: Remove the empty line.

The empty line at line 23 does not serve any purpose and can be removed to improve code readability.

Apply this diff to remove the empty line:

-
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 3807142 and ce6a19e.

Files selected for processing (4)
  • internal/config/config.go (1 hunks)
  • internal/images/get-images.go (1 hunks)
  • internal/utils/utils.go (1 hunks)
  • main.go (1 hunks)
Additional comments not posted (15)
internal/images/get-images.go (1)

13-21: LGTM!

The ImageList struct definition is well-structured and follows Go conventions. The use of anonymous structs provides a concise way to represent the hierarchy of a registry, repositories, and images. The JSON tags ensure proper serialization and deserialization of the struct fields.

internal/utils/utils.go (6)

33-40: LGTM!

The function correctly handles the own registry address and port and sets the Zot URL. It also returns an error if the registry address or port is invalid.


43-52: LGTM!

The function correctly launches the default Zot registry using the Zot config path. It also handles the error cases and returns an error if the registry fails to launch.


55-58: LGTM!

The function correctly checks if a given string is a valid URL. It also handles the error case and returns false if the URL is invalid.


61-71: LGTM!

The function correctly gets the absolute file path of the input file path and checks if it exists. It also handles the error cases and returns an error if the current working directory cannot be retrieved or if the file does not exist.


74-76: LGTM!

The function correctly checks if a given string contains invalid path characters. It returns true if the string contains any of the invalid characters and false otherwise.


79-90: LGTM!

The function correctly parses the images.json file and decodes it into the ImageList struct. It handles the error cases and returns an error if the file cannot be opened or if the JSON data cannot be decoded. It also defers the closing of the file to ensure that the file is closed after the function returns.

main.go (8)

79-84: LGTM!

The initConfig function correctly initializes the application configuration and returns any errors that occur during the process.


86-89: LGTM!

The setupContext function correctly sets up a context with signal handling for graceful shutdown.


91-104: LGTM!

The setupServerApp function correctly sets up the server application with a router and middleware. It uses the server.NewDefaultRouter function to create a new router with a prefix of /api/v1, the server.LoggingMiddleware to add logging middleware to the router, and the server.NewApp function to create a new server application with the router, context, logger, configuration, and registrars.


106-125: LGTM!

The handleRegistrySetup function correctly handles the setup of the registry based on the configuration. If the own_registry configuration is set to true, the function calls the utils.HandleOwnRegistry function to handle the setup of the own registry. If the own_registry configuration is set to false, the function launches the default registry using the utils.LaunchDefaultZotRegistry function. The function returns any errors that occur during the setup of the registry.


127-141: LGTM!

The processInput function correctly processes the input URL or file path and returns an image fetcher. If the input is a valid URL, the function uses the store.RemoteImageListFetcher function to create a remote image fetcher and sets the URL configuration using the utils.SetUrlConfig function. If the input is not a valid URL, the function checks if it is a valid file path using the validateFilePath function. If the input is a valid file path, the function calls the setupFileFetcher function to create a file image fetcher. The function returns any errors that occur during the processing of the input.


143-153: LGTM!

The validateFilePath function correctly validates the file path and returns an error if the path is invalid or the file does not exist. The function checks if the path contains invalid characters using the utils.HasInvalidPathChars function and checks if the file exists using the utils.GetAbsFilePath function. The function returns an error if the path contains invalid characters or the file does not exist.


155-166: LGTM!

The setupFileFetcher function correctly sets up a file image fetcher and returns it along with any errors that occur during the setup. The function uses the store.FileImageListFetcher function to create a file image fetcher, parses the images JSON file using the utils.ParseImagesJsonFile function, and sets the registry environment variables using the utils.SetRegistryEnvVars function. The function returns the file image fetcher and any errors that occur during the setup.


33-77: LGTM!

The run function correctly orchestrates the setup and execution of the satellite application. It initializes the configuration and logger, sets up the context and cancellation, sets up the server application, handles the registry setup, processes the input URL or file path, loads the environment variables, creates a new in-memory store, creates a new replicator, creates a new satellite service, starts the satellite service, and waits for the satellite service to finish.

internal/config/config.go Outdated Show resolved Hide resolved
internal/config/config.go Outdated Show resolved Hide resolved
internal/utils/utils.go Outdated Show resolved Hide resolved
internal/utils/utils.go Show resolved Hide resolved
internal/utils/utils.go Outdated Show resolved Hide resolved
@bupd
Copy link
Collaborator

bupd commented Sep 13, 2024

@Mehul-Kumar-27

img, err := crane.Copy(imageRef, crane.WithAuth(auth), crane.WithInsecure(insecure))

We use crane.WithInsecure for testing and dev environments to work with http(Insecure) seamlessly. but this should be avoided in production. Add a env variable production, if production is set to true. Should only use secure methods https(Secure).

// Example usage:
if os.Getenv("PRODUCTION") == "true" {
   // Do not use insecure methods
 } else {
   // Allow insecure communication for non-production environments
 }

@Vad1mo
Copy link
Contributor

Vad1mo commented Sep 14, 2024

@Mehul-Kumar-27

img, err := crane.Copy(imageRef, crane.WithAuth(auth), crane.WithInsecure(insecure))

We use crane.WithInsecure for testing and dev environments to work with http(Insecure) seamlessly. but this should be avoided in production. Add a env variable production, if production is set to true. Should only use secure methods https(Secure).

// Example usage:

if os.Getenv("PRODUCTION") == "true" {

   // Do not use insecure methods

 } else {

   // Allow insecure communication for non-production environments

 }

The PROD env car should not have a side effect, I rather would prefere an UNSECURE env var, in cases where it needs to be. Security should be the default, not exception.

@Mehul-Kumar-27 Mehul-Kumar-27 marked this pull request as ready for review September 15, 2024 10:23
@Mehul-Kumar-27 Mehul-Kumar-27 changed the title WIP: refactoring harbor satellite and fixing test cases. refactoring harbor satellite and fixing test cases. Sep 15, 2024
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 4a34c2b and 5f75ed8.

Files selected for processing (5)
  • .env (1 hunks)
  • internal/config/config.go (1 hunks)
  • internal/replicate/replicate.go (5 hunks)
  • internal/store/http-fetch.go (3 hunks)
  • main.go (1 hunks)
Additional comments not posted (23)
.env (2)

5-5: LGTM! Consider secure configurations in the development environment.

Setting the ENV variable to dev is a good practice to differentiate between development and production environments. This allows for conditionally executing code based on the environment.

However, please ensure that sensitive information or insecure configurations are not exposed in the development environment. It's important to maintain security best practices across all environments.


4-4: Clarify the purpose of the empty TOKEN variable.

The TOKEN variable is explicitly defined but left empty. If this variable is used for authentication or authorization purposes, leaving it empty could potentially lead to security vulnerabilities or unauthorized access.

Please provide clarification on the purpose of the empty TOKEN variable and ensure that it is properly set with a secure token value when required.

If the empty value is intentional for development or testing purposes, consider adding a comment to indicate this and ensure that the token is properly set in production environments.

internal/store/http-fetch.go (5)

14-14: LGTM!

The import statement for the config package is correctly added.


21-25: LGTM!

The new fields username, password, use_unsecure, and zot_url are correctly added to the RemoteImageList struct to store authentication and connection settings.


35-39: LGTM!

The RemoteImageListFetcher function is correctly modified to initialize the new fields of RemoteImageList using the config package functions.


126-126: LGTM!

The crane.Digest function is correctly called with the imageRef and options arguments.


129-129: LGTM!

The error handling code is correctly simplified by directly returning an empty string and the error.

main.go (7)

72-77: LGTM!

The function correctly initializes the application configuration and handles errors appropriately.


79-82: LGTM!

The function correctly sets up a context with signal handling for graceful shutdown.


84-97: LGTM!

The function correctly sets up the server application with a router, middleware, and necessary dependencies.


99-118: LGTM!

The function correctly handles the setup of the container registry based on the configuration. It appropriately logs messages, handles errors, and manages the context cancellation.


120-134: LGTM!

The function correctly processes the input (file or URL) and returns an appropriate image fetcher based on the input type. It handles errors and sets up the necessary configurations.


136-146: LGTM!

The function correctly validates the file path by checking for invalid characters and attempting to get the absolute file path. It handles errors appropriately and returns them with descriptive messages.


148-159: LGTM!

The function correctly sets up a file-based image fetcher by parsing the images.json file and setting the registry environment variables. It handles errors appropriately and returns them with descriptive messages.

internal/config/config.go (4)

13-31: LGTM!

The Config struct correctly encapsulates the necessary configuration parameters. The naming convention is consistent across all fields.


33-123: LGTM!

The getter and setter functions provide a clean and consistent interface for accessing and modifying configuration parameters. The naming convention is consistent across all functions.


125-157: LGTM!

The LoadConfig function correctly reads the configuration settings from the TOML file using the viper library. It populates the Config instance with the retrieved values and includes appropriate error handling. The error message includes the path of the configuration file, which is helpful for debugging.


137-142: The current implementation aligns with the suggested approach for handling insecure communication.

The code segment checks the value of the USE_UNSECURE environment variable to determine whether to allow insecure communication. If the value is "true", it sets the use_unsecure variable to true; otherwise, it defaults to false.

This approach aligns with the suggestion made in the pull request comments to use an UNSECURE variable and default to secure communication unless explicitly specified otherwise.

internal/replicate/replicate.go (5)

24-29: LGTM!

The addition of the new fields to the BasicReplicator struct is a good change. It allows the replicator to utilize configuration values for authentication and connection settings, enhancing its functionality and flexibility.


46-51: LGTM!

The changes to the NewReplicator function are appropriate. Initializing the new fields of the BasicReplicator struct using values from the config package allows for better integration with external configuration management. This enhances the configurability and flexibility of the replicator.


191-194: LGTM!

The modification to the CopyImage function to conditionally include the crane.Insecure option based on the config.UseUnsecure() value is a good change. It allows for configurable handling of insecure connections, providing flexibility in different environments.


205-208: LGTM!

The modification to the CopyImage function to conditionally include the crane.Insecure option in the push_options slice based on the config.UseUnsecure() value is a good change. It allows for configurable handling of insecure connections during image pushing, providing flexibility in different environments.


11-11: LGTM!

The import of the "container-registry.com/harbor-satellite/internal/config" package is a good practice. It suggests a modular and organized approach to configuration management, which can improve the maintainability and readability of the codebase.

.env Show resolved Hide resolved
internal/store/http-fetch.go Show resolved Hide resolved
internal/config/config.go Outdated Show resolved Hide resolved
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Outside diff range and nitpick comments (2)
.github/workflows/build.yaml (1)

39-39: Typographical Error in Step Name

The step name should be "Generate the Dagger Go code" instead of "Generated the dagger go code" to accurately describe the action being performed.

Apply this diff to correct the typo:

-        - name: Generated the dagger go code
+        - name: Generate the Dagger Go code
test/e2e/satellite_test.go (1)

111-116: Variable Naming Convention

The variable PATH_TO_CONFIG uses uppercase letters, which in Go is typically reserved for constants or exported variables. Since this variable is not exported and seems to be mutable, consider renaming it using camelCase to adhere to Go naming conventions.

Apply this diff to rename the variable:

-    var PATH_TO_CONFIG string
+    var pathToConfig string
     if ABS {
-        PATH_TO_CONFIG = absolute_path
+        pathToConfig = absolute_path
     } else {
-        PATH_TO_CONFIG = relative_path
+        pathToConfig = relative_path
     }
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 8ce1b19 and 67e49c2.

Files selected for processing (2)
  • .github/workflows/build.yaml (2 hunks)
  • test/e2e/satellite_test.go (3 hunks)
Additional comments not posted (1)
.github/workflows/build.yaml (1)

46-46: Good Practice: Added Dependency on 'run-tests' Job

Adding needs: run-tests ensures that the build jobs only run after the tests have passed, enhancing the reliability of the build process.

Also applies to: 81-81

.github/workflows/build.yaml Show resolved Hide resolved
.github/workflows/build.yaml Show resolved Hide resolved
.github/workflows/build.yaml Show resolved Hide resolved
test/e2e/satellite_test.go Show resolved Hide resolved
test/e2e/satellite_test.go Show resolved Hide resolved
test/e2e/satellite_test.go Show resolved Hide resolved
@Vad1mo Vad1mo self-requested a review September 20, 2024 13:09
Copy link
Collaborator

@bupd bupd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove

Remove execute-tests-for-ground-control & execute-tests-for-satellite dagger functions. Since its no longer used.

@Vad1mo Vad1mo merged commit 72219c0 into container-registry:main Sep 20, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation golang
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants