Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(compose): template files during compose operations #686

Merged
merged 18 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions docs/getting-started/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,9 @@ provider:

The constant's keys should be in the format `.const.<key>` and should not contain any '-' or '.' characters, as this will not respect the go text/template format.

> [!IMPORTANT]
> Due to viper limitations, all constants should be referenced in the template as lowercase values.

#### Variables

A sample `variables` section of a `lula-config.yaml` file is as follows:
Expand Down
260 changes: 260 additions & 0 deletions docs/getting-started/templating.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,260 @@
# Templating

Lula supports composition of both Component Definition and Lula Validation template files. See the [configuration](./configuration.md) documentation for more information on how to configure Lula to use templating. See the [compose CLI command](../cli-commands/lula_tools_compose.md) documentation for more information on the `lula tools compose` command flags to control how templating is applied.

## Component Definition Templating

Component Definition templates can be used to create modular component definitions using values from the `lula-config.yaml` file.

Example:
```yaml
component-definition:
uuid: E6A291A4-2BC8-43A0-B4B2-FD67CAAE1F8F
metadata:
title: {{ .const.title }}
last-modified: "2022-09-13T12:00:00Z"
version: "20220913"
oscal-version: 1.1.2
parties:
- uuid: C18F4A9F-A402-415B-8D13-B51739D689FF
type: organization
name: Lula Development
links:
- href: {{ .const.website }}
rel: website
```

lula-config.yaml:
```yaml
constants:
title: Lula Demo
website: https://github.com/defenseunicorns/lula
```

When this is `composed` with templating applied (`lula tools compose -f <file> --render all`) with the associated `lula-config.yaml`, the resulting component definition will be:

```yaml
component-definition:
uuid: E6A291A4-2BC8-43A0-B4B2-FD67CAAE1F8F
metadata:
title: Lula Demo
last-modified: "2022-09-13T12:00:00Z"
version: "20220913"
oscal-version: 1.1.2
parties:
- uuid: C18F4A9F-A402-415B-8D13-B51739D689FF
type: organization
name: Lula Development
links:
- href: https://github.com/defenseunicorns/lula
rel: website
```

## Validation Templating

Validation templates can be used to create modular Lula Validations using values from the `lula-config.yaml` file. These can be composed into the component definition using the `lula tools compose` command.

Example:
```yaml
component-definition:
uuid: E6A291A4-2BC8-43A0-B4B2-FD67CAAE1F8F
metadata:
title: Lula Demo
last-modified: "2022-09-13T12:00:00Z"
version: "20220913"
oscal-version: 1.1.2 # This version should remain one version behind latest version for `lula dev upgrade` demo
parties:
# Should be consistent across all of the packages, but where is ground truth?
- uuid: C18F4A9F-A402-415B-8D13-B51739D689FF
type: organization
name: Lula Development
links:
- href: https://github.com/defenseunicorns/lula
rel: website
components:
- uuid: A9D5204C-7E5B-4C43-BD49-34DF759B9F04
type: {{ .const.type }}
title: {{ .const.title }}
description: |
Lula - the Compliance Validator
purpose: Validate compliance controls
responsible-roles:
- role-id: provider
party-uuids:
- C18F4A9F-A402-415B-8D13-B51739D689FF # matches parties entry for Defense Unicorns
control-implementations:
- uuid: A584FEDC-8CEA-4B0C-9F07-85C2C4AE751A
source: https://raw.githubusercontent.com/usnistgov/oscal-content/master/nist.gov/SP800-53/rev5/json/NIST_SP-800-53_rev5_catalog.json
description: Validate generic security requirements
implemented-requirements:
- uuid: 42C2FFDC-5F05-44DF-A67F-EEC8660AEFFD
control-id: ID-1
description: >-
This control validates that the demo-pod pod in the validation-test namespace contains the required pod label foo=bar in order to establish compliance.
links:
- href: "./validation.tmpl.yaml"
text: local path template validation
rel: lula
```

Where `./validation.tmpl.yaml` is:
```yaml
metadata:
name: Test validation with templating
uuid: 99fc662c-109a-4e26-8398-75f3db67f862
domain:
type: kubernetes
kubernetes-spec:
resources:
- name: podvt
resource-rule:
name: {{ .const.resources.name }}
version: v1
resource: pods
namespaces: [{{ .const.resources.namespace }}]
provider:
type: opa
opa-spec:
rego: |
package validate
import rego.v1

# Default values
default validate := false
default msg := "Not evaluated"

# Validation result
validate if {
{ "one", "two", "three" } == { {{ .const.resources.exemptions | concatToRegoList }} }
"{{ .var.some_env_var }}" == "my-env-var"
"{{ .var.some_lula_secret }}" == "********"
}
msg = validate.msg

value_of_my_secret := {{ .var.some_lula_secret }}
```

Executing `lula tools compose -f ./component-definition.yaml --render all --render-validations` will result in:

```yaml
component-definition:
back-matter:
resources:
- description: |
domain:
kubernetes-spec:
create-resources: null
resources:
- description: ""
name: podvt
resource-rule:
group: ""
name: test-pod-label
namespaces:
- validation-test
resource: pods
version: v1
type: kubernetes
lula-version: ""
metadata:
name: Test validation with templating
uuid: 99fc662c-109a-4e26-8398-75f3db67f862
provider:
opa-spec:
rego: |
package validate
import rego.v1

# Default values
default validate := false
default msg := "Not evaluated"

# Validation result
validate if {
{ "one", "two", "three" } == { "one", "two", "three" }
"this-should-be-overridden" == "my-env-var"
"" == "********"
}
msg = validate.msg

value_of_my_secret :=
type: opa
title: Test validation with templating
uuid: 99fc662c-109a-4e26-8398-75f3db67f862
components:
- control-implementations:
- description: Validate generic security requirements
implemented-requirements:
- control-id: ID-1
description: This control validates that the demo-pod pod in the validation-test namespace contains the required pod label foo=bar in order to establish compliance.
links:
- href: '#99fc662c-109a-4e26-8398-75f3db67f862'
rel: lula
text: local path template validation
uuid: 42C2FFDC-5F05-44DF-A67F-EEC8660AEFFD
source: https://raw.githubusercontent.com/usnistgov/oscal-content/master/nist.gov/SP800-53/rev5/json/NIST_SP-800-53_rev5_catalog.json
uuid: A584FEDC-8CEA-4B0C-9F07-85C2C4AE751A
description: |
Lula - the Compliance Validator
purpose: Validate compliance controls
responsible-roles:
- party-uuids:
- C18F4A9F-A402-415B-8D13-B51739D689FF
role-id: provider
title: lula
type: software
uuid: A9D5204C-7E5B-4C43-BD49-34DF759B9F04
metadata:
last-modified: XXX
oscal-version: 1.1.2
parties:
- links:
- href: https://github.com/defenseunicorns/lula
rel: website
name: Lula Development
type: organization
uuid: C18F4A9F-A402-415B-8D13-B51739D689FF
title: Lula Demo
version: "20220913"
uuid: E6A291A4-2BC8-43A0-B4B2-FD67CAAE1F8F
```

### Composing Validation Templates

If validations are composed into a component definition AND the validation is still intended to be a template, it must be a valid yaml document. For example, the above `validation.tmpl.yaml` is invalid yaml, as the `resource-rule.name` field is not ecapsulated in quotes. A valid yaml version of the above template would be:

```yaml
metadata:
name: Test validation with templating
uuid: 99fc662c-109a-4e26-8398-75f3db67f862
domain:
type: kubernetes
kubernetes-spec:
resources:
- name: podvt
resource-rule:
name: "{{ .const.resources.name }}"
version: v1
resource: pods
namespaces: ["{{ .const.resources.namespace }}"]
provider:
type: opa
opa-spec:
rego: |
package validate
import rego.v1

# Default values
default validate := false
default msg := "Not evaluated"

# Validation result
validate if {
{ "one", "two", "three" } == { {{ .const.resources.exemptions | concatToRegoList }} }
"{{ .var.some_env_var }}" == "my-env-var"
"{{ .var.some_lula_secret }}" == "********"
}
msg = validate.msg

value_of_my_secret := {{ .var.some_lula_secret }}
```
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ require (
github.com/charmbracelet/x/exp/teatest v0.0.0-20240919170804-a4978c8e603a
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc
github.com/defenseunicorns/go-oscal v0.6.0
github.com/google/go-cmp v0.6.0
github.com/hashicorp/go-version v1.7.0
github.com/kyverno/kyverno-json v0.0.3
github.com/mattn/go-runewidth v0.0.16
Expand Down Expand Up @@ -73,7 +74,6 @@ require (
github.com/golang/protobuf v1.5.4 // indirect
github.com/google/btree v1.1.2 // indirect
github.com/google/gnostic-models v0.6.9-0.20230804172637-c7be7c783f49 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/google/uuid v1.6.0 // indirect
Expand Down
18 changes: 16 additions & 2 deletions src/cmd/common/viper.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,12 @@ var (

// Viper configuration error
vConfigError error

// Template config values
TemplateConstants map[string]interface{}

// Template config values
TemplateVariables []template.VariableConfig
)

// InitViper initializes the viper singleton for the CLI
Expand Down Expand Up @@ -66,6 +72,14 @@ func InitViper() *viper.Viper {
// Set default values for viper
setDefaults()

// Load template config
constants, variables, err := GetTemplateConfig()
if err != nil {
panic(err)
}
TemplateConstants = constants
TemplateVariables = variables

return v
}

Expand All @@ -76,8 +90,8 @@ func GetViper() *viper.Viper {

// GetTemplateConfig loads the constants and variables from the viper config
func GetTemplateConfig() (map[string]interface{}, []template.VariableConfig, error) {
var constants map[string]interface{}
var variables []template.VariableConfig
constants := make(map[string]interface{})
variables := make([]template.VariableConfig, 0)

err := v.UnmarshalKey(VConstants, &constants)
if err != nil {
Expand Down
Loading