feat: add audit mode and reporting in mutate and generate rules

proposals/mutate-and-generate-reporting.md

@@ -0,0 +1,165 @@
+# Meta
+[meta]: #meta
+- Name: Audit mode and reporting in mutate and generate rules
+- Start Date: 12 August 2024
+- Update data (optional): 12 August 2024
+- Author(s): vishal-chdhry
+
+# Table of Contents
+[table-of-contents]: #table-of-contents
+- [Meta](#meta)
+- [Table of Contents](#table-of-contents)
+- [Overview](#overview)
+- [Motivation](#motivation)
+- [Proposal](#proposal)
+- [Implementation](#implementation)
+  - [Link to the Implementation PR](#link-to-the-implementation-pr)
+- [Drawbacks](#drawbacks)
+- [Alternatives](#alternatives)
+- [Unresolved Questions](#unresolved-questions)
+
+# Overview
+[overview]: #overview
+This proposal discusses ways to capture mutate and generate rule results in existing policy reports.
+
+# Motivation
+[motivation]: #motivation
+
+Currently, reporting is only supported in validate rules. While this is nice in that it allows a report to be generated showing the cluster resources that violate the validation rules, it doesn't show the resources that would have either been mutated or generated by Kyverno policy. As an operator, administrator, or manager who may have multiple clusters under management, this makes it impossible to know if the mutate or generate rules about to be pushed out are safe and effective.
+
+# Proposal
+
+The `properties` field in policy report result is used to add information regarding mutation and generation. A `type` property is used to indicate if the report is for mutate, validate or generate.
+For mutate rules, the policy report should indicate, minimally,:
+- jsonpatch of the mutation
+- target of the mutation
+
+The report results will have a property called `patches` which has 2 properties: `jsonpatches` and `target`, `target` is the resource that is being mutated and `jsonpatches` are the patches applied to the target.
+```json
+{
+  "patches": [
+    {
+      "target": {
+        "apiVersion": "v1",
+        "kind": "Pod",
+        "namespace": "default",
+        "name": "sample-resource",
+        "uid": "bb228314-0e3e-42c2-b945-63efe9279ad4"
+      },
+      "jsonpatches": [
+        {
+          "op": "replace",
+          "path": "/baz",
+          "value": "boo"
+        },
+        {
+          "op": "add",
+          "path": "/hello",
+          "value": ["world"]
+        },
+        {
+          "op": "remove",
+          "path": "/foo"
+        }
+      ]
+    }
+  ]
+}
+```
+
+Example:
+```yaml
+apiVersion: wgpolicyk8s.io/v1alpha2
+kind: PolicyReport
+metadata:
+  name: sample-mutation-policy-report
+summary:
+  pass: 8
+  fail: 2
+  warn: 0
+  error: 0
+  skip: 0
+results:
+  - policy: mutation-policy
+    message: resource `sample-resource` mutated
+    result: pass
+    rule: mutation-rule
+    scored: true
+    severity: medium
+    source: kyverno
+    timestamp:
+      nanos: 0
+      seconds: 1681659534
+    properties:
+      type: mutation
+      jsonpatch: '[{ "op": "replace", "path": "/baz", "value": "boo" },{ "op": "add", "path": "/hello", "value": ["world"] },{ "op": "remove", "path": "/foo" }]'
+  - policy: mutate-target-policy
+    message: resource `sample-resource` mutated
+    result: pass
+    rule: mutation-target-rule
+    scored: true
+    severity: medium
+    source: kyverno
+    timestamp:
+      nanos: 0
+      seconds: 1681659534
+    properties:
+      type: mutation
+      patches: '[{"target":{"apiVersion": "v1", "kind": "Pod", "namespace": "default", "name": "sample-resource", "uid": "bb228314-0e3e-42c2-b945-63efe9279ad4"},"jsonPatches": [{ "op": "replace", "path": "/baz", "value": "boo" },{ "op": "add", "path": "/hello", "value": ["world"] },{ "op": "remove", "path": "/foo" }]}]'
+```
+
+For generate rules:
+- List of resources that were generated by the policy
+Example:
+```yaml
+apiVersion: wgpolicyk8s.io/v1alpha2
+kind: PolicyReport
+metadata:
+  name: sample-generate-policy-report
+summary:
+  pass: 8
+  fail: 2
+  warn: 0
+  error: 0
+  skip: 0
+results:
+  - policy: generate-policy
+    message: resource `sample-resource` mutated
+    result: pass
+    rule: generate-rule
+    scored: true
+    severity: medium
+    source: kyverno
+    timestamp:
+      nanos: 0
+      seconds: 1681659534
+    properties:
+      type: generation
+      resources: '[{"apiVersion": "v1", "kind": "Pod", "namespace": "default", "name": "sample-resource", "uid": "bb228314-0e3e-42c2-b945-63efe9279ad4"}]'
+```
+
+# Implementation
+
+This is the technical portion of the KDP, where you explain the design in sufficient detail.
+
+The section should return to the examples given in the previous section, and explain more fully how the detailed proposal makes those examples work.
+
+## Link to the Implementation PR
+
+# Drawbacks
+
+Changes the structure of how we manages engine response and patches internally for mutation and resource generation in generate.
+
+# Alternatives
+
+For mutation, we can show the final resource in the reports instead of the jsonpatches, as the jsonpatch can get overwritten by other rules
+
+# Unresolved Questions
+
+- Currently, when the policy is deleted, the report for that policy also gets deleted. When OrphanDownstreamOnPolicyDelete is set to true, the resource stays after the policy is deleted, should the report also get orphaned when OrphanDownstreamOnPolicyDelete is `true`?
+- Should the report for mention details such as
+  For generate:
+    - Has the resource has been cloned from a list?
+    - Is it being synchronised?
+  For mutate:
+    - Is it a mutate existing rule?