Use run image digest if it exists in local daemon copy

[zmackie]

Description

In exploring the issue of reproducibility, I unpacked why we have different configurations for the following cases:

Case 1 (create image locally and push)

± zm |master → origin ✓| → pack build -p apps --builder cnbs/sample-builder:alpine zmackie/localtest:local-push
alpine: Pulling from cnbs/sample-builder
Digest: sha256:9e3cfea3f90fb4fbbe855a2cc9ce505087ae10d6805cfcb44bd67a4b72628641
Status: Image is up to date for cnbs/sample-builder:alpine
alpine: Pulling from cnbs/sample-stack-run
Digest: sha256:f611081e086fcc7f7b8b1eb3b9844f05a34f65dba40656b7bf0d31742964ebbc
Status: Image is up to date for cnbs/sample-stack-run:alpine
===> DETECTING
[detector] io.buildpacks.samples.hello-world 0.0.1
[detector] io.buildpacks.samples.hello-moon  0.0.1
....SNIP....
[builder] ---> Done
===> EXPORTING
[exporter] Adding layer 'launcher'
[exporter] Adding 1/1 app layer(s)
[exporter] Adding layer 'config'
[exporter] *** Images (dc9d923ceb08):
[exporter]       index.docker.io/zmackie/localtest:local-push
Successfully built image zmackie/localtest:local-push

± zm |master → origin ✓| → docker push zmackie/localtest:local-push
The push refers to repository [docker.io/zmackie/localtest]
ffb0454fbd92: Pushed
ae47863df37c: Pushed
6fb83b26bebe: Pushed
06121a438e08: Layer already exists
ec604747e691: Layer already exists
531743b7098c: Layer already exists
local-push: digest: sha256:96c284524fddcaf72fe4da5cc266ef50a2d8c2d033adc47470ec220ae2253453 size: 1575

± zm |master → origin ✓| → crane digest zmackie/localtest:local-push
sha256:96c284524fddcaf72fe4da5cc266ef50a2d8c2d033adc47470ec220ae2253453

*Case 2 (create image remotely with --publish)

± zm |master → origin ✓| → pack build -p apps --builder cnbs/sample-builder:alpine zmackie/localtest:publish --publish
alpine: Pulling from cnbs/sample-builder
Digest: sha256:9e3cfea3f90fb4fbbe855a2cc9ce505087ae10d6805cfcb44bd67a4b72628641
Status: Image is up to date for cnbs/sample-builder:alpine
===> DETECTING
[detector] io.buildpacks.samples.hello-world 0.0.1
[detector] io.buildpacks.samples.hello-moon  0.0.1
===> ANALYZING
[analyzer] Warning: Image "index.docker.io/zmackie/localtest:publish" not found
===> RESTORING
===> BUILDING
...SNIP....
===> EXPORTING
[exporter] Adding layer 'launcher'
[exporter] Adding 1/1 app layer(s)
[exporter] Adding layer 'config'
[exporter] *** Images (sha256:9498b3674d4c3f21a9a48841a909ec1f0d5b84bb458a96839a79b24714e6f66f):
[exporter]       index.docker.io/zmackie/localtest:publish
Successfully built image zmackie/localtest:publish

± zm |master → origin ✓| → crane digest zmackie/localtest:publish
sha256:9498b3674d4c3f21a9a48841a909ec1f0d5b84bb458a96839a79b24714e6f66f

I then fetched the configs and dug into them, which turned up the following diff, which is what I thought would be the case:

± zm |master → origin ?:4 ✗| → diff localtest:local-push-metadata.json localtest:publish-metadata.json
27c27
<     "reference": "30f444709a09ed2e1ab0673d42f823e0e14ba9b6bc7fc897d2295dc9ea509ccf"
---
>     "reference": "index.docker.io/cnbs/sample-stack-run@sha256:f611081e086fcc7f7b8b1eb3b9844f05a34f65dba40656b7bf0d31742964ebbc"

However, the run image used in the local case exists in my local daemon, and has a digest reference:

± zm |master → origin ?:4 ✗| → docker images cnbs/sample-stack-run --digests
REPOSITORY              TAG                 DIGEST                                                                    IMAGE ID            CREATED             SIZE
cnbs/sample-stack-run   alpine              sha256:f611081e086fcc7f7b8b1eb3b9844f05a34f65dba40656b7bf0d31742964ebbc   30f444709a09        6 days ago          8.09MB


± zm |master → origin ?:4 ✗| → docker inspect 30f444709a09
[
    {
        "Id": "sha256:30f444709a09ed2e1ab0673d42f823e0e14ba9b6bc7fc897d2295dc9ea509ccf",
        "RepoTags": [
            "cnbs/sample-stack-run:alpine"
        ],
        "RepoDigests": [
            "cnbs/sample-stack-run@sha256:f611081e086fcc7f7b8b1eb3b9844f05a34f65dba40656b7bf0d31742964ebbc"
        ],
...SNIP...

Proposed solution

If the local image has a single digest entry in its docker config (config.RepoDigest) we can populate the io.buildpacks.lifecycle.metadata.reference field with that data and therefore end up with the same configuration.

Describe alternatives you've considered

Additional context

[dgageot]

I have the feeling that this should append in the lifecycle, not in pack, right?

[zmackie]

@dgageot That's probably right. @ekcasey what you think?

[ekcasey]

@dgageot is right that the lifecycle adds this label. I think the change actually needs to be made in imgutil however. A LocalImage should return a DigestIdentifier if it finds one with a repository matching the original name. I'll move this to lifecycle b/c of the broader context and create a lower level issue on imgutil

[dfreilich]

lower level issue: buildpacks/imgutil#32

[natalieparellano]

Potentially related buildpacks/rfcs#203