Skip to content

Conversation

technosophos
Copy link
Member

This fixes some conflicts between 101 and 103 regarding how image mapping is to be done.

Closes #113

This fixes some conflicts between 101 and 103 regarding how image mapping is to be done.

Closes cnabio#113
@technosophos technosophos self-assigned this Feb 26, 2019
@ghost ghost added the review label Feb 26, 2019
@technosophos
Copy link
Member Author

/cc @simonferquel @garethr @jeremyrickard @lachie83

This PR tries to fix the ambiguities mentioned in #113. However, I'm unhappy with my attempt to fix it. The reason is that it pushes the burden on to the bundle developer. This almost seems unavoidable, though, since any other work-around would require the CNAB Runtime to know how to alter and save any image format. That would be an utter nightmare.

Also, there is a statement in @simonferquel 's section that says:

The run tool MAY use this file to modify its behavior, if declarative substitution is not enough.

I am not sure where declarative substitution is defined or what that phrase means.

At the end of the day, I feel like this represents something that can work, but which is not ideal. I tried to devise another way of doing this that would be simpler, but I cannot think of anything.

So... this might be incrementally better than what was there before, but it's not a complete solution.

Copy link
Contributor

@glyn glyn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for mentioning image relocation.


TODO: How do we specify URI is a VM image (or Jar or other) instead of a Docker-style image? Or do we? And if not, why not?
CNAB Runtimes MAY dynamically create or override `refs` sections when appropriate. For example, if an image is relocated as part of a CNAB installation, the runtime may create a `refs` section to indicate that change.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@technosophos Actually, I see image relocation as a responsibility of an invocation image rather than something runtimes need to be concerned about. (From that perspective, the refs section isn't needed in the spec.)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To this point in time, we've assumed the following:

  • Invocation images are never responsible for image management (partly because they may not even know the image type (eg. VM, OCI, MSIX, etc)
  • Information about an image location must be injected into the bundle from outside

That raised the problem that we're trying to solve here. An example goes like this:

I have a bundle that contains a Kubernetes manifest. That manifest points to example.com/foo/bar:v1, but I just exported the image and imported it into an isolated network. The new image was loaded to inner-network.local/foo/bar:v1

The thing that knows about inner-network.local is the CNAB Runtime. The invocation image is blissfully unaware of whether it's on the public network or the inner network, and blissfully unaware of its network constraints. It will work just as well in the external public network as in the internal network.

BUT... when we moved the images, we now need to notify the invocation image that it needs to rewrite its Kubernetes manifest to point to the new inner-network.local/foo/bar:v1 location.

Now, you raise a really good point: If the invocation image knows its own internal file system layout (which it does), then the refs field is not particularly useful. The invocation image itself should have all the internal logic it needs. So I suppose we could just ax that part of the spec. What we would seem to need in this case is just a mapping of old image names to their new image names. /cc @simonferquel <-- is that what you had in mind in your earlier PR?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To this point in time, we've assumed the following:

  • Invocation images are never responsible for image management (partly because they may not even know the image type (eg. VM, OCI, MSIX, etc)

I'd like to challenge this assumption.

I wonder if invocation images might actually be better able to support the image types in their bundles than a runtime, which would need a pretty complex extension mechanism to cope with arbitrary image types.

This would avoid the need for explicit support for image management in the spec or runtime.

We could reduce the complexity for bundle authors, by building management features for docker (and other types of) images into base invocation images and/or adding tooling features (such as porter mixins).

  • Information about an image location must be injected into the bundle from outside

I'd also like to challenge this assumption.

In the use case of image relocation, it would be sufficient to pass a parameter describing the target registry. This would avoid the need for explicit support in the spec or runtime.

That raised the problem that we're trying to solve here. An example goes like this:

I have a bundle that contains a Kubernetes manifest. That manifest points to example.com/foo/bar:v1, but I just exported the image and imported it into an isolated network. The new image was loaded to inner-network.local/foo/bar:v1

The thing that knows about inner-network.local is the CNAB Runtime. The invocation image is blissfully unaware of whether it's on the public network or the inner network, and blissfully unaware of its network constraints. It will work just as well in the external public network as in the internal network.

BUT... when we moved the images, we now need to notify the invocation image that it needs to rewrite its Kubernetes manifest to point to the new inner-network.local/foo/bar:v1 location.

Now, you raise a really good point: If the invocation image knows its own internal file system layout (which it does), then the refs field is not particularly useful. The invocation image itself should have all the internal logic it needs. So I suppose we could just ax that part of the spec.

I agree we could ax that part of the spec.

I'd also like us to consider the option of images being provided to a bundle separately to the invocation image: this is the subject of #95.

What we would seem to need in this case is just a mapping of old image names to their new image names. /cc @simonferquel <-- is that what you had in mind in your earlier PR?

Of course, if we made invocation images responsible for image relocation and passed a target registry as a parameter, then the spec and runtime wouldn't need to provide a mapping.

[An aside: the more features we can omit from v1 of the specification, the easier it will be to finalise the specification and implement runtimes.

Additional spec features may turn out to be desirable as the community gains experience in building runtimes for v1, but I think that's a healthy way to discover spec requirements. Trying to nail down all the spec requirements in v1 is a tall order given that only a few runtimes are currently being developed.

In general, it would be much easier to add features to future spec versions than have to cope with features that later turned out to be unnecessary.]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To my understanding, refs "declarative" replacements were to be handled by the CNAB runtime, whereas invocation images may support an imperative approach (by consuming the injected image map)

That is, after creation of the container, but before starting it, the driver would apply the refs rules without requiring anything within the invocation image.

In my mind, the refs are completely ignored by the invocation image then.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@glyn you make some very good points there.

For now, I am going to focus on removing the refs section as part of this PR. Would you mind opening a new issue that specifically deals with adding whatever is necessary to make it the bundle's responsibility? We can then shift the related conversation to that thread and not lose track of it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@glyn you make some very good points there.

For now, I am going to focus on removing the refs section as part of this PR.

Thanks!

Would you mind opening a new issue that specifically deals with adding whatever is necessary to make it the bundle's responsibility? We can then shift the related conversation to that thread and not lose track of it.

#122

Copy link
Contributor

@glyn glyn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Image relocation need not concern the runtime.

Copy link
Member

@vdice vdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm afraid I won't be much help as to offering an opinion on this PR as a whole, as I'd still need to catch up on the discussion history, but I did have one question after reading through the changes here.

{
"field": "service.image",
"path": "/cnab/charts/values.yaml",
"expressionType": `css-selector`
Copy link
Member

@vdice vdice Feb 27, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was slightly confused to see the expressionType of css-selector in this example for setting a chart value. Was this intentional? Or should it be pcre?

On that note, what would the behavior be if css-selector was used? If I understand the spec correctly, the invocation image would attempt the substitution and should produce an error if it can't be made?

@simonferquel
Copy link
Contributor

After a lot of thought, I think the refs should actually be dropped: it would make the CNAB runtime drivers really complex if we want to implement declarative replacements handling on the runtime side, and as the invocation image itself has the knowledge of its internal layout, it does not really bring any value if we handle them within the invocation image.

If you look at the bundles created by docker-app, you will see that we declare images without refs, and that is the responsibility of the docker-app cnab/run tool to actually handle them properly (the implementation does not do file manipulation, but directly modifies in-memory structures after parsing).

@technosophos
Copy link
Member Author

Okay, I have come around to @simonferquel and @glyn 's point of view. I now think that the ref field should be dropped.

The determining factor to me was realizing that keeping refs as spec'ed either requires (a) the invocation images to get very complex, or (b) the CNAB Runtime to modify the invocation image. Both have some deep problems.

As @simonferquel commented a few days ago, the "general purpose" refs field is actually more complex than just delegating the substitution task directly to the authors of the invocation image. (@glyn also made a similar point).

So I am going to redo this section to simply remove the refs section from the spec.

technosophos added a commit to technosophos/cnab-spec that referenced this pull request Mar 4, 2019
This removes the `refs` section from the specification, and intentionally backs out the definition of how invocation images are to perform substitutions of image names. This decision was arrived at on issue cnabio#117 after trying a few other ways of handling this behavior.

Closes #1113
@technosophos
Copy link
Member Author

I am now closing this issue, superseded by #122 and #123

@ghost ghost removed the review label Mar 4, 2019
technosophos added a commit to technosophos/cnab-spec that referenced this pull request Mar 11, 2019
This removes the `refs` section from the specification, and intentionally backs out the definition of how invocation images are to perform substitutions of image names. This decision was arrived at on issue cnabio#117 after trying a few other ways of handling this behavior.

Closes #1113
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Address redundancy in spec regarding declarative image references and runtime references
4 participants