Skip to content

Conversation

jellonek
Copy link
Contributor

@jellonek jellonek commented Oct 9, 2018

Closes #736


This change is Reviewable

Copy link
Contributor

@ivan4th ivan4th left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you implement my suggestions about wrapping diskimage.Put(), adding a test case to TestDomainDefinitions will be trivial. I'm ok with e2e as a followup but pls add some minimal coverage, it's easy with Golden Master.

Reviewable status: 0 of 2 approvals obtained (waiting on @jellonek)


docs/configmap-as-files.md, line 1 at r1 (raw file):

# Tuning rootfs of VM using config map as files

Maybe "Adding files from ConfigMaps to the root filesystem" ?


docs/configmap-as-files.md, line 6 at r1 (raw file):

image pointed in `container` part of pod definition by using an user selected
[Config Map](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/)
as a source of data.

Virtlet makes it possible to add a set of files from a ConfigMap to the root filesystem of a VM.


docs/configmap-as-files.md, line 10 at r1 (raw file):

## Usage

Prepare config map:

Create ConfigMap:


docs/configmap-as-files.md, line 15 at r1 (raw file):

then add to pod annotations: VirtletConfigMapAsFiles: sample-data.

then add the following to the pod annotations:


docs/README.md, line 16 at r1 (raw file):

    * [Image Name Translation](image-name-translation.md)
* [Diagnostics](diagnostics.md)
* [Tuning rootfs of VM using config map as files](configmap-as-files.md)

Adding files from ConfigMaps to the root filesystem


pkg/diskimage/diskimage_linux.go, line 96 at r1 (raw file):

}

// Put insert files to image creating all necessary subdirs

// Put adds files to the image, making all the necessary subdirs


pkg/diskimage/diskimage_linux.go, line 97 at r1 (raw file):

// Put insert files to image creating all necessary subdirs
func Put(image string, files map[string][]byte) error {

Please add a placeholder for this function to diskimage_unsupported.go, too.


pkg/diskimage/diskimage_linux.go, line 117 at r1 (raw file):

	}

	// Run the libguestfs back-end.

// Run the libguestfs backend


pkg/diskimage/diskimage_linux.go, line 124 at r1 (raw file):

	// Get the list of devices.  Because we only added one drive
	// above, we expect that this list should contain a single
	// element.

// Get the list of devices. As we've added just one drive above, we
// expect this list to contain just one item.


pkg/diskimage/diskimage_linux.go, line 141 at r1 (raw file):

	}

	// Try to mount fist partition of image as /.

Try to mount the first partition of the image as /


pkg/diskimage/diskimage_linux.go, line 150 at r1 (raw file):

		if len(dir) > 1 {
			// Just in case - try to create directory structure
			// as it can be missing.

// Make sure all the containing directories exist


pkg/libvirttools/extdata.go, line 66 at r1 (raw file):

}

func loadConfigMapAsFilesMap(ns, name string) (map[string][]byte, error) {

loadConfigMapAsFileMap


pkg/libvirttools/persistentroot_volumesource.go, line 96 at r1 (raw file):

	}

	if err := diskimage.Put(v.dmPath(), v.config.ParsedAnnotations.FilesForRootfs); err != nil {

This makes this part of code untestable. Look how volume formatting is done in qcow2_flexvolume.go. You need to add PutFiles() method to virt.StorageVolume interface, then add a method that calls the actual Put() to pkg/virtualization/libvirt_storage.go, and a fake implementation that records the call (with the contents of the files!) to pkg/virt/fake/fake_storage.go.

Another thing, you should probably skip this call altogether if v.config.ParsedAnnotations.FilesForRootfs is empty.


pkg/libvirttools/root_volumesource.go, line 106 at r1 (raw file):

	}

	if err := diskimage.Put(volPath, v.config.ParsedAnnotations.FilesForRootfs); err != nil {

Same as above. Don't add direct call to diskimage here. Also, you should probably skip this call altogether if v.config.ParsedAnnotations.FilesForRootfs is empty.


pkg/metadata/types/annotations.go, line 208 at r1 (raw file):

	}

	if configMapAsFilesStr, found := podAnnotations[configMapAsFilesKeyName]; found {

...; found && externalConfigMapLoader != nil


pkg/metadata/types/annotations.go, line 213 at r1 (raw file):

			va.FilesForRootfs, err = externalConfigMapLoader(ns, configMapAsFilesStr)
			if err != nil {
				return fmt.Errorf("error loading config map %q as files: %v",

ConfigMap

@jellonek jellonek changed the title Tuning rootfs using config map as files source [WIP] Tuning rootfs using config map as files source Oct 12, 2018
@jellonek jellonek force-pushed the jell/gfscm branch 4 times, most recently from e0188c4 to ac2b4b4 Compare October 12, 2018 14:25
Copy link
Contributor Author

@jellonek jellonek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 change requests, 0 of 2 approvals obtained (waiting on @jellonek and @ivan4th)


docs/README.md, line 16 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

Adding files from ConfigMaps to the root filesystem

Reworded.


pkg/diskimage/diskimage_linux.go, line 96 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

// Put adds files to the image, making all the necessary subdirs

Done.


pkg/diskimage/diskimage_linux.go, line 97 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

Please add a placeholder for this function to diskimage_unsupported.go, too.

Done.


pkg/diskimage/diskimage_linux.go, line 117 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

// Run the libguestfs backend

Done. Also updated in place from which that was copied.


pkg/diskimage/diskimage_linux.go, line 124 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

// Get the list of devices. As we've added just one drive above, we
// expect this list to contain just one item.

Done. As above.


pkg/diskimage/diskimage_linux.go, line 141 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

Try to mount the first partition of the image as /

Done.


pkg/diskimage/diskimage_linux.go, line 150 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

// Make sure all the containing directories exist

Done.


pkg/libvirttools/extdata.go, line 66 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

loadConfigMapAsFileMap

Kinda done ;)


pkg/libvirttools/persistentroot_volumesource.go, line 96 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

This makes this part of code untestable. Look how volume formatting is done in qcow2_flexvolume.go. You need to add PutFiles() method to virt.StorageVolume interface, then add a method that calls the actual Put() to pkg/virtualization/libvirt_storage.go, and a fake implementation that records the call (with the contents of the files!) to pkg/virt/fake/fake_storage.go.

Another thing, you should probably skip this call altogether if v.config.ParsedAnnotations.FilesForRootfs is empty.

I'll rething how to do that using virt.StorageVolume interface...
Imo there is no need to double check for FilesForRootfs emptiness as it's the first thing done upon entering to this func, in which case it returns immediately.


pkg/libvirttools/root_volumesource.go, line 106 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

Same as above. Don't add direct call to diskimage here. Also, you should probably skip this call altogether if v.config.ParsedAnnotations.FilesForRootfs is empty.

Again. Imo there is no need for double check for emptiness (which is also done as a first thing in Put) and error if call occurred. That way it's imo cleaner.


pkg/metadata/types/annotations.go, line 208 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

...; found && externalConfigMapLoader != nil

Done.


pkg/metadata/types/annotations.go, line 213 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

ConfigMap

Reworded.


docs/configmap-as-files.md, line 1 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

Maybe "Adding files from ConfigMaps to the root filesystem" ?

Reworded.


docs/configmap-as-files.md, line 6 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

Virtlet makes it possible to add a set of files from a ConfigMap to the root filesystem of a VM.

Done.


docs/configmap-as-files.md, line 10 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

Create ConfigMap:

Done.


docs/configmap-as-files.md, line 15 at r1 (raw file):

Previously, ivan4th (Ivan Shvedunov) wrote…

then add the following to the pod annotations:

Done.

@jellonek jellonek force-pushed the jell/gfscm branch 2 times, most recently from 46a0102 to 0b4ab9b Compare October 12, 2018 16:22
@jellonek jellonek changed the title [WIP] Tuning rootfs using config map as files source Tuning rootfs using config map as files source Oct 17, 2018
Copy link
Contributor

@ivan4th ivan4th left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 5 of 11 files at r1, 8 of 8 files at r2.
Reviewable status: 1 change requests, 0 of 2 approvals obtained

Copy link
Contributor

@ivan4th ivan4th left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: 1 of 2 approvals obtained

Copy link
Contributor

@lukaszo lukaszo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewable status: 1 of 2 approvals obtained (waiting on @jellonek)


docs/writing-files-to-rootfs.md, line 3 at r2 (raw file):

# Writing files to root filesystem before VM boot

Virtlet makes it possible to write set of files from to the root filesystem of a VM using

from to?


docs/writing-files-to-rootfs.md, line 21 at r2 (raw file):

where entry is any name, entry_name contains path where file should be

instead of any name I would write any text

@lukaszo lukaszo merged commit 20cec5e into master Oct 17, 2018
@lukaszo lukaszo deleted the jell/gfscm branch October 17, 2018 10:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants