I dunno, I think this could confuse people into thinking their privileged containers are being userns-ed as well if they don't look into the documentation or something... but idk

ooo can you put the test stuff in a serperate PR we should merge that soon :)

Thanks @jfrazelle, moved the integration test changes to a separate commit #20117.
Do you think the behavior is still unclear even if you user must explicitly specify --skip-userns-remap or --disable-userns?
@estesp what do you think?

I think there are two things to decide based on this and prior discussions with @mrunalp (and even @rhatdan) on the topic:

• When user namespaces are enabled daemon-wide, is the right answer either (a) --privileged is not available (this is the case today with the current implementation), or (b) should --privileged "turn off" user namespaces for that one container process/instance and also enable the few additional permissions that --privileged turns on?
• Should there be a separate "disable user ns" feature (as is implemented in this PR) which is separate from the notion of --privileged which only disables the use of user and group mappings for a specific container instance/process?

My biggest issue, even though it may seem superfluous, with this particular (second case) implementation is that yet-another-flag seems clunky. At least with --privileged it is somewhat clear what is happening: e.g. I want a "super-admin" container in the current daemon runtime of (usually) user-namespaced processes to do something more privileged as real root. The drawback is for the use case of just wanting the lack of uid/gid mappings without the extra/added capabilities of --privileged

Are there cases where having a "--privileged turns off uid/gid mapping" feature is not sufficient for the desired result?

If you don't make --privileged turn off user namespace, then this is going to break a lot of admin tools that are used to manage the system. This prevents us from considering turning on user namespace. We want to ship SPC (Super Privileged Containers) which can manage the host system and have no need of user namespace.

I am not a fan of what we have done with User Namespace to this point, since it does not give me separation between the containers. But without --privileged turning off all security, I believe we have broken the definition of --privileged.

There are some use-cases where customers need specific 'enhanced' capabilities (e.g., pid=host) and using full-blown --privileged basically violates the principle of least privilege.

Also, in addition to skipping userns remapping, what additional actions must be taken to enable SPC (with respect to current implementation)?

Another simpler option that is inline with other namespaces is having something like --userns=host. We can allow --privileged when that option is set.

Sent from my iPhone

On Feb 9, 2016, at 3:53 AM, Phil Estes notifications@github.com wrote:

I think there are two things to decide based on this and prior discussions with @mrunalp (and even @rhatdan) on the topic:

When user namespaces are enabled daemon-wide, is the right answer either (a) --privileged is not available (this is the case today with the current implementation), or (b) should --privileged "turn off" user namespaces for that one container process/instance and also enable the few additional permissions that --privileged turns on?
Should there be a separate "disable user ns" feature (as is implemented in this PR) which is separate from the notion of --privileged which only disables the use of user and group mappings for a specific container instance/process?
My biggest issue, even though it may seem superfluous, with this particular (second case) implementation is that yet-another-flag seems clunky. At least with --privileged it is somewhat clear what is happening: e.g. I want a "super-admin" container in the current daemon runtime of (usually) user-namespaced processes to do something more privileged as real root. The drawback is for the use case of just wanting the lack of uid/gid mappings without the extra/added capabilities of --privileged

Are there cases where having a "--privileged turns off uid/gid mapping" feature is not sufficient for the desired result?

—
Reply to this email directly or view it on GitHub.

SPC are basically turning off Security Separation and one or more namespaces. --pid=host --net=host --ipc=host --userns=host ...

You also need to volume mount parts of the Host OS into the container. For example

docker run --privileged -v /:/host fedora chroot /host useradd dwalsh

I wrote about it over a year ago.

http://developerblog.redhat.com/2014/11/06/introducing-a-super-privileged-container-concept/

When you move to Container Hosts platforms like Atomic Host and CoreOS, you want to be able ship all software as containers, but some of those containers need to be able to manage the host or manage other containers.

I think @mrunalp's suggestion is the most attractive option I've heard as it keeps it in line with all other flags that turn off namespace separation and provides "commonality" with the host (or another container).

@rhatdan I'm pretty aware you aren't a fan :) but I assume you are well aware that until we have a solid solution for the conflict between layer sharing & file ownership there is no way to use present-day Docker's graph implementation and have separation between containers. I've blogged about and discussed with others many times the "phase 2" ideas, but for now the choices were (a) do nothing at all with userns because we can't do "everything", or (b) do something limited for now until we have the underlying solution required for custom mappings per container. If you can help us drive that into the Linux VFS and/or filesystem community, the faster we can have a reasonable implementation in Docker.

Also, nothing is broken with --privileged today as it is turned off when user namespaces are enabled in the daemon. You get an error if you try to run a privileged container in Docker 1.10 if user namespaces are enabled. This proposal, and a few other issues linked from the first comment are trying to solve that in a clean way, and when we have agreed on the solution, --privileged will work as you have described.

I would love to see the VFS thing fixed, but while I have tried to raise these issues within Red Hat, I am not hopeful for anything changing soon. I believe the File System kernel engineers are not keen on fixing it, probably for justifiable reasons. I do have some on my team investigating a chown -R solution, which would at least give us Container Separation, with a slow down on at container creation time, and could cause certain backends to explode in size, but might work OK for Devmapper and BTRFS.

I am in agreement with --userns=HOST as a good solution. But blocking --privileged when you run with user namespace with the daemon, makes no sense to me. If that is the case we should really block --pid=host and --ipc=host, since the intention of seeing proceses on the host or sharing IPC would also be blocked by DAC. Which would just generate customer problems. In SELinux world if you share IPC or PID with the host, then SELinux gets disabled, since it would just blow things up. When you do --PID=container:UUID --IPC=container:UUID, we make sure the SELinux labels work. We need to do similar things with USER NAMESPACE.

Only bad thing I see happening if you run with --privileged and usernamespace turned off would be if the admin writes to the COW file system, his content will get UID==0 rather then UID=DOCKERROOT, and this admin would be responsible for cleaning up the mess. But in most cases where he would be running with --privileged he probably wants the app to just work, and is not concerned about Security separation. And probably would not run the container again without --privileged.

Not sure what you mean by saying nothing is broken by --privileged not being allowed if User Namespace is enabled on the host. --privileged is broken. I would argue that one of the first thing that users do when faced with a container being blocked by "Permission Denied" is to run the container with --privileged, but if they turn on UserNamespace, they will not be allowed.

So do we have an agreement on one of the following behaviors?

• --userns=HOST - no user namespaces remapping (allow all operations).
• --userns= or nothing: same as today (block --privileged, --pid=host, --ipc=host, ....).

• --userns=HOST - no user namespaces remapping (allow all operation).
• --privileged - no user namespaces remapping (allow all operation).
• --userns or nothing: block --pid=host, --ipc=host, .... but not --privileged.

Also, what is the expected behavior if the user specifies userns=uid?

I would prefer 2. And document the fact that content in the COW image will not have the correct UID/GID when you go to run it with User Namespaces in the future.

Last month in chatting with @mrunalp and others on this topic of mismatched file ownership on new files, I was reminded that the only way that CoW is even involved in this discussion is if we are talking about using docker commit and then tagging this image for running by another user who won't run as privileged and/or with user namespaces disabled.

We can and should make a docs comment about that "commit" scenario, but if you run a privileged container, it has zero impact on the image it was run from, and cannot create files that will be seen by any other user of that image. We can also comment about volumes as volumes shared between user namespace-enabled and disabled containers will have this issue for new files created. But volumes are already an advanced topic, especially when mixing with user namespaces, and there is an understanding this has to be a "self-managed" scenario when you care about file ownership and multiple users in volumes.

@estesp are you ok with proposal 2?

@liron-l my clarification on proposal 2 is that --privileged and --userns=host would, together, resume today's behavior of --privileged alone without user namespaces enabled.

I would still want a user that asks for --privileged without explicitly specifying --userns=host to have the current Docker 1.10 behavior (when the daemon has user namespaces enabled, of course) of getting an error message that --privileged and user namespaces are conflicting options. This way the user has to explicitly ask for --privileged along in combination with disabling a user namespace mapping for that container. Without this, as @jfrazelle mentioned, there may be an expectation that you are running a privileged container with user namespaces still enabled, which would not be the case.

With the known limitations, I don't think a --userns= flag would have any other options than "host" at first. Future use would depend on kernel capabilities or changes. One nice future use would be the specification of a user/uid/group/gid pair which would control the lookups in /etc/sub{u,g}id for handling per-tenant unique mappings when that "phase 2" capability is developed.

@estesp I am fine with this solution, and would also be fine with --privileged and userns enabled at the same time. Bottom line what I don't like is no way to be able to run --privileged if I have enabled userns in the docker daemon.

Thanks @rhatdan, @estesp, I've changed the PR accordingly.
I've also created a PR in docker-archive-public/docker.engine-api#84 with the userns flag @mrunalp suggested.

Ping @estesp, will appreciate a review for this change and the accompanied engine-api change in docker-archive-public/docker.engine-api#84. Thanks

Maintainers discussion; we're okay with an option where using --privileged requires you to set --userns=HOST as well (not sure which of the above options that is).

Possibly add a warning somewhere..

ping @liron-l were you working on this?

Thanks @thaJeztah, I'm waiting for a review for the new parameter in engine-api.
Except from the changes to the API, the code in this commit is complete (and includes testing).

@liron-l is it currently implemented as #20111 (comment)? (sorry didn't dive into the code)

@thaJeztah yes.
If usernamespace mode is enabled, If you specify --userns=HOST everything works normally, while without it --privileged fails (same as today).

@liron-l spotted one more issue, but LGTM otherwise

Thanks @thaJeztah, fixed it.

oh, looks like it needs a rebase now 😢 sorry

Thanks @thaJeztah, rebased.

LGTM, thank you!

ping @vdemeester @MHBauer @SvenDowideit ptal

Exciting 😄

Just one question : should we say something about windows ? (like it's ignored under windows)

LGTM 🐮

WindowsTP4 networking issues are unrelated to this change. Merging 🎉

Belated congrats @liron-l -- thanks for getting this through the process!

Thanks @estesp, @thaJeztah , @jfrazelle , @rhatdan for all the help!

You're welcome! Thanks for contributing!