(currently this guid uses rsync to have the same files in the docker container as in the CLion projects, since this is the default and gives CLion control over what is happening. It is probably preffered to use docker volume mounting instead. Wdyt?)

Do you think, it's a good idea to put together a dedicated Dockerfile for development? That might save some steps from the tutorial.

@mrks What do you think?

I'm currently a bit swamped trying to get the CI server running again. Will look at this soon.

I have no idea about Docker and what that means for the compiling/execution performance of Hyrise, but since a lot of students appear to use CLion, it sounds like a good idea to put that guide down. Maybe not in the README to keep it concise, but linked in the Wiki?

We discussed this in the Fortnightly before you joined. @julianmenzler will test the tutorial just to make sure, after which we can dicuss the best location.

Nice. Would really love to see getting this merged and supported. It'll be a tremendous time saver for future DYOD students running Linux systems. And for anybody who has to debug issue that are only reproducible in the CI environment using the familiar CLion debugger.

I have no idea about Docker and what that means for the compiling/execution performance of Hyrise, but since a lot of students appear to use CLion, it sounds like a good idea to put that guide down. Maybe not in the README to keep it concise, but linked in the Wiki?

When the docker host runs Linux, there will be almost no overhead, since the processes in the docker container run directly in the host's kernel. They only use lightweight namespace isolation primitives like cgroups, an isolated file system and isolated networking.

On Windows there will be the overhead of WSL (ie. Linux syscalls are translated to Windows Kernel Syscalls) (only for the latest versions of Docker).
Mac users would pay the full overhead of a Linux VM that Docker for Mac uses under the hood. But I think the project already supports Mac users quite well in its current configuration anyway ;).

@julianmenzler let me know if you have any questions. ✌️

In the context of the Bachelor's project, we are currently having a discussion whether docker is the right technology for creating development environments. On the plus side: we already have a docker setup for the CI but setting up a development environment in a docker apparently conflicts with a couple of Docker's dogmas. @Bouncner mentioned that we used Vagrant for Hyrise1 which focuses on a simple, lightweight setup for a development environment.

What would be the advantage of that?

lightweight setup

Doesn't it start a VM?

I don't know. The Docker disciples just said/wrote that using docker containers for development is against Docker's philosophy. Actually, I don't care.

Compile errors under macOS:

https://gist.github.com/julianmenzler/166b64a3fdf0447ba67d7239a97fa79c

Could you try building Hyrise in the container without CLion (i.e., creating your own build folder, running cmake, make)

Compile errors under macOS:
https://gist.github.com/julianmenzler/166b64a3fdf0447ba67d7239a97fa79c

Also had that problem before when trying to get hyrise running on my ubuntu (non docker) dev machine. (I recall fixing it by changing GCC to CLang). Takes some time to figure this out tho. ✌️

But if you cant build it that it should fail in CI also tho, right?

Very weird.

But if you cant build it that it should fail in CI also tho, right?

That's what I would expect, too. Especially as this runs in a VM and Mac should have no influence.

Could you try building Hyrise in the container without CLion (i.e., creating your own build folder, running cmake, make)

I tried this today. Same issue.

Also, I made a clean checkout from within the container and started compiling using make -j8. This time no problems with libjemalloc, but other errors occurred.
However, I remembered your suggestion (@mrks) from our meeting:make -j1 did the trick. Problems with macOS and Clion remain though.

As a next step, I'll try out the tutorial in a Linux setting. I already set up a VM with Linux and Clion today. I'll keep you posted.

Problems with macOS and Clion remain though.

Could you try this: https://youtrack.jetbrains.com/issue/CPP-349#focus=streamItem-27-890460.0-0

Update

• Tutorial works for Ubuntu 20.04 and Clion 2020.1
• I give up on getting this to work under macOS Catalina and Clion 2020.1

Could you try this: https://youtrack.jetbrains.com/issue/CPP-349#focus=streamItem-27-890460.0-0

Clion triggers the following command

/cmake-3.16.0/bin/cmake --build /tmp/tmp.dzpGmw5myc/cmake-build-debug_docker --target hyriseTest -j 1

So -j 1 is not a problem. Something else seems to get messed up when Clion copies over the project files I guess.
(To make sure, I also executed make -j1 manually, same issue with jemalloc)

I suggest we put a disclaimer in front of the tutorial. Something like "Works on Linux systems only."

Another question to be answered: Shall we make this a separate Wiki-page?

Shall we make this a separate Wiki-page?

Yes, and link it under "Development" in the menu. Could you make the necessary changes to the text and move it there? I'd also link it in README.md.

Shall we make this a separate Wiki-page?

Yes, and link it under "Development" in the menu. Could you make the necessary changes to the text and move it there? I'd also link it in README.md.

Ok, I take care of that.

Ok, I'm done. Here it is: https://github.com/hyrise/hyrise/wiki/Use-Docker-with-CLion

Great! Thanks to both of you!

Where does ENABLE_JIT_SUPPORT come from? That has been removed a long time ago. We should grep the code and the wiki for it.

If you change this PR to simply link the wiki page, we can merge and close it, too.

Could you merge the current master, please? Otherwise, this might fail because of that server test.

Great! Thanks to both of you!

Where does ENABLE_JIT_SUPPORT come from? That has been removed a long time ago. We should grep the code and the wiki for it.

If you change this PR to simply link the wiki page, we can merge and close it, too.

Good to know! I copied that part from my personal config. I removed it.

If you change this PR to simply link the wiki page, we can merge and close it, too.

Done

By the way, is the following paragraph still relevant?

Inside the container, you can then checkout Hyrise and run ./install_dependencies.sh to download the required submodules.

From my view, the Dockerfile already ensures that dependencies get set up. Or is there any difference?

The submodules are not part of the Docker container. The rest should be there, but as the CI run also runs the script, I'd suggest that people do so, too. That way, we can be reasonably sure that everything that works in CI also works for them.

Alright, thanks for clarifying. 👍

Why I put the install_dependency section in was because I needed a way of only installing the submodules and not the other dependencies that would be installed in the hosts apt-system.

So maybe just a reminder in the tutorial that the submodules have to be set up to make Docker work could suffice.

Have you thought about whether we should switch to a volume-mounting approach instead of rsyncing all the files? I guess that volume mounting has a bunch of advantages. Then there is no overhead for copying the files and you wouldn't need to provide the disc space for 2 copies of hyrise.

So maybe just a reminder in the tutorial that the submodules have to be set up to make Docker work could suffice.

The Dockerfile takes care of fetching all dependencies required for compiling Hyrise. Project files, including the git submodules, have to be cloned by the developer himself.

Have you thought about whether we should switch to a volume-mounting approach instead of rsyncing all the files? I guess that volume mounting has a bunch of advantages. Then there is no overhead for copying the files and you wouldn't need to provide the disc space for 2 copies of hyrise.

Indeed, I thought about this. But it might break CLion's remote workflow and requires further testing. Most people will set up a Hyrise dev environment without Docker anyway. So I guess spending more resources on this guide is not necessary. At least for now.