Skip to content

Conversation

@dariusarnold
Copy link
Contributor

@dariusarnold dariusarnold commented Mar 8, 2025

This allows both interactive use in IDEs/editors with dev container support as well as building inside the dev container from the command line. The changes are based on https://github.com/kon-foo/InfiniSim/tree/docker-builder-image.

The original PR was extended with the following features:

  • ccache and ninja are installed for optional faster builds. Using ninja instead of make speeds up the build from ~ 28 s to 17 s on my machine with 16 threads.
  • git and sudo are added for interactive use of the devcontainer
  • Store bash history in volume to preserve it across rebuilds
  • The non-root user ubuntu with sudo password "ubuntu" was added and is used to avoid producing build artifacts owned by root
  • Split args into CMake generation time and CMake build time args
  • Allow passing the build directory in docker run

kon-foo and others added 4 commits February 24, 2025 18:29
This allows both interactive use in IDEs/editors with dev container support as well as building inside the dev container from the command line. The changes are based on https://github.com/kon-foo/InfiniSim/tree/docker-builder-image.

The original PR was extended with the following features:
- ccache and ninja are installed for optional faster builds. Using ninja instead of make speeds up the build from ~ 28 s to 17 s on my machine with 16 threads.
- git and sudo are added for interactive use of the devcontainer
- Store bash history in volume to preserve it across rebuilds
- The non-root user infinitime with sudo password "it" was added and is used to avoid producing build artifacts owned by root
- Split args into CMake generation time and CMake build time args
- Allow passing the build directory in docker run
@NeroBurner NeroBurner added the enhancement New feature or request label Mar 9, 2025
dariusarnold and others added 3 commits March 9, 2025 17:50
Co-authored-by: NeroBurner <pyro4hell@gmail.com>
- Change the container user to ubuntu:ubuntu since that one already exists anyway. There were also permisison issues stemming from not using ubuntu since the external user would likely have UID 1000 (at least on single user systems). That would lead to external files mapped into the container being owned by ubuntu, not infinitime, which has UID 1001
- Split installation into three sections to increase layer granularity for caching
- Remove node install script after usage to reduce container size
@dariusarnold
Copy link
Contributor Author

dariusarnold commented Mar 9, 2025

Thank you for the review @NeroBurner, I think I integrated all your findings.

I also changed the following:

  • Change the container user to ubuntu:ubuntu since that one already exists anyway. There were also permisison issues stemming from not using ubuntu since the external user would likely have UID 1000 (at least on single user systems). That would lead to external files mapped into the container being owned by ubuntu, not infinitime, which has UID 1001
  • Split installation into three sections to increase layer granularity for caching

If you want, I can look into using the container in the pipeline as well. That would ensure that the container itself builds and the application builds inside the container. I would create a second PR for that if you are interested.

Copy link
Collaborator

@NeroBurner NeroBurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the effort! The dev-container will make it easier for new contributors to quickly jump in, thanks!

Regarding whether or not to use the docker container for the CI action I'm undecided. On the one hand I like to reduce code duplication, which is in favor of using docker container. On the other hand the current workflow is minimal, and I don't know what the runtime impact is of building the container vs doing the installs directly. Additionally I don't know what special runner is needed to create a docker container in CI (as far as I understand you need a docker in docker capable runner to create a docker container in CI).

So I'm generally open for docker in CI, just need a bit of hand holding until I can agree

dariusarnold and others added 5 commits March 11, 2025 15:31
…o matching entries in passwd file" when build dev container in CLion

Clion tries to read the local /etc/password when building to change the UID of ther user in the container to the one from outside. If your /etc/passwd does not have this entry, it will fail to build.
Comment this in if you do want to use the dev container CLion (at least CLion 2024.3.4, Build #JBC-243.25659.42. It is reported to Jetbrains).
Suggestions from code review

Co-authored-by: NeroBurner <pyro4hell@gmail.com>
@dariusarnold
Copy link
Contributor Author

dariusarnold commented Mar 11, 2025

I added all review suggestions and tested this in CLion CLion 2024.3.4 and VsCode and (had to add a workaround for a bug in CLion), the devcontainer works in both.
Regarding CI usage, I can experiment in my fork to find out about the runtime impact. It is a separate change anyway.

Copy link
Collaborator

@NeroBurner NeroBurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the improvements!

@NeroBurner NeroBurner merged commit 8cb1ef4 into InfiniTimeOrg:main Jun 16, 2025
1 check passed
&& chown -R $USERNAME /commandhistory \
&& echo "$SNIPPET" >> "/home/$USERNAME/.bashrc"

USER ubuntu
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should have tried this earlier. But still. Adding the USER ubuntu line breaks the container for me when using podman

I get the following:

$ podman run --rm -it -v ${PWD}:/sources infinisim-build
CMake Error: Unable to (re)create the private pkgRedirects directory:
/sources/build/CMakeFiles/pkgRedirects

Removing the USER ubuntu line everything builds fine

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NeroBurner I am not familiar with podman, I have only used Docker so far. I guess you have a permission issue. You bind mount the sources into the container, which will keep the UID and GID from your local user. Everything inside the container runs as the ubuntu user, which likely has a different UID and GID than the local user. So the processes inside the container can't write to /sources due to missing permissions.
The dev container implementation automatically updates the UID/GID from the container user to match the local one (see the updateRemoteUserUID property in devcontainer.json), but when using just podman as a container engine, you will have to specify that explicitly (I guess, at least for Docker you would have to). Again, I have not used Podman, but it seems it supports that with --userns/--uidmap/--gidmap.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

podman runs in userland without the need of a docker-daemon running as root. And as such some permissions are restricted.

When running inside the container (without the USER ubuntu) with podman the shown user is root. But the resulting binaries of the bind mount (the build dir) have the host user as owner.

So I'd like to remove the USER ubuntu part of the Dockerfile and update the README.md with --user $(id -u):$(id -g) for docker users.

could you try that out if that would work (as I don't have docker installed)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested the following after removing the USER ubuntu from the Dockerfile (all without a user ubuntu on my local machine):

  1. Rebuilding the dev container with VsCode, building InfiniSim inside the container
    -> The binaries are owned by the local user and can be run outside of the container as well.
  2. Building InfiniSim from command line with docker run --rm -it -v ${PWD}:/sources --user $(id -u):$(id -g) infinisim-build
    -> Same result as in 1.
  3. Build the container in CLion and build InfiniSim in the IDE
    -> Same result as in 1.

Good results and all use cases still work.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome! thanks for checking! 🙇

would you be willing to open a PR and update the readme and docker container? if not it's also OK as you were already very helpful!

Copy link
Contributor Author

@dariusarnold dariusarnold Jun 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I can do that.

Edit: Done, please check #178

dariusarnold added a commit to dariusarnold/InfiniSim that referenced this pull request Jun 19, 2025
Following the issue described in InfiniTimeOrg#174 (comment).
This prevents permission issues when using the dev container with podman, which is running rootless. Usage with docker requires explicitly passing UID and GID of the local user, which is added to the README.
NeroBurner added a commit that referenced this pull request Jun 20, 2025
Following the issue described in #174 (comment).
This prevents permission issues when using the dev container with `podman`, which is running rootless.
Usage with `docker` requires explicitly passing `UID` and `GID` of the local user, which is added to the README.

Add podman command to `README.md`

Co-authored-by: NeroBurner <pyro4hell@gmail.com>

---------

Co-authored-by: NeroBurner <pyro4hell@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants