Add this to Fedora proper?

[Conan-Kudo]

Have you considered getting LXD into Fedora proper? The process to do so isn't particularly difficult...

[ganto]

Thanks for your interest in my packages 😄

To answer your question: yes and no

• Yes, because it would make the user experience much smoother and trustworthy
• No, because it will still be a lot of work to get there from a packaging perspective. Especially the lxd.spec itself needs a big cleanup. First it guess gopkg.in/lxc/go-lxc.v2 has to be split into a separate package. The tests somehow have to be made passing and then the SELinux topic has to be approached. So far I haven't found too many technical people from the Fedora community being interested in LXC/LXD (anymore). Until a few months ago, the lxc-fedora template to create Fedora containers was totally neglected (see Fedora template maintenance lxc/lxc#1356) and my try to get in contact with the Fedora lxc package maintainer to review my rewrite remained unanswered. I think the Fedora world is turning towards atomic and docker which are great technologies, so I doubt the effort for "upstreaming" these packages is worth the time.

I created this repository because I was interested in LXD and didn't have an Ubuntu around 😏. lxd will help me to run Debian containers on my Fedora to test the various DebOps and other Ansible roles I'm developing and maintaining. I'm not using LXD anywhere near production, so the current quality of the packages is good enough for my use case but in my opinion definitely not for an official Fedora package.

Are you using my COPR repo? What do you think about it and about (the future of) LXD on Fedora?

[Conan-Kudo]

@ganto If you're having problems getting in touch with the lxc maintainer and there's a bug report in bugzilla for fixes for lxc packaging, you could petition to become co-maintainer of lxc itself. You can also ask for help on the devel list on what to do about unresponsive maintainers.

That said, I'm one of the snapd maintainers in Fedora, and @kyrofa is one of the Snapcraft developers interested in getting all the Snapcraft functionality working on Fedora. Snapcraft supports using LXD as a backend for "clean" snap builds, so I'm very interested in seeing all of this land in Fedora itself so that Snapcraft can leverage it.

One of the things we're working towards in Snapcraft and snapd is supporting building Fedora-backed snaps, which means having a Fedora environment available in LXD, among other things. Obviously, this means Fedora's LXC support needs to be enhanced and LXD should land in Fedora. :)

As for LXD itself, it's quite nice, and I'd like it to be available in Fedora, regardless of the Snapcraft thing.

[Conan-Kudo]

@ganto As for the SELinux issue, you should probably work with @rhatdan on figuring out what needs to be added to container-selinux module to ensure LXD can work well under an SELinux enforcing system.

[ganto]

Hmn, I don't fully understand. What are you using lxd for when building snaps? When I searched for it I found snapcraft.io: Build on LXD or Docker. Why do you need to use LXD to setup a Ubuntu container to run snap builds in? I would imagine the "Fedora-way" of this would rather be to run snapcraft in a Fedora systemd-nspawn container.

As I can't see any use-case for lxd on a Red Hat-based system in any productive environment that I maintain, I won't get payed for it nor am I motivated to put a lot more effort into this. I'll still try to clean up the RPM as time permits and update the repository if new versions will be released and nothing will become too broken.

If more people interested in a proper lxd package in Fedora will show up or new exciting features will be added to lxd, this might change my motivation. Or maybe there is someone willing to continue from what I built up so far. In either case, leave us a comment below.

[kyrofa]

Hey guys, thanks for looping me in. Indeed, as @Conan-Kudo mentioned snapcraft supports LXD, but doesn't require it. By default it'll build the snap on the host itself, but that would potentially install packages on the host, which is sometimes not desired. So it also supports a "cleanbuild" that spawns a temporary container via LXD to run the build and leave the host alone. Whether or not this container is Ubuntu- or Fedora-based doesn't matter so much as we're working on RPM support.

It's not currently on the roadmap, but we could potentially refactor the cleanbuild system to support multiple backends.

[Conan-Kudo]

@ganto Well, the difference between nspawn and LXD is that nspawn has no templates. You'd need to depend on a tool to build the OS filesystem (dnf/yum, zypper, debootstrap, etc.). That takes a lot more time...

@kyrofa If you're able to, maybe you could help him make LXD on Fedora a thing? :)

[kyrofa]

@ganto I have zero Fedora packaging experience, but it's something I want to learn. Would you be willing to work with a n00b?

[Conan-Kudo]

@kyrofa I'm also willing to help you get started on this, if you'd like.

[kyrofa]

Thanks @Conan-Kudo, I've put this on my backlog. Give me a few weeks to wrap up my current tasking and then I'll surely be reaching out for some advice 😃 .

[Conan-Kudo]

@ganto You could start with some of the generic Go and Python dependencies to get them into Fedora. We can work our way up from the dependencies to the lxd package itself.

[Conan-Kudo]

An update to this: I've proposed changes to the container-selinux policy to fix LXD for Fedora.

@stgraber is working on fixing the SELinux support in LXD itself. So there's no reason that LXD can't start making its way into Fedora proper now.

[ganto]

@Conan-Kudo Neat, thanks the info. Started cleaning up the spec files. However, for lxd-2.18 we first need the version bump to lxc-2.1.0.

[Conan-Kudo]

@ganto According to the LXD documentation, lxc-2.0.0 or newer is fine. Why do you think we need lxc-2.1.0?

[stgraber]

LXD 2.18 will work just fine with LXC 2.0.x. We do version detection and dynamically update our container configs to match the liblxc version you have installed.

[ganto]

Ok, good to hear, then I'll remove the forced version dependency again.

What made me think it's required in the first place was https://github.com/lxc/lxd-pkg-ubuntu/commit/089f11941011b1d80c2915ae809b9da5fe0d9a58 on the corresponding Ubuntu package.

[stgraber]

Yeah, we did the specific version dependency just to avoid upgrade ordering issues. LXD itself is fine either way, it should doesn't like it when you bump from 2.0 to 2.1 while it's running :)

[Conan-Kudo]

@ganto @kyrofa @stgraber:

My PR was merged: containers/container-selinux@0620186

And new packages are being built with the changes now: containers/container-selinux#42 (comment)

[HalisCz]

Hi guys,

I just want to say that I am really excited for LXD and I am looking for proper package on Fedora. I have zero fedora-packaging experience, but if there is anything I can help you with, just let me know.

[ganto]

@Conan-Kudo Thanks a lot for you work on projectatomic/container-selinux for the necessary SELinux label definitions. I'm happy to see that your changes were merged into container-selinux-2.43.0. Currently Fedora 27 is already with container-selinux-2.42.0, so I hope that it will be possible soon to run lxd with SELinux enabled on Fedora 27 👍

Regarding my packaging efforts. I just uploaded lxd-2.20 to COPR which is a huge update in regard of the required Go packages. I had to add or update more than 30 packages just to satisfy the lxd build dependencies. Maybe I could have bundled many of them within lxd but I used the chance to get more experience with Go packaging. With the learnings I cleaned up the Go module spec files and hopefully make them more compliant with the Fedora packaging guidelines. I'm not sure that's fun to maintain them in Fedora upstream or if a bundling would be acceptable. Let's see how it works as it is for now.

On the other hand I also started by first official Fedora package review for a lxd-unrelated RPM (BZ1537398) and so I can get some experience with adding a package to "Fedora proper". Now it seems that I need to find a "sponsor" which would accept me in the "packager" group and attest my skills. If someone reads this who can help me with this, please contact me.

[Conan-Kudo]

On the other hand I also started by first official Fedora package review for a lxd-unrelated RPM (BZ1537398) and so I can get some experience with adding a package to "Fedora proper". Now it seems that I need to find a "sponsor" which would accept me in the "packager" group and attest my skills. If someone reads this who can help me with this, please contact me.

You should be all set there...

[Conan-Kudo]

Any progress on this issue?

[HalisCz]

@ganto any progress on this issue? Is there anything I can help you with?

[ganto]

Thanks a lot for your interest. I'm currently working on getting the lxc-3.0/lxd-3.0 release ready for COPR via ganto/copr-lxc3. There were again a lot of new features and dependency changes which need to be reflected in the RPMs. I hope to have something ready by the end of the easter holidays.

The best thing you can do, if you want to help, is to review the current spec files and RPMs and report every issue you find in the corresponding Github projects:

• ganto/copr-lxc for LXC 2.1
• ganto/copr-lxd for LXD 2.21
• ganto/copr-lxc3 for LXC/LXD 3.0

[ganto]

As you may have seen, initial packages for LXC/LXD 3.0 are now available via ganto/lxc3 COPR repository. Here the following points that still needs to be solved, before LXD can realistically be added to Fedora proper:

• LXD has a huge dependency chain of go modules and many of them are very Canonical-flavoured (e.g. custom forks, juju modules) with sometimes a bit unstable APIs and circular dependencies when running the test suites. As you can see in this repo, i tried to create RPMs for all of the golang modules to build LXD in the past (up to release 2.21) but especially the jump from 2.19 to 2.20 took more than a month of my spare time for packaging all the required Go dependencies and even there I still took some shortcuts by e.g. not including the entire modules in some cases. LXD 3.0 added another bunch more but I definitely don't have time to do that "properly" anymore. If there are not more people helping to maintain this thing, then the only realistic way is to include it to Fedora as bundled built or don't include at all. The current package is mostly in a "works-for-me" state now.

• An additional issue related to bundling was introduced with LXD 3.0. Now LXD ships an embedded version of libsqlitewith a custom implementation for database clustering. An initial announcement of this work by @freeekanayaka was done here. A quick search didn't reveal any result what's the current state regarding upstream integration. Might also be that this needs to be shipped in a separate package when being added to Fedora.

• Although @Conan-Kudo did some great working making sure, the LXD files are labeled properly, SELinux is still an issue (see Starting the lxd.socket with SELinux fails with permission denied #12). The problem relies in the hard-coded path for the unix socket which doesn't conform to what SELinux allows so far. One option would be to get the required policy added to the container-selinux policy set. However, to my understanding this will slightly weaken the entire container policy. The nicer option would be to make the socket path configurable within LXD, so that Fedora could use a path more suitable. This would need some coding skills (btw. I'm not a developer) and cooperation with the LXD developers who might have their own reasons for choosing the current implementation.

• So far I only tested SELinux with the directory backend. There is definitely more testing with other storage backends and different networking setups required here.

• There is an issue with the TestEndpoints_LocalUnknownUnixGroup() test when building with mock on Fedora 27. I patched it with lxd-2.20-000-Fix-TestEndpoints_LocalUnknownUnixGroup-test.patch but this somehow broke the test again for Fedora 28 and Rawhide. Someone has to figure out, what exactly is the problem here...

• The TestHeartbeat from the github.com/lxc/lxd/lxd/cluster fails. Haven't looked into that yet.

If you want to help, I would be glad if you could look into one of the issues mentioned above, help me cleanup and fix the spec files (I'll gladly accept PRs) or simply use LXD from my COPR repository, experiment with one of the many features (various networking setups, various storage backends, clustering) and report when something doesn't work.

[Conan-Kudo]

@ganto

Fedora allows bundling in this particular case, since there's too many weird/forked dependencies. The origin, kubernetes, and docker packages all do this, but you'll need to describe the bundled dependencies via Provides: bundled(golang(dep)) = <githash> statements. You can probably write a simple script to fill in the bundled dep statements similar to what the Origin guys do: https://github.com/openshift/origin/blob/master/.tito/lib/origin/builder/__init__.py#L41-L56

If @kyrofa is still interested in helping, he might be able to help adapt the tito tagger stuff for LXD from the Origin guys so that this works as intended. He's very proficient in Python 3 code. :)

The issue with the LXD socket path is definitely a problem for @stgraber, though. He should be able to do something to make this better for us. Same goes for the tests, as he's the one able to do this stuff.

[ganto]

The latest build lxd-3.0.0-0.2 does now include the new upstream code which allows to set a custom unix socket and now works with SELinux enabled. 👍