[ANN] Mollymawk and Albatross orchestrating all virtual machines

Community
, ,
1

Dear everyone,

we just finished and deployed Albatross and Mollymawk to support virtual machines that are not MirageOS unikernels.

The reasoning is simple: we embedded so many nice features (metrics, console output, deploying via web UI, multi-tenant, startup dependencies, restart-on-failure,..) that we really wanted to deploy our non-MirageOS virtual machines as well with the same mechanism and don’t have duplicated code all over.

For the time being, it only supports FreeBSD BHyve, but let us know if you’re interested in other virtualization technologies and we will prioritize that work!

We also wrote a brief blog article with screenshots: Robur's blog - Mollymawk supporting other virtual machines

11 Likes
2

Congratulations!

Have you considered using libvirt as an abstraction to underlying hypervisor? And do you think solo5 tender could be controlled via libvirt?

I’m researching how to setup a Xen hypervisor and the possibility of managing virtual machines and unikernels on Xen via Albatross would be neat.

3

I have briefly looked into libvirt, and it was rather complex, so I decided to not use it in Albatross.

Albatross has not been developed for the Xen use-case – which is pretty different from the hvt approach, so I’d expect there’d be quite some work needed, and in addition quite some code not used in Albatross. I’m not sure how widely people are using Xen these days (apart from QubesOS), so I’m a bit sceptical whether “Xen support in Albatross” would be a worthwhile goal. Do you use Xen on a server yourself?

4

Fair enough. My arguments for using libvirt is that it abstracts over other popular hypervisors like KVM (or Xen) and not be tied to Bhyve. But I don’t know much about libvirt and can understand that the complexity was not worth it.

Not yet, just researching best fits. Xen appeals to me as a type-1 hypervisor and thus much smaller trusted computing base.

5

Great work! I am setting up my server, and I would be interested in using Albatross and Mollymawk to manage non-MirageOS on a Linux-based system.

6

I find the support for FreeBSD’s bhyve interesting. How would you compare with Linux’s KVM in terms of performance, reliability and feature set ? Some top level insights would be useful to me. Are there areas where bhyve is actually better than KVM ?

7

Thanks for your interest. I didn’t conduct any benchmarks, but I’m sure you’ll find some on the Internet or be able to conduct some yourself.

I won’t go into the “FreeBSD vs Linux” discussion in here, I’m sure there are plenty of (opinionated) resources available that will help you to get a top level insight.

8

So what is your current invocation how you start a virtual machine? Is it using qemu? What kind of parameters do you pass?

9

Thanks ! Ok I understand that you don’t want to get drawn into any FreeBSD/Linux comparison.

In my experience there is often poor quality information available about OSes other than Linux/Windows on the internet. The usage of FreeBSD is quite miniscule so that is to be expected.

I’m not interested in benchmarks… rather your general experience with bhyve if you’d like to share that with everyone. No need to compare with Linux if you don’t want to.

10

Sorry, I haven’t started using VM, I am still in the process of defining what I will run and how.

11

I use since more than two decades FreeBSD as my daily driver, and also on servers I maintain.

Since more than a decade I use the zfs file system (ZFS - Wikipedia) – nowadays available on GNU/Linux via OpenZFS as well. I do use geli for encryption of drives (plus some testruns of zfs encryption).

I have to say, I don’t understand how people can run services and laptops without such a snapshottable file system. Updating is easy. Backups are easy. Documentation is great.

Also, when it comes to documentation, whenever I’m unsure I look up in the FreeBSD handbook (FreeBSD Handbook | FreeBSD Documentation Portal) – a well-wrtitten, up-to-date book on how to configure FreeBSD. I also find the manual pages very well written and understandable.

Why I use FreeBSD is as well that to me it feels like home, I have an easy time to debug and change the system I’m using, compiling a new kernel (or a user-space utility) without any fuzz (as example, recently I had to connect to a wireless network with enterprise (802.1x) authentication, and it used legacy ciphers – so I needed to change the call in wpa_supplicant to OpenSSL – I wouldn’t know where to start and what to do on a Linux system). Various utilities are shipped with the base system, which makes the pure base already very usable (ifconfig / tcpdump / …).

I tried to use GNU/Linux as my daily driver, and was overwhelmed by the number of daemons that are running, and how to figure out where the source code of something is and how to compile it (as a package) feels very tricky to me. Also, their system of initrd etc. is very complex to my brain.

I do understand and appreciate that other people are using and are happy with their Linux systems. It’s just not my cup of tea.

Now, BHyve in my perspective performs well, works nicely, and has its limitations: when I first tried on my laptop I couldn’t execute more than 1600 virtual machines (doing the MirageOS unikernel development) – I asked the original author, and he mentioned that this is uncharted territory. In production, I do run fewer virtual machines, and I don’t run into issues there.

Obviously the development of FreeBSD (similar to Linux as far as I understand) is driven a lot by commercial users who employ developers (for FreeBSD, this is for sure NetApp, Netflix, etc. – take a look at Donors | FreeBSD Foundation is this is interesting.

4 Likes
12

Really appreciate your response regarding BHyve and FreeBSD in general.

I also use FreeBSD occasionally and agree with a lot of the points you made in your reply. I use vm as the Bhyve front end and while basic, it gets the job done.

I have to say, I don’t understand how people can run services and laptops without such a snapshottable file system. Updating is easy. Backups are easy.

The same thing is definitely possible in Linux via btrfs snapshots – there are some utilities available that ease the whole process. However the simplicity of FreeBSD’s bectlexperience is unbeatable.

I tried to use GNU/Linux as my daily driver, and was overwhelmed by the number of daemons that are running, and how to figure out where the source code of something is and how to compile it (as a package) feels very tricky to me. Also, their system of initrd etc. is very complex to my brain.

I came into FreeBSD from Linux. I would say that FreeBSD is an excellent system but not any less complex than Linux. It’s just a matter of getting used to things. I personally love Fedora and Chimera Linux (incidentally Chimera Linux is quite similar in philosophy to FreeBSD in the Linux world).

For me, learning to use FreeBSD has been a very pleasant experience. Once you use ZFS you understand what the fuss is about. You also realise the value of a cohesive system in which kernel and userspace are developed together. It’s also great that clang is the FreeBSD system compiler and the FreeBSD libc is so simple in comparison to Linux’s glibc. (musl libc is quite good though).

To anybody reading this I would recommend trying out FreeBSD. It will give your unix/computing concepts a nice boost. I still prefer Linux but I love FreeBSD too :slight_smile: .