Calcx Returns: The Dark Side of CI/CD

I have mentioned my longest-running programming project so far a couple of times before on this blog. If you do not know what calcx is, read the following and come back afterwards: First Post about Calcx Second Post about Calcx

Now that you know what's up, here's a quick rundown of what has been happening since the posts were written: First of all, I migrated from GitHub to Codeberg. The new link is now: https://codeberg.org/benoto/calcx I will also maintain the GitHub page, at least to some extend, but main development will stay on the Codeberg repository. The reason for this migration is Codeberg's stance against AI and their smaller, self-hosted-like nature.

Now about the actual program: I have actually completed quite a lot of the basic stuff for the calculator so that I feel confident to publish a first compiled version available for download on the GitHub / Codeberg page. Since I have done everything in pure-rust, compilation should not present a huge obstacle since rust is quite good with cross-compilation. However, I do not want to compile on the same machine I do the programming on; all of the packages and libraries required for cross-compilation take up way too much space on the hard drive and the CPU in this rather old ThinkPad would also not like such a rough handling. Therefore, I decided to do it the right way; at least partly. After research, the only real option I found to be sufficient was a self-hosted woodpecker instance. I am aware that GitHub supposedly is quite good at offering their own CI/CD workflow for free; and Codeberg, too, does have an integrated tool for it, but I decided against those to do it myself and hopefully learn some stuff along the way...

So I set off to install and set up woodpecker¹ on my own hardware. Since I am running Proxmox on my server, creating a new service was as easy as clicking “create new LXC container”. I chose Debian (13) as my distribution of choice, since Woodpecker offers a native package for it. Installation then was as easy as copy pasting into the terminal (yes, I know, insecure... It was totally an accident):

RELEASE_VERSION=$(curl -s https://api.github.com/repos/woodpecker-ci/woodpecker/releases/latest | grep -Po '"tag_name":\s"v\K[^"]+')

# Debian/Ubuntu (x86_64)
curl -fLOOO "https://github.com/woodpecker-ci/woodpecker/releases/download/v${RELEASE_VERSION}/woodpecker-{server,agent,cli}_${RELEASE_VERSION}_amd64.deb"
sudo apt --fix-broken install ./woodpecker-{server,agent,cli}_${RELEASE_VERSION}_amd64.deb

After the installation, I renamed the example configuration files and tried to start; which after some fiddling worked. However, what became a problem afterwards was the OAuth2.0 Authorization, which I could not get to work. But it turns out, I was just too ignorant to simply read the documentation. OAuth redirect URL requires this format: https://[url-to-woodpecker-host]/authorize and guess what I have been missing? The /authorize of course... yay...

After having changed that, woodpecker correctly connected to my Codeberg repository.

However, after some trial and error, I was unable to get woodpecker to work. Additionally, before setting up a more complex system for creating binaries, I first wanted to make sure they are cross-compile-able in the first place which first led me to cargo build targets. But before putting in more effort into getting those to compile correctly with my used packages, I first wanted to try another tool supposedly being very good for cross-compilation: cross

Their own documentation says it's basically magic, so let's see if it'll be able to produce working binaries for other systems such as windows and mac...

Supposedly, the only command I need to build the project is: cross build --target aarch64-unknown-linux-gnu (Obviously while altering the target for each compilation) So lets see if this works: ... It seems cross is not more magical than cargo build --target aarch64-unknown-linux-gnu and additionally, it cannot solve the problem of a dependency apparently not being cross-compile-able. So let's find out how to fix that... The dependency in question here is gmp-mpfr-sys. Since it's not directly a dependency in my Cargo.toml, I figured it must have come from rug, which in their documentation says it's simply a wrapper for GMP. Okay, so that's where it comes from. The error also directly suggests a “solution” but says it might not work. I need to enable the cross-compile feature for my gmp-mpfr-sys dependency. Since it's only an indirect one, my only options is to make it a direct one and now pay a lot of attention to the version numbers...

Since I build with cross, after removing the panic by gmp-mpfr-sys, it should compile still to a working binary for the other systems. So let's try that out! Well, it did not. When compiling for aarch, the build fails with errors: no such instruction xxxx. I guess that's bad... So what I tried next is to instead of forcing cross compilation, using the use-system-libs feature flag. I do expect, however, that this might compile, but not run on most systems. Potentially, if I want genuine cross-compatibility, I might need to

  1. Either customize rug so that it does compile for my use case (unlikely)
  2. Build my own rug-alternative that is able to cross-compile (might be fun, but also unlikely)

All in all, if I cannot get cross-compilation working with low-to-medium effort, I will simply give up. I, personally, only use Linux and probably most people that would use such a project do too. But I will not give up just yet!

So what I did now is that I had another error when trying to compile for windows. To try to solve this, I switched to the newest version of cross by compiling the git version my self. At first, it seemed to have worked with the force-cross feature flag for gmp-mpfr-sys. The compilation part itself worked fine and there were no errors. But what of course has not been tested yet is if the binaries actually execute on a windows machine. First and foremost, I was simply curious how large the binary might be, answer: roughly 3.2 MB. That's pretty good. But now, let's get this binary running on an actual windows machine! I won't bore you with the details of how I got the .exe from my virtual machine to a real windows host (let's call it a lot of hops through copy and paste and scp and USB thumb drives...) but eventually, I managed... Even if it took way longer than I'd like to admit. There was even a VPS involved... xD

Eventually, I got it on a windows host, double clicked and waited. And there it was! The terminal, just like on Linux! And mostly (of course mostly...) it worked fine. The mostly comes from a slight annoyance that the cursor was not at the spot where new characters appeared. The cursor in general appeared to be quite buggy... But! Everything else worked fine! So let's call this a 2/3 lose, 1/3 win.

Fortunately for me, after a bit of research I found out this might be related to the ANSI codes I use for colors which led to me simply disable colors when on Windows.

So that's Windows taken care of: I have a fully working version of my program as a binary (.exe) for Windows. Now, I need to decide what else I want to support. To get a running Mac-build seems to be very difficult, at least from what I have read so far. Nonetheless, I will give it a try, but if it fails, I won't put in as much work as I did for Windows.

First and foremost, I'll try to compile for x86 Macs (so the old intel-based ones), since I have much greater chances of succeeding here; only after that worked will I even try to compile to apples new ARM hardware. Okay, I guess Apple users might be out of luck, neither x86 nor ARM compiles with my current compiler (cross). Sorry...

I also tried building: a 32-bit Linux version, failed; a 64-bit-freebsd version, failed; and a 64-bit-linux-musl version compiled fine.

All of this concludes to these two lists:

Working and compiled version: – Generic x86_64-bit-linux-gnu – Generic x86_64-bit-linux-musl – Windows 10/11 GNU

Build failed: – Apple x86 – Apple ARM – x86_64-bit-freebsd – Linux ARM – Windows ARM – 32-bit Linux

That's not great, but something I can work with. Being able to support the operating system still having the biggest market share is quite the achievement for me when originally having built the project entirely for Linux.

By now, I do have binaries for the aforementioned three operating systems. Nonetheless, I still haven't figured out a good working system to compile my newest version and get it quickly up to GitHub. I believe for now, I will be fine with a rather improvised non-permanent solution which will be manually compiling and just using an improvised HTTP server to get all builds off of the build-VM to upload as the final release binary. That'll be way better than having to scp my binaries around for each and every release.

Soon, I'll be able to publish my very first release of this long-term project; first, I need to make some demos, however. Who would want to install a software before having seen what it does and can do? Not me, for sure.

So, I guess this wraps it up now.

CUL8TR!

1: (which is what Codeberg is also using under the hood)

Update: Wow, it gets better! I was able to compile the 32-bit Linux version with cargo instead of cross!