Demystify GNU Artanis service deployment in product environment

Byggsteg is a CI/CD project written in Guile Scheme, which I recently ported to GNU Artanis 1.0.0. Choosing an existing libre-licensed project for this system integration test was a practical and logical decision.

Since these two dependencies are in the same repo, we build them in a row.

git clone https://github.com/opencog/guile-dbi.git
cd guile-dbi
git checkout guile-dbi-2.1.8
cd guile-dbi
./autogen.sh
./configure --prefix=/usr
make -j5
sudo make install
ldconfig
cd ..

git checkout guile-dbd-mysql-2.1.8
cd guile-dbd-mysql
./autogen.sh
./configure --prefix=/usr
make -j5
sudo make install
sudo ldconfig

git clone https://github.com/spk121/guile-curl.git
cd guile-curl
git checkout v0.9
./bootstrap && ./configure --prefix=/usr
make -j5
sudo make install
sudo ln -s /usr/lib/guile/3.0/extensions/libguile-curl.* /usr/lib/
sudo ldconfig

Please note that you need to make the soft link under Debian/Ubuntu. Otherwise the lib may not be recognized.

We're not going to advocate the cloud provider, but we can tell you the machine configuration.

• Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
  • 2 CPU cores
• RAM 8GB
• Ubuntu 24.04 live server
  • Linux 6.8.0-1021
• GNU Guile 3.0.9
• GNU Artanis 1.0.0

The benchmark test is based on the multi server feature of GNU Artanis for best throughput performance.

In this benchmark, we set the multi server instances to 2 since the cloud machine has only 2 CPU cores. We also compare the throughput with the unique instance case.

server.multi = true

The multi.server can let you start multiple GNU Artanis instances to handle the requests concurrently. This feature is essential for high-performance web applications. However, one may have to manage each instance by manually, it's a bit tricky since it's designed for distributed situation.

NOTE: If you're looking for an easy setup for higher performance in just one instance, there's experimental **server.workers feature in the coming v1.1.0.**

First, let me define the purpose of this benchmark. According to our experiences, after enabled the Linux Kernel Vaccine, some of the performance will be affacted, for example I/O. So we have to make sure the performance of GNU Artanis will not be down too much under Linux Kernel Vaccine.

Do we need to tune the kernel prarameters?

I don't think so. Tuning kernel parameters is a complex task. Without careful research and adjustment, simple tweaks can potentially degrade performance. In this post, the benchmark is designed to compare the differences between the pre- and post-vaccine stages. The kernel parameters tuning is out of the topic, but could be further discussed in another post.

Specifically, I enabled the multi-server mode in GNU Artanis, which is a feature that allows multiple threads to handle coroutine workqueues concurrently. This feature is essential for high-performance web applications.

In this post, we put byggsteg into /var/www, this will affect the Nginx config file later.

sudo apt install wrk # for the benchmark

git clone https://codeberg.org/jjba23/byggsteg.git
cd byggsteg
make create # will ask for authenication, please input the password, feel free the check the Makefile for the details
make prepare-dirs

# run with tmux or screen
tmux new -t byggsteg
art work
# ctrl+b d to detach

# restore the session
# tmux a -t byggsteg

We're going to test the API performance with 100 long connections, 10 threads, and 20 seconds duration. The brief meaning of the API is to check the building instance log and return a json response. Before the test, we need to start an instance with byggsteg.

curl 'localhost:3000/api/v1/jobs/manage/submit' -X POST -H 'Content-Type: application/x-www-form-urlencoded' \
     --data-urlencode 'job-code=`((project . "free-alacarte")(branch-name . "trunk")(task . "stack-test")(clone-url . "https://codeberg.org/jjba23/free-alacarte"))'
# ==> response a json
# {"log-filename":"667265652d616c6163617274655f5f31323a34303a34345f5f32382d31322d323032342e62796767737465672e6c6f670a"}

According to the response, we can get the log file name, and we can use it to check the log. The API usage is trivial in this post, if you're interested, please visit the byggsteg project.

Now let's test the performance of the API.

wrk -c 100 -t 10 -d20s --latency http://localhost:3000/api/v1/logs/667265652d616c6163617274655f5f31323a34303a34345f5f32382d31322d323032342e62796767737465672e6c6f670a

# the response will be like this json
# {"success":"true","failure":"false","in-progress":"false","log-filename":"free-alacarte__12:40:44__28-12-2024.byggsteg.log","log-data":"0a7374617274696e67206e6577206a6f622e2e2e0a0a0a"}

The tested byggsteg code is not a fast implementation, there're frequent disk I/O operations, and we don't set any memory cache intendedly. The purpose of this test is to show the performance (included I/O, string parsing and JIT) difference before and after enabling the Linux Kernel Vaccine.

The performance of GNU Artanis was reduced by around 5%~7% after enabling the VED Linux Kernel Vaccine. This result is within the expected range, and the performance is still acceptable for a production environment. Considering the security enhancements provided by the Vaccine, the trade-off is reasonable.

There's also ARM64 benchmark you may want to take a look.