About Artanis LLM plugin
Mae govannen, nothlir!
Today we're pleased to anounce the release of Artanis LLM 0.0.1. This is the first plugin of Artanis. The name explains itself, LLM stands for Large Language Model. It is a plugin that provides a simple interface to use large language models in Artanis.
The design
In this plugin, we are not going to inference LLM with GNU Guile, instead, we use the RESTful API provided by LLM service vendors. Artanis LLM aims to be the gateway of all LLM vendors, so that you can use any possible LLM you like with GNU Artanis.
Why an Artanis plugin?
The client module of GNU Artanis is designed to be compliant with the async non-blocking server core. So that your controller will not be blocked by the LLM inference process.
Why not use the LLM directly?
Inference a LLM model is a heavy task, it may take a long time to get the result. This plugin is designed to provide an easy way to use LLM in your web application, and it will not block your server.
I'm happy to see anyone to write LLM inferencing module to use LLM directly in GNU Guile, however, this is out of the topic.
How to use
You may want to take a look at the README of Artanis LLM.
Feedback are always welcome
Please feel free to report any issues to our GitLab repo.
And feel free to send mail to [email protected] for any questions.
Happy hacking!