AI4OS LLM¶
As part of the tool repertoire that the AI4OS stack is offering to increase our user’s productivity, we are current offering an LLM Chatbot service (in beta!) that allows users to summarize information, get code recommendations, ask questions about the documentation, etc.
We care about user privacy, so it’s important to notice that your chat history will be erased whenever you delete it, and no data will be retained by the platform (privacy policy).
Login¶
The service is available at: https://llm.dev.ai4eosc.eu
To access the LLM service, you first need to register in the platform.

Once you login, you will arrive to a landing page where you will be able to select the model which you want to interact with.

The current available models are:
Nemo-2407
: a small-size model from the Mistral family (12B parameter) with a large context window (128K tokens), released on July 2024,Small-2409
: a medium-size model from the Mistral family (22B parameters) with a smaller context window (32K tokens), released on September 2024,Assistant
: our custom model designed to help you navigate our documentation,
Now, let’s explore some common usages of the tool. Keep in mind that the AI4OS LLM is built with OpenWebUI so you always find further information in their documentation.
Chat with the LLM¶
We can ask generic questions to the model.

Remember that if you answer relies on up-to-date information, you can always enable Web search
under the button.
Summarize a document¶
Under the button, you can select Upload files
.
This will allow you to query a document with questions.

Ask questions about the documentation¶
Important
This service is currently under development, so it might not be accessible to you.
In the upper left corner, you can select the AI4EOSC/Assistant
model to ask questions about the platform. The LLM with use our documentation as knowledge base to provide truthful answers to your questions.

Integrate it with your own services¶
Under → Settings
→ Account
you will be able to generate your own API keys to access the service. Learn more on how to use API keys to integrate the AI4OS LLM into your own services (endpoints are compatible with the OpenAI API spec).
