The upcoming version of AI Runner features a large number of improvements. Take a look at the list below.

LLM Improvements

• Switched to Ministral 8b quantized to 4bits
• Improved LLM quantization, smaller model size
• RAG now indexes based on conversations
• Store indices to disc for reloading
• Ability to index on news articles
• Improved system prompts
• Improved agent implementation to match that of LlamaIndex
• Improved tool usage
• Adds various LLM tools
• Improved mood tracking
• Added conversation summaries used for better system prompts
• Improved prompting and personalities
• Now using an proper embedding model when indexing RAG
• Added weather data
• Added ability to get user latitude / longitude
• Added OS information to system prompt

Text-to-Speech improvements

• Added ability to change pitch (Speech T5)
• Added ability to change voice style (Speech T5)
• Removed unused Speech T5 properties from preferences
• Fixed ability to switch between espeak and Speech T5
• Improvements to text processing

Inpainting improvements

• Fixes inpainting for SDXL
• Fixes inpainting implementation in general as it was not working quite right in previous versions
• Adds missing inpaint files during installation

Experimental features

News Scraper

A tool was added which will scrape the news from various RSS Feeds. these articles are stored in the database and used for RAG results when chatting with the LLM.

Fine tuning

The ability to fine-tune the model on arbitrary data has been added. Fine tuned weights are stored as PEFT adapters and loaded along with the base model. This experimental feature is being used to see how easily we can finetune information about the user into a PEFT adapter so that the LLM “remembers” information about the user.

Stable Diffusion improvements

• Makes SDXL Turbo models work
• Added diffuser model quantization (less VRAM)
• Implemented tiny autoencoder (faster inference)
• No longer reload prompt embeds each time we generate an image (performance enhancement)

Misc features and improvements

• Forces the user to re-install if coming from a previous version
• Added more license agreements to installation step