AltaeraAI

This repository is deprecated and was shifted into an organisational one: https://github.com/ThinkThroughLabs/AltaeraAI

README parts:

• What is it?
• What is it about?
• Current Models List
• Installation
• Launching & Updating
• Access Inference on external devices
• Technical Support
• License
• TODO

What is it?

AltaeraAI is a Termux wrapper that packages KoboldCpp and Kobold Lite UI for native usage on Android devices.

What is it about?

AltaeraAI is a Free and Open Source solution for running GGML/GGUF models with the power of your smartphone. It wraps around Termux instructions for installing Artix Linux with all the necessary dependencies in the “PRoot Distro” environment, and then installs KoboldCpp as both the back-end and the front-end UI (KoboldLite). Lastly, it adds lines of arguments in your “bash.bashrc” file, so that you can easily reach out for MENU by simply typing “ae“.

• The main premise of it is to automate AI Inference using a smartphone, with the help of cherry-picked, Open-Source tools and software. This allows you for instant, localized access to Artificial Intelligence - whether that be companionship, assistance or just text generation. At the same time, you maintain 100% privacy over your stories - you own the software due to the nature of locality, in the form of a self contained distributable.

• The instalation process will ask whether you want to install the pre-packaged KoboldCpp, or if you want to have it built directly on your device. The same applies each time you want you update your AltaeraAI.

• The default implemented model is RWKV-4; other options include: KobbleTinyV2-1.1B, KobbleTinyV2-1.1B (imatrix), TinyDolphin (imatrix/laser), TinyLlama, TinyVicuna, LLaMa-3, LLaMA-2, Mistral, Vicuna-v1.5, Mamba, Phi-3, Phi-SoSerious-Mini-V1 (GGUF), Phi-SoSerious-Mini-V1-imatrix (GGUF), Phi-2,Yi-1.5-6B-Chat (GGUF) and Gemma-2B/7B (GGUF) at this very moment. The installation process will allow you to either choose AI model via an auto-detection mechanism (currently reserved for 'RWKV-4-World-Claude-for-Mobile'), or you can select the model size manually. The former runs a simple algorithm to detect amount of RAM installed on device, then downloads an AI model accordingly.

AltaeraAI works as a platform in order to provide easy instructions and support for AI deployment on Android devices with the use of Termux. You can read more at: altaera.ai

Current Models List:

• RWKV-4 (GGML) - [No-Blas]
• KobbleTinyV2-1.1B (GGUF) - [No-Blas] / [OpenBLAS]
• KobbleTinyV2-1.1B (imatrix) (GGUF) - [No-Blas] / [OpenBLAS]
• TinyDolphin (imatrix/laser) (GGUF) - [No-Blas] / [OpenBLAS]
• TinyLlama (GGUF) - [No-Blas] / [OpenBLAS]
• TinyVicuna (GGUF) - [No-Blas] / [OpenBLAS]
• LLaMa-3 (GGUF) - [No-Blas] / [OpenBLAS]
• LLaMA-2 (GGUF) - [No-Blas] / [OpenBLAS]
• Mistral (GGUF) - [No-Blas] / [OpenBLAS]
• Vicuna-v1.5 (GGUF) - [No-Blas] / [OpenBLAS]
• Mamba (GGUF) - [No-Blas] / [OpenBLAS]
• Phi-3 (GGUF) - [No-Blas] / [OpenBLAS]
• Phi-SoSerious-Mini-V1 (GGUF) - [No-Blas] / [OpenBLAS]
• Phi-SoSerious-Mini-V1-imatrix (GGUF) - [No-Blas] / [OpenBLAS]
• Phi-2 (GGUF) - [No-Blas] / [OpenBLAS]
• Yi-1.5-6B-Chat (GGUF) - [No-Blas] / [OpenBLAS]
• Gemma-2B/7B (GGUF) - [No-Blas] / [OpenBLAS]

Installation

1. Download and install Termux

2. Open Termux and paste in:

   curl -o install https://raw.githubusercontent.com/ThinkThroughLabs/AltaeraAI/main/scripts/install && chmod +x install && ./install
   

• Then press ENTER and wait for the installation to finish. Make sure that you are connected to Wi-Fi because the process will download 3GB+ of data, depending on device. Installation will take 5min+ of time, depending on the device and Internet speed.

Installation.mp4

Launching & Updating

• Open Termux, type in 'ae' – you will be welcomed with the MENU screen.

The “Start AltaeraAI” button will ask you to choose from currently installed models, and then forward you to a browser with deployed UI.

Access Inference on external devices

You can access your AI Inference on external devices like PCs, laptops, etc., with the use of Secure Tunnelling [ngrok] - AltaeraAI has this function implemented in its code. You can learn more at: ngrok Secure Tunnels - AltaeraAI

Technical Support

• Please, visit: Technical Support - AltaeraAI
• or just open an issue on this github

License [derived from KoboldCpp]

• The original GGML library and llama.cpp by ggerganov are licensed under the MIT License
• However, Kobold Lite is licensed under the AGPL v3.0 License
• The other files are also under the AGPL v3.0 License unless otherwise stated

TODO

• A lot of things ;)