Tools for merging pretrained Large Language Models and create Mixture of Experts (MoE) from open-source models.

An attempt at a glass pane through which to view the internals of power. 🪟

Mergekit Assistant is a cutting-edge toolkit designed for the seamless merging of pre-trained language models. It supports an array of models, offers various merging methods, and optimizes for low-resource environments with both CPU and GPU compatibility.

Copy of https://tieknots.how with a few tweaks