EAGLE: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders

🔥🔥🔥 A curated list of papers on LLMs-based multimodal generation (image, video, 3D and audio).

[EMNLP 2024 Industry Track] This is the official PyTorch implementation of "LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit".

[NeurIPS 2024] This repo contains evaluation code for the paper "Are We on the Right Way for Evaluating Large Vision-Language Models"

up-to-date curated list of state-of-the-art Large vision language models hallucinations research work, papers & resources

Large Visual Language Model(LVLM), Large Language Model(LLM), Multimodal Large Language Model(MLLM), Alignment, Agent, AI System, Survey

CLIP-MoE: Mixture of Experts for CLIP

A benchmark dataset and simple code examples for measuring the perception and reasoning of multi-sensor Vision Language models.

LEMMA: An effective and explainable way to detect multimodal misinformation with LVLM and external knowledge augmentation, incorporating the intuition and reasoning capbility inside LVLM.

HiRED strategically drops visual tokens in the image encoding stage to improve inference efficiency for High-Resolution Vision-Language Models (e.g., LLaVA-Next) under a fixed token budget.

Code for USENIX Security 2024 paper: Moderating Illicit Online Image Promotion for Unsafe User Generated Content Games Using Large Vision-Language Models.

Novel approach that leverages LVLMs to efficiently generate high-quality synthetic VQA-NLE datasets.

A powerful Streamlit application that allows users to analyze and interact with YouTube video content through natural language questions.