Revert "Generic Multimodal Support" (huggingface#1146)

Revert "Generic Multimodal Support (huggingface#1021)" This reverts commit 57f8934.
ice91 · May 15, 2024 · 545882c · 545882c
1 parent c6b428a
commit 545882c
Show file tree

Hide file tree

Showing 25 changed files with 342 additions and 831 deletions.
diff --git a/README.md b/README.md
@@ -230,7 +230,7 @@ The following is the default `chatPromptTemplate`, although newlines and indenti
 
 #### Multi modal model
 
-We currently support [IDEFICS](https://huggingface.co/blog/idefics) (hosted on TGI), OpenAI and Claude 3 as multimodal models. You can enable it by setting `multimodal: true` in your `MODELS` configuration. For IDEFICS, you must have a [PRO HF Api token](https://huggingface.co/settings/tokens). For OpenAI, see the [OpenAI section](#OpenAI). For Anthropic, see the [Anthropic section](#Anthropic).
+We currently only support IDEFICS as a multimodal model, hosted on TGI. You can enable it by using the following config (if you have a PRO HF Api token):
 
 ```env
     {
@@ -465,34 +465,14 @@ MODELS=`[
 
 #### Anthropic
 
-We also support Anthropic models (including multimodal ones via `multmodal: true`) through the official SDK. You may provide your API key via the `ANTHROPIC_API_KEY` env variable, or alternatively, through the `endpoints.apiKey` as per the following example.
+We also support Anthropic models through the official SDK. You may provide your API key via the `ANTHROPIC_API_KEY` env variable, or alternatively, through the `endpoints.apiKey` as per the following example.
 
 ```
 MODELS=`[
-  {
-      "name": "claude-3-haiku-20240307",
-      "displayName": "Claude 3 Haiku",
-      "description": "Fastest and most compact model for near-instant responsiveness",
-      "multimodal": true,
-      "parameters": {
-        "max_new_tokens": 4096,
-      },
-      "endpoints": [
-        {
-          "type": "anthropic",
-          // optionals
-          "apiKey": "sk-ant-...",
-          "baseURL": "https://api.anthropic.com",
-          "defaultHeaders": {},
-          "defaultQuery": {}
-        }
-      ]
-  },
   {
       "name": "claude-3-sonnet-20240229",
       "displayName": "Claude 3 Sonnet",
       "description": "Ideal balance of intelligence and speed",
-      "multimodal": true,
       "parameters": {
         "max_new_tokens": 4096,
       },
@@ -511,7 +491,6 @@ MODELS=`[
       "name": "claude-3-opus-20240229",
       "displayName": "Claude 3 Opus",
       "description": "Most powerful model for highly complex tasks",
-      "multimodal": true,
       "parameters": {
          "max_new_tokens": 4096
       },
@@ -537,7 +516,6 @@ MODELS=`[
       "name": "claude-3-sonnet@20240229",
       "displayName": "Claude 3 Sonnet",
       "description": "Ideal balance of intelligence and speed",
-      "multimodal": true,
       "parameters": {
         "max_new_tokens": 4096,
       },
@@ -556,7 +534,6 @@ MODELS=`[
       "name": "claude-3-haiku@20240307",
       "displayName": "Claude 3 Haiku",
       "description": "Fastest, most compact model for near-instant responsiveness",
-      "multimodal": true,
       "parameters": {
          "max_new_tokens": 4096
       },