support Shanghai_AI_Laboratory/internlm-xcomposer2d5-ol-7b:audio

[Jintao-Huang]

InternLM/InternLM-XComposer#450

[Jintao-Huang]

model: https://modelscope.cn/models/Shanghai_AI_Laboratory/internlm-xcomposer2d5-ol-7b

inference code:

# pip install git+https://github.com/modelscope/ms-swift.git
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '0'

from swift.llm import PtEngine, InferRequest, RequestConfig
import torch
engine = PtEngine('Shanghai_AI_Laboratory/internlm-xcomposer2d5-ol-7b:audio', torch_dtype=torch.float16)
request_config = RequestConfig(max_tokens=512)
resp_list = engine.infer([InferRequest(messages=[{'role': 'user', 'content': '<audio>Detect the language and recognize the speech.'}],
                                       audios=['https://modelscope-open.oss-cn-hangzhou.aliyuncs.com/images/chinese.mp3'])])
print(f'response: {resp_list[0].choices[0].message.content}')