✨ feat!(llm): 重构并升级大语言模型服务为全新 AI 智能体框架 (#2146)

* ✨ feat!(llm): 重构并升级大语言模型服务为全新 AI 智能体框架

- 【重构】将原 services/llm 重构并迁移至全新的 services/ai 架构,提供向下兼容垫片
- 【新增】引入 Agent、Team、Workflow 三大智能体与工作流编排范式
- 【新增】引入基于 RAG 的长期向量记忆与中期槽位记忆系统
- 【新增】引入基于 Docker 的安全代码执行沙箱环境
- 【新增】支持 MCP 协议,允许动态管理和调用 MCP 服务
- 【新增】引入输入输出安全合规护栏与自愈反思机制
- 【优化】重构并优化多厂商 API 适配器 (Gemini, OpenAI, DeepSeek, GLM 等)
- 【优化】优化日志脱敏与 Token 预估机制
- 【移除】移除旧版 llm default 和 llm reset-key 命令,新增 llm mcp 管理命令

* 🔧 chore(deps): 更新项目依赖与配置

- 添加 mcp、jieba 和 aiodocker 依赖到配置文件及 requirements.txt
- 在 pyright 配置中设置 reportMissingImports 为 none
- 调整 .gitignore 中 resources 目录的忽略规则

* ♻️ refactor(tools): 重构工具终止机制并清理知识库日志输出

- 统一使用 `context.state["__end_run__"]` 替代 `EndRunResult` 控制任务结束
- 移除文件系统和向量知识库检索工具中 `ToolResult` 的 `.with_log` 调用
- 调整指令处理器(Directive)的返回值为 `tool_res.output`
- 修复部分类型检查警告并优化联合类型判断语法

* ♻️ refactor(tools): 重构工具副作用指令与控制流熔断机制

- 引入 `DirectivePayload` 及 `ToolResult` 的子类以结构化表达工具副作用
- 移除通过 `context.state` 传递魔术变量的隐式控制流设计
- 重构 `DirectiveManager` 处理器接口,直接在处理器中修改 `AgentState` 并构建 `AgentRunResult`
- 在 `StandardAgentExecutor` 中统一通过 `directive_manager` 调度工具返回的副作用指令
- 补全 `MessageBuilder` 中部分核心方法的文档注释

* 🐛 fix(sandbox): 修复 Docker 沙箱容器状态检测与会话清理逻辑

-【修复】修正 `is_alive` 中直接读取私有属性的问题,改用 `show()` 返回值
-【修复】解决 `execute_code` 中缓存的执行器与当前会话不一致的问题
-【优化】在清理工作区前增加容器存活检测,避免向已死容器发送请求
-【优化】创建容器时增加运行状态校验,若已停止则自动从缓存中移除并重建
-【优化】优化容器销毁和清理逻辑,静默处理容器不存在 (404) 的异常

* 📝 docs(core): 补充核心模块初始化方法的文档注释

* 🚨 auto fix by pre-commit hooks

---------

Co-authored-by: webjoin111 <455457521@qq.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Rumio
2026-07-03 08:53:56 +08:00
committed by GitHub
co-authored by webjoin111 pre-commit-ci[bot]
parent bdc1374848
commit 80fc5b86a7
223 changed files with 41885 additions and 10006 deletions
@@ -0,0 +1,38 @@
"""
LLM 适配器模块
提供不同LLM服务商的API适配器实现,统一接口调用方式。
"""
from .base import BaseAdapter, RequestData, ResponseData
from .deepseek import DeepSeekAdapter
from .doubao import DoubaoAdapter
from .factory import LLMAdapterFactory, get_adapter_for_api_type, register_adapter
from .gemini import GeminiAdapter
from .glm import GLMAdapter
from .jina import JinaAdapter
from .mimo import MiMoAdapter
from .minimax import MiniMaxAdapter
from .openai import OpenAIAdapter, OpenAICompatAdapter
from .openrouter import OpenRouterAdapter
LLMAdapterFactory.initialize()
__all__ = [
"BaseAdapter",
"DeepSeekAdapter",
"DoubaoAdapter",
"GLMAdapter",
"GeminiAdapter",
"JinaAdapter",
"LLMAdapterFactory",
"MiMoAdapter",
"MiniMaxAdapter",
"OpenAIAdapter",
"OpenAICompatAdapter",
"OpenRouterAdapter",
"RequestData",
"ResponseData",
"get_adapter_for_api_type",
"register_adapter",
]
+702
View File
@@ -0,0 +1,702 @@
"""
LLM 适配器基类和通用数据结构
"""
from __future__ import annotations
from abc import ABC, abstractmethod
import inspect
import json
from pathlib import Path
from typing import TYPE_CHECKING, Any
import uuid
import httpx
from pydantic import BaseModel, Field
from zhenxun.configs.path_config import TEMP_PATH
from zhenxun.services.ai.core.engine.token_counter import parse_usage_info
from zhenxun.services.ai.core.exceptions import (
AuthenticationException,
ConfigurationException,
ContentFilteredException,
ContextLengthExceededException,
InvalidRequestException,
LLMException,
LocationNotSupportedException,
QuotaExceededException,
RateLimitException,
ResponseParseException,
UpstreamServerException,
)
from zhenxun.services.ai.core.messages import (
AudioResponse,
ChatRequest,
ChatResponse,
EmbeddingRequest,
EmbeddingResponse,
ImagePart,
ImageRequest,
ImageResponse,
LLMContentPart,
RerankRequest,
RerankResponse,
RerankResult,
SpeechRequest,
TextPart,
ThoughtPart,
)
from zhenxun.services.ai.core.models import ModelIdentity
from zhenxun.services.log import logger
from zhenxun.utils.log_sanitizer import sanitize_for_logging
if TYPE_CHECKING:
from zhenxun.services.ai.llm.adapters.handlers.base import (
BaseAudioHandler,
BaseEmbeddingHandler,
BaseImageHandler,
BaseRerankHandler,
BaseTextHandler,
)
class RequestData(BaseModel):
"""标准化的请求载体,用于向上层 HTTP 客户端传递请求参数。"""
method: str = "POST"
url: str
headers: dict[str, str]
body: dict[str, Any]
files: dict[str, Any] | list[tuple[str, Any]] | None = None
class ResponseData(BaseModel):
"""标准化的响应载体,统一承接文本、多模态与附加元数据。"""
content_parts: list[LLMContentPart] = Field(default_factory=list)
usage_info: dict[str, Any] | None = None
raw_response: dict[str, Any] | None = None
grounding_metadata: Any | None = None
cache_info: Any | None = None
@property
def text(self) -> str:
"""提取并拼接所有 `TextPart` 文本内容。"""
return "".join(
p.text for p in self.content_parts if isinstance(p, TextPart)
).strip()
@text.setter
def text(self, value: str):
"""设置首个 `TextPart`,不存在则追加新的 `TextPart`。"""
for p in self.content_parts:
if isinstance(p, TextPart):
p.text = value
return
self.content_parts.append(TextPart(text=value))
@property
def thought_text(self) -> str | None:
"""提取并拼接所有思维片段文本,未命中则返回 `None`。"""
thoughts = [
p.thought_text for p in self.content_parts if isinstance(p, ThoughtPart)
]
return "\n".join(thoughts).strip() if thoughts else None
@property
def thought_signature(self) -> str | None:
"""从末尾向前查找思维签名,用于后续连续推理场景。"""
for p in reversed(self.content_parts):
if (
hasattr(p, "metadata")
and p.metadata
and "thought_signature" in p.metadata
):
return p.metadata["thought_signature"]
return None
@property
def images(self) -> list[bytes | Path | str]:
"""收集图片内容,按 URL / 原始字节 / 本地路径顺序返回。"""
imgs = []
for p in self.content_parts:
if isinstance(p, ImagePart):
if p.url:
imgs.append(p.url)
elif p.raw:
imgs.append(p.raw)
elif p.path:
imgs.append(p.path)
return imgs
@images.setter
def images(self, value: list[bytes | Path | str]):
"""覆盖图片片段并按输入类型重建 `ImagePart` 列表。"""
self.content_parts = [
p for p in self.content_parts if not isinstance(p, ImagePart)
]
for img in value:
if isinstance(img, str) and img.startswith(("http://", "https://")):
self.content_parts.append(ImagePart(url=img))
elif isinstance(img, bytes):
self.content_parts.append(ImagePart(raw=img))
else:
self.content_parts.append(ImagePart(path=Path(img)))
code_execution_results: list[dict[str, Any]] | None = None
search_results: list[dict[str, Any]] | None = None
function_calls: list[dict[str, Any]] | None = None
safety_ratings: list[dict[str, Any]] | None = None
citations: list[dict[str, Any]] | None = None
def process_image_data(image_data: bytes) -> bytes | Path:
"""处理图片二进制数据:超过 2MB 时落盘并返回文件路径。"""
max_inline_size = 2 * 1024 * 1024
if len(image_data) > max_inline_size:
save_dir = TEMP_PATH / "llm"
save_dir.mkdir(parents=True, exist_ok=True)
file_name = f"{uuid.uuid4()}.png"
file_path = save_dir / file_name
file_path.write_bytes(image_data)
logger.info(
f"图片数据过大 ({len(image_data)} bytes),已保存到临时文件: {file_path}",
"LLMAdapter",
)
return file_path.resolve()
return image_data
class BaseAdapter(ABC):
"""
LLM API适配器基类 (门面模式 Facade)。
负责维护厂商级别的通用配置(如 URL 拼接、请求头构建、通用错误拦截),
而将具体的模态序列化与反序列化逻辑委派给各路 Handler。
"""
text_handler: "BaseTextHandler | None" = None
image_handler: "BaseImageHandler | None" = None
embedding_handler: "BaseEmbeddingHandler | None" = None
rerank_handler: "BaseRerankHandler | None" = None
audio_handler: "BaseAudioHandler | None" = None
@property
def log_sanitization_context(self) -> str:
"""用于日志清洗的上下文名称,默认 'default'"""
return "default"
@property
@abstractmethod
def api_type(self) -> str:
"""API类型标识"""
pass
@property
@abstractmethod
def supported_api_types(self) -> list[str]:
"""支持的API类型列表"""
pass
async def prepare_payload(
self, identity: ModelIdentity, api_key: str, request: Any
) -> RequestData:
"""泛型请求构建分发入口 (Polymorphic Dispatch)"""
dispatch = {
ChatRequest: self.prepare_advanced_request,
EmbeddingRequest: self.prepare_embedding_request,
RerankRequest: self.prepare_rerank_request,
ImageRequest: self.prepare_image_request,
SpeechRequest: self.prepare_speech_request,
}
handler = dispatch.get(type(request))
if not handler:
raise ValueError(
f"适配器 {self.api_type} 不支持的请求类型: {type(request)}"
)
res = handler(identity, api_key, request)
if inspect.isawaitable(res):
return await res
return res
async def parse_payload(
self, identity: ModelIdentity, request: Any, raw_response: httpx.Response
) -> Any:
"""泛型响应解析分发入口 (Polymorphic Dispatch)"""
if isinstance(request, SpeechRequest):
res = self.parse_speech_response(identity, raw_response)
if inspect.isawaitable(res):
return await res
return res
response_bytes = await raw_response.aread()
logger.debug(f"📦 响应体已完整读取 ({len(response_bytes)} bytes)")
try:
response_json = json.loads(response_bytes)
except json.JSONDecodeError:
raise ResponseParseException(
"API 返回了非 JSON 格式的内容,可能是 URL 路径错误或中转站配置异常。",
details={
"raw_response": response_bytes.decode("utf-8", errors="ignore")[
:500
]
},
)
sanitizer_req_context = self.log_sanitization_context
sanitizer_resp_context = sanitizer_req_context.replace("_request", "_response")
if sanitizer_resp_context == sanitizer_req_context:
sanitizer_resp_context = f"{sanitizer_req_context}_response"
sanitized_response = sanitize_for_logging(
response_json, context=sanitizer_resp_context
)
response_json_str = json.dumps(sanitized_response, ensure_ascii=False, indent=2)
logger.debug(f"📋 响应JSON: {response_json_str}")
dispatch = {
EmbeddingRequest: self._parse_embedding_payload,
RerankRequest: self._parse_rerank_payload,
ImageRequest: self._parse_image_payload,
ChatRequest: self._parse_chat_payload,
}
handler = dispatch.get(type(request))
if not handler:
raise ValueError(
f"适配器 {self.api_type} 不支持的请求类型解析: {type(request)}"
)
return handler(identity, response_json)
def _parse_embedding_payload(
self, identity: ModelIdentity, response_json: dict
) -> EmbeddingResponse:
self.validate_embedding_response(response_json)
embeddings = self.parse_embedding_response(response_json)
return EmbeddingResponse(
embeddings=embeddings,
usage=parse_usage_info(
response_json.get("usage") or response_json.get("usageMetadata")
),
model_name=identity.model_name,
)
def _parse_rerank_payload(
self, identity: ModelIdentity, response_json: dict
) -> RerankResponse:
return RerankResponse(results=self.parse_rerank_response(response_json))
def _parse_image_payload(
self, identity: ModelIdentity, response_json: dict
) -> ImageResponse:
response_data = self.parse_image_response(response_json)
return ImageResponse(
content_parts=response_data.content_parts, raw_response=response_json
)
def _parse_chat_payload(
self, identity: ModelIdentity, response_json: dict
) -> ChatResponse:
response_data = self.parse_response(identity, response_json, is_advanced=True)
return ChatResponse(
content_parts=response_data.content_parts,
usage_info=response_data.usage_info,
raw_response=response_data.raw_response,
grounding_metadata=response_data.grounding_metadata,
)
async def prepare_simple_request(
self,
identity: ModelIdentity,
api_key: str,
prompt: str,
history: list[dict[str, str]] | None = None,
) -> RequestData:
"""准备简单文本生成请求
默认实现:将简单请求转换为高级请求格式
子类可以重写此方法以提供特定的优化实现
"""
from zhenxun.services.ai.core.messages import (
AssistantMessage,
SystemMessage,
TextPart,
UserMessage,
)
messages: list[Any] = []
if history:
for msg in history:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "system":
messages.append(SystemMessage(content=[TextPart(text=content)]))
elif role == "assistant":
messages.append(AssistantMessage(content=[TextPart(text=content)]))
else:
messages.append(UserMessage(content=[TextPart(text=content)]))
messages.append(UserMessage(content=[TextPart(text=prompt)]))
config = identity.generation_config
return await self.prepare_advanced_request(
identity=identity,
api_key=api_key,
request=ChatRequest(messages=messages, config=config),
)
async def prepare_advanced_request(
self,
identity: ModelIdentity,
api_key: str,
request: ChatRequest,
) -> RequestData:
"""准备高级对话请求并委派给 `text_handler` 完成序列化。"""
if self.text_handler:
return await self.text_handler.prepare_text_request(
adapter=self,
identity=identity,
api_key=api_key,
request=request,
)
raise NotImplementedError(
f"API 类型 '{self.api_type}' 未装配 TextHandler,暂不支持文本对话能力。"
)
def parse_response(
self,
identity: ModelIdentity,
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
"""解析文本响应并委派给 `text_handler`。"""
if self.text_handler:
return self.text_handler.parse_text_response(
adapter=self,
identity=identity,
response_json=response_json,
is_advanced=is_advanced,
)
raise NotImplementedError(f"API 类型 '{self.api_type}' 未装配 TextHandler。")
async def prepare_embedding_request(
self,
identity: ModelIdentity,
api_key: str,
request: EmbeddingRequest,
) -> RequestData:
"""准备文本/多模态嵌入请求并委派给 `embedding_handler`。"""
if self.embedding_handler:
return await self.embedding_handler.prepare_embedding_request(
adapter=self,
identity=identity,
api_key=api_key,
request=request,
)
raise NotImplementedError(
f"API 类型 '{self.api_type}' 未装配 EmbeddingHandler,暂不支持向量嵌入。"
)
def parse_embedding_response(
self, response_json: dict[str, Any]
) -> list[list[float]]:
"""解析文本嵌入响应并委派给 `embedding_handler`。"""
if self.embedding_handler:
return self.embedding_handler.parse_embedding_response(
adapter=self, response_json=response_json
)
raise NotImplementedError(
f"API 类型 '{self.api_type}' 未装配 EmbeddingHandler。"
)
def prepare_rerank_request(
self,
identity: ModelIdentity,
api_key: str,
request: RerankRequest,
) -> RequestData:
"""准备重排请求并委派给 `rerank_handler`。"""
if self.rerank_handler:
return self.rerank_handler.prepare_rerank_request(
adapter=self,
identity=identity,
api_key=api_key,
request=request,
)
raise NotImplementedError(
f"API 类型 '{self.api_type}' 未装配 RerankHandler,暂不支持文本重排。"
)
def parse_rerank_response(
self, response_json: dict[str, Any]
) -> list[RerankResult]:
"""解析重排响应并委派给 `rerank_handler`。"""
if self.rerank_handler:
return self.rerank_handler.parse_rerank_response(
adapter=self, response_json=response_json
)
raise NotImplementedError(f"API 类型 '{self.api_type}' 未装配 RerankHandler。")
def prepare_image_request(
self,
identity: ModelIdentity,
api_key: str,
request: ImageRequest,
) -> RequestData:
"""准备图像请求并委派给 `image_handler`。"""
if self.image_handler:
return self.image_handler.prepare_image_request(
adapter=self,
identity=identity,
api_key=api_key,
request=request,
)
raise NotImplementedError(
f"API 类型 '{self.api_type}' 未装配 ImageHandler,暂不支持图像生成。"
)
def parse_image_response(self, response_json: dict[str, Any]) -> ResponseData:
"""解析图像响应并委派给 `image_handler`。"""
if self.image_handler:
return self.image_handler.parse_image_response(
adapter=self, response_json=response_json
)
raise NotImplementedError(f"API 类型 '{self.api_type}' 未装配 ImageHandler。")
def prepare_speech_request(
self,
identity: ModelIdentity,
api_key: str,
request: SpeechRequest,
) -> RequestData:
"""准备语音生成请求并委派给 `audio_handler`。"""
if self.audio_handler:
return self.audio_handler.prepare_speech_request(
adapter=self,
identity=identity,
api_key=api_key,
request=request,
)
raise NotImplementedError(
f"API 类型 '{self.api_type}' 未装配 AudioHandler,暂不支持语音生成。"
)
async def parse_speech_response(
self, identity: ModelIdentity, raw_response: httpx.Response
) -> AudioResponse:
"""解析语音响应并委派给 `audio_handler`。
注意传入的是 httpx.Response 的 raw 对象"""
if self.audio_handler:
return await self.audio_handler.parse_speech_response(
adapter=self, identity=identity, raw_response=raw_response
)
raise NotImplementedError(f"API 类型 '{self.api_type}' 未装配 AudioHandler。")
def validate_embedding_response(self, response_json: dict[str, Any]) -> None:
"""验证嵌入接口响应,检测 `error` 并转换为统一异常。"""
if response_json.get("error"):
error_info = response_json["error"]
msg = (
error_info.get("message", str(error_info))
if isinstance(error_info, dict)
else str(error_info)
)
raise UpstreamServerException(
f"嵌入API错误: {msg}",
details=response_json,
)
def get_api_url(self, identity: ModelIdentity, endpoint: str) -> str:
"""拼接最终请求 URL,兼容 `path_prefix` 与端点前后斜杠。"""
if not identity.api_base:
raise ConfigurationException(
f"模型 {identity.model_name} 的 api_base 未设置",
)
base_url = identity.api_base.rstrip("/")
prefix = identity.path_prefix.strip("/") if identity.path_prefix else ""
ep = endpoint.lstrip("/")
if prefix:
return f"{base_url}/{prefix}/{ep}"
return f"{base_url}/{ep}"
def get_base_headers(self, api_key: str) -> dict[str, str]:
"""构建默认请求头,包含 UA、JSON 类型与 Bearer 鉴权。"""
from zhenxun.utils.user_agent import get_user_agent
headers = get_user_agent()
headers.update(
{
"Content-Type": "application/json",
"Authorization": f"Bearer {api_key}",
}
)
return headers
def validate_response(self, response_json: dict[str, Any]) -> None:
"""统一校验文本/多模态响应并映射平台错误码。"""
if response_json.get("error"):
error_info = response_json["error"]
error_message = str(error_info)
if isinstance(error_info, dict):
error_message = error_info.get("message", error_message)
error_code = error_info.get("code", "unknown")
if (
error_code in ("invalid_api_key", "authentication_failed")
or "permission" in error_message.lower()
):
raise AuthenticationException(
f"鉴权失败: {error_message}", details={"api_error": error_info}
)
elif error_code in ("insufficient_quota", "quota_exceeded"):
raise QuotaExceededException(
f"配额耗尽: {error_message}", details={"api_error": error_info}
)
elif error_code == "rate_limit_exceeded":
raise RateLimitException(
f"请求限流: {error_message}", details={"api_error": error_info}
)
elif error_code in ("model_not_found", "invalid_model"):
raise ConfigurationException(
f"模型配置错误: {error_message}",
details={"api_error": error_info},
)
elif error_code in (
"context_length_exceeded",
"max_tokens_exceeded",
"1261",
):
raise ContextLengthExceededException(
f"上下文超限: {error_message}",
details={"api_error": error_info},
)
elif error_code in ("invalid_request_error", "invalid_parameter"):
raise InvalidRequestException(
f"请求参数错误: {error_message}",
details={"api_error": error_info},
)
raise UpstreamServerException(
f"API请求报错: {error_message}",
details={"api_error": error_info},
)
if "candidates" in response_json:
candidates = response_json.get("candidates", [])
if candidates:
candidate = candidates[0]
finish_reason = candidate.get("finishReason")
if finish_reason in ["SAFETY", "RECITATION"]:
raise ContentFilteredException(
f"内容被模型安全策略过滤: {finish_reason}",
details={
"finish_reason": finish_reason,
},
)
if not response_json:
raise UpstreamServerException(
"API返回空响应",
details={"response": response_json},
)
def handle_http_error(self, response: httpx.Response) -> LLMException | None:
"""
处理 HTTP 错误响应。
如果响应状态码表示成功 (200),返回 None;否则构造 LLMException 供外部捕获。
"""
if response.status_code == 200:
return None
error_text = response.content.decode("utf-8", errors="ignore")
error_status = ""
error_msg = error_text
try:
error_json = json.loads(error_text)
if isinstance(error_json, dict) and "error" in error_json:
error_info = error_json["error"]
if isinstance(error_info, dict):
error_msg = error_info.get("message", error_msg)
raw_status = error_info.get("status") or error_info.get("code")
error_status = str(raw_status) if raw_status is not None else ""
elif error_info is not None:
error_msg = str(error_info)
error_status = error_msg
except Exception:
pass
status_upper = error_status.upper() if error_status else ""
text_upper = error_text.upper()
if response.status_code == 400:
if (
"FAILED_PRECONDITION" in status_upper
or "LOCATION IS NOT SUPPORTED" in text_upper
):
return LocationNotSupportedException(
"当前地区不支持该服务", details={"response": error_text}
)
elif "API_KEY_INVALID" in text_upper or "API KEY NOT VALID" in text_upper:
return AuthenticationException(
"API Key 无效", details={"response": error_text}
)
elif (
status_upper
in ["1261", "STRING_ABOVE_MAX_LENGTH", "CONTEXT_LENGTH_EXCEEDED"]
or "EXCEEDS MAX LENGTH" in text_upper
or "STRING TOO LONG" in text_upper
):
return ContextLengthExceededException(
"上下文超长", details={"response": error_text}
)
else:
return InvalidRequestException(
f"参数错误: {error_msg}", details={"response": error_text}
)
elif response.status_code in [401, 403]:
if "country" in error_msg.lower() or "unsupported" in error_msg.lower():
return LocationNotSupportedException(
"地区受限", details={"response": error_text}
)
else:
return AuthenticationException(
"鉴权失败/权限不足", details={"response": error_text}
)
elif response.status_code == 404:
return ConfigurationException(
"端点或模型未找到", details={"response": error_text}
)
elif response.status_code == 429:
if (
"RESOURCE_EXHAUSTED" in status_upper
or "INSUFFICIENT_QUOTA" in status_upper
or ("quota" in error_msg.lower() if error_msg else False)
):
return QuotaExceededException(
"API 配额耗尽", details={"response": error_text}
)
else:
return RateLimitException(
"请求频繁被限流", details={"response": error_text}
)
elif response.status_code in [402, 413]:
return QuotaExceededException(
"资源耗尽/文件过大", details={"response": error_text}
)
elif response.status_code >= 500:
return UpstreamServerException(
f"HTTP请求失败: {response.status_code} ({error_status or 'Unknown'})",
details={
"status_code": response.status_code,
"response": error_text,
},
)
return UpstreamServerException(
f"未知网络错误 {response.status_code}: {error_msg}"
)
@@ -0,0 +1,140 @@
from typing import Any
from zhenxun.services.ai.core.models import (
ModelCapabilities,
ModelDetail,
ModelIdentity,
)
from zhenxun.services.ai.core.options import GenerationConfig
from zhenxun.services.ai.llm.adapters.handlers.openai_handlers import (
OpenAIConfigMapper,
OpenAITextHandler,
OpenAIToolSerializer,
)
from zhenxun.services.ai.llm.adapters.openai import OpenAICompatAdapter
class DeepSeekToolSerializer(OpenAIToolSerializer):
"""
专门针对 DeepSeek 的工具序列化器。
负责抹平 Pydantic Schema 与 DeepSeek Strict Mode 之间的差异。
"""
def __init__(self, api_type: str = "deepseek"):
"""初始化 DeepSeek 工具序列化器。"""
super().__init__(api_type=api_type)
def sanitize_schema(self, schema: dict[str, Any]) -> dict[str, Any]:
from zhenxun.services.ai.llm.engine.schema_transformer import (
DeepSeekFallbackTransformer,
OpenAIUnionFlattenTransformer,
RefComplianceTransformer,
RemoveUnsupportedKeysTransformer,
RootRefInlineTransformer,
SchemaPipeline,
StrictObjectTransformer,
TypeEnforcerTransformer,
)
unsupported_keys = [
"default",
"minLength",
"maxLength",
"pattern",
"format",
"minimum",
"maximum",
"multipleOf",
"patternProperties",
"propertyNames",
"minItems",
"maxItems",
"uniqueItems",
"$schema",
"title",
]
pipeline = SchemaPipeline(
[
RootRefInlineTransformer(),
OpenAIUnionFlattenTransformer(),
TypeEnforcerTransformer(),
RemoveUnsupportedKeysTransformer(unsupported_keys),
StrictObjectTransformer(),
DeepSeekFallbackTransformer(),
RefComplianceTransformer(),
]
)
return pipeline.run(schema)
class DeepSeekConfigMapper(OpenAIConfigMapper):
"""DeepSeek 的专属配置映射器"""
def map_config(
self,
config: GenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
"""映射生成参数并处理 DeepSeek 专有 `thinking` 与响应格式差异。"""
params = super().map_config(config, model_detail, capabilities)
if "response_format" in params:
rf = params["response_format"]
if isinstance(rf, dict) and rf.get("type") == "json_schema":
params["response_format"] = {"type": "json_object"}
if config.common.reasoning_effort:
effort = str(config.common.reasoning_effort).lower()
if effort == "none":
params["thinking"] = {"type": "disabled"}
else:
params["thinking"] = {"type": "enabled"}
elif (
hasattr(config, "deepseek_options")
and config.deepseek_options.thinking is not None
):
if config.deepseek_options.thinking is True:
params["thinking"] = {"type": "enabled"}
elif config.deepseek_options.thinking is False:
params["thinking"] = {"type": "disabled"}
return params
class DeepSeekTextHandler(OpenAITextHandler):
"""DeepSeek 专有文本处理器,替换了特定序列化组件"""
def __init__(self, api_type: str = "deepseek"):
"""替换 OpenAI 默认组件为 DeepSeek 专用实现。"""
super().__init__(api_type=api_type)
self.serializer = DeepSeekToolSerializer(api_type=api_type)
self.mapper = DeepSeekConfigMapper(api_type=api_type)
class DeepSeekAdapter(OpenAICompatAdapter):
"""DeepSeek 官方 API 适配器"""
def __init__(self):
"""初始化 DeepSeek 适配器并挂载文本处理器。"""
super().__init__()
self.text_handler = DeepSeekTextHandler(api_type=self.api_type)
@property
def log_sanitization_context(self) -> str:
"""返回 DeepSeek 请求日志清洗上下文。"""
return "openai_request"
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "deepseek"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["deepseek"]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
"""返回对话端点,优先使用模型级自定义端点。"""
return "/v1/chat/completions"
@@ -0,0 +1,31 @@
from __future__ import annotations
from zhenxun.services.ai.core.models import ModelIdentity
from .handlers.openai_handlers import (
OpenAITextHandler,
)
from .openai import OpenAICompatAdapter
class DoubaoAdapter(OpenAICompatAdapter):
"""火山方舟 (Doubao) API 适配器"""
def __init__(self):
"""初始化 Doubao 适配器并复用 OpenAI 兼容处理器。"""
super().__init__()
self.text_handler = OpenAITextHandler(api_type=self.api_type)
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "doubao"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["doubao"]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
"""返回对话端点,优先使用模型级自定义端点。"""
return "/v3/chat/completions"
+167
View File
@@ -0,0 +1,167 @@
"""
LLM 适配器工厂类
"""
from __future__ import annotations
import fnmatch
from typing import Any, ClassVar
import httpx
from zhenxun.services.ai.core.exceptions import ConfigurationException
from zhenxun.services.ai.core.models import ModelIdentity
from .base import BaseAdapter, RequestData
class LLMAdapterFactory:
"""适配器注册与按 API 类型分发的统一入口。"""
_adapters: ClassVar[dict[str, BaseAdapter]] = {}
_api_type_mapping: ClassVar[dict[str, str]] = {}
@classmethod
def initialize(cls) -> None:
"""初始化默认适配器"""
if cls._adapters:
return
from .deepseek import DeepSeekAdapter
from .doubao import DoubaoAdapter
from .gemini import GeminiAdapter
from .glm import GLMAdapter
from .jina import JinaAdapter
from .mimo import MiMoAdapter
from .minimax import MiniMaxAdapter
from .openai import OpenAIAdapter
from .openrouter import OpenRouterAdapter
cls.register_adapter(OpenAIAdapter())
cls.register_adapter(OpenRouterAdapter())
cls.register_adapter(DeepSeekAdapter())
cls.register_adapter(JinaAdapter())
cls.register_adapter(GeminiAdapter())
cls.register_adapter(GLMAdapter())
cls.register_adapter(SmartAdapter())
cls.register_adapter(MiMoAdapter())
cls.register_adapter(MiniMaxAdapter())
cls.register_adapter(DoubaoAdapter())
@classmethod
def register_adapter(cls, adapter: BaseAdapter) -> None:
"""注册适配器"""
adapter_key = adapter.api_type
cls._adapters[adapter_key] = adapter
for api_type in adapter.supported_api_types:
cls._api_type_mapping[api_type] = adapter_key
@classmethod
def get_adapter(cls, api_type: str) -> BaseAdapter:
"""获取适配器"""
cls.initialize()
adapter_key = cls._api_type_mapping.get(api_type)
if not adapter_key:
raise ConfigurationException(
f"不支持的API类型: {api_type}",
details={
"api_type": api_type,
"supported_types": list(cls._api_type_mapping.keys()),
},
)
return cls._adapters[adapter_key]
@classmethod
def list_supported_types(cls) -> list[str]:
"""列出所有支持的API类型"""
cls.initialize()
return list(cls._api_type_mapping.keys())
@classmethod
def list_adapters(cls) -> dict[str, BaseAdapter]:
"""列出所有注册的适配器"""
cls.initialize()
return cls._adapters.copy()
def get_adapter_for_api_type(api_type: str) -> BaseAdapter:
"""按 API 类型获取适配器实例。"""
return LLMAdapterFactory.get_adapter(api_type)
def register_adapter(adapter: BaseAdapter) -> None:
"""向工厂注册新的适配器实例。"""
LLMAdapterFactory.register_adapter(adapter)
class SmartAdapter(BaseAdapter):
"""
智能路由适配器。
本身不处理序列化,而是根据规则委托给 OpenAIAdapter 或 GeminiAdapter。
"""
@property
def log_sanitization_context(self) -> str:
"""返回智能路由适配器的默认日志清洗上下文。"""
return "openai_request"
_ROUTING_RULES: ClassVar[list[tuple[str, str]]] = [
("*nano-banana*", "gemini"),
("*gemini*", "gemini"),
("*deepseek*", "deepseek"),
("*minimax*", "minimax"),
("*gpt*", "openai_responses"),
]
_DEFAULT_API_TYPE: ClassVar[str] = "openai"
def __init__(self):
"""初始化模型名到目标适配器的路由缓存。"""
self._adapter_cache: dict[str, BaseAdapter] = {}
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "smart"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["smart"]
def _get_delegate_adapter(self, identity: ModelIdentity) -> BaseAdapter:
"""
核心路由逻辑:决定使用哪个适配器 (带缓存)
"""
if identity.api_type and identity.api_type != "smart":
return get_adapter_for_api_type(identity.api_type)
model_name = identity.model_name
if model_name in self._adapter_cache:
return self._adapter_cache[model_name]
target_api_type = self._DEFAULT_API_TYPE
model_name_lower = model_name.lower()
for pattern, api_type in self._ROUTING_RULES:
if fnmatch.fnmatch(model_name_lower, pattern):
target_api_type = api_type
break
adapter = get_adapter_for_api_type(target_api_type)
self._adapter_cache[model_name] = adapter
return adapter
async def prepare_payload(
self, identity: ModelIdentity, api_key: str, request: Any
) -> RequestData:
adapter = self._get_delegate_adapter(identity)
return await adapter.prepare_payload(identity, api_key, request)
async def parse_payload(
self, identity: ModelIdentity, request: Any, raw_response: httpx.Response
) -> Any:
adapter = self._get_delegate_adapter(identity)
return await adapter.parse_payload(identity, request, raw_response)
@@ -0,0 +1,59 @@
"""
Gemini API 适配器
"""
from __future__ import annotations
from zhenxun.services.ai.core.models import ModelIdentity
from zhenxun.services.ai.core.options import GenerationConfig
from .base import BaseAdapter
from .handlers.gemini_handlers import (
GeminiAudioHandler,
GeminiEmbeddingHandler,
GeminiImageHandler,
GeminiTextHandler,
)
class GeminiAdapter(BaseAdapter):
"""Gemini API 适配器"""
def __init__(self):
"""初始化 Gemini 适配器并挂载各模态处理器。"""
super().__init__()
self.text_handler = GeminiTextHandler()
self.image_handler = GeminiImageHandler()
self.embedding_handler = GeminiEmbeddingHandler()
self.audio_handler = GeminiAudioHandler()
@property
def log_sanitization_context(self) -> str:
"""返回 Gemini 请求日志清洗上下文。"""
return "gemini_request"
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "gemini"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["gemini"]
def get_base_headers(self, api_key: str) -> dict[str, str]:
"""获取基础请求头"""
from zhenxun.utils.user_agent import get_user_agent
headers = get_user_agent()
headers.update({"Content-Type": "application/json"})
headers["x-goog-api-key"] = api_key
return headers
def _get_gemini_endpoint(
self, identity: ModelIdentity, config: GenerationConfig | None = None
) -> str:
"""返回Gemini generateContent 端点"""
return f"/v1beta/models/{identity.model_name}:generateContent"
+78
View File
@@ -0,0 +1,78 @@
from zhenxun.services.ai.core.messages import RerankRequest
from zhenxun.services.ai.core.models import ModelIdentity
from zhenxun.services.ai.llm.adapters.base import BaseAdapter, RequestData
from zhenxun.services.ai.llm.adapters.handlers.openai_handlers import (
OpenAIConfigMapper,
OpenAIEmbeddingHandler,
OpenAIRerankHandler,
OpenAITextHandler,
)
from zhenxun.services.ai.llm.adapters.openai import OpenAICompatAdapter
class GLMRerankHandler(OpenAIRerankHandler):
"""GLM 专有的重排处理器(重写了端点构建逻辑)"""
def prepare_rerank_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: RerankRequest,
) -> RequestData:
"""构建 GLM 重排请求,统一将文档归一化为字符串列表。"""
endpoint = "/api/paas/v4/rerank"
url = adapter.get_api_url(identity, endpoint)
headers = adapter.get_base_headers(api_key)
safe_documents = []
for doc in request.documents:
if isinstance(doc, dict):
safe_documents.append(doc.get("text", str(doc)))
else:
safe_documents.append(str(doc))
body = {
"model": identity.model_name,
"query": request.query,
"documents": safe_documents,
"top_n": request.top_n,
}
return RequestData(url=url, headers=headers, body=body)
class GLMTextHandler(OpenAITextHandler):
def __init__(self, api_type: str = "glm"):
super().__init__(api_type=api_type)
self.mapper = OpenAIConfigMapper(api_type=api_type)
class GLMAdapter(OpenAICompatAdapter):
"""GLM (智谱) 大模型专有适配器 (继承 OpenAI 兼容协议处理标准聊天)"""
def __init__(self):
"""初始化 GLM 适配器并装配专有图像/重排处理器。"""
super().__init__()
self.text_handler = GLMTextHandler(api_type=self.api_type)
self.embedding_handler = OpenAIEmbeddingHandler()
self.rerank_handler = GLMRerankHandler()
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "glm"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["glm"]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
"""返回对话端点,优先使用模型级自定义端点。"""
if None:
return None
return "/api/paas/v4/chat/completions"
def get_embedding_endpoint(self, identity: ModelIdentity) -> str:
"""返回嵌入端点。"""
return "/api/paas/v4/embeddings"
@@ -0,0 +1 @@
@@ -0,0 +1,223 @@
from __future__ import annotations
from abc import ABC, abstractmethod
import asyncio
from typing import Any
import httpx
from zhenxun.services.ai.core.messages import (
AudioResponse,
ChatRequest,
EmbeddingRequest,
ImageRequest,
LLMMessage,
RerankRequest,
RerankResult,
SpeechRequest,
)
from zhenxun.services.ai.core.models import (
ModelCapabilities,
ModelDetail,
ModelIdentity,
ToolDefinition,
)
from zhenxun.services.ai.core.options import (
GenerationConfig,
)
from zhenxun.services.ai.llm.adapters.base import BaseAdapter, RequestData, ResponseData
class ConfigMapper(ABC):
@abstractmethod
def map_config(
self,
config: GenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
"""将通用生成配置转换为特定 API 的参数字典"""
...
class MessageConverter(ABC):
@abstractmethod
async def convert_messages_async(
self, messages: list[LLMMessage]
) -> list[dict[str, Any]] | dict[str, Any]:
"""将通用消息列表异步转换为特定 API 的消息格式"""
...
class ToolSerializer(ABC):
def serialize_tools(
self, tools: list[ToolDefinition]
) -> list[dict[str, Any]] | None:
"""将通用工具定义转换为特定 API 的工具格式 (模板方法)"""
if not tools:
return None
serialized_tools = []
for tool in tools:
raw_schema = tool.parameters.copy() if tool.parameters else {}
sanitized_schema = self.sanitize_schema(raw_schema)
tool_payload = self.format_tool_payload(
tool_name=tool.name,
tool_description=tool.description or "",
sanitized_schema=sanitized_schema,
)
serialized_tools.append(tool_payload)
return serialized_tools
@abstractmethod
def format_tool_payload(
self, tool_name: str, tool_description: str, sanitized_schema: dict[str, Any]
) -> dict[str, Any]:
"""由子类实现:格式化单一工具的 Payload"""
...
@abstractmethod
def sanitize_schema(self, schema: dict[str, Any]) -> dict[str, Any]:
"""对 JSON Schema 进行特定 API 的清洗和格式化"""
...
@abstractmethod
def serialize_server_tools(
self, tools: list[Any], capabilities: ModelCapabilities
) -> list[dict[str, Any]]:
"""将系统内置的原生云端工具转译为底层 API 载荷(执行双重能力校验)"""
...
class ResponseParser(ABC):
@abstractmethod
def parse(self, response_json: dict[str, Any]) -> ResponseData:
"""将特定 API 的响应解析为通用响应数据"""
...
class BaseTextHandler(ABC):
"""
文本对话生成处理器接口。
负责将真寻的通用消息与工具列表转换为底层 API 的请求格式,并解析响应。
"""
async def _resolve_and_split_tools(
self, tools: list[Any] | None
) -> tuple[list[Any], list[Any], list[Any]]:
"""
统一的工具解析逻辑:分离客户端工具与服务端内置工具,并发获取 Schema。
返回: (tool_defs, client_executables, server_tools)
"""
client_executables, server_tools, tool_defs = [], [], []
if tools:
raw_tools = list(tools.values()) if isinstance(tools, dict) else tools
for tool in raw_tools:
if getattr(tool, "execution_side", "client") == "server":
server_tools.append(tool)
elif hasattr(tool, "get_definition"):
client_executables.append(tool)
if definition_tasks := [t.get_definition() for t in client_executables]:
tool_defs = [td for td in await asyncio.gather(*definition_tasks) if td]
return tool_defs, client_executables, server_tools
@abstractmethod
async def prepare_text_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: ChatRequest,
) -> RequestData: ...
@abstractmethod
def parse_text_response(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData: ...
class BaseEmbeddingHandler(ABC):
"""
文本嵌入向量处理器接口。
"""
@abstractmethod
async def prepare_embedding_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: EmbeddingRequest,
) -> RequestData: ...
@abstractmethod
def parse_embedding_response(
self, adapter: BaseAdapter, response_json: dict[str, Any]
) -> list[list[float]]: ...
class BaseImageHandler(ABC):
"""
图像生成/编辑处理器接口。
"""
@abstractmethod
def prepare_image_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: ImageRequest,
) -> RequestData: ...
@abstractmethod
def parse_image_response(
self, adapter: BaseAdapter, response_json: dict[str, Any]
) -> ResponseData: ...
class BaseRerankHandler(ABC):
"""
文本重排处理器接口。
"""
@abstractmethod
def prepare_rerank_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: RerankRequest,
) -> RequestData: ...
@abstractmethod
def parse_rerank_response(
self, adapter: BaseAdapter, response_json: dict[str, Any]
) -> list[RerankResult]: ...
class BaseAudioHandler(ABC):
"""
文本转语音 (TTS) 处理器接口。
"""
@abstractmethod
def prepare_speech_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: SpeechRequest,
) -> RequestData: ...
@abstractmethod
async def parse_speech_response(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
raw_response: httpx.Response,
) -> AudioResponse: ...
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,199 @@
import base64
import json
from typing import Any
import httpx
from zhenxun.services.ai.core.messages import (
AudioPart,
AudioResponse,
ImagePart,
LLMMessage,
SpeechRequest,
TextPart,
UsageInfo,
VideoPart,
)
from zhenxun.services.ai.core.models import (
ModelCapabilities,
ModelDetail,
ModelIdentity,
)
from zhenxun.services.ai.core.options import GenerationConfig, TTSConfig
from zhenxun.services.ai.llm.adapters.base import BaseAdapter, RequestData
from zhenxun.services.ai.llm.adapters.handlers.base import BaseAudioHandler
from zhenxun.services.ai.llm.adapters.handlers.openai_handlers import (
OpenAIConfigMapper,
OpenAIMessageConverter,
OpenAITextHandler,
OpenAIToolSerializer,
)
class MiMoToolSerializer(OpenAIToolSerializer):
"""MiMo 工具序列化器,负责拦截并构造独有的 web_search 工具"""
def serialize_server_tools(
self, tools: list[Any], capabilities: ModelCapabilities
) -> list[dict[str, Any]]:
res = []
for t in tools:
type_id = getattr(t, "type_id", "unknown")
if type_id not in capabilities.supported_native_tools:
continue
if type_id == "web_search":
res.append(
{
"type": "web_search",
"max_keyword": getattr(t, "max_keyword", 3),
"force_search": getattr(t, "force_search", True),
"limit": getattr(t, "limit", 1),
}
)
return res
class MiMoConfigMapper(OpenAIConfigMapper):
"""MiMo 配置映射器,处理深度思考参数差异"""
def map_config(
self,
config: GenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
params = super().map_config(config, model_detail, capabilities)
if config.common.reasoning_effort:
effort = str(config.common.reasoning_effort).lower()
if effort == "none":
params["thinking"] = {"type": "disabled"}
else:
params["thinking"] = {"type": "enabled"}
elif (
hasattr(config, "deepseek_options")
and config.deepseek_options.thinking is not None
):
if config.deepseek_options.thinking is True:
params["thinking"] = {"type": "enabled"}
elif config.deepseek_options.thinking is False:
params["thinking"] = {"type": "disabled"}
return params
class MiMoMessageConverter(OpenAIMessageConverter):
"""MiMo 消息转换器,拦截处理特有的音视频多模态结构"""
async def convert_messages_async(
self, messages: list[LLMMessage]
) -> list[dict[str, Any]]:
openai_messages = await super().convert_messages_async(messages)
for o_msg, o_orig in zip(openai_messages, messages):
if o_msg["role"] == "user":
content_parts = []
for part in o_orig.content:
if isinstance(part, TextPart):
content_parts.append({"type": "text", "text": part.text})
elif isinstance(part, ImagePart):
src = (
part.url
if part.url
else await part.get_data_uri(part.mime_type or "image/jpeg")
)
content_parts.append(
{"type": "image_url", "image_url": {"url": src}}
)
elif isinstance(part, VideoPart):
src = (
part.url
if part.url
else await part.get_data_uri(part.mime_type or "video/mp4")
)
content_parts.append(
{
"type": "video_url",
"video_url": {"url": src},
"fps": getattr(part, "fps", 2),
"media_resolution": getattr(
part, "media_resolution", "default"
),
}
)
elif isinstance(part, AudioPart):
src = (
part.url
if part.url
else await part.get_data_uri(part.mime_type or "audio/mp3")
)
content_parts.append(
{"type": "input_audio", "input_audio": {"data": src}}
)
o_msg["content"] = content_parts
return openai_messages
class MiMoTextHandler(OpenAITextHandler):
"""MiMo 文本对话处理器集成"""
def __init__(self, api_type: str = "mimo"):
super().__init__(api_type=api_type)
self.converter = MiMoMessageConverter(api_type=api_type)
self.serializer = MiMoToolSerializer(api_type=api_type)
self.mapper = MiMoConfigMapper(api_type=api_type)
class MiMoAudioHandler(BaseAudioHandler):
"""MiMo TTS 接口实现:挂载在 Chat Completions 端点上"""
def prepare_speech_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: SpeechRequest,
) -> RequestData:
input_text = request.input_text
config = request.config or TTSConfig()
config_voice = (
config.mimo_options.voice_id if hasattr(config, "mimo_options") else None
)
voice = (
config_voice
or request.voice
or identity.capabilities.default_voice_id
or "mimo_default"
)
url = adapter.get_api_url(identity, "/v1/chat/completions")
headers = adapter.get_base_headers(api_key)
body = {
"model": identity.model_name,
"messages": [{"role": "assistant", "content": input_text}],
"audio": {
"format": config.response_format
if config.response_format in ("wav", "pcm16")
else "wav",
"voice": voice,
},
}
return RequestData(url=url, headers=headers, body=body)
async def parse_speech_response(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
raw_response: httpx.Response,
) -> AudioResponse:
data = json.loads(await raw_response.aread())
adapter.validate_response(data)
audio_b64 = data["choices"][0]["message"]["audio"]["data"]
return AudioResponse(
audio_bytes=base64.b64decode(audio_b64),
audio_format="wav",
usage=UsageInfo(),
model_name=identity.model_name,
)
File diff suppressed because it is too large Load Diff
+129
View File
@@ -0,0 +1,129 @@
from zhenxun.services.ai.core.messages import EmbeddingRequest
from zhenxun.services.ai.core.models import ModelIdentity
from zhenxun.services.ai.core.options import LLMEmbeddingConfig
from zhenxun.services.ai.llm.adapters.base import BaseAdapter, RequestData
from zhenxun.services.ai.llm.adapters.handlers.openai_handlers import (
OpenAIEmbeddingHandler,
OpenAIRerankHandler,
)
from zhenxun.services.ai.llm.adapters.openai import OpenAICompatAdapter
class JinaEmbeddingHandler(OpenAIEmbeddingHandler):
"""Jina 专属文本/多模态嵌入处理器"""
async def prepare_embedding_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: EmbeddingRequest,
) -> RequestData:
batch = request.batch
config = request.config or LLMEmbeddingConfig()
endpoint = getattr(adapter, "get_embedding_endpoint")(identity)
url = adapter.get_api_url(identity, endpoint)
headers = adapter.get_base_headers(api_key)
is_omni = "omni" in identity.model_name.lower()
inputs_payload = []
if is_omni:
from zhenxun.services.ai.core.messages import (
AudioPart,
FilePart,
ImagePart,
TextPart,
VideoPart,
)
from zhenxun.services.log import logger
for payload in batch.payloads:
jina_content = []
for part in payload.parts:
if isinstance(part, TextPart):
jina_content.append({"text": part.text})
elif isinstance(part, ImagePart):
if part.url:
jina_content.append({"image": part.url})
else:
jina_content.append(
{"image": await part.get_data_uri("image/png")}
)
elif isinstance(part, AudioPart):
if part.url:
jina_content.append({"audio": part.url})
else:
jina_content.append(
{"audio": await part.get_data_uri("audio/mp3")}
)
elif isinstance(part, VideoPart):
if part.url:
jina_content.append({"video": part.url})
else:
jina_content.append(
{"video": await part.get_data_uri("video/mp4")}
)
elif isinstance(part, FilePart):
logger.warning(
f"Jina 暂不明确支持 Base64 内联 "
f"{type(part).__name__},已忽略。"
)
if not jina_content:
jina_content.append({"text": " "})
inputs_payload.append({"content": jina_content})
else:
inputs_payload = batch.to_text_only(
f"{identity.model_name} (API: {adapter.api_type})"
)
body = {
"model": identity.model_name,
"input": inputs_payload,
}
if config.output_dimensionality:
body["dimensions"] = config.output_dimensionality
if config.task_type:
task_mapping = {
"RETRIEVAL_QUERY": "retrieval.query",
"RETRIEVAL_DOCUMENT": "retrieval.passage",
"SEMANTIC_SIMILARITY": "text-matching",
"CLASSIFICATION": "classification",
"CLUSTERING": "clustering",
}
body["task"] = task_mapping.get(config.task_type, config.task_type)
if config.encoding_format and config.encoding_format != "float":
body["embedding_type"] = config.encoding_format
return RequestData(url=url, headers=headers, body=body)
class JinaAdapter(OpenAICompatAdapter):
"""Jina API 专有适配器 (仅支持 Embedding 和 Rerank)"""
def __init__(self):
super().__init__()
self.text_handler = None
self.embedding_handler = JinaEmbeddingHandler()
self.rerank_handler = OpenAIRerankHandler()
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "jina"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["jina"]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
raise NotImplementedError("Jina API 专精于检索,暂不支持常规对话生成。")
def get_embedding_endpoint(self, identity: ModelIdentity) -> str:
return "/v1/embeddings"
+30
View File
@@ -0,0 +1,30 @@
from zhenxun.services.ai.core.models import ModelIdentity
from zhenxun.services.ai.llm.adapters.handlers.mimo_handlers import (
MiMoAudioHandler,
MiMoTextHandler,
)
from zhenxun.services.ai.llm.adapters.handlers.openai_handlers import (
OpenAIImageHandler,
)
from zhenxun.services.ai.llm.adapters.openai import OpenAICompatAdapter
class MiMoAdapter(OpenAICompatAdapter):
"""小米 MiMo 大模型专有适配器"""
def __init__(self):
super().__init__()
self.text_handler = MiMoTextHandler(api_type=self.api_type)
self.image_handler = OpenAIImageHandler()
self.audio_handler = MiMoAudioHandler()
@property
def api_type(self) -> str:
return "mimo"
@property
def supported_api_types(self) -> list[str]:
return ["mimo"]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
return "/v1/chat/completions"
+243
View File
@@ -0,0 +1,243 @@
from __future__ import annotations
import json
from typing import Any
import httpx
from zhenxun.services.ai.core.exceptions import ResponseParseException
from zhenxun.services.ai.core.messages import (
AssistantMessage,
AudioResponse,
LLMMessage,
SpeechRequest,
ThoughtPart,
)
from zhenxun.services.ai.core.models import (
ModelCapabilities,
ModelDetail,
ModelIdentity,
)
from zhenxun.services.ai.core.options import GenerationConfig, TTSConfig
from zhenxun.services.ai.llm.adapters.base import BaseAdapter, RequestData
from zhenxun.services.ai.llm.adapters.handlers.base import BaseAudioHandler
from .handlers.openai_handlers import (
OpenAIConfigMapper,
OpenAIMessageConverter,
OpenAITextHandler,
)
from .openai import OpenAICompatAdapter
class MiniMaxAudioHandler(BaseAudioHandler):
"""MiniMax 专有文本转语音处理器"""
def prepare_speech_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: SpeechRequest,
) -> RequestData:
input_text = request.input_text
config = request.config or TTSConfig()
config_voice = config.minimax_options.voice_id
voice = (
config_voice
or request.voice
or identity.capabilities.default_voice_id
or "female-shaonv"
)
endpoint = "/v1/t2a_v2"
base_url = (
identity.api_base.rstrip("/")
if identity.api_base
else "https://api.minimaxi.com"
)
if base_url.endswith("/v1"):
base_url = base_url[:-3]
url = f"{base_url}{endpoint}"
headers = adapter.get_base_headers(api_key)
voice_setting: dict[str, Any] = {"voice_id": voice}
if config.speed != 1.0:
voice_setting["speed"] = config.speed
if config.minimax_options.vol is not None:
voice_setting["vol"] = config.minimax_options.vol
if config.minimax_options.pitch is not None:
voice_setting["pitch"] = config.minimax_options.pitch
if config.minimax_options.emotion is not None:
voice_setting["emotion"] = config.minimax_options.emotion
target_format = config.response_format
if target_format not in ("mp3", "pcm", "flac", "wav"):
target_format = "mp3"
audio_setting = {
"sample_rate": 32000,
"bitrate": 128000,
"format": target_format,
"channel": 1,
}
body: dict[str, Any] = {
"model": identity.model_name,
"text": input_text,
"stream": False,
"voice_setting": voice_setting,
"audio_setting": audio_setting,
}
if config.minimax_options.timbre_weights:
body["timbre_weights"] = config.minimax_options.timbre_weights
body["voice_setting"]["voice_id"] = ""
if config.minimax_options.pronunciation_dict:
body["pronunciation_dict"] = config.minimax_options.pronunciation_dict
return RequestData(url=url, headers=headers, body=body)
async def parse_speech_response(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
raw_response: httpx.Response,
) -> AudioResponse:
resp_bytes = await raw_response.aread()
data = json.loads(resp_bytes)
base_resp = data.get("base_resp", {})
if base_resp.get("status_code", 0) != 0:
raise ResponseParseException(
f"MiniMax 语音合成失败: {base_resp.get('status_msg')}",
details=data,
)
try:
audio_hex = data["data"]["audio"]
audio_bytes = bytes.fromhex(audio_hex)
except (KeyError, ValueError) as e:
raise ResponseParseException(
f"解析 MiniMax 语音 Hex 数据失败: {e}", details=data
)
extra_info = data.get("extra_info", {})
audio_format = extra_info.get("audio_format", "mp3")
usage_chars = extra_info.get("usage_characters", 0)
from zhenxun.services.ai.core.messages import AudioResponse, UsageInfo
usage = UsageInfo()
usage.prompt_tokens = usage_chars
return AudioResponse(
audio_bytes=audio_bytes,
audio_format=audio_format,
usage=usage,
model_name=identity.model_name,
raw_response=data,
)
class MiniMaxMessageConverter(OpenAIMessageConverter):
"""MiniMax 消息转换器,处理特有的 reasoning_details 格式回传"""
async def convert_messages_async(
self, messages: list[LLMMessage]
) -> list[dict[str, Any]]:
openai_messages = await super().convert_messages_async(messages)
assistant_msgs = [m for m in messages if isinstance(m, AssistantMessage)]
ast_idx = 0
for o_msg in openai_messages:
if o_msg.get("role") == "assistant":
if ast_idx < len(assistant_msgs):
orig_ast = assistant_msgs[ast_idx]
ast_idx += 1
if "reasoning_content" in o_msg:
del o_msg["reasoning_content"]
thought_parts = [
p for p in orig_ast.content if isinstance(p, ThoughtPart)
]
if thought_parts:
part = thought_parts[0]
raw_details = (
part.metadata.get("raw_reasoning_details")
if part.metadata
else None
)
if raw_details:
o_msg["reasoning_details"] = raw_details
else:
o_msg["reasoning_details"] = [
{"type": "reasoning.text", "text": part.thought_text}
]
return openai_messages
class MiniMaxConfigMapper(OpenAIConfigMapper):
"""MiniMax 专属配置映射器"""
def map_config(
self,
config: GenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
params = super().map_config(config, model_detail, capabilities)
params["reasoning_split"] = True
return params
class MiniMaxTextHandler(OpenAITextHandler):
"""MiniMax 复合文本处理器"""
def __init__(self, api_type: str = "minimax"):
super().__init__(api_type=api_type)
self.converter = MiniMaxMessageConverter(api_type=api_type)
self.mapper = MiniMaxConfigMapper(api_type=api_type)
class MiniMaxAdapter(OpenAICompatAdapter):
"""
MiniMax API 适配器。
"""
def __init__(self):
"""初始化 MiniMax 适配器并挂载专属处理器。"""
super().__init__()
self.text_handler = MiniMaxTextHandler(api_type=self.api_type)
self.audio_handler = MiniMaxAudioHandler()
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "minimax"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["minimax"]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
"""根据官方兼容要求,重写获取端点,允许自定义覆盖"""
return "/v1/chat/completions"
def _get_base_url(self, identity: ModelIdentity) -> str:
base_url = (
identity.api_base.rstrip("/")
if identity.api_base
else "https://api.minimaxi.com"
)
if base_url.endswith("/v1"):
base_url = base_url[:-3]
return base_url
+122
View File
@@ -0,0 +1,122 @@
"""
OpenAI API 适配器
支持 OpenAI、智谱AI 等 OpenAI 兼容的 API 服务。
"""
from __future__ import annotations
from abc import abstractmethod
from zhenxun.services.ai.core.models import ModelIdentity
from .base import (
BaseAdapter,
RequestData,
)
from .handlers.openai_handlers import (
CompositeOpenAITextHandler,
OpenAIAudioHandler,
OpenAIEmbeddingHandler,
OpenAIImageHandler,
OpenAIRerankHandler,
)
class OpenAICompatAdapter(BaseAdapter):
"""
OpenAI 兼容 API 适配器基类。
保留端点获取等基础逻辑。
"""
@property
def log_sanitization_context(self) -> str:
"""返回 OpenAI 系列请求的日志清洗上下文。"""
return "openai_request"
@abstractmethod
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
"""子类必须实现,返回 chat completions 的端点"""
pass
def get_embedding_endpoint(self, identity: ModelIdentity) -> str:
"""返回 embeddings 的默认端点"""
return "/v1/embeddings"
async def prepare_simple_request(
self,
identity: ModelIdentity,
api_key: str,
prompt: str,
history: list[dict[str, str]] | None = None,
) -> RequestData:
"""准备简单文本生成请求"""
from zhenxun.services.ai.core.messages import (
AssistantMessage,
SystemMessage,
TextPart,
UserMessage,
)
messages = []
if history:
for msg in history:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "system":
messages.append(SystemMessage(content=[TextPart(text=content)]))
elif role == "assistant":
messages.append(AssistantMessage(content=[TextPart(text=content)]))
else:
messages.append(UserMessage(content=[TextPart(text=content)]))
messages.append(UserMessage(content=[TextPart(text=prompt)]))
config = identity.generation_config
from zhenxun.services.ai.core.messages import ChatRequest
request = ChatRequest(messages=messages, config=config)
return await self.prepare_advanced_request(
identity=identity,
api_key=api_key,
request=request,
)
class OpenAIAdapter(OpenAICompatAdapter):
"""OpenAI 系列适配器,统一装配文本/图像/嵌入/重排处理链。"""
def __init__(self):
"""初始化并挂载复合文本处理器与通用多模态处理器。"""
super().__init__()
self.text_handler = CompositeOpenAITextHandler(api_type=self.api_type)
self.image_handler = OpenAIImageHandler()
self.embedding_handler = OpenAIEmbeddingHandler()
self.rerank_handler = OpenAIRerankHandler()
self.audio_handler = OpenAIAudioHandler()
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "openai"
@property
def supported_api_types(self) -> list[str]:
"""支持的 API 类型及别名。"""
return [
"openai",
"openai_responses",
]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
"""返回聊天完成端点"""
current_api_type = identity.api_type
if current_api_type == "openai_responses":
return "/v1/responses"
if current_api_type == "doubao":
return "/api/v3/chat/completions"
return "/v1/chat/completions"
def get_embedding_endpoint(self, identity: ModelIdentity) -> str:
"""返回嵌入端点。"""
return "/v1/embeddings"
@@ -0,0 +1,202 @@
import base64
from pathlib import Path
from typing import Any
from zhenxun.services.ai.core.exceptions import LLMException
from zhenxun.services.ai.core.messages import (
ImagePart,
ImageRequest,
LLMMessage,
ThoughtPart,
)
from zhenxun.services.ai.core.models import ModelIdentity
from zhenxun.services.ai.llm.adapters.base import (
BaseAdapter,
RequestData,
ResponseData,
process_image_data,
)
from zhenxun.services.ai.llm.adapters.handlers.base import BaseImageHandler
from zhenxun.services.ai.llm.adapters.handlers.openai_handlers import (
CompositeOpenAITextHandler,
OpenAIMessageConverter,
)
from zhenxun.services.ai.llm.adapters.openai import OpenAIAdapter
class OpenRouterMessageConverter(OpenAIMessageConverter):
"""OpenRouter 专有消息转换器:处理 reasoning_details 的无损回传"""
async def convert_messages_async(
self, messages: list[LLMMessage]
) -> list[dict[str, Any]]:
openai_messages = await super().convert_messages_async(messages)
assistant_msgs = [m for m in messages if getattr(m, "role", "") == "assistant"]
ast_idx = 0
for o_msg in openai_messages:
if o_msg.get("role") == "assistant":
if ast_idx < len(assistant_msgs):
orig_ast = assistant_msgs[ast_idx]
ast_idx += 1
thought_parts = [
p for p in orig_ast.content if isinstance(p, ThoughtPart)
]
if thought_parts:
part = thought_parts[0]
raw_details = (
part.metadata.get("raw_reasoning_details")
if part.metadata
else None
)
if raw_details:
o_msg["reasoning_details"] = raw_details
o_msg.pop("reasoning_content", None)
o_msg.pop("reasoning", None)
return openai_messages
class OpenRouterTextHandler(CompositeOpenAITextHandler):
"""OpenRouter 专有文本处理器,挂载专有 Converter"""
def __init__(self, api_type: str = "openrouter"):
super().__init__(api_type=api_type)
self._standard_handler.converter = OpenRouterMessageConverter(api_type=api_type)
class OpenRouterImageHandler(BaseImageHandler):
"""OpenRouter 专有的图像生成处理器"""
def prepare_image_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: ImageRequest,
) -> RequestData:
headers = adapter.get_base_headers(api_key)
endpoint = "/v1/chat/completions"
url = adapter.get_api_url(identity, endpoint)
body: dict[str, Any] = {
"model": identity.model_name,
"modalities": ["image", "text"],
}
if request.images:
content_list: list[dict[str, Any]] = [
{"type": "text", "text": request.prompt}
]
for img_source in request.images:
img_bytes = None
if isinstance(img_source, bytes):
img_bytes = img_source
elif hasattr(img_source, "read_bytes"):
img_bytes = img_source.read_bytes()
elif isinstance(img_source, str) and img_source.startswith(
"data:image"
):
content_list.append(
{"type": "image_url", "image_url": {"url": img_source}}
)
continue
else:
raise LLMException(
"OpenRouter 图像生成仅支持 bytes/Path/base64 URI"
)
if img_bytes:
mime_type = "image/jpeg"
if img_bytes.startswith(b"\x89PNG\r\n\x1a\n"):
mime_type = "image/png"
elif img_bytes.startswith(b"GIF87a") or img_bytes.startswith(
b"GIF89a"
):
mime_type = "image/gif"
elif img_bytes.startswith(b"RIFF") and img_bytes[8:12] == b"WEBP":
mime_type = "image/webp"
b64_str = base64.b64encode(img_bytes).decode("utf-8")
content_list.append(
{
"type": "image_url",
"image_url": {"url": f"data:{mime_type};base64,{b64_str}"},
}
)
body["messages"] = [{"role": "user", "content": content_list}]
else:
body["messages"] = [{"role": "user", "content": request.prompt}]
if request.config:
image_config = {}
if request.config.media.aspect_ratio:
image_config["aspect_ratio"] = str(request.config.media.aspect_ratio)
if request.config.media.resolution:
image_config["image_size"] = str(
request.config.media.resolution
).upper()
if image_config:
body["image_config"] = image_config
return RequestData(url=url, headers=headers, body=body)
def parse_image_response(
self, adapter: BaseAdapter, response_json: dict[str, Any]
) -> ResponseData:
adapter.validate_response(response_json)
images_data = []
choices = response_json.get("choices", [])
if choices:
message = choices[0].get("message", {})
if "images" in message:
for img_data in message["images"]:
img_url_obj = img_data.get("image_url", {})
url_str = img_url_obj.get("url", "")
if url_str.startswith("data:image"):
try:
b64_data = url_str.split(",", 1)[1]
decoded = base64.b64decode(b64_data)
images_data.append(process_image_data(decoded))
except Exception:
pass
elif url_str:
images_data.append(url_str)
content_parts = []
for img in images_data:
if isinstance(img, str) and img.startswith("http"):
content_parts.append(ImagePart(url=img))
elif isinstance(img, bytes):
content_parts.append(ImagePart(raw=img))
else:
content_parts.append(ImagePart(path=Path(img)))
if not content_parts:
raise LLMException("OpenRouter 图像生成响应中未找到有效的图片数据")
return ResponseData(content_parts=content_parts, raw_response=response_json)
class OpenRouterAdapter(OpenAIAdapter):
"""OpenRouter 平台适配器"""
def __init__(self):
super().__init__()
self.text_handler = OpenRouterTextHandler(api_type=self.api_type)
self.image_handler = OpenRouterImageHandler()
@property
def api_type(self) -> str:
return "openrouter"
@property
def supported_api_types(self) -> list[str]:
return ["openrouter"]