✨ feat!(llm): 重构并升级大语言模型服务为全新 AI 智能体框架 (#2146)

* ✨ feat!(llm): 重构并升级大语言模型服务为全新 AI 智能体框架

- 【重构】将原 services/llm 重构并迁移至全新的 services/ai 架构,提供向下兼容垫片
- 【新增】引入 Agent、Team、Workflow 三大智能体与工作流编排范式
- 【新增】引入基于 RAG 的长期向量记忆与中期槽位记忆系统
- 【新增】引入基于 Docker 的安全代码执行沙箱环境
- 【新增】支持 MCP 协议,允许动态管理和调用 MCP 服务
- 【新增】引入输入输出安全合规护栏与自愈反思机制
- 【优化】重构并优化多厂商 API 适配器 (Gemini, OpenAI, DeepSeek, GLM 等)
- 【优化】优化日志脱敏与 Token 预估机制
- 【移除】移除旧版 llm default 和 llm reset-key 命令,新增 llm mcp 管理命令

* 🔧 chore(deps): 更新项目依赖与配置

- 添加 mcp、jieba 和 aiodocker 依赖到配置文件及 requirements.txt
- 在 pyright 配置中设置 reportMissingImports 为 none
- 调整 .gitignore 中 resources 目录的忽略规则

* ♻️ refactor(tools): 重构工具终止机制并清理知识库日志输出

- 统一使用 `context.state["__end_run__"]` 替代 `EndRunResult` 控制任务结束
- 移除文件系统和向量知识库检索工具中 `ToolResult` 的 `.with_log` 调用
- 调整指令处理器(Directive)的返回值为 `tool_res.output`
- 修复部分类型检查警告并优化联合类型判断语法

* ♻️ refactor(tools): 重构工具副作用指令与控制流熔断机制

- 引入 `DirectivePayload` 及 `ToolResult` 的子类以结构化表达工具副作用
- 移除通过 `context.state` 传递魔术变量的隐式控制流设计
- 重构 `DirectiveManager` 处理器接口,直接在处理器中修改 `AgentState` 并构建 `AgentRunResult`
- 在 `StandardAgentExecutor` 中统一通过 `directive_manager` 调度工具返回的副作用指令
- 补全 `MessageBuilder` 中部分核心方法的文档注释

* 🐛 fix(sandbox): 修复 Docker 沙箱容器状态检测与会话清理逻辑

-【修复】修正 `is_alive` 中直接读取私有属性的问题,改用 `show()` 返回值
-【修复】解决 `execute_code` 中缓存的执行器与当前会话不一致的问题
-【优化】在清理工作区前增加容器存活检测,避免向已死容器发送请求
-【优化】创建容器时增加运行状态校验,若已停止则自动从缓存中移除并重建
-【优化】优化容器销毁和清理逻辑,静默处理容器不存在 (404) 的异常

* 📝 docs(core): 补充核心模块初始化方法的文档注释

* 🚨 auto fix by pre-commit hooks

---------

Co-authored-by: webjoin111 <455457521@qq.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Rumio
2026-07-03 08:53:56 +08:00
committed by GitHub
co-authored by webjoin111 pre-commit-ci[bot]
parent bdc1374848
commit 80fc5b86a7
223 changed files with 41885 additions and 10006 deletions
+48
View File
@@ -0,0 +1,48 @@
"""
LLM 服务模块 - 公共 API 入口
提供统一的 AI 服务调用接口、核心数据契约和配置工具。
"""
from zhenxun.services.ai.core.exceptions import LLMException
from zhenxun.services.ai.core.messages import (
AudioResponse,
ChatResponse,
LLMContentPart,
LLMMessage,
)
from zhenxun.services.ai.core.options import (
TTSConfig,
)
from .api import (
chat,
create_image,
create_speech,
embed,
generate,
generate_structured,
rerank,
)
from .builder import (
IntentBuilder,
)
from .manager import get_default_model
__all__ = [
"AudioResponse",
"ChatResponse",
"IntentBuilder",
"LLMContentPart",
"LLMException",
"LLMMessage",
"TTSConfig",
"chat",
"create_image",
"create_speech",
"embed",
"generate",
"generate_structured",
"get_default_model",
"rerank",
]
@@ -0,0 +1,38 @@
"""
LLM 适配器模块
提供不同LLM服务商的API适配器实现,统一接口调用方式。
"""
from .base import BaseAdapter, RequestData, ResponseData
from .deepseek import DeepSeekAdapter
from .doubao import DoubaoAdapter
from .factory import LLMAdapterFactory, get_adapter_for_api_type, register_adapter
from .gemini import GeminiAdapter
from .glm import GLMAdapter
from .jina import JinaAdapter
from .mimo import MiMoAdapter
from .minimax import MiniMaxAdapter
from .openai import OpenAIAdapter, OpenAICompatAdapter
from .openrouter import OpenRouterAdapter
LLMAdapterFactory.initialize()
__all__ = [
"BaseAdapter",
"DeepSeekAdapter",
"DoubaoAdapter",
"GLMAdapter",
"GeminiAdapter",
"JinaAdapter",
"LLMAdapterFactory",
"MiMoAdapter",
"MiniMaxAdapter",
"OpenAIAdapter",
"OpenAICompatAdapter",
"OpenRouterAdapter",
"RequestData",
"ResponseData",
"get_adapter_for_api_type",
"register_adapter",
]
+702
View File
@@ -0,0 +1,702 @@
"""
LLM 适配器基类和通用数据结构
"""
from __future__ import annotations
from abc import ABC, abstractmethod
import inspect
import json
from pathlib import Path
from typing import TYPE_CHECKING, Any
import uuid
import httpx
from pydantic import BaseModel, Field
from zhenxun.configs.path_config import TEMP_PATH
from zhenxun.services.ai.core.engine.token_counter import parse_usage_info
from zhenxun.services.ai.core.exceptions import (
AuthenticationException,
ConfigurationException,
ContentFilteredException,
ContextLengthExceededException,
InvalidRequestException,
LLMException,
LocationNotSupportedException,
QuotaExceededException,
RateLimitException,
ResponseParseException,
UpstreamServerException,
)
from zhenxun.services.ai.core.messages import (
AudioResponse,
ChatRequest,
ChatResponse,
EmbeddingRequest,
EmbeddingResponse,
ImagePart,
ImageRequest,
ImageResponse,
LLMContentPart,
RerankRequest,
RerankResponse,
RerankResult,
SpeechRequest,
TextPart,
ThoughtPart,
)
from zhenxun.services.ai.core.models import ModelIdentity
from zhenxun.services.log import logger
from zhenxun.utils.log_sanitizer import sanitize_for_logging
if TYPE_CHECKING:
from zhenxun.services.ai.llm.adapters.handlers.base import (
BaseAudioHandler,
BaseEmbeddingHandler,
BaseImageHandler,
BaseRerankHandler,
BaseTextHandler,
)
class RequestData(BaseModel):
"""标准化的请求载体,用于向上层 HTTP 客户端传递请求参数。"""
method: str = "POST"
url: str
headers: dict[str, str]
body: dict[str, Any]
files: dict[str, Any] | list[tuple[str, Any]] | None = None
class ResponseData(BaseModel):
"""标准化的响应载体,统一承接文本、多模态与附加元数据。"""
content_parts: list[LLMContentPart] = Field(default_factory=list)
usage_info: dict[str, Any] | None = None
raw_response: dict[str, Any] | None = None
grounding_metadata: Any | None = None
cache_info: Any | None = None
@property
def text(self) -> str:
"""提取并拼接所有 `TextPart` 文本内容。"""
return "".join(
p.text for p in self.content_parts if isinstance(p, TextPart)
).strip()
@text.setter
def text(self, value: str):
"""设置首个 `TextPart`,不存在则追加新的 `TextPart`。"""
for p in self.content_parts:
if isinstance(p, TextPart):
p.text = value
return
self.content_parts.append(TextPart(text=value))
@property
def thought_text(self) -> str | None:
"""提取并拼接所有思维片段文本,未命中则返回 `None`。"""
thoughts = [
p.thought_text for p in self.content_parts if isinstance(p, ThoughtPart)
]
return "\n".join(thoughts).strip() if thoughts else None
@property
def thought_signature(self) -> str | None:
"""从末尾向前查找思维签名,用于后续连续推理场景。"""
for p in reversed(self.content_parts):
if (
hasattr(p, "metadata")
and p.metadata
and "thought_signature" in p.metadata
):
return p.metadata["thought_signature"]
return None
@property
def images(self) -> list[bytes | Path | str]:
"""收集图片内容,按 URL / 原始字节 / 本地路径顺序返回。"""
imgs = []
for p in self.content_parts:
if isinstance(p, ImagePart):
if p.url:
imgs.append(p.url)
elif p.raw:
imgs.append(p.raw)
elif p.path:
imgs.append(p.path)
return imgs
@images.setter
def images(self, value: list[bytes | Path | str]):
"""覆盖图片片段并按输入类型重建 `ImagePart` 列表。"""
self.content_parts = [
p for p in self.content_parts if not isinstance(p, ImagePart)
]
for img in value:
if isinstance(img, str) and img.startswith(("http://", "https://")):
self.content_parts.append(ImagePart(url=img))
elif isinstance(img, bytes):
self.content_parts.append(ImagePart(raw=img))
else:
self.content_parts.append(ImagePart(path=Path(img)))
code_execution_results: list[dict[str, Any]] | None = None
search_results: list[dict[str, Any]] | None = None
function_calls: list[dict[str, Any]] | None = None
safety_ratings: list[dict[str, Any]] | None = None
citations: list[dict[str, Any]] | None = None
def process_image_data(image_data: bytes) -> bytes | Path:
"""处理图片二进制数据:超过 2MB 时落盘并返回文件路径。"""
max_inline_size = 2 * 1024 * 1024
if len(image_data) > max_inline_size:
save_dir = TEMP_PATH / "llm"
save_dir.mkdir(parents=True, exist_ok=True)
file_name = f"{uuid.uuid4()}.png"
file_path = save_dir / file_name
file_path.write_bytes(image_data)
logger.info(
f"图片数据过大 ({len(image_data)} bytes),已保存到临时文件: {file_path}",
"LLMAdapter",
)
return file_path.resolve()
return image_data
class BaseAdapter(ABC):
"""
LLM API适配器基类 (门面模式 Facade)。
负责维护厂商级别的通用配置(如 URL 拼接、请求头构建、通用错误拦截),
而将具体的模态序列化与反序列化逻辑委派给各路 Handler。
"""
text_handler: "BaseTextHandler | None" = None
image_handler: "BaseImageHandler | None" = None
embedding_handler: "BaseEmbeddingHandler | None" = None
rerank_handler: "BaseRerankHandler | None" = None
audio_handler: "BaseAudioHandler | None" = None
@property
def log_sanitization_context(self) -> str:
"""用于日志清洗的上下文名称,默认 'default'"""
return "default"
@property
@abstractmethod
def api_type(self) -> str:
"""API类型标识"""
pass
@property
@abstractmethod
def supported_api_types(self) -> list[str]:
"""支持的API类型列表"""
pass
async def prepare_payload(
self, identity: ModelIdentity, api_key: str, request: Any
) -> RequestData:
"""泛型请求构建分发入口 (Polymorphic Dispatch)"""
dispatch = {
ChatRequest: self.prepare_advanced_request,
EmbeddingRequest: self.prepare_embedding_request,
RerankRequest: self.prepare_rerank_request,
ImageRequest: self.prepare_image_request,
SpeechRequest: self.prepare_speech_request,
}
handler = dispatch.get(type(request))
if not handler:
raise ValueError(
f"适配器 {self.api_type} 不支持的请求类型: {type(request)}"
)
res = handler(identity, api_key, request)
if inspect.isawaitable(res):
return await res
return res
async def parse_payload(
self, identity: ModelIdentity, request: Any, raw_response: httpx.Response
) -> Any:
"""泛型响应解析分发入口 (Polymorphic Dispatch)"""
if isinstance(request, SpeechRequest):
res = self.parse_speech_response(identity, raw_response)
if inspect.isawaitable(res):
return await res
return res
response_bytes = await raw_response.aread()
logger.debug(f"📦 响应体已完整读取 ({len(response_bytes)} bytes)")
try:
response_json = json.loads(response_bytes)
except json.JSONDecodeError:
raise ResponseParseException(
"API 返回了非 JSON 格式的内容,可能是 URL 路径错误或中转站配置异常。",
details={
"raw_response": response_bytes.decode("utf-8", errors="ignore")[
:500
]
},
)
sanitizer_req_context = self.log_sanitization_context
sanitizer_resp_context = sanitizer_req_context.replace("_request", "_response")
if sanitizer_resp_context == sanitizer_req_context:
sanitizer_resp_context = f"{sanitizer_req_context}_response"
sanitized_response = sanitize_for_logging(
response_json, context=sanitizer_resp_context
)
response_json_str = json.dumps(sanitized_response, ensure_ascii=False, indent=2)
logger.debug(f"📋 响应JSON: {response_json_str}")
dispatch = {
EmbeddingRequest: self._parse_embedding_payload,
RerankRequest: self._parse_rerank_payload,
ImageRequest: self._parse_image_payload,
ChatRequest: self._parse_chat_payload,
}
handler = dispatch.get(type(request))
if not handler:
raise ValueError(
f"适配器 {self.api_type} 不支持的请求类型解析: {type(request)}"
)
return handler(identity, response_json)
def _parse_embedding_payload(
self, identity: ModelIdentity, response_json: dict
) -> EmbeddingResponse:
self.validate_embedding_response(response_json)
embeddings = self.parse_embedding_response(response_json)
return EmbeddingResponse(
embeddings=embeddings,
usage=parse_usage_info(
response_json.get("usage") or response_json.get("usageMetadata")
),
model_name=identity.model_name,
)
def _parse_rerank_payload(
self, identity: ModelIdentity, response_json: dict
) -> RerankResponse:
return RerankResponse(results=self.parse_rerank_response(response_json))
def _parse_image_payload(
self, identity: ModelIdentity, response_json: dict
) -> ImageResponse:
response_data = self.parse_image_response(response_json)
return ImageResponse(
content_parts=response_data.content_parts, raw_response=response_json
)
def _parse_chat_payload(
self, identity: ModelIdentity, response_json: dict
) -> ChatResponse:
response_data = self.parse_response(identity, response_json, is_advanced=True)
return ChatResponse(
content_parts=response_data.content_parts,
usage_info=response_data.usage_info,
raw_response=response_data.raw_response,
grounding_metadata=response_data.grounding_metadata,
)
async def prepare_simple_request(
self,
identity: ModelIdentity,
api_key: str,
prompt: str,
history: list[dict[str, str]] | None = None,
) -> RequestData:
"""准备简单文本生成请求
默认实现:将简单请求转换为高级请求格式
子类可以重写此方法以提供特定的优化实现
"""
from zhenxun.services.ai.core.messages import (
AssistantMessage,
SystemMessage,
TextPart,
UserMessage,
)
messages: list[Any] = []
if history:
for msg in history:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "system":
messages.append(SystemMessage(content=[TextPart(text=content)]))
elif role == "assistant":
messages.append(AssistantMessage(content=[TextPart(text=content)]))
else:
messages.append(UserMessage(content=[TextPart(text=content)]))
messages.append(UserMessage(content=[TextPart(text=prompt)]))
config = identity.generation_config
return await self.prepare_advanced_request(
identity=identity,
api_key=api_key,
request=ChatRequest(messages=messages, config=config),
)
async def prepare_advanced_request(
self,
identity: ModelIdentity,
api_key: str,
request: ChatRequest,
) -> RequestData:
"""准备高级对话请求并委派给 `text_handler` 完成序列化。"""
if self.text_handler:
return await self.text_handler.prepare_text_request(
adapter=self,
identity=identity,
api_key=api_key,
request=request,
)
raise NotImplementedError(
f"API 类型 '{self.api_type}' 未装配 TextHandler,暂不支持文本对话能力。"
)
def parse_response(
self,
identity: ModelIdentity,
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
"""解析文本响应并委派给 `text_handler`。"""
if self.text_handler:
return self.text_handler.parse_text_response(
adapter=self,
identity=identity,
response_json=response_json,
is_advanced=is_advanced,
)
raise NotImplementedError(f"API 类型 '{self.api_type}' 未装配 TextHandler。")
async def prepare_embedding_request(
self,
identity: ModelIdentity,
api_key: str,
request: EmbeddingRequest,
) -> RequestData:
"""准备文本/多模态嵌入请求并委派给 `embedding_handler`。"""
if self.embedding_handler:
return await self.embedding_handler.prepare_embedding_request(
adapter=self,
identity=identity,
api_key=api_key,
request=request,
)
raise NotImplementedError(
f"API 类型 '{self.api_type}' 未装配 EmbeddingHandler,暂不支持向量嵌入。"
)
def parse_embedding_response(
self, response_json: dict[str, Any]
) -> list[list[float]]:
"""解析文本嵌入响应并委派给 `embedding_handler`。"""
if self.embedding_handler:
return self.embedding_handler.parse_embedding_response(
adapter=self, response_json=response_json
)
raise NotImplementedError(
f"API 类型 '{self.api_type}' 未装配 EmbeddingHandler。"
)
def prepare_rerank_request(
self,
identity: ModelIdentity,
api_key: str,
request: RerankRequest,
) -> RequestData:
"""准备重排请求并委派给 `rerank_handler`。"""
if self.rerank_handler:
return self.rerank_handler.prepare_rerank_request(
adapter=self,
identity=identity,
api_key=api_key,
request=request,
)
raise NotImplementedError(
f"API 类型 '{self.api_type}' 未装配 RerankHandler,暂不支持文本重排。"
)
def parse_rerank_response(
self, response_json: dict[str, Any]
) -> list[RerankResult]:
"""解析重排响应并委派给 `rerank_handler`。"""
if self.rerank_handler:
return self.rerank_handler.parse_rerank_response(
adapter=self, response_json=response_json
)
raise NotImplementedError(f"API 类型 '{self.api_type}' 未装配 RerankHandler。")
def prepare_image_request(
self,
identity: ModelIdentity,
api_key: str,
request: ImageRequest,
) -> RequestData:
"""准备图像请求并委派给 `image_handler`。"""
if self.image_handler:
return self.image_handler.prepare_image_request(
adapter=self,
identity=identity,
api_key=api_key,
request=request,
)
raise NotImplementedError(
f"API 类型 '{self.api_type}' 未装配 ImageHandler,暂不支持图像生成。"
)
def parse_image_response(self, response_json: dict[str, Any]) -> ResponseData:
"""解析图像响应并委派给 `image_handler`。"""
if self.image_handler:
return self.image_handler.parse_image_response(
adapter=self, response_json=response_json
)
raise NotImplementedError(f"API 类型 '{self.api_type}' 未装配 ImageHandler。")
def prepare_speech_request(
self,
identity: ModelIdentity,
api_key: str,
request: SpeechRequest,
) -> RequestData:
"""准备语音生成请求并委派给 `audio_handler`。"""
if self.audio_handler:
return self.audio_handler.prepare_speech_request(
adapter=self,
identity=identity,
api_key=api_key,
request=request,
)
raise NotImplementedError(
f"API 类型 '{self.api_type}' 未装配 AudioHandler,暂不支持语音生成。"
)
async def parse_speech_response(
self, identity: ModelIdentity, raw_response: httpx.Response
) -> AudioResponse:
"""解析语音响应并委派给 `audio_handler`。
注意传入的是 httpx.Response 的 raw 对象"""
if self.audio_handler:
return await self.audio_handler.parse_speech_response(
adapter=self, identity=identity, raw_response=raw_response
)
raise NotImplementedError(f"API 类型 '{self.api_type}' 未装配 AudioHandler。")
def validate_embedding_response(self, response_json: dict[str, Any]) -> None:
"""验证嵌入接口响应,检测 `error` 并转换为统一异常。"""
if response_json.get("error"):
error_info = response_json["error"]
msg = (
error_info.get("message", str(error_info))
if isinstance(error_info, dict)
else str(error_info)
)
raise UpstreamServerException(
f"嵌入API错误: {msg}",
details=response_json,
)
def get_api_url(self, identity: ModelIdentity, endpoint: str) -> str:
"""拼接最终请求 URL,兼容 `path_prefix` 与端点前后斜杠。"""
if not identity.api_base:
raise ConfigurationException(
f"模型 {identity.model_name} 的 api_base 未设置",
)
base_url = identity.api_base.rstrip("/")
prefix = identity.path_prefix.strip("/") if identity.path_prefix else ""
ep = endpoint.lstrip("/")
if prefix:
return f"{base_url}/{prefix}/{ep}"
return f"{base_url}/{ep}"
def get_base_headers(self, api_key: str) -> dict[str, str]:
"""构建默认请求头,包含 UA、JSON 类型与 Bearer 鉴权。"""
from zhenxun.utils.user_agent import get_user_agent
headers = get_user_agent()
headers.update(
{
"Content-Type": "application/json",
"Authorization": f"Bearer {api_key}",
}
)
return headers
def validate_response(self, response_json: dict[str, Any]) -> None:
"""统一校验文本/多模态响应并映射平台错误码。"""
if response_json.get("error"):
error_info = response_json["error"]
error_message = str(error_info)
if isinstance(error_info, dict):
error_message = error_info.get("message", error_message)
error_code = error_info.get("code", "unknown")
if (
error_code in ("invalid_api_key", "authentication_failed")
or "permission" in error_message.lower()
):
raise AuthenticationException(
f"鉴权失败: {error_message}", details={"api_error": error_info}
)
elif error_code in ("insufficient_quota", "quota_exceeded"):
raise QuotaExceededException(
f"配额耗尽: {error_message}", details={"api_error": error_info}
)
elif error_code == "rate_limit_exceeded":
raise RateLimitException(
f"请求限流: {error_message}", details={"api_error": error_info}
)
elif error_code in ("model_not_found", "invalid_model"):
raise ConfigurationException(
f"模型配置错误: {error_message}",
details={"api_error": error_info},
)
elif error_code in (
"context_length_exceeded",
"max_tokens_exceeded",
"1261",
):
raise ContextLengthExceededException(
f"上下文超限: {error_message}",
details={"api_error": error_info},
)
elif error_code in ("invalid_request_error", "invalid_parameter"):
raise InvalidRequestException(
f"请求参数错误: {error_message}",
details={"api_error": error_info},
)
raise UpstreamServerException(
f"API请求报错: {error_message}",
details={"api_error": error_info},
)
if "candidates" in response_json:
candidates = response_json.get("candidates", [])
if candidates:
candidate = candidates[0]
finish_reason = candidate.get("finishReason")
if finish_reason in ["SAFETY", "RECITATION"]:
raise ContentFilteredException(
f"内容被模型安全策略过滤: {finish_reason}",
details={
"finish_reason": finish_reason,
},
)
if not response_json:
raise UpstreamServerException(
"API返回空响应",
details={"response": response_json},
)
def handle_http_error(self, response: httpx.Response) -> LLMException | None:
"""
处理 HTTP 错误响应。
如果响应状态码表示成功 (200),返回 None;否则构造 LLMException 供外部捕获。
"""
if response.status_code == 200:
return None
error_text = response.content.decode("utf-8", errors="ignore")
error_status = ""
error_msg = error_text
try:
error_json = json.loads(error_text)
if isinstance(error_json, dict) and "error" in error_json:
error_info = error_json["error"]
if isinstance(error_info, dict):
error_msg = error_info.get("message", error_msg)
raw_status = error_info.get("status") or error_info.get("code")
error_status = str(raw_status) if raw_status is not None else ""
elif error_info is not None:
error_msg = str(error_info)
error_status = error_msg
except Exception:
pass
status_upper = error_status.upper() if error_status else ""
text_upper = error_text.upper()
if response.status_code == 400:
if (
"FAILED_PRECONDITION" in status_upper
or "LOCATION IS NOT SUPPORTED" in text_upper
):
return LocationNotSupportedException(
"当前地区不支持该服务", details={"response": error_text}
)
elif "API_KEY_INVALID" in text_upper or "API KEY NOT VALID" in text_upper:
return AuthenticationException(
"API Key 无效", details={"response": error_text}
)
elif (
status_upper
in ["1261", "STRING_ABOVE_MAX_LENGTH", "CONTEXT_LENGTH_EXCEEDED"]
or "EXCEEDS MAX LENGTH" in text_upper
or "STRING TOO LONG" in text_upper
):
return ContextLengthExceededException(
"上下文超长", details={"response": error_text}
)
else:
return InvalidRequestException(
f"参数错误: {error_msg}", details={"response": error_text}
)
elif response.status_code in [401, 403]:
if "country" in error_msg.lower() or "unsupported" in error_msg.lower():
return LocationNotSupportedException(
"地区受限", details={"response": error_text}
)
else:
return AuthenticationException(
"鉴权失败/权限不足", details={"response": error_text}
)
elif response.status_code == 404:
return ConfigurationException(
"端点或模型未找到", details={"response": error_text}
)
elif response.status_code == 429:
if (
"RESOURCE_EXHAUSTED" in status_upper
or "INSUFFICIENT_QUOTA" in status_upper
or ("quota" in error_msg.lower() if error_msg else False)
):
return QuotaExceededException(
"API 配额耗尽", details={"response": error_text}
)
else:
return RateLimitException(
"请求频繁被限流", details={"response": error_text}
)
elif response.status_code in [402, 413]:
return QuotaExceededException(
"资源耗尽/文件过大", details={"response": error_text}
)
elif response.status_code >= 500:
return UpstreamServerException(
f"HTTP请求失败: {response.status_code} ({error_status or 'Unknown'})",
details={
"status_code": response.status_code,
"response": error_text,
},
)
return UpstreamServerException(
f"未知网络错误 {response.status_code}: {error_msg}"
)
@@ -0,0 +1,140 @@
from typing import Any
from zhenxun.services.ai.core.models import (
ModelCapabilities,
ModelDetail,
ModelIdentity,
)
from zhenxun.services.ai.core.options import GenerationConfig
from zhenxun.services.ai.llm.adapters.handlers.openai_handlers import (
OpenAIConfigMapper,
OpenAITextHandler,
OpenAIToolSerializer,
)
from zhenxun.services.ai.llm.adapters.openai import OpenAICompatAdapter
class DeepSeekToolSerializer(OpenAIToolSerializer):
"""
专门针对 DeepSeek 的工具序列化器。
负责抹平 Pydantic Schema 与 DeepSeek Strict Mode 之间的差异。
"""
def __init__(self, api_type: str = "deepseek"):
"""初始化 DeepSeek 工具序列化器。"""
super().__init__(api_type=api_type)
def sanitize_schema(self, schema: dict[str, Any]) -> dict[str, Any]:
from zhenxun.services.ai.llm.engine.schema_transformer import (
DeepSeekFallbackTransformer,
OpenAIUnionFlattenTransformer,
RefComplianceTransformer,
RemoveUnsupportedKeysTransformer,
RootRefInlineTransformer,
SchemaPipeline,
StrictObjectTransformer,
TypeEnforcerTransformer,
)
unsupported_keys = [
"default",
"minLength",
"maxLength",
"pattern",
"format",
"minimum",
"maximum",
"multipleOf",
"patternProperties",
"propertyNames",
"minItems",
"maxItems",
"uniqueItems",
"$schema",
"title",
]
pipeline = SchemaPipeline(
[
RootRefInlineTransformer(),
OpenAIUnionFlattenTransformer(),
TypeEnforcerTransformer(),
RemoveUnsupportedKeysTransformer(unsupported_keys),
StrictObjectTransformer(),
DeepSeekFallbackTransformer(),
RefComplianceTransformer(),
]
)
return pipeline.run(schema)
class DeepSeekConfigMapper(OpenAIConfigMapper):
"""DeepSeek 的专属配置映射器"""
def map_config(
self,
config: GenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
"""映射生成参数并处理 DeepSeek 专有 `thinking` 与响应格式差异。"""
params = super().map_config(config, model_detail, capabilities)
if "response_format" in params:
rf = params["response_format"]
if isinstance(rf, dict) and rf.get("type") == "json_schema":
params["response_format"] = {"type": "json_object"}
if config.common.reasoning_effort:
effort = str(config.common.reasoning_effort).lower()
if effort == "none":
params["thinking"] = {"type": "disabled"}
else:
params["thinking"] = {"type": "enabled"}
elif (
hasattr(config, "deepseek_options")
and config.deepseek_options.thinking is not None
):
if config.deepseek_options.thinking is True:
params["thinking"] = {"type": "enabled"}
elif config.deepseek_options.thinking is False:
params["thinking"] = {"type": "disabled"}
return params
class DeepSeekTextHandler(OpenAITextHandler):
"""DeepSeek 专有文本处理器,替换了特定序列化组件"""
def __init__(self, api_type: str = "deepseek"):
"""替换 OpenAI 默认组件为 DeepSeek 专用实现。"""
super().__init__(api_type=api_type)
self.serializer = DeepSeekToolSerializer(api_type=api_type)
self.mapper = DeepSeekConfigMapper(api_type=api_type)
class DeepSeekAdapter(OpenAICompatAdapter):
"""DeepSeek 官方 API 适配器"""
def __init__(self):
"""初始化 DeepSeek 适配器并挂载文本处理器。"""
super().__init__()
self.text_handler = DeepSeekTextHandler(api_type=self.api_type)
@property
def log_sanitization_context(self) -> str:
"""返回 DeepSeek 请求日志清洗上下文。"""
return "openai_request"
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "deepseek"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["deepseek"]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
"""返回对话端点,优先使用模型级自定义端点。"""
return "/v1/chat/completions"
@@ -0,0 +1,31 @@
from __future__ import annotations
from zhenxun.services.ai.core.models import ModelIdentity
from .handlers.openai_handlers import (
OpenAITextHandler,
)
from .openai import OpenAICompatAdapter
class DoubaoAdapter(OpenAICompatAdapter):
"""火山方舟 (Doubao) API 适配器"""
def __init__(self):
"""初始化 Doubao 适配器并复用 OpenAI 兼容处理器。"""
super().__init__()
self.text_handler = OpenAITextHandler(api_type=self.api_type)
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "doubao"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["doubao"]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
"""返回对话端点,优先使用模型级自定义端点。"""
return "/v3/chat/completions"
+167
View File
@@ -0,0 +1,167 @@
"""
LLM 适配器工厂类
"""
from __future__ import annotations
import fnmatch
from typing import Any, ClassVar
import httpx
from zhenxun.services.ai.core.exceptions import ConfigurationException
from zhenxun.services.ai.core.models import ModelIdentity
from .base import BaseAdapter, RequestData
class LLMAdapterFactory:
"""适配器注册与按 API 类型分发的统一入口。"""
_adapters: ClassVar[dict[str, BaseAdapter]] = {}
_api_type_mapping: ClassVar[dict[str, str]] = {}
@classmethod
def initialize(cls) -> None:
"""初始化默认适配器"""
if cls._adapters:
return
from .deepseek import DeepSeekAdapter
from .doubao import DoubaoAdapter
from .gemini import GeminiAdapter
from .glm import GLMAdapter
from .jina import JinaAdapter
from .mimo import MiMoAdapter
from .minimax import MiniMaxAdapter
from .openai import OpenAIAdapter
from .openrouter import OpenRouterAdapter
cls.register_adapter(OpenAIAdapter())
cls.register_adapter(OpenRouterAdapter())
cls.register_adapter(DeepSeekAdapter())
cls.register_adapter(JinaAdapter())
cls.register_adapter(GeminiAdapter())
cls.register_adapter(GLMAdapter())
cls.register_adapter(SmartAdapter())
cls.register_adapter(MiMoAdapter())
cls.register_adapter(MiniMaxAdapter())
cls.register_adapter(DoubaoAdapter())
@classmethod
def register_adapter(cls, adapter: BaseAdapter) -> None:
"""注册适配器"""
adapter_key = adapter.api_type
cls._adapters[adapter_key] = adapter
for api_type in adapter.supported_api_types:
cls._api_type_mapping[api_type] = adapter_key
@classmethod
def get_adapter(cls, api_type: str) -> BaseAdapter:
"""获取适配器"""
cls.initialize()
adapter_key = cls._api_type_mapping.get(api_type)
if not adapter_key:
raise ConfigurationException(
f"不支持的API类型: {api_type}",
details={
"api_type": api_type,
"supported_types": list(cls._api_type_mapping.keys()),
},
)
return cls._adapters[adapter_key]
@classmethod
def list_supported_types(cls) -> list[str]:
"""列出所有支持的API类型"""
cls.initialize()
return list(cls._api_type_mapping.keys())
@classmethod
def list_adapters(cls) -> dict[str, BaseAdapter]:
"""列出所有注册的适配器"""
cls.initialize()
return cls._adapters.copy()
def get_adapter_for_api_type(api_type: str) -> BaseAdapter:
"""按 API 类型获取适配器实例。"""
return LLMAdapterFactory.get_adapter(api_type)
def register_adapter(adapter: BaseAdapter) -> None:
"""向工厂注册新的适配器实例。"""
LLMAdapterFactory.register_adapter(adapter)
class SmartAdapter(BaseAdapter):
"""
智能路由适配器。
本身不处理序列化,而是根据规则委托给 OpenAIAdapter 或 GeminiAdapter。
"""
@property
def log_sanitization_context(self) -> str:
"""返回智能路由适配器的默认日志清洗上下文。"""
return "openai_request"
_ROUTING_RULES: ClassVar[list[tuple[str, str]]] = [
("*nano-banana*", "gemini"),
("*gemini*", "gemini"),
("*deepseek*", "deepseek"),
("*minimax*", "minimax"),
("*gpt*", "openai_responses"),
]
_DEFAULT_API_TYPE: ClassVar[str] = "openai"
def __init__(self):
"""初始化模型名到目标适配器的路由缓存。"""
self._adapter_cache: dict[str, BaseAdapter] = {}
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "smart"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["smart"]
def _get_delegate_adapter(self, identity: ModelIdentity) -> BaseAdapter:
"""
核心路由逻辑:决定使用哪个适配器 (带缓存)
"""
if identity.api_type and identity.api_type != "smart":
return get_adapter_for_api_type(identity.api_type)
model_name = identity.model_name
if model_name in self._adapter_cache:
return self._adapter_cache[model_name]
target_api_type = self._DEFAULT_API_TYPE
model_name_lower = model_name.lower()
for pattern, api_type in self._ROUTING_RULES:
if fnmatch.fnmatch(model_name_lower, pattern):
target_api_type = api_type
break
adapter = get_adapter_for_api_type(target_api_type)
self._adapter_cache[model_name] = adapter
return adapter
async def prepare_payload(
self, identity: ModelIdentity, api_key: str, request: Any
) -> RequestData:
adapter = self._get_delegate_adapter(identity)
return await adapter.prepare_payload(identity, api_key, request)
async def parse_payload(
self, identity: ModelIdentity, request: Any, raw_response: httpx.Response
) -> Any:
adapter = self._get_delegate_adapter(identity)
return await adapter.parse_payload(identity, request, raw_response)
@@ -0,0 +1,59 @@
"""
Gemini API 适配器
"""
from __future__ import annotations
from zhenxun.services.ai.core.models import ModelIdentity
from zhenxun.services.ai.core.options import GenerationConfig
from .base import BaseAdapter
from .handlers.gemini_handlers import (
GeminiAudioHandler,
GeminiEmbeddingHandler,
GeminiImageHandler,
GeminiTextHandler,
)
class GeminiAdapter(BaseAdapter):
"""Gemini API 适配器"""
def __init__(self):
"""初始化 Gemini 适配器并挂载各模态处理器。"""
super().__init__()
self.text_handler = GeminiTextHandler()
self.image_handler = GeminiImageHandler()
self.embedding_handler = GeminiEmbeddingHandler()
self.audio_handler = GeminiAudioHandler()
@property
def log_sanitization_context(self) -> str:
"""返回 Gemini 请求日志清洗上下文。"""
return "gemini_request"
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "gemini"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["gemini"]
def get_base_headers(self, api_key: str) -> dict[str, str]:
"""获取基础请求头"""
from zhenxun.utils.user_agent import get_user_agent
headers = get_user_agent()
headers.update({"Content-Type": "application/json"})
headers["x-goog-api-key"] = api_key
return headers
def _get_gemini_endpoint(
self, identity: ModelIdentity, config: GenerationConfig | None = None
) -> str:
"""返回Gemini generateContent 端点"""
return f"/v1beta/models/{identity.model_name}:generateContent"
+78
View File
@@ -0,0 +1,78 @@
from zhenxun.services.ai.core.messages import RerankRequest
from zhenxun.services.ai.core.models import ModelIdentity
from zhenxun.services.ai.llm.adapters.base import BaseAdapter, RequestData
from zhenxun.services.ai.llm.adapters.handlers.openai_handlers import (
OpenAIConfigMapper,
OpenAIEmbeddingHandler,
OpenAIRerankHandler,
OpenAITextHandler,
)
from zhenxun.services.ai.llm.adapters.openai import OpenAICompatAdapter
class GLMRerankHandler(OpenAIRerankHandler):
"""GLM 专有的重排处理器(重写了端点构建逻辑)"""
def prepare_rerank_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: RerankRequest,
) -> RequestData:
"""构建 GLM 重排请求,统一将文档归一化为字符串列表。"""
endpoint = "/api/paas/v4/rerank"
url = adapter.get_api_url(identity, endpoint)
headers = adapter.get_base_headers(api_key)
safe_documents = []
for doc in request.documents:
if isinstance(doc, dict):
safe_documents.append(doc.get("text", str(doc)))
else:
safe_documents.append(str(doc))
body = {
"model": identity.model_name,
"query": request.query,
"documents": safe_documents,
"top_n": request.top_n,
}
return RequestData(url=url, headers=headers, body=body)
class GLMTextHandler(OpenAITextHandler):
def __init__(self, api_type: str = "glm"):
super().__init__(api_type=api_type)
self.mapper = OpenAIConfigMapper(api_type=api_type)
class GLMAdapter(OpenAICompatAdapter):
"""GLM (智谱) 大模型专有适配器 (继承 OpenAI 兼容协议处理标准聊天)"""
def __init__(self):
"""初始化 GLM 适配器并装配专有图像/重排处理器。"""
super().__init__()
self.text_handler = GLMTextHandler(api_type=self.api_type)
self.embedding_handler = OpenAIEmbeddingHandler()
self.rerank_handler = GLMRerankHandler()
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "glm"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["glm"]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
"""返回对话端点,优先使用模型级自定义端点。"""
if None:
return None
return "/api/paas/v4/chat/completions"
def get_embedding_endpoint(self, identity: ModelIdentity) -> str:
"""返回嵌入端点。"""
return "/api/paas/v4/embeddings"
@@ -0,0 +1 @@
@@ -0,0 +1,223 @@
from __future__ import annotations
from abc import ABC, abstractmethod
import asyncio
from typing import Any
import httpx
from zhenxun.services.ai.core.messages import (
AudioResponse,
ChatRequest,
EmbeddingRequest,
ImageRequest,
LLMMessage,
RerankRequest,
RerankResult,
SpeechRequest,
)
from zhenxun.services.ai.core.models import (
ModelCapabilities,
ModelDetail,
ModelIdentity,
ToolDefinition,
)
from zhenxun.services.ai.core.options import (
GenerationConfig,
)
from zhenxun.services.ai.llm.adapters.base import BaseAdapter, RequestData, ResponseData
class ConfigMapper(ABC):
@abstractmethod
def map_config(
self,
config: GenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
"""将通用生成配置转换为特定 API 的参数字典"""
...
class MessageConverter(ABC):
@abstractmethod
async def convert_messages_async(
self, messages: list[LLMMessage]
) -> list[dict[str, Any]] | dict[str, Any]:
"""将通用消息列表异步转换为特定 API 的消息格式"""
...
class ToolSerializer(ABC):
def serialize_tools(
self, tools: list[ToolDefinition]
) -> list[dict[str, Any]] | None:
"""将通用工具定义转换为特定 API 的工具格式 (模板方法)"""
if not tools:
return None
serialized_tools = []
for tool in tools:
raw_schema = tool.parameters.copy() if tool.parameters else {}
sanitized_schema = self.sanitize_schema(raw_schema)
tool_payload = self.format_tool_payload(
tool_name=tool.name,
tool_description=tool.description or "",
sanitized_schema=sanitized_schema,
)
serialized_tools.append(tool_payload)
return serialized_tools
@abstractmethod
def format_tool_payload(
self, tool_name: str, tool_description: str, sanitized_schema: dict[str, Any]
) -> dict[str, Any]:
"""由子类实现:格式化单一工具的 Payload"""
...
@abstractmethod
def sanitize_schema(self, schema: dict[str, Any]) -> dict[str, Any]:
"""对 JSON Schema 进行特定 API 的清洗和格式化"""
...
@abstractmethod
def serialize_server_tools(
self, tools: list[Any], capabilities: ModelCapabilities
) -> list[dict[str, Any]]:
"""将系统内置的原生云端工具转译为底层 API 载荷(执行双重能力校验)"""
...
class ResponseParser(ABC):
@abstractmethod
def parse(self, response_json: dict[str, Any]) -> ResponseData:
"""将特定 API 的响应解析为通用响应数据"""
...
class BaseTextHandler(ABC):
"""
文本对话生成处理器接口。
负责将真寻的通用消息与工具列表转换为底层 API 的请求格式,并解析响应。
"""
async def _resolve_and_split_tools(
self, tools: list[Any] | None
) -> tuple[list[Any], list[Any], list[Any]]:
"""
统一的工具解析逻辑:分离客户端工具与服务端内置工具,并发获取 Schema。
返回: (tool_defs, client_executables, server_tools)
"""
client_executables, server_tools, tool_defs = [], [], []
if tools:
raw_tools = list(tools.values()) if isinstance(tools, dict) else tools
for tool in raw_tools:
if getattr(tool, "execution_side", "client") == "server":
server_tools.append(tool)
elif hasattr(tool, "get_definition"):
client_executables.append(tool)
if definition_tasks := [t.get_definition() for t in client_executables]:
tool_defs = [td for td in await asyncio.gather(*definition_tasks) if td]
return tool_defs, client_executables, server_tools
@abstractmethod
async def prepare_text_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: ChatRequest,
) -> RequestData: ...
@abstractmethod
def parse_text_response(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData: ...
class BaseEmbeddingHandler(ABC):
"""
文本嵌入向量处理器接口。
"""
@abstractmethod
async def prepare_embedding_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: EmbeddingRequest,
) -> RequestData: ...
@abstractmethod
def parse_embedding_response(
self, adapter: BaseAdapter, response_json: dict[str, Any]
) -> list[list[float]]: ...
class BaseImageHandler(ABC):
"""
图像生成/编辑处理器接口。
"""
@abstractmethod
def prepare_image_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: ImageRequest,
) -> RequestData: ...
@abstractmethod
def parse_image_response(
self, adapter: BaseAdapter, response_json: dict[str, Any]
) -> ResponseData: ...
class BaseRerankHandler(ABC):
"""
文本重排处理器接口。
"""
@abstractmethod
def prepare_rerank_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: RerankRequest,
) -> RequestData: ...
@abstractmethod
def parse_rerank_response(
self, adapter: BaseAdapter, response_json: dict[str, Any]
) -> list[RerankResult]: ...
class BaseAudioHandler(ABC):
"""
文本转语音 (TTS) 处理器接口。
"""
@abstractmethod
def prepare_speech_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: SpeechRequest,
) -> RequestData: ...
@abstractmethod
async def parse_speech_response(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
raw_response: httpx.Response,
) -> AudioResponse: ...
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,199 @@
import base64
import json
from typing import Any
import httpx
from zhenxun.services.ai.core.messages import (
AudioPart,
AudioResponse,
ImagePart,
LLMMessage,
SpeechRequest,
TextPart,
UsageInfo,
VideoPart,
)
from zhenxun.services.ai.core.models import (
ModelCapabilities,
ModelDetail,
ModelIdentity,
)
from zhenxun.services.ai.core.options import GenerationConfig, TTSConfig
from zhenxun.services.ai.llm.adapters.base import BaseAdapter, RequestData
from zhenxun.services.ai.llm.adapters.handlers.base import BaseAudioHandler
from zhenxun.services.ai.llm.adapters.handlers.openai_handlers import (
OpenAIConfigMapper,
OpenAIMessageConverter,
OpenAITextHandler,
OpenAIToolSerializer,
)
class MiMoToolSerializer(OpenAIToolSerializer):
"""MiMo 工具序列化器,负责拦截并构造独有的 web_search 工具"""
def serialize_server_tools(
self, tools: list[Any], capabilities: ModelCapabilities
) -> list[dict[str, Any]]:
res = []
for t in tools:
type_id = getattr(t, "type_id", "unknown")
if type_id not in capabilities.supported_native_tools:
continue
if type_id == "web_search":
res.append(
{
"type": "web_search",
"max_keyword": getattr(t, "max_keyword", 3),
"force_search": getattr(t, "force_search", True),
"limit": getattr(t, "limit", 1),
}
)
return res
class MiMoConfigMapper(OpenAIConfigMapper):
"""MiMo 配置映射器,处理深度思考参数差异"""
def map_config(
self,
config: GenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
params = super().map_config(config, model_detail, capabilities)
if config.common.reasoning_effort:
effort = str(config.common.reasoning_effort).lower()
if effort == "none":
params["thinking"] = {"type": "disabled"}
else:
params["thinking"] = {"type": "enabled"}
elif (
hasattr(config, "deepseek_options")
and config.deepseek_options.thinking is not None
):
if config.deepseek_options.thinking is True:
params["thinking"] = {"type": "enabled"}
elif config.deepseek_options.thinking is False:
params["thinking"] = {"type": "disabled"}
return params
class MiMoMessageConverter(OpenAIMessageConverter):
"""MiMo 消息转换器,拦截处理特有的音视频多模态结构"""
async def convert_messages_async(
self, messages: list[LLMMessage]
) -> list[dict[str, Any]]:
openai_messages = await super().convert_messages_async(messages)
for o_msg, o_orig in zip(openai_messages, messages):
if o_msg["role"] == "user":
content_parts = []
for part in o_orig.content:
if isinstance(part, TextPart):
content_parts.append({"type": "text", "text": part.text})
elif isinstance(part, ImagePart):
src = (
part.url
if part.url
else await part.get_data_uri(part.mime_type or "image/jpeg")
)
content_parts.append(
{"type": "image_url", "image_url": {"url": src}}
)
elif isinstance(part, VideoPart):
src = (
part.url
if part.url
else await part.get_data_uri(part.mime_type or "video/mp4")
)
content_parts.append(
{
"type": "video_url",
"video_url": {"url": src},
"fps": getattr(part, "fps", 2),
"media_resolution": getattr(
part, "media_resolution", "default"
),
}
)
elif isinstance(part, AudioPart):
src = (
part.url
if part.url
else await part.get_data_uri(part.mime_type or "audio/mp3")
)
content_parts.append(
{"type": "input_audio", "input_audio": {"data": src}}
)
o_msg["content"] = content_parts
return openai_messages
class MiMoTextHandler(OpenAITextHandler):
"""MiMo 文本对话处理器集成"""
def __init__(self, api_type: str = "mimo"):
super().__init__(api_type=api_type)
self.converter = MiMoMessageConverter(api_type=api_type)
self.serializer = MiMoToolSerializer(api_type=api_type)
self.mapper = MiMoConfigMapper(api_type=api_type)
class MiMoAudioHandler(BaseAudioHandler):
"""MiMo TTS 接口实现:挂载在 Chat Completions 端点上"""
def prepare_speech_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: SpeechRequest,
) -> RequestData:
input_text = request.input_text
config = request.config or TTSConfig()
config_voice = (
config.mimo_options.voice_id if hasattr(config, "mimo_options") else None
)
voice = (
config_voice
or request.voice
or identity.capabilities.default_voice_id
or "mimo_default"
)
url = adapter.get_api_url(identity, "/v1/chat/completions")
headers = adapter.get_base_headers(api_key)
body = {
"model": identity.model_name,
"messages": [{"role": "assistant", "content": input_text}],
"audio": {
"format": config.response_format
if config.response_format in ("wav", "pcm16")
else "wav",
"voice": voice,
},
}
return RequestData(url=url, headers=headers, body=body)
async def parse_speech_response(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
raw_response: httpx.Response,
) -> AudioResponse:
data = json.loads(await raw_response.aread())
adapter.validate_response(data)
audio_b64 = data["choices"][0]["message"]["audio"]["data"]
return AudioResponse(
audio_bytes=base64.b64decode(audio_b64),
audio_format="wav",
usage=UsageInfo(),
model_name=identity.model_name,
)
File diff suppressed because it is too large Load Diff
+129
View File
@@ -0,0 +1,129 @@
from zhenxun.services.ai.core.messages import EmbeddingRequest
from zhenxun.services.ai.core.models import ModelIdentity
from zhenxun.services.ai.core.options import LLMEmbeddingConfig
from zhenxun.services.ai.llm.adapters.base import BaseAdapter, RequestData
from zhenxun.services.ai.llm.adapters.handlers.openai_handlers import (
OpenAIEmbeddingHandler,
OpenAIRerankHandler,
)
from zhenxun.services.ai.llm.adapters.openai import OpenAICompatAdapter
class JinaEmbeddingHandler(OpenAIEmbeddingHandler):
"""Jina 专属文本/多模态嵌入处理器"""
async def prepare_embedding_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: EmbeddingRequest,
) -> RequestData:
batch = request.batch
config = request.config or LLMEmbeddingConfig()
endpoint = getattr(adapter, "get_embedding_endpoint")(identity)
url = adapter.get_api_url(identity, endpoint)
headers = adapter.get_base_headers(api_key)
is_omni = "omni" in identity.model_name.lower()
inputs_payload = []
if is_omni:
from zhenxun.services.ai.core.messages import (
AudioPart,
FilePart,
ImagePart,
TextPart,
VideoPart,
)
from zhenxun.services.log import logger
for payload in batch.payloads:
jina_content = []
for part in payload.parts:
if isinstance(part, TextPart):
jina_content.append({"text": part.text})
elif isinstance(part, ImagePart):
if part.url:
jina_content.append({"image": part.url})
else:
jina_content.append(
{"image": await part.get_data_uri("image/png")}
)
elif isinstance(part, AudioPart):
if part.url:
jina_content.append({"audio": part.url})
else:
jina_content.append(
{"audio": await part.get_data_uri("audio/mp3")}
)
elif isinstance(part, VideoPart):
if part.url:
jina_content.append({"video": part.url})
else:
jina_content.append(
{"video": await part.get_data_uri("video/mp4")}
)
elif isinstance(part, FilePart):
logger.warning(
f"Jina 暂不明确支持 Base64 内联 "
f"{type(part).__name__},已忽略。"
)
if not jina_content:
jina_content.append({"text": " "})
inputs_payload.append({"content": jina_content})
else:
inputs_payload = batch.to_text_only(
f"{identity.model_name} (API: {adapter.api_type})"
)
body = {
"model": identity.model_name,
"input": inputs_payload,
}
if config.output_dimensionality:
body["dimensions"] = config.output_dimensionality
if config.task_type:
task_mapping = {
"RETRIEVAL_QUERY": "retrieval.query",
"RETRIEVAL_DOCUMENT": "retrieval.passage",
"SEMANTIC_SIMILARITY": "text-matching",
"CLASSIFICATION": "classification",
"CLUSTERING": "clustering",
}
body["task"] = task_mapping.get(config.task_type, config.task_type)
if config.encoding_format and config.encoding_format != "float":
body["embedding_type"] = config.encoding_format
return RequestData(url=url, headers=headers, body=body)
class JinaAdapter(OpenAICompatAdapter):
"""Jina API 专有适配器 (仅支持 Embedding 和 Rerank)"""
def __init__(self):
super().__init__()
self.text_handler = None
self.embedding_handler = JinaEmbeddingHandler()
self.rerank_handler = OpenAIRerankHandler()
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "jina"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["jina"]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
raise NotImplementedError("Jina API 专精于检索,暂不支持常规对话生成。")
def get_embedding_endpoint(self, identity: ModelIdentity) -> str:
return "/v1/embeddings"
+30
View File
@@ -0,0 +1,30 @@
from zhenxun.services.ai.core.models import ModelIdentity
from zhenxun.services.ai.llm.adapters.handlers.mimo_handlers import (
MiMoAudioHandler,
MiMoTextHandler,
)
from zhenxun.services.ai.llm.adapters.handlers.openai_handlers import (
OpenAIImageHandler,
)
from zhenxun.services.ai.llm.adapters.openai import OpenAICompatAdapter
class MiMoAdapter(OpenAICompatAdapter):
"""小米 MiMo 大模型专有适配器"""
def __init__(self):
super().__init__()
self.text_handler = MiMoTextHandler(api_type=self.api_type)
self.image_handler = OpenAIImageHandler()
self.audio_handler = MiMoAudioHandler()
@property
def api_type(self) -> str:
return "mimo"
@property
def supported_api_types(self) -> list[str]:
return ["mimo"]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
return "/v1/chat/completions"
+243
View File
@@ -0,0 +1,243 @@
from __future__ import annotations
import json
from typing import Any
import httpx
from zhenxun.services.ai.core.exceptions import ResponseParseException
from zhenxun.services.ai.core.messages import (
AssistantMessage,
AudioResponse,
LLMMessage,
SpeechRequest,
ThoughtPart,
)
from zhenxun.services.ai.core.models import (
ModelCapabilities,
ModelDetail,
ModelIdentity,
)
from zhenxun.services.ai.core.options import GenerationConfig, TTSConfig
from zhenxun.services.ai.llm.adapters.base import BaseAdapter, RequestData
from zhenxun.services.ai.llm.adapters.handlers.base import BaseAudioHandler
from .handlers.openai_handlers import (
OpenAIConfigMapper,
OpenAIMessageConverter,
OpenAITextHandler,
)
from .openai import OpenAICompatAdapter
class MiniMaxAudioHandler(BaseAudioHandler):
"""MiniMax 专有文本转语音处理器"""
def prepare_speech_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: SpeechRequest,
) -> RequestData:
input_text = request.input_text
config = request.config or TTSConfig()
config_voice = config.minimax_options.voice_id
voice = (
config_voice
or request.voice
or identity.capabilities.default_voice_id
or "female-shaonv"
)
endpoint = "/v1/t2a_v2"
base_url = (
identity.api_base.rstrip("/")
if identity.api_base
else "https://api.minimaxi.com"
)
if base_url.endswith("/v1"):
base_url = base_url[:-3]
url = f"{base_url}{endpoint}"
headers = adapter.get_base_headers(api_key)
voice_setting: dict[str, Any] = {"voice_id": voice}
if config.speed != 1.0:
voice_setting["speed"] = config.speed
if config.minimax_options.vol is not None:
voice_setting["vol"] = config.minimax_options.vol
if config.minimax_options.pitch is not None:
voice_setting["pitch"] = config.minimax_options.pitch
if config.minimax_options.emotion is not None:
voice_setting["emotion"] = config.minimax_options.emotion
target_format = config.response_format
if target_format not in ("mp3", "pcm", "flac", "wav"):
target_format = "mp3"
audio_setting = {
"sample_rate": 32000,
"bitrate": 128000,
"format": target_format,
"channel": 1,
}
body: dict[str, Any] = {
"model": identity.model_name,
"text": input_text,
"stream": False,
"voice_setting": voice_setting,
"audio_setting": audio_setting,
}
if config.minimax_options.timbre_weights:
body["timbre_weights"] = config.minimax_options.timbre_weights
body["voice_setting"]["voice_id"] = ""
if config.minimax_options.pronunciation_dict:
body["pronunciation_dict"] = config.minimax_options.pronunciation_dict
return RequestData(url=url, headers=headers, body=body)
async def parse_speech_response(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
raw_response: httpx.Response,
) -> AudioResponse:
resp_bytes = await raw_response.aread()
data = json.loads(resp_bytes)
base_resp = data.get("base_resp", {})
if base_resp.get("status_code", 0) != 0:
raise ResponseParseException(
f"MiniMax 语音合成失败: {base_resp.get('status_msg')}",
details=data,
)
try:
audio_hex = data["data"]["audio"]
audio_bytes = bytes.fromhex(audio_hex)
except (KeyError, ValueError) as e:
raise ResponseParseException(
f"解析 MiniMax 语音 Hex 数据失败: {e}", details=data
)
extra_info = data.get("extra_info", {})
audio_format = extra_info.get("audio_format", "mp3")
usage_chars = extra_info.get("usage_characters", 0)
from zhenxun.services.ai.core.messages import AudioResponse, UsageInfo
usage = UsageInfo()
usage.prompt_tokens = usage_chars
return AudioResponse(
audio_bytes=audio_bytes,
audio_format=audio_format,
usage=usage,
model_name=identity.model_name,
raw_response=data,
)
class MiniMaxMessageConverter(OpenAIMessageConverter):
"""MiniMax 消息转换器,处理特有的 reasoning_details 格式回传"""
async def convert_messages_async(
self, messages: list[LLMMessage]
) -> list[dict[str, Any]]:
openai_messages = await super().convert_messages_async(messages)
assistant_msgs = [m for m in messages if isinstance(m, AssistantMessage)]
ast_idx = 0
for o_msg in openai_messages:
if o_msg.get("role") == "assistant":
if ast_idx < len(assistant_msgs):
orig_ast = assistant_msgs[ast_idx]
ast_idx += 1
if "reasoning_content" in o_msg:
del o_msg["reasoning_content"]
thought_parts = [
p for p in orig_ast.content if isinstance(p, ThoughtPart)
]
if thought_parts:
part = thought_parts[0]
raw_details = (
part.metadata.get("raw_reasoning_details")
if part.metadata
else None
)
if raw_details:
o_msg["reasoning_details"] = raw_details
else:
o_msg["reasoning_details"] = [
{"type": "reasoning.text", "text": part.thought_text}
]
return openai_messages
class MiniMaxConfigMapper(OpenAIConfigMapper):
"""MiniMax 专属配置映射器"""
def map_config(
self,
config: GenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
params = super().map_config(config, model_detail, capabilities)
params["reasoning_split"] = True
return params
class MiniMaxTextHandler(OpenAITextHandler):
"""MiniMax 复合文本处理器"""
def __init__(self, api_type: str = "minimax"):
super().__init__(api_type=api_type)
self.converter = MiniMaxMessageConverter(api_type=api_type)
self.mapper = MiniMaxConfigMapper(api_type=api_type)
class MiniMaxAdapter(OpenAICompatAdapter):
"""
MiniMax API 适配器。
"""
def __init__(self):
"""初始化 MiniMax 适配器并挂载专属处理器。"""
super().__init__()
self.text_handler = MiniMaxTextHandler(api_type=self.api_type)
self.audio_handler = MiniMaxAudioHandler()
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "minimax"
@property
def supported_api_types(self) -> list[str]:
"""当前适配器支持的 API 类型列表。"""
return ["minimax"]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
"""根据官方兼容要求,重写获取端点,允许自定义覆盖"""
return "/v1/chat/completions"
def _get_base_url(self, identity: ModelIdentity) -> str:
base_url = (
identity.api_base.rstrip("/")
if identity.api_base
else "https://api.minimaxi.com"
)
if base_url.endswith("/v1"):
base_url = base_url[:-3]
return base_url
+122
View File
@@ -0,0 +1,122 @@
"""
OpenAI API 适配器
支持 OpenAI、智谱AI 等 OpenAI 兼容的 API 服务。
"""
from __future__ import annotations
from abc import abstractmethod
from zhenxun.services.ai.core.models import ModelIdentity
from .base import (
BaseAdapter,
RequestData,
)
from .handlers.openai_handlers import (
CompositeOpenAITextHandler,
OpenAIAudioHandler,
OpenAIEmbeddingHandler,
OpenAIImageHandler,
OpenAIRerankHandler,
)
class OpenAICompatAdapter(BaseAdapter):
"""
OpenAI 兼容 API 适配器基类。
保留端点获取等基础逻辑。
"""
@property
def log_sanitization_context(self) -> str:
"""返回 OpenAI 系列请求的日志清洗上下文。"""
return "openai_request"
@abstractmethod
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
"""子类必须实现,返回 chat completions 的端点"""
pass
def get_embedding_endpoint(self, identity: ModelIdentity) -> str:
"""返回 embeddings 的默认端点"""
return "/v1/embeddings"
async def prepare_simple_request(
self,
identity: ModelIdentity,
api_key: str,
prompt: str,
history: list[dict[str, str]] | None = None,
) -> RequestData:
"""准备简单文本生成请求"""
from zhenxun.services.ai.core.messages import (
AssistantMessage,
SystemMessage,
TextPart,
UserMessage,
)
messages = []
if history:
for msg in history:
role = msg.get("role", "user")
content = msg.get("content", "")
if role == "system":
messages.append(SystemMessage(content=[TextPart(text=content)]))
elif role == "assistant":
messages.append(AssistantMessage(content=[TextPart(text=content)]))
else:
messages.append(UserMessage(content=[TextPart(text=content)]))
messages.append(UserMessage(content=[TextPart(text=prompt)]))
config = identity.generation_config
from zhenxun.services.ai.core.messages import ChatRequest
request = ChatRequest(messages=messages, config=config)
return await self.prepare_advanced_request(
identity=identity,
api_key=api_key,
request=request,
)
class OpenAIAdapter(OpenAICompatAdapter):
"""OpenAI 系列适配器,统一装配文本/图像/嵌入/重排处理链。"""
def __init__(self):
"""初始化并挂载复合文本处理器与通用多模态处理器。"""
super().__init__()
self.text_handler = CompositeOpenAITextHandler(api_type=self.api_type)
self.image_handler = OpenAIImageHandler()
self.embedding_handler = OpenAIEmbeddingHandler()
self.rerank_handler = OpenAIRerankHandler()
self.audio_handler = OpenAIAudioHandler()
@property
def api_type(self) -> str:
"""适配器主类型标识。"""
return "openai"
@property
def supported_api_types(self) -> list[str]:
"""支持的 API 类型及别名。"""
return [
"openai",
"openai_responses",
]
def get_chat_endpoint(self, identity: ModelIdentity) -> str:
"""返回聊天完成端点"""
current_api_type = identity.api_type
if current_api_type == "openai_responses":
return "/v1/responses"
if current_api_type == "doubao":
return "/api/v3/chat/completions"
return "/v1/chat/completions"
def get_embedding_endpoint(self, identity: ModelIdentity) -> str:
"""返回嵌入端点。"""
return "/v1/embeddings"
@@ -0,0 +1,202 @@
import base64
from pathlib import Path
from typing import Any
from zhenxun.services.ai.core.exceptions import LLMException
from zhenxun.services.ai.core.messages import (
ImagePart,
ImageRequest,
LLMMessage,
ThoughtPart,
)
from zhenxun.services.ai.core.models import ModelIdentity
from zhenxun.services.ai.llm.adapters.base import (
BaseAdapter,
RequestData,
ResponseData,
process_image_data,
)
from zhenxun.services.ai.llm.adapters.handlers.base import BaseImageHandler
from zhenxun.services.ai.llm.adapters.handlers.openai_handlers import (
CompositeOpenAITextHandler,
OpenAIMessageConverter,
)
from zhenxun.services.ai.llm.adapters.openai import OpenAIAdapter
class OpenRouterMessageConverter(OpenAIMessageConverter):
"""OpenRouter 专有消息转换器:处理 reasoning_details 的无损回传"""
async def convert_messages_async(
self, messages: list[LLMMessage]
) -> list[dict[str, Any]]:
openai_messages = await super().convert_messages_async(messages)
assistant_msgs = [m for m in messages if getattr(m, "role", "") == "assistant"]
ast_idx = 0
for o_msg in openai_messages:
if o_msg.get("role") == "assistant":
if ast_idx < len(assistant_msgs):
orig_ast = assistant_msgs[ast_idx]
ast_idx += 1
thought_parts = [
p for p in orig_ast.content if isinstance(p, ThoughtPart)
]
if thought_parts:
part = thought_parts[0]
raw_details = (
part.metadata.get("raw_reasoning_details")
if part.metadata
else None
)
if raw_details:
o_msg["reasoning_details"] = raw_details
o_msg.pop("reasoning_content", None)
o_msg.pop("reasoning", None)
return openai_messages
class OpenRouterTextHandler(CompositeOpenAITextHandler):
"""OpenRouter 专有文本处理器,挂载专有 Converter"""
def __init__(self, api_type: str = "openrouter"):
super().__init__(api_type=api_type)
self._standard_handler.converter = OpenRouterMessageConverter(api_type=api_type)
class OpenRouterImageHandler(BaseImageHandler):
"""OpenRouter 专有的图像生成处理器"""
def prepare_image_request(
self,
adapter: BaseAdapter,
identity: ModelIdentity,
api_key: str,
request: ImageRequest,
) -> RequestData:
headers = adapter.get_base_headers(api_key)
endpoint = "/v1/chat/completions"
url = adapter.get_api_url(identity, endpoint)
body: dict[str, Any] = {
"model": identity.model_name,
"modalities": ["image", "text"],
}
if request.images:
content_list: list[dict[str, Any]] = [
{"type": "text", "text": request.prompt}
]
for img_source in request.images:
img_bytes = None
if isinstance(img_source, bytes):
img_bytes = img_source
elif hasattr(img_source, "read_bytes"):
img_bytes = img_source.read_bytes()
elif isinstance(img_source, str) and img_source.startswith(
"data:image"
):
content_list.append(
{"type": "image_url", "image_url": {"url": img_source}}
)
continue
else:
raise LLMException(
"OpenRouter 图像生成仅支持 bytes/Path/base64 URI"
)
if img_bytes:
mime_type = "image/jpeg"
if img_bytes.startswith(b"\x89PNG\r\n\x1a\n"):
mime_type = "image/png"
elif img_bytes.startswith(b"GIF87a") or img_bytes.startswith(
b"GIF89a"
):
mime_type = "image/gif"
elif img_bytes.startswith(b"RIFF") and img_bytes[8:12] == b"WEBP":
mime_type = "image/webp"
b64_str = base64.b64encode(img_bytes).decode("utf-8")
content_list.append(
{
"type": "image_url",
"image_url": {"url": f"data:{mime_type};base64,{b64_str}"},
}
)
body["messages"] = [{"role": "user", "content": content_list}]
else:
body["messages"] = [{"role": "user", "content": request.prompt}]
if request.config:
image_config = {}
if request.config.media.aspect_ratio:
image_config["aspect_ratio"] = str(request.config.media.aspect_ratio)
if request.config.media.resolution:
image_config["image_size"] = str(
request.config.media.resolution
).upper()
if image_config:
body["image_config"] = image_config
return RequestData(url=url, headers=headers, body=body)
def parse_image_response(
self, adapter: BaseAdapter, response_json: dict[str, Any]
) -> ResponseData:
adapter.validate_response(response_json)
images_data = []
choices = response_json.get("choices", [])
if choices:
message = choices[0].get("message", {})
if "images" in message:
for img_data in message["images"]:
img_url_obj = img_data.get("image_url", {})
url_str = img_url_obj.get("url", "")
if url_str.startswith("data:image"):
try:
b64_data = url_str.split(",", 1)[1]
decoded = base64.b64decode(b64_data)
images_data.append(process_image_data(decoded))
except Exception:
pass
elif url_str:
images_data.append(url_str)
content_parts = []
for img in images_data:
if isinstance(img, str) and img.startswith("http"):
content_parts.append(ImagePart(url=img))
elif isinstance(img, bytes):
content_parts.append(ImagePart(raw=img))
else:
content_parts.append(ImagePart(path=Path(img)))
if not content_parts:
raise LLMException("OpenRouter 图像生成响应中未找到有效的图片数据")
return ResponseData(content_parts=content_parts, raw_response=response_json)
class OpenRouterAdapter(OpenAIAdapter):
"""OpenRouter 平台适配器"""
def __init__(self):
super().__init__()
self.text_handler = OpenRouterTextHandler(api_type=self.api_type)
self.image_handler = OpenRouterImageHandler()
@property
def api_type(self) -> str:
return "openrouter"
@property
def supported_api_types(self) -> list[str]:
return ["openrouter"]
+526
View File
@@ -0,0 +1,526 @@
"""
LLM 服务的高级 API 接口 - 便捷函数入口 (无状态)
"""
from pathlib import Path
from typing import Any, Literal, TypeVar, overload
from pydantic import BaseModel
from zhenxun.services.ai.core.exceptions import (
LLMException,
UpstreamServerException,
get_user_friendly_error_message,
)
from zhenxun.services.ai.core.messages import (
AudioResponse,
ChatRequest,
ChatResponse,
EmbeddingRequest,
EmbeddingResponse,
ImageRequest,
ImageResponse,
LLMMessage,
PromptInput,
RerankRequest,
RerankResult,
SpeechRequest,
)
from zhenxun.services.ai.core.models import ModelName
from zhenxun.services.ai.core.options import (
GenerationConfig,
LLMEmbeddingConfig,
TTSConfig,
)
from zhenxun.services.ai.guardrails import GuardrailSource
from zhenxun.services.ai.llm.engine.router import LLMOrchestrator
from zhenxun.services.log import logger
from .builder import IntentBuilder
T = TypeVar("T", bound=BaseModel)
async def chat(
message: PromptInput | list[LLMMessage],
*,
model: ModelName = None,
instruction: str | None = None,
config: GenerationConfig | IntentBuilder | None = None,
timeout: float | None = None,
) -> ChatResponse:
"""
无状态的聊天对话便捷函数,单次执行后立即销毁上下文。
示例:
response = await chat("你好", model="OpenAI/gpt-4o", instruction="你是一个助手")
print(response.text)
参数:
message: 用户输入的消息内容,支持多种格式。
model: 要使用的模型名称,如果为None则使用默认模型。
instruction: 系统指令,用于指导AI的行为和回复风格。
config: (可选) 配置构建器 IntentBuilder 或 GenerationConfig 对象。
timeout: (可选) HTTP 请求超时时间(秒)。
返回:
ChatResponse: 包含AI回复内容、使用信息和工具调用等的完整响应对象。
异常:
LLMException: 当网络超时、模型不存在或 API 返回错误时抛出,建议外层捕获。
"""
try:
from zhenxun.services.ai.message_builder import MessageBuilder
messages = await MessageBuilder.normalize_to_llm_messages(
message, instruction=instruction
)
return await generate(
messages=messages,
model=model,
config=config,
timeout=timeout,
)
except LLMException as e:
raise e.with_traceback(None) from None
except Exception as e:
friendly_msg = get_user_friendly_error_message(e)
logger.error(f"执行 chat 函数失败: {e} | 建议: {friendly_msg}", e=e)
raise LLMException(f"聊天执行失败: {friendly_msg}").with_traceback(
None
) from None
@overload
async def embed(
input_batch: PromptInput,
*,
model: ModelName = None,
task: Literal[
"general", "query", "document", "similarity", "classification", "clustering"
] = "general",
dimensions: int | None = None,
multimodal: bool | list[str] = False,
config: LLMEmbeddingConfig | None = None,
) -> EmbeddingResponse: ...
@overload
async def embed(
input_batch: list[Any],
*,
model: ModelName = None,
task: Literal[
"general", "query", "document", "similarity", "classification", "clustering"
] = "general",
dimensions: int | None = None,
multimodal: bool | list[str] = False,
config: LLMEmbeddingConfig | None = None,
) -> EmbeddingResponse: ...
async def embed(
input_batch: PromptInput | list[Any],
*,
model: ModelName = None,
task: Literal[
"general", "query", "document", "similarity", "classification", "clustering"
] = "general",
dimensions: int | None = None,
multimodal: bool | list[str] = False,
config: LLMEmbeddingConfig | None = None,
) -> EmbeddingResponse:
"""
无状态的向量嵌入便捷函数,支持文本批量与图文多模态融合 (Fused Embeddings)。
参数:
input_batch: 要生成嵌入的内容。传入单条字符串/消息视为单向量;传入多条视为批量。
model: 要使用的嵌入模型名称,如果为None则使用默认模型。
task: 生成意图
(query检索词 / document目标文档 / similarity相似度 等),将自动翻译到底层。
dimensions: 强制降低返回的向量维度 (降维)。
multimodal: 是否开启多模态嵌入提取。默认 False (极速安全的纯文本模式)。
config: 嵌入配置对象。
返回:
EmbeddingResponse: 包含向量和 Token 消耗统计的富响应对象。
"""
final_config = config or LLMEmbeddingConfig()
if multimodal is not False:
final_config.multimodal = multimodal
from zhenxun.services.ai.message_builder import MessageBuilder
batch = await MessageBuilder.normalize_to_embed_batch(
input_batch, config=final_config
)
if not batch.payloads:
from zhenxun.services.ai.core.messages import UsageInfo
return EmbeddingResponse(
embeddings=[], usage=UsageInfo(), model_name=str(model)
)
if dimensions is not None:
final_config.output_dimensionality = dimensions
if task != "general":
task_map = {
"query": "RETRIEVAL_QUERY",
"document": "RETRIEVAL_DOCUMENT",
"similarity": "SEMANTIC_SIMILARITY",
"classification": "CLASSIFICATION",
"clustering": "CLUSTERING",
}
final_config.task_type = task_map.get(task)
try:
request = EmbeddingRequest(batch=batch, config=final_config)
return await LLMOrchestrator.invoke(request, model_name=model, task="embedding")
except LLMException as e:
raise e.with_traceback(None) from None
except Exception as e:
friendly_msg = get_user_friendly_error_message(e)
logger.error(f"文本嵌入失败: {e} | 建议: {friendly_msg}", e=e)
raise UpstreamServerException(
f"文本嵌入失败: {friendly_msg}",
cause=e,
).with_traceback(None) from None
async def rerank(
query: str,
documents: list[str | dict[str, str]],
top_n: int = 3,
*,
model: ModelName = None,
) -> list[RerankResult]:
"""
无状态的文本重排便捷函数。
参数:
query: 用户查询问题
documents: 候选文档列表 (支持纯文本或 {"image": "url", "text": "xxx"} 图文格式)
top_n: 返回匹配度最高的前 n 个文档
model: 重排模型名称 (如 BAAI/bge-reranker-v2-m3)
"""
try:
request = RerankRequest(query=query, documents=documents, top_n=top_n)
response = await LLMOrchestrator.invoke(
request, model_name=model, task="rerank"
)
return response.results
except Exception as e:
friendly_msg = get_user_friendly_error_message(e)
logger.error(f"文档重排失败: {e} | 建议: {friendly_msg}", e=e)
raise LLMException(f"文档重排失败: {friendly_msg}").with_traceback(
None
) from None
async def generate_structured(
message: PromptInput | list[LLMMessage],
response_model: type[T],
*,
guardrails: list[GuardrailSource] | None = None,
model: ModelName = None,
config: GenerationConfig | IntentBuilder | None = None,
max_retries: int | None = None,
error_prompt_template: str | None = None,
instruction: str | None = None,
timeout: float | None = None,
) -> T:
"""
请求大模型生成结构化数据,并自动验证/解析为指定的 Pydantic 模型。
示例:
class UserInfo(BaseModel):
name: str
info = await generate_structured("提取张三的信息", response_model=UserInfo)
参数:
message: 输入的消息内容,支持纯文本、UniMessage、消息对象列表等。
response_model: 目标结构化输出的强类型 Pydantic 模型类。
guardrails: 护栏来源列表,支持自然语言规则、自定义校验函数
model: 强制指定调用的模型路由或名称,若为空则使用默认模型。
config: 大模型生成的通用配置或意图构建器。
max_retries: 格式解析或护栏校验失败时的最大自我反思重试次数(IVR),若为空则使用全局配置。
error_prompt_template: 自定义校验失败时引导大模型自我修正的提示词模板。
instruction: 注入到系统提示词中的全局任务指令或前置设定。
timeout: 本次 API 请求的超时时间限制(秒)。
返回:
T: 解析验证通过后的 Pydantic 模型实例。
""" # noqa: E501
try:
from zhenxun.services.ai.config import get_llm_config
from zhenxun.services.ai.core.engine.structured_parser import (
BaseOutputProcessor,
)
from zhenxun.services.ai.core.options import (
OutputFormatConfig,
ResponseFormat,
StructuredOutputStrategy,
)
if max_retries is None:
max_retries = get_llm_config().client_settings.structured_retries
from zhenxun.services.ai.guardrails import parse_guardrails
parsed_guardrails = parse_guardrails(guardrails)
output_processor = BaseOutputProcessor(
response_model=response_model,
error_template=error_prompt_template,
)
json_schema = output_processor.get_json_schema()
structured_config = GenerationConfig(
output=OutputFormatConfig(
response_format=ResponseFormat.JSON,
response_schema=json_schema,
structured_output_strategy=StructuredOutputStrategy.NATIVE,
)
)
prompt_parts: list[str] = []
if instruction:
prompt_parts.append(instruction)
import json
schema_str = json.dumps(json_schema, ensure_ascii=False, indent=2)
prompt_parts.append(
"### ⚠️ [结构化输出要求]\n"
"请严格按照以下 JSON Schema 格式进行回复,禁止包含任何额外纯文本解释:\n"
f"```json\n{schema_str}\n```"
)
system_prompt = "\n\n".join(prompt_parts) if prompt_parts else None
from zhenxun.services.ai.message_builder import MessageBuilder
messages = await MessageBuilder.normalize_to_llm_messages(
message if message is not None else [], instruction=system_prompt
)
if isinstance(config, IntentBuilder):
config = config.build()
final_config = (
structured_config.merge_with(config) if config else structured_config
)
from zhenxun.services.ai.capabilities.builtin import (
ReflexionCapability,
)
extra_context = {
"output_processor": output_processor,
"guardrails": parsed_guardrails,
"max_retries": max_retries,
"__sys_capabilities": [ReflexionCapability()],
}
response = await generate(
messages=messages,
model=model,
config=final_config,
timeout=timeout,
extra=extra_context,
)
if not hasattr(response, "parsed_obj") or response.parsed_obj is None:
raise LLMException("结构化输出失败:中间件未返回解析后的对象。")
return response.parsed_obj
except LLMException as e:
raise e.with_traceback(None) from None
except Exception as e:
friendly_msg = get_user_friendly_error_message(e)
logger.error(f"生成结构化响应失败: {e} | 建议: {friendly_msg}", e=e)
raise LLMException(f"生成结构化响应失败: {friendly_msg}").with_traceback(
None
) from None
async def generate(
messages: list[LLMMessage],
*,
model: ModelName = None,
config: GenerationConfig | IntentBuilder | None = None,
timeout: float | None = None,
extra: dict[str, Any] | None = None,
) -> ChatResponse:
"""
[内部 API/高级用法] 直接传入底层消息实体列表生成响应。一般业务插件推荐使用 `chat`。
参数:
messages: 完整的消息历史列表,包括系统指令、用户消息和助手回复。
model: 要使用的模型名称,如果为None则使用默认模型。
config: (可选) 生成配置对象,将与默认配置合并后传递。
返回:
ChatResponse: 包含AI回复内容、使用信息和工具调用等的完整响应对象。
"""
try:
resolved_config: GenerationConfig | None = None
if isinstance(config, IntentBuilder):
resolved_config = config.build()
else:
resolved_config = config
request = ChatRequest(
messages=messages,
config=resolved_config,
timeout=timeout,
extra=extra or {},
)
sys_caps = request.extra.pop("__sys_capabilities", [])
run_ctx = request.extra.pop("run_context", None)
if sys_caps:
from zhenxun.services.ai.capabilities import CombinedCapability
from zhenxun.services.ai.core.models import LLMContext
from zhenxun.services.ai.run import RunContext
run_context = run_ctx or RunContext()
llm_context = LLMContext(request=request)
combined_cap = CombinedCapability(sys_caps)
async def inner_handler(ctx: LLMContext[Any, Any]) -> ChatResponse:
return await LLMOrchestrator.invoke(
ctx.request,
model_name=model,
task="chat",
override_config=resolved_config,
)
return await combined_cap.wrap_model_request(
run_context, llm_context, inner_handler
)
else:
return await LLMOrchestrator.invoke(
request, model_name=model, task="chat", override_config=resolved_config
)
except LLMException as e:
raise e.with_traceback(None) from None
except Exception as e:
friendly_msg = get_user_friendly_error_message(e)
logger.error(f"生成响应失败: {e} | 建议: {friendly_msg}", e=e)
raise LLMException(f"生成响应失败: {friendly_msg}").with_traceback(
None
) from None
@overload
async def create_image(
prompt: str | Any,
*,
images: None = None,
model: ModelName = None,
config: GenerationConfig | IntentBuilder | None = None,
) -> ImageResponse:
"""根据文本提示生成一张新图片。"""
...
@overload
async def create_image(
prompt: str | Any,
*,
images: list[Path | bytes | str] | Path | bytes | str,
model: ModelName = None,
config: GenerationConfig | IntentBuilder | None = None,
) -> ImageResponse:
"""在给定图片的基础上,根据文本提示进行编辑或重新生成。"""
...
async def create_image(
prompt: str | Any,
*,
images: list[Path | bytes | str] | Path | bytes | str | None = None,
model: ModelName = None,
config: GenerationConfig | IntentBuilder | None = None,
) -> ImageResponse:
"""
多模态图片生成/编辑函数。
示例:
res = await create_image("画一只猫", model="OpenAI/dall-e-3")
img_bytes = res.images[0]
说明:
- 若 `images` 为 None,执行文本生成图片 (Text-to-Image)。
- 若提供 `images`,执行图像编辑 (Image-to-Image)。
"""
text_prompt = getattr(prompt, "extract_plain_text", lambda: str(prompt))()
image_list = []
if images:
if isinstance(images, list):
image_list.extend(images)
else:
image_list.append(images)
if isinstance(config, IntentBuilder):
config = config.build()
config = config or GenerationConfig()
try:
request = ImageRequest(
prompt=text_prompt,
images=image_list if image_list else None,
config=config,
)
return await LLMOrchestrator.invoke(
request, model_name=model, task="image", override_config=config
)
except LLMException as e:
raise e.with_traceback(None) from None
except Exception as e:
friendly_msg = get_user_friendly_error_message(e)
logger.error(f"图片生成执行发生未知错误: {e} | 建议: {friendly_msg}", e=e)
raise LLMException(f"图片生成失败: {friendly_msg}").with_traceback(
None
) from None
async def create_speech(
text: str,
voice: str | None = None,
*,
model: ModelName = None,
config: TTSConfig | None = None,
) -> AudioResponse:
"""
通用文本转语音便捷函数。
参数:
text: 待合成的文本内容。
voice: 快捷音色指定,若为空则自动使用目标模型的缺省最优音色。
model: 指定生成语音的模型名称。
config: 语音生成的额外设置。
示例:
res = await create_speech("你好,世界", voice="alloy", model="OpenAI/tts-1")
Path("out.mp3").write_bytes(res.audio_bytes)
"""
if not text:
raise LLMException("TTS 输入文本不能为空")
try:
request = SpeechRequest(input_text=text, voice=voice, config=config)
return await LLMOrchestrator.invoke(request, model_name=model, task="tts")
except LLMException as e:
raise e.with_traceback(None) from None
except Exception as e:
friendly_msg = get_user_friendly_error_message(e)
logger.error(f"语音生成执行发生未知错误: {e} | 建议: {friendly_msg}", e=e)
raise LLMException(f"语音生成失败: {friendly_msg}").with_traceback(
None
) from None
+228
View File
@@ -0,0 +1,228 @@
"""
LLM 生成配置相关类和函数
"""
from typing import Any, Literal
from typing_extensions import Self
from zhenxun.services.ai.core.exceptions import ConfigurationException
from zhenxun.services.ai.core.options import (
GenerationConfig,
ResponseFormat,
)
from zhenxun.services.log import logger
from zhenxun.utils.pydantic_compat import model_json_schema, model_validate
class GeminiIntentNamespace:
"""Gemini 专属高级参数构建域"""
def __init__(self, builder: "IntentBuilder"):
self._builder = builder
def set_safety_threshold(self, threshold: str) -> "IntentBuilder":
"""强制设置 Gemini 安全阈值 (如 BLOCK_NONE, BLOCK_ONLY_HIGH)"""
self._builder._config.gemini_options.safety_settings = {
"HARM_CATEGORY_HARASSMENT": threshold,
"HARM_CATEGORY_HATE_SPEECH": threshold,
"HARM_CATEGORY_SEXUALLY_EXPLICIT": threshold,
"HARM_CATEGORY_DANGEROUS_CONTENT": threshold,
}
return self._builder
class OpenAIIntentNamespace:
"""OpenAI 专属高级参数构建域"""
def __init__(self, builder: "IntentBuilder"):
self._builder = builder
def enable_server_storage(self, store: bool = True) -> "IntentBuilder":
"""设置是否在 OpenAI 服务端留存请求记录"""
self._builder._config.openai_options.store = store
return self._builder
class DeepSeekIntentNamespace:
"""DeepSeek 专属高级参数构建域"""
def __init__(self, builder: "IntentBuilder"):
self._builder = builder
def disable_thinking(self) -> "IntentBuilder":
"""显式关闭 DeepSeek 的思维链"""
self._builder._config.deepseek_options.thinking = False
return self._builder
class IntentBuilder:
"""
基于能力意图声明的构建器 (Intent-Driven Builder)。
完全屏蔽底层厂商参数差异,面向开发者提供 Fluent API。
"""
def __init__(self):
self._config = GenerationConfig()
@property
def gemini(self) -> GeminiIntentNamespace:
return GeminiIntentNamespace(self)
@property
def openai(self) -> OpenAIIntentNamespace:
return OpenAIIntentNamespace(self)
@property
def deepseek(self) -> DeepSeekIntentNamespace:
return DeepSeekIntentNamespace(self)
def with_reasoning(self, level: str | None = None) -> Self:
"""
跨厂商统一的思考/推理等级意图声明。
自动向下转换为底层合法参数,并阻止不兼容模型的非法调用。
"""
if level:
self._config.common.reasoning_effort = level
if level.lower() != "none":
self._config.gemini_options.include_thoughts = True
self._config.deepseek_options.thinking = True
else:
self._config.deepseek_options.thinking = False
return self
def with_local_cache(self, ttl: int = 3600) -> Self:
"""
显式开启本次 LLM 网络请求的极速本地缓存。
对于相同模型、相同参数、相同 Prompt 的请求,将直接返回本地记忆,免去网络开销。
适用于 Embedding、确定性的结构化抽取或工作流节点。
"""
self._config.custom_kwargs["__cache_ttl__"] = ttl
return self
def with_json_output(self) -> Self:
"""
基础结构化意图:要求大模型输出通用 JSON 格式(不校验 Schema)。
"""
self._config.output.response_format = ResponseFormat.JSON
self._config.output.response_mime_type = "application/json"
self._config.output.structured_output_strategy = "native"
return self
def require_structured_output(self, schema: Any, strict: bool = True) -> Self:
"""
强制要求结构化输出意图。
支持自动处理 Pydantic 模型并转换为厂商所需的 JSON Schema。
"""
import inspect
from pydantic import BaseModel
from zhenxun.services.ai.core.options import StructuredOutputStrategy
self._config.output.response_format = ResponseFormat.JSON
self._config.output.response_mime_type = "application/json"
if schema:
if inspect.isclass(schema) and issubclass(schema, BaseModel):
self._config.output.response_schema = model_json_schema(schema)
else:
from typing import cast
self._config.output.response_schema = cast(dict[str, Any], schema)
if strict:
self._config.output.structured_output_strategy = (
StructuredOutputStrategy.NATIVE
)
return self
def config_core(
self,
temperature: float | None = None,
max_tokens: int | None = None,
top_p: float | None = None,
) -> Self:
"""
配置底层核心采样参数(如 temperature, max_tokens 等)。
"""
if temperature is not None:
self._config.common.temperature = temperature
if max_tokens is not None:
self._config.common.max_tokens = max_tokens
if top_p is not None:
self._config.common.top_p = top_p
return self
def with_safety_level(self, level: str = "moderate") -> Self:
"""
安全合规意图。
level 取值: 'strict' (最严格), 'moderate' (中等), 'none' (完全无限制)。
"""
from zhenxun.services.ai.config import get_gemini_safety_threshold
if level == "strict":
self.gemini.set_safety_threshold("BLOCK_LOW_AND_ABOVE")
elif level == "none":
self.gemini.set_safety_threshold("BLOCK_NONE")
else:
self.gemini.set_safety_threshold(get_gemini_safety_threshold())
return self
def with_image_generation_params(
self, aspect_ratio: str = "16:9", resolution: str = "1K"
) -> Self:
"""
生图意图:统一配置图像生成的比例与分辨率。
"""
self._config.media.aspect_ratio = aspect_ratio
self._config.media.resolution = resolution
return self
def with_vision_optimization(
self, quality: Literal["low", "medium", "high", "standard", "hd"] = "high"
) -> Self:
"""
视觉优化意图。
"""
self._config.media.quality = quality
return self
def with_provider_raw_kwargs(self, provider_name: str, **kwargs) -> Self:
"""厂商逃生舱:直接注入特有参数"""
provider_name = provider_name.lower()
if provider_name == "openai":
for k, v in kwargs.items():
setattr(self._config.openai_options, k, v)
elif provider_name == "gemini":
for k, v in kwargs.items():
setattr(self._config.gemini_options, k, v)
else:
self._config.custom_kwargs.update(kwargs)
return self
def build(self) -> GenerationConfig:
"""构建最终的配置对象"""
return self._config
def validate_override_params(
override_config: dict[str, Any] | GenerationConfig | None,
) -> GenerationConfig:
"""验证和标准化覆盖参数"""
if override_config is None:
return GenerationConfig()
if isinstance(override_config, GenerationConfig):
return override_config
if isinstance(override_config, dict):
try:
return model_validate(GenerationConfig, override_config)
except Exception as e:
logger.warning(f"覆盖配置参数验证失败: {e}")
raise ConfigurationException(
f"无效的覆盖配置参数: {e}",
cause=e,
)
raise ConfigurationException(
f"不支持的配置类型: {type(override_config)}",
)
@@ -0,0 +1,3 @@
"""
LLM 执行引擎模块。
"""
@@ -0,0 +1,641 @@
import asyncio
import hashlib
import json
import re
import time
from typing import Any, ClassVar, cast
from aiocache import SimpleMemoryCache
import httpx
from zhenxun.services.ai.core.exceptions import (
ConfigurationException,
LLMException,
NetworkTimeoutException,
UpstreamServerException,
)
from zhenxun.services.ai.core.messages import (
AudioPart,
AudioResponse,
ChatRequest,
ChatResponse,
EmbeddingRequest,
EmbeddingResponse,
FilePart,
ImagePart,
ImageRequest,
ImageResponse,
RerankRequest,
RerankResponse,
SpeechRequest,
TextPart,
VideoPart,
)
from zhenxun.services.ai.core.models import (
LLMContext,
ModelCapabilities,
ModelIdentity,
ModelModality,
)
from zhenxun.services.ai.core.options import (
GenerationConfig,
)
from zhenxun.services.ai.core.protocols.middleware import LLMMiddleware, NextCall
from zhenxun.services.ai.llm.adapters.base import (
BaseAdapter,
RequestData,
process_image_data,
)
from zhenxun.services.ai.llm.system.models import RetryConfig
from zhenxun.services.ai.llm.system.network import (
HealthManager,
LLMHttpClient,
)
from zhenxun.services.log import logger
from zhenxun.utils.http_utils import AsyncHttpx
from zhenxun.utils.log_sanitizer import sanitize_for_logging
from zhenxun.utils.pydantic_compat import (
dump_json_safely,
model_copy,
model_dump,
parse_as,
)
_LLM_API_CACHE = SimpleMemoryCache(namespace="zhenxun_llm_api_cache")
class MiddlewarePipeline:
"""中间件管线组装器"""
def __init__(self):
self.middlewares: list[LLMMiddleware] = []
def add_middleware(self, middleware: LLMMiddleware) -> None:
"""按顺序追加中间件,先加入的将处在调用链的最外层"""
self.middlewares.append(middleware)
def build(self, terminal_handler: NextCall[Any, Any]) -> NextCall[Any, Any]:
handler = terminal_handler
for middleware in reversed(self.middlewares):
def _wrap(
mw: LLMMiddleware[Any, Any], next_c: NextCall[Any, Any]
) -> NextCall[Any, Any]:
async def _handler(context: LLMContext[Any, Any]) -> Any:
return await mw(context, next_c)
return _handler
handler = _wrap(middleware, handler)
return handler
class LLMCacheMiddleware:
"""
大模型极速缓存中间件:
只在开发者显式配置了 __cache_ttl__ 时生效。
拦截高成本的 API 网络请求,直接返回本地缓存。
"""
_RESPONSE_TYPE_MAP: ClassVar[dict[type, type]] = {
ChatRequest: ChatResponse,
EmbeddingRequest: EmbeddingResponse,
ImageRequest: ImageResponse,
SpeechRequest: AudioResponse,
RerankRequest: RerankResponse,
}
def __init__(self, model_name: str):
self.model_name = model_name
def _generate_cache_key(self, context: LLMContext[Any, Any]) -> str:
"""构造绝对纯净的请求哈希键,剔除时间戳等干扰项"""
payload = {
"model": self.model_name,
"type": type(context.request).__name__,
"request": context.request.get_cache_hash_payload(),
}
json_str = json.dumps(payload, sort_keys=True, ensure_ascii=False, default=str)
return hashlib.md5(json_str.encode("utf-8")).hexdigest()
async def __call__(
self, context: LLMContext[Any, Any], next_call: NextCall[Any, Any]
) -> Any:
ttl = None
if hasattr(context.request, "config") and context.request.config:
ttl = getattr(context.request.config, "custom_kwargs", {}).get(
"__cache_ttl__"
)
if ttl is None:
return await next_call(context)
cache_key = self._generate_cache_key(context)
cached_data = await _LLM_API_CACHE.get(cache_key)
if cached_data is not None:
logger.debug(
f"⚡ [LLMCache] 命中本地极速缓存 - "
f"model: {self.model_name}, type: {type(context.request).__name__}"
)
response_type = self._RESPONSE_TYPE_MAP.get(type(context.request))
if response_type:
cached_resp = parse_as(response_type, cached_data)
else:
cached_resp = cached_data
if isinstance(cached_resp, ChatResponse):
cached_resp.usage_info = {
"is_cache_hit": True,
"total_tokens": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"promptTokenCount": 0,
"candidatesTokenCount": 0,
"totalTokenCount": 0,
}
return cached_resp
response = await next_call(context)
await _LLM_API_CACHE.set(cache_key, model_dump(response), ttl=ttl)
return response
class FailoverAndRetryMiddleware:
"""
故障转移与重试中间件:
结合了密钥轮询 (Key Selection) 与异常退避重试 (Retry) 逻辑。
"""
def __init__(
self,
retry_config: RetryConfig,
health_manager: HealthManager,
provider_name: str,
api_keys: list[str],
):
self.retry_config = retry_config
self.health_manager = health_manager
self.provider_name = provider_name
self.api_keys = api_keys
self._failed_keys: set[str] = set()
def _raise_with_masked_key(self, e: LLMException, api_key: str) -> None:
"""辅助函数:掩码 API Key 并原样抛出异常,防止密钥泄露"""
masked = f"{api_key[:8]}..." if api_key else "unknown"
if isinstance(e.details, dict):
e.details["api_key"] = masked
raise e.with_traceback(None) from None
async def __call__(
self, context: LLMContext[Any, Any], next_call: NextCall[Any, Any]
) -> Any:
last_exception: Exception | None = None
is_routed = context.request.extra.get("_is_routed_call", False)
max_retries = 0 if is_routed else self.retry_config.max_retries
total_attempts = max_retries + 1
for attempt in range(total_attempts):
selected_key = await self.health_manager.get_next_available_key(
self.provider_name,
self.api_keys,
exclude_keys=self._failed_keys,
strict_mode=is_routed,
)
if not selected_key:
raise ConfigurationException(
f"提供商 {self.provider_name} 无可用 API Key"
)
context.runtime_state["api_key"] = selected_key
context.runtime_state["provider_name"] = self.provider_name
try:
context.runtime_state["attempt"] = attempt + 1
return await next_call(context)
except LLMException as e:
last_exception = e
await self.health_manager.record_key_failure(
self.provider_name, selected_key, e
)
if e.should_rotate_key:
self._failed_keys.add(selected_key)
if not e.is_retryable:
self._raise_with_masked_key(e, selected_key)
if attempt == total_attempts - 1:
self._raise_with_masked_key(e, selected_key)
wait_time = self.retry_config.retry_delay
if self.retry_config.exponential_backoff:
wait_time *= 2**attempt
logger.warning(
f"请求失败,{wait_time:.2f}秒后重试"
f" (第{attempt + 1}/{max_retries}次重试): {e}"
)
await asyncio.sleep(wait_time)
except Exception as e:
logger.error(f"非预期异常,停止重试: {e}", e=e)
raise e.with_traceback(None) from None
if last_exception:
raise last_exception.with_traceback(None) from None
raise LLMException("重试循环异常结束").with_traceback(None) from None
class LoggingMiddleware:
"""
日志中间件:
职责归位后,统一负责 HTTP Payload 的生成、安全脱敏以及完整生命周期的日志记录。
"""
def __init__(
self,
provider_name: str,
model_name: str,
adapter: BaseAdapter,
identity: ModelIdentity,
log_context: str = "Generation",
):
self.provider_name = provider_name
self.model_name = model_name
self.adapter = adapter
self.identity = identity
self.log_context = log_context
async def __call__(
self, context: LLMContext[Any, Any], next_call: NextCall[Any, Any]
) -> Any:
attempt = context.runtime_state.get("attempt", 1)
api_key = context.runtime_state.get("api_key", "unknown")
masked_key = f"{api_key[:8]}..."
logger.info(
f"🌐 发起LLM请求 (尝试 {attempt}) - {self.provider_name}/{self.model_name} "
f"[{self.log_context}] Key: {masked_key}"
)
request_data = await self.adapter.prepare_payload(
identity=self.identity,
api_key=api_key,
request=context.request,
)
context.runtime_state["request_data"] = request_data
logger.debug(f"📡 请求URL: {request_data.url}")
logger.debug(f"📋 请求头: {dict(request_data.headers)}")
if self.identity.api_type == "smart":
from zhenxun.services.ai.llm.adapters.factory import SmartAdapter
smart_adapter = cast(SmartAdapter, self.adapter)
delegate_adapter = smart_adapter._get_delegate_adapter(self.identity)
sanitizer_req_context = f"{delegate_adapter.api_type}_request"
else:
sanitizer_req_context = self.adapter.log_sanitization_context
sanitized_body = sanitize_for_logging(
request_data.body, context=sanitizer_req_context
)
if request_data.files and isinstance(sanitized_body, dict):
file_info: list[str] = []
file_count = 0
if isinstance(request_data.files, list):
file_count = len(request_data.files)
for key, value in request_data.files:
filename = (
value[0]
if isinstance(value, tuple) and len(value) > 0
else "..."
)
file_info.append(f"{key}='{filename}'")
elif isinstance(request_data.files, dict):
file_count = len(request_data.files)
file_info = list(request_data.files.keys())
sanitized_body["[MULTIPART_FILES]"] = f"Count: {file_count} | {file_info}"
request_body_str = dump_json_safely(
sanitized_body, ensure_ascii=False, indent=2
)
logger.debug(f"📦 请求体: {request_body_str}")
try:
start_time = time.monotonic()
response = await next_call(context)
duration = (time.monotonic() - start_time) * 1000
logger.debug(f"🎯 LLM响应成功 [{self.log_context}] 耗时: {duration:.2f}ms")
return response
except Exception as e:
raise e.with_traceback(None) from None
class HttpExecutionMiddleware:
"""
终端 HTTP 执行中间件:
只负责将上游构建好的 Payload 发送出去,并拦截纯粹的 HTTP 网络故障。
"""
def __init__(
self,
http_client: LLMHttpClient,
identity: ModelIdentity,
health_manager: HealthManager,
adapter: BaseAdapter,
):
self.http_client = http_client
self.identity = identity
self.health_manager = health_manager
self.adapter = adapter
async def __call__(
self, context: LLMContext[Any, Any], next_call: NextCall[Any, Any]
) -> Any:
api_key = context.runtime_state["api_key"]
provider_name = self.identity.provider_name
route_id = f"{self.identity.provider_name}/{self.identity.model_name}"
request_data: RequestData = context.runtime_state["request_data"]
if context.cancellation_token:
context.cancellation_token.raise_if_cancelled()
start_time = time.monotonic()
try:
method = getattr(request_data, "method", "POST").upper()
req_kwargs = {
"headers": request_data.headers,
"timeout": context.request.timeout,
}
if method in ("POST", "PUT", "PATCH"):
if request_data.files:
req_kwargs["data"] = request_data.body
req_kwargs["files"] = request_data.files
else:
req_kwargs["content"] = json.dumps(
request_data.body, ensure_ascii=False
)
elif method == "GET" and request_data.body:
req_kwargs["params"] = request_data.body
post_task = asyncio.create_task(
self.http_client.request(method, request_data.url, **req_kwargs)
)
if context.cancellation_token:
context.cancellation_token.link_future(post_task)
raw_engine_output = await post_task
logger.debug(f"📥 HTTP响应状态码: {raw_engine_output.status_code}")
if exception := self.adapter.handle_http_error(raw_engine_output):
error_text = raw_engine_output.content.decode("utf-8", errors="ignore")
logger.debug(f"💥 完整错误响应: {error_text}")
raise exception.with_traceback(None) from None
latency = (time.monotonic() - start_time) * 1000
await self.health_manager.record_key_success(provider_name, api_key)
await self.health_manager.record_route_success(route_id, latency)
return await self.adapter.parse_payload(
identity=self.identity,
request=context.request,
raw_response=raw_engine_output,
)
except asyncio.CancelledError:
logger.warning(f"网络请求已被取消: {request_data.url}")
raise
except httpx.TimeoutException as e:
await self.health_manager.record_route_failure(route_id, e)
raise NetworkTimeoutException(f"HTTP请求超时: {e}", cause=e)
except httpx.NetworkError as e:
await self.health_manager.record_route_failure(route_id, e)
raise UpstreamServerException(f"网络连接中断: {e}", cause=e)
except LLMException as e:
if e.should_failover:
await self.health_manager.record_route_failure(route_id, e)
raise e.with_traceback(None) from None
except Exception as e:
logger.error(f"解析响应失败或发生未知错误: {e}")
masked_key = (
f"{api_key[:8]}...{api_key[-4:] if len(api_key) > 12 else '***'}"
if api_key
else "N/A"
)
raise UpstreamServerException(
f"网络请求异常: {type(e).__name__} - {e}",
details={"api_key": masked_key},
cause=e,
).with_traceback(None) from None
class ModalityFilterMiddleware:
"""模态过滤中间件:负责自动剔除当前模型不支持的多模态输入"""
def __init__(self, model_name: str, capabilities: ModelCapabilities):
self.model_name = model_name
self.capabilities = capabilities
async def __call__(
self, context: LLMContext[Any, Any], next_call: NextCall[Any, Any]
) -> Any:
request = context.request
if isinstance(request, ChatRequest):
filtered_messages = []
_warned_image, _warned_audio, _warned_video = False, False, False
for msg in request.messages:
new_content = []
for part in msg.content:
if (
isinstance(part, ImagePart)
and ModelModality.IMAGE
not in self.capabilities.input_modalities
):
if not _warned_image:
logger.warning(
f"模型 {self.model_name} 不支持图像输入,"
"已自动将图片替换为占位符"
)
_warned_image = True
new_content.append(TextPart(text="<图片>"))
elif (
isinstance(part, AudioPart)
and ModelModality.AUDIO
not in self.capabilities.input_modalities
):
if not _warned_audio:
logger.warning(
f"模型 {self.model_name} 不支持音频输入,"
"已自动将音频替换为占位符"
)
_warned_audio = True
new_content.append(TextPart(text="<音频>"))
elif (
isinstance(part, VideoPart)
and ModelModality.VIDEO
not in self.capabilities.input_modalities
):
if not _warned_video:
logger.warning(
f"模型 {self.model_name} 不支持视频输入,"
"已自动将视频替换为占位符"
)
_warned_video = True
new_content.append(TextPart(text="<视频>"))
elif (
isinstance(part, FilePart)
and ModelModality.FILE not in self.capabilities.input_modalities
):
new_content.append(TextPart(text="<文件>"))
else:
new_content.append(part)
filtered_messages.append(
model_copy(msg, update={"content": new_content})
)
context.request = model_copy(
request, update={"messages": filtered_messages}
)
return await next_call(context)
class ConfigMergeMiddleware:
"""配置合并中间件:合并覆盖配置,统一填充默认参数"""
def __init__(self, generation_config: GenerationConfig | None):
self.generation_config = generation_config
async def __call__(
self, context: LLMContext[Any, Any], next_call: NextCall[Any, Any]
) -> Any:
request = context.request
updates = {}
if hasattr(request, "tools") and getattr(request, "tools", None) is not None:
tools = getattr(request, "tools")
updates["tools"] = (
list(tools.values())
if isinstance(tools, dict)
else (tools if isinstance(tools, list) else [tools])
)
if hasattr(request, "config"):
req_config = getattr(request, "config", None)
if isinstance(req_config, GenerationConfig) and self.generation_config:
updates["config"] = self.generation_config.merge_with(req_config)
elif (
req_config is None
and self.generation_config
and hasattr(request, "messages")
):
updates["config"] = self.generation_config
if updates:
context.request = model_copy(request, update=updates)
return await next_call(context)
class ResponseRescueMiddleware:
"""响应挽救中间件:对于没有按要求返回图片链接的模型,尝试进行正则兜底下载"""
async def __call__(
self, context: LLMContext[Any, Any], next_call: NextCall[Any, Any]
) -> Any:
response = await next_call(context)
request = context.request
if isinstance(request, ChatRequest) and isinstance(response, ChatResponse):
gen_config = request.config
policy = gen_config.validation_policy if gen_config else None
should_rescue_image = policy and policy.get("require_image")
if (
should_rescue_image
and not response.images
and response.text
and gen_config
):
markdown_matches = re.findall(
r"(!?\[.*?\]\((https?://[^\)]+)\))", response.text
)
if markdown_matches:
logger.info(
f"检测到 {len(markdown_matches)} 个链接,尝试自动下载清洗。"
)
current_text = response.text
other_parts = [
p for p in response.content_parts if not isinstance(p, TextPart)
]
downloaded_urls = set()
for full_tag, url in markdown_matches:
try:
if url not in downloaded_urls:
content = await AsyncHttpx.get_content(url)
processed = process_image_data(content)
if isinstance(processed, bytes):
img_part = ImagePart(raw=processed)
else:
img_part = ImagePart(path=processed)
other_parts.append(img_part)
downloaded_urls.add(url)
current_text = current_text.replace(full_tag, "")
except Exception as exc:
logger.warning(f"自动下载图片失败: {url}, 错误: {exc}")
response.content_parts = [
TextPart(text=current_text.strip()),
*other_parts,
]
return response
class OutputValidationMiddleware:
"""输出验证中间件:负责策略校验与自定义格式验证)"""
async def __call__(
self, context: LLMContext[Any, Any], next_call: NextCall[Any, Any]
) -> Any:
response = await next_call(context)
request = context.request
if isinstance(request, ChatRequest) and isinstance(response, ChatResponse):
gen_config = request.config
if not gen_config:
return response
if gen_config.response_validator:
try:
gen_config.response_validator(response)
except Exception as exc:
raise LLMException(
f"响应内容未通过自定义验证器: {exc}",
details={"validator_error": str(exc)},
).with_traceback(None) from None
policy = gen_config.validation_policy
if policy and policy.get("require_image") and not response.images:
prompt_had_image = any(
isinstance(p, ImagePart)
for msg in request.messages
for p in msg.content
)
if not prompt_had_image:
logger.debug("提示词中未包含图片,跳过要求图片返回的重试特判。")
else:
raise LLMException(
"响应验证失败:要求返回图片但未找到图片数据。",
details={"policy": policy, "text_response": response.text},
)
return response
+162
View File
@@ -0,0 +1,162 @@
from abc import ABC, abstractmethod
from typing import Any
from zhenxun.services.ai.core.exceptions import (
ConfigurationException,
LLMException,
UpstreamServerException,
)
from zhenxun.services.ai.core.options import GenerationConfig
from zhenxun.services.ai.llm.manager import (
_get_group_name,
_resolve_model_group,
get_default_model,
get_model_instance,
list_available_models,
)
from zhenxun.services.ai.llm.system.capabilities import get_model_capabilities
from zhenxun.services.ai.llm.system.network import health_manager
from zhenxun.services.log import logger
class BaseModelRouter(ABC):
@abstractmethod
async def route(
self,
request: Any,
model_names: list[str],
task: str,
override_config: GenerationConfig | dict | None,
cancellation_token: Any | None,
) -> Any:
pass
class FallbackRouter(BaseModelRouter):
"""主备故障转移路由器"""
async def route(
self,
request: Any,
model_names: list[str],
task: str,
override_config: GenerationConfig | dict | None,
cancellation_token: Any | None,
) -> Any:
errors = []
all_nodes_bypassed = True
is_routed_call = len(model_names) > 1
request.extra["_is_routed_call"] = is_routed_call
start_idx = request.extra.get("_working_route_index", 0)
indices_to_try = list(range(start_idx, len(model_names))) + list(
range(0, start_idx)
)
for idx in indices_to_try:
m_name = model_names[idx]
if not health_manager.is_route_healthy(m_name, strict_mode=is_routed_call):
logger.debug(f"👉 [Orchestrator] 节点 '{m_name}' 熔断中,已跳过")
errors.append(f"{m_name}(熔断中)")
continue
caps = get_model_capabilities(m_name)
if not caps.supports_task(task):
errors.append(f"{m_name}(Unsupported Task: {task})")
continue
all_nodes_bypassed = False
try:
if len(model_names) > 1:
if idx != start_idx:
logger.debug(f"🔄 [Orchestrator] 切换至备用节点: '{m_name}'...")
async with await get_model_instance(
m_name, override_config, task=task
) as instance:
response = await instance.invoke(request, cancellation_token)
request.extra["_working_route_index"] = idx
if run_ctx := request.extra.get("run_context"):
run_ctx.state["_working_route_index"] = idx
return response
except LLMException as e:
if not e.should_failover:
logger.warning(
f"🚫 [Orchestrator] 节点 '{m_name}' "
f"返回不可恢复错误 ({e.__class__.__name__}),停止故障转移。"
)
raise e
logger.warning(
f"⚠️ [Orchestrator] 节点 '{m_name}' "
f"错误 ({e.__class__.__name__}),触发故障转移..."
)
errors.append(f"{m_name}({e.__class__.__name__})")
except Exception as e:
logger.warning(
f"⚠️ [Orchestrator] 节点 '{m_name}' 发生未知异常,触发故障转移: {e}"
)
errors.append(f"{m_name}(Error)")
if all_nodes_bypassed and len(model_names) > 1:
fallback_model = health_manager.get_best_fallback_route(model_names)
logger.warning(
f"⚠️ [Orchestrator] 路由组所有节点均已宕机!"
f"强制放行 '{fallback_model}' 探活..."
)
try:
async with await get_model_instance(
fallback_model, override_config, task=task
) as instance:
return await instance.invoke(request, cancellation_token)
except Exception as e:
errors.append(f"{fallback_model}(保底探活彻底失败:{e})")
err_msg = f"所有路由尝试均已失败: {', '.join(errors)}"
raise UpstreamServerException(err_msg)
class BaseOrchestrator:
"""顶层大模型请求编排器,负责组解析与路由策略委派。"""
def __init__(self, router: BaseModelRouter | None = None):
self.router = router or FallbackRouter()
async def invoke(
self,
request: Any,
model_name: str | None = None,
task: str = "chat",
override_config: GenerationConfig | dict | None = None,
cancellation_token: Any | None = None,
) -> Any:
resolved_model_name = model_name
if resolved_model_name is None:
resolved_model_name = get_default_model(task)
if resolved_model_name is None:
available_models = list_available_models()
if not available_models:
raise ConfigurationException("未配置任何AI模型")
resolved_model_name = available_models[0]["full_name"]
logger.warning(f"未指定模型,使用第一个可用模型: {resolved_model_name}")
group_name = _get_group_name(resolved_model_name)
if group_name is not None:
model_names = _resolve_model_group(group_name)
if not model_names:
raise ConfigurationException(
f"模型路由组 '{group_name}' 解析失败或为空,请检查配置。"
)
else:
model_names = [resolved_model_name]
return await self.router.route(
request=request,
model_names=model_names,
task=task,
override_config=override_config,
cancellation_token=cancellation_token,
)
LLMOrchestrator = BaseOrchestrator()
@@ -0,0 +1,356 @@
from abc import ABC, abstractmethod
import copy
from typing import Any
class BaseSchemaTransformer(ABC):
"""JSON Schema 节点处理器基类"""
@abstractmethod
def process_node(
self, node: dict[str, Any], is_root: bool = False
) -> dict[str, Any]:
"""处理单个 Schema 节点并返回修改后的节点"""
pass
class SchemaPipeline:
"""JSON Schema 转换管道执行器"""
def __init__(self, transformers: list[BaseSchemaTransformer]):
self.transformers = transformers
def run(self, schema: dict[str, Any]) -> dict[str, Any]:
"""执行管道,递归遍历并处理整个 Schema AST"""
if not isinstance(schema, dict):
return schema
schema_copy = copy.deepcopy(schema)
return self._walk(schema_copy, is_root=True)
def _walk(self, node: Any, is_root: bool = False) -> Any:
"""核心递归遍历算法,定向深入标准的 Schema 容器键"""
if isinstance(node, list):
return [self._walk(item, is_root=False) for item in node]
if not isinstance(node, dict):
return node
current_node = node
for transformer in self.transformers:
current_node = transformer.process_node(current_node, is_root=is_root)
if not isinstance(current_node, dict):
return current_node
for dict_key in ["properties", "patternProperties", "$defs", "definitions"]:
if dict_key in current_node and isinstance(current_node[dict_key], dict):
current_node[dict_key] = {
k: self._walk(v, is_root=False)
for k, v in current_node[dict_key].items()
}
for list_key in ["anyOf", "allOf", "oneOf", "prefixItems"]:
if list_key in current_node and isinstance(current_node[list_key], list):
current_node[list_key] = [
self._walk(v, is_root=False) for v in current_node[list_key]
]
for single_key in ["items", "additionalProperties", "contains"]:
if single_key in current_node and isinstance(
current_node[single_key], dict
):
current_node[single_key] = self._walk(
current_node[single_key], is_root=False
)
return current_node
class RootRefInlineTransformer(BaseSchemaTransformer):
"""将根节点的 $ref 展开,保留 $defs 供内部递归使用"""
def process_node(
self, node: dict[str, Any], is_root: bool = False
) -> dict[str, Any]:
if is_root and "$ref" in node:
ref_path = node.pop("$ref")
ref_name = ref_path.split("/")[-1]
defs = node.get("$defs") or node.get("definitions") or {}
if ref_name in defs:
def_content = defs[ref_name].copy()
for k, v in def_content.items():
if k not in node:
node[k] = v
return node
class RemoveUnsupportedKeysTransformer(BaseSchemaTransformer):
"""移除不支持的键"""
def __init__(self, keys_to_remove: list[str]):
self.keys_to_remove = keys_to_remove
def process_node(
self, node: dict[str, Any], is_root: bool = False
) -> dict[str, Any]:
for key in self.keys_to_remove:
node.pop(key, None)
return node
class GeminiEnumTransformer(BaseSchemaTransformer):
"""将 const 转换为 enum (Gemini 专用)"""
def process_node(
self, node: dict[str, Any], is_root: bool = False
) -> dict[str, Any]:
if "const" in node:
node["enum"] = [node.pop("const")]
return node
class GeminiNullableUnionTransformer(BaseSchemaTransformer):
"""处理 anyOf 和 type 列表中的 null,转换为 nullable: True (Gemini 专用)"""
def process_node(
self, node: dict[str, Any], is_root: bool = False
) -> dict[str, Any]:
if "type" in node and isinstance(node["type"], list):
types_list = node["type"]
if "null" in types_list:
node["nullable"] = True
types_list = [t for t in types_list if t != "null"]
node["type"] = types_list[0] if len(types_list) == 1 else types_list
if "anyOf" in node and isinstance(node["anyOf"], list):
any_of = node["anyOf"]
has_null = any(
isinstance(x, dict) and x.get("type") == "null" for x in any_of
)
if has_null:
node["nullable"] = True
new_any_of = [
x
for x in any_of
if not (isinstance(x, dict) and x.get("type") == "null")
]
if len(new_any_of) == 1:
node.update(new_any_of[0])
node.pop("anyOf", None)
else:
node["anyOf"] = new_any_of
return node
class GeminiFormatTransformer(BaseSchemaTransformer):
"""清理不支持的 format 格式 (Gemini 专用)"""
def process_node(
self, node: dict[str, Any], is_root: bool = False
) -> dict[str, Any]:
if node.get("format") and node["format"] not in ["enum", "date-time"]:
node.pop("format", None)
return node
class StrictObjectTransformer(BaseSchemaTransformer):
"""
强制对象必须关闭 additionalProperties,
并将所有 properties 设为 required (OpenAI/DeepSeek 专用)
"""
def process_node(
self, node: dict[str, Any], is_root: bool = False
) -> dict[str, Any]:
if node.get("type") == "object" or "properties" in node:
node["type"] = "object"
node["additionalProperties"] = False
if "properties" not in node:
node["properties"] = {}
node["required"] = list(node["properties"].keys())
return node
class OpenAIUnionFlattenTransformer(BaseSchemaTransformer):
"""
将 anyOf/allOf/oneOf 拍平,选取第一个非 null 的类型作为降级方案
(OpenAI 严格模式不支持复杂 Union)
"""
def process_node(
self, node: dict[str, Any], is_root: bool = False
) -> dict[str, Any]:
for union_key in ["anyOf", "allOf", "oneOf"]:
if union_key in node:
union_list = node.pop(union_key)
if isinstance(union_list, list) and len(union_list) > 0:
fallback = {}
for item in union_list:
if isinstance(item, dict) and item.get("type") != "null":
fallback = item
break
if not fallback and isinstance(union_list[0], dict):
fallback = union_list[0]
for k, v in fallback.items():
if k not in node:
node[k] = v
return node
class TypeEnforcerTransformer(BaseSchemaTransformer):
"""强制赋予空节点类型 (默认 string)"""
def process_node(
self, node: dict[str, Any], is_root: bool = False
) -> dict[str, Any]:
if "type" not in node and "properties" not in node and "$ref" not in node:
node["type"] = "string"
return node
class DeepSeekFallbackTransformer(BaseSchemaTransformer):
"""非根空对象转为字符串 (DeepSeek 专用避坑)"""
def process_node(
self, node: dict[str, Any], is_root: bool = False
) -> dict[str, Any]:
if node.get("type") == "object" and not is_root and not node.get("properties"):
node["type"] = "string"
for k in ["properties", "required", "additionalProperties"]:
node.pop(k, None)
node["description"] = (
f"{node.get('description', '')} (Please provide a JSON string)".strip()
)
return node
class GeminiCyclicRefTransformer(BaseSchemaTransformer):
"""
识别并处理循环引用:
Gemini 要求循环引用的 $ref 决不能出现在父级的 required 列表中
"""
def __init__(self, full_schema: dict):
self.cyclic_refs = self._detect_cycles(full_schema)
def _detect_cycles(self, schema: dict) -> set[str]:
defs = schema.get("$defs") or schema.get("definitions") or {}
cyclic = set()
def check_cycle(def_name: str, visited: set, path: list):
if def_name in path:
for node in path[path.index(def_name) :]:
cyclic.add(f"#/$defs/{node}")
cyclic.add(f"#/definitions/{node}")
return
if def_name in visited:
return
visited.add(def_name)
node = defs.get(def_name, {})
def find_refs(n: Any) -> list[str]:
refs = []
if isinstance(n, dict):
if "$ref" in n:
refs.append(n["$ref"])
for v in n.values():
refs.extend(find_refs(v))
elif isinstance(n, list):
for v in n:
refs.extend(find_refs(v))
return refs
for ref in find_refs(node):
if ref.startswith("#/$defs/") or ref.startswith("#/definitions/"):
next_def = ref.split("/")[-1]
check_cycle(next_def, visited, [*path, def_name])
visited_set = set()
for name in defs:
check_cycle(name, visited_set, [])
return cyclic
def process_node(
self, node: dict[str, Any], is_root: bool = False
) -> dict[str, Any]:
if "properties" in node and isinstance(node.get("required"), list):
new_required = []
for req_key in node["required"]:
prop = node.get("properties", {}).get(req_key, {})
is_cyclic = False
if prop.get("$ref") in self.cyclic_refs:
is_cyclic = True
elif prop.get("type") == "array" and isinstance(
prop.get("items"), dict
):
if prop["items"].get("$ref") in self.cyclic_refs:
is_cyclic = True
if not is_cyclic:
new_required.append(req_key)
if not new_required:
node.pop("required")
else:
node["required"] = new_required
return node
class RefComplianceTransformer(BaseSchemaTransformer):
"""如果包含 $ref,同级不允许出现任何非 $ 开头的键 (如 description, title)"""
def process_node(
self, node: dict[str, Any], is_root: bool = False
) -> dict[str, Any]:
if "$ref" in node:
keys_to_remove = [k for k in node.keys() if not k.startswith("$")]
for k in keys_to_remove:
node.pop(k)
return node
class GeminiDeepRefInlineTransformer(BaseSchemaTransformer):
"""
深度展开所有 $ref,并清理 $defs (Gemini Function Calling 专用)。
大模型 API 网关层面严格拒绝 $defs 与 $ref,所以进行全量物理替换。
如果遇到循环引用,将安全退化为 string 类型防止栈溢出死循环。
"""
def process_node(
self, node: dict[str, Any], is_root: bool = False
) -> dict[str, Any]:
if not is_root:
return node
defs = {}
if "$defs" in node:
defs.update(node["$defs"])
if "definitions" in node:
defs.update(node["definitions"])
def _resolve_refs(current: Any, visited: set) -> Any:
if isinstance(current, list):
return [_resolve_refs(item, visited) for item in current]
elif isinstance(current, dict):
if "$ref" in current:
ref_path = current["$ref"]
ref_name = ref_path.split("/")[-1]
if ref_name in defs:
if ref_name in visited:
return {
"type": "string",
"description": "Cyclic reference omitted",
}
new_visited = visited | {ref_name}
resolved = _resolve_refs(defs[ref_name], new_visited)
result = current.copy()
result.pop("$ref")
for k, v in resolved.items():
if k not in result:
result[k] = v
return result
return {k: _resolve_refs(v, visited) for k, v in current.items()}
return current
node = _resolve_refs(node, set())
node.pop("$defs", None)
node.pop("definitions", None)
return node
+280
View File
@@ -0,0 +1,280 @@
"""
LLM 模型实现类
包含 LLM 模型的抽象基类和具体实现,负责与各种 AI 提供商的 API 交互。
"""
from __future__ import annotations
from typing import Any, TypeVar
from pydantic import BaseModel
from zhenxun.services.ai.config import ProviderConfig, get_llm_config
from zhenxun.services.ai.core.exceptions import ConfigurationException
from zhenxun.services.ai.core.messages import (
AudioResponse,
BaseRequest,
ChatRequest,
ChatResponse,
EmbeddingRequest,
EmbeddingResponse,
ImageRequest,
ImageResponse,
RerankRequest,
RerankResponse,
SpeechRequest,
)
from zhenxun.services.ai.core.models import (
CancellationToken,
LLMContext,
ModelCapabilities,
ModelDetail,
ModelIdentity,
)
from zhenxun.services.ai.core.options import (
GenerationConfig,
)
from zhenxun.services.ai.core.protocols.llm import (
SupportsChat,
SupportsImageGeneration,
SupportsReranking,
SupportsSpeechSynthesis,
SupportsTextEmbedding,
)
from zhenxun.services.ai.core.protocols.middleware import LLMMiddleware
from zhenxun.services.ai.llm.system.models import RetryConfig
from zhenxun.services.ai.llm.system.network import HealthManager, LLMHttpClient
from zhenxun.services.log import logger
T = TypeVar("T", bound=BaseModel)
class LLMModel(
SupportsChat,
SupportsTextEmbedding,
SupportsSpeechSynthesis,
SupportsReranking,
SupportsImageGeneration,
):
"""LLM 模型实现类"""
def __init__(
self,
provider_config: ProviderConfig,
model_detail: ModelDetail,
health_manager: HealthManager,
http_client: LLMHttpClient,
capabilities: ModelCapabilities,
config_override: GenerationConfig | None = None,
):
self.provider_config = provider_config
self.model_detail = model_detail
self.health_manager = health_manager
self.http_client: LLMHttpClient = http_client
self.capabilities = capabilities
self._generation_config = config_override
self.provider_name = provider_config.name
self.api_type = model_detail.api_type or provider_config.api_type
self.api_base = provider_config.api_base
self.path_prefix = model_detail.path_prefix
self.api_keys = (
[provider_config.api_key]
if isinstance(provider_config.api_key, str)
else provider_config.api_key
)
self.model_name = model_detail.model_name
self.temperature = model_detail.temperature
self.generation_max_tokens = model_detail.generation_max_tokens
self._is_closed = False
self._ref_count = 0
self.identity = ModelIdentity(
provider_name=self.provider_name,
model_name=self.model_name,
api_type=self.api_type,
api_base=self.api_base,
path_prefix=self.path_prefix,
capabilities=self.capabilities,
generation_config=self._generation_config,
)
from zhenxun.services.ai.llm.engine.middlewares import MiddlewarePipeline
self.pipeline = MiddlewarePipeline()
self._setup_default_pipeline()
def add_middleware(self, middleware: LLMMiddleware) -> None:
"""注册一个中间件到处理管道的最外层"""
self.pipeline.add_middleware(middleware)
def _setup_default_pipeline(self) -> None:
from zhenxun.services.ai.llm.adapters.factory import get_adapter_for_api_type
from zhenxun.services.ai.llm.engine.middlewares import (
ConfigMergeMiddleware,
FailoverAndRetryMiddleware,
LLMCacheMiddleware,
LoggingMiddleware,
ModalityFilterMiddleware,
OutputValidationMiddleware,
ResponseRescueMiddleware,
)
client_settings = get_llm_config().client_settings
retry_config = RetryConfig(
max_retries=client_settings.max_retries,
retry_delay=client_settings.retry_delay,
)
adapter = get_adapter_for_api_type(self.api_type)
self.pipeline.add_middleware(LLMCacheMiddleware(self.model_name))
self.pipeline.add_middleware(ConfigMergeMiddleware(self._generation_config))
self.pipeline.add_middleware(
ModalityFilterMiddleware(self.model_name, self.capabilities)
)
self.pipeline.add_middleware(
FailoverAndRetryMiddleware(
retry_config, self.health_manager, self.provider_name, self.api_keys
)
)
self.pipeline.add_middleware(OutputValidationMiddleware())
self.pipeline.add_middleware(ResponseRescueMiddleware())
self.pipeline.add_middleware(
LoggingMiddleware(
self.provider_name, self.model_name, adapter, self.identity
)
)
async def _select_api_key(self, failed_keys: set[str] | None = None) -> str:
"""选择可用的API密钥(使用轮询策略)"""
if not self.api_keys:
raise ConfigurationException(
f"提供商 {self.provider_name} 没有配置API密钥",
)
selected_key = await self.health_manager.get_next_available_key(
self.provider_name, self.api_keys, failed_keys
)
if not selected_key:
raise ConfigurationException(
f"提供商 {self.provider_name} 的所有API密钥当前都不可用",
details={
"total_keys": len(self.api_keys),
"failed_keys": len(failed_keys or set()),
},
)
return selected_key
async def close(self):
"""标记模型实例的当前使用周期结束"""
if self._is_closed:
return
self._is_closed = True
logger.debug(
f"LLMModel实例的使用周期已结束: {self} (共享HTTP客户端状态不受影响)"
)
async def __aenter__(self):
if self._is_closed:
logger.debug(
f"Re-entering context for closed LLMModel {self}. "
f"Resetting _is_closed to False."
)
self._is_closed = False
self._check_not_closed()
self._ref_count += 1
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
"""异步上下文管理器出口"""
_ = exc_type, exc_val, exc_tb
self._ref_count -= 1
if self._ref_count <= 0:
self._ref_count = 0
await self.close()
def _check_not_closed(self):
"""检查实例是否已关闭"""
if self._is_closed:
raise RuntimeError(f"LLMModel实例已关闭: {self}")
async def invoke(
self,
request: BaseRequest,
cancellation_token: CancellationToken | None = None,
) -> Any:
"""
大一统命令执行核心入口 (Command Pattern)。
整合所有中间件执行管线,屏蔽具体模态差异。
"""
self._check_not_closed()
context = LLMContext(
request=request,
cancellation_token=cancellation_token,
)
from zhenxun.services.ai.llm.adapters.factory import get_adapter_for_api_type
from zhenxun.services.ai.llm.engine.middlewares import HttpExecutionMiddleware
adapter = get_adapter_for_api_type(self.api_type)
execution_middleware = HttpExecutionMiddleware(
http_client=self.http_client,
identity=self.identity,
health_manager=self.health_manager,
adapter=adapter,
)
async def terminal_handler(ctx: LLMContext[Any, Any]) -> Any:
async def _noop(c: LLMContext[Any, Any]) -> Any:
raise RuntimeError("HttpExecutionMiddleware 不应调用 next_call")
return await execution_middleware(ctx, _noop)
handler = self.pipeline.build(terminal_handler)
return await handler(context)
async def generate_response(
self,
request: ChatRequest,
cancellation_token: CancellationToken | None = None,
) -> ChatResponse:
return await self.invoke(request, cancellation_token)
async def generate_embeddings(
self,
request: EmbeddingRequest,
) -> EmbeddingResponse:
return await self.invoke(request)
async def rerank(
self,
request: RerankRequest,
) -> RerankResponse:
return await self.invoke(request)
async def generate_image(
self,
request: ImageRequest,
) -> ImageResponse:
return await self.invoke(request)
async def generate_speech(
self,
request: SpeechRequest,
) -> AudioResponse:
return await self.invoke(request)
def __str__(self) -> str:
status = "closed" if self._is_closed else "active"
return f"LLMModel({self.provider_name}/{self.model_name}, {status})"
def __repr__(self) -> str:
status = "closed" if self._is_closed else "active"
return (
f"LLMModel(provider={self.provider_name}, model={self.model_name}, "
f"api_type={self.api_type}, status={status})"
)
+311
View File
@@ -0,0 +1,311 @@
"""
LLM 模型管理器
对外提供统一的配置查询、模型发现与实例化入口。
"""
from typing import Any
from zhenxun.services.ai.config import (
ProviderConfig,
get_ai_config,
get_llm_config,
)
from zhenxun.services.ai.core.exceptions import ConfigurationException
from zhenxun.services.ai.core.models import ModelDetail
from zhenxun.services.ai.core.options import GenerationConfig
from zhenxun.services.ai.llm.system.capabilities import get_model_capabilities
from zhenxun.services.ai.llm.system.network import health_manager
from zhenxun.services.log import logger
from zhenxun.utils.manager.priority_manager import PriorityLifecycle
from zhenxun.utils.pydantic_compat import model_dump
_RESOLVED_GROUP_CACHE: dict[str, list[str]] = {}
"""路由组解析缓存,避免每次调用重复打印剔除警告并提升性能"""
def clear_resolved_group_cache() -> None:
global _RESOLVED_GROUP_CACHE
_RESOLVED_GROUP_CACHE.clear()
def parse_provider_model_string(name_str: str | None) -> tuple[str | None, str | None]:
"""解析 'ProviderName/ModelName' 格式的字符串"""
if not name_str or "/" not in name_str:
return None, None
parts = name_str.split("/", 1)
if len(parts) == 2 and parts[0].strip() and parts[1].strip():
return parts[0].strip(), parts[1].strip()
return None, None
def _get_group_name(name_str: str) -> str | None:
"""判断名称是否是组名,如果是则提取并返回组名,否则返回 None"""
name_str = name_str.strip()
if "/" not in name_str:
return name_str
return None
def get_default_api_base_for_type(api_type: str) -> str | None:
"""根据API类型获取默认的API基础地址"""
default_api_bases = {
"openai": "https://api.openai.com",
"doubao": "https://ark.cn-beijing.volces.com/api",
"deepseek": "https://api.deepseek.com",
"jina": "https://api.jina.ai",
"glm": "https://open.bigmodel.cn",
"gemini": "https://generativelanguage.googleapis.com",
"openrouter": "https://openrouter.ai/api",
"smart": None,
"openai_responses": None,
}
return default_api_bases.get(api_type)
def get_configured_providers() -> list[ProviderConfig]:
"""从配置中获取Provider列表"""
ai_config = get_ai_config()
providers = ai_config.get("PROVIDERS", [])
if not isinstance(providers, list):
logger.error("配置项 AI.PROVIDERS 的值不是一个列表,将使用空列表。")
return []
valid_providers = []
for i, item in enumerate(providers):
if isinstance(item, ProviderConfig):
if not item.api_base:
default_api_base = get_default_api_base_for_type(item.api_type)
if default_api_base:
item.api_base = default_api_base
valid_providers.append(item)
else:
logger.warning(
f"配置文件中第 {i + 1} 项未能正确解析为 ProviderConfig 对象,已跳过。"
)
return valid_providers
def find_model_config(
provider_name: str, model_name: str
) -> tuple[ProviderConfig, ModelDetail] | None:
"""在配置中查找指定 Provider 与 ModelDetail。"""
providers = get_configured_providers()
for provider in providers:
if provider.name.lower() == provider_name.lower():
for model_detail in provider.models:
if model_detail.model_name.lower() == model_name.lower():
return provider, model_detail
return None
def _resolve_model_group(group_name: str, visited: set | None = None) -> list[str]:
"""递归解析模型组,展开为扁平的真实模型列表,并防止循环嵌套。"""
global _RESOLVED_GROUP_CACHE
if visited is None and group_name in _RESOLVED_GROUP_CACHE:
return _RESOLVED_GROUP_CACHE[group_name]
if visited is None:
visited = set()
if group_name in visited:
logger.warning(f"检测到模型路由组嵌套死循环: {group_name},已安全跳过该分支。")
return []
visited.add(group_name)
llm_config = get_llm_config()
if group_name not in llm_config.model_groups:
logger.warning(f"模型路由组 '{group_name}' 不存在于配置中。")
return []
resolved_models = []
for item in llm_config.model_groups[group_name]:
item = item.strip()
sub_group = _get_group_name(item)
if sub_group:
resolved_models.extend(_resolve_model_group(sub_group, visited.copy()))
else:
prov_mod = parse_provider_model_string(item)
if prov_mod[0] and prov_mod[1]:
if find_model_config(prov_mod[0], prov_mod[1]):
if item not in resolved_models:
resolved_models.append(item)
else:
logger.warning(
f"⚠️ [Router] 路由组 '{group_name}' 中的模型 "
f"'{item}' 未在配置,已被自动剔除!"
)
else:
logger.warning(f"路由组 '{group_name}' 包含无效格式的项目 '{item}'。")
if len(visited) == 1:
_RESOLVED_GROUP_CACHE[group_name] = resolved_models
return resolved_models
def _get_model_identifiers(provider_name: str, model_detail: ModelDetail) -> list[str]:
"""获取模型的所有可用标识符"""
return [f"{provider_name}/{model_detail.model_name}"]
def list_available_models() -> list[dict[str, Any]]:
"""列出所有已配置的可用模型及其信息。"""
providers = get_configured_providers()
model_list = []
for provider in providers:
for model_detail in provider.models:
caps = get_model_capabilities(model_detail.model_name)
model_info = {
"provider_name": provider.name,
"model_name": model_detail.model_name,
"full_name": f"{provider.name}/{model_detail.model_name}",
"api_type": provider.api_type or "auto-detect",
"api_base": provider.api_base,
"is_available": model_detail.is_available,
"is_embedding_model": caps.is_embedding_model,
"max_input_tokens": caps.max_input_tokens,
"available_identifiers": _get_model_identifiers(
provider.name, model_detail
),
}
model_list.append(model_info)
return model_list
def list_embedding_models() -> list[dict[str, Any]]:
"""列出所有支持嵌入能力的模型。"""
all_models = list_available_models()
return [model for model in all_models if model.get("is_embedding_model", False)]
def list_model_identifiers() -> dict[str, list[str]]:
"""列出所有模型的可用标识符映射。"""
providers = get_configured_providers()
result = {}
for provider in providers:
for model_detail in provider.models:
full_name = f"{provider.name}/{model_detail.model_name}"
identifiers = _get_model_identifiers(provider.name, model_detail)
result[full_name] = identifiers
return result
def get_default_model(task: str = "chat") -> str | None:
"""根据任务类型获取默认模型名称"""
config = get_llm_config()
return getattr(config.default_models, task, None)
async def get_key_usage_stats() -> dict[str, Any]:
"""获取所有 Provider 的 Key 使用统计。"""
providers = get_configured_providers()
stats = {}
for provider in providers:
keys = (
[provider.api_key]
if isinstance(provider.api_key, str)
else provider.api_key
)
provider_stats = {}
provider_state = health_manager.state.providers.get(provider.name)
if provider_state:
for k in keys:
stat_data = provider_state.api_keys.get(k)
if stat_data:
provider_stats[health_manager._get_key_id(k)] = model_dump(
stat_data
)
stats[provider.name] = {
"total_keys": len(
[provider.api_key]
if isinstance(provider.api_key, str)
else provider.api_key
),
"key_stats": provider_stats,
}
return stats
async def reset_key_status(provider_name: str, api_key: str | None = None) -> bool:
"""重置指定 Provider 的 Key 状态。"""
providers = get_configured_providers()
target_provider = None
for provider in providers:
if provider.name.lower() == provider_name.lower():
target_provider = provider
break
if not target_provider:
logger.error(f"未找到Provider: {provider_name}")
return False
provider_keys = (
[target_provider.api_key]
if isinstance(target_provider.api_key, str)
else target_provider.api_key
)
if api_key:
if api_key in provider_keys:
await health_manager.reset_key_status(target_provider.name, api_key)
logger.info(f"已重置Provider '{provider_name}' 的指定Key状态")
return True
else:
logger.error(f"指定的Key不属于Provider '{provider_name}'")
return False
else:
for key in provider_keys:
await health_manager.reset_key_status(target_provider.name, key)
logger.info(f"已重置Provider '{provider_name}' 的所有Key状态")
return True
async def get_model_instance(
provider_model_name: str | None = None,
override_config: dict[str, Any] | GenerationConfig | None = None,
task: str = "chat",
) -> Any:
"""作为门面 API,解析字符串并调用底层的 get_or_create_model"""
resolved_model_name_str = provider_model_name
if resolved_model_name_str is None:
resolved_model_name_str = get_default_model(task)
if resolved_model_name_str is None:
available_models_list = list_available_models()
if not available_models_list:
raise ConfigurationException("未配置任何AI模型")
resolved_model_name_str = available_models_list[0]["full_name"]
logger.warning(f"未指定模型,使用第一个可用模型: {resolved_model_name_str}")
prov_name_str, mod_name_str = parse_provider_model_string(resolved_model_name_str)
if not prov_name_str or not mod_name_str:
raise ConfigurationException(f"无效的模型名称格式: '{resolved_model_name_str}'")
config_tuple_found = find_model_config(prov_name_str, mod_name_str)
if not config_tuple_found:
raise ConfigurationException(f"未找到模型: '{resolved_model_name_str}'. ")
provider_config_found, model_detail_found = config_tuple_found
from zhenxun.services.ai.llm.system.cache import get_or_create_model
return await get_or_create_model(
provider_config_found, model_detail_found, override_config
)
def clear_all_cache() -> None:
"""
清空模型实例缓存与路由组解析缓存。
"""
from zhenxun.services.ai.llm.system.cache import clear_model_cache
clear_model_cache()
clear_resolved_group_cache()
logger.debug("已清空全局模型实例与路由组缓存")
@PriorityLifecycle.on_startup(priority=10)
async def _init_llm_config_on_startup():
"""启动时初始化 LLM 配置、密钥状态并预热工具提供者管理器。"""
logger.info("正在初始化 LLM 配置并加载遥测状态...")
try:
from zhenxun.services.ai.config import get_llm_config
from zhenxun.services.ai.llm.system.network import health_manager
from zhenxun.services.ai.tools.engine.registry import tool_provider_manager
get_llm_config()
await health_manager.initialize()
await tool_provider_manager.initialize()
except Exception as e:
logger.error(f"LLM 配置或遥测状态初始化时发生错误: {e}", e=e)
@@ -0,0 +1,3 @@
"""
LLM 运维与调度管理模块。
"""
+173
View File
@@ -0,0 +1,173 @@
import hashlib
import time
from typing import Any
from zhenxun.services.ai.config import ProviderConfig, get_llm_config
from zhenxun.services.ai.core.exceptions import ConfigurationException, LLMException
from zhenxun.services.ai.core.models import ModelDetail, ModelModality
from zhenxun.services.ai.core.options import GenerationConfig
from zhenxun.services.ai.llm.builder import validate_override_params
from zhenxun.services.ai.llm.engine.service import LLMModel
from zhenxun.services.ai.llm.system.capabilities import get_model_capabilities
from zhenxun.services.ai.llm.system.network import health_manager, http_client_manager
from zhenxun.services.log import logger
from zhenxun.utils.pydantic_compat import dump_json_safely, model_copy, model_dump
_model_cache: dict[str, tuple[LLMModel, float]] = {}
_cache_ttl = 3600
_max_cache_size = 10
def _make_cache_key(
provider_name: str,
model_name: str,
override_config: dict | GenerationConfig | None,
) -> str:
"""生成缓存键"""
config_str = (
dump_json_safely(override_config, sort_keys=True) if override_config else "None"
)
key_data = f"{provider_name}/{model_name}:{config_str}"
return hashlib.md5(key_data.encode()).hexdigest()
def _get_cached_model(cache_key: str) -> LLMModel | None:
"""从缓存获取模型"""
if cache_key in _model_cache:
model, created_time = _model_cache[cache_key]
current_time = time.time()
if current_time - created_time > _cache_ttl:
del _model_cache[cache_key]
logger.debug(f"模型缓存已过期: {cache_key}")
return None
if model._is_closed:
logger.debug(
f"缓存的模型 {cache_key} ({model.provider_name}/{model.model_name}) "
f"处于_is_closed=True状态,重置为False以供复用。"
)
model._is_closed = False
logger.debug(
f"使用缓存的模型: {cache_key} -> {model.provider_name}/{model.model_name}"
)
return model
return None
def _cache_model(cache_key: str, model: LLMModel):
"""缓存模型实例"""
current_time = time.time()
if len(_model_cache) >= _max_cache_size:
oldest_key = min(_model_cache.keys(), key=lambda k: _model_cache[k][1])
del _model_cache[oldest_key]
_model_cache[cache_key] = (model, current_time)
def clear_model_cache():
"""仅清空内存中的模型实例缓存。"""
global _model_cache
_model_cache.clear()
logger.debug("已清空模型实例缓存")
async def get_or_create_model(
provider_config_found: ProviderConfig,
model_detail_found: ModelDetail,
override_config: dict[str, Any] | GenerationConfig | None = None,
) -> Any:
"""组装或获取底层 LLMModel 状态机实例 (不包含任何字符串解析)。"""
prov_name_str = provider_config_found.name
mod_name_str = model_detail_found.model_name
cache_key = _make_cache_key(prov_name_str, mod_name_str, override_config)
cached_model = _get_cached_model(cache_key)
def _get_clean_log_config(cfg: GenerationConfig) -> dict:
"""辅助函数:剔除超长 Schema 以防止日志刷屏"""
log_dict = model_dump(cfg, exclude_none=True)
if "output" in log_dict and isinstance(log_dict["output"], dict):
if "response_schema" in log_dict["output"]:
log_dict["output"]["response_schema"] = (
"<JSON Schema Hidden for brevity>"
)
return log_dict
if cached_model:
if override_config:
validated_override = validate_override_params(override_config)
if cached_model._generation_config != validated_override:
cached_model._generation_config = validated_override
cached_model.identity.generation_config = validated_override
logger.debug(
f"对缓存模型 {prov_name_str}/{mod_name_str} 应用新的覆盖配置: "
f"{_get_clean_log_config(validated_override)}"
)
return cached_model
capabilities = get_model_capabilities(model_detail_found.model_name)
capabilities = model_copy(capabilities, deep=True)
if model_detail_found.task_type == "image_generation":
capabilities.output_modalities.add(ModelModality.IMAGE)
capabilities.supports_tool_calling = False
llm_config = get_llm_config()
client_settings = llm_config.client_settings
default_timeout = (
provider_config_found.timeout
if provider_config_found.timeout is not None
else client_settings.timeout
)
config_for_http_client = ProviderConfig(
name=provider_config_found.name,
api_key=provider_config_found.api_key,
models=provider_config_found.models,
timeout=default_timeout,
api_base=provider_config_found.api_base,
api_type=provider_config_found.api_type,
openai_compat=provider_config_found.openai_compat,
temperature=provider_config_found.temperature,
generation_max_tokens=provider_config_found.generation_max_tokens,
)
shared_http_client = await http_client_manager.get_client(config_for_http_client)
try:
model_instance = LLMModel(
provider_config=config_for_http_client,
model_detail=model_detail_found,
health_manager=health_manager,
http_client=shared_http_client,
capabilities=capabilities,
)
if override_config:
validated_override_params = validate_override_params(override_config)
model_instance._generation_config = validated_override_params
model_instance.identity.generation_config = validated_override_params
logger.debug(
f"为新模型 {prov_name_str}/{mod_name_str} 应用配置覆盖: "
f"{_get_clean_log_config(validated_override_params)}"
)
_cache_model(cache_key, model_instance)
logger.debug(
f"创建并缓存了新模型: {cache_key} -> {prov_name_str}/{mod_name_str}"
)
return model_instance
except LLMException:
raise
except Exception as e:
logger.error(
f"实例化 LLMModel ({prov_name_str}/{mod_name_str}) 时发生内部错误: {e!s}",
e=e,
)
raise ConfigurationException(
f"初始化模型 '{prov_name_str}/{mod_name_str}' 失败: {e!s}",
cause=e,
)
@@ -0,0 +1,352 @@
import fnmatch
from zhenxun.services.ai.core.models import (
ModelCapabilities,
ModelModality,
ReasoningMode,
)
from zhenxun.utils.pydantic_compat import model_copy
CTX_1M = 1_000_000
CTX_400K = 400_000
CTX_256K = 256_000
CTX_200K = 204_800
CTX_128K = 128_000
CTX_8K = 8_192
CAP_MULTIMODAL_EMBEDDING = ModelCapabilities(
input_modalities={
ModelModality.TEXT,
ModelModality.IMAGE,
ModelModality.AUDIO,
ModelModality.VIDEO,
ModelModality.FILE,
},
is_embedding_model=True,
supports_tool_calling=False,
)
STANDARD_TEXT_TOOL_CAPABILITIES = ModelCapabilities(
input_modalities={ModelModality.TEXT},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
supported_native_tools={
"web_search",
"code_execution",
"computer_use",
"file_search",
},
)
CAP_GEMINI_2_5 = ModelCapabilities(
input_modalities={
ModelModality.TEXT,
ModelModality.IMAGE,
ModelModality.AUDIO,
ModelModality.VIDEO,
},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
reasoning_mode=ReasoningMode.BUDGET,
reasoning_visibility="visible",
supported_native_tools={
"web_search",
"code_execution",
"google_map",
"url_context",
},
)
CAP_GEMINI_3_BASE = ModelCapabilities(
input_modalities={
ModelModality.TEXT,
ModelModality.IMAGE,
ModelModality.AUDIO,
ModelModality.VIDEO,
},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
reasoning_mode=ReasoningMode.LEVEL,
reasoning_visibility="visible",
supported_native_tools={
"web_search",
"code_execution",
"google_map",
"url_context",
},
features={
"mixed_tools",
"server_side_tool_invocations",
},
)
CAP_GEMINI_3_PRO = model_copy(
CAP_GEMINI_3_BASE,
update={
"reasoning_effort_map": {
"max": "high",
"xhigh": "high",
"minimal": "low",
"none": "low",
}
},
)
CAP_GEMINI_3_FLASH = model_copy(
CAP_GEMINI_3_BASE,
update={
"reasoning_effort_map": {"max": "high", "xhigh": "high", "none": "minimal"}
},
)
CAP_OPENAI_REASONING = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.IMAGE},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
reasoning_mode=ReasoningMode.EFFORT,
reasoning_visibility="hidden",
supported_native_tools={
"web_search",
"code_execution",
"computer_use",
"file_search",
},
reasoning_effort_map={"max": "xhigh", "minimal": "none"},
)
CAP_OPENAI_MULTIMODAL = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.IMAGE},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
supported_native_tools={
"web_search",
"computer_use",
"file_search",
},
reasoning_effort_map={"max": "xhigh", "minimal": "none"},
)
CAP_DEEPSEEK_V4 = ModelCapabilities(
input_modalities={ModelModality.TEXT},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
reasoning_mode=ReasoningMode.EFFORT,
reasoning_visibility="visible",
reasoning_effort_map={"minimal": "low"},
)
CAP_MINIMAX_REASONING = ModelCapabilities(
input_modalities={ModelModality.TEXT},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
reasoning_mode=ReasoningMode.EFFORT,
reasoning_visibility="visible",
)
CAP_GLM_MULTIMODAL = ModelCapabilities(
input_modalities={
ModelModality.TEXT,
ModelModality.IMAGE,
ModelModality.VIDEO,
ModelModality.FILE,
},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
)
CAP_MINIMAX_MULTIMODAL = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.IMAGE, ModelModality.VIDEO},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
)
CAP_MIMO_TEXT = ModelCapabilities(
input_modalities={ModelModality.TEXT},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
supported_native_tools={"web_search"},
reasoning_effort_map={"max": "high", "xhigh": "high", "minimal": "low"},
)
CAP_MIMO_MULTIMODAL = ModelCapabilities(
input_modalities={
ModelModality.TEXT,
ModelModality.IMAGE,
ModelModality.AUDIO,
ModelModality.VIDEO,
},
output_modalities={ModelModality.TEXT, ModelModality.AUDIO},
supports_tool_calling=True,
supported_native_tools={"web_search"},
reasoning_effort_map={"max": "high", "xhigh": "high", "minimal": "low"},
)
CAP_OPENAI_TTS = ModelCapabilities(
input_modalities={ModelModality.TEXT},
output_modalities={ModelModality.AUDIO},
supports_tool_calling=False,
default_voice_id="alloy",
)
CAP_GEMINI_TTS = ModelCapabilities(
input_modalities={ModelModality.TEXT},
output_modalities={ModelModality.AUDIO},
supports_tool_calling=False,
default_voice_id="Aoede",
)
CAP_MINIMAX_TTS = ModelCapabilities(
input_modalities={ModelModality.TEXT},
output_modalities={ModelModality.AUDIO},
supports_tool_calling=False,
default_voice_id="female-shaonv",
)
CAP_MIMO_TTS = ModelCapabilities(
input_modalities={ModelModality.TEXT},
output_modalities={ModelModality.AUDIO},
supports_tool_calling=False,
default_voice_id="mimo_default",
)
CAP_TEXT_EMBEDDING = ModelCapabilities(
input_modalities={ModelModality.TEXT},
is_embedding_model=True,
supports_tool_calling=False,
)
CAP_RERANK_ONLY = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.IMAGE},
is_rerank_model=True,
)
CAP_OPENAI_IMAGE = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.IMAGE},
output_modalities={ModelModality.TEXT, ModelModality.IMAGE},
supports_tool_calling=False,
)
CAP_GEMINI_IMAGE = ModelCapabilities(
input_modalities={
ModelModality.TEXT,
ModelModality.IMAGE,
ModelModality.AUDIO,
ModelModality.VIDEO,
},
output_modalities={ModelModality.TEXT, ModelModality.IMAGE},
supports_tool_calling=True,
supported_native_tools={
"web_search",
},
)
DEFAULT_PERMISSIVE_CAPABILITIES = ModelCapabilities(
input_modalities={
ModelModality.TEXT,
ModelModality.IMAGE,
ModelModality.AUDIO,
ModelModality.VIDEO,
},
output_modalities={
ModelModality.TEXT,
ModelModality.IMAGE,
ModelModality.AUDIO,
},
supports_tool_calling=True,
)
MODEL_ALIAS_MAPPING: dict[str, str] = {
"*DeepSeek-V4-Pro*": "deepseek-v4-pro",
"*DeepSeek-V4-Flash*": "deepseek-v4-flash",
}
_ROUTING_TABLE: list[tuple[list[str], ModelCapabilities, int]] = [
(["mimo-*tts*"], CAP_MIMO_TTS, CTX_8K),
(["gemini-*tts*"], CAP_GEMINI_TTS, CTX_8K),
(["*minimax-*tts*", "*MiniMax-*tts*"], CAP_MINIMAX_TTS, CTX_8K),
(["*tts*"], CAP_OPENAI_TTS, CTX_8K),
(["*gpt*image*"], CAP_OPENAI_IMAGE, CTX_128K),
(["*gemini*image*", "*nano-banana*"], CAP_GEMINI_IMAGE, CTX_128K),
(["glm-4.6v*"], CAP_GLM_MULTIMODAL, CTX_128K),
(["glm-4.7-flash*"], STANDARD_TEXT_TOOL_CAPABILITIES, CTX_128K),
(["deepseek-v4-pro*", "deepseek-v4-flash*"], CAP_DEEPSEEK_V4, CTX_1M),
(["glm-4-long*"], STANDARD_TEXT_TOOL_CAPABILITIES, CTX_1M),
(["*MiniMax-M3*"], CAP_MINIMAX_MULTIMODAL, CTX_1M),
(["mimo-v2.5-pro*", "mimo-v2-pro*", "mimo-v2-flash*"], CAP_MIMO_TEXT, CTX_1M),
(["mimo-v2.5", "mimo-v2-omni*"], CAP_MIMO_MULTIMODAL, CTX_1M),
(["gpt-5.5*", "gpt-5.4*"], CAP_OPENAI_MULTIMODAL, CTX_1M),
(["gemini-3*pro*"], CAP_GEMINI_3_PRO, CTX_1M),
(["gemini-3*"], CAP_GEMINI_3_FLASH, CTX_1M),
(
["gemini-2.5-pro*", "gemini-2.5-flash*"],
CAP_GEMINI_2_5,
CTX_1M,
),
(
["gpt-5*", "gpt-5-mini*", "gpt-5-nano*", "*codex*"],
CAP_OPENAI_MULTIMODAL,
CTX_400K,
),
(
["kimi-k2.7*", "kimi-k2.6*", "kimi-k2.5*"],
DEFAULT_PERMISSIVE_CAPABILITIES,
CTX_256K,
),
(["glm-5v*"], CAP_GLM_MULTIMODAL, CTX_200K),
(["glm-5*", "glm-4.7*", "glm-4.6*"], STANDARD_TEXT_TOOL_CAPABILITIES, CTX_200K),
(["*MiniMax-M2*", "*minimax-m2*"], CAP_MINIMAX_REASONING, CTX_200K),
(["gpt-4*", "gpt-3.5*", "gpt-*"], CAP_OPENAI_MULTIMODAL, CTX_128K),
(["o1-*", "o3-*"], CAP_OPENAI_REASONING, CTX_128K),
(["glm-4v*"], CAP_GLM_MULTIMODAL, CTX_128K),
(
["glm-4.5*", "glm-4-flashx-*", "glm-4*"],
STANDARD_TEXT_TOOL_CAPABILITIES,
CTX_128K,
),
(
["gemini-embedding-2*", "jina-embeddings-v5-omni*"],
CAP_MULTIMODAL_EMBEDDING,
CTX_8K,
),
(
["*embedding*", "*Embedding*", "jina-embeddings-*", "bge-m3*", "*bge-large*"],
CAP_TEXT_EMBEDDING,
CTX_8K,
),
(["*reranker*", "*rerank*", "jina-colbert-*"], CAP_RERANK_ONLY, CTX_8K),
]
def _build_registry() -> dict[str, ModelCapabilities]:
"""构建模型能力注册表 (基于声明式路由表)"""
registry: dict[str, ModelCapabilities] = {}
for patterns, cap_template, ctx_limit in _ROUTING_TABLE:
cap_instance = model_copy(cap_template, update={"max_input_tokens": ctx_limit})
for pattern in patterns:
registry[pattern] = cap_instance
return registry
MODEL_CAPABILITIES_REGISTRY = _build_registry()
def get_model_capabilities(model_name: str) -> ModelCapabilities:
"""
从注册表获取模型能力,支持别名映射和通配符匹配。
"""
canonical_name = model_name
for alias_pattern, c_name in MODEL_ALIAS_MAPPING.items():
if fnmatch.fnmatch(model_name, alias_pattern):
canonical_name = c_name
break
parts = canonical_name.split("/")
names_to_check = ["/".join(parts[i:]) for i in range(len(parts))]
for name in names_to_check:
if name in MODEL_CAPABILITIES_REGISTRY:
return MODEL_CAPABILITIES_REGISTRY[name]
for pattern, capabilities in MODEL_CAPABILITIES_REGISTRY.items():
if "*" in pattern and fnmatch.fnmatch(name, pattern):
return capabilities
return DEFAULT_PERMISSIVE_CAPABILITIES
+112
View File
@@ -0,0 +1,112 @@
"""
LLM 服务内部状态与配置数据模型。
"""
from enum import Enum
from pydantic import BaseModel, Field
class HttpClientConfig(BaseModel):
"""HTTP 客户端底层连接池配置"""
timeout: int = 180
"""请求超时时间(秒)"""
max_connections: int = 100
"""连接池最大允许的总连接数"""
max_keepalive_connections: int = 20
"""连接池允许保持复用的最大 Keep-Alive 连接数"""
class RouteHealthState(str, Enum):
"""模型路由级别健康状态枚举"""
CLOSED = "CLOSED"
"""闭合状态:健康正常,放行所有请求"""
OPEN = "OPEN"
"""打开状态:熔断器已触发,拒绝所有请求"""
HALF_OPEN = "HALF_OPEN"
"""半开状态:冷却期结束,允许少量探活请求试探服务是否恢复"""
class KeyHealthStatus(BaseModel):
"""API 密钥凭证级别的详细状态信息"""
status: str = "HEALTHY"
"""当前密钥的健康状态 (HEALTHY, WARNING, COOLDOWN, DISABLED 等)"""
successes: int = 0
"""该密钥请求成功的总次数"""
failures: int = 0
"""该密钥请求失败的总次数"""
cooldown_until: float = 0.0
"""冷却解锁的 Unix 时间戳,0.0 表示无冷却"""
last_error: str | None = None
"""最近一次请求失败记录的错误原因"""
class RouteHealthStatus(BaseModel):
"""模型路由级别的遥测与熔断状态信息"""
state: RouteHealthState = RouteHealthState.CLOSED
"""当前路由节点的熔断器状态"""
cooldown_until: float = 0.0
"""熔断器探活冷却解锁的 Unix 时间戳"""
success_rate: float = 100.0
"""该路由的请求成功率百分比 (0.0 - 100.0)"""
successes: int = 0
"""该路由请求成功的总次数"""
failures: int = 0
"""该路由请求失败的总次数"""
latency_ema: float = 0.0
"""平滑指数移动平均 (EMA) 的响应延迟预估值(毫秒)"""
last_error: str | None = None
"""该路由最近一次发生服务端故障的错误原因"""
class ProviderHealthStatus(BaseModel):
"""单一提供商(服务商)的综合健康状态"""
api_keys: dict[str, KeyHealthStatus] = Field(default_factory=dict)
"""该提供商下绑定的所有 API 密钥状态映射表"""
class GlobalHealthState(BaseModel):
"""全局 AI 服务遥测与健康状态基座持久化模型"""
providers: dict[str, ProviderHealthStatus] = Field(default_factory=dict)
"""所有提供商维度的状态映射表"""
routes: dict[str, RouteHealthStatus] = Field(default_factory=dict)
"""所有模型路由端点维度的状态映射表"""
class CircuitBreakerPolicy(BaseModel):
"""熔断器与密钥冷却策略配置"""
rate_limit_cooldown: int = Field(default=60)
"""被限流时的短时冷却时间(秒)"""
auth_error_cooldown: int = Field(default=31536000)
"""凭证错误/鉴权失败时的长时冷却(默认1年即为禁用)"""
server_error_cooldown: int = Field(default=120)
"""服务端发生崩溃(500)等异常时的模型节点熔断时间(秒)"""
quota_error_cooldown: int = Field(default=3600)
"""API 额度耗尽时的冷却时间(秒)"""
class RetryConfig:
"""请求重试与轮询策略配置"""
def __init__(
self,
max_retries: int = 3,
retry_delay: float = 1.0,
exponential_backoff: bool = True,
key_rotation: bool = True,
):
self.max_retries = max_retries
"""发生可恢复异常时的最大重试次数"""
self.retry_delay = retry_delay
"""基础重试等待延迟(秒)"""
self.exponential_backoff = exponential_backoff
"""是否启用指数退避策略(延迟时间随重试次数翻倍)"""
self.key_rotation = key_rotation
"""发生凭证错误或限流时,是否自动轮换到下一个可用的 API Key"""
+594
View File
@@ -0,0 +1,594 @@
"""
LLM 核心基础设施模块
包含执行 LLM 请求所需的底层组件,如 HTTP 客户端、API Key 存储和智能重试逻辑。
"""
import asyncio
import json
import os
from pathlib import Path
import time
from typing import Any
import aiofiles
import httpx
import nonebot
from zhenxun.configs.config import BotConfig
from zhenxun.configs.path_config import DATA_PATH
from zhenxun.services.ai.config import ProviderConfig
from zhenxun.services.ai.core.exceptions import (
AuthenticationException,
ConfigurationException,
LocationNotSupportedException,
QuotaExceededException,
RateLimitException,
)
from zhenxun.services.log import logger
from zhenxun.utils.pydantic_compat import model_dump, parse_as
from zhenxun.utils.user_agent import get_user_agent
from .models import (
CircuitBreakerPolicy,
GlobalHealthState,
HttpClientConfig,
KeyHealthStatus,
ProviderHealthStatus,
RouteHealthState,
RouteHealthStatus,
)
driver = nonebot.get_driver()
class LLMHttpClient:
"""[内部 API] LLM 服务专用异步 HTTP 客户端封装。"""
def __init__(self, config: HttpClientConfig | None = None):
"""初始化 LLM 服务专用 HTTP 客户端"""
self.config = config or HttpClientConfig()
self._client: httpx.AsyncClient | None = None
self._active_requests = 0
self._lock = asyncio.Lock()
async def _ensure_client_initialized(self) -> httpx.AsyncClient:
"""确保底层的 AsyncClient 已完成初始化"""
if self._client is None or self._client.is_closed:
async with self._lock:
if self._client is None or self._client.is_closed:
logger.debug(
f"LLMHttpClient: 正在初始化新的 httpx.AsyncClient "
f"配置: {self.config}"
)
headers = get_user_agent()
limits = httpx.Limits(
max_connections=self.config.max_connections,
max_keepalive_connections=self.config.max_keepalive_connections,
)
timeout = httpx.Timeout(self.config.timeout)
client_kwargs = {}
if BotConfig.system_proxy:
try:
version_parts = httpx.__version__.split(".")
major = int(
"".join(c for c in version_parts[0] if c.isdigit())
)
minor = (
int("".join(c for c in version_parts[1] if c.isdigit()))
if len(version_parts) > 1
else 0
)
if (major, minor) >= (0, 28):
client_kwargs["proxy"] = BotConfig.system_proxy
else:
client_kwargs["proxies"] = BotConfig.system_proxy
except (ValueError, IndexError):
client_kwargs["proxies"] = BotConfig.system_proxy
logger.warning(
f"无法解析 httpx version '{httpx.__version__}',"
"LLM模块将默认使用旧版 'proxies' 参数语法。"
)
self._client = httpx.AsyncClient(
headers=headers,
limits=limits,
timeout=timeout,
follow_redirects=True,
**client_kwargs,
)
if self._client is None:
raise ConfigurationException(
"HTTP 客户端初始化失败。",
)
return self._client
async def request(self, method: str, url: str, **kwargs: Any) -> httpx.Response:
"""发送异步 HTTP 请求"""
client = await self._ensure_client_initialized()
async with self._lock:
self._active_requests += 1
try:
return await client.request(method, url, **kwargs)
finally:
async with self._lock:
self._active_requests -= 1
async def post(self, url: str, **kwargs: Any) -> httpx.Response:
"""发送异步 POST 请求"""
return await self.request("POST", url, **kwargs)
async def close(self):
"""安全关闭 HTTP 客户端并释放连接池"""
async with self._lock:
if self._client and not self._client.is_closed:
logger.debug(
f"LLMHttpClient: 正在关闭,配置: {self.config}. "
f"活跃请求数: {self._active_requests}"
)
if self._active_requests > 0:
logger.warning(
f"LLMHttpClient: 关闭时仍有 {self._active_requests} "
f"个请求处于活跃状态。"
)
await self._client.aclose()
self._client = None
logger.debug(f"配置为 {self.config} 的 LLMHttpClient 已完全关闭。")
@property
def is_closed(self) -> bool:
"""检查底层客户端是否已关闭"""
return self._client is None or self._client.is_closed
class LLMHttpClientManager:
"""[内部 API] 负责管理与复用 LLMHttpClient 连接池。"""
def __init__(self):
"""初始化客户端管理器"""
self._clients: dict[tuple[str, str, int], LLMHttpClient] = {}
self._lock = asyncio.Lock()
def _get_client_key(self, provider_config: ProviderConfig) -> tuple[str, str, int]:
"""获取客户端唯一缓存标识"""
api_base = provider_config.api_base or ""
return (provider_config.api_type, api_base, provider_config.timeout)
async def get_client(self, provider_config: ProviderConfig) -> LLMHttpClient:
"""获取或创建指定配置的 HTTP 客户端实例"""
key = self._get_client_key(provider_config)
async with self._lock:
client = self._clients.get(key)
if client and not client.is_closed:
logger.debug(
f"LLMHttpClientManager: 复用现有的 LLMHttpClient 密钥: {key}"
)
return client
if client and client.is_closed:
logger.debug(
f"LLMHttpClientManager: 发现密钥 {key} 对应的客户端已关闭。"
f"正在创建新的客户端。"
)
logger.debug(f"LLMHttpClientManager: 为密钥 {key} 创建新的 LLMHttpClient")
http_client_config = HttpClientConfig(timeout=provider_config.timeout)
new_client = LLMHttpClient(config=http_client_config)
self._clients[key] = new_client
return new_client
async def shutdown(self):
"""关闭所有托管的客户端连接池"""
async with self._lock:
logger.info(
f"LLMHttpClientManager: 正在关闭。关闭 {len(self._clients)} 个客户端。"
)
close_tasks = [
client.close()
for client in self._clients.values()
if client and not client.is_closed
]
if close_tasks:
await asyncio.gather(*close_tasks, return_exceptions=True)
self._clients.clear()
logger.info("LLMHttpClientManager: 关闭完成。")
http_client_manager = LLMHttpClientManager()
async def create_llm_http_client(
timeout: int = 180,
) -> LLMHttpClient:
"""创建并返回一个新的 HTTP 客户端"""
config = HttpClientConfig(timeout=timeout)
return LLMHttpClient(config)
class HealthStatePersister:
"""后台异步持久化管理器"""
def __init__(self, state: GlobalHealthState, file_path: Path):
"""初始化持久化管理器"""
self.state = state
self.file_path = file_path
self._is_dirty = False
self._lock = asyncio.Lock()
self._watchdog_task: asyncio.Task | None = None
self._stop_event = asyncio.Event()
def start(self):
"""启动后台异步保存定时任务"""
if self._watchdog_task is None or self._watchdog_task.done():
self._stop_event.clear()
self._watchdog_task = asyncio.create_task(self._watchdog_loop())
def mark_dirty(self):
"""标记内存状态已脏,需要存盘"""
self._is_dirty = True
async def _watchdog_loop(self):
"""后台循环检测并持久化脏数据"""
while not self._stop_event.is_set():
await asyncio.sleep(5)
if self._is_dirty:
await self.force_save()
async def force_save(self):
"""强制将内存中的状态同步写入磁盘文件"""
if not self._is_dirty:
return
async with self._lock:
self._is_dirty = False
data_to_save = model_dump(self.state)
try:
self.file_path.parent.mkdir(parents=True, exist_ok=True)
temp_path = self.file_path.with_suffix(".json.tmp")
async with aiofiles.open(temp_path, "w", encoding="utf-8") as f:
await f.write(
json.dumps(data_to_save, ensure_ascii=False, indent=2)
)
if self.file_path.exists():
self.file_path.unlink()
os.rename(temp_path, self.file_path)
except Exception as e:
logger.error(f"保存密钥状态到文件失败: {e}", e=e)
async def stop(self):
"""停止后台任务并保存所有脏状态"""
self._stop_event.set()
if self._watchdog_task:
self._watchdog_task.cancel()
try:
await self._watchdog_task
except asyncio.CancelledError:
pass
await self.force_save()
class KeyRotationManager:
"""专门负责 API Key 的负载均衡与冷却状态维护"""
def __init__(self, state: GlobalHealthState):
"""初始化 API Key 轮询管理器"""
self.state = state
self._provider_key_index: dict[str, int] = {}
def get_next_available_key(
self,
provider_name: str,
api_keys: list[str],
exclude_keys: set[str] | None = None,
strict_mode: bool = False,
) -> str | None:
"""轮询策略获取下一个健康可用的 API Key"""
if not api_keys:
return None
exclude_keys = exclude_keys or set()
provider_state = self.state.providers.setdefault(
provider_name, ProviderHealthStatus()
)
for key in api_keys:
if key not in provider_state.api_keys:
provider_state.api_keys[key] = KeyHealthStatus()
now = time.time()
available_keys = [
key
for key in api_keys
if key not in exclude_keys
and provider_state.api_keys[key].cooldown_until <= now
]
if not available_keys:
if strict_mode:
return None
return api_keys[0]
current_index = self._provider_key_index.get(provider_name, 0)
selected_key = available_keys[current_index % len(available_keys)]
self._provider_key_index[provider_name] = current_index + 1
stats = provider_state.api_keys[selected_key]
total_usage = stats.successes + stats.failures
logger.debug(f"轮询选择API密钥 (使用次数: {total_usage})")
return selected_key
def record_key_success(self, provider_name: str, api_key: str):
"""记录指定 API Key 调用成功状态"""
provider_state = self.state.providers.setdefault(
provider_name, ProviderHealthStatus()
)
stats = provider_state.api_keys.setdefault(api_key, KeyHealthStatus())
stats.cooldown_until = 0.0
stats.successes += 1
stats.status = "HEALTHY"
stats.last_error = None
def record_key_failure(
self,
provider_name: str,
api_key: str,
exception: Exception,
policy: CircuitBreakerPolicy,
):
"""记录并处理指定 API Key 的调用失败冷却"""
now = time.time()
cooldown_duration = 0
error_message = str(exception)
if isinstance(exception, LocationNotSupportedException):
provider_state = self.state.providers.setdefault(
provider_name, ProviderHealthStatus()
)
stats = provider_state.api_keys.setdefault(api_key, KeyHealthStatus())
stats.failures += 1
stats.last_error = error_message[:256]
return
if isinstance(exception, QuotaExceededException):
cooldown_duration = policy.quota_error_cooldown
elif isinstance(exception, AuthenticationException):
cooldown_duration = policy.auth_error_cooldown
elif isinstance(exception, RateLimitException):
cooldown_duration = policy.rate_limit_cooldown
provider_state = self.state.providers.setdefault(
provider_name, ProviderHealthStatus()
)
stats = provider_state.api_keys.setdefault(api_key, KeyHealthStatus())
if cooldown_duration > 0:
stats.cooldown_until = now + cooldown_duration
stats.status = (
"COOLDOWN"
if cooldown_duration < policy.auth_error_cooldown
else "DISABLED"
)
stats.failures += 1
stats.last_error = error_message[:256]
def reset_key_status(self, provider_name: str, api_key: str):
"""重置 API Key 状态为健康"""
provider_state = self.state.providers.setdefault(
provider_name, ProviderHealthStatus()
)
stats = provider_state.api_keys.setdefault(api_key, KeyHealthStatus())
stats.cooldown_until = 0.0
stats.last_error = None
stats.status = "HEALTHY"
class CircuitBreakerManager:
"""专门负责模型路由级别的熔断与探活 (无锁化设计)"""
def __init__(self, state: GlobalHealthState):
"""初始化路由级熔断管理器"""
self.state = state
def is_route_healthy(self, route_name: str, strict_mode: bool = True) -> bool:
"""检查路由节点的健康与熔断状态"""
stats = self.state.routes.get(route_name)
if not stats:
return True
if stats.state == RouteHealthState.CLOSED:
return True
if stats.state == RouteHealthState.OPEN:
if time.time() >= stats.cooldown_until:
stats.state = RouteHealthState.HALF_OPEN
logger.info(
f"🔄 [Route-Level] 节点 '{route_name}' "
"冷却期结束,进入 HALF_OPEN 半开试探状态。"
)
return True
if not strict_mode:
logger.debug(
f"👉 [Route-Level] 节点 '{route_name}' 处于熔断状态(OPEN),"
"但因非严格模式(单模型直调),强制放行执行探活。"
)
return True
return False
if stats.state == RouteHealthState.HALF_OPEN:
if not strict_mode:
return True
return False
return True
def record_route_success(self, route_name: str, latency: float):
"""记录路由请求成功并尝试闭合熔断器"""
stats = self.state.routes.setdefault(route_name, RouteHealthStatus())
stats.successes += 1
total = stats.successes + stats.failures
stats.success_rate = (stats.successes / total) * 100
if stats.latency_ema == 0.0:
stats.latency_ema = latency
else:
stats.latency_ema = 0.2 * latency + 0.8 * stats.latency_ema
if stats.state != RouteHealthState.CLOSED:
logger.info(
f"✅ [Route-Level] 节点 '{route_name}' "
"试探成功!已完全恢复健康状态 (CLOSED)。"
)
stats.state = RouteHealthState.CLOSED
stats.cooldown_until = 0.0
stats.last_error = None
def record_route_failure(
self, route_name: str, exception: Exception, policy: CircuitBreakerPolicy
):
"""记录路由失败并开启熔断状态"""
now = time.time()
cooldown_duration = policy.server_error_cooldown
stats = self.state.routes.setdefault(route_name, RouteHealthStatus())
stats.failures += 1
total = stats.successes + stats.failures
stats.success_rate = (stats.successes / total) * 100
stats.state = RouteHealthState.OPEN
stats.cooldown_until = now + cooldown_duration
stats.last_error = str(exception)[:256]
def get_best_fallback_route(self, route_names: list[str]) -> str:
"""选择处于熔断冷却最少或最健康的备选路由"""
def get_cooldown(name: str) -> float:
stats = self.state.routes.get(name)
return stats.cooldown_until if stats else 0.0
return sorted(route_names, key=get_cooldown)[0]
class HealthManager:
"""全局 AI 健康与遥测门面"""
def __init__(self):
"""初始化健康监测与遥测门面"""
self.state = GlobalHealthState()
self._file_path = DATA_PATH / "ai" / "api_key.json"
self._persister: HealthStatePersister | None = None
self._key_manager = KeyRotationManager(self.state)
self._circuit_manager = CircuitBreakerManager(self.state)
self.policy = CircuitBreakerPolicy()
async def initialize(self):
"""从本地文件异步加载遥测状态"""
if not self._file_path.exists():
logger.debug("未找到遥测状态文件,将使用内存状态启动。")
else:
try:
import aiofiles
logger.debug(f"正在从 {self._file_path} 加载密钥状态...")
async with aiofiles.open(self._file_path, encoding="utf-8") as f:
content = await f.read()
if content:
self.state = parse_as(GlobalHealthState, json.loads(content))
self._key_manager.state = self.state
self._circuit_manager.state = self.state
total_keys = sum(
len(provider.api_keys) for provider in self.state.providers.values()
)
logger.debug(f"成功加载 {total_keys} 个密钥的状态。")
except json.JSONDecodeError:
logger.error(f"遥测状态文件 {self._file_path} 格式错误,无法解析。")
except Exception as e:
logger.error(f"加载遥测状态文件时发生错误: {e}", e=e)
self._persister = HealthStatePersister(self.state, self._file_path)
self._persister.start()
async def shutdown(self):
"""在应用关闭时安全地持久化健康状态"""
if self._persister:
await self._persister.stop()
logger.debug("HealthManager 已在关闭前保存遥测状态。")
async def get_next_available_key(
self,
provider_name: str,
api_keys: list[str],
exclude_keys: set[str] | None = None,
strict_mode: bool = False,
) -> str | None:
"""路由获取下一个可用 API Key"""
return self._key_manager.get_next_available_key(
provider_name, api_keys, exclude_keys, strict_mode
)
def is_route_healthy(self, route_name: str, strict_mode: bool = True) -> bool:
"""路由判断指定模型路由是否健康"""
return self._circuit_manager.is_route_healthy(route_name, strict_mode)
async def record_route_success(self, route_name: str, latency: float):
"""路由记录模型请求成功"""
self._circuit_manager.record_route_success(route_name, latency)
if self._persister:
self._persister.mark_dirty()
async def record_route_failure(self, route_name: str, exception: Exception):
"""路由记录模型请求失败并记录熔断"""
self._circuit_manager.record_route_failure(route_name, exception, self.policy)
if self._persister:
self._persister.mark_dirty()
logger.warning(
f"🚨 [Route-Level] 节点 '{route_name}' 发生服务端故障,"
f"已触发熔断 (OPEN)。错误: {exception}"
)
def get_best_fallback_route(self, route_names: list[str]) -> str:
"""路由获取最佳备选健康节点"""
return self._circuit_manager.get_best_fallback_route(route_names)
async def record_key_success(self, provider_name: str, api_key: str):
"""路由记录 API Key 成功状态"""
self._key_manager.record_key_success(provider_name, api_key)
if self._persister:
self._persister.mark_dirty()
async def record_key_failure(
self,
provider_name: str,
api_key: str,
exception: Exception,
):
"""路由记录 API Key 失败冷却状态"""
self._key_manager.record_key_failure(
provider_name, api_key, exception, self.policy
)
if self._persister:
self._persister.mark_dirty()
key_id = self._get_key_id(api_key)
logger.debug(f"API Key {key_id} 发生失败: {exception}")
async def reset_key_status(self, provider_name: str, api_key: str):
"""路由重置 API Key 的健康状态"""
self._key_manager.reset_key_status(provider_name, api_key)
if self._persister:
self._persister.mark_dirty()
logger.info(f"重置API密钥状态: {self._get_key_id(api_key)}")
def _get_key_id(self, api_key: str) -> str:
"""脱敏获取用于日志的 API Key 摘要 ID"""
if len(api_key) <= 8:
return api_key
return f"{api_key[:4]}...{api_key[-4:]}"
health_manager = HealthManager()
@driver.on_shutdown
async def _shutdown_health_manager():
await health_manager.shutdown()