Diuse/zhenxun_bot
1
0
Fork 0
mirror of https://github.com/zhenxun-org/zhenxun_bot.git synced 2026-10-17 18:20:01 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

✨ feat!(llm): 重构并升级大语言模型服务为全新 AI 智能体框架 (#2146)

Browse Source 
* ✨ feat!(llm): 重构并升级大语言模型服务为全新 AI 智能体框架

- 【重构】将原 services/llm 重构并迁移至全新的 services/ai 架构，提供向下兼容垫片
- 【新增】引入 Agent、Team、Workflow 三大智能体与工作流编排范式
- 【新增】引入基于 RAG 的长期向量记忆与中期槽位记忆系统
- 【新增】引入基于 Docker 的安全代码执行沙箱环境
- 【新增】支持 MCP 协议，允许动态管理和调用 MCP 服务
- 【新增】引入输入输出安全合规护栏与自愈反思机制
- 【优化】重构并优化多厂商 API 适配器 (Gemini, OpenAI, DeepSeek, GLM 等)
- 【优化】优化日志脱敏与 Token 预估机制
- 【移除】移除旧版 llm default 和 llm reset-key 命令，新增 llm mcp 管理命令

* 🔧 chore(deps): 更新项目依赖与配置

- 添加 mcp、jieba 和 aiodocker 依赖到配置文件及 requirements.txt
- 在 pyright 配置中设置 reportMissingImports 为 none
- 调整 .gitignore 中 resources 目录的忽略规则

* ♻️ refactor(tools): 重构工具终止机制并清理知识库日志输出

- 统一使用 `context.state["__end_run__"]` 替代 `EndRunResult` 控制任务结束
- 移除文件系统和向量知识库检索工具中 `ToolResult` 的 `.with_log` 调用
- 调整指令处理器（Directive）的返回值为 `tool_res.output`
- 修复部分类型检查警告并优化联合类型判断语法

* ♻️ refactor(tools): 重构工具副作用指令与控制流熔断机制

- 引入 `DirectivePayload` 及 `ToolResult` 的子类以结构化表达工具副作用
- 移除通过 `context.state` 传递魔术变量的隐式控制流设计
- 重构 `DirectiveManager` 处理器接口，直接在处理器中修改 `AgentState` 并构建 `AgentRunResult`
- 在 `StandardAgentExecutor` 中统一通过 `directive_manager` 调度工具返回的副作用指令
- 补全 `MessageBuilder` 中部分核心方法的文档注释

* 🐛 fix(sandbox): 修复 Docker 沙箱容器状态检测与会话清理逻辑

-【修复】修正 `is_alive` 中直接读取私有属性的问题，改用 `show()` 返回值
-【修复】解决 `execute_code` 中缓存的执行器与当前会话不一致的问题
-【优化】在清理工作区前增加容器存活检测，避免向已死容器发送请求
-【优化】创建容器时增加运行状态校验，若已停止则自动从缓存中移除并重建
-【优化】优化容器销毁和清理逻辑，静默处理容器不存在 (404) 的异常

* 📝 docs(core): 补充核心模块初始化方法的文档注释

* 🚨 auto fix by pre-commit hooks

---------

Co-authored-by: webjoin111 <455457521@qq.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Rumio
2026-07-03 08:53:56 +08:00
committed by GitHub
co-authored by webjoin111 pre-commit-ci[bot]
parent bdc1374848
commit 80fc5b86a7
223 changed files with 41885 additions and 10006 deletions
Download Patch File Download Diff File Expand all files Collapse all files
zhenxun/services/llm/__init__.py
+359 -92
View File
@@ -1,120 +1,387 @@
"""
LLM 服务模块 - 公共 API 入口
向下兼容门面 (Facade)
提供统一的 AI 服务调用接口、核心类型定义和模型管理功能。
利用 sys.modules 魔法，将对 zhenxun.services.llm 及其子模块的导入
无缝重定向到 zhenxun.services.ai.llm 下，避免破坏任何第三方插件。
同时在此提供最常用的类导出，作为 LLM 服务的统一入口。
"""
from .api import (
chat,
code,
create_image,
embed,
embed_documents,
embed_query,
generate,
generate_structured,
search,
)
from .config import (
CommonOverrides,
GenConfigBuilder,
LLMGenerationConfig,
register_llm_configs,
import sys
from types import ModuleType
from typing import Any
from zhenxun.services import logger
logger.warning(
"zhenxun.services.llm 已被重构迁移至 zhenxun.services.ai.llm，"
"为了更好的性能和兼容性，请及时更新您的插件导入路径。"
)
register_llm_configs()
from .api import ModelName
from .manager import (
clear_model_cache,
get_cache_stats,
get_global_default_model_name,
from pydantic import BaseModel, ConfigDict
import zhenxun.services.ai.core
import zhenxun.services.ai.core.exceptions
import zhenxun.services.ai.core.messages
from zhenxun.services.ai.core.messages import ChatResponse as LLMResponse
from zhenxun.services.ai.core.messages import LLMContentPart, ToolCallPart
import zhenxun.services.ai.core.models
from zhenxun.services.ai.core.models import ToolDefinition
import zhenxun.services.ai.core.options
import zhenxun.services.ai.core.protocols
from zhenxun.services.ai.core.protocols.tool import ToolExecutable
import zhenxun.services.ai.llm
import zhenxun.services.ai.llm.system.capabilities
from zhenxun.services.ai.run import RunContext as RealRunContext
import zhenxun.services.ai.tools
from zhenxun.services.ai.tools.models import ToolResult
_legacy_types_module = ModuleType("zhenxun.services.llm.types")
for _mod in (
zhenxun.services.ai.core.options,
zhenxun.services.ai.core.exceptions,
zhenxun.services.ai.core.messages,
zhenxun.services.ai.core.models,
):
for _name in dir(_mod):
if not _name.startswith("_"):
setattr(_legacy_types_module, _name, getattr(_mod, _name))
sys.modules["zhenxun.services.llm.types"] = _legacy_types_module
sys.modules["zhenxun.services.llm.types.models"] = zhenxun.services.ai.core.models
sys.modules["zhenxun.services.llm.types.protocols"] = zhenxun.services.ai.core.protocols
sys.modules["zhenxun.services.llm.types.exceptions"] = (
zhenxun.services.ai.core.exceptions
)
sys.modules["zhenxun.services.llm.types.capabilities"] = (
zhenxun.services.ai.llm.system.capabilities
)
prefix_old = "zhenxun.services.llm"
prefix_new = "zhenxun.services.ai.llm"
for module_name, module_obj in list(sys.modules.items()):
if module_name.startswith(prefix_new):
old_module_name = module_name.replace(prefix_new, prefix_old, 1)
if old_module_name not in sys.modules:
sys.modules[old_module_name] = module_obj
sys.modules["zhenxun.services.llm.tools"] = zhenxun.services.ai.tools
class LLMToolFunction(BaseModel):
name: str
arguments: str
class LLMToolCall(BaseModel):
id: str
function: LLMToolFunction
thought_signature: str | None = None
type: str = "function"
class _FakeFunction:
def __init__(self, name, args):
self.name = name
self.arguments = (
args
if isinstance(args, str)
else __import__("json").dumps(args, ensure_ascii=False)
)
@property
def _legacy_function(self) -> _FakeFunction:
return _FakeFunction(self.tool_name, self.args)
setattr(ToolCallPart, "function", _legacy_function) # type: ignore
@property
def _legacy_thought_signature(self) -> Any:
return self.metadata.get("thought_signature") if self.metadata else None
@_legacy_thought_signature.setter
def _legacy_thought_signature(self, value: Any) -> None:
if self.metadata is None:
self.metadata = {}
self.metadata["thought_signature"] = value
setattr(ToolCallPart, "thought_signature", _legacy_thought_signature) # type: ignore
from zhenxun.services.ai.core.messages import LLMMessage
_original_assistant_tool_calls = LLMMessage.assistant_tool_calls
@classmethod
def _shim_assistant_tool_calls(
cls, tool_calls: Any, content: Any = "", scope: Any = None
) -> Any:
converted = []
for tc in tool_calls:
if hasattr(tc, "function") and not isinstance(tc, ToolCallPart):
converted.append(
ToolCallPart(
id=tc.id, tool_name=tc.function.name, args=tc.function.arguments
)
)
else:
converted.append(tc)
return _original_assistant_tool_calls(converted, content, scope) # type: ignore
setattr(LLMMessage, "assistant_tool_calls", _shim_assistant_tool_calls) # type: ignore
if not hasattr(LLMMessage, "name"):
@property
def _legacy_name(self) -> Any:
return getattr(self, "source_name", None)
@_legacy_name.setter
def _legacy_name(self, value: Any) -> None:
self.source_name = value
setattr(LLMMessage, "name", _legacy_name) # type: ignore
if not hasattr(LLMMessage, "tool_call_id"):
@property
def _legacy_tool_call_id(self) -> Any:
return None
@_legacy_tool_call_id.setter
def _legacy_tool_call_id(self, value: Any) -> None:
pass
setattr(LLMMessage, "tool_call_id", _legacy_tool_call_id) # type: ignore
from zhenxun.services.ai.core.options import GenerationConfig, ReasoningEffort
mod_config_gen = ModuleType("zhenxun.services.llm.config.generation")
sys.modules["zhenxun.services.llm.config"] = ModuleType("zhenxun.services.llm.config")
sys.modules["zhenxun.services.llm.config.generation"] = mod_config_gen
class ReasoningConfig(BaseModel):
model_config = ConfigDict(extra="allow") # type: ignore
class ToolConfig(BaseModel):
model_config = ConfigDict(extra="allow") # type: ignore
setattr(mod_config_gen, "LLMGenerationConfig", GenerationConfig) # type: ignore
setattr(mod_config_gen, "ReasoningConfig", ReasoningConfig) # type: ignore
setattr(mod_config_gen, "ToolConfig", ToolConfig) # type: ignore
setattr(mod_config_gen, "ReasoningEffort", ReasoningEffort) # type: ignore
_target_models_mod = sys.modules["zhenxun.services.llm.types.models"]
setattr(_target_models_mod, "ToolResult", ToolResult) # type: ignore
setattr(_target_models_mod, "LLMToolCall", LLMToolCall) # type: ignore
setattr(_target_models_mod, "LLMToolFunction", LLMToolFunction) # type: ignore
setattr(_target_models_mod, "LLMContentPart", LLMContentPart) # type: ignore
setattr(_target_models_mod, "ToolDefinition", ToolDefinition) # type: ignore
setattr(_target_models_mod, "LLMResponse", LLMResponse) # type: ignore
setattr(_target_models_mod, "LLMMessage", LLMMessage) # type: ignore
setattr(
sys.modules["zhenxun.services.llm.types.protocols"],
"ToolExecutable",
ToolExecutable,
) # type: ignore
class LegacyRunContextShim:
"""拦截旧版 RunContext(extra={...}) 的调用，转化为新版支持的 state"""
def __new__(cls, session_id=None, extra=None, scope=None, **kwargs):
state = kwargs.get("state", {})
if extra:
state.update(extra)
if scope:
state.update(scope)
ctx = RealRunContext(session_id=session_id, state=state)
ctx.extra = ctx.state # type: ignore
ctx.scope = ctx.state # type: ignore
return ctx
setattr(sys.modules["zhenxun.services.llm.tools"], "RunContext", LegacyRunContextShim) # type: ignore
class ToolInvoker:
"""向下兼容垫片：代替被重构移除的旧版 ToolInvoker"""
def __init__(self, callbacks=None):
self.callbacks = callbacks or []
async def execute_tool_call(self, tool_call, available_tools, context=None):
import json
if hasattr(tool_call, "function") and hasattr(tool_call.function, "name"):
tool_name = tool_call.function.name
args_raw = tool_call.function.arguments
else:
tool_name = getattr(tool_call, "tool_name", "unknown")
args_raw = getattr(tool_call, "args", "{}")
arguments = {}
if args_raw:
if isinstance(args_raw, str):
try:
arguments = json.loads(args_raw)
except Exception:
pass
elif isinstance(args_raw, dict):
arguments = args_raw
executable = available_tools.get(tool_name)
if not executable:
return tool_call, ToolResult(output=f"Error: Tool '{tool_name}' not found.")
try:
result = await executable.execute(context=context, **arguments)
if not hasattr(result, "output"):
result = ToolResult(output=result)
return tool_call, result
except Exception as e:
return tool_call, ToolResult(output=f"System Execution Error: {e!s}")
import zhenxun.services.ai.tools
setattr(zhenxun.services.ai.tools, "ToolInvoker", ToolInvoker) # type: ignore
setattr(sys.modules["zhenxun.services.llm.tools"], "ToolInvoker", ToolInvoker) # type: ignore
class CommonOverrides:
"""向下兼容垫片：由于该类已被废弃，此处提供空实现以防止旧插件导入报错。"""
@staticmethod
def _fallback(*args, **kwargs):
from zhenxun.services.ai.llm.builder import IntentBuilder
return IntentBuilder().build()
gemini_json = _fallback
gemini_2_5_thinking = _fallback
gemini_3_thinking = _fallback
gemini_structured = _fallback
gemini_safe = _fallback
gemini_code_execution = _fallback
gemini_grounding = _fallback
gemini_nano_banana = _fallback
gemini_high_res = _fallback
from zhenxun.services.ai.core.options import GenerationConfig, OutputFormatConfig
class OutputConfig(OutputFormatConfig):
"""向下兼容垫片：旧版 OutputConfig 等价于 OutputFormatConfig。"""
AIConfig = GenerationConfig
LLMGenerationConfig = GenerationConfig
from zhenxun.services.ai.llm import * # noqa: F403
from zhenxun.services.ai.llm.manager import (
get_default_model,
get_model_instance,
list_available_models,
list_embedding_models,
list_model_identifiers,
set_global_default_model_name,
)
from .memory import (
AIConfig,
BaseMemory,
MemoryProcessor,
set_default_memory_backend,
)
from .session import AI
from .tools import RunContext, ToolInvoker, function_tool, tool_provider_manager
from .types import (
EmbeddingTaskType,
LLMContentPart,
LLMErrorCode,
LLMException,
LLMMessage,
LLMResponse,
ModelDetail,
ModelInfo,
ModelProvider,
ResponseFormat,
TaskType,
ToolCategory,
ToolMetadata,
UsageInfo,
)
from .types.models import (
GeminiCodeExecution,
GeminiGoogleSearch,
GeminiUrlContext,
)
from .utils import create_multimodal_message, message_to_unimessage, unimsg_to_llm_parts
from zhenxun.services.ai.message_builder import MessageBuilder
message_to_unimessage = MessageBuilder.message_to_unimessage
unimsg_to_llm_parts = MessageBuilder.unimsg_to_llm_parts
from zhenxun.services.ai.core.exceptions import LLMException
from zhenxun.services.ai.core.messages import ChatRequest
from zhenxun.services.ai.core.messages import ChatResponse as LLMResponse
from zhenxun.services.ai.llm.api import generate_structured as _new_generate_structured
from zhenxun.services.ai.llm.engine.router import LLMOrchestrator
from zhenxun.services.ai.tools import tool as function_tool
class AI:
"""向下兼容垫片：代替被彻底删除的旧版 AI 类"""
def __init__(self, session_id=None, **kwargs):
self.session_id = session_id
async def generate_internal(
self,
messages,
model=None,
config=None,
tools=None,
tool_choice=None,
timeout=None,
):
req = ChatRequest(
messages=messages,
config=config,
tools=tools,
tool_choice=tool_choice,
timeout=timeout,
)
if self.session_id:
req.extra["session_id"] = self.session_id
return await LLMOrchestrator.invoke(req, model_name=model, task="chat")
async def generate_structured(
self,
message,
response_model,
model=None,
tools=None,
tool_choice=None,
instruction=None,
timeout=None,
template_vars=None,
config=None,
max_validation_retries=None,
validation_callback=None,
error_prompt_template=None,
auto_thinking=False,
):
return await _new_generate_structured(
message=message,
response_model=response_model,
model=model,
instruction=instruction,
timeout=timeout,
config=config,
max_retries=max_validation_retries,
error_prompt_template=error_prompt_template,
)
__all__ = [
"AI",
"AIConfig",
"BaseMemory",
"CommonOverrides",
"EmbeddingTaskType",
"GeminiCodeExecution",
"GeminiGoogleSearch",
"GeminiUrlContext",
"GenConfigBuilder",
"GenerationConfig",
"LLMContentPart",
"LLMErrorCode",
"LLMException",
"LLMGenerationConfig",
"LLMMessage",
"LLMResponse",
"MemoryProcessor",
"ModelDetail",
"ModelInfo",
"ModelName",
"ModelProvider",
"ResponseFormat",
"RunContext",
"TaskType",
"ToolCategory",
"LLMToolCall",
"LLMToolFunction",
"OutputConfig",
"ToolDefinition",
"ToolExecutable",
"ToolInvoker",
"ToolMetadata",
"UsageInfo",
"chat",
"clear_model_cache",
"code",
"create_image",
"create_multimodal_message",
"embed",
"embed_documents",
"embed_query",
"ToolResult",
"function_tool",
"generate",
"generate_structured",
"get_cache_stats",
"get_global_default_model_name",
"get_default_model",
"get_model_instance",
"list_available_models",
"list_embedding_models",
"list_model_identifiers",
"message_to_unimessage",
"register_llm_configs",
"search",
"set_default_memory_backend",
"set_global_default_model_name",
"tool_provider_manager",
"unimsg_to_llm_parts",
]
zhenxun/services/llm/adapters/__init__.py
-26
View File
@@ -1,26 +0,0 @@
"""
LLM 适配器模块
提供不同LLM服务商的API适配器实现，统一接口调用方式。
"""
from .base import BaseAdapter, OpenAICompatAdapter, RequestData, ResponseData
from .factory import LLMAdapterFactory, get_adapter_for_api_type, register_adapter
from .gemini import GeminiAdapter
from .openai import DeepSeekAdapter, OpenAIAdapter, OpenAIImageAdapter
LLMAdapterFactory.initialize()
__all__ = [
"BaseAdapter",
"DeepSeekAdapter",
"GeminiAdapter",
"LLMAdapterFactory",
"OpenAIAdapter",
"OpenAICompatAdapter",
"OpenAIImageAdapter",
"RequestData",
"ResponseData",
"get_adapter_for_api_type",
"register_adapter",
]
zhenxun/services/llm/adapters/base.py
-595
View File
@@ -1,595 +0,0 @@
"""
LLM 适配器基类和通用数据结构
"""
from abc import ABC, abstractmethod
import json
from pathlib import Path
from typing import TYPE_CHECKING, Any
import uuid
import httpx
from pydantic import BaseModel
from zhenxun.configs.path_config import TEMP_PATH
from zhenxun.services.log import logger
from ..types import LLMContentPart
from ..types.exceptions import LLMErrorCode, LLMException
from ..types.models import LLMToolCall
if TYPE_CHECKING:
from ..config.generation import LLMEmbeddingConfig, LLMGenerationConfig
from ..service import LLMModel
from ..types import LLMMessage
from ..types.models import ToolChoice
class RequestData(BaseModel):
"""请求数据封装"""
url: str
headers: dict[str, str]
body: dict[str, Any]
files: dict[str, Any] | list[tuple[str, Any]] | None = None
class ResponseData(BaseModel):
"""响应数据封装 - 支持所有高级功能"""
text: str
content_parts: list[LLMContentPart] | None = None
images: list[bytes | Path] | None = None
usage_info: dict[str, Any] | None = None
raw_response: dict[str, Any] | None = None
tool_calls: list[LLMToolCall] | None = None
code_executions: list[Any] | None = None
grounding_metadata: Any | None = None
cache_info: Any | None = None
thought_text: str | None = None
thought_signature: str | None = None
code_execution_results: list[dict[str, Any]] | None = None
search_results: list[dict[str, Any]] | None = None
function_calls: list[dict[str, Any]] | None = None
safety_ratings: list[dict[str, Any]] | None = None
citations: list[dict[str, Any]] | None = None
def process_image_data(image_data: bytes) -> bytes | Path:
"""
处理图片数据：若超过 2MB 则保存到临时目录，避免占用内存。
"""
max_inline_size = 2 * 1024 * 1024
if len(image_data) > max_inline_size:
save_dir = TEMP_PATH / "llm"
save_dir.mkdir(parents=True, exist_ok=True)
file_name = f"{uuid.uuid4()}.png"
file_path = save_dir / file_name
file_path.write_bytes(image_data)
logger.info(
f"图片数据过大 ({len(image_data)} bytes)，已保存到临时文件: {file_path}",
"LLMAdapter",
)
return file_path.resolve()
return image_data
class BaseAdapter(ABC):
"""LLM API适配器基类"""
@property
def log_sanitization_context(self) -> str:
"""用于日志清洗的上下文名称，默认 'default'"""
return "default"
@property
@abstractmethod
def api_type(self) -> str:
"""API类型标识"""
pass
@property
@abstractmethod
def supported_api_types(self) -> list[str]:
"""支持的API类型列表"""
pass
async def prepare_simple_request(
self,
model: "LLMModel",
api_key: str,
prompt: str,
history: list[dict[str, str]] | None = None,
) -> RequestData:
"""准备简单文本生成请求
默认实现：将简单请求转换为高级请求格式
子类可以重写此方法以提供特定的优化实现
"""
from ..types import LLMMessage
messages: list[LLMMessage] = []
if history:
for msg in history:
role = msg.get("role", "user")
content = msg.get("content", "")
messages.append(LLMMessage(role=role, content=content))
messages.append(LLMMessage(role="user", content=prompt))
config = model._generation_config
return await self.prepare_advanced_request(
model=model,
api_key=api_key,
messages=messages,
config=config,
tools=None,
tool_choice=None,
)
@abstractmethod
async def prepare_advanced_request(
self,
model: "LLMModel",
api_key: str,
messages: list["LLMMessage"],
config: "LLMGenerationConfig | None" = None,
tools: list[Any] | None = None,
tool_choice: "str | dict[str, Any] | ToolChoice | None" = None,
) -> RequestData:
"""准备高级请求"""
pass
@abstractmethod
def parse_response(
self,
model: "LLMModel",
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
"""解析API响应"""
pass
@abstractmethod
def prepare_embedding_request(
self,
model: "LLMModel",
api_key: str,
texts: list[str],
config: "LLMEmbeddingConfig",
) -> RequestData:
"""准备文本嵌入请求"""
pass
@abstractmethod
def parse_embedding_response(
self, response_json: dict[str, Any]
) -> list[list[float]]:
"""解析文本嵌入响应"""
pass
@abstractmethod
def convert_generation_config(
self, config: "LLMGenerationConfig", model: "LLMModel"
) -> dict[str, Any]:
"""将通用生成配置转换为特定API的参数字典"""
pass
def validate_embedding_response(self, response_json: dict[str, Any]) -> None:
"""验证嵌入API响应"""
if response_json.get("error"):
error_info = response_json["error"]
msg = (
error_info.get("message", str(error_info))
if isinstance(error_info, dict)
else str(error_info)
)
raise LLMException(
f"嵌入API错误: {msg}",
code=LLMErrorCode.EMBEDDING_FAILED,
details=response_json,
)
def get_api_url(self, model: "LLMModel", endpoint: str) -> str:
"""构建API URL"""
if not model.api_base:
raise LLMException(
f"模型 {model.model_name} 的 api_base 未设置",
code=LLMErrorCode.CONFIGURATION_ERROR,
)
return f"{model.api_base.rstrip('/')}{endpoint}"
def _get_provider_extra_headers(self, model: "LLMModel | None") -> dict[str, str]:
if not model:
return {}
raw_headers = getattr(model.provider_config, "extra_headers", None)
if not isinstance(raw_headers, dict):
return {}
headers: dict[str, str] = {}
for key, value in raw_headers.items():
key_text = str(key).strip()
if not key_text or value is None:
continue
headers[key_text] = str(value)
return headers
def get_base_headers(
self, api_key: str, model: "LLMModel | None" = None
) -> dict[str, str]:
"""获取基础请求头"""
from zhenxun.utils.user_agent import get_user_agent
headers = get_user_agent()
headers.update(
{
"Content-Type": "application/json",
"Authorization": f"Bearer {api_key}",
}
)
headers.update(self._get_provider_extra_headers(model))
return headers
def validate_response(self, response_json: dict[str, Any]) -> None:
"""验证API响应，解析不同API的错误结构"""
if response_json.get("error"):
error_info = response_json["error"]
if isinstance(error_info, dict):
error_message = error_info.get("message", "未知错误")
error_code = error_info.get("code", "unknown")
error_type = error_info.get("type", "api_error")
error_code_mapping = {
"invalid_api_key": LLMErrorCode.API_KEY_INVALID,
"authentication_failed": LLMErrorCode.API_KEY_INVALID,
"insufficient_quota": LLMErrorCode.API_QUOTA_EXCEEDED,
"rate_limit_exceeded": LLMErrorCode.API_RATE_LIMITED,
"quota_exceeded": LLMErrorCode.API_RATE_LIMITED,
"model_not_found": LLMErrorCode.MODEL_NOT_FOUND,
"invalid_model": LLMErrorCode.MODEL_NOT_FOUND,
"context_length_exceeded": LLMErrorCode.CONTEXT_LENGTH_EXCEEDED,
"max_tokens_exceeded": LLMErrorCode.CONTEXT_LENGTH_EXCEEDED,
"invalid_request_error": LLMErrorCode.INVALID_PARAMETER,
"invalid_parameter": LLMErrorCode.INVALID_PARAMETER,
}
llm_error_code = error_code_mapping.get(
error_code, LLMErrorCode.API_RESPONSE_INVALID
)
logger.error(
f"API返回错误: {error_message} "
f"(代码: {error_code}, 类型: {error_type})"
)
else:
error_message = str(error_info)
error_code = "unknown"
llm_error_code = LLMErrorCode.API_RESPONSE_INVALID
logger.error(f"API返回错误: {error_message}")
raise LLMException(
f"API请求失败: {error_message}",
code=llm_error_code,
details={"api_error": error_info, "error_code": error_code},
)
if "candidates" in response_json:
candidates = response_json.get("candidates", [])
if candidates:
candidate = candidates[0]
finish_reason = candidate.get("finishReason")
if finish_reason in ["SAFETY", "RECITATION"]:
safety_ratings = candidate.get("safetyRatings", [])
logger.warning(
f"Gemini内容被安全过滤: {finish_reason}, "
f"安全评级: {safety_ratings}"
)
raise LLMException(
f"内容被安全过滤: {finish_reason}",
code=LLMErrorCode.CONTENT_FILTERED,
details={
"finish_reason": finish_reason,
"safety_ratings": safety_ratings,
},
)
if not response_json:
logger.error("API返回空响应")
raise LLMException(
"API返回空响应",
code=LLMErrorCode.API_RESPONSE_INVALID,
details={"response": response_json},
)
def _apply_generation_config(
self,
model: "LLMModel",
config: "LLMGenerationConfig | None" = None,
) -> dict[str, Any]:
"""通用的配置应用逻辑"""
if config is not None:
return self.convert_generation_config(config, model)
if model._generation_config:
return self.convert_generation_config(model._generation_config, model)
return {}
def apply_config_override(
self,
model: "LLMModel",
body: dict[str, Any],
config: "LLMGenerationConfig | None" = None,
) -> dict[str, Any]:
"""应用配置覆盖"""
config_params = self._apply_generation_config(model, config)
body.update(config_params)
return body
def handle_http_error(self, response: httpx.Response) -> LLMException | None:
"""
处理 HTTP 错误响应。
如果响应状态码表示成功 (200)，返回 None；否则构造 LLMException 供外部捕获。
"""
if response.status_code == 200:
return None
error_text = response.content.decode("utf-8", errors="ignore")
error_status = ""
error_msg = error_text
try:
error_json = json.loads(error_text)
if isinstance(error_json, dict) and "error" in error_json:
error_info = error_json["error"]
if isinstance(error_info, dict):
error_msg = error_info.get("message", error_msg)
raw_status = error_info.get("status") or error_info.get("code")
error_status = str(raw_status) if raw_status is not None else ""
elif error_info is not None:
error_msg = str(error_info)
error_status = error_msg
except Exception:
pass
status_upper = error_status.upper() if error_status else ""
text_upper = error_text.upper()
error_code = LLMErrorCode.API_REQUEST_FAILED
if response.status_code == 400:
if (
"FAILED_PRECONDITION" in status_upper
or "LOCATION IS NOT SUPPORTED" in text_upper
):
error_code = LLMErrorCode.USER_LOCATION_NOT_SUPPORTED
elif "INVALID_ARGUMENT" in status_upper:
error_code = LLMErrorCode.INVALID_PARAMETER
elif "API_KEY_INVALID" in text_upper or "API KEY NOT VALID" in text_upper:
error_code = LLMErrorCode.API_KEY_INVALID
else:
error_code = LLMErrorCode.INVALID_PARAMETER
elif response.status_code in [401, 403]:
if error_msg and (
"country" in error_msg.lower()
or "region" in error_msg.lower()
or "unsupported" in error_msg.lower()
):
error_code = LLMErrorCode.USER_LOCATION_NOT_SUPPORTED
elif "PERMISSION_DENIED" in status_upper:
error_code = LLMErrorCode.API_KEY_INVALID
else:
error_code = LLMErrorCode.API_KEY_INVALID
elif response.status_code == 404:
error_code = LLMErrorCode.MODEL_NOT_FOUND
elif response.status_code == 429:
if (
"RESOURCE_EXHAUSTED" in status_upper
or "INSUFFICIENT_QUOTA" in status_upper
or ("quota" in error_msg.lower() if error_msg else False)
):
error_code = LLMErrorCode.API_QUOTA_EXCEEDED
else:
error_code = LLMErrorCode.API_RATE_LIMITED
elif response.status_code in [402, 413]:
error_code = LLMErrorCode.API_QUOTA_EXCEEDED
elif response.status_code == 422:
error_code = LLMErrorCode.GENERATION_FAILED
elif response.status_code >= 500:
error_code = LLMErrorCode.API_TIMEOUT
return LLMException(
f"HTTP请求失败: {response.status_code} ({error_status or 'Unknown'})",
code=error_code,
details={
"status_code": response.status_code,
"api_status": error_status,
"response": error_text,
},
)
class OpenAICompatAdapter(BaseAdapter):
"""
处理所有 OpenAI 兼容 API 的通用适配器。
"""
@property
def log_sanitization_context(self) -> str:
return "openai_request"
@abstractmethod
def get_chat_endpoint(self, model: "LLMModel") -> str:
"""子类必须实现，返回 chat completions 的端点"""
pass
@abstractmethod
def get_embedding_endpoint(self, model: "LLMModel") -> str:
"""子类必须实现，返回 embeddings 的端点"""
pass
async def prepare_simple_request(
self,
model: "LLMModel",
api_key: str,
prompt: str,
history: list[dict[str, str]] | None = None,
) -> RequestData:
"""准备简单文本生成请求 - OpenAI兼容API的通用实现"""
url = self.get_api_url(model, self.get_chat_endpoint(model))
headers = self.get_base_headers(api_key, model)
messages = []
if history:
messages.extend(history)
messages.append({"role": "user", "content": prompt})
body = {
"model": model.model_name,
"messages": messages,
}
body = self.apply_config_override(model, body)
return RequestData(url=url, headers=headers, body=body)
async def prepare_advanced_request(
self,
model: "LLMModel",
api_key: str,
messages: list["LLMMessage"],
config: "LLMGenerationConfig | None" = None,
tools: list[Any] | None = None,
tool_choice: "str | dict[str, Any] | ToolChoice | None" = None,
) -> RequestData:
"""准备高级请求 - OpenAI兼容格式"""
url = self.get_api_url(model, self.get_chat_endpoint(model))
headers = self.get_base_headers(api_key, model)
if model.api_type == "openrouter":
headers.update(
{
"HTTP-Referer": "https://github.com/zhenxun-org/zhenxun_bot",
"X-Title": "Zhenxun Bot",
}
)
from .components.openai_components import OpenAIMessageConverter
converter = OpenAIMessageConverter()
openai_messages = converter.convert_messages(messages)
body = {
"model": model.model_name,
"messages": openai_messages,
}
openai_tools: list[dict[str, Any]] | None = None
executables: list[Any] = []
if tools:
for tool in tools:
if hasattr(tool, "get_definition"):
executables.append(tool)
if executables:
import asyncio
from zhenxun.utils.pydantic_compat import model_dump
definition_tasks = [
executable.get_definition() for executable in executables
]
tool_defs = []
if definition_tasks:
tool_defs = await asyncio.gather(*definition_tasks)
if tool_defs:
openai_tools = [
{"type": "function", "function": model_dump(tool)}
for tool in tool_defs
]
if openai_tools:
body["tools"] = openai_tools
if tool_choice:
body["tool_choice"] = tool_choice
body = self.apply_config_override(model, body, config)
return RequestData(url=url, headers=headers, body=body)
def parse_response(
self,
model: "LLMModel",
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
"""解析响应 - 直接使用组件化 ResponseParser"""
_ = model, is_advanced
from .components.openai_components import OpenAIResponseParser
parser = OpenAIResponseParser()
return parser.parse(response_json)
def prepare_embedding_request(
self,
model: "LLMModel",
api_key: str,
texts: list[str],
config: "LLMEmbeddingConfig",
) -> RequestData:
"""准备嵌入请求 - OpenAI兼容格式"""
url = self.get_api_url(model, self.get_embedding_endpoint(model))
headers = self.get_base_headers(api_key, model)
body = {
"model": model.model_name,
"input": texts,
}
if config.output_dimensionality:
body["dimensions"] = config.output_dimensionality
if config.task_type:
body["task"] = config.task_type
if config.encoding_format and config.encoding_format != "float":
body["encoding_format"] = config.encoding_format
return RequestData(url=url, headers=headers, body=body)
def parse_embedding_response(
self, response_json: dict[str, Any]
) -> list[list[float]]:
"""解析嵌入响应 - OpenAI兼容格式"""
self.validate_embedding_response(response_json)
try:
data = response_json.get("data", [])
if not data:
raise LLMException(
"嵌入响应中没有数据",
code=LLMErrorCode.EMBEDDING_FAILED,
details=response_json,
)
embeddings = []
for item in data:
if "embedding" in item:
embeddings.append(item["embedding"])
else:
raise LLMException(
"嵌入响应格式错误：缺少embedding字段",
code=LLMErrorCode.EMBEDDING_FAILED,
details=item,
)
return embeddings
except Exception as e:
logger.error(f"解析嵌入响应失败: {e}", e=e)
raise LLMException(
f"解析嵌入响应失败: {e}",
code=LLMErrorCode.EMBEDDING_FAILED,
cause=e,
)
zhenxun/services/llm/adapters/components/__init__.py
-1
View File
@@ -1 +0,0 @@
zhenxun/services/llm/adapters/components/gemini_components.py
-606
View File
@@ -1,606 +0,0 @@
import base64
import json
from pathlib import Path
from typing import Any
from zhenxun.services.llm.adapters.base import ResponseData, process_image_data
from zhenxun.services.llm.adapters.components.interfaces import (
ConfigMapper,
MessageConverter,
ResponseParser,
ToolSerializer,
)
from zhenxun.services.llm.config.generation import (
ImageAspectRatio,
LLMGenerationConfig,
ReasoningEffort,
ResponseFormat,
)
from zhenxun.services.llm.config.providers import get_gemini_safety_threshold
from zhenxun.services.llm.types import (
CodeExecutionOutcome,
LLMContentPart,
LLMMessage,
)
from zhenxun.services.llm.types.capabilities import ModelCapabilities
from zhenxun.services.llm.types.exceptions import LLMErrorCode, LLMException
from zhenxun.services.llm.types.models import (
LLMGroundingAttribution,
LLMGroundingMetadata,
LLMToolCall,
LLMToolFunction,
ModelDetail,
ToolDefinition,
)
from zhenxun.services.llm.utils import (
resolve_json_schema_refs,
sanitize_schema_for_llm,
)
from zhenxun.services.log import logger
from zhenxun.utils.http_utils import AsyncHttpx
from zhenxun.utils.pydantic_compat import model_copy, model_dump
class GeminiConfigMapper(ConfigMapper):
def map_config(
self,
config: LLMGenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
params: dict[str, Any] = {}
if config.core:
if config.core.temperature is not None:
params["temperature"] = config.core.temperature
if config.core.max_tokens is not None:
params["maxOutputTokens"] = config.core.max_tokens
if config.core.top_k is not None:
params["topK"] = config.core.top_k
if config.core.top_p is not None:
params["topP"] = config.core.top_p
if config.output:
if config.output.response_format == ResponseFormat.JSON:
params["responseMimeType"] = "application/json"
if config.output.response_schema:
params["responseJsonSchema"] = config.output.response_schema
elif config.output.response_mime_type is not None:
params["responseMimeType"] = config.output.response_mime_type
if (
config.output.response_schema is not None
and "responseJsonSchema" not in params
):
params["responseJsonSchema"] = config.output.response_schema
if config.output.response_modalities:
params["responseModalities"] = config.output.response_modalities
if config.tool_config:
fc_config: dict[str, Any] = {"mode": config.tool_config.mode}
if (
config.tool_config.allowed_function_names
and config.tool_config.mode == "ANY"
):
builtins = {"code_execution", "google_search", "google_map"}
user_funcs = [
name
for name in config.tool_config.allowed_function_names
if name not in builtins
]
if user_funcs:
fc_config["allowedFunctionNames"] = user_funcs
params["toolConfig"] = {"functionCallingConfig": fc_config}
if config.reasoning:
thinking_config = params.setdefault("thinkingConfig", {})
if config.reasoning.budget_tokens is not None:
if (
config.reasoning.budget_tokens <= 0
or config.reasoning.budget_tokens >= 1
):
budget_value = int(config.reasoning.budget_tokens)
else:
budget_value = int(config.reasoning.budget_tokens * 32768)
thinking_config["thinkingBudget"] = budget_value
elif config.reasoning.effort:
if config.reasoning.effort == ReasoningEffort.MEDIUM:
thinking_config["thinkingLevel"] = "HIGH"
else:
thinking_config["thinkingLevel"] = config.reasoning.effort.value
if config.reasoning.show_thoughts is not None:
thinking_config["includeThoughts"] = config.reasoning.show_thoughts
elif capabilities and capabilities.reasoning_visibility == "visible":
thinking_config["includeThoughts"] = True
if config.visual:
image_config: dict[str, Any] = {}
if config.visual.aspect_ratio is not None:
ar_value = (
config.visual.aspect_ratio.value
if isinstance(config.visual.aspect_ratio, ImageAspectRatio)
else config.visual.aspect_ratio
)
image_config["aspectRatio"] = ar_value
if config.visual.resolution:
image_config["imageSize"] = config.visual.resolution
if image_config:
params["imageConfig"] = image_config
if config.visual.media_resolution:
media_value = config.visual.media_resolution.upper()
if not media_value.startswith("MEDIA_RESOLUTION_"):
media_value = f"MEDIA_RESOLUTION_{media_value}"
params["mediaResolution"] = media_value
if config.custom_params:
mapped_custom = config.custom_params.copy()
if "max_tokens" in mapped_custom:
mapped_custom["maxOutputTokens"] = mapped_custom.pop("max_tokens")
if "top_k" in mapped_custom:
mapped_custom["topK"] = mapped_custom.pop("top_k")
if "top_p" in mapped_custom:
mapped_custom["topP"] = mapped_custom.pop("top_p")
for key in (
"code_execution_timeout",
"grounding_config",
"dynamic_threshold",
"user_location",
"reflexion_retries",
):
mapped_custom.pop(key, None)
for unsupported in [
"frequency_penalty",
"presence_penalty",
"repetition_penalty",
]:
if unsupported in mapped_custom:
mapped_custom.pop(unsupported)
params.update(mapped_custom)
safety_settings: list[dict[str, Any]] = []
if config.safety and config.safety.safety_settings:
for category, threshold in config.safety.safety_settings.items():
safety_settings.append({"category": category, "threshold": threshold})
else:
threshold = get_gemini_safety_threshold()
for category in [
"HARM_CATEGORY_HARASSMENT",
"HARM_CATEGORY_HATE_SPEECH",
"HARM_CATEGORY_SEXUALLY_EXPLICIT",
"HARM_CATEGORY_DANGEROUS_CONTENT",
]:
safety_settings.append({"category": category, "threshold": threshold})
if safety_settings:
params["safetySettings"] = safety_settings
return params
class GeminiMessageConverter(MessageConverter):
async def convert_part(self, part: LLMContentPart) -> dict[str, Any]:
"""将单个内容部分转换为 Gemini API 格式"""
def _get_gemini_resolution_dict() -> dict[str, Any]:
if part.media_resolution:
value = part.media_resolution.upper()
if not value.startswith("MEDIA_RESOLUTION_"):
value = f"MEDIA_RESOLUTION_{value}"
return {"media_resolution": {"level": value}}
return {}
if part.type == "text":
return {"text": part.text}
if part.type == "thought":
return {"text": part.thought_text, "thought": True}
if part.type == "image":
if not part.image_source:
raise ValueError("图像类型的内容必须包含image_source")
if part.is_image_base64():
base64_info = part.get_base64_data()
if base64_info:
mime_type, data = base64_info
payload = {"inlineData": {"mimeType": mime_type, "data": data}}
payload.update(_get_gemini_resolution_dict())
return payload
raise ValueError(f"无法解析Base64图像数据: {part.image_source[:50]}...")
if part.is_image_url():
logger.debug(f"正在为Gemini下载并编码URL图片: {part.image_source}")
try:
image_bytes = await AsyncHttpx.get_content(part.image_source)
mime_type = part.mime_type or "image/jpeg"
base64_data = base64.b64encode(image_bytes).decode("utf-8")
payload = {
"inlineData": {"mimeType": mime_type, "data": base64_data}
}
payload.update(_get_gemini_resolution_dict())
return payload
except Exception as e:
logger.error(f"下载或编码URL图片失败: {e}", e=e)
raise ValueError(f"无法处理图片URL: {e}")
raise ValueError(f"不支持的图像源格式: {part.image_source[:50]}...")
if part.type == "video":
if not part.video_source:
raise ValueError("视频类型的内容必须包含video_source")
if part.video_source.startswith("data:"):
try:
header, data = part.video_source.split(",", 1)
mime_type = header.split(";")[0].replace("data:", "")
payload = {"inlineData": {"mimeType": mime_type, "data": data}}
payload.update(_get_gemini_resolution_dict())
return payload
except (ValueError, IndexError):
raise ValueError(
f"无法解析Base64视频数据: {part.video_source[:50]}..."
)
raise ValueError(
"Gemini API 的视频处理需要通过 File API 上传，不支持直接 URL"
)
if part.type == "audio":
if not part.audio_source:
raise ValueError("音频类型的内容必须包含audio_source")
if part.audio_source.startswith("data:"):
try:
header, data = part.audio_source.split(",", 1)
mime_type = header.split(";")[0].replace("data:", "")
payload = {"inlineData": {"mimeType": mime_type, "data": data}}
payload.update(_get_gemini_resolution_dict())
return payload
except (ValueError, IndexError):
raise ValueError(
f"无法解析Base64音频数据: {part.audio_source[:50]}..."
)
raise ValueError(
"Gemini API 的音频处理需要通过 File API 上传，不支持直接 URL"
)
if part.type == "file":
if part.file_uri:
payload = {
"fileData": {"mimeType": part.mime_type, "fileUri": part.file_uri}
}
payload.update(_get_gemini_resolution_dict())
return payload
if part.file_source:
file_name = (
part.metadata.get("name", "file") if part.metadata else "file"
)
return {"text": f"[文件: {file_name}]\n{part.file_source}"}
raise ValueError("文件类型的内容必须包含file_uri或file_source")
raise ValueError(f"不支持的内容类型: {part.type}")
async def convert_messages_async(
self, messages: list[LLMMessage]
) -> list[dict[str, Any]]:
gemini_contents: list[dict[str, Any]] = []
for msg in messages:
current_parts: list[dict[str, Any]] = []
if msg.role == "system":
continue
elif msg.role == "user":
if isinstance(msg.content, str):
current_parts.append({"text": msg.content})
elif isinstance(msg.content, list):
for part_obj in msg.content:
current_parts.append(await self.convert_part(part_obj))
gemini_contents.append({"role": "user", "parts": current_parts})
elif msg.role == "assistant" or msg.role == "model":
if isinstance(msg.content, str) and msg.content:
current_parts.append({"text": msg.content})
elif isinstance(msg.content, list):
for part_obj in msg.content:
part_dict = await self.convert_part(part_obj)
if "executableCode" in part_dict:
part_dict["executable_code"] = part_dict.pop(
"executableCode"
)
if "codeExecutionResult" in part_dict:
part_dict["code_execution_result"] = part_dict.pop(
"codeExecutionResult"
)
if (
part_obj.metadata
and "thought_signature" in part_obj.metadata
):
part_dict["thoughtSignature"] = part_obj.metadata[
"thought_signature"
]
current_parts.append(part_dict)
if msg.tool_calls:
for call in msg.tool_calls:
fc_part = {
"functionCall": {
"name": call.function.name,
"args": json.loads(call.function.arguments),
}
}
if call.thought_signature:
fc_part["thoughtSignature"] = call.thought_signature
current_parts.append(fc_part)
if current_parts:
gemini_contents.append({"role": "model", "parts": current_parts})
elif msg.role == "tool":
if not msg.name:
raise ValueError("Gemini 工具消息必须包含 'name' 字段（函数名）。")
try:
content_str = (
msg.content
if isinstance(msg.content, str)
else str(msg.content)
)
tool_result_obj = json.loads(content_str)
except json.JSONDecodeError:
content_str = (
msg.content
if isinstance(msg.content, str)
else str(msg.content)
)
tool_result_obj = {"raw_output": content_str}
if isinstance(tool_result_obj, list):
final_response_payload = {"result": tool_result_obj}
elif not isinstance(tool_result_obj, dict):
final_response_payload = {"result": tool_result_obj}
else:
final_response_payload = tool_result_obj
current_parts.append(
{
"functionResponse": {
"name": msg.name,
"response": final_response_payload,
}
}
)
if gemini_contents and gemini_contents[-1]["role"] == "function":
gemini_contents[-1]["parts"].extend(current_parts)
else:
gemini_contents.append({"role": "function", "parts": current_parts})
return gemini_contents
def convert_messages(self, messages: list[LLMMessage]) -> list[dict[str, Any]]:
raise NotImplementedError("Use convert_messages_async for Gemini")
class GeminiToolSerializer(ToolSerializer):
def serialize_tools(self, tools: list[ToolDefinition]) -> list[dict[str, Any]]:
function_declarations: list[dict[str, Any]] = []
for tool_def in tools:
tool_copy = model_copy(tool_def)
tool_copy.parameters = resolve_json_schema_refs(tool_copy.parameters)
tool_copy.parameters = sanitize_schema_for_llm(
tool_copy.parameters, api_type="gemini"
)
function_declarations.append(model_dump(tool_copy))
return function_declarations
class GeminiResponseParser(ResponseParser):
def validate_response(self, response_json: dict[str, Any]) -> None:
if error := response_json.get("error"):
code = error.get("code")
message = error.get("message", "")
status = error.get("status")
details = error.get("details", [])
if code == 429 or status == "RESOURCE_EXHAUSTED":
is_quota = any(
d.get("reason") in ("QUOTA_EXCEEDED", "SERVICE_DISABLED")
for d in details
if isinstance(d, dict)
)
if is_quota or "quota" in message.lower():
raise LLMException(
f"Gemini配额耗尽: {message}",
code=LLMErrorCode.API_QUOTA_EXCEEDED,
details=error,
)
raise LLMException(
f"Gemini速率限制: {message}",
code=LLMErrorCode.API_RATE_LIMITED,
details=error,
)
if code == 400 or status in ("INVALID_ARGUMENT", "FAILED_PRECONDITION"):
raise LLMException(
f"Gemini参数错误: {message}",
code=LLMErrorCode.INVALID_PARAMETER,
details=error,
recoverable=False,
)
if prompt_feedback := response_json.get("promptFeedback"):
if block_reason := prompt_feedback.get("blockReason"):
raise LLMException(
f"内容被安全过滤: {block_reason}",
code=LLMErrorCode.CONTENT_FILTERED,
details={
"block_reason": block_reason,
"safety_ratings": prompt_feedback.get("safetyRatings"),
},
)
def parse(self, response_json: dict[str, Any]) -> ResponseData:
self.validate_response(response_json)
if "image_generation" in response_json and isinstance(
response_json["image_generation"], dict
):
candidates_source = response_json["image_generation"]
else:
candidates_source = response_json
candidates = candidates_source.get("candidates", [])
usage_info = response_json.get("usageMetadata")
if not candidates:
return ResponseData(text="", raw_response=response_json)
candidate = candidates[0]
thought_signature: str | None = None
content_data = candidate.get("content", {})
parts = content_data.get("parts", [])
text_content = ""
images_payload: list[bytes | Path] = []
parsed_tool_calls: list[LLMToolCall] | None = None
parsed_code_executions: list[dict[str, Any]] = []
content_parts: list[LLMContentPart] = []
thought_summary_parts: list[str] = []
answer_parts = []
for part in parts:
part_signature = part.get("thoughtSignature")
if part_signature and thought_signature is None:
thought_signature = part_signature
part_metadata: dict[str, Any] | None = None
if part_signature:
part_metadata = {"thought_signature": part_signature}
if part.get("thought") is True:
t_text = part.get("text", "")
thought_summary_parts.append(t_text)
content_parts.append(LLMContentPart.thought_part(t_text))
elif "text" in part:
answer_parts.append(part["text"])
c_part = LLMContentPart(
type="text", text=part["text"], metadata=part_metadata
)
content_parts.append(c_part)
elif "thoughtSummary" in part:
thought_summary_parts.append(part["thoughtSummary"])
content_parts.append(
LLMContentPart.thought_part(part["thoughtSummary"])
)
elif "inlineData" in part:
inline_data = part["inlineData"]
if "data" in inline_data:
decoded = base64.b64decode(inline_data["data"])
images_payload.append(process_image_data(decoded))
elif "functionCall" in part:
if parsed_tool_calls is None:
parsed_tool_calls = []
fc_data = part["functionCall"]
fc_sig = part_signature
try:
call_id = f"call_gemini_{len(parsed_tool_calls)}"
parsed_tool_calls.append(
LLMToolCall(
id=call_id,
thought_signature=fc_sig,
function=LLMToolFunction(
name=fc_data["name"],
arguments=json.dumps(fc_data["args"]),
),
)
)
except Exception as e:
logger.warning(
f"解析Gemini functionCall时出错: {fc_data}, 错误: {e}"
)
elif "executableCode" in part:
exec_code = part["executableCode"]
lang = exec_code.get("language", "PYTHON")
code = exec_code.get("code", "")
content_parts.append(LLMContentPart.executable_code_part(lang, code))
answer_parts.append(f"\n[生成代码 ({lang})]:\n```python\n{code}\n```\n")
elif "codeExecutionResult" in part:
result = part["codeExecutionResult"]
outcome = result.get("outcome", CodeExecutionOutcome.OUTCOME_UNKNOWN)
output = result.get("output", "")
content_parts.append(
LLMContentPart.execution_result_part(outcome, output)
)
parsed_code_executions.append(result)
if outcome == CodeExecutionOutcome.OUTCOME_OK:
answer_parts.append(f"\n[代码执行结果]:\n```\n{output}\n```\n")
else:
answer_parts.append(f"\n[代码执行失败 ({outcome})]:\n{output}\n")
full_answer = "".join(answer_parts).strip()
text_content = full_answer
final_thought_text = (
"\n\n".join(thought_summary_parts).strip()
if thought_summary_parts
else None
)
grounding_metadata_obj = None
if grounding_data := candidate.get("groundingMetadata"):
try:
sep_content = None
sep_field = grounding_data.get("searchEntryPoint")
if isinstance(sep_field, dict):
sep_content = sep_field.get("renderedContent")
attributions = []
if chunks := grounding_data.get("groundingChunks"):
for chunk in chunks:
if web := chunk.get("web"):
attributions.append(
LLMGroundingAttribution(
title=web.get("title"),
uri=web.get("uri"),
snippet=web.get("snippet"),
confidence_score=None,
)
)
grounding_metadata_obj = LLMGroundingMetadata(
web_search_queries=grounding_data.get("webSearchQueries"),
grounding_attributions=attributions or None,
search_suggestions=grounding_data.get("searchSuggestions"),
search_entry_point=sep_content,
map_widget_token=grounding_data.get("googleMapsWidgetContextToken"),
)
except Exception as e:
logger.warning(f"无法解析Grounding元数据: {grounding_data}, {e}")
return ResponseData(
text=text_content,
tool_calls=parsed_tool_calls,
code_executions=parsed_code_executions if parsed_code_executions else None,
content_parts=content_parts if content_parts else None,
images=images_payload if images_payload else None,
usage_info=usage_info,
raw_response=response_json,
grounding_metadata=grounding_metadata_obj,
thought_text=final_thought_text,
thought_signature=thought_signature,
)
zhenxun/services/llm/adapters/components/interfaces.py
-43
View File
@@ -1,43 +0,0 @@
from abc import ABC, abstractmethod
from typing import Any
from zhenxun.services.llm.adapters.base import ResponseData
from zhenxun.services.llm.config.generation import LLMGenerationConfig
from zhenxun.services.llm.types import LLMMessage
from zhenxun.services.llm.types.capabilities import ModelCapabilities
from zhenxun.services.llm.types.models import ModelDetail, ToolDefinition
class ConfigMapper(ABC):
@abstractmethod
def map_config(
self,
config: LLMGenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
"""将通用生成配置转换为特定 API 的参数字典"""
...
class MessageConverter(ABC):
@abstractmethod
def convert_messages(
self, messages: list[LLMMessage]
) -> list[dict[str, Any]] | dict[str, Any]:
"""将通用消息列表转换为特定 API 的消息格式"""
...
class ToolSerializer(ABC):
@abstractmethod
def serialize_tools(self, tools: list[ToolDefinition]) -> Any:
"""将通用工具定义转换为特定 API 的工具格式"""
...
class ResponseParser(ABC):
@abstractmethod
def parse(self, response_json: dict[str, Any]) -> ResponseData:
"""将特定 API 的响应解析为通用响应数据"""
...
zhenxun/services/llm/adapters/components/openai_components.py
-347
View File
@@ -1,347 +0,0 @@
import base64
import binascii
import json
from pathlib import Path
from typing import Any
from zhenxun.services.llm.adapters.base import ResponseData, process_image_data
from zhenxun.services.llm.adapters.components.interfaces import (
ConfigMapper,
MessageConverter,
ResponseParser,
ToolSerializer,
)
from zhenxun.services.llm.config.generation import (
ImageAspectRatio,
LLMGenerationConfig,
ResponseFormat,
StructuredOutputStrategy,
)
from zhenxun.services.llm.types import LLMMessage
from zhenxun.services.llm.types.capabilities import ModelCapabilities
from zhenxun.services.llm.types.exceptions import LLMErrorCode, LLMException
from zhenxun.services.llm.types.models import (
LLMToolCall,
LLMToolFunction,
ModelDetail,
ToolDefinition,
)
from zhenxun.services.llm.utils import sanitize_schema_for_llm
from zhenxun.services.log import logger
from zhenxun.utils.pydantic_compat import model_dump
class OpenAIConfigMapper(ConfigMapper):
def __init__(self, api_type: str = "openai"):
self.api_type = api_type
def map_config(
self,
config: LLMGenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
params: dict[str, Any] = {}
strategy = config.output.structured_output_strategy if config.output else None
if strategy is None:
strategy = (
StructuredOutputStrategy.TOOL_CALL
if self.api_type == "deepseek"
else StructuredOutputStrategy.NATIVE
)
if config.core:
if config.core.temperature is not None:
params["temperature"] = config.core.temperature
if config.core.max_tokens is not None:
params["max_tokens"] = config.core.max_tokens
if config.core.top_k is not None:
params["top_k"] = config.core.top_k
if config.core.top_p is not None:
params["top_p"] = config.core.top_p
if config.core.frequency_penalty is not None:
params["frequency_penalty"] = config.core.frequency_penalty
if config.core.presence_penalty is not None:
params["presence_penalty"] = config.core.presence_penalty
if config.core.stop is not None:
params["stop"] = config.core.stop
if config.core.repetition_penalty is not None:
if self.api_type == "openai":
logger.warning("OpenAI官方API不支持repetition_penalty参数，已忽略")
else:
params["repetition_penalty"] = config.core.repetition_penalty
if config.reasoning and config.reasoning.effort:
params["reasoning_effort"] = config.reasoning.effort.value.lower()
if config.output:
if isinstance(config.output.response_format, dict):
params["response_format"] = config.output.response_format
elif (
config.output.response_format == ResponseFormat.JSON
and strategy == StructuredOutputStrategy.NATIVE
):
if config.output.response_schema:
sanitized = sanitize_schema_for_llm(
config.output.response_schema, api_type="openai"
)
params["response_format"] = {
"type": "json_schema",
"json_schema": {
"name": "structured_response",
"schema": sanitized,
"strict": True,
},
}
else:
params["response_format"] = {"type": "json_object"}
if config.tool_config:
mode = config.tool_config.mode
if mode == "NONE":
params["tool_choice"] = "none"
elif mode == "AUTO":
params["tool_choice"] = "auto"
elif mode == "ANY":
params["tool_choice"] = "required"
if config.visual and config.visual.aspect_ratio:
size_map = {
ImageAspectRatio.SQUARE: "1024x1024",
ImageAspectRatio.LANDSCAPE_16_9: "1792x1024",
ImageAspectRatio.PORTRAIT_9_16: "1024x1792",
}
ar = config.visual.aspect_ratio
if isinstance(ar, ImageAspectRatio):
mapped_size = size_map.get(ar)
if mapped_size:
params["size"] = mapped_size
elif isinstance(ar, str):
params["size"] = ar
if config.custom_params:
mapped_custom = config.custom_params.copy()
if "repetition_penalty" in mapped_custom and self.api_type == "openai":
mapped_custom.pop("repetition_penalty")
if "stop" in mapped_custom:
stop_value = mapped_custom["stop"]
if isinstance(stop_value, str):
mapped_custom["stop"] = [stop_value]
params.update(mapped_custom)
return params
class OpenAIMessageConverter(MessageConverter):
def convert_messages(self, messages: list[LLMMessage]) -> list[dict[str, Any]]:
openai_messages: list[dict[str, Any]] = []
for msg in messages:
openai_msg: dict[str, Any] = {"role": msg.role}
if msg.role == "tool":
openai_msg["tool_call_id"] = msg.tool_call_id
openai_msg["name"] = msg.name
openai_msg["content"] = msg.content
else:
if isinstance(msg.content, str):
openai_msg["content"] = msg.content
else:
content_parts = []
for part in msg.content:
if part.type == "text":
content_parts.append({"type": "text", "text": part.text})
elif part.type == "image":
content_parts.append(
{
"type": "image_url",
"image_url": {"url": part.image_source},
}
)
openai_msg["content"] = content_parts
if msg.role == "assistant" and msg.tool_calls:
assistant_tool_calls = []
for call in msg.tool_calls:
assistant_tool_calls.append(
{
"id": call.id,
"type": "function",
"function": {
"name": call.function.name,
"arguments": call.function.arguments,
},
}
)
openai_msg["tool_calls"] = assistant_tool_calls
if msg.name and msg.role != "tool":
openai_msg["name"] = msg.name
openai_messages.append(openai_msg)
return openai_messages
class OpenAIToolSerializer(ToolSerializer):
def serialize_tools(
self, tools: list[ToolDefinition]
) -> list[dict[str, Any]] | None:
if not tools:
return None
openai_tools = []
for tool in tools:
tool_dict = model_dump(tool)
parameters = tool_dict.get("parameters")
if parameters:
tool_dict["parameters"] = sanitize_schema_for_llm(
parameters, api_type="openai"
)
tool_dict["strict"] = True
openai_tools.append({"type": "function", "function": tool_dict})
return openai_tools
class OpenAIResponseParser(ResponseParser):
def validate_response(self, response_json: dict[str, Any]) -> None:
if response_json.get("error"):
error_info = response_json["error"]
if isinstance(error_info, dict):
error_message = error_info.get("message", "未知错误")
error_code = error_info.get("code", "unknown")
error_code_mapping = {
"invalid_api_key": LLMErrorCode.API_KEY_INVALID,
"authentication_failed": LLMErrorCode.API_KEY_INVALID,
"insufficient_quota": LLMErrorCode.API_QUOTA_EXCEEDED,
"rate_limit_exceeded": LLMErrorCode.API_RATE_LIMITED,
"quota_exceeded": LLMErrorCode.API_RATE_LIMITED,
"model_not_found": LLMErrorCode.MODEL_NOT_FOUND,
"invalid_model": LLMErrorCode.MODEL_NOT_FOUND,
"context_length_exceeded": LLMErrorCode.CONTEXT_LENGTH_EXCEEDED,
"max_tokens_exceeded": LLMErrorCode.CONTEXT_LENGTH_EXCEEDED,
"invalid_request_error": LLMErrorCode.INVALID_PARAMETER,
"invalid_parameter": LLMErrorCode.INVALID_PARAMETER,
}
llm_error_code = error_code_mapping.get(
error_code, LLMErrorCode.API_RESPONSE_INVALID
)
else:
error_message = str(error_info)
error_code = "unknown"
llm_error_code = LLMErrorCode.API_RESPONSE_INVALID
raise LLMException(
f"API请求失败: {error_message}",
code=llm_error_code,
details={"api_error": error_info, "error_code": error_code},
)
def parse(self, response_json: dict[str, Any]) -> ResponseData:
self.validate_response(response_json)
choices = response_json.get("choices", [])
if not choices:
return ResponseData(text="", raw_response=response_json)
choice = choices[0]
message = choice.get("message", {})
content = message.get("content", "")
reasoning_content = message.get("reasoning_content", None)
refusal = message.get("refusal")
if refusal:
raise LLMException(
f"模型拒绝生成请求: {refusal}",
code=LLMErrorCode.CONTENT_FILTERED,
details={"refusal": refusal},
recoverable=False,
)
if content:
content = content.strip()
images_payload: list[bytes | Path] = []
if content and content.startswith("{") and content.endswith("}"):
try:
content_json = json.loads(content)
if "b64_json" in content_json:
b64_str = content_json["b64_json"]
if isinstance(b64_str, str) and b64_str.startswith("data:"):
b64_str = b64_str.split(",", 1)[1]
decoded = base64.b64decode(b64_str)
images_payload.append(process_image_data(decoded))
content = "[图片已生成]"
elif "data" in content_json and isinstance(content_json["data"], str):
b64_str = content_json["data"]
if b64_str.startswith("data:"):
b64_str = b64_str.split(",", 1)[1]
decoded = base64.b64decode(b64_str)
images_payload.append(process_image_data(decoded))
content = "[图片已生成]"
except (json.JSONDecodeError, KeyError, binascii.Error):
pass
elif (
"images" in message
and isinstance(message["images"], list)
and message["images"]
):
for image_info in message["images"]:
if image_info.get("type") == "image_url":
image_url_obj = image_info.get("image_url", {})
url_str = image_url_obj.get("url", "")
if url_str.startswith("data:image"):
try:
b64_data = url_str.split(",", 1)[1]
decoded = base64.b64decode(b64_data)
images_payload.append(process_image_data(decoded))
except (IndexError, binascii.Error) as e:
logger.warning(f"解析OpenRouter Base64图片数据失败: {e}")
if images_payload:
content = content if content else "[图片已生成]"
parsed_tool_calls: list[LLMToolCall] | None = None
if message_tool_calls := message.get("tool_calls"):
parsed_tool_calls = []
for tc_data in message_tool_calls:
try:
if tc_data.get("type") == "function":
parsed_tool_calls.append(
LLMToolCall(
id=tc_data["id"],
function=LLMToolFunction(
name=tc_data["function"]["name"],
arguments=tc_data["function"]["arguments"],
),
)
)
except KeyError as e:
logger.warning(
f"解析OpenAI工具调用数据时缺少键: {tc_data}, 错误: {e}"
)
except Exception as e:
logger.warning(
f"解析OpenAI工具调用数据时出错: {tc_data}, 错误: {e}"
)
if not parsed_tool_calls:
parsed_tool_calls = None
final_text = content if content is not None else ""
if not final_text and parsed_tool_calls:
final_text = f"请求调用 {len(parsed_tool_calls)} 个工具。"
usage_info = response_json.get("usage")
return ResponseData(
text=final_text,
tool_calls=parsed_tool_calls,
usage_info=usage_info,
images=images_payload if images_payload else None,
raw_response=response_json,
thought_text=reasoning_content,
)
zhenxun/services/llm/adapters/factory.py
-183
View File
@@ -1,183 +0,0 @@
"""
LLM 适配器工厂类
"""
import fnmatch
from typing import TYPE_CHECKING, Any, ClassVar
from ..types.exceptions import LLMErrorCode, LLMException
from ..types.models import ToolChoice
from .base import BaseAdapter, RequestData, ResponseData
if TYPE_CHECKING:
from ..config.generation import LLMEmbeddingConfig, LLMGenerationConfig
from ..service import LLMModel
from ..types import LLMMessage
class LLMAdapterFactory:
"""LLM适配器工厂类"""
_adapters: ClassVar[dict[str, BaseAdapter]] = {}
_api_type_mapping: ClassVar[dict[str, str]] = {}
@classmethod
def initialize(cls) -> None:
"""初始化默认适配器"""
if cls._adapters:
return
from .gemini import GeminiAdapter
from .openai import DeepSeekAdapter, OpenAIAdapter, OpenAIImageAdapter
cls.register_adapter(OpenAIAdapter())
cls.register_adapter(DeepSeekAdapter())
cls.register_adapter(GeminiAdapter())
cls.register_adapter(SmartAdapter())
cls.register_adapter(OpenAIImageAdapter())
@classmethod
def register_adapter(cls, adapter: BaseAdapter) -> None:
"""注册适配器"""
adapter_key = adapter.api_type
cls._adapters[adapter_key] = adapter
for api_type in adapter.supported_api_types:
cls._api_type_mapping[api_type] = adapter_key
@classmethod
def get_adapter(cls, api_type: str) -> BaseAdapter:
"""获取适配器"""
cls.initialize()
adapter_key = cls._api_type_mapping.get(api_type)
if not adapter_key:
raise LLMException(
f"不支持的API类型: {api_type}",
code=LLMErrorCode.UNKNOWN_API_TYPE,
details={
"api_type": api_type,
"supported_types": list(cls._api_type_mapping.keys()),
},
)
return cls._adapters[adapter_key]
@classmethod
def list_supported_types(cls) -> list[str]:
"""列出所有支持的API类型"""
cls.initialize()
return list(cls._api_type_mapping.keys())
@classmethod
def list_adapters(cls) -> dict[str, BaseAdapter]:
"""列出所有注册的适配器"""
cls.initialize()
return cls._adapters.copy()
def get_adapter_for_api_type(api_type: str) -> BaseAdapter:
"""获取指定API类型的适配器"""
return LLMAdapterFactory.get_adapter(api_type)
def register_adapter(adapter: BaseAdapter) -> None:
"""注册新的适配器"""
LLMAdapterFactory.register_adapter(adapter)
class SmartAdapter(BaseAdapter):
"""
智能路由适配器。
本身不处理序列化，而是根据规则委托给 OpenAIAdapter 或 GeminiAdapter。
"""
@property
def log_sanitization_context(self) -> str:
return "openai_request"
_ROUTING_RULES: ClassVar[list[tuple[str, str]]] = [
("*nano-banana*", "gemini"),
("*gemini*", "gemini"),
]
_DEFAULT_API_TYPE: ClassVar[str] = "openai"
def __init__(self):
self._adapter_cache: dict[str, BaseAdapter] = {}
@property
def api_type(self) -> str:
return "smart"
@property
def supported_api_types(self) -> list[str]:
return ["smart"]
def _get_delegate_adapter(self, model: "LLMModel") -> BaseAdapter:
"""
核心路由逻辑：决定使用哪个适配器 (带缓存)
"""
if model.model_detail.api_type:
return get_adapter_for_api_type(model.model_detail.api_type)
model_name = model.model_name
if model_name in self._adapter_cache:
return self._adapter_cache[model_name]
target_api_type = self._DEFAULT_API_TYPE
model_name_lower = model_name.lower()
for pattern, api_type in self._ROUTING_RULES:
if fnmatch.fnmatch(model_name_lower, pattern):
target_api_type = api_type
break
adapter = get_adapter_for_api_type(target_api_type)
self._adapter_cache[model_name] = adapter
return adapter
async def prepare_advanced_request(
self,
model: "LLMModel",
api_key: str,
messages: list["LLMMessage"],
config: "LLMGenerationConfig | None" = None,
tools: list[Any] | None = None,
tool_choice: "str | dict[str, Any] | ToolChoice | None" = None,
) -> RequestData:
adapter = self._get_delegate_adapter(model)
return await adapter.prepare_advanced_request(
model, api_key, messages, config, tools, tool_choice
)
def parse_response(
self,
model: "LLMModel",
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
adapter = self._get_delegate_adapter(model)
return adapter.parse_response(model, response_json, is_advanced)
def prepare_embedding_request(
self,
model: "LLMModel",
api_key: str,
texts: list[str],
config: "LLMEmbeddingConfig",
) -> RequestData:
adapter = self._get_delegate_adapter(model)
return adapter.prepare_embedding_request(model, api_key, texts, config)
def parse_embedding_response(
self, response_json: dict[str, Any]
) -> list[list[float]]:
return get_adapter_for_api_type("openai").parse_embedding_response(
response_json
)
def convert_generation_config(
self, config: "LLMGenerationConfig", model: "LLMModel"
) -> dict[str, Any]:
adapter = self._get_delegate_adapter(model)
return adapter.convert_generation_config(config, model)
zhenxun/services/llm/adapters/gemini.py
-326
View File
@@ -1,326 +0,0 @@
"""
Gemini API 适配器
"""
from typing import TYPE_CHECKING, Any
from zhenxun.services.log import logger
from ..config.generation import ResponseFormat
from ..types import LLMContentPart
from ..types.exceptions import LLMErrorCode, LLMException
from ..types.models import BasePlatformTool, ToolChoice
from .base import BaseAdapter, RequestData, ResponseData
from .components.gemini_components import (
GeminiConfigMapper,
GeminiMessageConverter,
GeminiResponseParser,
GeminiToolSerializer,
)
if TYPE_CHECKING:
from ..config.generation import LLMEmbeddingConfig, LLMGenerationConfig
from ..service import LLMModel
from ..types import LLMMessage
class GeminiAdapter(BaseAdapter):
"""Gemini API 适配器"""
@property
def log_sanitization_context(self) -> str:
return "gemini_request"
@property
def api_type(self) -> str:
return "gemini"
@property
def supported_api_types(self) -> list[str]:
return ["gemini"]
def get_base_headers(
self, api_key: str, model: "LLMModel | None" = None
) -> dict[str, str]:
"""获取基础请求头"""
from zhenxun.utils.user_agent import get_user_agent
headers = get_user_agent()
headers.update({"Content-Type": "application/json"})
headers["x-goog-api-key"] = api_key
headers.update(self._get_provider_extra_headers(model))
return headers
async def prepare_advanced_request(
self,
model: "LLMModel",
api_key: str,
messages: list["LLMMessage"],
config: "LLMGenerationConfig | None" = None,
tools: list[Any] | None = None,
tool_choice: str | dict[str, Any] | ToolChoice | None = None,
) -> RequestData:
"""准备高级请求"""
effective_config = config if config is not None else model._generation_config
if tools:
from ..types.models import GeminiUrlContext
context_urls: list[str] = []
for tool in tools:
if isinstance(tool, GeminiUrlContext):
context_urls.extend(tool.urls)
if context_urls and messages:
last_msg = messages[-1]
if last_msg.role == "user":
url_text = "\n\n[Context URLs]:\n" + "\n".join(context_urls)
if isinstance(last_msg.content, str):
last_msg.content += url_text
elif isinstance(last_msg.content, list):
last_msg.content.append(LLMContentPart.text_part(url_text))
has_function_tools = False
if tools:
has_function_tools = any(hasattr(tool, "get_definition") for tool in tools)
is_structured = False
if effective_config and effective_config.output:
if (
effective_config.output.response_schema
or effective_config.output.response_format == ResponseFormat.JSON
or effective_config.output.response_mime_type == "application/json"
):
is_structured = True
if (has_function_tools or is_structured) and effective_config:
if effective_config.reasoning is None:
from ..config.generation import ReasoningConfig
effective_config.reasoning = ReasoningConfig()
if (
effective_config.reasoning.budget_tokens is None
and effective_config.reasoning.effort is None
):
reason_desc = "工具调用" if has_function_tools else "结构化输出"
logger.debug(
f"检测到{reason_desc}，自动为模型 {model.model_name} 开启思维链增强"
)
effective_config.reasoning.budget_tokens = -1
endpoint = self._get_gemini_endpoint(model, effective_config)
url = self.get_api_url(model, endpoint)
headers = self.get_base_headers(api_key, model)
converter = GeminiMessageConverter()
system_instruction_parts: list[dict[str, Any]] | None = None
for msg in messages:
if msg.role == "system":
if isinstance(msg.content, str):
system_instruction_parts = [{"text": msg.content}]
elif isinstance(msg.content, list):
system_instruction_parts = [
await converter.convert_part(part) for part in msg.content
]
continue
gemini_contents = await converter.convert_messages_async(messages)
body: dict[str, Any] = {"contents": gemini_contents}
if system_instruction_parts:
body["systemInstruction"] = {"parts": system_instruction_parts}
all_tools_for_request = []
has_user_functions = False
if tools:
from ..types.protocols import ToolExecutable
function_tools: list[ToolExecutable] = []
gemini_tools_dict: dict[str, Any] = {}
for tool in tools:
if isinstance(tool, BasePlatformTool):
declaration = tool.get_tool_declaration()
if declaration:
gemini_tools_dict.update(declaration)
elif hasattr(tool, "get_definition"):
function_tools.append(tool)
if function_tools:
import asyncio
definition_tasks = [
executable.get_definition() for executable in function_tools
]
tool_definitions = await asyncio.gather(*definition_tasks)
serializer = GeminiToolSerializer()
function_declarations = serializer.serialize_tools(tool_definitions)
if function_declarations:
gemini_tools_dict["functionDeclarations"] = function_declarations
has_user_functions = True
if gemini_tools_dict:
all_tools_for_request.append(gemini_tools_dict)
if all_tools_for_request:
body["tools"] = all_tools_for_request
tool_config_updates: dict[str, Any] = {}
if (
effective_config
and effective_config.custom_params
and "user_location" in effective_config.custom_params
):
tool_config_updates["retrievalConfig"] = {
"latLng": effective_config.custom_params["user_location"]
}
if tool_config_updates:
body.setdefault("toolConfig", {}).update(tool_config_updates)
converted_params: dict[str, Any] = {}
if effective_config:
converted_params = self.convert_generation_config(effective_config, model)
if converted_params:
if "toolConfig" in converted_params:
tool_config_payload = converted_params.pop("toolConfig")
fc_config = tool_config_payload.get("functionCallingConfig")
should_apply_fc = has_user_functions or (
fc_config and fc_config.get("mode") == "NONE"
)
if should_apply_fc:
body.setdefault("toolConfig", {}).update(tool_config_payload)
elif fc_config and fc_config.get("mode") != "AUTO":
logger.debug(
"Gemini: 忽略针对纯内置工具的 functionCallingConfig (API限制)"
)
if "safetySettings" in converted_params:
body["safetySettings"] = converted_params.pop("safetySettings")
if converted_params:
body["generationConfig"] = converted_params
return RequestData(url=url, headers=headers, body=body)
def apply_config_override(
self,
model: "LLMModel",
body: dict[str, Any],
config: "LLMGenerationConfig | None" = None,
) -> dict[str, Any]:
"""应用配置覆盖 - Gemini 不需要额外的配置覆盖"""
return body
def _get_gemini_endpoint(
self, model: "LLMModel", config: "LLMGenerationConfig | None" = None
) -> str:
"""返回Gemini generateContent 端点"""
return f"/v1beta/models/{model.model_name}:generateContent"
def parse_response(
self,
model: "LLMModel",
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
"""解析 Gemini API 响应"""
_ = model, is_advanced
parser = GeminiResponseParser()
return parser.parse(response_json)
def prepare_embedding_request(
self,
model: "LLMModel",
api_key: str,
texts: list[str],
config: "LLMEmbeddingConfig",
) -> RequestData:
"""准备文本嵌入请求"""
api_model_name = model.model_name
if not api_model_name.startswith("models/"):
api_model_name = f"models/{api_model_name}"
if not model.api_base:
raise LLMException(
f"模型 {model.model_name} 的 api_base 未设置",
code=LLMErrorCode.CONFIGURATION_ERROR,
)
base_url = model.api_base.rstrip("/")
url = f"{base_url}/v1beta/{api_model_name}:batchEmbedContents"
headers = self.get_base_headers(api_key, model)
requests_payload = []
for text_content in texts:
safe_text = text_content if text_content else " "
request_item: dict[str, Any] = {
"model": api_model_name,
"content": {"parts": [{"text": safe_text}]},
}
if config.task_type:
request_item["task_type"] = str(config.task_type).upper()
if config.title:
request_item["title"] = config.title
if config.output_dimensionality:
request_item["output_dimensionality"] = config.output_dimensionality
requests_payload.append(request_item)
body = {"requests": requests_payload}
return RequestData(url=url, headers=headers, body=body)
def parse_embedding_response(
self, response_json: dict[str, Any]
) -> list[list[float]]:
"""解析文本嵌入响应"""
try:
embeddings_data = response_json["embeddings"]
return [item["values"] for item in embeddings_data]
except KeyError as e:
logger.error(f"解析Gemini嵌入响应时缺少键: {e}. 响应: {response_json}")
raise LLMException(
"Gemini嵌入响应格式错误",
code=LLMErrorCode.RESPONSE_PARSE_ERROR,
details={"error": str(e)},
)
except Exception as e:
logger.error(
f"解析Gemini嵌入响应时发生未知错误: {e}. 响应: {response_json}"
)
raise LLMException(
f"解析Gemini嵌入响应失败: {e}",
code=LLMErrorCode.RESPONSE_PARSE_ERROR,
cause=e,
)
def validate_embedding_response(self, response_json: dict[str, Any]) -> None:
"""验证嵌入响应"""
super().validate_embedding_response(response_json)
if "embeddings" not in response_json or not isinstance(
response_json["embeddings"], list
):
raise LLMException(
"Gemini嵌入响应缺少'embeddings'字段或格式不正确",
code=LLMErrorCode.RESPONSE_PARSE_ERROR,
details=response_json,
)
for item in response_json["embeddings"]:
if "values" not in item:
raise LLMException(
"Gemini嵌入响应的条目中缺少'values'字段",
code=LLMErrorCode.RESPONSE_PARSE_ERROR,
details=response_json,
)
def convert_generation_config(
self, config: "LLMGenerationConfig", model: "LLMModel"
) -> dict[str, Any]:
mapper = GeminiConfigMapper()
return mapper.map_config(config, model.model_detail, model.capabilities)
zhenxun/services/llm/adapters/openai.py
-599
View File
@@ -1,599 +0,0 @@
"""
OpenAI API 适配器
支持 OpenAI、智谱AI 等 OpenAI 兼容的 API 服务。
"""
from abc import ABC, abstractmethod
import base64
from pathlib import Path
from typing import TYPE_CHECKING, Any
import json_repair
from zhenxun.services.llm.config.generation import ImageAspectRatio
from zhenxun.services.llm.types.exceptions import LLMErrorCode, LLMException
from zhenxun.services.log import logger
from zhenxun.utils.http_utils import AsyncHttpx
from ..types import StructuredOutputStrategy
from ..types.models import ToolChoice
from ..utils import sanitize_schema_for_llm
from .base import (
BaseAdapter,
OpenAICompatAdapter,
RequestData,
ResponseData,
process_image_data,
)
from .components.openai_components import (
OpenAIConfigMapper,
OpenAIMessageConverter,
OpenAIResponseParser,
OpenAIToolSerializer,
)
if TYPE_CHECKING:
from ..config.generation import LLMEmbeddingConfig, LLMGenerationConfig
from ..service import LLMModel
from ..types import LLMMessage
class APIProtocol(ABC):
"""API 协议策略基类"""
@abstractmethod
def build_request_body(
self,
model: "LLMModel",
messages: list["LLMMessage"],
tools: list[dict[str, Any]] | None,
tool_choice: Any,
) -> dict[str, Any]:
"""构建不同协议下的请求体"""
pass
@abstractmethod
def parse_response(self, response_json: dict[str, Any]) -> ResponseData:
"""解析不同协议下的响应"""
pass
class StandardProtocol(APIProtocol):
"""标准 OpenAI 协议策略"""
def __init__(self, adapter: "OpenAICompatAdapter"):
self.adapter = adapter
def build_request_body(
self,
model: "LLMModel",
messages: list["LLMMessage"],
tools: list[dict[str, Any]] | None,
tool_choice: Any,
) -> dict[str, Any]:
converter = OpenAIMessageConverter()
openai_messages = converter.convert_messages(messages)
body: dict[str, Any] = {
"model": model.model_name,
"messages": openai_messages,
}
if tools:
body["tools"] = tools
if tool_choice:
body["tool_choice"] = tool_choice
return body
def parse_response(self, response_json: dict[str, Any]) -> ResponseData:
parser = OpenAIResponseParser()
return parser.parse(response_json)
class ResponsesProtocol(APIProtocol):
"""/v1/responses 新版协议策略"""
def __init__(self, adapter: "OpenAICompatAdapter"):
self.adapter = adapter
def build_request_body(
self,
model: "LLMModel",
messages: list["LLMMessage"],
tools: list[dict[str, Any]] | None,
tool_choice: Any,
) -> dict[str, Any]:
input_items: list[dict[str, Any]] = []
for msg in messages:
role = msg.role
content_list: list[dict[str, Any]] = []
raw_contents = (
msg.content if isinstance(msg.content, list) else [msg.content]
)
for part in raw_contents:
if part is None:
continue
if isinstance(part, str):
content_list.append({"type": "input_text", "text": part})
continue
if hasattr(part, "type"):
part_type = getattr(part, "type", None)
if part_type == "text":
content_list.append(
{"type": "input_text", "text": getattr(part, "text", "")}
)
elif part_type == "image":
content_list.append(
{
"type": "input_image",
"image_url": getattr(part, "image_source", ""),
}
)
continue
if isinstance(part, dict):
part_type = part.get("type")
if part_type == "text":
content_list.append(
{"type": "input_text", "text": part.get("text", "")}
)
elif part_type in {"image", "image_url"}:
image_src = part.get("image_url") or part.get(
"image_source", ""
)
content_list.append(
{
"type": "input_image",
"image_url": image_src,
}
)
input_items.append({"role": role, "content": content_list})
body: dict[str, Any] = {
"model": model.model_name,
"input": input_items,
}
if tools:
body["tools"] = tools
if tool_choice:
body["tool_choice"] = tool_choice
return body
def parse_response(self, response_json: dict[str, Any]) -> ResponseData:
self.adapter.validate_response(response_json)
text_content = ""
for item in response_json.get("output", []):
if item.get("type") == "message" and item.get("role") == "assistant":
for content_item in item.get("content", []):
if content_item.get("type") == "output_text":
text_content += content_item.get("text", "")
return ResponseData(
text=text_content,
usage_info=response_json.get("usage"),
raw_response=response_json,
)
class OpenAIAdapter(OpenAICompatAdapter):
"""OpenAI兼容API适配器"""
@property
def api_type(self) -> str:
return "openai"
@property
def supported_api_types(self) -> list[str]:
return [
"openai",
"zhipu",
"ark",
"openrouter",
"openai_responses",
]
def get_chat_endpoint(self, model: "LLMModel") -> str:
"""返回聊天完成端点"""
if model.model_detail.endpoint:
return model.model_detail.endpoint
current_api_type = model.model_detail.api_type or model.api_type
if current_api_type == "openai_responses":
return "/v1/responses"
if current_api_type == "ark":
return "/api/v3/chat/completions"
if current_api_type == "zhipu":
return "/api/paas/v4/chat/completions"
return "/v1/chat/completions"
def _get_protocol_strategy(self, model: "LLMModel") -> APIProtocol:
"""根据 API 类型获取对应的处理策略"""
current_api_type = model.model_detail.api_type or model.api_type
if current_api_type == "openai_responses":
return ResponsesProtocol(self)
return StandardProtocol(self)
def get_embedding_endpoint(self, model: "LLMModel") -> str:
"""根据API类型返回嵌入端点"""
if model.api_type == "zhipu":
return "/v4/embeddings"
return "/v1/embeddings"
def convert_generation_config(
self, config: "LLMGenerationConfig", model: "LLMModel"
) -> dict[str, Any]:
mapper = OpenAIConfigMapper(api_type=self.api_type)
return mapper.map_config(config, model.model_detail, model.capabilities)
async def prepare_advanced_request(
self,
model: "LLMModel",
api_key: str,
messages: list["LLMMessage"],
config: "LLMGenerationConfig | None" = None,
tools: list[Any] | None = None,
tool_choice: str | dict[str, Any] | ToolChoice | None = None,
) -> "RequestData":
"""根据不同协议策略构建高级请求"""
url = self.get_api_url(model, self.get_chat_endpoint(model))
headers = self.get_base_headers(api_key, model)
if model.api_type == "openrouter":
headers.update(
{
"HTTP-Referer": "https://github.com/zhenxun-org/zhenxun_bot",
"X-Title": "Zhenxun Bot",
}
)
default_config = getattr(model, "_generation_config", None)
effective_config = config if config is not None else default_config
structured_strategy = (
effective_config.output.structured_output_strategy
if effective_config and effective_config.output
else None
)
if structured_strategy is None:
structured_strategy = StructuredOutputStrategy.NATIVE
openai_tools: list[dict[str, Any]] | None = None
executables: list[Any] = []
if tools:
if isinstance(tools, dict):
executables = list(tools.values())
else:
for tool in tools:
if hasattr(tool, "get_definition"):
executables.append(tool)
definition_tasks = [executable.get_definition() for executable in executables]
tool_defs: list[Any] = []
if definition_tasks:
import asyncio
tool_defs = await asyncio.gather(*definition_tasks)
if tool_defs:
serializer = OpenAIToolSerializer()
openai_tools = serializer.serialize_tools(tool_defs)
final_tool_choice = tool_choice
if final_tool_choice is None:
if (
effective_config
and effective_config.tool_config
and effective_config.tool_config.mode == "ANY"
):
allowed = effective_config.tool_config.allowed_function_names
if allowed:
if len(allowed) == 1:
final_tool_choice = {
"type": "function",
"function": {"name": allowed[0]},
}
else:
logger.warning(
"OpenAI API 不支持多个 allowed_function_names，降级为"
" required。"
)
final_tool_choice = "required"
else:
final_tool_choice = "required"
if (
structured_strategy == StructuredOutputStrategy.TOOL_CALL
and effective_config
and effective_config.output
and effective_config.output.response_schema
):
sanitized_schema = sanitize_schema_for_llm(
effective_config.output.response_schema, api_type="openai"
)
structured_tool = {
"type": "function",
"function": {
"name": "return_structured_response",
"description": "Return the final structured response.",
"parameters": sanitized_schema,
"strict": True if model.api_type != "deepseek" else False,
},
}
if openai_tools is None:
openai_tools = []
openai_tools.append(structured_tool)
final_tool_choice = {
"type": "function",
"function": {"name": "return_structured_response"},
}
protocol_strategy = self._get_protocol_strategy(model)
body = protocol_strategy.build_request_body(
model=model,
messages=messages,
tools=openai_tools,
tool_choice=final_tool_choice,
)
body = self.apply_config_override(model, body, config)
if final_tool_choice is not None:
body["tool_choice"] = final_tool_choice
response_format = body.get("response_format", {})
inject_prompt = (
structured_strategy == StructuredOutputStrategy.NATIVE
and isinstance(response_format, dict)
and response_format.get("type") == "json_object"
)
if inject_prompt:
messages_list = body.get("messages", [])
has_json_keyword = False
for msg in messages_list:
content = msg.get("content")
if isinstance(content, str) and "json" in content.lower():
has_json_keyword = True
break
if isinstance(content, list):
for part in content:
if (
isinstance(part, dict)
and part.get("type") == "text"
and "json" in part.get("text", "").lower()
):
has_json_keyword = True
break
if has_json_keyword:
break
if not has_json_keyword:
injection_text = (
"请务必输出合法的 JSON 格式，避免额外的文本、Markdown 或解释。"
)
system_msg = next(
(m for m in messages_list if m.get("role") == "system"), None
)
if system_msg:
if isinstance(system_msg.get("content"), str):
system_msg["content"] += " " + injection_text
elif isinstance(system_msg.get("content"), list):
system_msg["content"].append(
{"type": "text", "text": injection_text}
)
else:
messages_list.insert(
0, {"role": "system", "content": injection_text}
)
body["messages"] = messages_list
return RequestData(url=url, headers=headers, body=body)
def parse_response(
self,
model: "LLMModel",
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
"""解析响应 - 使用策略模式委托处理"""
_ = is_advanced
protocol_strategy = self._get_protocol_strategy(model)
response_data = protocol_strategy.parse_response(response_json)
if response_data.tool_calls:
target_tool = next(
(
tc
for tc in response_data.tool_calls
if tc.function.name == "return_structured_response"
),
None,
)
if target_tool:
response_data.text = json_repair.repair_json(
target_tool.function.arguments
)
remaining = [
tc
for tc in response_data.tool_calls
if tc.function.name != "return_structured_response"
]
response_data.tool_calls = remaining or None
return response_data
class DeepSeekAdapter(OpenAIAdapter):
"""DeepSeek 专用适配器 (基于 OpenAI 协议)"""
@property
def api_type(self) -> str:
return "deepseek"
@property
def supported_api_types(self) -> list[str]:
return ["deepseek"]
class OpenAIImageAdapter(BaseAdapter):
"""OpenAI 图像生成/编辑适配器"""
@property
def api_type(self) -> str:
return "openai_image"
@property
def log_sanitization_context(self) -> str:
return "openai_request"
@property
def supported_api_types(self) -> list[str]:
return ["openai_image", "nano_banana"]
async def prepare_advanced_request(
self,
model: "LLMModel",
api_key: str,
messages: list["LLMMessage"],
config: "LLMGenerationConfig | None" = None,
tools: list[Any] | None = None,
tool_choice: "str | dict[str, Any] | ToolChoice | None" = None,
) -> RequestData:
_ = tools, tool_choice
effective_config = config if config is not None else model._generation_config
headers = self.get_base_headers(api_key, model)
prompt = ""
images_bytes_list: list[bytes] = []
for msg in reversed(messages):
if msg.role != "user":
continue
if isinstance(msg.content, str):
prompt = msg.content
elif isinstance(msg.content, list):
for part in msg.content:
if part.type == "text" and not prompt:
prompt = part.text
elif part.type == "image":
if part.is_image_base64():
if b64_data := part.get_base64_data():
_, b64_str = b64_data
images_bytes_list.append(base64.b64decode(b64_str))
elif part.is_image_url() and part.image_source:
images_bytes_list.append(
await AsyncHttpx.get_content(part.image_source)
)
if prompt:
break
if not prompt and not images_bytes_list:
raise LLMException(
"图像生成需要提供 Prompt",
code=LLMErrorCode.CONFIGURATION_ERROR,
)
body: dict[str, Any] = {
"model": model.model_name,
"prompt": prompt,
"response_format": "b64_json",
}
if effective_config:
if effective_config.visual:
if effective_config.visual.aspect_ratio:
ar = effective_config.visual.aspect_ratio
size_map = {
ImageAspectRatio.SQUARE: "1024x1024",
ImageAspectRatio.LANDSCAPE_16_9: "1792x1024",
ImageAspectRatio.PORTRAIT_9_16: "1024x1792",
}
if isinstance(ar, ImageAspectRatio) and ar in size_map:
body["size"] = size_map[ar]
body["aspect_ratio"] = ar.value
elif isinstance(ar, str):
if "x" in ar:
body["size"] = ar
else:
body["aspect_ratio"] = ar
if effective_config.visual.resolution:
res_val = effective_config.visual.resolution
if not isinstance(res_val, str):
res_val = getattr(res_val, "value", res_val)
body["image_size"] = res_val
if effective_config.custom_params:
body.update(effective_config.custom_params)
if images_bytes_list:
b64_images = []
for img_bytes in images_bytes_list:
b64_str = base64.b64encode(img_bytes).decode("utf-8")
b64_images.append(b64_str)
body["image"] = b64_images
endpoint = "/v1/images/generations"
url = self.get_api_url(model, endpoint)
return RequestData(url=url, headers=headers, body=body)
def parse_response(
self,
model: "LLMModel",
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
_ = model, is_advanced
self.validate_response(response_json)
images_data: list[bytes | Path] = []
data_list = response_json.get("data", [])
for item in data_list:
if "b64_json" in item:
try:
b64_str = item["b64_json"]
if b64_str.startswith("data:"):
b64_str = b64_str.split(",", 1)[1]
img = base64.b64decode(b64_str)
images_data.append(process_image_data(img))
except Exception as exc:
logger.error(f"Base64 解码失败: {exc}")
elif "url" in item:
logger.warning(
f"API 返回了 URL 而不是 Base64: {item.get('url', 'unknown')}"
)
text_summary = (
f"已生成 {len(images_data)} 张图片。"
if images_data
else "图像生成接口调用成功，但未解析到图片数据。"
)
return ResponseData(
text=text_summary,
images=images_data if images_data else None,
raw_response=response_json,
)
def prepare_embedding_request(
self,
model: "LLMModel",
api_key: str,
texts: list[str],
config: "LLMEmbeddingConfig",
) -> RequestData:
raise NotImplementedError("OpenAIImageAdapter 不支持 Embedding")
def parse_embedding_response(
self, response_json: dict[str, Any]
) -> list[list[float]]:
raise NotImplementedError("OpenAIImageAdapter 不支持 Embedding")
def convert_generation_config(
self, config: "LLMGenerationConfig", model: "LLMModel"
) -> dict[str, Any]:
_ = config, model
return {}
zhenxun/services/llm/api.py
-407
View File
@@ -1,407 +0,0 @@
"""
LLM 服务的高级 API 接口 - 便捷函数入口 (无状态)
"""
from collections.abc import Awaitable, Callable
from pathlib import Path
from typing import Any, TypeVar, overload
from nonebot_plugin_alconna.uniseg import UniMessage
from pydantic import BaseModel
from zhenxun.services.log import logger
from .config import CommonOverrides
from .config.generation import (
GenConfigBuilder,
LLMEmbeddingConfig,
LLMGenerationConfig,
OutputConfig,
)
from .manager import get_model_instance
from .session import AI
from .types import (
LLMContentPart,
LLMErrorCode,
LLMException,
LLMMessage,
LLMResponse,
ModelName,
ToolChoice,
)
from .types.exceptions import get_user_friendly_error_message
from .types.models import GeminiGoogleSearch
from .utils import create_multimodal_message
T = TypeVar("T", bound=BaseModel)
async def chat(
message: str | UniMessage | LLMMessage | list[LLMContentPart],
*,
model: ModelName = None,
instruction: str | None = None,
tools: list[Any] | None = None,
tool_choice: str | dict[str, Any] | ToolChoice | None = None,
config: LLMGenerationConfig | GenConfigBuilder | None = None,
timeout: float | None = None,
) -> LLMResponse:
"""
无状态的聊天对话便捷函数，通过临时的AI会话实例与LLM模型交互。
参数:
message: 用户输入的消息内容，支持多种格式。
model: 要使用的模型名称，如果为None则使用默认模型。
instruction: 系统指令，用于指导AI的行为和回复风格。
tools: 可用的工具列表，支持字典配置或字符串标识符。
tool_choice: 工具选择策略，控制AI如何选择和使用工具。
config: (可选) 生成配置对象，将与默认配置合并后传递。
timeout: (可选) HTTP 请求超时时间（秒）。
返回:
LLMResponse: 包含AI回复内容、使用信息和工具调用等的完整响应对象。
"""
try:
ai_session = AI()
return await ai_session.chat(
message,
model=model,
instruction=instruction,
tools=tools,
tool_choice=tool_choice,
config=config,
timeout=timeout,
)
except LLMException:
raise
except Exception as e:
friendly_msg = get_user_friendly_error_message(e)
logger.error(f"执行 chat 函数失败: {e} | 建议: {friendly_msg}", e=e)
raise LLMException(f"聊天执行失败: {friendly_msg}", cause=e)
async def code(
prompt: str,
*,
model: ModelName = None,
timeout: int | None = None,
) -> LLMResponse:
"""
无状态的代码执行便捷函数，支持在沙箱环境中执行代码。
参数:
prompt: 代码执行的提示词，描述要执行的代码任务。
model: 要使用的模型名称，默认使用Gemini/gemini-2.0-flash。
timeout: 代码执行超时时间（秒），防止长时间运行的代码阻塞。
返回:
LLMResponse: 包含代码执行结果的完整响应对象。
"""
resolved_model = model
config = CommonOverrides.gemini_code_execution()
if timeout:
config.custom_params = config.custom_params or {}
config.custom_params["code_execution_timeout"] = timeout
return await chat(prompt, model=resolved_model, config=config)
async def embed(
texts: list[str] | str,
*,
model: ModelName = None,
config: LLMEmbeddingConfig | None = None,
) -> list[list[float]]:
"""
无状态的文本嵌入便捷函数，将文本转换为向量表示。
参数:
texts: 要生成嵌入的文本内容，支持单个字符串或字符串列表。
model: 要使用的嵌入模型名称，如果为None则使用默认模型。
config: 嵌入配置对象。
返回:
list[list[float]]: 文本对应的嵌入向量列表，每个向量为浮点数列表。
"""
if isinstance(texts, str):
texts = [texts]
if not texts:
return []
final_config = config or LLMEmbeddingConfig()
try:
async with await get_model_instance(model) as model_instance:
return await model_instance.generate_embeddings(texts, config=final_config)
except LLMException:
raise
except Exception as e:
friendly_msg = get_user_friendly_error_message(e)
logger.error(f"文本嵌入失败: {e} | 建议: {friendly_msg}", e=e)
raise LLMException(
f"文本嵌入失败: {friendly_msg}",
code=LLMErrorCode.EMBEDDING_FAILED,
cause=e,
)
async def embed_query(
text: str,
*,
model: ModelName = None,
dimensions: int | None = None,
) -> list[float]:
"""
语义化便捷 API：为检索查询生成嵌入。
"""
config = LLMEmbeddingConfig(
task_type="RETRIEVAL_QUERY",
output_dimensionality=dimensions,
)
vectors = await embed([text], model=model, config=config)
return vectors[0] if vectors else []
async def embed_documents(
texts: list[str],
*,
model: ModelName = None,
dimensions: int | None = None,
title: str | None = None,
) -> list[list[float]]:
"""
语义化便捷 API：为文档集合生成嵌入。
"""
config = LLMEmbeddingConfig(
task_type="RETRIEVAL_DOCUMENT",
output_dimensionality=dimensions,
title=title,
)
return await embed(texts, model=model, config=config)
async def generate_structured(
message: str | UniMessage | LLMMessage | list[LLMContentPart],
response_model: type[T],
*,
model: ModelName = None,
tools: list[Any] | None = None,
tool_choice: str | dict[str, Any] | ToolChoice | None = None,
max_validation_retries: int | None = None,
validation_callback: Callable[[T], Any | Awaitable[Any]] | None = None,
error_prompt_template: str | None = None,
auto_thinking: bool = False,
instruction: str | None = None,
timeout: float | None = None,
) -> T:
"""
无状态地生成结构化响应，并自动解析为指定的Pydantic模型。
参数:
message: 用户输入的消息内容，支持多种格式。
response_model: 用于解析和验证响应的Pydantic模型类。
max_validation_retries: 校验失败时的最大重试次数，默认为 None (使用全局配置)。
validation_callback: 自定义校验回调函数，抛出异常视为校验失败。
error_prompt_template: 自定义错误反馈提示词模板。
auto_thinking: 是否自动开启思维链 (CoT) 包装。适用于不支持原生思考的模型
model: 要使用的模型名称，如果为None则使用默认模型。
instruction: 系统指令，用于指导AI生成符合要求的结构化输出。
timeout: HTTP 请求超时时间（秒）。
返回:
T: 解析后的Pydantic模型实例，类型为response_model指定的类型。
"""
try:
ai_session = AI()
return await ai_session.generate_structured(
message,
response_model,
model=model,
tools=tools,
tool_choice=tool_choice,
max_validation_retries=max_validation_retries,
validation_callback=validation_callback,
error_prompt_template=error_prompt_template,
auto_thinking=auto_thinking,
instruction=instruction,
timeout=timeout,
)
except LLMException:
raise
except Exception as e:
friendly_msg = get_user_friendly_error_message(e)
logger.error(f"生成结构化响应失败: {e} | 建议: {friendly_msg}", e=e)
raise LLMException(f"生成结构化响应失败: {friendly_msg}", cause=e)
async def generate(
messages: list[LLMMessage],
*,
model: ModelName = None,
tools: list[Any] | None = None,
tool_choice: str | dict[str, Any] | ToolChoice | None = None,
config: LLMGenerationConfig | GenConfigBuilder | None = None,
) -> LLMResponse:
"""
根据完整的消息列表生成一次性响应，这是一个无状态的底层函数。
参数:
messages: 完整的消息历史列表，包括系统指令、用户消息和助手回复。
model: 要使用的模型名称，如果为None则使用默认模型。
tools: 可用的工具列表，支持字典配置或字符串标识符。
tool_choice: 工具选择策略，控制AI如何选择和使用工具。
config: (可选) 生成配置对象，将与默认配置合并后传递。
返回:
LLMResponse: 包含AI回复内容、使用信息和工具调用等的完整响应对象。
"""
try:
if isinstance(config, GenConfigBuilder):
config = config.build()
async with await get_model_instance(
model, override_config=None
) as model_instance:
return await model_instance.generate_response(
messages,
config=config,
tools=tools, # type: ignore[arg-type]
tool_choice=tool_choice,
)
except LLMException:
raise
except Exception as e:
friendly_msg = get_user_friendly_error_message(e)
logger.error(f"生成响应失败: {e} | 建议: {friendly_msg}", e=e)
raise LLMException(f"生成响应失败: {friendly_msg}", cause=e)
async def _generate_image_from_message(
message: UniMessage,
model: ModelName = None,
config: LLMGenerationConfig | GenConfigBuilder | None = None,
) -> LLMResponse:
"""
[内部] 从 UniMessage 生成图片的核心辅助函数。
"""
from .utils import normalize_to_llm_messages
if isinstance(config, GenConfigBuilder):
config = config.build()
config = config or LLMGenerationConfig()
config.validation_policy = {"require_image": True}
if config.output is None:
config.output = OutputConfig()
config.output.response_modalities = ["IMAGE", "TEXT"]
try:
messages = await normalize_to_llm_messages(message)
async with await get_model_instance(model) as model_instance:
response = await model_instance.generate_response(messages, config=config)
if not response.images:
error_text = response.text or "模型未返回图片数据。"
logger.warning(f"图片生成调用未返回图片，返回文本内容: {error_text}")
return response
except LLMException:
raise
except Exception as e:
friendly_msg = get_user_friendly_error_message(e)
logger.error(f"执行图片生成时发生未知错误: {e} | 建议: {friendly_msg}", e=e)
raise LLMException(f"图片生成失败: {friendly_msg}", cause=e)
@overload
async def create_image(
prompt: str | UniMessage,
*,
images: None = None,
model: ModelName = None,
) -> LLMResponse:
"""根据文本提示生成一张新图片。"""
...
@overload
async def create_image(
prompt: str | UniMessage,
*,
images: list[Path | bytes | str] | Path | bytes | str,
model: ModelName = None,
) -> LLMResponse:
"""在给定图片的基础上，根据文本提示进行编辑或重新生成。"""
...
async def create_image(
prompt: str | UniMessage,
*,
images: list[Path | bytes | str] | Path | bytes | str | None = None,
model: ModelName = None,
config: LLMGenerationConfig | GenConfigBuilder | None = None,
) -> LLMResponse:
"""
智能图片生成/编辑函数。
- 如果 `images` 为 None，执行文生图。
- 如果提供了 `images`，执行图+文生图，支持多张图片输入。
"""
text_prompt = (
prompt.extract_plain_text() if isinstance(prompt, UniMessage) else str(prompt)
)
image_list = []
if images:
if isinstance(images, list):
image_list.extend(images)
else:
image_list.append(images)
message = create_multimodal_message(text=text_prompt, images=image_list)
return await _generate_image_from_message(message, model=model, config=config)
async def search(
query: str | UniMessage | LLMMessage | list[LLMContentPart],
*,
model: ModelName = None,
instruction: str = (
"你是一位强大的信息检索和整合专家。请利用可用的搜索工具，"
"根据用户的查询找到最相关的信息，并进行总结和回答。"
),
config: LLMGenerationConfig | GenConfigBuilder | None = None,
) -> LLMResponse:
"""
无状态的信息搜索便捷函数，利用搜索工具获取实时信息。
参数:
query: 搜索查询内容，支持多种输入格式。
model: 要使用的模型名称，如果为None则使用默认模型。
config: (可选) 生成配置对象，将与预设配置合并后传递。
instruction: 搜索任务的系统指令，指导AI如何处理搜索结果。
返回:
LLMResponse: 包含搜索结果和AI整合回复的完整响应对象。
"""
logger.debug("执行无状态 'search' 任务...")
search_config = CommonOverrides.gemini_grounding()
if isinstance(config, GenConfigBuilder):
config = config.build()
final_config = search_config.merge_with(config)
return await chat(
query,
model=model,
instruction=instruction,
config=final_config,
tools=[GeminiGoogleSearch()],
)
zhenxun/services/llm/config/__init__.py
-35
View File
@@ -1,35 +0,0 @@
"""
LLM 配置模块
提供生成配置、预设配置和配置验证功能。
"""
from .generation import (
CommonOverrides,
GenConfigBuilder,
LLMEmbeddingConfig,
LLMGenerationConfig,
validate_override_params,
)
from .providers import (
LLMConfig,
get_gemini_safety_threshold,
get_llm_config,
register_llm_configs,
set_default_model,
validate_llm_config,
)
__all__ = [
"CommonOverrides",
"GenConfigBuilder",
"LLMConfig",
"LLMEmbeddingConfig",
"LLMGenerationConfig",
"get_gemini_safety_threshold",
"get_llm_config",
"register_llm_configs",
"set_default_model",
"validate_llm_config",
"validate_override_params",
]
zhenxun/services/llm/config/generation.py
-523
View File
@@ -1,523 +0,0 @@
"""
LLM 生成配置相关类和函数
"""
from collections.abc import Callable
from enum import Enum
from typing import Any, Literal
from typing_extensions import Self
from pydantic import BaseModel, ConfigDict, Field
from zhenxun.services.log import logger
from zhenxun.utils.pydantic_compat import model_copy, model_dump, model_validate
from ..types import LLMResponse, ResponseFormat, StructuredOutputStrategy
from ..types.exceptions import LLMErrorCode, LLMException
from .providers import get_gemini_safety_threshold
class ReasoningEffort(str, Enum):
"""推理努力程度枚举"""
LOW = "LOW"
MEDIUM = "MEDIUM"
HIGH = "HIGH"
class ImageAspectRatio(str, Enum):
"""图像宽高比枚举"""
SQUARE = "1:1"
LANDSCAPE_16_9 = "16:9"
PORTRAIT_9_16 = "9:16"
LANDSCAPE_4_3 = "4:3"
PORTRAIT_3_4 = "3:4"
LANDSCAPE_3_2 = "3:2"
PORTRAIT_2_3 = "2:3"
class ImageResolution(str, Enum):
"""图像分辨率/质量枚举"""
STANDARD = "STANDARD"
HD = "HD"
class CoreConfig(BaseModel):
"""核心生成参数"""
temperature: float | None = Field(
default=None, ge=0.0, le=2.0, description="生成温度"
)
"""生成温度"""
max_tokens: int | None = Field(default=None, gt=0, description="最大输出token数")
"""最大输出token数"""
top_p: float | None = Field(default=None, ge=0.0, le=1.0, description="核采样参数")
"""核采样参数"""
top_k: int | None = Field(default=None, gt=0, description="Top-K采样参数")
"""Top-K采样参数"""
frequency_penalty: float | None = Field(
default=None, ge=-2.0, le=2.0, description="频率惩罚"
)
"""频率惩罚"""
presence_penalty: float | None = Field(
default=None, ge=-2.0, le=2.0, description="存在惩罚"
)
"""存在惩罚"""
repetition_penalty: float | None = Field(
default=None, ge=0.0, le=2.0, description="重复惩罚"
)
"""重复惩罚"""
stop: list[str] | str | None = Field(default=None, description="停止序列")
"""停止序列"""
class ReasoningConfig(BaseModel):
"""推理能力配置"""
effort: ReasoningEffort | None = Field(
default=None, description="推理努力程度 (适用于 O1, Gemini 3)"
)
"""推理努力程度 (适用于 O1, Gemini 3)"""
budget_tokens: int | None = Field(
default=None, description="具体的思考 Token 预算 (适用于 Gemini 2.5)"
)
"""具体的思考 Token 预算 (适用于 Gemini 2.5)"""
show_thoughts: bool | None = Field(
default=None, description="是否在响应中显式包含思维链内容"
)
"""是否在响应中显式包含思维链内容"""
class VisualConfig(BaseModel):
"""视觉生成配置"""
aspect_ratio: ImageAspectRatio | str | None = Field(
default=None, description="宽高比"
)
"""宽高比"""
resolution: ImageResolution | str | None = Field(
default=None, description="生成质量/分辨率"
)
"""生成质量/分辨率"""
media_resolution: str | None = Field(
default=None,
description="输入媒体的解析度 (Gemini 3+): 'LOW', 'MEDIUM', 'HIGH'",
)
"""输入媒体的解析度 (Gemini 3+): 'LOW', 'MEDIUM', 'HIGH'"""
style: str | None = Field(
default=None, description="图像风格 (如 DALL-E 3 vivid/natural)"
)
"""图像风格 (如 DALL-E 3 vivid/natural)"""
class OutputConfig(BaseModel):
"""输出格式控制"""
response_format: ResponseFormat | dict[str, Any] | None = Field(
default=None, description="期望的响应格式"
)
"""期望的响应格式"""
response_mime_type: str | None = Field(
default=None, description="响应MIME类型（Gemini专用）"
)
"""响应MIME类型（Gemini专用）"""
response_schema: dict[str, Any] | None = Field(
default=None, description="JSON响应模式"
)
"""JSON响应模式"""
response_modalities: list[str] | None = Field(
default=None, description="响应模态类型 (TEXT, IMAGE, AUDIO)"
)
"""响应模态类型 (TEXT, IMAGE, AUDIO)"""
structured_output_strategy: StructuredOutputStrategy | str | None = Field(
default=None, description="结构化输出策略 (NATIVE/TOOL_CALL/PROMPT)"
)
"""结构化输出策略 (NATIVE/TOOL_CALL/PROMPT)"""
class SafetyConfig(BaseModel):
"""安全设置"""
safety_settings: dict[str, str] | None = Field(default=None, description="安全设置")
"""安全设置"""
class ToolConfig(BaseModel):
"""工具调用控制配置"""
mode: Literal["AUTO", "ANY", "NONE"] = Field(
default="AUTO",
description="工具调用模式: AUTO(自动), ANY(强制), NONE(禁用)",
)
"""工具调用模式: AUTO(自动), ANY(强制), NONE(禁用)"""
allowed_function_names: list[str] | None = Field(
default=None,
description="当 mode 为 ANY 时，允许调用的函数名称白名单",
)
"""当 mode 为 ANY 时，允许调用的函数名称白名单"""
class LLMGenerationConfig(BaseModel):
"""
LLM 生成配置
采用组件化设计，不再扁平化参数。
"""
core: CoreConfig | None = Field(default=None, description="基础生成参数")
"""基础生成参数"""
reasoning: ReasoningConfig | None = Field(default=None, description="推理能力配置")
"""推理能力配置"""
visual: VisualConfig | None = Field(default=None, description="视觉生成配置")
"""视觉生成配置"""
output: OutputConfig | None = Field(default=None, description="输出格式配置")
"""输出格式配置"""
safety: SafetyConfig | None = Field(default=None, description="安全配置")
"""安全配置"""
tool_config: ToolConfig | None = Field(default=None, description="工具调用策略配置")
"""工具调用策略配置"""
enable_caching: bool | None = Field(default=None, description="是否启用响应缓存")
"""是否启用响应缓存"""
custom_params: dict[str, Any] | None = Field(default=None, description="自定义参数")
"""自定义参数"""
validation_policy: dict[str, Any] | None = Field(
default=None, description="声明式的响应验证策略 (例如: {'require_image': True})"
)
"""声明式的响应验证策略 (例如: {'require_image': True})"""
response_validator: Callable[[LLMResponse], None] | None = Field(
default=None,
description="一个高级回调函数，用于验证响应，验证失败时应抛出异常",
)
"""一个高级回调函数，用于验证响应，验证失败时应抛出异常"""
model_config = ConfigDict(arbitrary_types_allowed=True)
@classmethod
def builder(cls) -> "GenConfigBuilder":
"""创建一个新的配置构建器"""
return GenConfigBuilder()
def to_dict(self) -> dict[str, Any]:
"""
转换为字典，排除None值。
注意：这会返回嵌套结构的字典。适配器需要处理这种嵌套。
"""
return model_dump(self, exclude_none=True)
def merge_with(self, other: "LLMGenerationConfig | None") -> "LLMGenerationConfig":
"""
与另一个配置对象进行深度合并。
other 中的非 None 字段会覆盖当前配置中的对应字段。
返回一个新的配置对象，原对象不变。
"""
if not other:
return model_copy(self, deep=True)
new_config = model_copy(self, deep=True)
def _merge_component(base_comp, override_comp, comp_cls):
if override_comp is None:
return base_comp
if base_comp is None:
return override_comp
updates = model_dump(override_comp, exclude_none=True)
return model_copy(base_comp, update=updates)
new_config.core = _merge_component(new_config.core, other.core, CoreConfig)
new_config.reasoning = _merge_component(
new_config.reasoning, other.reasoning, ReasoningConfig
)
new_config.visual = _merge_component(
new_config.visual, other.visual, VisualConfig
)
new_config.output = _merge_component(
new_config.output, other.output, OutputConfig
)
new_config.safety = _merge_component(
new_config.safety, other.safety, SafetyConfig
)
new_config.tool_config = _merge_component(
new_config.tool_config, other.tool_config, ToolConfig
)
if other.enable_caching is not None:
new_config.enable_caching = other.enable_caching
if other.custom_params:
if new_config.custom_params is None:
new_config.custom_params = {}
new_config.custom_params.update(other.custom_params)
if other.validation_policy:
if new_config.validation_policy is None:
new_config.validation_policy = {}
new_config.validation_policy.update(other.validation_policy)
if other.response_validator:
new_config.response_validator = other.response_validator
return new_config
class LLMEmbeddingConfig(BaseModel):
"""Embedding 专用配置"""
task_type: str | None = Field(default=None, description="任务类型 (Gemini/Jina)")
"""任务类型 (Gemini/Jina)"""
output_dimensionality: int | None = Field(
default=None, description="输出维度/压缩维度 (Gemini/Jina/OpenAI)"
)
"""输出维度/压缩维度 (Gemini/Jina/OpenAI)"""
title: str | None = Field(
default=None, description="仅用于 Gemini RETRIEVAL_DOCUMENT 任务的标题"
)
"""仅用于 Gemini RETRIEVAL_DOCUMENT 任务的标题"""
encoding_format: str | None = Field(
default="float", description="编码格式 (float/base64)"
)
"""编码格式 (float/base64)"""
model_config = ConfigDict(arbitrary_types_allowed=True)
class GenConfigBuilder:
"""
LLM 生成配置的语义化构建器。
设计原则：高频业务场景优先，低频参数命名空间化。
"""
def __init__(self):
self._config = LLMGenerationConfig()
def _ensure_core(self) -> CoreConfig:
if self._config.core is None:
self._config.core = CoreConfig()
return self._config.core
def _ensure_output(self) -> OutputConfig:
if self._config.output is None:
self._config.output = OutputConfig()
return self._config.output
def _ensure_reasoning(self) -> ReasoningConfig:
if self._config.reasoning is None:
self._config.reasoning = ReasoningConfig()
return self._config.reasoning
def as_json(self, schema: dict[str, Any] | None = None) -> Self:
"""
[高频] 强制模型输出 JSON 格式。
"""
out = self._ensure_output()
out.response_format = ResponseFormat.JSON
if schema:
out.response_schema = schema
return self
def enable_thinking(
self, budget_tokens: int = -1, show_thoughts: bool = False
) -> Self:
"""
[高频] 启用模型的思考/推理能力 (如 Gemini 2.0 Flash Thinking, DeepSeek R1)。
"""
reasoning = self._ensure_reasoning()
reasoning.budget_tokens = budget_tokens
reasoning.show_thoughts = show_thoughts
return self
def config_core(
self,
temperature: float | None = None,
max_tokens: int | None = None,
top_p: float | None = None,
top_k: int | None = None,
stop: list[str] | str | None = None,
frequency_penalty: float | None = None,
presence_penalty: float | None = None,
) -> Self:
"""
[低频] 配置核心生成参数。
"""
core = self._ensure_core()
if temperature is not None:
core.temperature = temperature
if max_tokens is not None:
core.max_tokens = max_tokens
if top_p is not None:
core.top_p = top_p
if top_k is not None:
core.top_k = top_k
if stop is not None:
core.stop = stop
if frequency_penalty is not None:
core.frequency_penalty = frequency_penalty
if presence_penalty is not None:
core.presence_penalty = presence_penalty
return self
def config_safety(self, settings: dict[str, str]) -> Self:
"""
[低频] 配置安全过滤设置。
"""
if self._config.safety is None:
self._config.safety = SafetyConfig()
self._config.safety.safety_settings = settings
return self
def config_visual(
self,
aspect_ratio: ImageAspectRatio | str | None = None,
resolution: ImageResolution | str | None = None,
) -> Self:
"""
[低频] 配置视觉生成参数 (DALL-E 3 / Gemini Imagen)。
"""
if self._config.visual is None:
self._config.visual = VisualConfig()
if aspect_ratio:
self._config.visual.aspect_ratio = aspect_ratio
if resolution:
self._config.visual.resolution = resolution
return self
def set_custom_param(self, key: str, value: Any) -> Self:
"""设置特定于厂商的自定义参数"""
if self._config.custom_params is None:
self._config.custom_params = {}
self._config.custom_params[key] = value
return self
def build(self) -> LLMGenerationConfig:
"""构建最终的配置对象"""
return self._config
def validate_override_params(
override_config: dict[str, Any] | LLMGenerationConfig | None,
) -> LLMGenerationConfig:
"""验证和标准化覆盖参数"""
if override_config is None:
return LLMGenerationConfig()
if isinstance(override_config, LLMGenerationConfig):
return override_config
if isinstance(override_config, dict):
try:
return model_validate(LLMGenerationConfig, override_config)
except Exception as e:
logger.warning(f"覆盖配置参数验证失败: {e}")
raise LLMException(
f"无效的覆盖配置参数: {e}",
code=LLMErrorCode.CONFIGURATION_ERROR,
cause=e,
)
raise LLMException(
f"不支持的配置类型: {type(override_config)}",
code=LLMErrorCode.CONFIGURATION_ERROR,
)
class CommonOverrides:
"""常用的配置覆盖预设"""
@staticmethod
def gemini_json() -> LLMGenerationConfig:
"""Gemini JSON模式：强制JSON输出"""
return LLMGenerationConfig(
core=CoreConfig(),
output=OutputConfig(
response_format=ResponseFormat.JSON,
response_mime_type="application/json",
),
)
@staticmethod
def gemini_2_5_thinking(tokens: int = -1) -> LLMGenerationConfig:
"""Gemini 2.5 思考模式：默认 -1 (动态思考)，0 为禁用，>=1024 为固定预算"""
return LLMGenerationConfig(
core=CoreConfig(temperature=1.0),
reasoning=ReasoningConfig(budget_tokens=tokens, show_thoughts=True),
)
@staticmethod
def gemini_3_thinking(level: str = "HIGH") -> LLMGenerationConfig:
"""Gemini 3 深度思考模式：使用思考等级"""
try:
effort = ReasoningEffort(level.upper())
except ValueError:
effort = ReasoningEffort.HIGH
return LLMGenerationConfig(
core=CoreConfig(),
reasoning=ReasoningConfig(effort=effort, show_thoughts=True),
)
@staticmethod
def gemini_structured(schema: dict[str, Any]) -> LLMGenerationConfig:
"""Gemini 结构化输出：自定义JSON模式"""
return LLMGenerationConfig(
core=CoreConfig(),
output=OutputConfig(
response_mime_type="application/json", response_schema=schema
),
)
@staticmethod
def gemini_safe() -> LLMGenerationConfig:
"""Gemini 安全模式：使用配置的安全设置"""
threshold = get_gemini_safety_threshold()
return LLMGenerationConfig(
core=CoreConfig(),
safety=SafetyConfig(
safety_settings={
"HARM_CATEGORY_HARASSMENT": threshold,
"HARM_CATEGORY_HATE_SPEECH": threshold,
"HARM_CATEGORY_SEXUALLY_EXPLICIT": threshold,
"HARM_CATEGORY_DANGEROUS_CONTENT": threshold,
}
),
)
@staticmethod
def gemini_code_execution() -> LLMGenerationConfig:
"""Gemini 代码执行模式：启用代码执行功能"""
return LLMGenerationConfig(
core=CoreConfig(),
custom_params={"code_execution_timeout": 30},
)
@staticmethod
def gemini_grounding() -> LLMGenerationConfig:
"""Gemini 信息来源关联模式：启用Google搜索"""
return LLMGenerationConfig(
core=CoreConfig(),
custom_params={
"grounding_config": {"dynamicRetrievalConfig": {"mode": "MODE_DYNAMIC"}}
},
)
@staticmethod
def gemini_nano_banana(aspect_ratio: str = "16:9") -> LLMGenerationConfig:
"""Gemini Nano Banana Pro：自定义比例生图"""
try:
ar = ImageAspectRatio(aspect_ratio)
except ValueError:
ar = ImageAspectRatio.LANDSCAPE_16_9
return LLMGenerationConfig(
core=CoreConfig(),
visual=VisualConfig(aspect_ratio=ar),
)
@staticmethod
def gemini_high_res() -> LLMGenerationConfig:
"""Gemini 3: 强制使用高解析度处理输入媒体"""
return LLMGenerationConfig(
visual=VisualConfig(media_resolution="HIGH", resolution=ImageResolution.HD)
)
zhenxun/services/llm/config/providers.py
-451
View File
@@ -1,451 +0,0 @@
"""
LLM 提供商配置管理
负责注册和管理 AI 服务提供商的配置项。
"""
from functools import lru_cache
from typing import Any
from pydantic import BaseModel, Field
from zhenxun.configs.config import Config
from zhenxun.configs.utils import parse_as
from zhenxun.services.log import logger
from zhenxun.utils.manager.priority_manager import PriorityLifecycle
from zhenxun.utils.pydantic_compat import model_dump
from ..core import key_store
from ..tools import tool_provider_manager
from ..types.models import ModelDetail, ProviderConfig
AI_CONFIG_GROUP = "AI"
PROVIDERS_CONFIG_KEY = "PROVIDERS"
class DebugLogOptions(BaseModel):
"""调试日志细粒度控制"""
show_tools: bool = Field(
default=True, description="是否在日志中显示工具定义(JSON Schema)"
)
show_schema: bool = Field(
default=True, description="是否在日志中显示结构化输出Schema(response_format)"
)
show_safety: bool = Field(
default=True, description="是否在日志中显示安全设置(safetySettings)"
)
def __bool__(self) -> bool:
"""支持 bool(debug_options) 的语法，方便兼容旧逻辑。"""
return self.show_tools or self.show_schema or self.show_safety
class ClientSettings(BaseModel):
"""LLM 客户端通用设置"""
timeout: int = Field(default=300, description="API请求超时时间（秒）")
max_retries: int = Field(default=3, description="请求失败时的最大重试次数")
retry_delay: int = Field(default=2, description="请求重试的基础延迟时间（秒）")
structured_retries: int = Field(
default=2, description="结构化生成校验失败时的最大重试次数 (IVR)"
)
proxy: str | None = Field(
default=None,
description="网络代理，例如 http://127.0.0.1:7890",
)
class LLMConfig(BaseModel):
"""LLM 服务配置类"""
default_model_name: str | None = Field(
default=None,
description="LLM服务全局默认使用的模型名称 (格式: ProviderName/ModelName)",
)
client_settings: ClientSettings = Field(
default_factory=ClientSettings, description="客户端连接与重试配置"
)
providers: list[ProviderConfig] = Field(
default_factory=list, description="配置多个 AI 服务提供商及其模型信息"
)
debug_log: DebugLogOptions | bool = Field(
default_factory=DebugLogOptions,
description="LLM请求日志详情开关。支持 bool (全开/全关) 或 dict (细粒度控制)。",
)
def get_provider_by_name(self, name: str) -> ProviderConfig | None:
"""根据名称获取提供商配置
参数:
name: 提供商名称
返回:
ProviderConfig | None: 提供商配置，如果未找到则返回 None
"""
for provider in self.providers:
if provider.name == name:
return provider
return None
def get_model_by_provider_and_name(
self, provider_name: str, model_name: str
) -> tuple[ProviderConfig, ModelDetail] | None:
"""根据提供商名称和模型名称获取配置
参数:
provider_name: 提供商名称
model_name: 模型名称
返回:
tuple[ProviderConfig, ModelDetail] | None: 提供商配置和模型详情的元组，
如果未找到则返回 None
"""
provider = self.get_provider_by_name(provider_name)
if not provider:
return None
for model in provider.models:
if model.model_name == model_name:
return provider, model
return None
def list_available_models(self) -> list[dict[str, Any]]:
"""列出所有可用的模型
返回:
list[dict[str, Any]]: 模型信息列表
"""
models = []
for provider in self.providers:
for model in provider.models:
models.append(
{
"provider_name": provider.name,
"model_name": model.model_name,
"full_name": f"{provider.name}/{model.model_name}",
"is_available": model.is_available,
"is_embedding_model": model.is_embedding_model,
"api_type": provider.api_type,
}
)
return models
def validate_model_name(self, provider_model_name: str) -> bool:
"""验证模型名称格式是否正确
参数:
provider_model_name: 格式为 "ProviderName/ModelName" 的字符串
返回:
bool: 是否有效
"""
if not provider_model_name or "/" not in provider_model_name:
return False
parts = provider_model_name.split("/", 1)
if len(parts) != 2:
return False
provider_name, model_name = parts
return (
self.get_model_by_provider_and_name(provider_name, model_name) is not None
)
def get_ai_config():
"""获取 AI 配置组"""
return Config.get(AI_CONFIG_GROUP)
def get_default_providers() -> list[dict[str, Any]]:
"""获取默认的提供商配置
返回:
list[dict[str, Any]]: 默认提供商配置列表
"""
return [
{
"name": "DeepSeek",
"api_key": "YOUR_ARK_API_KEY",
"api_base": "https://api.deepseek.com",
"api_type": "openai",
"extra_headers": {},
"models": [
{
"model_name": "deepseek-chat",
"max_tokens": 4096,
"temperature": 0.7,
},
{
"model_name": "deepseek-reasoner",
},
],
},
{
"name": "ARK",
"api_key": "YOUR_ARK_API_KEY",
"api_base": "https://ark.cn-beijing.volces.com",
"api_type": "ark",
"models": [
{"model_name": "deepseek-r1-250528"},
{"model_name": "doubao-seed-1-6-250615"},
{"model_name": "doubao-seed-1-6-flash-250615"},
{"model_name": "doubao-seed-1-6-thinking-250615"},
],
},
{
"name": "siliconflow",
"api_key": "YOUR_ARK_API_KEY",
"api_base": "https://api.siliconflow.cn",
"api_type": "openai",
"models": [
{"model_name": "deepseek-ai/DeepSeek-V3"},
],
},
{
"name": "GLM",
"api_key": "YOUR_ARK_API_KEY",
"api_base": "https://open.bigmodel.cn",
"api_type": "zhipu",
"models": [
{"model_name": "glm-4-flash"},
{"model_name": "glm-4-plus"},
],
},
{
"name": "Gemini",
"api_key": [
"AIzaSy*****************************",
"AIzaSy*****************************",
"AIzaSy*****************************",
],
"api_base": "https://generativelanguage.googleapis.com",
"api_type": "gemini",
"models": [
{"model_name": "gemini-2.5-flash"},
{"model_name": "gemini-2.5-pro"},
{"model_name": "gemini-2.5-flash-lite"},
],
},
{
"name": "OpenRouter",
"api_key": "YOUR_OPENROUTER_API_KEY",
"api_base": "https://openrouter.ai/api",
"api_type": "openrouter",
"models": [
{"model_name": "google/gemini-2.5-pro"},
{"model_name": "google/gemini-2.5-flash"},
{"model_name": "x-ai/grok-4"},
],
},
]
def register_llm_configs():
"""注册 LLM 服务的配置项"""
logger.info("注册 LLM 服务的配置项")
llm_config = LLMConfig()
Config.add_plugin_config(
AI_CONFIG_GROUP,
"default_model_name",
llm_config.default_model_name,
help="LLM服务全局默认使用的模型名称 (格式: ProviderName/ModelName)",
type=str,
)
Config.add_plugin_config(
AI_CONFIG_GROUP,
"client_settings",
model_dump(llm_config.client_settings),
help=(
"LLM客户端高级设置。\n"
"包含: timeout(超时秒数), max_retries(重试次数), "
"retry_delay(重试延迟), structured_retries(结构化生成重试), proxy(代理)"
),
type=dict,
)
Config.add_plugin_config(
AI_CONFIG_GROUP,
"debug_log",
{"show_tools": True, "show_schema": True, "show_safety": True},
help=(
"LLM日志详情开关。示例: {'show_tools': True, 'show_schema': False, "
"'show_safety': False}"
),
type=dict,
)
Config.add_plugin_config(
AI_CONFIG_GROUP,
"gemini_safety_threshold",
"BLOCK_NONE",
help=(
"Gemini 安全过滤阈值 "
"(BLOCK_LOW_AND_ABOVE: 阻止低级别及以上, "
"BLOCK_MEDIUM_AND_ABOVE: 阻止中等级别及以上, "
"BLOCK_ONLY_HIGH: 只阻止高级别, "
"BLOCK_NONE: 不阻止)"
),
type=str,
)
Config.add_plugin_config(
AI_CONFIG_GROUP,
PROVIDERS_CONFIG_KEY,
get_default_providers(),
help=(
"配置多个 AI 服务提供商及其模型信息。\n"
"注意：可以在特定模型配置下添加 'api_type' 以覆盖提供商的全局设置。\n"
"可选：在 provider 下添加 'extra_headers' 传递额外请求头，"
"用于 AI 网关鉴权（例如 cf-aig-authorization）。\n"
"支持的 api_type 包括:\n"
"- 'openai': 标准 OpenAI 格式 (DeepSeek, SiliconFlow, Moonshot 等)\n"
"- 'gemini': Google Gemini API\n"
"- 'zhipu': 智谱 AI (GLM)\n"
"- 'ark': 字节跳动火山引擎 (Doubao)\n"
"- 'openrouter': OpenRouter 聚合平台\n"
"- 'openai_image': OpenAI 兼容的图像生成接口 (DALL-E)\n"
"- 'openai_responses': 支持新版 responses 格式的 OpenAI 兼容接口\n"
"- 'smart': 智能路由模式 (主要用于第三方中转场景，自动根据模型名"
"分发请求到 openai 或 gemini)"
),
default_value=[],
type=list[ProviderConfig],
)
@lru_cache(maxsize=1)
def get_llm_config() -> LLMConfig:
"""获取 LLM 配置实例"""
ai_config = get_ai_config()
raw_debug = ai_config.get("debug_log", False)
if isinstance(raw_debug, bool):
debug_log_val = DebugLogOptions(
show_tools=raw_debug, show_schema=raw_debug, show_safety=raw_debug
)
else:
debug_log_val = raw_debug
config_data = {
"default_model_name": ai_config.get("default_model_name"),
"client_settings": ai_config.get("client_settings", {}),
"debug_log": debug_log_val,
PROVIDERS_CONFIG_KEY: ai_config.get(PROVIDERS_CONFIG_KEY, []),
}
return parse_as(LLMConfig, config_data)
def get_gemini_safety_threshold() -> str:
"""获取 Gemini 安全过滤阈值配置
返回:
str: 安全过滤阈值
"""
ai_config = get_ai_config()
return ai_config.get("gemini_safety_threshold", "BLOCK_MEDIUM_AND_ABOVE")
def validate_llm_config() -> tuple[bool, list[str]]:
"""验证 LLM 配置的有效性
返回:
tuple[bool, list[str]]: (是否有效, 错误信息列表)
"""
errors = []
try:
llm_config = get_llm_config()
if llm_config.client_settings.timeout <= 0:
errors.append("timeout 必须大于 0")
if llm_config.client_settings.max_retries < 0:
errors.append("max_retries 不能小于 0")
if llm_config.client_settings.retry_delay <= 0:
errors.append("retry_delay 必须大于 0")
if not llm_config.providers:
errors.append("至少需要配置一个 AI 服务提供商")
else:
provider_names = set()
for provider in llm_config.providers:
if provider.name in provider_names:
errors.append(f"提供商名称重复: {provider.name}")
provider_names.add(provider.name)
if not provider.api_key:
errors.append(f"提供商 {provider.name} 缺少 API Key")
if not provider.models:
errors.append(f"提供商 {provider.name} 没有配置任何模型")
else:
model_names = set()
for model in provider.models:
if model.model_name in model_names:
errors.append(
f"提供商 {provider.name} 中模型名称重复: "
f"{model.model_name}"
)
model_names.add(model.model_name)
if llm_config.default_model_name:
if not llm_config.validate_model_name(llm_config.default_model_name):
errors.append(
f"默认模型 {llm_config.default_model_name} 在配置中不存在"
)
except Exception as e:
errors.append(f"配置解析失败: {e!s}")
return len(errors) == 0, errors
def set_default_model(provider_model_name: str | None) -> bool:
"""设置默认模型
参数:
provider_model_name: 模型名称，格式为 "ProviderName/ModelName"，None 表示清除
返回:
bool: 是否设置成功
"""
if provider_model_name:
llm_config = get_llm_config()
if not llm_config.validate_model_name(provider_model_name):
logger.error(f"模型 {provider_model_name} 在配置中不存在")
return False
Config.set_config(
AI_CONFIG_GROUP, "default_model_name", provider_model_name, auto_save=True
)
if provider_model_name:
logger.info(f"默认模型已设置为: {provider_model_name}")
else:
logger.info("默认模型已清除")
return True
@PriorityLifecycle.on_startup(priority=10)
async def _init_llm_config_on_startup():
"""
在服务启动时主动调用一次 get_llm_config 和 key_store.initialize，
并预热工具提供者管理器。
"""
logger.info("正在初始化 LLM 配置并加载密钥状态...")
try:
get_llm_config()
await key_store.initialize()
logger.debug("LLM 配置和密钥状态初始化完成。")
logger.debug("正在预热 LLM 工具提供者管理器...")
await tool_provider_manager.initialize()
logger.debug("LLM 工具提供者管理器预热完成。")
except Exception as e:
logger.error(f"LLM 配置或密钥状态初始化时发生错误: {e}", e=e)
zhenxun/services/llm/core.py
-596
View File
@@ -1,596 +0,0 @@
"""
LLM 核心基础设施模块
包含执行 LLM 请求所需的底层组件，如 HTTP 客户端、API Key 存储和智能重试逻辑。
"""
import asyncio
from dataclasses import asdict, dataclass
from enum import IntEnum
import json
import os
import time
from typing import Any
import aiofiles
import httpx
import nonebot
from pydantic import BaseModel
from zhenxun.configs.path_config import DATA_PATH
from zhenxun.services.log import logger
from zhenxun.utils.user_agent import get_user_agent
from .types import ProviderConfig
from .types.exceptions import LLMErrorCode, LLMException
driver = nonebot.get_driver()
class HttpClientConfig(BaseModel):
"""HTTP客户端配置"""
timeout: int = 180
max_connections: int = 100
max_keepalive_connections: int = 20
proxy: str | None = None
class LLMHttpClient:
"""LLM服务专用HTTP客户端"""
def __init__(self, config: HttpClientConfig | None = None):
self.config = config or HttpClientConfig()
self._client: httpx.AsyncClient | None = None
self._active_requests = 0
self._lock = asyncio.Lock()
async def _ensure_client_initialized(self) -> httpx.AsyncClient:
if self._client is None or self._client.is_closed:
async with self._lock:
if self._client is None or self._client.is_closed:
logger.debug(
f"LLMHttpClient: 正在初始化新的 httpx.AsyncClient "
f"配置: {self.config}"
)
headers = get_user_agent()
limits = httpx.Limits(
max_connections=self.config.max_connections,
max_keepalive_connections=self.config.max_keepalive_connections,
)
timeout = httpx.Timeout(self.config.timeout)
client_kwargs = {}
if self.config.proxy:
try:
version_parts = httpx.__version__.split(".")
major = int(
"".join(c for c in version_parts[0] if c.isdigit())
)
minor = (
int("".join(c for c in version_parts[1] if c.isdigit()))
if len(version_parts) > 1
else 0
)
if (major, minor) >= (0, 28):
client_kwargs["proxy"] = self.config.proxy
else:
client_kwargs["proxies"] = self.config.proxy
except (ValueError, IndexError):
client_kwargs["proxies"] = self.config.proxy
logger.warning(
f"无法解析 httpx 版本 '{httpx.__version__}'，"
"LLM模块将默认使用旧版 'proxies' 参数语法。"
)
self._client = httpx.AsyncClient(
headers=headers,
limits=limits,
timeout=timeout,
follow_redirects=True,
**client_kwargs,
)
if self._client is None:
raise LLMException(
"HTTP 客户端初始化失败。", LLMErrorCode.CONFIGURATION_ERROR
)
return self._client
async def post(self, url: str, **kwargs: Any) -> httpx.Response:
client = await self._ensure_client_initialized()
async with self._lock:
self._active_requests += 1
try:
return await client.post(url, **kwargs)
finally:
async with self._lock:
self._active_requests -= 1
async def close(self):
async with self._lock:
if self._client and not self._client.is_closed:
logger.debug(
f"LLMHttpClient: 正在关闭，配置: {self.config}. "
f"活跃请求数: {self._active_requests}"
)
if self._active_requests > 0:
logger.warning(
f"LLMHttpClient: 关闭时仍有 {self._active_requests} "
f"个请求处于活跃状态。"
)
await self._client.aclose()
self._client = None
logger.debug(f"配置为 {self.config} 的 LLMHttpClient 已完全关闭。")
@property
def is_closed(self) -> bool:
return self._client is None or self._client.is_closed
class LLMHttpClientManager:
"""管理 LLMHttpClient 实例的工厂和池"""
def __init__(self):
self._clients: dict[tuple[int, str | None], LLMHttpClient] = {}
self._lock = asyncio.Lock()
def _get_client_key(
self, provider_config: ProviderConfig
) -> tuple[int, str | None]:
return (provider_config.timeout, provider_config.proxy)
async def get_client(self, provider_config: ProviderConfig) -> LLMHttpClient:
key = self._get_client_key(provider_config)
async with self._lock:
client = self._clients.get(key)
if client and not client.is_closed:
logger.debug(
f"LLMHttpClientManager: 复用现有的 LLMHttpClient 密钥: {key}"
)
return client
if client and client.is_closed:
logger.debug(
f"LLMHttpClientManager: 发现密钥 {key} 对应的客户端已关闭。"
f"正在创建新的客户端。"
)
logger.debug(f"LLMHttpClientManager: 为密钥 {key} 创建新的 LLMHttpClient")
http_client_config = HttpClientConfig(
timeout=provider_config.timeout, proxy=provider_config.proxy
)
new_client = LLMHttpClient(config=http_client_config)
self._clients[key] = new_client
return new_client
async def shutdown(self):
async with self._lock:
logger.info(
f"LLMHttpClientManager: 正在关闭。关闭 {len(self._clients)} 个客户端。"
)
close_tasks = [
client.close()
for client in self._clients.values()
if client and not client.is_closed
]
if close_tasks:
await asyncio.gather(*close_tasks, return_exceptions=True)
self._clients.clear()
logger.info("LLMHttpClientManager: 关闭完成。")
http_client_manager = LLMHttpClientManager()
async def create_llm_http_client(
timeout: int = 180,
proxy: str | None = None,
) -> LLMHttpClient:
"""
创建LLM HTTP客户端
参数:
timeout: 超时时间（秒）。
proxy: 代理服务器地址。
返回:
LLMHttpClient: HTTP客户端实例。
"""
config = HttpClientConfig(timeout=timeout, proxy=proxy)
return LLMHttpClient(config)
class KeyStatus(IntEnum):
"""用于排序和展示的密钥状态枚举"""
DISABLED = 0
ERROR = 1
COOLDOWN = 2
WARNING = 3
HEALTHY = 4
UNUSED = 5
@dataclass
class KeyStats:
"""单个API Key的详细状态和统计信息"""
cooldown_until: float = 0.0
success_count: int = 0
failure_count: int = 0
total_latency: float = 0.0
last_error_info: str | None = None
@property
def is_available(self) -> bool:
"""检查Key当前是否可用"""
return time.time() >= self.cooldown_until
@property
def avg_latency(self) -> float:
"""计算平均延迟"""
return (
self.total_latency / self.success_count if self.success_count > 0 else 0.0
)
@property
def success_rate(self) -> float:
"""计算成功率"""
total = self.success_count + self.failure_count
return self.success_count / total * 100 if total > 0 else 100.0
@property
def status(self) -> KeyStatus:
"""根据当前统计数据动态计算状态"""
now = time.time()
cooldown_left = max(0, self.cooldown_until - now)
if cooldown_left > 31536000 - 60:
return KeyStatus.DISABLED
if cooldown_left > 0:
return KeyStatus.COOLDOWN
total_calls = self.success_count + self.failure_count
if total_calls == 0:
return KeyStatus.UNUSED
if self.success_rate < 70:
return KeyStatus.ERROR
if total_calls >= 5 and self.avg_latency > 15000:
return KeyStatus.WARNING
return KeyStatus.HEALTHY
@property
def suggested_action(self) -> str:
"""根据状态给出建议操作"""
status_actions = {
KeyStatus.DISABLED: "更换Key",
KeyStatus.ERROR: "检查网络/重置",
KeyStatus.COOLDOWN: "等待/重置",
KeyStatus.WARNING: "观察",
KeyStatus.HEALTHY: "-",
KeyStatus.UNUSED: "-",
}
return status_actions.get(self.status, "未知")
class RetryConfig:
"""重试配置"""
def __init__(
self,
max_retries: int = 3,
retry_delay: float = 1.0,
exponential_backoff: bool = True,
key_rotation: bool = True,
):
self.max_retries = max_retries
self.retry_delay = retry_delay
self.exponential_backoff = exponential_backoff
self.key_rotation = key_rotation
def _should_retry_llm_error(
error: LLMException, attempt: int, max_retries: int
) -> bool:
"""判断LLM错误是否应该重试"""
non_retryable_errors = {
LLMErrorCode.MODEL_NOT_FOUND,
LLMErrorCode.CONTEXT_LENGTH_EXCEEDED,
LLMErrorCode.USER_LOCATION_NOT_SUPPORTED,
LLMErrorCode.INVALID_PARAMETER,
LLMErrorCode.CONFIGURATION_ERROR,
LLMErrorCode.API_KEY_INVALID,
}
if error.code in non_retryable_errors:
return False
retryable_errors = {
LLMErrorCode.API_REQUEST_FAILED,
LLMErrorCode.API_TIMEOUT,
LLMErrorCode.API_RATE_LIMITED,
LLMErrorCode.API_RESPONSE_INVALID,
LLMErrorCode.RESPONSE_PARSE_ERROR,
LLMErrorCode.GENERATION_FAILED,
LLMErrorCode.CONTENT_FILTERED,
LLMErrorCode.API_QUOTA_EXCEEDED,
}
if error.code in retryable_errors:
if error.code == LLMErrorCode.API_QUOTA_EXCEEDED:
return attempt < min(2, max_retries)
return True
return False
class KeyStatusStore:
"""API Key 状态管理存储 - 支持持久化"""
def __init__(self):
self._key_stats: dict[str, KeyStats] = {}
self._provider_key_index: dict[str, int] = {}
self._lock = asyncio.Lock()
self._file_path = DATA_PATH / "llm" / "key_status.json"
async def initialize(self):
"""从文件异步加载密钥状态，在应用启动时调用"""
async with self._lock:
if not self._file_path.exists():
logger.info("未找到密钥状态文件，将使用内存状态启动。")
return
try:
logger.info(f"正在从 {self._file_path} 加载密钥状态...")
async with aiofiles.open(self._file_path, encoding="utf-8") as f:
content = await f.read()
if not content:
logger.warning("密钥状态文件为空。")
return
data = json.loads(content)
for key, stats_dict in data.items():
self._key_stats[key] = KeyStats(**stats_dict)
logger.info(f"成功加载 {len(self._key_stats)} 个密钥的状态。")
except json.JSONDecodeError:
logger.error(f"密钥状态文件 {self._file_path} 格式错误，无法解析。")
except Exception as e:
logger.error(f"加载密钥状态文件时发生错误: {e}", e=e)
async def _save_to_file_internal(self):
"""
[内部方法] 将当前密钥状态安全地写入JSON文件。
假定调用方已持有锁。
"""
data_to_save = {key: asdict(stats) for key, stats in self._key_stats.items()}
try:
self._file_path.parent.mkdir(parents=True, exist_ok=True)
temp_path = self._file_path.with_suffix(".json.tmp")
async with aiofiles.open(temp_path, "w", encoding="utf-8") as f:
await f.write(json.dumps(data_to_save, ensure_ascii=False, indent=2))
if self._file_path.exists():
self._file_path.unlink()
os.rename(temp_path, self._file_path)
logger.debug("密钥状态已成功持久化到文件。")
except Exception as e:
logger.error(f"保存密钥状态到文件失败: {e}", e=e)
async def shutdown(self):
"""在应用关闭时安全地保存状态"""
async with self._lock:
await self._save_to_file_internal()
logger.info("KeyStatusStore 已在关闭前保存状态。")
async def get_next_available_key(
self,
provider_name: str,
api_keys: list[str],
exclude_keys: set[str] | None = None,
) -> str | None:
"""
获取下一个可用的API密钥（轮询策略）
参数:
provider_name: 提供商名称。
api_keys: API密钥列表。
exclude_keys: 要排除的密钥集合。
返回:
str | None: 可用的API密钥，如果没有可用密钥则返回None。
"""
if not api_keys:
return None
exclude_keys = exclude_keys or set()
async with self._lock:
for key in api_keys:
if key not in self._key_stats:
self._key_stats[key] = KeyStats()
available_keys = [
key
for key in api_keys
if key not in exclude_keys and self._key_stats[key].is_available
]
if not available_keys:
return api_keys[0]
current_index = self._provider_key_index.get(provider_name, 0)
selected_key = available_keys[current_index % len(available_keys)]
self._provider_key_index[provider_name] = current_index + 1
total_usage = (
self._key_stats[selected_key].success_count
+ self._key_stats[selected_key].failure_count
)
logger.debug(
f"轮询选择API密钥: {self._get_key_id(selected_key)} "
f"(使用次数: {total_usage})"
)
return selected_key
async def record_success(self, api_key: str, latency: float):
"""记录成功使用，并持久化"""
async with self._lock:
stats = self._key_stats.setdefault(api_key, KeyStats())
stats.cooldown_until = 0.0
stats.success_count += 1
stats.total_latency += latency
stats.last_error_info = None
await self._save_to_file_internal()
logger.debug(
f"记录API密钥成功使用: {self._get_key_id(api_key)}, 延迟: {latency:.2f}ms"
)
async def record_failure(
self, api_key: str, status_code: int | None, error_message: str
):
"""
记录失败使用，并设置冷却时间
参数:
api_key: API密钥。
status_code: HTTP状态码。
error_message: 错误信息。
"""
key_id = self._get_key_id(api_key)
now = time.time()
cooldown_duration = 300
location_not_supported = error_message and (
"USER_LOCATION_NOT_SUPPORTED" in error_message
or "User location is not supported" in error_message
)
if location_not_supported:
logger.warning(
f"API Key {key_id} 请求失败，原因是地区不支持 (Gemini)。"
" 这通常是代理节点问题，Key 本身可能是正常的。跳过冷却。"
)
async with self._lock:
stats = self._key_stats.setdefault(api_key, KeyStats())
stats.failure_count += 1
stats.last_error_info = error_message[:256]
await self._save_to_file_internal()
return
if error_message and (
"API_QUOTA_EXCEEDED" in error_message
or "insufficient_quota" in error_message.lower()
):
cooldown_duration = 3600
logger.warning(f"API Key {key_id} 额度耗尽，冷却 1 小时。")
is_key_invalid = status_code == 401 or (
status_code == 400
and error_message
and (
"API_KEY_INVALID" in error_message
or "API key not valid" in error_message
)
)
if is_key_invalid:
cooldown_duration = 31536000
log_level = "error"
log_message = f"API密钥认证/权限/路径错误，将永久禁用: {key_id}"
elif status_code == 403:
cooldown_duration = 3600
log_level = "warning"
log_message = f"API密钥权限不足或地区不支持(403)，冷却1小时: {key_id}"
elif status_code == 404:
log_level = "error"
log_message = "API请求返回 404 (未找到)，可能是模型名称错误或接口地址"
f"错误，不冷却密钥: {key_id}"
elif status_code == 422:
cooldown_duration = 0
log_level = "warning"
log_message = f"API请求无法处理(422)，可能是生成故障，不冷却密钥: {key_id}"
elif status_code == 429:
cooldown_duration = 60
log_level = "warning"
log_message = f"API密钥被限流，冷却60秒: {key_id}"
elif error_message and (
"ConnectError" in error_message
or "NetworkError" in error_message
or "Connection refused" in error_message
or "RemoteProtocolError" in error_message
or "ProxyError" in error_message
):
cooldown_duration = 0
log_level = "warning"
log_message = f"网络连接层异常(代理/DNS)，不冷却密钥: {key_id}"
else:
log_level = "warning"
log_message = f"API密钥遇到临时性错误，冷却{cooldown_duration}秒: {key_id}"
async with self._lock:
stats = self._key_stats.setdefault(api_key, KeyStats())
stats.cooldown_until = now + cooldown_duration
stats.failure_count += 1
stats.last_error_info = error_message[:256]
await self._save_to_file_internal()
getattr(logger, log_level)(log_message)
async def reset_key_status(self, api_key: str):
"""重置密钥状态，并持久化"""
async with self._lock:
stats = self._key_stats.setdefault(api_key, KeyStats())
stats.cooldown_until = 0.0
stats.last_error_info = None
await self._save_to_file_internal()
logger.info(f"重置API密钥状态: {self._get_key_id(api_key)}")
async def get_key_stats(self, api_keys: list[str]) -> dict[str, dict]:
"""
获取密钥使用统计，并计算出用于展示的派生数据。
参数:
api_keys: API密钥列表。
返回:
dict[str, dict]: 包含丰富状态和统计信息的密钥字典。
"""
stats_dict = {}
now = time.time()
async with self._lock:
for key in api_keys:
key_id = self._get_key_id(key)
stats = self._key_stats.get(key, KeyStats())
stats_dict[key_id] = {
"status_enum": stats.status,
"cooldown_seconds_left": max(0, stats.cooldown_until - now),
"total_calls": stats.success_count + stats.failure_count,
"success_count": stats.success_count,
"failure_count": stats.failure_count,
"success_rate": stats.success_rate,
"avg_latency": stats.avg_latency,
"last_error": stats.last_error_info,
"suggested_action": stats.suggested_action,
}
return stats_dict
def _get_key_id(self, api_key: str) -> str:
"""获取API密钥的标识符（用于日志）"""
if len(api_key) <= 8:
return api_key
return f"{api_key[:4]}...{api_key[-4:]}"
key_store = KeyStatusStore()
@driver.on_shutdown
async def _shutdown_key_store():
await key_store.shutdown()
zhenxun/services/llm/manager.py
-474
View File
@@ -1,474 +0,0 @@
"""
LLM 模型管理器
负责模型实例的创建、缓存、配置管理和生命周期管理。
"""
import hashlib
import time
from typing import Any
from zhenxun.configs.config import Config
from zhenxun.services.log import logger
from zhenxun.utils.pydantic_compat import dump_json_safely
from .config import validate_override_params
from .config.generation import LLMGenerationConfig
from .config.providers import (
AI_CONFIG_GROUP,
PROVIDERS_CONFIG_KEY,
get_ai_config,
get_llm_config,
)
from .core import http_client_manager, key_store
from .service import LLMModel
from .types import LLMErrorCode, LLMException, ModelDetail, ProviderConfig
from .types.capabilities import get_model_capabilities
DEFAULT_MODEL_NAME_KEY = "default_model_name"
_model_cache: dict[str, tuple[LLMModel, float]] = {}
_cache_ttl = 3600
_max_cache_size = 10
def parse_provider_model_string(name_str: str | None) -> tuple[str | None, str | None]:
"""解析 'ProviderName/ModelName' 格式的字符串"""
if not name_str or "/" not in name_str:
return None, None
parts = name_str.split("/", 1)
if len(parts) == 2 and parts[0].strip() and parts[1].strip():
return parts[0].strip(), parts[1].strip()
return None, None
def _make_cache_key(
provider_model_name: str | None,
override_config: dict | LLMGenerationConfig | None,
) -> str:
"""生成缓存键"""
config_str = (
dump_json_safely(override_config, sort_keys=True) if override_config else "None"
)
key_data = f"{provider_model_name}:{config_str}"
return hashlib.md5(key_data.encode()).hexdigest()
def _get_cached_model(cache_key: str) -> LLMModel | None:
"""从缓存获取模型"""
if cache_key in _model_cache:
model, created_time = _model_cache[cache_key]
current_time = time.time()
if current_time - created_time > _cache_ttl:
del _model_cache[cache_key]
logger.debug(f"模型缓存已过期: {cache_key}")
return None
if model._is_closed:
logger.debug(
f"缓存的模型 {cache_key} ({model.provider_name}/{model.model_name}) "
f"处于_is_closed=True状态，重置为False以供复用。"
)
model._is_closed = False
logger.debug(
f"使用缓存的模型: {cache_key} -> {model.provider_name}/{model.model_name}"
)
return model
return None
def _cache_model(cache_key: str, model: LLMModel):
"""缓存模型实例"""
current_time = time.time()
if len(_model_cache) >= _max_cache_size:
oldest_key = min(_model_cache.keys(), key=lambda k: _model_cache[k][1])
del _model_cache[oldest_key]
_model_cache[cache_key] = (model, current_time)
def clear_model_cache():
"""
清空模型缓存，释放所有缓存的模型实例。
用于在内存不足或需要强制重新加载模型配置时清理缓存。
"""
global _model_cache
_model_cache.clear()
logger.info("已清空模型缓存")
def get_cache_stats() -> dict[str, Any]:
"""
获取模型缓存的统计信息。
返回:
dict[str, Any]: 包含缓存大小、最大容量、TTL和已缓存模型列表的统计信息。
"""
return {
"cache_size": len(_model_cache),
"max_cache_size": _max_cache_size,
"cache_ttl": _cache_ttl,
"cached_models": list(_model_cache.keys()),
}
def get_default_api_base_for_type(api_type: str) -> str | None:
"""根据API类型获取默认的API基础地址"""
default_api_bases = {
"openai": "https://api.openai.com",
"deepseek": "https://api.deepseek.com/beta",
"zhipu": "https://open.bigmodel.cn",
"gemini": "https://generativelanguage.googleapis.com",
"openrouter": "https://openrouter.ai/api",
"smart": None,
"openai_responses": None,
}
return default_api_bases.get(api_type)
def get_configured_providers() -> list[ProviderConfig]:
"""从配置中获取Provider列表 - 简化和修正版本"""
ai_config = get_ai_config()
providers = ai_config.get(PROVIDERS_CONFIG_KEY, [])
if not isinstance(providers, list):
logger.error(
f"配置项 {AI_CONFIG_GROUP}.{PROVIDERS_CONFIG_KEY} 的值不是一个列表，"
f"将使用空列表。"
)
return []
valid_providers = []
for i, item in enumerate(providers):
if isinstance(item, ProviderConfig):
if not item.api_base:
default_api_base = get_default_api_base_for_type(item.api_type)
if default_api_base:
item.api_base = default_api_base
valid_providers.append(item)
else:
logger.warning(
f"配置文件中第 {i + 1} 项未能正确解析为 ProviderConfig 对象，已跳过。"
f"实际类型: {type(item)}"
)
return valid_providers
def find_model_config(
provider_name: str, model_name: str
) -> tuple[ProviderConfig, ModelDetail] | None:
"""
在配置中查找指定的 Provider 和 ModelDetail
参数:
provider_name: 提供商名称
model_name: 模型名称
返回:
tuple[ProviderConfig, ModelDetail] | None: 找到的配置元组，未找到则返回 None
"""
providers = get_configured_providers()
for provider in providers:
if provider.name.lower() == provider_name.lower():
for model_detail in provider.models:
if model_detail.model_name.lower() == model_name.lower():
return provider, model_detail
return None
def list_available_models() -> list[dict[str, Any]]:
"""
列出所有配置的可用模型及其详细信息。
返回:
list[dict[str, Any]]: 模型信息列表，每个字典包含提供商名称、模型名称、
能力信息、是否为嵌入模型等详细信息。
"""
providers = get_configured_providers()
model_list = []
for provider in providers:
for model_detail in provider.models:
model_info = {
"provider_name": provider.name,
"model_name": model_detail.model_name,
"full_name": f"{provider.name}/{model_detail.model_name}",
"api_type": provider.api_type or "auto-detect",
"api_base": provider.api_base,
"is_available": model_detail.is_available,
"is_embedding_model": model_detail.is_embedding_model,
"available_identifiers": _get_model_identifiers(
provider.name, model_detail
),
}
model_list.append(model_info)
return model_list
def _get_model_identifiers(provider_name: str, model_detail: ModelDetail) -> list[str]:
"""获取模型的所有可用标识符"""
return [f"{provider_name}/{model_detail.model_name}"]
def list_model_identifiers() -> dict[str, list[str]]:
"""
列出所有模型的可用标识符
返回:
dict[str, list[str]]: 字典，键为模型的完整名称，值为该模型的所有可用标识符列表
"""
providers = get_configured_providers()
result = {}
for provider in providers:
for model_detail in provider.models:
full_name = f"{provider.name}/{model_detail.model_name}"
identifiers = _get_model_identifiers(provider.name, model_detail)
result[full_name] = identifiers
return result
def list_embedding_models() -> list[dict[str, Any]]:
"""
列出所有配置的嵌入模型。
返回:
list[dict[str, Any]]: 嵌入模型信息列表，从所有可用模型中筛选出
支持嵌入功能的模型。
"""
all_models = list_available_models()
return [model for model in all_models if model.get("is_embedding_model", False)]
async def get_model_instance(
provider_model_name: str | None = None,
override_config: dict[str, Any] | LLMGenerationConfig | None = None,
) -> LLMModel:
"""
根据 'ProviderName/ModelName' 字符串获取并实例化 LLMModel (异步版本)
参数:
provider_model_name: 模型名称，格式为 'ProviderName/ModelName'。
override_config: 覆盖配置字典。
返回:
LLMModel: 模型实例。
"""
cache_key = _make_cache_key(provider_model_name, override_config)
cached_model = _get_cached_model(cache_key)
if cached_model:
if override_config:
validated_override = validate_override_params(override_config)
if cached_model._generation_config != validated_override:
cached_model._generation_config = validated_override
logger.debug(
f"对缓存模型 {provider_model_name} 应用新的覆盖配置: "
f"{validated_override.to_dict()}"
)
return cached_model
resolved_model_name_str = provider_model_name
if resolved_model_name_str is None:
resolved_model_name_str = get_global_default_model_name()
if resolved_model_name_str is None:
available_models_list = list_available_models()
if not available_models_list:
raise LLMException(
"未配置任何AI模型", code=LLMErrorCode.CONFIGURATION_ERROR
)
resolved_model_name_str = available_models_list[0]["full_name"]
logger.warning(f"未指定模型，使用第一个可用模型: {resolved_model_name_str}")
prov_name_str, mod_name_str = parse_provider_model_string(resolved_model_name_str)
if not prov_name_str or not mod_name_str:
raise LLMException(
f"无效的模型名称格式: '{resolved_model_name_str}'",
code=LLMErrorCode.MODEL_NOT_FOUND,
)
config_tuple_found = find_model_config(prov_name_str, mod_name_str)
if not config_tuple_found:
all_models = list_available_models()
raise LLMException(
f"未找到模型: '{resolved_model_name_str}'. "
f"可用: {[m['full_name'] for m in all_models]}",
code=LLMErrorCode.MODEL_NOT_FOUND,
)
provider_config_found, model_detail_found = config_tuple_found
capabilities = get_model_capabilities(model_detail_found.model_name)
model_detail_found.is_embedding_model = capabilities.is_embedding_model
llm_config = get_llm_config()
client_settings = llm_config.client_settings
default_timeout = (
provider_config_found.timeout
if provider_config_found.timeout is not None
else client_settings.timeout
)
config_for_http_client = ProviderConfig(
name=provider_config_found.name,
api_key=provider_config_found.api_key,
models=provider_config_found.models,
timeout=default_timeout,
proxy=client_settings.proxy,
api_base=provider_config_found.api_base,
api_type=provider_config_found.api_type,
openai_compat=provider_config_found.openai_compat,
temperature=provider_config_found.temperature,
max_tokens=provider_config_found.max_tokens,
extra_headers=provider_config_found.extra_headers,
)
shared_http_client = await http_client_manager.get_client(config_for_http_client)
try:
model_instance = LLMModel(
provider_config=config_for_http_client,
model_detail=model_detail_found,
key_store=key_store,
http_client=shared_http_client,
capabilities=capabilities,
)
if override_config:
validated_override_params = validate_override_params(override_config)
model_instance._generation_config = validated_override_params
logger.debug(
f"为新模型 {resolved_model_name_str} 应用配置覆盖: "
f"{validated_override_params.to_dict()}"
)
_cache_model(cache_key, model_instance)
logger.debug(
f"创建并缓存了新模型: {cache_key} -> {prov_name_str}/{mod_name_str}"
)
return model_instance
except LLMException:
raise
except Exception as e:
logger.error(
f"实例化 LLMModel ({resolved_model_name_str}) 时发生内部错误: {e!s}", e=e
)
raise LLMException(
f"初始化模型 '{resolved_model_name_str}' 失败: {e!s}",
code=LLMErrorCode.MODEL_INIT_FAILED,
cause=e,
)
def get_global_default_model_name() -> str | None:
"""获取全局默认模型名称"""
ai_config = get_ai_config()
return ai_config.get(DEFAULT_MODEL_NAME_KEY)
def set_global_default_model_name(provider_model_name: str | None) -> bool:
"""
设置全局默认模型名称
参数:
provider_model_name: 模型名称，格式为 'ProviderName/ModelName'。
返回:
bool: 设置是否成功。
"""
if provider_model_name:
prov_name, mod_name = parse_provider_model_string(provider_model_name)
if not prov_name or not mod_name or not find_model_config(prov_name, mod_name):
logger.error(
f"尝试设置的全局默认模型 '{provider_model_name}' 无效或未配置。"
)
return False
Config.set_config(
AI_CONFIG_GROUP, DEFAULT_MODEL_NAME_KEY, provider_model_name, auto_save=True
)
if provider_model_name:
logger.info(f"LLM 服务全局默认模型已更新为: {provider_model_name}")
else:
logger.info("LLM 服务全局默认模型已清除。")
return True
async def get_key_usage_stats() -> dict[str, Any]:
"""
获取所有Provider的Key使用统计
返回:
dict[str, Any]: 包含所有Provider的Key使用统计信息。
"""
providers = get_configured_providers()
stats = {}
for provider in providers:
provider_stats = await key_store.get_key_stats(
[provider.api_key]
if isinstance(provider.api_key, str)
else provider.api_key
)
stats[provider.name] = {
"total_keys": len(
[provider.api_key]
if isinstance(provider.api_key, str)
else provider.api_key
),
"key_stats": provider_stats,
}
return stats
async def reset_key_status(provider_name: str, api_key: str | None = None) -> bool:
"""
重置指定Provider的Key状态
参数:
provider_name: 提供商名称。
api_key: 要重置的特定API密钥，如果为None则重置所有密钥。
返回:
bool: 重置是否成功。
"""
providers = get_configured_providers()
target_provider = None
for provider in providers:
if provider.name.lower() == provider_name.lower():
target_provider = provider
break
if not target_provider:
logger.error(f"未找到Provider: {provider_name}")
return False
provider_keys = (
[target_provider.api_key]
if isinstance(target_provider.api_key, str)
else target_provider.api_key
)
if api_key:
if api_key in provider_keys:
await key_store.reset_key_status(api_key)
logger.info(f"已重置Provider '{provider_name}' 的指定Key状态")
return True
else:
logger.error(f"指定的Key不属于Provider '{provider_name}'")
return False
else:
for key in provider_keys:
await key_store.reset_key_status(key)
logger.info(f"已重置Provider '{provider_name}' 的所有Key状态")
return True
zhenxun/services/llm/memory.py
-243
View File
@@ -1,243 +0,0 @@
"""
LLM 服务 - 会话记忆模块
定义了LLM会话记忆的存储、策略和处理接口。
"""
from abc import ABC, abstractmethod
from collections import defaultdict
from collections.abc import Callable
from typing import Any
from pydantic import BaseModel, Field
from zhenxun.services.llm.types import LLMMessage
from zhenxun.services.log import logger
class AIConfig(BaseModel):
"""AI配置类 (为保持独立性而在此处保留一个副本，实际使用中可能来自更高层)"""
model: Any = None
default_embedding_model: Any = None
default_preserve_media_in_history: bool = False
tool_providers: list[Any] = Field(default_factory=list)
def __post_init__(self):
"""初始化后从配置中读取默认值"""
pass
class BaseMessageStore(ABC):
"""
底层存储接口 (DAO - Data Access Object)。
这是一个抽象基类，定义了消息数据最底层的 **持久化与检索 (CRUD)** 接口。
它只关心数据的存取，不涉及任何业务逻辑（如历史记录修剪）。
开发者如果希望将对话历史存储到 Redis、数据库或其他持久化后端，
应当实现这个接口。
"""
@abstractmethod
async def get_messages(self, session_id: str) -> list[LLMMessage]:
"""
根据会话ID获取完整的消息列表。
"""
raise NotImplementedError
@abstractmethod
async def add_messages(self, session_id: str, messages: list[LLMMessage]) -> None:
"""追加消息"""
raise NotImplementedError
@abstractmethod
async def set_messages(self, session_id: str, messages: list[LLMMessage]) -> None:
"""
完全覆盖指定会话ID的消息列表。
主要用于历史记录修剪等场景。
"""
raise NotImplementedError
@abstractmethod
async def clear(self, session_id: str) -> None:
"""清空指定会话ID的所有消息数据。"""
raise NotImplementedError
class InMemoryMessageStore(BaseMessageStore):
"""
一个基于内存的 `BaseMessageStore` 实现。
它使用一个Python字典来存储所有会话的消息，提供了最简单、最快速的存储方案。
这是框架的默认存储方式，实现了开箱即用。
注意：此实现是 **非持久化** 的，当应用程序重启时，所有对话历史都会丢失。
适用于测试、简单应用或不需要长期记忆的场景。
"""
def __init__(self):
self._data: dict[str, list[LLMMessage]] = defaultdict(list)
async def get_messages(self, session_id: str) -> list[LLMMessage]:
"""从内存字典中获取消息列表的副本。"""
return self._data.get(session_id, []).copy()
async def add_messages(self, session_id: str, messages: list[LLMMessage]) -> None:
"""向内存中的消息列表追加消息。"""
self._data[session_id].extend(messages)
async def set_messages(self, session_id: str, messages: list[LLMMessage]) -> None:
"""在内存中直接替换指定会话的消息列表。"""
self._data[session_id] = messages
async def clear(self, session_id: str) -> None:
"""从内存字典中删除指定会话的条目。"""
if session_id in self._data:
del self._data[session_id]
class BaseMemory(ABC):
"""
记忆系统上层逻辑基类 (Strategy Layer)。
此抽象基类定义了记忆系统的 **策略层** 接口。它负责对外提供统一的记忆操作
接口，并封装了具体的记忆管理策略，如历史记录的修剪、摘要生成等。
`AI` 会话客户端直接与此接口交互，而不关心底层的存储实现。
开发者可以通过实现此接口来创建自定义的记忆管理策略，例如：
- `SummarizationMemory`: 在历史记录过长时，自动调用LLM生成摘要来压缩历史。
- `VectorStoreMemory`: 将对话历史向量化并存入向量数据库，实现长期记忆检索。
"""
@abstractmethod
async def get_history(self, session_id: str) -> list[LLMMessage]:
"""获取用于构建模型输入的完整历史消息列表。"""
raise NotImplementedError
async def add_message(self, session_id: str, message: LLMMessage) -> None:
"""向记忆中添加单条消息。默认实现是调用 `add_messages`。"""
await self.add_messages(session_id, [message])
@abstractmethod
async def add_messages(self, session_id: str, messages: list[LLMMessage]) -> None:
"""向记忆中添加多条消息，并可能触发内部的记忆管理策略（如修剪）。"""
raise NotImplementedError
@abstractmethod
async def clear_history(self, session_id: str) -> None:
"""清空指定会话的全部记忆。"""
raise NotImplementedError
class ChatMemory(BaseMemory):
"""
标准聊天记忆实现：组合 Store + 滑动窗口策略。
这是 `BaseMemory` 的默认实现，它通过组合一个 `BaseMessageStore` 实例来
完成实际的数据存储，并在此之上实现了一个简单的“滑动窗口”记忆修剪策略。
"""
def __init__(self, store: BaseMessageStore, max_messages: int = 50):
self.store = store
self._max_messages = max_messages
async def _trim_history(self, session_id: str) -> None:
"""
记忆修剪策略：确保历史记录不超过 `_max_messages` 条。
如果存在系统消息 (System Prompt)，它将被永久保留在列表的第一位。
"""
history = await self.store.get_messages(session_id)
if len(history) <= self._max_messages:
return
has_system = history and history[0].role == "system"
new_history: list[LLMMessage] = []
if has_system:
keep_count = max(0, self._max_messages - 1)
new_history = [history[0], *history[-keep_count:]]
else:
new_history = history[-self._max_messages :]
await self.store.set_messages(session_id, new_history)
async def get_history(self, session_id: str) -> list[LLMMessage]:
"""直接从底层存储获取历史记录。"""
return await self.store.get_messages(session_id)
async def add_messages(self, session_id: str, messages: list[LLMMessage]) -> None:
"""添加消息到历史记录，并立即执行修剪策略。"""
await self.store.add_messages(session_id, messages)
await self._trim_history(session_id)
async def clear_history(self, session_id: str) -> None:
"""清空底层存储中的历史记录。"""
await self.store.clear(session_id)
class MemoryProcessor(ABC):
"""
记忆处理器接口 (Hook/Observer)。
这是一个扩展接口，允许开发者创建自定义的“记忆处理器”，以在记忆被修改后
执行额外的操作（“钩子”）。
当 `AI` 实例的记忆更新时，它会依次调用所有注册的 `MemoryProcessor`。
使用场景示例：
- `LoggingMemoryProcessor`: 将每一轮对话异步记录到外部日志系统。
- `SummarizationProcessor`: 在后台任务中检查对话长度，并在需要时生成摘要。
- `EntityExtractionProcessor`: 从对话中提取关键实体（如人名、地名）并存储。
"""
@abstractmethod
async def process(self, session_id: str, new_messages: list[LLMMessage]) -> None:
"""处理新添加到记忆中的消息。"""
pass
_default_memory_factory: Callable[[], BaseMemory] | None = None
def set_default_memory_backend(factory: Callable[[], BaseMemory]):
"""
设置全局默认记忆后端工厂，允许统一替换会话的记忆实现。
这是一个高级依赖注入函数，允许插件或项目在启动时用自定义的 `BaseMemory`
实现替换掉默认的 `ChatMemory(InMemoryMessageStore())`。
Args:
factory: 一个无参数的、返回 `BaseMemory` 实例的函数或类。
"""
global _default_memory_factory
_default_memory_factory = factory
def _get_default_memory() -> BaseMemory:
"""
[内部函数] 获取一个默认的记忆后端实例。
它会首先检查是否有通过 `set_default_memory_backend` 设置的全局工厂，
如果有，则使用该工厂创建实例；否则，返回一个标准的内存记忆实例。
"""
if _default_memory_factory:
logger.debug("使用自定义的默认记忆后端工厂构建实例。")
return _default_memory_factory()
logger.debug("未配置自定义记忆后端，使用默认的 ChatMemory。")
return ChatMemory(store=InMemoryMessageStore())
__all__ = [
"AIConfig",
"BaseMemory",
"BaseMessageStore",
"ChatMemory",
"InMemoryMessageStore",
"MemoryProcessor",
"_get_default_memory",
"set_default_memory_backend",
]
zhenxun/services/llm/service.py
-830
View File
@@ -1,830 +0,0 @@
"""
LLM 模型实现类
包含 LLM 模型的抽象基类和具体实现，负责与各种 AI 提供商的 API 交互。
"""
from abc import ABC, abstractmethod
import asyncio
from collections.abc import Awaitable, Callable
import json
import re
import time
from typing import Any, Literal, TypeVar, cast
import httpx
from pydantic import BaseModel, ConfigDict, Field
from zhenxun.services.log import logger
from zhenxun.utils.http_utils import AsyncHttpx
from zhenxun.utils.log_sanitizer import sanitize_for_logging
from zhenxun.utils.pydantic_compat import dump_json_safely
from .adapters.base import BaseAdapter, RequestData, process_image_data
from .config import LLMGenerationConfig
from .config.generation import LLMEmbeddingConfig
from .config.providers import get_llm_config
from .core import (
KeyStatusStore,
LLMHttpClient,
RetryConfig,
_should_retry_llm_error,
http_client_manager,
)
from .types import (
LLMErrorCode,
LLMException,
LLMMessage,
LLMResponse,
LLMToolCall,
ModelDetail,
ProviderConfig,
ToolChoice,
)
from .types.capabilities import ModelCapabilities, ModelModality
T = TypeVar("T", bound=BaseModel)
def _sanitize_request_headers(headers: dict[str, Any]) -> dict[str, str]:
sanitized: dict[str, str] = {}
sensitive_parts = ("authorization", "token", "api-key", "api_key", "secret")
for key, value in headers.items():
key_text = str(key)
value_text = str(value)
lowered = key_text.lower()
if any(part in lowered for part in sensitive_parts):
if " " in value_text:
prefix = value_text.split(" ", 1)[0]
sanitized[key_text] = f"{prefix} ***"
else:
sanitized[key_text] = "***"
else:
sanitized[key_text] = value_text
return sanitized
class LLMContext(BaseModel):
"""LLM 执行上下文，用于在中间件管道中传递请求状态"""
messages: list[LLMMessage]
config: LLMGenerationConfig | LLMEmbeddingConfig
tools: list[Any] | None
tool_choice: str | dict[str, Any] | ToolChoice | None
timeout: float | None
extra: dict[str, Any] = Field(default_factory=dict)
request_type: Literal["generation", "embedding"] = "generation"
runtime_state: dict[str, Any] = Field(
default_factory=dict,
description="中间件运行时的临时状态存储(api_key, retry_count等)",
)
model_config = ConfigDict(arbitrary_types_allowed=True)
NextCall = Callable[[LLMContext], Awaitable[LLMResponse]]
LLMMiddleware = Callable[[LLMContext, NextCall], Awaitable[LLMResponse]]
class BaseLLMMiddleware(ABC):
"""LLM 中间件抽象基类"""
@abstractmethod
async def __call__(self, context: LLMContext, next_call: NextCall) -> LLMResponse:
"""
执行中间件逻辑
Args:
context: 请求上下文，包含配置和运行时状态
next_call: 调用链中的下一个处理函数
Returns:
LLMResponse: 模型响应结果
"""
pass
class LLMModelBase(ABC):
"""LLM模型抽象基类"""
@abstractmethod
async def generate_response(
self,
messages: list[LLMMessage],
config: LLMGenerationConfig | None = None,
tools: list[Any] | None = None,
tool_choice: str | dict[str, Any] | ToolChoice | None = None,
timeout: float | None = None,
) -> LLMResponse:
"""生成高级响应"""
pass
@abstractmethod
async def generate_embeddings(
self,
texts: list[str],
config: LLMEmbeddingConfig,
) -> list[list[float]]:
"""生成文本嵌入向量"""
pass
class LLMModel(LLMModelBase):
"""LLM 模型实现类"""
def __init__(
self,
provider_config: ProviderConfig,
model_detail: ModelDetail,
key_store: KeyStatusStore,
http_client: LLMHttpClient,
capabilities: ModelCapabilities,
config_override: LLMGenerationConfig | None = None,
):
self.provider_config = provider_config
self.model_detail = model_detail
self.key_store = key_store
self.http_client: LLMHttpClient = http_client
self.capabilities = capabilities
self._generation_config = config_override
self.provider_name = provider_config.name
self.api_type = provider_config.api_type
self.api_base = provider_config.api_base
self.api_keys = (
[provider_config.api_key]
if isinstance(provider_config.api_key, str)
else provider_config.api_key
)
self.model_name = model_detail.model_name
self.temperature = model_detail.temperature
self.max_tokens = model_detail.max_tokens
self._is_closed = False
self._ref_count = 0
self._middlewares: list[LLMMiddleware] = []
def _has_modality(self, modality: ModelModality, is_input: bool = True) -> bool:
target_set = (
self.capabilities.input_modalities
if is_input
else self.capabilities.output_modalities
)
return modality in target_set
@property
def can_process_images(self) -> bool:
"""检查模型是否支持图片作为输入。"""
return self._has_modality(ModelModality.IMAGE)
@property
def can_process_video(self) -> bool:
"""检查模型是否支持视频作为输入。"""
return self._has_modality(ModelModality.VIDEO)
@property
def can_process_audio(self) -> bool:
"""检查模型是否支持音频作为输入。"""
return self._has_modality(ModelModality.AUDIO)
@property
def can_generate_images(self) -> bool:
"""检查模型是否支持生成图片。"""
return self._has_modality(ModelModality.IMAGE, is_input=False)
@property
def can_generate_audio(self) -> bool:
"""检查模型是否支持生成音频 (TTS)。"""
return self._has_modality(ModelModality.AUDIO, is_input=False)
@property
def is_embedding_model(self) -> bool:
"""检查这是否是一个嵌入模型。"""
return self.capabilities.is_embedding_model
def add_middleware(self, middleware: LLMMiddleware) -> None:
"""注册一个中间件到处理管道的最外层"""
self._middlewares.append(middleware)
def _build_pipeline(self) -> NextCall:
"""
构建完整的中间件调用链。顺序为：
用户自定义中间件 -> Retry -> Logging -> KeySelection -> Network (终结者)
"""
from .adapters import get_adapter_for_api_type
client_settings = get_llm_config().client_settings
retry_config = RetryConfig(
max_retries=client_settings.max_retries,
retry_delay=client_settings.retry_delay,
)
adapter = get_adapter_for_api_type(self.api_type)
network_middleware = NetworkRequestMiddleware(self, adapter)
async def terminal_handler(ctx: LLMContext) -> LLMResponse:
async def _noop(_: LLMContext) -> LLMResponse:
raise RuntimeError("NetworkRequestMiddleware 不应调用 next_call")
return await network_middleware(ctx, _noop)
def _wrap(middleware: LLMMiddleware, next_call: NextCall) -> NextCall:
async def _handler(inner_ctx: LLMContext) -> LLMResponse:
return await middleware(inner_ctx, next_call)
return _handler
handler: NextCall = terminal_handler
handler = _wrap(
KeySelectionMiddleware(self.key_store, self.provider_name, self.api_keys),
handler,
)
handler = _wrap(
LoggingMiddleware(self.provider_name, self.model_name),
handler,
)
handler = _wrap(
RetryMiddleware(retry_config, self.key_store),
handler,
)
for middleware in reversed(self._middlewares):
handler = _wrap(middleware, handler)
return handler
def _get_effective_api_type(self) -> str:
"""
获取实际生效的 API 类型。
主要用于 Smart 模式下，判断日志净化应该使用哪种格式。
"""
if self.api_type != "smart":
return self.api_type
if self.model_detail.api_type:
return self.model_detail.api_type
if (
"gemini" in self.model_name.lower()
and "openai" not in self.model_name.lower()
):
return "gemini"
return "openai"
async def _get_http_client(self) -> LLMHttpClient:
"""获取HTTP客户端"""
if self.http_client.is_closed:
logger.debug(
f"LLMModel {self.provider_name}/{self.model_name} 的 HTTP 客户端已关闭,"
"正在获取新的客户端"
)
self.http_client = await http_client_manager.get_client(
self.provider_config
)
return self.http_client
async def _select_api_key(self, failed_keys: set[str] | None = None) -> str:
"""选择可用的API密钥（使用轮询策略）"""
if not self.api_keys:
raise LLMException(
f"提供商 {self.provider_name} 没有配置API密钥",
code=LLMErrorCode.NO_AVAILABLE_KEYS,
)
selected_key = await self.key_store.get_next_available_key(
self.provider_name, self.api_keys, failed_keys
)
if not selected_key:
raise LLMException(
f"提供商 {self.provider_name} 的所有API密钥当前都不可用",
code=LLMErrorCode.NO_AVAILABLE_KEYS,
details={
"total_keys": len(self.api_keys),
"failed_keys": len(failed_keys or set()),
},
)
return selected_key
async def close(self):
"""标记模型实例的当前使用周期结束"""
if self._is_closed:
return
self._is_closed = True
logger.debug(
f"LLMModel实例的使用周期已结束: {self} (共享HTTP客户端状态不受影响)"
)
async def __aenter__(self):
if self._is_closed:
logger.debug(
f"Re-entering context for closed LLMModel {self}. "
f"Resetting _is_closed to False."
)
self._is_closed = False
self._check_not_closed()
self._ref_count += 1
return self
async def __aexit__(self, exc_type, exc_val, exc_tb):
"""异步上下文管理器出口"""
_ = exc_type, exc_val, exc_tb
self._ref_count -= 1
if self._ref_count <= 0:
self._ref_count = 0
await self.close()
def _check_not_closed(self):
"""检查实例是否已关闭"""
if self._is_closed:
raise RuntimeError(f"LLMModel实例已关闭: {self}")
async def _execute_core_generation(self, context: LLMContext) -> LLMResponse:
"""
[内核] 执行核心生成逻辑：构建管道并执行。
此方法作为中间件管道的终点被调用。
"""
pipeline_handler = self._build_pipeline()
return await pipeline_handler(context)
async def generate_response(
self,
messages: list[LLMMessage],
config: LLMGenerationConfig | None = None,
tools: list[Any] | None = None,
tool_choice: str | dict[str, Any] | ToolChoice | None = None,
timeout: float | None = None,
) -> LLMResponse:
"""
生成高级响应 (支持中间件管道)。
"""
self._check_not_closed()
if self._generation_config and config:
final_request_config = self._generation_config.merge_with(config)
elif config:
final_request_config = config
else:
final_request_config = self._generation_config or LLMGenerationConfig()
normalized_tools: list[Any] | None = None
if tools:
if isinstance(tools, dict):
normalized_tools = list(tools.values())
elif isinstance(tools, list):
normalized_tools = tools
else:
normalized_tools = [tools]
context = LLMContext(
messages=messages,
config=final_request_config,
tools=normalized_tools,
tool_choice=tool_choice,
timeout=timeout,
)
return await self._execute_core_generation(context)
async def generate_embeddings(
self,
texts: list[str],
config: LLMEmbeddingConfig | None = None,
) -> list[list[float]]:
"""生成文本嵌入向量"""
self._check_not_closed()
if not texts:
return []
final_config = config or LLMEmbeddingConfig()
context = LLMContext(
messages=[],
config=final_config,
tools=None,
tool_choice=None,
timeout=None,
request_type="embedding",
extra={"texts": texts},
)
pipeline = self._build_pipeline()
response = await pipeline(context)
embeddings = (
response.cache_info.get("embeddings") if response.cache_info else None
)
if embeddings is None:
raise LLMException(
"嵌入请求未返回 embeddings 数据",
code=LLMErrorCode.EMBEDDING_FAILED,
)
return embeddings
def __str__(self) -> str:
status = "closed" if self._is_closed else "active"
return f"LLMModel({self.provider_name}/{self.model_name}, {status})"
def __repr__(self) -> str:
status = "closed" if self._is_closed else "active"
return (
f"LLMModel(provider={self.provider_name}, model={self.model_name}, "
f"api_type={self.api_type}, status={status})"
)
class RetryMiddleware(BaseLLMMiddleware):
"""
重试中间件：处理异常捕获与重试循环
"""
def __init__(self, retry_config: RetryConfig, key_store: KeyStatusStore):
self.retry_config = retry_config
self.key_store = key_store
async def __call__(self, context: LLMContext, next_call: NextCall) -> LLMResponse:
last_exception: Exception | None = None
total_attempts = self.retry_config.max_retries + 1
for attempt in range(total_attempts):
try:
context.runtime_state["attempt"] = attempt + 1
return await next_call(context)
except LLMException as e:
last_exception = e
api_key = context.runtime_state.get("api_key")
if api_key:
status_code = e.details.get("status_code")
error_msg = f"({e.code.name}) {e.message}"
await self.key_store.record_failure(api_key, status_code, error_msg)
if not _should_retry_llm_error(
e, attempt, self.retry_config.max_retries
):
raise e
if attempt == total_attempts - 1:
raise e
wait_time = self.retry_config.retry_delay
if self.retry_config.exponential_backoff:
wait_time *= 2**attempt
logger.warning(
f"请求失败，{wait_time:.2f}秒后重试"
f" (第{attempt + 1}/{self.retry_config.max_retries}次重试): {e}"
)
await asyncio.sleep(wait_time)
except Exception as e:
logger.error(f"非预期异常，停止重试: {e}", e=e)
raise e
if last_exception:
raise last_exception
raise LLMException("重试循环异常结束")
class KeySelectionMiddleware(BaseLLMMiddleware):
"""
密钥选择中间件：负责轮询获取可用 API Key
"""
def __init__(
self, key_store: KeyStatusStore, provider_name: str, api_keys: list[str]
):
self.key_store = key_store
self.provider_name = provider_name
self.api_keys = api_keys
self._failed_keys: set[str] = set()
async def __call__(self, context: LLMContext, next_call: NextCall) -> LLMResponse:
selected_key = await self.key_store.get_next_available_key(
self.provider_name, self.api_keys, exclude_keys=self._failed_keys
)
if not selected_key:
raise LLMException(
f"提供商 {self.provider_name} 无可用 API Key",
code=LLMErrorCode.NO_AVAILABLE_KEYS,
)
context.runtime_state["api_key"] = selected_key
try:
response = await next_call(context)
return response
except LLMException as e:
self._failed_keys.add(selected_key)
masked = f"{selected_key[:8]}..."
if isinstance(e.details, dict):
e.details["api_key"] = masked
raise e
class LoggingMiddleware(BaseLLMMiddleware):
"""
日志中间件：负责请求和响应的日志记录与脱敏
"""
def __init__(
self, provider_name: str, model_name: str, log_context: str = "Generation"
):
self.provider_name = provider_name
self.model_name = model_name
self.log_context = log_context
async def __call__(self, context: LLMContext, next_call: NextCall) -> LLMResponse:
attempt = context.runtime_state.get("attempt", 1)
api_key = context.runtime_state.get("api_key", "unknown")
masked_key = f"{api_key[:8]}..."
logger.info(
f"🌐 发起LLM请求 (尝试 {attempt}) - {self.provider_name}/{self.model_name} "
f"[{self.log_context}] Key: {masked_key}"
)
try:
start_time = time.monotonic()
response = await next_call(context)
duration = (time.monotonic() - start_time) * 1000
logger.info(f"🎯 LLM响应成功 [{self.log_context}] 耗时: {duration:.2f}ms")
return response
except Exception as e:
logger.error(f"❌ 请求异常 [{self.log_context}]: {type(e).__name__} - {e}")
raise e
class NetworkRequestMiddleware(BaseLLMMiddleware):
"""
网络请求中间件：执行 Adapter 转换和 HTTP 请求
"""
def __init__(self, model_instance: "LLMModel", adapter: "BaseAdapter"):
self.model = model_instance
self.http_client = model_instance.http_client
self.adapter = adapter
self.key_store = model_instance.key_store
async def __call__(self, context: LLMContext, next_call: NextCall) -> LLMResponse:
api_key = context.runtime_state["api_key"]
request_data: RequestData
gen_config: LLMGenerationConfig | None = None
embed_config: LLMEmbeddingConfig | None = None
if context.request_type == "embedding":
embed_config = cast(LLMEmbeddingConfig, context.config)
texts = (context.extra or {}).get("texts", [])
request_data = self.adapter.prepare_embedding_request(
model=self.model,
api_key=api_key,
texts=texts,
config=embed_config,
)
else:
gen_config = cast(LLMGenerationConfig, context.config)
request_data = await self.adapter.prepare_advanced_request(
model=self.model,
api_key=api_key,
messages=context.messages,
config=gen_config,
tools=context.tools,
tool_choice=context.tool_choice,
)
masked_key = (
f"{api_key[:8]}...{api_key[-4:] if len(api_key) > 12 else '***'}"
if api_key
else "N/A"
)
logger.debug(f"🔑 API密钥: {masked_key}")
logger.debug(f"📡 请求URL: {request_data.url}")
logger.debug(f"📋 请求头: {_sanitize_request_headers(request_data.headers)}")
if self.model.api_type == "smart":
effective_type = self.model._get_effective_api_type()
sanitizer_req_context = f"{effective_type}_request"
else:
sanitizer_req_context = self.adapter.log_sanitization_context
sanitized_body = sanitize_for_logging(
request_data.body, context=sanitizer_req_context
)
if request_data.files and isinstance(sanitized_body, dict):
file_info: list[str] = []
file_count = 0
if isinstance(request_data.files, list):
file_count = len(request_data.files)
for key, value in request_data.files:
filename = (
value[0]
if isinstance(value, tuple) and len(value) > 0
else "..."
)
file_info.append(f"{key}='{filename}'")
elif isinstance(request_data.files, dict):
file_count = len(request_data.files)
file_info = list(request_data.files.keys())
sanitized_body["[MULTIPART_FILES]"] = f"Count: {file_count} | {file_info}"
request_body_str = dump_json_safely(
sanitized_body, ensure_ascii=False, indent=2
)
logger.debug(f"📦 请求体: {request_body_str}")
start_time = time.monotonic()
try:
http_response = await self.http_client.post(
request_data.url,
headers=request_data.headers,
content=dump_json_safely(request_data.body, ensure_ascii=False)
if not request_data.files
else None,
data=request_data.body if request_data.files else None,
files=request_data.files,
timeout=context.timeout,
)
logger.debug(f"📥 响应状态码: {http_response.status_code}")
if exception := self.adapter.handle_http_error(http_response):
error_text = http_response.content.decode("utf-8", errors="ignore")
logger.debug(f"💥 完整错误响应: {error_text}")
await self.key_store.record_failure(
api_key, http_response.status_code, error_text
)
raise exception
response_bytes = await http_response.aread()
logger.debug(f"📦 响应体已完整读取 ({len(response_bytes)} bytes)")
response_json = json.loads(response_bytes)
sanitizer_resp_context = sanitizer_req_context.replace(
"_request", "_response"
)
if sanitizer_resp_context == sanitizer_req_context:
sanitizer_resp_context = f"{sanitizer_req_context}_response"
sanitized_response = sanitize_for_logging(
response_json, context=sanitizer_resp_context
)
response_json_str = json.dumps(
sanitized_response, ensure_ascii=False, indent=2
)
logger.debug(f"📋 响应JSON: {response_json_str}")
if context.request_type == "embedding":
self.adapter.validate_embedding_response(response_json)
embeddings = self.adapter.parse_embedding_response(response_json)
latency = (time.monotonic() - start_time) * 1000
await self.key_store.record_success(api_key, latency)
return LLMResponse(
text="",
raw_response=response_json,
cache_info={"embeddings": embeddings},
)
response_data = self.adapter.parse_response(
self.model, response_json, is_advanced=True
)
should_rescue_image = (
gen_config
and gen_config.validation_policy
and gen_config.validation_policy.get("require_image")
)
if (
should_rescue_image
and not response_data.images
and response_data.text
and gen_config
):
markdown_matches = re.findall(
r"(!?\[.*?\]\((https?://[^\)]+)\))", response_data.text
)
if markdown_matches:
logger.info(
f"检测到 {len(markdown_matches)} "
"个资源链接，尝试自动下载并清洗。"
)
if response_data.images is None:
response_data.images = []
downloaded_urls = set()
for full_tag, url in markdown_matches:
try:
if url not in downloaded_urls:
content = await AsyncHttpx.get_content(url)
response_data.images.append(process_image_data(content))
downloaded_urls.add(url)
response_data.text = response_data.text.replace(
full_tag, ""
)
except Exception as exc:
logger.warning(
f"自动下载生成的图片失败: {url}, 错误: {exc}"
)
response_data.text = response_data.text.strip()
latency = (time.monotonic() - start_time) * 1000
await self.key_store.record_success(api_key, latency)
response_tool_calls: list[LLMToolCall] = []
if response_data.tool_calls:
for tc_data in response_data.tool_calls:
if isinstance(tc_data, LLMToolCall):
response_tool_calls.append(tc_data)
elif isinstance(tc_data, dict):
try:
response_tool_calls.append(LLMToolCall(**tc_data))
except Exception:
pass
final_response = LLMResponse(
text=response_data.text,
content_parts=response_data.content_parts,
usage_info=response_data.usage_info,
images=response_data.images,
raw_response=response_data.raw_response,
tool_calls=response_tool_calls if response_tool_calls else None,
code_executions=response_data.code_executions,
grounding_metadata=response_data.grounding_metadata,
cache_info=response_data.cache_info,
thought_text=response_data.thought_text,
thought_signature=response_data.thought_signature,
)
if context.request_type == "generation" and gen_config:
if gen_config.response_validator:
try:
gen_config.response_validator(final_response)
except Exception as exc:
raise LLMException(
f"响应内容未通过自定义验证器: {exc}",
code=LLMErrorCode.API_RESPONSE_INVALID,
details={"validator_error": str(exc)},
cause=exc,
) from exc
policy = gen_config.validation_policy
if policy:
effective_type = self.model._get_effective_api_type()
if policy.get("require_image") and not final_response.images:
if effective_type == "gemini" and response_data.raw_response:
usage_metadata = response_data.raw_response.get(
"usageMetadata", {}
)
prompt_token_details = usage_metadata.get(
"promptTokensDetails", []
)
prompt_had_image = any(
detail.get("modality") == "IMAGE"
for detail in prompt_token_details
)
if prompt_had_image:
raise LLMException(
"响应验证失败：模型接收了图片输入但未生成图片。",
code=LLMErrorCode.API_RESPONSE_INVALID,
details={
"policy": policy,
"text_response": final_response.text,
"raw_response": response_data.raw_response,
},
)
else:
logger.debug(
"Gemini提示词中未包含图片，跳过图片要求重试。"
)
else:
raise LLMException(
"响应验证失败：要求返回图片但未找到图片数据。",
code=LLMErrorCode.API_RESPONSE_INVALID,
details={
"policy": policy,
"text_response": final_response.text,
},
)
return final_response
except Exception as e:
if isinstance(e, LLMException):
raise e
logger.error(f"解析响应失败或发生未知错误: {e}")
if not isinstance(e, httpx.NetworkError | httpx.TimeoutException):
await self.key_store.record_failure(api_key, None, str(e))
raise LLMException(
f"网络请求异常: {type(e).__name__} - {e}",
code=LLMErrorCode.API_REQUEST_FAILED,
details={"api_key": masked_key},
cause=e,
)
zhenxun/services/llm/session.py
-793
View File
@@ -1,793 +0,0 @@
"""
LLM 服务 - 会话客户端
提供一个有状态的、面向会话的 LLM 客户端，用于进行多轮对话和复杂交互。
"""
from collections.abc import Awaitable, Callable
import copy
import json
from typing import Any, TypeVar, cast
import uuid
from jinja2 import Template
from nonebot.utils import is_coroutine_callable
from nonebot_plugin_alconna.uniseg import UniMessage
from pydantic import BaseModel
from zhenxun.services.log import logger
from zhenxun.utils.pydantic_compat import model_json_schema
from .config import (
CommonOverrides,
GenConfigBuilder,
LLMEmbeddingConfig,
LLMGenerationConfig,
)
from .config.generation import OutputConfig
from .config.providers import get_llm_config
from .manager import get_global_default_model_name, get_model_instance
from .memory import (
AIConfig,
BaseMemory,
MemoryProcessor,
_get_default_memory,
)
from .tools import tool_provider_manager
from .types import (
LLMContentPart,
LLMErrorCode,
LLMException,
LLMMessage,
LLMResponse,
ModelName,
ResponseFormat,
StructuredOutputStrategy,
ToolChoice,
ToolExecutable,
)
from .types.models import (
GeminiCodeExecution,
GeminiGoogleSearch,
)
from .utils import (
create_cot_wrapper,
normalize_to_llm_messages,
parse_and_validate_json,
should_apply_autocot,
)
T = TypeVar("T", bound=BaseModel)
DEFAULT_IVR_TEMPLATE = (
"你的响应未能通过结构校验。\n"
"错误详情: {error_msg}\n\n"
"请执行以下步骤进行修正：\n"
"1. 反思：分析为什么会出现这个错误。\n"
"2. 修正：生成一个新的、符合 Schema 要求的 JSON 对象。\n"
"请直接输出修正后的 JSON，不要包含 Markdown 标记或其他解释。"
)
class AI:
"""
统一的AI服务类 - 提供了带记忆的会话接口。
不再执行自主工具循环，当LLM返回工具调用时，会直接将请求返回给调用者。
"""
def __init__(
self,
session_id: str | None = None,
config: AIConfig | None = None,
memory: BaseMemory | None = None,
default_generation_config: LLMGenerationConfig | None = None,
processors: list[MemoryProcessor] | None = None,
):
"""
初始化AI服务
参数:
session_id: 唯一的会话ID，用于隔离记忆。
config: AI 配置.
memory: 可选的自定义记忆后端。如果为None，则使用默认的 ChatMemory
(InMemoryMessageStore)。
default_generation_config: 此AI实例的默认生成配置。
processors: 记忆处理器列表，在添加记忆后触发。
"""
self.session_id = session_id or str(uuid.uuid4())
self.config = config or AIConfig()
self.memory = memory or _get_default_memory()
self.default_generation_config = (
default_generation_config or LLMGenerationConfig()
)
self.processors = processors or []
global_providers = tool_provider_manager._providers
config_providers = self.config.tool_providers
self._tool_providers = list(dict.fromkeys(global_providers + config_providers))
self.message_buffer: list[LLMMessage] = []
async def clear_history(self):
"""清空当前会话的历史记录。"""
await self.memory.clear_history(self.session_id)
logger.info(f"AI会话历史记录已清空 (session_id: {self.session_id})")
async def add_observation(
self, message: str | UniMessage | LLMMessage | list[LLMContentPart]
):
"""
将一条观察消息加入缓冲区，不立即触发模型调用。
返回:
int: 缓冲区中消息的数量。
"""
current_message = await self._normalize_input_to_message(message)
self.message_buffer.append(current_message)
content_preview = str(current_message.content)[:50]
logger.debug(
f"[放入观察] {content_preview} (缓冲区大小: {len(self.message_buffer)})",
"AI_MEMORY",
)
return len(self.message_buffer)
async def add_user_message_to_history(
self, message: str | LLMMessage | list[LLMContentPart]
):
"""
将一条用户消息标准化并添加到会话历史中。
参数:
message: 用户消息内容。
"""
user_message = await self._normalize_input_to_message(message)
await self.memory.add_message(self.session_id, user_message)
async def add_assistant_response_to_history(self, response_text: str):
"""
将助手的文本回复添加到会话历史中。
参数:
response_text: 助手的回复文本。
"""
assistant_message = LLMMessage.assistant_text_response(response_text)
await self.memory.add_message(self.session_id, assistant_message)
def _sanitize_message_for_history(self, message: LLMMessage) -> LLMMessage:
"""
净化用于存入历史记录的消息。
将非文本的多模态内容部分替换为文本占位符，以避免重复处理。
"""
if not isinstance(message.content, list):
return message
sanitized_message = copy.deepcopy(message)
content_list = sanitized_message.content
if not isinstance(content_list, list):
return sanitized_message
new_content_parts: list[LLMContentPart] = []
has_multimodal_content = False
for part in content_list:
if isinstance(part, LLMContentPart) and part.type == "text":
new_content_parts.append(part)
else:
has_multimodal_content = True
if has_multimodal_content:
placeholder = "[用户发送了媒体文件，内容已在首次分析时处理]"
text_part_found = False
for part in new_content_parts:
if part.type == "text":
part.text = f"{placeholder} {part.text or ''}".strip()
text_part_found = True
break
if not text_part_found:
new_content_parts.insert(0, LLMContentPart.text_part(placeholder))
sanitized_message.content = new_content_parts
return sanitized_message
async def _normalize_input_to_message(
self, message: str | UniMessage | LLMMessage | list[LLMContentPart]
) -> LLMMessage:
"""
内部辅助方法，将各种输入类型统一转换为单个 LLMMessage 对象。
它调用共享的工具函数并提取最后一条消息（通常是用户输入）。
"""
messages = await normalize_to_llm_messages(message)
if not messages:
raise LLMException(
"无法将输入标准化为有效的消息。", code=LLMErrorCode.CONFIGURATION_ERROR
)
return messages[-1]
async def generate_internal(
self,
messages: list[LLMMessage],
*,
model: ModelName = None,
config: LLMGenerationConfig | GenConfigBuilder | None = None,
tools: list[Any] | dict[str, ToolExecutable] | None = None,
tool_choice: str | dict[str, Any] | ToolChoice | None = None,
timeout: float | None = None,
model_instance: Any = None,
) -> LLMResponse:
"""
内部生成核心方法，负责配置合并、工具解析和模型调用。
此方法不处理历史记录的存储，供 AgentExecutor 或 chat 方法调用。
"""
final_config = self.default_generation_config
if isinstance(config, GenConfigBuilder):
config = config.build()
if config:
final_config = final_config.merge_with(config)
final_tools_list = []
if tools:
if isinstance(tools, dict):
final_tools_list = list(tools.values())
elif isinstance(tools, list):
to_resolve: list[Any] = []
for t in tools:
if isinstance(t, str | dict):
to_resolve.append(t)
else:
final_tools_list.append(t)
if to_resolve:
resolved_dict = await self._resolve_tools(to_resolve)
final_tools_list.extend(resolved_dict.values())
if model_instance:
return await model_instance.generate_response(
messages,
config=final_config,
tools=final_tools_list if final_tools_list else None,
tool_choice=tool_choice,
timeout=timeout,
)
resolved_model_name = self._resolve_model_name(model or self.config.model)
async with await get_model_instance(
resolved_model_name,
override_config=None,
) as instance:
return await instance.generate_response(
messages,
config=final_config,
tools=final_tools_list if final_tools_list else None,
tool_choice=tool_choice,
timeout=timeout,
)
async def chat(
self,
message: str | UniMessage | LLMMessage | list[LLMContentPart] | None,
*,
model: ModelName = None,
instruction: str | None = None,
template_vars: dict[str, Any] | None = None,
preserve_media_in_history: bool | None = None,
tools: list[Any] | dict[str, ToolExecutable] | None = None,
tool_choice: str | dict[str, Any] | ToolChoice | None = None,
config: LLMGenerationConfig | GenConfigBuilder | None = None,
use_buffer: bool = False,
timeout: float | None = None,
) -> LLMResponse:
"""
核心交互方法，管理会话历史并执行单次LLM调用。
参数:
message: 用户输入的消息内容，支持文本、UniMessage、LLMMessage或
内容部分列表。
model: 要使用的模型名称，如果为None则使用配置中的默认模型。
instruction: 本次调用的特定系统指令，会与全局指令合并。
template_vars: 模板变量字典，用于在指令中进行变量替换。
preserve_media_in_history: 是否在历史记录中保留媒体内容，
None时使用默认配置。
tools: 可用的工具列表或工具字典，支持临时工具和预配置工具。
tool_choice: 工具选择策略，控制AI如何选择和使用工具。
config: 生成配置对象，用于覆盖默认的生成参数。
use_buffer: 是否刷新并包含消息缓冲区的内容，在此次对话中一次性提交。
timeout: HTTP 请求超时时间（秒）。
返回:
LLMResponse: 包含AI回复、工具调用请求、使用信息等的完整响应对象。
"""
messages_to_add: list[LLMMessage] = []
if message:
current_message = await self._normalize_input_to_message(message)
messages_to_add.append(current_message)
if use_buffer and self.message_buffer:
messages_to_add = self.message_buffer + messages_to_add
self.message_buffer.clear()
messages_for_run = []
final_instruction = instruction
if final_instruction and template_vars:
try:
template = Template(final_instruction)
final_instruction = template.render(**template_vars)
logger.debug(f"渲染后的系统指令: {final_instruction}")
except Exception as e:
logger.error(f"渲染系统指令模板失败: {e}", e=e)
if final_instruction:
messages_for_run.append(LLMMessage.system(final_instruction))
current_history = await self.memory.get_history(self.session_id)
messages_for_run.extend(current_history)
messages_for_run.extend(messages_to_add)
try:
response = await self.generate_internal(
messages_for_run,
model=model,
config=config,
tools=tools,
tool_choice=tool_choice,
timeout=timeout,
)
should_preserve = (
preserve_media_in_history
if preserve_media_in_history is not None
else self.config.default_preserve_media_in_history
)
msgs_to_store: list[LLMMessage] = []
for msg in messages_to_add:
store_msg = (
msg if should_preserve else self._sanitize_message_for_history(msg)
)
msgs_to_store.append(store_msg)
if response.content_parts:
assistant_response_msg = LLMMessage(
role="assistant",
content=response.content_parts,
tool_calls=response.tool_calls,
)
else:
assistant_response_msg = LLMMessage.assistant_text_response(
response.text
)
if response.tool_calls:
assistant_response_msg = LLMMessage.assistant_tool_calls(
response.tool_calls, response.text
)
await self.memory.add_messages(
self.session_id, [*msgs_to_store, assistant_response_msg]
)
if self.processors:
for processor in self.processors:
await processor.process(
self.session_id, [*msgs_to_store, assistant_response_msg]
)
return response
except Exception as e:
raise (
e
if isinstance(e, LLMException)
else LLMException(f"聊天执行失败: {e}", cause=e)
)
async def code(
self,
prompt: str,
*,
model: ModelName = None,
timeout: int | None = None,
config: LLMGenerationConfig | GenConfigBuilder | None = None,
) -> LLMResponse:
"""
代码执行
参数:
prompt: 代码执行的提示词。
model: 要使用的模型名称。
timeout: 代码执行超时时间（秒）。
config: (可选) 覆盖默认的生成配置。
返回:
LLMResponse: 包含执行结果的完整响应对象。
"""
resolved_model = model or self.config.model
code_config = CommonOverrides.gemini_code_execution()
if timeout:
code_config.custom_params = code_config.custom_params or {}
code_config.custom_params["code_execution_timeout"] = timeout
if isinstance(config, GenConfigBuilder):
config = config.build()
if config:
code_config = code_config.merge_with(config)
return await self.chat(prompt, model=resolved_model, config=code_config)
async def search(
self,
query: UniMessage,
*,
model: ModelName = None,
instruction: str = (
"你是一位强大的信息检索和整合专家。请利用可用的搜索工具，"
"根据用户的查询找到最相关的信息，并进行总结和回答。"
),
template_vars: dict[str, Any] | None = None,
config: LLMGenerationConfig | GenConfigBuilder | None = None,
) -> LLMResponse:
"""
信息搜索的便捷入口，原生支持多模态查询。
"""
logger.info("执行 'search' 任务...")
search_config = CommonOverrides.gemini_grounding()
if isinstance(config, GenConfigBuilder):
config = config.build()
if config:
search_config = search_config.merge_with(config)
return await self.chat(
query,
model=model,
instruction=instruction,
template_vars=template_vars,
config=search_config,
tools=[GeminiGoogleSearch()],
)
async def generate_structured(
self,
message: str | UniMessage | LLMMessage | list[LLMContentPart] | None,
response_model: type[T],
*,
model: ModelName = None,
tools: list[Any] | dict[str, ToolExecutable] | None = None,
tool_choice: str | dict[str, Any] | ToolChoice | None = None,
instruction: str | None = None,
timeout: float | None = None,
template_vars: dict[str, Any] | None = None,
config: LLMGenerationConfig | GenConfigBuilder | None = None,
max_validation_retries: int | None = None,
validation_callback: Callable[[T], Any | Awaitable[Any]] | None = None,
error_prompt_template: str | None = None,
auto_thinking: bool = False,
) -> T:
"""
生成结构化响应，并自动解析为指定的Pydantic模型。
参数:
message: 用户输入的消息内容，支持多种格式。为None时只使用历史+缓冲区。
response_model: 用于解析和验证响应的Pydantic模型类。
model: 要使用的模型名称，如果为None则使用配置中的默认模型。
instruction: 本次调用的特定系统指令，会与JSON Schema指令合并。
timeout: HTTP 请求超时时间（秒）。
template_vars: 系统指令中的模板变量，用于动态渲染。
config: 生成配置对象，用于覆盖默认的生成参数。
返回:
T: 解析后的Pydantic模型实例，类型为response_model指定的类型。
异常:
LLMException: 如果模型返回的不是有效的JSON或验证失败。
"""
if isinstance(config, GenConfigBuilder):
config = config.build()
final_config = self.default_generation_config.merge_with(config)
if final_config is None:
final_config = LLMGenerationConfig()
if max_validation_retries is None:
max_validation_retries = get_llm_config().client_settings.structured_retries
resolved_model_name = self._resolve_model_name(model or self.config.model)
request_autocot = True if auto_thinking is False else auto_thinking
effective_auto_thinking = should_apply_autocot(
request_autocot, resolved_model_name, final_config
)
target_model: type[T] = response_model
if effective_auto_thinking:
target_model = cast(type[T], create_cot_wrapper(response_model))
response_model = target_model
cot_instruction = (
"请务必先在 `reasoning` 字段中进行详细的一步步推理，确保逻辑正确，"
"然后再填充 `result` 字段。"
)
if instruction:
instruction = f"{instruction}\n\n{cot_instruction}"
else:
instruction = cot_instruction
final_instruction = instruction
if final_instruction and template_vars:
try:
template = Template(final_instruction)
final_instruction = template.render(**template_vars)
except Exception as e:
logger.error(f"渲染结构化指令模板失败: {e}", e=e)
try:
json_schema = model_json_schema(response_model)
except AttributeError:
json_schema = response_model.schema()
schema_str = json.dumps(json_schema, ensure_ascii=False, indent=2)
prompt_prefix = f"{final_instruction}\n\n" if final_instruction else ""
structured_strategy = (
final_config.output.structured_output_strategy
if final_config.output
else None
)
if structured_strategy == StructuredOutputStrategy.TOOL_CALL:
system_prompt = prompt_prefix + "请调用提供的工具提交结构化数据。"
else:
system_prompt = (
prompt_prefix
+ "请严格按照以下 JSON Schema 格式进行响应。不应包含任何额外的解释、"
"注释或代码块标记，只返回一个合法的 JSON 对象。\n\n"
)
system_prompt += f"JSON Schema:\n```json\n{schema_str}\n```"
structured_strategy = (
final_config.output.structured_output_strategy
if final_config.output
else StructuredOutputStrategy.NATIVE
)
final_tools_list: list[ToolExecutable] | None = None
if structured_strategy != StructuredOutputStrategy.NATIVE:
if tools:
final_tools_list = []
if isinstance(tools, dict):
final_tools_list = list(tools.values())
elif isinstance(tools, list):
to_resolve: list[Any] = []
for t in tools:
if isinstance(t, str | dict):
to_resolve.append(t)
else:
final_tools_list.append(t)
if to_resolve:
resolved_dict = await self._resolve_tools(to_resolve)
final_tools_list.extend(resolved_dict.values())
elif tools:
logger.warning(
"检测到在 generate_structured (NATIVE 策略) 中传入了 tools。"
"为了避免 API 冲突(Gemini)及输出歧义(OpenAI)，这些"
"tools 将被本次请求忽略。"
"若需使用工具，请使用 chat() 方法或 Agent 流程。"
)
if final_config.output is None:
final_config.output = OutputConfig()
final_config.output.response_format = ResponseFormat.JSON
final_config.output.response_schema = json_schema
messages_for_run = [LLMMessage.system(system_prompt)]
current_history = await self.memory.get_history(self.session_id)
messages_for_run.extend(current_history)
messages_for_run.extend(self.message_buffer)
if message:
normalized_message = await self._normalize_input_to_message(message)
messages_for_run.append(normalized_message)
ivr_messages = list(messages_for_run)
last_exception: Exception | None = None
for attempt in range(max_validation_retries + 1):
current_response_text: str = ""
async with await get_model_instance(
resolved_model_name,
override_config=None,
) as model_instance:
response = await model_instance.generate_response(
ivr_messages,
config=final_config,
tools=final_tools_list if final_tools_list else None,
tool_choice=tool_choice,
timeout=timeout,
)
current_response_text = response.text
try:
parsed_obj = parse_and_validate_json(response.text, target_model)
final_obj: T = cast(T, parsed_obj)
if effective_auto_thinking:
logger.debug(
f"AutoCoT 思考过程: {getattr(parsed_obj, 'reasoning', '')}"
)
final_obj = cast(T, getattr(parsed_obj, "result"))
if validation_callback:
if is_coroutine_callable(validation_callback):
await validation_callback(final_obj)
else:
validation_callback(final_obj)
return final_obj
except Exception as e:
is_llm_error = isinstance(e, LLMException)
llm_error: LLMException | None = (
cast(LLMException, e) if is_llm_error else None
)
last_exception = e
if attempt < max_validation_retries:
error_msg = (
llm_error.details.get("validation_error", str(e))
if llm_error
else str(e)
)
raw_response = current_response_text or (
llm_error.details.get("raw_response", "") if llm_error else ""
)
logger.warning(
f"结构化校验失败 (尝试 {attempt + 1}/"
f"{max_validation_retries + 1})。正在尝试 IVR 修复... 错误:"
f"{error_msg}"
)
if raw_response:
ivr_messages.append(
LLMMessage.assistant_text_response(raw_response)
)
else:
logger.warning(
"IVR 警告: 无法获取上一轮生成的原始文本，"
"模型将在无上下文情况下尝试修复。"
)
template = error_prompt_template or DEFAULT_IVR_TEMPLATE
feedback_prompt = template.format(error_msg=error_msg)
ivr_messages.append(LLMMessage.user(feedback_prompt))
continue
if llm_error and not llm_error.recoverable:
raise llm_error
if last_exception:
raise last_exception
raise LLMException(
"IVR 循环异常结束，未能生成有效结果。", code=LLMErrorCode.GENERATION_FAILED
)
def _resolve_model_name(self, model_name: ModelName) -> str:
"""解析模型名称"""
if model_name:
return model_name
default_model = get_global_default_model_name()
if default_model:
return default_model
raise LLMException(
"未指定模型名称且未设置全局默认模型",
code=LLMErrorCode.MODEL_NOT_FOUND,
)
async def embed(
self,
texts: list[str] | str,
*,
model: ModelName = None,
config: LLMEmbeddingConfig | None = None,
) -> list[list[float]]:
"""
生成文本嵌入向量，将文本转换为数值向量表示。
参数:
texts: 要生成嵌入的文本内容，支持单个字符串或字符串列表。
model: 嵌入模型名称，如果为None则使用配置中的默认嵌入模型。
config: 嵌入配置
返回:
list[list[float]]: 文本对应的嵌入向量列表，每个向量为浮点数列表。
异常:
LLMException: 当嵌入生成失败或模型配置错误时抛出
"""
if isinstance(texts, str):
texts = [texts]
if not texts:
return []
try:
resolved_model_str = (
model or self.config.default_embedding_model or self.config.model
)
if not resolved_model_str:
raise LLMException(
"使用 embed 方法时未指定嵌入模型名称，"
"且 AIConfig 未设置 default_embedding_model。",
code=LLMErrorCode.MODEL_NOT_FOUND,
)
resolved_model_str = self._resolve_model_name(resolved_model_str)
final_config = config or LLMEmbeddingConfig()
async with await get_model_instance(
resolved_model_str,
override_config=None,
) as embedding_model_instance:
return await embedding_model_instance.generate_embeddings(
texts, config=final_config
)
except LLMException:
raise
except Exception as e:
logger.error(f"文本嵌入失败: {e}", e=e)
raise LLMException(
f"文本嵌入失败: {e}", code=LLMErrorCode.EMBEDDING_FAILED, cause=e
)
async def _resolve_tools(
self,
tool_configs: list[Any],
) -> dict[str, ToolExecutable]:
"""
使用注入的 ToolProvider 异步解析 ad-hoc（临时）工具配置。
返回一个从工具名称到可执行对象的字典。
"""
resolved: dict[str, ToolExecutable] = {}
for config in tool_configs:
if isinstance(config, str):
if config == "google_search":
resolved[config] = GeminiGoogleSearch() # type: ignore[arg-type]
continue
elif config == "code_execution":
resolved[config] = GeminiCodeExecution() # type: ignore[arg-type]
continue
elif config == "url_context":
pass
name = config if isinstance(config, str) else config.get("name")
if not name:
raise LLMException(
"工具配置字典必须包含 'name' 字段。",
code=LLMErrorCode.CONFIGURATION_ERROR,
)
if isinstance(config, str):
config_dict = {"name": name, "type": "function"}
elif isinstance(config, dict):
config_dict = config
else:
raise TypeError(f"不支持的工具配置类型: {type(config)}")
executable = None
for provider in self._tool_providers:
executable = await provider.get_tool_executable(name, config_dict)
if executable:
break
if not executable:
raise LLMException(
f"没有为 ad-hoc 工具 '{name}' 找到合适的提供者。",
code=LLMErrorCode.CONFIGURATION_ERROR,
)
resolved[name] = executable
return resolved
zhenxun/services/llm/tools.py
-851
View File
@@ -1,851 +0,0 @@
"""
工具模块
整合了工具参数解析器、工具提供者管理器与工具执行逻辑，便于在 LLM 服务层统一调用。
"""
import asyncio
from collections.abc import Callable
from enum import Enum
import inspect
import json
import re
import time
from typing import (
Annotated,
Any,
Optional,
Union,
cast,
get_args,
get_origin,
get_type_hints,
)
from typing_extensions import Self, override
from httpx import NetworkError, TimeoutException
try:
import ujson as fast_json
except ImportError:
fast_json = json
import nonebot
from nonebot.dependencies import Dependent, Param
from nonebot.internal.adapter import Bot, Event
from nonebot.internal.params import (
BotParam,
DefaultParam,
DependParam,
DependsInner,
EventParam,
StateParam,
)
from pydantic import BaseModel, Field, ValidationError, create_model
from pydantic.fields import FieldInfo
from zhenxun.services.log import logger
from zhenxun.utils.decorator.retry import Retry
from zhenxun.utils.pydantic_compat import model_dump, model_fields, model_json_schema
from .types import (
LLMErrorCode,
LLMException,
LLMMessage,
LLMToolCall,
ToolExecutable,
ToolProvider,
ToolResult,
)
from .types.models import ToolDefinition
from .types.protocols import BaseCallbackHandler, ToolCallData
class ToolParam(Param):
"""
工具参数提取器。
用于在自定义工具函数（Function Tool）中，从 LLM 解析出的参数字典
(`state["_tool_params"]`)
中提取特定的参数值。通常配合 `Annotated` 和依赖注入系统使用。
"""
def __init__(self, *args: Any, name: str, **kwargs: Any):
super().__init__(*args, **kwargs)
self.name = name
def __repr__(self) -> str:
return f"ToolParam(name={self.name})"
@classmethod
@override
def _check_param(
cls, param: inspect.Parameter, allow_types: tuple[type[Param], ...]
) -> Optional["ToolParam"]:
if param.default is not inspect.Parameter.empty and isinstance(
param.default, DependsInner
):
return None
if get_origin(param.annotation) is Annotated:
for arg in get_args(param.annotation):
if isinstance(arg, DependsInner):
return None
if param.kind not in (
inspect.Parameter.VAR_POSITIONAL,
inspect.Parameter.VAR_KEYWORD,
):
return cls(name=param.name)
return None
@override
async def _solve(self, **kwargs: Any) -> Any:
state: dict[str, Any] = kwargs.get("state", {})
tool_params = state.get("_tool_params", {})
if self.name in tool_params:
return tool_params[self.name]
return None
class RunContext(BaseModel):
"""
依赖注入容器（DI Container），保留原有上下文信息的同时提升获取类型的能力。
"""
session_id: str | None = None
scope: dict[str, Any] = Field(default_factory=dict)
extra: dict[str, Any] = Field(default_factory=dict)
class Config:
arbitrary_types_allowed = True
class RunContextParam(Param):
"""自动注入 RunContext 的参数解析器"""
@classmethod
def _check_param(
cls, param: inspect.Parameter, allow_types: tuple[type[Param], ...]
) -> Optional["RunContextParam"]:
if param.annotation is RunContext:
return cls()
return None
async def _solve(self, **kwargs: Any) -> Any:
state = kwargs.get("state", {})
return state.get("_agent_context")
def _parse_docstring_params(docstring: str | None) -> dict[str, str]:
"""
解析文档字符串，提取参数描述。
支持 Google Style (Args:), ReST Style (:param:), 和中文风格 (参数:)。
"""
if not docstring:
return {}
params: dict[str, str] = {}
lines = docstring.splitlines()
rest_pattern = re.compile(r"[:@]param\s+(\w+)\s*:?\s*(.*)")
found_rest = False
for line in lines:
match = rest_pattern.search(line)
if match:
params[match.group(1)] = match.group(2).strip()
found_rest = True
if found_rest:
return params
section_header_pattern = re.compile(
r"^\s*(?:Args|Arguments|Parameters|参数)\s*[:：]\s*$"
)
param_section_active = False
google_pattern = re.compile(r"^\s*(\**\w+)(?:\s*\(.*?\))?\s*[:：]\s*(.*)")
for line in lines:
stripped_line = line.strip()
if not stripped_line:
continue
if section_header_pattern.match(line):
param_section_active = True
continue
if param_section_active:
if (
stripped_line.endswith(":") or stripped_line.endswith("：")
) and not google_pattern.match(line):
param_section_active = False
continue
match = google_pattern.match(line)
if match:
name = match.group(1).lstrip("*")
desc = match.group(2).strip()
params[name] = desc
return params
def _create_dynamic_model(func: Callable) -> type[BaseModel]:
"""根据函数签名动态创建 Pydantic 模型"""
sig = inspect.signature(func)
doc_params = _parse_docstring_params(func.__doc__)
type_hints = get_type_hints(func, include_extras=True)
fields = {}
for name, param in sig.parameters.items():
if name in ("self", "cls"):
continue
annotation = type_hints.get(name, Any)
default = param.default
is_run_context = False
if annotation is RunContext:
is_run_context = True
else:
origin = get_origin(annotation)
if origin is Union:
args = get_args(annotation)
if RunContext in args:
is_run_context = True
if is_run_context:
continue
if default is not inspect.Parameter.empty and isinstance(default, DependsInner):
continue
if get_origin(annotation) is Annotated:
args = get_args(annotation)
if any(isinstance(arg, DependsInner) for arg in args):
continue
description = doc_params.get(name)
if isinstance(default, FieldInfo):
if description and not getattr(default, "description", None):
default.description = description
fields[name] = (annotation, default)
else:
if default is inspect.Parameter.empty:
default = ...
fields[name] = (annotation, Field(default, description=description))
return create_model(f"{func.__name__}Params", **fields)
class FunctionExecutable(ToolExecutable):
"""一个 ToolExecutable 的实现，用于包装一个普通的 Python 函数。"""
def __init__(
self,
func: Callable,
name: str,
description: str,
params_model: type[BaseModel] | None = None,
unpack_args: bool = False,
):
self._func = func
self._name = name
self._description = description
self._params_model = params_model
self._unpack_args = unpack_args
self.dependent = Dependent[Any].parse(
call=func,
allow_types=(
DependParam,
BotParam,
EventParam,
StateParam,
RunContextParam,
ToolParam,
DefaultParam,
),
)
async def get_definition(self) -> ToolDefinition:
if not self._params_model:
return ToolDefinition(
name=self._name,
description=self._description,
parameters={"type": "object", "properties": {}},
)
schema = model_json_schema(self._params_model)
return ToolDefinition(
name=self._name,
description=self._description,
parameters={
"type": "object",
"properties": schema.get("properties", {}),
"required": schema.get("required", []),
},
)
async def execute(
self, context: RunContext | None = None, **kwargs: Any
) -> ToolResult:
context = context or RunContext()
tool_arguments = kwargs
if self._params_model:
try:
_fields = model_fields(self._params_model)
if isinstance(_fields, dict):
field_names = set(_fields)
else:
field_names = {
name
for field in _fields
for name in (
getattr(field, "name", None),
getattr(field, "alias", None),
)
if name
}
validation_input = {
key: value for key, value in kwargs.items() if key in field_names
}
validated_params = self._params_model(**validation_input)
if not self._unpack_args:
pass
else:
validated_dict = model_dump(validated_params)
tool_arguments = validated_dict
except ValidationError as e:
error_msgs = []
for err in e.errors():
loc = ".".join(str(x) for x in err["loc"])
msg = err["msg"]
error_msgs.append(f"Parameter '{loc}': {msg}")
formatted_error = "; ".join(error_msgs)
error_payload = {
"error_type": "InvalidArguments",
"message": f"Parameter validation failed: {formatted_error}",
"is_retryable": True,
}
return ToolResult(
output=json.dumps(error_payload, ensure_ascii=False),
display_content=f"Validation Error: {formatted_error}",
)
except Exception as e:
logger.error(
f"执行工具 '{self._name}' 时参数验证或实例化失败: {e}", e=e
)
raise
state = {
"_tool_params": tool_arguments,
"_agent_context": context,
}
bot: Bot | None = None
if context and context.scope.get("bot"):
bot = context.scope.get("bot")
if not bot:
try:
bot = nonebot.get_bot()
except ValueError:
pass
event: Event | None = None
if context and context.scope.get("event"):
event = context.scope.get("event")
raw_result = await self.dependent(
bot=bot,
event=event,
state=state,
)
return ToolResult(output=raw_result, display_content=str(raw_result))
class BuiltinFunctionToolProvider(ToolProvider):
"""一个内置的 ToolProvider，用于处理通过装饰器注册的函数。"""
def __init__(self):
self._functions: dict[str, dict[str, Any]] = {}
def register(
self,
name: str,
func: Callable,
description: str,
params_model: type[BaseModel] | None = None,
unpack_args: bool = False,
):
self._functions[name] = {
"func": func,
"description": description,
"params_model": params_model,
"unpack_args": unpack_args,
}
async def initialize(self) -> None:
pass
async def discover_tools(
self,
allowed_servers: list[str] | None = None,
excluded_servers: list[str] | None = None,
) -> dict[str, ToolExecutable]:
executables = {}
for name, info in self._functions.items():
executables[name] = FunctionExecutable(
func=info["func"],
name=name,
description=info["description"],
params_model=info["params_model"],
unpack_args=info.get("unpack_args", False),
)
return executables
async def get_tool_executable(
self, name: str, config: dict[str, Any]
) -> ToolExecutable | None:
if config.get("type", "function") == "function" and name in self._functions:
info = self._functions[name]
return FunctionExecutable(
func=info["func"],
name=name,
description=info["description"],
params_model=info["params_model"],
unpack_args=info.get("unpack_args", False),
)
return None
class ToolProviderManager:
"""工具提供者的中心化管理器，采用单例模式。"""
_instance: "ToolProviderManager | None" = None
def __new__(cls) -> Self:
if cls._instance is None:
cls._instance = super().__new__(cls)
return cast(Self, cls._instance)
def __init__(self):
if hasattr(self, "_initialized") and self._initialized:
return
self._providers: list[ToolProvider] = []
self._resolved_tools: dict[str, ToolExecutable] | None = None
self._init_lock = asyncio.Lock()
self._init_promise: asyncio.Task | None = None
self._builtin_function_provider = BuiltinFunctionToolProvider()
self.register(self._builtin_function_provider)
self._initialized = True
def register(self, provider: ToolProvider):
"""注册一个新的 ToolProvider。"""
if provider not in self._providers:
self._providers.append(provider)
logger.info(f"已注册工具提供者: {provider.__class__.__name__}")
def function_tool(
self,
name: str,
description: str,
params_model: type[BaseModel] | None = None,
):
"""装饰器：将一个函数注册为内置工具。"""
def decorator(func: Callable):
if name in self._builtin_function_provider._functions:
logger.warning(f"正在覆盖已注册的函数工具: {name}")
final_model = params_model
unpack_args = False
if final_model is None:
final_model = _create_dynamic_model(func)
unpack_args = True
self._builtin_function_provider.register(
name=name,
func=func,
description=description,
params_model=final_model,
unpack_args=unpack_args,
)
logger.info(f"已注册函数工具: '{name}'")
return func
return decorator
async def initialize(self) -> None:
"""懒加载初始化所有已注册的 ToolProvider。"""
if not self._init_promise:
async with self._init_lock:
if not self._init_promise:
self._init_promise = asyncio.create_task(
self._initialize_providers()
)
await self._init_promise
async def _initialize_providers(self) -> None:
"""内部初始化逻辑。"""
logger.info(f"开始初始化 {len(self._providers)} 个工具提供者...")
init_tasks = [provider.initialize() for provider in self._providers]
await asyncio.gather(*init_tasks, return_exceptions=True)
logger.info("所有工具提供者初始化完成。")
async def get_resolved_tools(
self,
allowed_servers: list[str] | None = None,
excluded_servers: list[str] | None = None,
) -> dict[str, ToolExecutable]:
"""
获取所有已发现和解析的工具。
此方法会触发懒加载初始化，并根据是否传入过滤器来决定是否使用全局缓存。
"""
await self.initialize()
has_filters = allowed_servers is not None or excluded_servers is not None
if not has_filters and self._resolved_tools is not None:
logger.debug("使用全局工具缓存。")
return self._resolved_tools
if has_filters:
logger.info("检测到过滤器，执行临时工具发现 (不使用缓存)。")
logger.debug(
f"过滤器详情: allowed_servers={allowed_servers}, "
f"excluded_servers={excluded_servers}"
)
else:
logger.info("未应用过滤器，开始全局工具发现...")
all_tools: dict[str, ToolExecutable] = {}
discover_tasks = []
for provider in self._providers:
sig = inspect.signature(provider.discover_tools)
params_to_pass = {}
if "allowed_servers" in sig.parameters:
params_to_pass["allowed_servers"] = allowed_servers
if "excluded_servers" in sig.parameters:
params_to_pass["excluded_servers"] = excluded_servers
discover_tasks.append(provider.discover_tools(**params_to_pass))
results = await asyncio.gather(*discover_tasks, return_exceptions=True)
for i, provider_result in enumerate(results):
provider_name = self._providers[i].__class__.__name__
if isinstance(provider_result, dict):
logger.debug(
f"提供者 '{provider_name}' 发现了 {len(provider_result)} 个工具。"
)
for name, executable in provider_result.items():
if name in all_tools:
logger.warning(
f"发现重复的工具名称 '{name}'，后发现的将覆盖前者。"
)
all_tools[name] = executable
elif isinstance(provider_result, Exception):
logger.error(
f"提供者 '{provider_name}' 在发现工具时出错: {provider_result}"
)
if not has_filters:
self._resolved_tools = all_tools
logger.info(f"全局工具发现完成，共找到并缓存了 {len(all_tools)} 个工具。")
else:
logger.info(f"带过滤器的工具发现完成，共找到 {len(all_tools)} 个工具。")
return all_tools
async def resolve_specific_tools(
self, tool_names: list[str]
) -> dict[str, ToolExecutable]:
"""
仅解析指定名称的工具，避免触发全量工具发现。
"""
resolved: dict[str, ToolExecutable] = {}
if not tool_names:
return resolved
await self.initialize()
for name in tool_names:
config: dict[str, Any] = {"name": name}
for provider in self._providers:
try:
executable = await provider.get_tool_executable(name, config)
except Exception as exc:
logger.error(
f"provider '{provider.__class__.__name__}' 在解析工具 '{name}'"
f"时出错: {exc}",
e=exc,
)
continue
if executable:
resolved[name] = executable
break
else:
logger.warning(f"没有找到名为 '{name}' 的工具，已跳过。")
return resolved
async def get_function_tools(
self, names: list[str] | None = None
) -> dict[str, ToolExecutable]:
"""
仅从内置的函数提供者中解析指定的工具。
"""
all_function_tools = await self._builtin_function_provider.discover_tools()
if names is None:
return all_function_tools
resolved_tools = {}
for name in names:
if name in all_function_tools:
resolved_tools[name] = all_function_tools[name]
else:
logger.warning(
f"本地函数工具 '{name}' 未通过 @function_tool 注册，将被忽略。"
)
return resolved_tools
tool_provider_manager = ToolProviderManager()
function_tool = tool_provider_manager.function_tool
class ToolErrorType(str, Enum):
"""结构化工具错误的类型枚举。"""
TOOL_NOT_FOUND = "ToolNotFound"
INVALID_ARGUMENTS = "InvalidArguments"
EXECUTION_ERROR = "ExecutionError"
USER_CANCELLATION = "UserCancellation"
class ToolErrorResult(BaseModel):
"""一个结构化的工具执行错误模型。"""
error_type: ToolErrorType = Field(..., description="错误的类型。")
message: str = Field(..., description="对错误的详细描述。")
is_retryable: bool = Field(False, description="指示这个错误是否可能通过重试解决。")
class ToolInvoker:
"""
全能工具执行器。
负责接收工具调用请求，解析参数，触发回调，执行工具，并返回标准化的结果。
"""
def __init__(self, callbacks: list[BaseCallbackHandler] | None = None):
self.callbacks = callbacks or []
async def _trigger_callbacks(self, event_name: str, *args, **kwargs: Any) -> None:
if not self.callbacks:
return
tasks = [
getattr(handler, event_name)(*args, **kwargs)
for handler in self.callbacks
if hasattr(handler, event_name)
]
await asyncio.gather(*tasks, return_exceptions=True)
async def execute_tool_call(
self,
tool_call: LLMToolCall,
available_tools: dict[str, ToolExecutable],
context: Any | None = None,
) -> tuple[LLMToolCall, ToolResult]:
tool_name = tool_call.function.name
arguments_str = tool_call.function.arguments
arguments: dict[str, Any] = {}
try:
if arguments_str:
arguments = json.loads(arguments_str)
except json.JSONDecodeError as e:
error_result = ToolErrorResult(
error_type=ToolErrorType.INVALID_ARGUMENTS,
message=f"参数解析失败: {e}",
is_retryable=False,
)
return tool_call, ToolResult(output=model_dump(error_result))
tool_data = ToolCallData(tool_name=tool_name, tool_args=arguments)
pre_calculated_result: ToolResult | None = None
for handler in self.callbacks:
res = await handler.on_tool_start(tool_call, tool_data)
if isinstance(res, ToolCallData):
tool_data = res
arguments = tool_data.tool_args
tool_call.function.arguments = json.dumps(arguments, ensure_ascii=False)
elif isinstance(res, ToolResult):
pre_calculated_result = res
break
if pre_calculated_result:
return tool_call, pre_calculated_result
executable = available_tools.get(tool_name)
if not executable:
error_result = ToolErrorResult(
error_type=ToolErrorType.TOOL_NOT_FOUND,
message=f"Tool '{tool_name}' not found.",
is_retryable=False,
)
return tool_call, ToolResult(output=model_dump(error_result))
from .config.providers import get_llm_config
if not get_llm_config().debug_log:
try:
definition = await executable.get_definition()
schema_payload = getattr(definition, "parameters", {})
schema_json = fast_json.dumps(
schema_payload,
ensure_ascii=False,
)
logger.debug(
f"🔍 [JIT Schema] {tool_name}: {schema_json}",
"ToolInvoker",
)
except Exception as e:
logger.trace(f"JIT Schema logging failed: {e}")
start_t = time.monotonic()
result: ToolResult | None = None
error: Exception | None = None
try:
@Retry.simple(stop_max_attempt=2, wait_fixed_seconds=1)
async def execute_with_retry():
return await executable.execute(context=context, **arguments)
result = await execute_with_retry()
except ValidationError as e:
error = e
error_msgs = []
for err in e.errors():
loc = ".".join(str(x) for x in err["loc"])
msg = err["msg"]
error_msgs.append(f"参数 '{loc}': {msg}")
formatted_error = "; ".join(error_msgs)
error_result = ToolErrorResult(
error_type=ToolErrorType.INVALID_ARGUMENTS,
message=f"参数验证失败。请根据错误修正你的输入: {formatted_error}",
is_retryable=True,
)
result = ToolResult(output=model_dump(error_result))
except (TimeoutException, NetworkError) as e:
error = e
error_result = ToolErrorResult(
error_type=ToolErrorType.EXECUTION_ERROR,
message=f"工具执行网络超时或连接失败: {e!s}",
is_retryable=False,
)
result = ToolResult(output=model_dump(error_result))
except Exception as e:
error = e
error_type = ToolErrorType.EXECUTION_ERROR
if (
isinstance(e, LLMException)
and e.code == LLMErrorCode.CONFIGURATION_ERROR
):
error_type = ToolErrorType.TOOL_NOT_FOUND
is_retryable = False
is_retryable = False
error_result = ToolErrorResult(
error_type=error_type, message=str(e), is_retryable=is_retryable
)
result = ToolResult(output=model_dump(error_result))
duration = time.monotonic() - start_t
await self._trigger_callbacks(
"on_tool_end",
result=result,
error=error,
tool_call=tool_call,
duration=duration,
)
if result is None:
raise LLMException("工具执行未返回任何结果。")
return tool_call, result
async def execute_batch(
self,
tool_calls: list[LLMToolCall],
available_tools: dict[str, ToolExecutable],
context: Any | None = None,
) -> list[LLMMessage]:
if not tool_calls:
return []
tasks = [
self.execute_tool_call(call, available_tools, context)
for call in tool_calls
]
results = await asyncio.gather(*tasks, return_exceptions=True)
tool_messages: list[LLMMessage] = []
for index, result_pair in enumerate(results):
original_call = tool_calls[index]
if isinstance(result_pair, Exception):
logger.error(
f"工具执行发生未捕获异常: {original_call.function.name}, "
f"错误: {result_pair}"
)
tool_messages.append(
LLMMessage.tool_response(
tool_call_id=original_call.id,
function_name=original_call.function.name,
result={
"error": f"System Execution Error: {result_pair}",
"status": "failed",
},
)
)
continue
tool_call_result = cast(tuple[LLMToolCall, ToolResult], result_pair)
_, tool_result = tool_call_result
tool_messages.append(
LLMMessage.tool_response(
tool_call_id=original_call.id,
function_name=original_call.function.name,
result=tool_result.output,
)
)
return tool_messages
__all__ = [
"RunContext",
"RunContextParam",
"ToolErrorResult",
"ToolErrorType",
"ToolInvoker",
"ToolParam",
"function_tool",
"tool_provider_manager",
]
zhenxun/services/llm/types/__init__.py
-76
View File
@@ -1,76 +0,0 @@
"""
LLM 类型定义模块
统一导出所有核心类型、协议和异常定义。
"""
from .capabilities import ModelCapabilities, ModelModality, get_model_capabilities
from .exceptions import LLMErrorCode, LLMException, get_user_friendly_error_message
from .models import (
CodeExecutionOutcome,
EmbeddingTaskType,
GeminiCodeExecution,
GeminiGoogleSearch,
GeminiUrlContext,
LLMCacheInfo,
LLMCodeExecution,
LLMContentPart,
LLMGroundingAttribution,
LLMGroundingMetadata,
LLMMessage,
LLMResponse,
LLMToolCall,
LLMToolFunction,
ModelDetail,
ModelInfo,
ModelName,
ModelProvider,
ProviderConfig,
ResponseFormat,
StructuredOutputStrategy,
TaskType,
ToolCategory,
ToolChoice,
ToolMetadata,
ToolResult,
UsageInfo,
)
from .protocols import ToolExecutable, ToolProvider
__all__ = [
"CodeExecutionOutcome",
"EmbeddingTaskType",
"GeminiCodeExecution",
"GeminiGoogleSearch",
"GeminiUrlContext",
"LLMCacheInfo",
"LLMCodeExecution",
"LLMContentPart",
"LLMErrorCode",
"LLMException",
"LLMGroundingAttribution",
"LLMGroundingMetadata",
"LLMMessage",
"LLMResponse",
"LLMToolCall",
"LLMToolFunction",
"ModelCapabilities",
"ModelDetail",
"ModelInfo",
"ModelModality",
"ModelName",
"ModelProvider",
"ProviderConfig",
"ResponseFormat",
"StructuredOutputStrategy",
"TaskType",
"ToolCategory",
"ToolChoice",
"ToolExecutable",
"ToolMetadata",
"ToolProvider",
"ToolResult",
"UsageInfo",
"get_model_capabilities",
"get_user_friendly_error_message",
]
zhenxun/services/llm/types/capabilities.py
-273
View File
@@ -1,273 +0,0 @@
"""
LLM 模型能力定义模块
定义模型的输入输出模态、工具调用支持等核心能力。
"""
from enum import Enum
import fnmatch
from typing import Literal
from pydantic import BaseModel, Field
from zhenxun.services.log import logger
class ModelModality(str, Enum):
TEXT = "text"
IMAGE = "image"
AUDIO = "audio"
VIDEO = "video"
EMBEDDING = "embedding"
class ReasoningMode(str, Enum):
"""推理/思考模式类型"""
NONE = "none"
BUDGET = "budget"
LEVEL = "level"
EFFORT = "effort"
PATTERNS_GEMINI_2_5 = [
"gemini-2.5*",
"gemini-flash*",
"gemini*lite*",
"gemini-flash-latest",
]
PATTERNS_GEMINI_3 = [
"gemini-3*",
"gemini-exp*",
]
PATTERNS_OPENAI_REASONING = [
"o1-*",
"o3-*",
"deepseek-r1*",
"deepseek-reasoner",
]
class ModelCapabilities(BaseModel):
"""定义一个模型的核心、稳定能力。"""
input_modalities: set[ModelModality] = Field(default={ModelModality.TEXT})
output_modalities: set[ModelModality] = Field(default={ModelModality.TEXT})
supports_tool_calling: bool = False
is_embedding_model: bool = False
reasoning_mode: ReasoningMode = ReasoningMode.NONE
reasoning_visibility: Literal["visible", "hidden", "none"] = "none"
STANDARD_TEXT_TOOL_CAPABILITIES = ModelCapabilities(
input_modalities={ModelModality.TEXT},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
)
CAP_GEMINI_2_5 = ModelCapabilities(
input_modalities={
ModelModality.TEXT,
ModelModality.IMAGE,
ModelModality.AUDIO,
ModelModality.VIDEO,
},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
reasoning_mode=ReasoningMode.BUDGET,
reasoning_visibility="visible",
)
CAP_GEMINI_3 = ModelCapabilities(
input_modalities={
ModelModality.TEXT,
ModelModality.IMAGE,
ModelModality.AUDIO,
ModelModality.VIDEO,
},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
reasoning_mode=ReasoningMode.LEVEL,
reasoning_visibility="visible",
)
CAP_GEMINI_IMAGE_GEN = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.IMAGE},
output_modalities={ModelModality.TEXT, ModelModality.IMAGE},
supports_tool_calling=True,
)
CAP_OPENAI_REASONING = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.IMAGE},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
reasoning_mode=ReasoningMode.EFFORT,
reasoning_visibility="hidden",
)
CAP_GPT_ADVANCED = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.IMAGE},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
)
CAP_GPT_MULTIMODAL_IO = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.AUDIO, ModelModality.IMAGE},
output_modalities={ModelModality.TEXT, ModelModality.AUDIO},
supports_tool_calling=True,
)
GPT_IMAGE_GENERATION_CAPABILITIES = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.IMAGE},
output_modalities={ModelModality.IMAGE},
supports_tool_calling=True,
)
GPT_VIDEO_GENERATION_CAPABILITIES = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.IMAGE, ModelModality.VIDEO},
output_modalities={ModelModality.VIDEO},
supports_tool_calling=True,
)
EMBEDDING_CAPABILITIES = ModelCapabilities(
input_modalities={ModelModality.TEXT},
output_modalities={ModelModality.EMBEDDING},
is_embedding_model=True,
)
DEFAULT_PERMISSIVE_CAPABILITIES = ModelCapabilities(
input_modalities={
ModelModality.TEXT,
ModelModality.IMAGE,
ModelModality.AUDIO,
ModelModality.VIDEO,
},
output_modalities={
ModelModality.TEXT,
ModelModality.IMAGE,
ModelModality.AUDIO,
ModelModality.VIDEO,
},
supports_tool_calling=True,
)
DOUBAO_ADVANCED_MULTIMODAL_CAPABILITIES = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.IMAGE, ModelModality.VIDEO},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
)
MODEL_ALIAS_MAPPING: dict[str, str] = {
"deepseek-v3*": "deepseek-chat",
"deepseek-ai/DeepSeek-V3": "deepseek-chat",
"deepseek-r1*": "deepseek-reasoner",
}
def _build_registry() -> dict[str, ModelCapabilities]:
"""构建模型能力注册表，展开模式列表以减少冗余"""
registry: dict[str, ModelCapabilities] = {}
def register_family(patterns: list[str], cap: ModelCapabilities) -> None:
for pattern in patterns:
registry[pattern] = cap
register_family(
["*gemini-*-image-preview*", "gemini-*-image*"], CAP_GEMINI_IMAGE_GEN
)
register_family(PATTERNS_GEMINI_2_5, CAP_GEMINI_2_5)
register_family(PATTERNS_GEMINI_3, CAP_GEMINI_3)
register_family(PATTERNS_OPENAI_REASONING, CAP_OPENAI_REASONING)
registry["gemini-*-tts"] = ModelCapabilities(
input_modalities={ModelModality.TEXT},
output_modalities={ModelModality.AUDIO},
)
registry["gemini-*-native-audio-*"] = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.AUDIO, ModelModality.VIDEO},
output_modalities={ModelModality.TEXT, ModelModality.AUDIO},
supports_tool_calling=True,
)
registry["gemini-2.0-flash-preview-image-generation"] = ModelCapabilities(
input_modalities={
ModelModality.TEXT,
ModelModality.IMAGE,
ModelModality.AUDIO,
ModelModality.VIDEO,
},
output_modalities={ModelModality.TEXT, ModelModality.IMAGE},
supports_tool_calling=True,
)
registry["GLM-4V-Flash"] = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.IMAGE},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
)
registry["GLM-4V-Plus*"] = ModelCapabilities(
input_modalities={ModelModality.TEXT, ModelModality.IMAGE, ModelModality.VIDEO},
output_modalities={ModelModality.TEXT},
supports_tool_calling=True,
)
register_family(
["glm-4-*", "glm-z1-*", "deepseek-chat"], STANDARD_TEXT_TOOL_CAPABILITIES
)
register_family(
["doubao-seed-*", "doubao-1-5-thinking-vision-pro"],
DOUBAO_ADVANCED_MULTIMODAL_CAPABILITIES,
)
register_family(["gpt-5*", "gpt-4.1*", "o4-mini*"], CAP_GPT_ADVANCED)
registry["gpt-4o*"] = CAP_GPT_MULTIMODAL_IO
registry["gpt image*"] = GPT_IMAGE_GENERATION_CAPABILITIES
registry["sora*"] = GPT_VIDEO_GENERATION_CAPABILITIES
registry["*embedding*"] = EMBEDDING_CAPABILITIES
return registry
MODEL_CAPABILITIES_REGISTRY = _build_registry()
def get_model_capabilities(model_name: str) -> ModelCapabilities:
"""
从注册表获取模型能力，支持别名映射和通配符匹配。
查找顺序: 1. 标准化名称 -> 2. 精确匹配 -> 3. 通配符匹配 -> 4. 默认值
"""
canonical_name = model_name
for alias_pattern, c_name in MODEL_ALIAS_MAPPING.items():
if fnmatch.fnmatch(model_name, alias_pattern):
canonical_name = c_name
break
parts = canonical_name.split("/")
names_to_check = ["/".join(parts[i:]) for i in range(len(parts))]
logger.trace(f"为 '{model_name}' 生成的检查列表: {names_to_check}")
for name in names_to_check:
if name in MODEL_CAPABILITIES_REGISTRY:
logger.debug(f"模型 '{model_name}' 通过精确匹配 '{name}' 找到能力定义。")
return MODEL_CAPABILITIES_REGISTRY[name]
for pattern, capabilities in MODEL_CAPABILITIES_REGISTRY.items():
if "*" in pattern and fnmatch.fnmatch(name, pattern):
logger.debug(
f"模型 '{model_name}' 通过通配符匹配 '{name}'(pattern: '{pattern}')"
f"找到能力定义。"
)
return capabilities
logger.warning(
f"模型 '{model_name}' 的能力定义未在注册表中找到，将使用默认的'全功能'回退配置"
)
return DEFAULT_PERMISSIVE_CAPABILITIES
zhenxun/services/llm/types/exceptions.py
-113
View File
@@ -1,113 +0,0 @@
"""
LLM 异常类型定义
"""
from enum import Enum
from typing import Any
class LLMErrorCode(Enum):
"""LLM 服务相关的错误代码枚举"""
MODEL_INIT_FAILED = 2000
MODEL_NOT_FOUND = 2001
API_REQUEST_FAILED = 2002
API_RESPONSE_INVALID = 2003
API_KEY_INVALID = 2004
API_QUOTA_EXCEEDED = 2005
API_TIMEOUT = 2006
API_RATE_LIMITED = 2007
NO_AVAILABLE_KEYS = 2008
UNKNOWN_API_TYPE = 2009
CONFIGURATION_ERROR = 2010
RESPONSE_PARSE_ERROR = 2011
CONTEXT_LENGTH_EXCEEDED = 2012
CONTENT_FILTERED = 2013
USER_LOCATION_NOT_SUPPORTED = 2014
INVALID_PARAMETER = 2017
GENERATION_FAILED = 2015
EMBEDDING_FAILED = 2016
class LLMException(Exception):
"""LLM 服务相关的基础异常类"""
def __init__(
self,
message: str,
code: LLMErrorCode = LLMErrorCode.API_REQUEST_FAILED,
details: dict[str, Any] | None = None,
recoverable: bool = True,
cause: Exception | None = None,
):
self.message = message
self.code = code
self.details = details or {}
self.recoverable = recoverable
self.cause = cause
super().__init__(message)
def __str__(self) -> str:
if self.details:
safe_details = {k: v for k, v in self.details.items() if k != "api_key"}
if safe_details:
return (
f"{self.message} (错误码: {self.code.name}, 详情: {safe_details})"
)
return f"{self.message} (错误码: {self.code.name})"
@property
def user_friendly_message(self) -> str:
"""返回适合向用户展示的错误消息"""
error_messages = {
LLMErrorCode.MODEL_NOT_FOUND: "AI模型未找到，请检查配置或联系管理员。",
LLMErrorCode.API_KEY_INVALID: "API密钥无效，请联系管理员更新配置。",
LLMErrorCode.API_QUOTA_EXCEEDED: (
"API使用配额已用尽，请稍后再试或联系管理员。"
),
LLMErrorCode.API_TIMEOUT: "AI服务响应超时，请稍后再试。",
LLMErrorCode.API_RATE_LIMITED: "请求过于频繁，已被AI服务限流，请稍后再试。",
LLMErrorCode.MODEL_INIT_FAILED: "AI模型初始化失败，请联系管理员检查配置。",
LLMErrorCode.NO_AVAILABLE_KEYS: (
"当前所有API密钥均不可用，请稍后再试或联系管理员。"
),
LLMErrorCode.USER_LOCATION_NOT_SUPPORTED: (
"当前网络环境不支持此 AI 模型 (如 Gemini/OpenAI)。\n"
"原因: 代理节点所在地区（如香港/国内/非支持区）被服务商屏蔽。\n"
"建议: 请尝试更换代理节点至支持的地区（如美国/日本/新加坡）。"
),
LLMErrorCode.API_REQUEST_FAILED: "AI服务请求失败，请稍后再试。",
LLMErrorCode.API_RESPONSE_INVALID: "AI服务响应异常，请稍后再试。",
LLMErrorCode.INVALID_PARAMETER: "请求参数错误，请检查输入内容。",
LLMErrorCode.CONFIGURATION_ERROR: "AI服务配置错误，请联系管理员。",
LLMErrorCode.CONTEXT_LENGTH_EXCEEDED: "输入内容过长，请缩短后重试。",
LLMErrorCode.CONTENT_FILTERED: "内容被安全过滤，请修改后重试。",
LLMErrorCode.RESPONSE_PARSE_ERROR: "AI服务响应解析失败，请稍后再试。",
LLMErrorCode.UNKNOWN_API_TYPE: "不支持的AI服务类型，请联系管理员。",
}
return error_messages.get(self.code, "AI服务暂时不可用，请稍后再试。")
def get_user_friendly_error_message(error: Exception) -> str:
"""将任何异常转换为用户友好的错误消息"""
if isinstance(error, LLMException):
return error.user_friendly_message
error_str = str(error).lower()
if "timeout" in error_str or "timed out" in error_str:
return "网络请求超时，请检查服务器网络或代理连接。"
if "connect" in error_str and ("refused" in error_str or "error" in error_str):
return "无法连接到 AI 服务商，请检查网络连接或代理设置。"
if "proxy" in error_str:
return "代理连接失败，请检查代理服务器是否正常运行。"
if "ssl" in error_str or "certificate" in error_str:
return "SSL 证书验证失败，请检查网络环境。"
if "permission" in error_str or "forbidden" in error_str:
return "权限不足，可能是 API Key 权限受限。"
if "not found" in error_str:
return "请求的资源未找到 (404)，请检查模型名称或端点配置。"
if "invalid" in error_str or "无效" in error_str:
return "请求参数无效，请检查输入。"
return f"服务暂时不可用 ({type(error).__name__})，请稍后再试。"
zhenxun/services/llm/types/models.py
-672
View File
@@ -1,672 +0,0 @@
"""
LLM 数据模型定义
包含模型信息、配置、工具定义和响应数据的模型类。
"""
import base64
from dataclasses import dataclass, field
from enum import Enum, auto
import mimetypes
from pathlib import Path
import sys
from typing import Any, Literal
import aiofiles
from pydantic import BaseModel, Field
from zhenxun.services.log import logger
if sys.version_info >= (3, 11):
from enum import StrEnum
else:
from strenum import StrEnum
class ModelProvider(Enum):
"""模型提供商枚举"""
OPENAI = "openai"
GEMINI = "gemini"
ZHIXPU = "zhipu"
CUSTOM = "custom"
class ResponseFormat(Enum):
"""响应格式枚举"""
TEXT = "text"
JSON = "json"
MULTIMODAL = "multimodal"
class StructuredOutputStrategy(str, Enum):
"""结构化输出策略"""
NATIVE = "native"
"""使用原生 API (如 OpenAI json_object/json_schema, Gemini mime_type)"""
TOOL_CALL = "tool_call"
"""构造虚假工具调用来强制输出结构化数据 (适用于指令跟随弱但工具调用强的模型)"""
PROMPT = "prompt"
"""仅在 Prompt 中追加 Schema 说明，依赖文本补全"""
class EmbeddingTaskType(str, Enum):
"""文本嵌入任务类型 (主要用于Gemini)"""
RETRIEVAL_QUERY = "RETRIEVAL_QUERY"
RETRIEVAL_DOCUMENT = "RETRIEVAL_DOCUMENT"
SEMANTIC_SIMILARITY = "SEMANTIC_SIMILARITY"
CLASSIFICATION = "CLASSIFICATION"
CLUSTERING = "CLUSTERING"
QUESTION_ANSWERING = "QUESTION_ANSWERING"
FACT_VERIFICATION = "FACT_VERIFICATION"
class ToolCategory(Enum):
"""工具分类枚举"""
FILE_SYSTEM = auto()
NETWORK = auto()
SYSTEM_INFO = auto()
CALCULATION = auto()
DATA_PROCESSING = auto()
CUSTOM = auto()
class CodeExecutionOutcome(StrEnum):
"""代码执行结果状态枚举"""
OUTCOME_OK = "OUTCOME_OK"
OUTCOME_FAILED = "OUTCOME_FAILED"
OUTCOME_DEADLINE_EXCEEDED = "OUTCOME_DEADLINE_EXCEEDED"
OUTCOME_COMPILATION_ERROR = "OUTCOME_COMPILATION_ERROR"
OUTCOME_RUNTIME_ERROR = "OUTCOME_RUNTIME_ERROR"
OUTCOME_UNKNOWN = "OUTCOME_UNKNOWN"
class TaskType(Enum):
"""任务类型枚举"""
CHAT = "chat"
CODE = "code"
SEARCH = "search"
ANALYSIS = "analysis"
GENERATION = "generation"
MULTIMODAL = "multimodal"
class LLMContentPart(BaseModel):
"""
LLM 消息内容部分 - 支持多模态内容。
这是一个联合体模型，`type` 字段决定了哪些其他字段是有效的。
例如：
- type='text': 使用 `text` 字段。
- type='image': 使用 `image_source` 字段。
- type='executable_code': 使用 `code_language` 和 `code_content` 字段。
"""
type: str
text: str | None = None
image_source: str | None = None
audio_source: str | None = None
video_source: str | None = None
document_source: str | None = None
file_uri: str | None = None
file_source: str | None = None
url: str | None = None
mime_type: str | None = None
thought_text: str | None = None
media_resolution: str | None = None
code_language: str | None = None
code_content: str | None = None
execution_outcome: str | None = None
execution_output: str | None = None
metadata: dict[str, Any] | None = None
def model_post_init(self, /, __context: Any) -> None:
"""验证内容部分的有效性"""
_ = __context
validation_rules = {
"text": lambda: self.text is not None,
"image": lambda: self.image_source,
"audio": lambda: self.audio_source,
"video": lambda: self.video_source,
"document": lambda: self.document_source,
"file": lambda: self.file_uri or self.file_source,
"url": lambda: self.url,
"thought": lambda: self.thought_text,
"executable_code": lambda: self.code_content is not None,
"execution_result": lambda: self.execution_outcome is not None,
}
if self.type in validation_rules:
if not validation_rules[self.type]():
raise ValueError(f"{self.type}类型的内容部分必须包含相应字段")
@classmethod
def text_part(cls, text: str) -> "LLMContentPart":
"""创建文本内容部分"""
return cls(type="text", text=text)
@classmethod
def thought_part(cls, text: str) -> "LLMContentPart":
"""创建思考过程内容部分"""
return cls(type="thought", thought_text=text)
@classmethod
def image_url_part(cls, url: str) -> "LLMContentPart":
"""创建图片URL内容部分"""
return cls(type="image", image_source=url)
@classmethod
def image_base64_part(
cls, data: str, mime_type: str = "image/png"
) -> "LLMContentPart":
"""创建Base64图片内容部分"""
data_url = f"data:{mime_type};base64,{data}"
return cls(type="image", image_source=data_url)
@classmethod
def audio_url_part(cls, url: str, mime_type: str = "audio/wav") -> "LLMContentPart":
"""创建音频URL内容部分"""
return cls(type="audio", audio_source=url, mime_type=mime_type)
@classmethod
def video_url_part(cls, url: str, mime_type: str = "video/mp4") -> "LLMContentPart":
"""创建视频URL内容部分"""
return cls(type="video", video_source=url, mime_type=mime_type)
@classmethod
def video_base64_part(
cls, data: str, mime_type: str = "video/mp4"
) -> "LLMContentPart":
"""创建Base64视频内容部分"""
data_url = f"data:{mime_type};base64,{data}"
return cls(type="video", video_source=data_url, mime_type=mime_type)
@classmethod
def audio_base64_part(
cls, data: str, mime_type: str = "audio/wav"
) -> "LLMContentPart":
"""创建Base64音频内容部分"""
data_url = f"data:{mime_type};base64,{data}"
return cls(type="audio", audio_source=data_url, mime_type=mime_type)
@classmethod
def file_uri_part(
cls,
file_uri: str,
mime_type: str | None = None,
metadata: dict[str, Any] | None = None,
) -> "LLMContentPart":
"""创建Gemini File API URI内容部分"""
return cls(
type="file",
file_uri=file_uri,
mime_type=mime_type,
metadata=metadata or {},
)
@classmethod
def executable_code_part(cls, language: str, code: str) -> "LLMContentPart":
"""创建可执行代码内容部分"""
return cls(type="executable_code", code_language=language, code_content=code)
@classmethod
def execution_result_part(
cls, outcome: str, output: str | None
) -> "LLMContentPart":
"""创建代码执行结果部分"""
return cls(
type="execution_result", execution_outcome=outcome, execution_output=output
)
@classmethod
async def from_path(
cls, path_like: str | Path, target_api: str | None = None
) -> "LLMContentPart | None":
"""
从本地文件路径创建 LLMContentPart。
自动检测MIME类型，并根据类型（如图片）可能加载为Base64。
target_api 可以用于提示如何最好地准备数据（例如 'gemini' 可能偏好 base64）
"""
try:
path = Path(path_like)
if not path.exists() or not path.is_file():
logger.warning(f"文件不存在或不是一个文件: {path}")
return None
mime_type, _ = mimetypes.guess_type(path.resolve().as_uri())
if not mime_type:
logger.warning(
f"无法猜测文件 {path.name} 的MIME类型，将尝试作为文本文件处理。"
)
try:
async with aiofiles.open(path, encoding="utf-8") as f:
text_content = await f.read()
return cls.text_part(text_content)
except Exception as e:
logger.error(f"读取文本文件 {path.name} 失败: {e}")
return None
if mime_type.startswith("image/"):
if target_api == "gemini" or not path.is_absolute():
try:
async with aiofiles.open(path, "rb") as f:
img_bytes = await f.read()
base64_data = base64.b64encode(img_bytes).decode("utf-8")
return cls.image_base64_part(
data=base64_data, mime_type=mime_type
)
except Exception as e:
logger.error(f"读取或编码图片文件 {path.name} 失败: {e}")
return None
else:
logger.warning(
f"为本地图片路径 {path.name} 生成 image_url_part。"
"实际API可能不支持 file:// URI。考虑使用Base64或公网URL。"
)
return cls.image_url_part(url=path.resolve().as_uri())
elif mime_type.startswith("audio/"):
return cls.audio_url_part(
url=path.resolve().as_uri(), mime_type=mime_type
)
elif mime_type.startswith("video/"):
if target_api == "gemini":
try:
async with aiofiles.open(path, "rb") as f:
video_bytes = await f.read()
base64_data = base64.b64encode(video_bytes).decode("utf-8")
return cls.video_base64_part(
data=base64_data, mime_type=mime_type
)
except Exception as e:
logger.error(f"读取或编码视频文件 {path.name} 失败: {e}")
return None
else:
return cls.video_url_part(
url=path.resolve().as_uri(), mime_type=mime_type
)
elif (
mime_type.startswith("text/")
or mime_type == "application/json"
or mime_type == "application/xml"
):
try:
async with aiofiles.open(path, encoding="utf-8") as f:
text_content = await f.read()
return cls.text_part(text_content)
except Exception as e:
logger.error(f"读取文本类文件 {path.name} 失败: {e}")
return None
else:
logger.info(
f"文件 {path.name} (MIME: {mime_type}) 将作为通用文件URI处理。"
)
return cls.file_uri_part(
file_uri=path.resolve().as_uri(),
mime_type=mime_type,
metadata={"name": path.name, "source": "local_path"},
)
except Exception as e:
logger.error(f"从路径 {path_like} 创建LLMContentPart时出错: {e}")
return None
def is_image_url(self) -> bool:
"""检查图像源是否为URL"""
if not self.image_source:
return False
return self.image_source.startswith(("http://", "https://"))
def is_image_base64(self) -> bool:
"""检查图像源是否为Base64 Data URL"""
if not self.image_source:
return False
return self.image_source.startswith("data:")
def get_base64_data(self) -> tuple[str, str] | None:
"""从Data URL中提取Base64数据和MIME类型"""
if not self.is_image_base64() or not self.image_source:
return None
try:
header, data = self.image_source.split(",", 1)
mime_part = header.split(";")[0].replace("data:", "")
return mime_part, data
except (ValueError, IndexError):
logger.warning(f"无法解析Base64图像数据: {self.image_source[:50]}...")
return None
class LLMMessage(BaseModel):
"""
LLM 消息对象，用于构建对话历史。
核心字段说明：
- role: 消息角色，推荐值为 'user', 'assistant', 'system', 'tool'。
- content: 消息内容，可以是纯文本字符串，也可以是 LLMContentPart 列表（用于多模态）
- tool_calls: (仅 assistant) 包含模型生成的工具调用请求。
- tool_call_id: (仅 tool) 对应 tool 消息响应的调用 ID。
- name: (仅 tool) 对应 tool 消息响应的函数名称。
"""
role: str
content: str | list[LLMContentPart]
name: str | None = None
tool_calls: list[Any] | None = None
tool_call_id: str | None = None
thought_signature: str | None = None
def model_post_init(self, /, __context: Any) -> None:
"""验证消息的有效性"""
_ = __context
if self.role == "tool":
if not self.tool_call_id:
raise ValueError("工具角色的消息必须包含 tool_call_id")
if not self.name:
raise ValueError("工具角色的消息必须包含函数名 (在 name 字段中)")
if self.role == "tool" and not isinstance(self.content, str):
logger.warning(
f"工具角色消息的内容期望是字符串，但得到的是: {type(self.content)}. "
"将尝试转换为字符串。"
)
try:
self.content = str(self.content)
except Exception as e:
raise ValueError(f"无法将工具角色的内容转换为字符串: {e}")
@classmethod
def user(cls, content: str | list[LLMContentPart]) -> "LLMMessage":
"""创建用户消息"""
return cls(role="user", content=content)
@classmethod
def assistant_tool_calls(
cls,
tool_calls: list[Any],
content: str | list[LLMContentPart] = "",
) -> "LLMMessage":
"""创建助手请求工具调用的消息"""
return cls(role="assistant", content=content, tool_calls=tool_calls)
@classmethod
def assistant_text_response(
cls, content: str | list[LLMContentPart]
) -> "LLMMessage":
"""创建助手纯文本回复的消息"""
return cls(role="assistant", content=content, tool_calls=None)
@classmethod
def tool_response(
cls,
tool_call_id: str,
function_name: str,
result: Any,
) -> "LLMMessage":
"""创建工具执行结果的消息"""
import json
try:
content_str = json.dumps(result)
except TypeError as e:
logger.error(
f"工具 '{function_name}' 的结果无法JSON序列化: {result}. 错误: {e}"
)
content_str = json.dumps(
{"error": "工具结果无法JSON序列化", "details": str(e)}
)
return cls(
role="tool",
content=content_str,
tool_call_id=tool_call_id,
name=function_name,
)
@classmethod
def system(cls, content: str) -> "LLMMessage":
"""创建系统消息"""
return cls(role="system", content=content)
class LLMResponse(BaseModel):
"""
LLM 响应对象，封装了模型生成的全部信息。
核心字段说明：
- text: 模型生成的文本内容。如果是纯文本回复，此字段即为结果。
- tool_calls: 如果模型决定调用工具，此列表包含调用详情。
- content_parts: 包含多模态或结构化内容的原始部分列表（如思维链、代码块）。
- raw_response: 原始的第三方 API 响应字典（用于调试）。
- images: 如果请求涉及生图，此处包含生成的图片数据。
"""
text: str
content_parts: list[Any] | None = None
images: list[bytes | Path] | None = None
usage_info: dict[str, Any] | None = None
raw_response: dict[str, Any] | None = None
tool_calls: list[Any] | None = None
code_executions: list[Any] | None = None
grounding_metadata: Any | None = None
cache_info: Any | None = None
thought_text: str | None = None
thought_signature: str | None = None
ModelName = str | None
class ToolDefinition(BaseModel):
"""
一个结构化的工具定义模型，用于向LLM描述工具。
"""
name: str = Field(..., description="工具的唯一名称标识")
description: str = Field(..., description="工具功能的清晰描述")
parameters: dict[str, Any] = Field(
default_factory=dict, description="符合JSON Schema规范的参数定义"
)
class ToolChoice(BaseModel):
"""统一的工具选择配置"""
mode: Literal["auto", "none", "any", "required"] = Field(
default="auto", description="工具调用模式"
)
allowed_function_names: list[str] | None = Field(
default=None, description="允许调用的函数名称列表"
)
class BasePlatformTool(BaseModel):
"""平台原生工具基类"""
class Config:
extra = "forbid"
def get_tool_declaration(self) -> dict[str, Any]:
"""获取放入 'tools' 列表中的声明对象 (Snake Case)"""
raise NotImplementedError
def get_tool_config(self) -> dict[str, Any] | None:
"""获取放入 'toolConfig' 中的配置对象 (Snake Case)"""
return None
class GeminiCodeExecution(BasePlatformTool):
"""Gemini 代码执行工具"""
def get_tool_declaration(self) -> dict[str, Any]:
return {"code_execution": {}}
class GeminiGoogleSearch(BasePlatformTool):
"""Gemini 谷歌搜索 (Grounding) 工具"""
mode: Literal["MODE_DYNAMIC"] = "MODE_DYNAMIC"
dynamic_threshold: float | None = Field(default=None)
def get_tool_declaration(self) -> dict[str, Any]:
return {"google_search": {}}
def get_tool_config(self) -> dict[str, Any] | None:
return None
class GeminiUrlContext(BasePlatformTool):
"""Gemini 网址上下文工具"""
urls: list[str] = Field(..., description="作为上下文的 URL 列表", max_length=20)
def get_tool_declaration(self) -> dict[str, Any]:
return {"google_search": {}, "url_context": {}}
def get_tool_config(self) -> dict[str, Any] | None:
return None
class ToolResult(BaseModel):
"""
一个结构化的工具执行结果模型。
"""
output: Any = Field(..., description="返回给LLM的、可JSON序列化的原始输出")
display_content: str | None = Field(
default=None, description="用于日志或UI展示的人类可读的执行摘要"
)
@dataclass(frozen=True)
class ModelInfo:
"""模型信息（不可变数据类）"""
name: str
provider: ModelProvider
max_tokens: int = 4096
supports_tools: bool = False
supports_vision: bool = False
supports_audio: bool = False
cost_per_1k_tokens: float = 0.0
@dataclass
class UsageInfo:
"""使用信息数据类"""
prompt_tokens: int = 0
completion_tokens: int = 0
total_tokens: int = 0
cost: float = 0.0
@property
def efficiency_ratio(self) -> float:
"""计算效率比（输出/输入）"""
return self.completion_tokens / max(self.prompt_tokens, 1)
@dataclass
class ToolMetadata:
"""工具元数据"""
name: str
description: str
category: ToolCategory
read_only: bool = True
destructive: bool = False
open_world: bool = False
parameters: dict[str, Any] = field(default_factory=dict)
required_params: list[str] = field(default_factory=list)
class ModelDetail(BaseModel):
"""模型详细信息"""
model_name: str
is_available: bool = True
is_embedding_model: bool = False
temperature: float | None = None
max_tokens: int | None = None
api_type: str | None = None
endpoint: str | None = None
class ProviderConfig(BaseModel):
"""LLM 提供商配置"""
name: str = Field(..., description="Provider 的唯一名称标识")
api_key: str | list[str] = Field(..., description="API Key 或 Key 列表")
api_base: str | None = Field(None, description="API Base URL，如果为空则使用默认值")
api_type: str = Field(default="openai", description="API 类型")
openai_compat: bool = Field(default=False, description="是否使用 OpenAI 兼容模式")
temperature: float | None = Field(default=0.7, description="默认温度参数")
max_tokens: int | None = Field(default=None, description="默认最大输出 token 限制")
models: list[ModelDetail] = Field(..., description="支持的模型列表")
timeout: int = Field(default=180, description="请求超时时间")
proxy: str | None = Field(default=None, description="代理设置")
extra_headers: dict[str, str] | None = Field(
default=None,
description="额外请求头，用于网关鉴权等场景",
)
class LLMToolFunction(BaseModel):
"""LLM 工具函数定义"""
name: str
arguments: str
class LLMToolCall(BaseModel):
"""LLM 工具调用"""
id: str
function: LLMToolFunction
thought_signature: str | None = None
class LLMCodeExecution(BaseModel):
"""代码执行结果"""
code: str
output: str | None = None
error: str | None = None
execution_time: float | None = None
files_generated: list[str] | None = None
class LLMGroundingAttribution(BaseModel):
"""信息来源关联"""
title: str | None = None
uri: str | None = None
snippet: str | None = None
confidence_score: float | None = None
class LLMGroundingMetadata(BaseModel):
"""信息来源关联元数据"""
web_search_queries: list[str] | None = None
grounding_attributions: list[LLMGroundingAttribution] | None = None
search_suggestions: list[dict[str, Any]] | None = None
search_entry_point: str | None = Field(
default=None, description="Google搜索建议的HTML片段(renderedContent)"
)
map_widget_token: str | None = Field(
default=None, description="Google Maps 前端组件令牌"
)
class LLMCacheInfo(BaseModel):
"""缓存信息"""
cache_hit: bool = False
cache_key: str | None = None
cache_ttl: int | None = None
created_at: str | None = None
zhenxun/services/llm/types/protocols.py
-156
View File
@@ -1,156 +0,0 @@
"""
LLM 模块的协议定义
"""
from abc import ABC
from typing import TYPE_CHECKING, Any, Protocol, Union
from pydantic import BaseModel
from .models import ToolDefinition, ToolResult
if TYPE_CHECKING:
from .models import LLMMessage, LLMResponse, LLMToolCall
class ToolCallData(BaseModel):
"""传递给 on_tool_start 的数据模型"""
tool_name: str
tool_args: dict[str, Any]
class ToolCallCompleteData(BaseModel):
"""传递给 on_tool_call_complete 的数据模型"""
id: str
name: str
arguments: str
result: "ToolResult"
class BaseCallbackHandler(ABC):
"""
Agent/LLM 生命周期回调处理器的基类。
下沉至 LLM 层以允许 ToolInvoker 直接调用。
"""
async def on_agent_start(self, messages: list["LLMMessage"], **kwargs: Any) -> None:
"""在 AgentExecutor 开始运行时调用。"""
pass
async def on_model_start(
self, model_name: str, messages: list["LLMMessage"], **kwargs: Any
) -> None:
"""在向LLM发起请求之前调用。"""
pass
async def on_model_end(
self, response: "LLMResponse", duration: float, **kwargs: Any
) -> None:
"""在收到LLM响应之后调用。"""
pass
async def on_tool_start(
self, tool_call: "LLMToolCall", data: ToolCallData, **kwargs: Any
) -> Union[ToolCallData, "ToolResult", None]:
"""
在单个工具即将被执行时调用。
返回:
ToolCallData: 修改参数并继续执行
ToolResult: 拦截执行并直接返回给模型
None: 正常继续
"""
pass
async def on_tool_end(
self,
result: Union["ToolResult", None],
error: Exception | None,
tool_call: "LLMToolCall",
duration: float,
**kwargs: Any,
) -> None:
"""在单个工具执行完毕后调用，无论成功或失败。"""
pass
async def on_tool_call_complete(
self, data: ToolCallCompleteData, **kwargs: Any
) -> None:
"""在工具调用完成并准备创建响应消息时调用。"""
pass
async def on_human_input_request(self, query: str, **kwargs: Any) -> str | None:
"""
当 Agent 需要人类输入时调用。
"""
return None
async def on_agent_end(
self, final_history: list["LLMMessage"], duration: float, **kwargs: Any
) -> None:
"""在 AgentExecutor 运行结束时调用。"""
pass
class ToolExecutable(Protocol):
"""
一个协议，定义了所有可被LLM调用的工具必须实现的行为。
它将工具的"定义"（给LLM看）和"执行"（由框架调用）封装在一起。
"""
async def get_definition(self) -> ToolDefinition:
"""
异步地获取一个结构化的工具定义。
"""
...
async def execute(self, context: Any | None = None, **kwargs: Any) -> ToolResult:
"""
异步执行工具并返回一个结构化的结果。
参数由LLM根据工具定义生成。
Args:
context: 运行时上下文 (RunContext)，可选注入
**kwargs: 工具参数
"""
...
class ToolProvider(Protocol):
"""
一个协议，定义了"工具提供者"的行为。
工具提供者负责发现或实例化具体的 ToolExecutable 对象。
"""
async def initialize(self) -> None:
"""
异步初始化提供者。
此方法应是幂等的，即多次调用只会执行一次初始化逻辑。
用于执行耗时的I/O操作，如网络请求或启动子进程。
"""
...
async def discover_tools(
self,
allowed_servers: list[str] | None = None,
excluded_servers: list[str] | None = None,
) -> dict[str, ToolExecutable]:
"""
异步发现此提供者提供的所有工具。
在 `initialize` 成功调用后才应被调用。
返回:
一个从工具名称到 ToolExecutable 实例的字典。
"""
...
async def get_tool_executable(
self, name: str, config: dict[str, Any]
) -> ToolExecutable | None:
"""
【保留】如果此提供者能处理名为 'name' 的工具，则返回一个可执行实例。
此方法主要用于按需解析 ad-hoc 工具。
"""
...
zhenxun/services/llm/utils.py
-593
View File
@@ -1,593 +0,0 @@
"""
LLM 模块的工具和转换函数
"""
import base64
from collections.abc import Awaitable, Callable
import io
from pathlib import Path
from typing import Any, TypeVar
import aiofiles
import json_repair
from nonebot.adapters import Message as PlatformMessage
from nonebot.compat import type_validate_json
from nonebot_plugin_alconna.uniseg import (
At,
File,
Image,
Reply,
Segment,
Text,
UniMessage,
Video,
Voice,
)
from PIL.Image import Image as PILImageType
from pydantic import BaseModel, Field, ValidationError, create_model
from zhenxun.services.log import logger
from zhenxun.utils.http_utils import AsyncHttpx
from zhenxun.utils.pydantic_compat import model_validate
from .types import LLMContentPart, LLMErrorCode, LLMException, LLMMessage
from .types.capabilities import ReasoningMode, get_model_capabilities
T = TypeVar("T", bound=BaseModel)
S = TypeVar("S", bound=Segment)
_SEGMENT_HANDLERS: dict[
type[Segment], Callable[[Any], Awaitable[LLMContentPart | None]]
] = {}
def register_segment_handler(seg_type: type[S]):
"""装饰器：注册 Uniseg 消息段的处理器"""
def decorator(func: Callable[[S], Awaitable[LLMContentPart | None]]):
_SEGMENT_HANDLERS[seg_type] = func
return func
return decorator
async def _process_media_data(seg: Any, default_mime: str) -> tuple[str, str] | None:
"""
[内部复用] 通用媒体数据处理：获取 Base64 数据和 MIME 类型。
优先顺序：Raw -> Path -> URL (下载)
"""
mime_type = getattr(seg, "mimetype", None) or default_mime
b64_data = None
if hasattr(seg, "raw") and seg.raw:
if isinstance(seg.raw, bytes):
b64_data = base64.b64encode(seg.raw).decode("utf-8")
elif getattr(seg, "path", None):
try:
path = Path(seg.path)
if path.exists():
async with aiofiles.open(path, "rb") as f:
content = await f.read()
b64_data = base64.b64encode(content).decode("utf-8")
except Exception as e:
logger.error(f"读取媒体文件失败: {seg.path}, 错误: {e}")
elif getattr(seg, "url", None):
try:
logger.debug(f"检测到媒体URL，开始下载: {seg.url}")
media_bytes = await AsyncHttpx.get_content(seg.url)
b64_data = base64.b64encode(media_bytes).decode("utf-8")
logger.debug(f"媒体文件下载成功，大小: {len(media_bytes)} bytes")
except Exception as e:
logger.error(f"从URL下载媒体失败: {seg.url}, 错误: {e}")
return None
if b64_data:
return mime_type, b64_data
return None
@register_segment_handler(Text)
async def _handle_text(seg: Text) -> LLMContentPart | None:
if seg.text.strip():
return LLMContentPart.text_part(seg.text)
return None
@register_segment_handler(Image)
async def _handle_image(seg: Image) -> LLMContentPart | None:
media_info = await _process_media_data(seg, "image/png")
if media_info:
mime, data = media_info
return LLMContentPart.image_base64_part(data, mime)
return None
@register_segment_handler(Voice)
async def _handle_voice(seg: Voice) -> LLMContentPart | None:
media_info = await _process_media_data(seg, "audio/wav")
if media_info:
mime, data = media_info
return LLMContentPart.audio_base64_part(data, mime)
return LLMContentPart.text_part(f"[语音消息: {seg.id or 'unknown'}]")
@register_segment_handler(Video)
async def _handle_video(seg: Video) -> LLMContentPart | None:
media_info = await _process_media_data(seg, "video/mp4")
if media_info:
mime, data = media_info
return LLMContentPart.video_base64_part(data, mime)
return LLMContentPart.text_part(f"[视频消息: {seg.id or 'unknown'}]")
@register_segment_handler(File)
async def _handle_file(seg: File) -> LLMContentPart | None:
if seg.path:
return await LLMContentPart.from_path(seg.path)
return LLMContentPart.text_part(f"[文件: {seg.name} (ID: {seg.id})]")
@register_segment_handler(At)
async def _handle_at(seg: At) -> LLMContentPart | None:
if seg.flag == "all":
return LLMContentPart.text_part("[提及所有人]")
return LLMContentPart.text_part(f"[提及用户: {seg.target}]")
@register_segment_handler(Reply)
async def _handle_reply(seg: Reply) -> LLMContentPart | None:
text = str(seg.msg) if seg.msg else ""
if text:
return LLMContentPart.text_part(f'[回复消息: "{text[:50]}..."]')
return LLMContentPart.text_part("[回复了一条消息]")
async def _transform_to_content_part(item: Any) -> LLMContentPart:
"""
将混合输入转换为统一的 LLMContentPart，便于 normalize_to_llm_messages 使用。
"""
if isinstance(item, LLMContentPart):
return item
if isinstance(item, str):
return LLMContentPart.text_part(item)
if isinstance(item, Path):
part = await LLMContentPart.from_path(item)
if part is None:
raise ValueError(f"无法从路径加载内容: {item}")
return part
if isinstance(item, dict):
return LLMContentPart(**item)
if PILImageType and isinstance(item, PILImageType):
buffer = io.BytesIO()
fmt = item.format or "PNG"
item.save(buffer, format=fmt)
b64_data = base64.b64encode(buffer.getvalue()).decode("utf-8")
mime_type = f"image/{fmt.lower()}"
return LLMContentPart.image_base64_part(b64_data, mime_type)
raise TypeError(f"不支持的输入类型用于构建 ContentPart: {type(item)}")
async def unimsg_to_llm_parts(message: UniMessage) -> list[LLMContentPart]:
"""
将 UniMessage 实例转换为一个 LLMContentPart 列表。
这是处理多模态输入的核心转换逻辑。
参数:
message: 要转换的UniMessage实例。
返回:
list[LLMContentPart]: 转换后的内容部分列表。
"""
if not _SEGMENT_HANDLERS:
pass
parts: list[LLMContentPart] = []
for seg in message:
handler = _SEGMENT_HANDLERS.get(type(seg))
if handler:
try:
part = await handler(seg)
if part:
parts.append(part)
except Exception as e:
logger.warning(f"处理消息段 {seg} 失败: {e}", "LLMUtils")
return parts
async def normalize_to_llm_messages(
message: str | UniMessage | LLMMessage | list[Any],
instruction: str | None = None,
) -> list[LLMMessage]:
"""
将多种输入格式标准化为 LLMMessage 列表，并可选地添加系统指令。
这是处理 LLM 输入的核心工具函数。
参数:
message: 要标准化的输入消息。
instruction: 可选的系统指令。
返回:
list[LLMMessage]: 标准化后的消息列表。
"""
messages = []
if instruction:
messages.append(LLMMessage.system(instruction))
if isinstance(message, LLMMessage):
messages.append(message)
elif isinstance(message, list) and all(isinstance(m, LLMMessage) for m in message):
messages.extend(message)
elif isinstance(message, str):
messages.append(LLMMessage.user(message))
elif isinstance(message, UniMessage):
content_parts = await unimsg_to_llm_parts(message)
messages.append(LLMMessage.user(content_parts))
elif isinstance(message, list):
parts = []
for item in message:
parts.append(await _transform_to_content_part(item))
messages.append(LLMMessage.user(parts))
else:
raise TypeError(f"不支持的消息类型: {type(message)}")
return messages
def create_multimodal_message(
text: str | None = None,
images: list[str | Path | bytes] | str | Path | bytes | None = None,
videos: list[str | Path | bytes] | str | Path | bytes | None = None,
audios: list[str | Path | bytes] | str | Path | bytes | None = None,
image_mimetypes: list[str] | str | None = None,
video_mimetypes: list[str] | str | None = None,
audio_mimetypes: list[str] | str | None = None,
) -> UniMessage:
"""
创建多模态消息的便捷函数
参数:
text: 文本内容
images: 图片数据，支持路径、字节数据或URL
videos: 视频数据
audios: 音频数据
image_mimetypes: 图片MIME类型，bytes数据时需要指定
video_mimetypes: 视频MIME类型，bytes数据时需要指定
audio_mimetypes: 音频MIME类型，bytes数据时需要指定
返回:
UniMessage: 构建好的多模态消息
"""
message = UniMessage()
if text:
message.append(Text(text))
if images is not None:
_add_media_to_message(message, images, image_mimetypes, Image, "image/png")
if videos is not None:
_add_media_to_message(message, videos, video_mimetypes, Video, "video/mp4")
if audios is not None:
_add_media_to_message(message, audios, audio_mimetypes, Voice, "audio/wav")
return message
def _add_media_to_message(
message: UniMessage,
media_items: list[str | Path | bytes] | str | Path | bytes,
mimetypes: list[str] | str | None,
media_class: type,
default_mimetype: str,
) -> None:
"""添加媒体文件到 UniMessage"""
if not isinstance(media_items, list):
media_items = [media_items]
mime_list = []
if mimetypes is not None:
if isinstance(mimetypes, str):
mime_list = [mimetypes] * len(media_items)
else:
mime_list = list(mimetypes)
for i, item in enumerate(media_items):
if isinstance(item, str | Path):
if str(item).startswith(("http://", "https://")):
message.append(media_class(url=str(item)))
else:
message.append(media_class(path=Path(item)))
elif isinstance(item, bytes):
mimetype = mime_list[i] if i < len(mime_list) else default_mimetype
message.append(media_class(raw=item, mimetype=mimetype))
def message_to_unimessage(message: PlatformMessage) -> UniMessage:
"""
将平台特定的 Message 对象转换为通用的 UniMessage。
主要用于处理引用消息等未被自动转换的消息体。
参数:
message: 平台特定的Message对象。
返回:
UniMessage: 转换后的通用消息对象。
"""
return UniMessage.of(message)
def resolve_json_schema_refs(schema: dict) -> dict:
"""
递归解析 JSON Schema 中的 $ref，将其替换为 $defs/definitions 中的定义。
用于兼容不支持 $ref 的 Gemini API。
"""
definitions = schema.get("$defs") or schema.get("definitions") or {}
def _resolve(node: Any) -> Any:
if isinstance(node, dict):
if "$ref" in node:
ref_name = node["$ref"].split("/")[-1]
if ref_name in definitions:
return _resolve(definitions[ref_name])
return {
key: _resolve(value)
for key, value in node.items()
if key not in ("$defs", "definitions")
}
if isinstance(node, list):
return [_resolve(item) for item in node]
return node
return _resolve(schema)
def sanitize_schema_for_llm(schema: Any, api_type: str) -> Any:
"""
递归地净化 JSON Schema，移除特定 LLM API 不支持的关键字。
"""
if isinstance(schema, list):
return [sanitize_schema_for_llm(item, api_type) for item in schema]
if isinstance(schema, dict):
schema_copy = schema.copy()
if api_type == "gemini":
if "const" in schema_copy:
schema_copy["enum"] = [schema_copy.pop("const")]
if "type" in schema_copy and isinstance(schema_copy["type"], list):
types_list = schema_copy["type"]
if "null" in types_list:
schema_copy["nullable"] = True
types_list = [t for t in types_list if t != "null"]
if len(types_list) == 1:
schema_copy["type"] = types_list[0]
else:
schema_copy["type"] = types_list
if "anyOf" in schema_copy:
any_of = schema_copy["anyOf"]
has_null = any(
isinstance(x, dict) and x.get("type") == "null" for x in any_of
)
if has_null:
schema_copy["nullable"] = True
new_any_of = [
x
for x in any_of
if not (isinstance(x, dict) and x.get("type") == "null")
]
if len(new_any_of) == 1:
schema_copy.update(new_any_of[0])
schema_copy.pop("anyOf", None)
else:
schema_copy["anyOf"] = new_any_of
unsupported_keys = [
"exclusiveMinimum",
"exclusiveMaximum",
"default",
"title",
"additionalProperties",
"$schema",
"$id",
]
for key in unsupported_keys:
schema_copy.pop(key, None)
if schema_copy.get("format") and schema_copy["format"] not in [
"enum",
"date-time",
]:
schema_copy.pop("format", None)
elif api_type == "openai":
unsupported_keys = [
"default",
"minLength",
"maxLength",
"pattern",
"format",
"minimum",
"maximum",
"multipleOf",
"patternProperties",
"minItems",
"maxItems",
"uniqueItems",
"$schema",
"title",
]
for key in unsupported_keys:
schema_copy.pop(key, None)
if "$ref" in schema_copy:
ref_key = schema_copy["$ref"].split("/")[-1]
defs = schema_copy.get("$defs") or schema_copy.get("definitions")
if defs and ref_key in defs:
schema_copy.pop("$ref", None)
schema_copy.update(defs[ref_key])
else:
return {"$ref": schema_copy["$ref"]}
is_object = (
schema_copy.get("type") == "object" or "properties" in schema_copy
)
if is_object:
schema_copy["type"] = "object"
schema_copy["additionalProperties"] = False
properties = schema_copy.get("properties", {})
required = schema_copy.get("required", [])
if properties:
existing_req = set(required)
for prop in properties.keys():
if prop not in existing_req:
required.append(prop)
schema_copy["required"] = required
for def_key in ["$defs", "definitions"]:
if def_key in schema_copy and isinstance(schema_copy[def_key], dict):
schema_copy[def_key] = {
k: sanitize_schema_for_llm(v, api_type)
for k, v in schema_copy[def_key].items()
}
recursive_keys = ["properties", "items", "allOf", "anyOf", "oneOf"]
for key in recursive_keys:
if key in schema_copy:
if key == "properties" and isinstance(schema_copy[key], dict):
schema_copy[key] = {
k: sanitize_schema_for_llm(v, api_type)
for k, v in schema_copy[key].items()
}
else:
schema_copy[key] = sanitize_schema_for_llm(
schema_copy[key], api_type
)
return schema_copy
else:
return schema
def extract_text_from_content(
content: str | list[LLMContentPart] | None,
) -> str:
"""
从消息内容中提取纯文本，自动过滤非文本部分，防止污染 Prompt。
"""
if content is None:
return ""
if isinstance(content, str):
return content
if isinstance(content, list):
return " ".join(
part.text for part in content if part.type == "text" and part.text
)
return str(content)
def parse_and_validate_json(text: str, response_model: type[T]) -> T:
"""
通用工具：尝试将文本解析为指定的 Pydantic 模型，并统一处理异常。
"""
try:
return type_validate_json(response_model, text)
except (ValidationError, ValueError) as e:
try:
logger.warning(f"标准JSON解析失败，尝试使用json_repair修复: {e}")
repaired_obj = json_repair.loads(text, skip_json_loads=True)
return model_validate(response_model, repaired_obj)
except Exception as repair_error:
logger.error(
f"LLM结构化输出校验最终失败: {repair_error}",
e=repair_error,
)
raise LLMException(
"LLM返回的JSON未能通过结构验证。",
code=LLMErrorCode.RESPONSE_PARSE_ERROR,
details={
"raw_response": text,
"validation_error": str(repair_error),
"original_error": repair_error,
},
cause=repair_error,
)
except Exception as e:
logger.error(f"解析LLM结构化输出时发生未知错误: {e}", e=e)
raise LLMException(
"解析LLM的JSON输出时失败。",
code=LLMErrorCode.RESPONSE_PARSE_ERROR,
details={"raw_response": text},
cause=e,
)
def create_cot_wrapper(inner_model: type[BaseModel]) -> type[BaseModel]:
"""
[动态运行时封装]
创建一个包含思维链 (Chain of Thought) 的包装模型。
强制模型在生成最终 JSON 结构前，先输出一个 reasoning 字段进行思考。
"""
wrapper_name = f"CoT_{inner_model.__name__}"
return create_model(
wrapper_name,
reasoning=(
str,
Field(
...,
min_length=10,
description=(
"在生成最终结果之前，请务必在此字段中详细描述你的推理步骤、计算过程或思考逻辑。禁止留空。"
),
),
),
result=(
inner_model,
Field(
...,
),
),
)
def should_apply_autocot(
requested: bool,
model_name: str | None,
config: Any,
) -> bool:
"""
[智能决策管道]
判断是否应该应用 AutoCoT (显式思维链包装)。
防止在模型已有原生思维能力时进行“双重思考”。
"""
if not requested:
return False
if config:
thinking_budget = getattr(config, "thinking_budget", 0) or 0
if thinking_budget > 0:
return False
if getattr(config, "thinking_level", None) is not None:
return False
if model_name:
caps = get_model_capabilities(model_name)
if caps.reasoning_mode != ReasoningMode.NONE:
return False
return True