Diuse/zhenxun_bot
1
0
Fork 0
mirror of https://github.com/zhenxun-org/zhenxun_bot.git synced 2026-10-20 03:30:11 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

♻️ refactor(llm): 重构 LLM 服务架构，引入中间件与组件化适配器 (#2073)
检查bot是否运行正常 / bot check (push) Has been cancelled
Details
CodeQL Code Security Analysis / Analyze (${{ matrix.language }}) (none, python) (push) Has been cancelled
Details
Sequential Lint and Type Check / ruff-call (push) Has been cancelled
Details
Release Drafter / Update Release Draft (push) Has been cancelled
Details
Force Sync to Aliyun / sync (push) Has been cancelled
Details
Update Version / update-version (push) Has been cancelled
Details
Sequential Lint and Type Check / pyright-call (push) Has been cancelled
Details

Browse Source 
* ♻️ refactor(llm): 重构 LLM 服务架构，引入中间件与组件化适配器

- 【重构】LLM 服务核心架构：
    - 引入中间件管道，统一处理请求生命周期（重试、密钥选择、日志、网络请求）。
    - 适配器重构为组件化设计，分离配置映射、消息转换、响应解析和工具序列化逻辑。
    - 移除 `with_smart_retry` 装饰器，其功能由中间件接管。
    - 移除 `LLMToolExecutor`，工具执行逻辑集成到 `ToolInvoker`。
- 【功能】增强配置系统：
    - `LLMGenerationConfig` 采用组件化结构（Core, Reasoning, Visual, Output, Safety, ToolConfig）。
    - 新增 `GenConfigBuilder` 提供语义化配置构建方式。
    - 新增 `LLMEmbeddingConfig` 用于嵌入专用配置。
    - `CommonOverrides` 迁移并更新至新配置结构。
- 【功能】强化工具系统：
    - 引入 `ToolInvoker` 实现更灵活的工具执行，支持回调与结构化错误。
    - `function_tool` 装饰器支持动态 Pydantic 模型创建和依赖注入 (`ToolParam`, `RunContext`)。
    - 平台原生工具支持 (`GeminiCodeExecution`, `GeminiGoogleSearch`, `GeminiUrlContext`)。
- 【功能】高级生成与嵌入：
    - `generate_structured` 方法支持 In-Context Validation and Repair (IVR) 循环和 AutoCoT (思维链) 包装。
    - 新增 `embed_query` 和 `embed_documents` 便捷嵌入 API。
    - `OpenAIImageAdapter` 支持 OpenAI 兼容的图像生成。
    - `SmartAdapter` 实现模型名称智能路由。
- 【重构】消息与类型系统：
    - `LLMContentPart` 扩展支持更多模态和代码执行相关内容。
    - `LLMMessage` 和 `LLMResponse` 结构更新，支持 `content_parts` 和思维链签名。
    - 统一 `LLMErrorCode` 和用户友好错误消息，提供更详细的网络/代理错误提示。
    - `pyproject.toml` 移除 `bilireq`，新增 `json_repair`。
- 【优化】日志与调试：
    - 引入 `DebugLogOptions`，提供细粒度日志脱敏控制。
    - 增强日志净化器，处理更多敏感数据和长字符串。
- 【清理】删除废弃模块：
    - `zhenxun/services/llm/memory.py`
    - `zhenxun/services/llm/executor.py`
    - `zhenxun/services/llm/config/presets.py`
    - `zhenxun/services/llm/types/content.py`
    - `zhenxun/services/llm/types/enums.py`
    - `zhenxun/services/llm/tools/__init__.py`
    - `zhenxun/services/llm/tools/manager.py`

* 📦️ build(deps): 移除 bilireq 并添加 json_repair 依赖

* 🐛 (llm): 移除图片生成模型能力预检查

* ♻️ refactor(llm.session): 重构记忆系统以分离存储和策略

* 🐛 fix(reload_setting): 重载配置时清除LLM缓存

* ✨ feat(llm): 支持结构化生成函数接收 UniMessage

* ✨ feat(search): 为搜索功能默认启用 Gemini Google Search 工具

* 🚨 auto fix by pre-commit hooks

---------

Co-authored-by: webjoin111 <455457521@qq.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Rumio
2025-12-14 20:27:02 +08:00
committed by GitHub
co-authored by webjoin111 pre-commit-ci[bot]
parent e5b2a872d3
commit c9f0a8b9d9
41 changed files with 6234 additions and 14360 deletions
Download Patch File Download Diff File Expand all files Collapse all files
zhenxun/services/llm/adapters/__init__.py
+3 -1
View File
@@ -7,16 +7,18 @@ LLM 适配器模块
from .base import BaseAdapter, OpenAICompatAdapter, RequestData, ResponseData
from .factory import LLMAdapterFactory, get_adapter_for_api_type, register_adapter
from .gemini import GeminiAdapter
from .openai import OpenAIAdapter
from .openai import DeepSeekAdapter, OpenAIAdapter, OpenAIImageAdapter
LLMAdapterFactory.initialize()
__all__ = [
"BaseAdapter",
"DeepSeekAdapter",
"GeminiAdapter",
"LLMAdapterFactory",
"OpenAIAdapter",
"OpenAICompatAdapter",
"OpenAIImageAdapter",
"RequestData",
"ResponseData",
"get_adapter_for_api_type",
zhenxun/services/llm/adapters/base.py
+179 -193
View File
@@ -3,24 +3,26 @@ LLM 适配器基类和通用数据结构
"""
from abc import ABC, abstractmethod
import base64
import binascii
import json
from pathlib import Path
from typing import TYPE_CHECKING, Any
import uuid
import httpx
from pydantic import BaseModel
from zhenxun.configs.path_config import TEMP_PATH
from zhenxun.services.log import logger
from ..types import LLMContentPart
from ..types.exceptions import LLMErrorCode, LLMException
from ..types.models import LLMToolCall
if TYPE_CHECKING:
from ..config.generation import LLMGenerationConfig
from ..config.generation import LLMEmbeddingConfig, LLMGenerationConfig
from ..service import LLMModel
from ..types.content import LLMMessage
from ..types.enums import EmbeddingTaskType
from ..types.protocols import ToolExecutable
from ..types import LLMMessage
from ..types.models import ToolChoice
class RequestData(BaseModel):
@@ -29,19 +31,23 @@ class RequestData(BaseModel):
url: str
headers: dict[str, str]
body: dict[str, Any]
files: dict[str, Any] | list[tuple[str, Any]] | None = None
class ResponseData(BaseModel):
"""响应数据封装 - 支持所有高级功能"""
text: str
images: list[bytes] | None = None
content_parts: list[LLMContentPart] | None = None
images: list[bytes | Path] | None = None
usage_info: dict[str, Any] | None = None
raw_response: dict[str, Any] | None = None
tool_calls: list[LLMToolCall] | None = None
code_executions: list[Any] | None = None
grounding_metadata: Any | None = None
cache_info: Any | None = None
thought_text: str | None = None
thought_signature: str | None = None
code_execution_results: list[dict[str, Any]] | None = None
search_results: list[dict[str, Any]] | None = None
@@ -50,9 +56,33 @@ class ResponseData(BaseModel):
citations: list[dict[str, Any]] | None = None
def process_image_data(image_data: bytes) -> bytes | Path:
"""
处理图片数据：若超过 2MB 则保存到临时目录，避免占用内存。
"""
max_inline_size = 2 * 1024 * 1024
if len(image_data) > max_inline_size:
save_dir = TEMP_PATH / "llm"
save_dir.mkdir(parents=True, exist_ok=True)
file_name = f"{uuid.uuid4()}.png"
file_path = save_dir / file_name
file_path.write_bytes(image_data)
logger.info(
f"图片数据过大 ({len(image_data)} bytes)，已保存到临时文件: {file_path}",
"LLMAdapter",
)
return file_path.resolve()
return image_data
class BaseAdapter(ABC):
"""LLM API适配器基类"""
@property
def log_sanitization_context(self) -> str:
"""用于日志清洗的上下文名称，默认 'default'"""
return "default"
@property
@abstractmethod
def api_type(self) -> str:
@@ -77,7 +107,7 @@ class BaseAdapter(ABC):
默认实现：将简单请求转换为高级请求格式
子类可以重写此方法以提供特定的优化实现
"""
from ..types.content import LLMMessage
from ..types import LLMMessage
messages: list[LLMMessage] = []
@@ -107,8 +137,8 @@ class BaseAdapter(ABC):
api_key: str,
messages: list["LLMMessage"],
config: "LLMGenerationConfig | None" = None,
tools: dict[str, "ToolExecutable"] | None = None,
tool_choice: str | dict[str, Any] | None = None,
tools: list[Any] | None = None,
tool_choice: "str | dict[str, Any] | ToolChoice | None" = None,
) -> RequestData:
"""准备高级请求"""
pass
@@ -129,8 +159,7 @@ class BaseAdapter(ABC):
model: "LLMModel",
api_key: str,
texts: list[str],
task_type: "EmbeddingTaskType | str",
**kwargs: Any,
config: "LLMEmbeddingConfig",
) -> RequestData:
"""准备文本嵌入请求"""
pass
@@ -142,9 +171,16 @@ class BaseAdapter(ABC):
"""解析文本嵌入响应"""
pass
@abstractmethod
def convert_generation_config(
self, config: "LLMGenerationConfig", model: "LLMModel"
) -> dict[str, Any]:
"""将通用生成配置转换为特定API的参数字典"""
pass
def validate_embedding_response(self, response_json: dict[str, Any]) -> None:
"""验证嵌入API响应"""
if "error" in response_json:
if response_json.get("error"):
error_info = response_json["error"]
msg = (
error_info.get("message", str(error_info))
@@ -179,158 +215,9 @@ class BaseAdapter(ABC):
)
return headers
def convert_messages_to_openai_format(
self, messages: list["LLMMessage"]
) -> list[dict[str, Any]]:
"""将LLMMessage转换为OpenAI格式 - 通用方法"""
openai_messages: list[dict[str, Any]] = []
for msg in messages:
openai_msg: dict[str, Any] = {"role": msg.role}
if msg.role == "tool":
openai_msg["tool_call_id"] = msg.tool_call_id
openai_msg["name"] = msg.name
openai_msg["content"] = msg.content
else:
if isinstance(msg.content, str):
openai_msg["content"] = msg.content
else:
content_parts = []
for part in msg.content:
if part.type == "text":
content_parts.append({"type": "text", "text": part.text})
elif part.type == "image":
content_parts.append(
{
"type": "image_url",
"image_url": {"url": part.image_source},
}
)
openai_msg["content"] = content_parts
if msg.role == "assistant" and msg.tool_calls:
assistant_tool_calls = []
for call in msg.tool_calls:
assistant_tool_calls.append(
{
"id": call.id,
"type": "function",
"function": {
"name": call.function.name,
"arguments": call.function.arguments,
},
}
)
openai_msg["tool_calls"] = assistant_tool_calls
if msg.name and msg.role != "tool":
openai_msg["name"] = msg.name
openai_messages.append(openai_msg)
return openai_messages
def parse_openai_response(self, response_json: dict[str, Any]) -> ResponseData:
"""解析OpenAI格式的响应 - 通用方法"""
self.validate_response(response_json)
try:
choices = response_json.get("choices", [])
if not choices:
logger.debug("OpenAI响应中没有choices，可能为空回复或流结束。")
return ResponseData(text="", raw_response=response_json)
choice = choices[0]
message = choice.get("message", {})
content = message.get("content", "")
if content:
content = content.strip()
images_bytes: list[bytes] = []
if content and content.startswith("{") and content.endswith("}"):
try:
content_json = json.loads(content)
if "b64_json" in content_json:
images_bytes.append(base64.b64decode(content_json["b64_json"]))
content = "[图片已生成]"
elif "data" in content_json and isinstance(
content_json["data"], str
):
images_bytes.append(base64.b64decode(content_json["data"]))
content = "[图片已生成]"
except (json.JSONDecodeError, KeyError, binascii.Error):
pass
elif (
"images" in message
and isinstance(message["images"], list)
and message["images"]
):
image_info = message["images"][0]
if image_info.get("type") == "image_url":
image_url_obj = image_info.get("image_url", {})
url_str = image_url_obj.get("url", "")
if url_str.startswith("data:image/png;base64,"):
try:
b64_data = url_str.split(",", 1)[1]
images_bytes.append(base64.b64decode(b64_data))
content = content if content else "[图片已生成]"
except (IndexError, binascii.Error) as e:
logger.warning(f"解析OpenRouter Base64图片数据失败: {e}")
parsed_tool_calls: list[LLMToolCall] | None = None
if message_tool_calls := message.get("tool_calls"):
from ..types.models import LLMToolFunction
parsed_tool_calls = []
for tc_data in message_tool_calls:
try:
if tc_data.get("type") == "function":
parsed_tool_calls.append(
LLMToolCall(
id=tc_data["id"],
function=LLMToolFunction(
name=tc_data["function"]["name"],
arguments=tc_data["function"]["arguments"],
),
)
)
except KeyError as e:
logger.warning(
f"解析OpenAI工具调用数据时缺少键: {tc_data}, 错误: {e}"
)
except Exception as e:
logger.warning(
f"解析OpenAI工具调用数据时出错: {tc_data}, 错误: {e}"
)
if not parsed_tool_calls:
parsed_tool_calls = None
final_text = content if content is not None else ""
if not final_text and parsed_tool_calls:
final_text = f"请求调用 {len(parsed_tool_calls)} 个工具。"
usage_info = response_json.get("usage")
return ResponseData(
text=final_text,
tool_calls=parsed_tool_calls,
usage_info=usage_info,
images=images_bytes if images_bytes else None,
raw_response=response_json,
)
except Exception as e:
logger.error(f"解析OpenAI格式响应失败: {e}", e=e)
raise LLMException(
f"解析API响应失败: {e}",
code=LLMErrorCode.RESPONSE_PARSE_ERROR,
cause=e,
)
def validate_response(self, response_json: dict[str, Any]) -> None:
"""验证API响应，解析不同API的错误结构"""
if "error" in response_json:
if response_json.get("error"):
error_info = response_json["error"]
if isinstance(error_info, dict):
@@ -341,12 +228,15 @@ class BaseAdapter(ABC):
error_code_mapping = {
"invalid_api_key": LLMErrorCode.API_KEY_INVALID,
"authentication_failed": LLMErrorCode.API_KEY_INVALID,
"insufficient_quota": LLMErrorCode.API_QUOTA_EXCEEDED,
"rate_limit_exceeded": LLMErrorCode.API_RATE_LIMITED,
"quota_exceeded": LLMErrorCode.API_RATE_LIMITED,
"model_not_found": LLMErrorCode.MODEL_NOT_FOUND,
"invalid_model": LLMErrorCode.MODEL_NOT_FOUND,
"context_length_exceeded": LLMErrorCode.CONTEXT_LENGTH_EXCEEDED,
"max_tokens_exceeded": LLMErrorCode.CONTEXT_LENGTH_EXCEEDED,
"invalid_request_error": LLMErrorCode.INVALID_PARAMETER,
"invalid_parameter": LLMErrorCode.INVALID_PARAMETER,
}
llm_error_code = error_code_mapping.get(
@@ -405,23 +295,12 @@ class BaseAdapter(ABC):
) -> dict[str, Any]:
"""通用的配置应用逻辑"""
if config is not None:
return config.to_api_params(model.api_type, model.model_name)
return self.convert_generation_config(config, model)
if model._generation_config is not None:
return model._generation_config.to_api_params(
model.api_type, model.model_name
)
if model._generation_config:
return self.convert_generation_config(model._generation_config, model)
base_config = {}
if model.temperature is not None:
base_config["temperature"] = model.temperature
if model.max_tokens is not None:
if model.api_type == "gemini":
base_config["maxOutputTokens"] = model.max_tokens
else:
base_config["max_tokens"] = model.max_tokens
return base_config
return {}
def apply_config_override(
self,
@@ -434,12 +313,96 @@ class BaseAdapter(ABC):
body.update(config_params)
return body
def handle_http_error(self, response: httpx.Response) -> LLMException | None:
"""
处理 HTTP 错误响应。
如果响应状态码表示成功 (200)，返回 None；否则构造 LLMException 供外部捕获。
"""
if response.status_code == 200:
return None
error_text = response.content.decode("utf-8", errors="ignore")
error_status = ""
error_msg = error_text
try:
error_json = json.loads(error_text)
if isinstance(error_json, dict) and "error" in error_json:
error_info = error_json["error"]
if isinstance(error_info, dict):
error_msg = error_info.get("message", error_msg)
raw_status = error_info.get("status") or error_info.get("code")
error_status = str(raw_status) if raw_status is not None else ""
elif error_info is not None:
error_msg = str(error_info)
error_status = error_msg
except Exception:
pass
status_upper = error_status.upper() if error_status else ""
text_upper = error_text.upper()
error_code = LLMErrorCode.API_REQUEST_FAILED
if response.status_code == 400:
if (
"FAILED_PRECONDITION" in status_upper
or "LOCATION IS NOT SUPPORTED" in text_upper
):
error_code = LLMErrorCode.USER_LOCATION_NOT_SUPPORTED
elif "INVALID_ARGUMENT" in status_upper:
error_code = LLMErrorCode.INVALID_PARAMETER
elif "API_KEY_INVALID" in text_upper or "API KEY NOT VALID" in text_upper:
error_code = LLMErrorCode.API_KEY_INVALID
else:
error_code = LLMErrorCode.INVALID_PARAMETER
elif response.status_code in [401, 403]:
if error_msg and (
"country" in error_msg.lower()
or "region" in error_msg.lower()
or "unsupported" in error_msg.lower()
):
error_code = LLMErrorCode.USER_LOCATION_NOT_SUPPORTED
elif "PERMISSION_DENIED" in status_upper:
error_code = LLMErrorCode.API_KEY_INVALID
else:
error_code = LLMErrorCode.API_KEY_INVALID
elif response.status_code == 404:
error_code = LLMErrorCode.MODEL_NOT_FOUND
elif response.status_code == 429:
if (
"RESOURCE_EXHAUSTED" in status_upper
or "INSUFFICIENT_QUOTA" in status_upper
or ("quota" in error_msg.lower() if error_msg else False)
):
error_code = LLMErrorCode.API_QUOTA_EXCEEDED
else:
error_code = LLMErrorCode.API_RATE_LIMITED
elif response.status_code in [402, 413]:
error_code = LLMErrorCode.API_QUOTA_EXCEEDED
elif response.status_code == 422:
error_code = LLMErrorCode.GENERATION_FAILED
elif response.status_code >= 500:
error_code = LLMErrorCode.API_TIMEOUT
return LLMException(
f"HTTP请求失败: {response.status_code} ({error_status or 'Unknown'})",
code=error_code,
details={
"status_code": response.status_code,
"api_status": error_status,
"response": error_text,
},
)
class OpenAICompatAdapter(BaseAdapter):
"""
处理所有 OpenAI 兼容 API 的通用适配器。
"""
@property
def log_sanitization_context(self) -> str:
return "openai_request"
@abstractmethod
def get_chat_endpoint(self, model: "LLMModel") -> str:
"""子类必须实现，返回 chat completions 的端点"""
@@ -481,8 +444,8 @@ class OpenAICompatAdapter(BaseAdapter):
api_key: str,
messages: list["LLMMessage"],
config: "LLMGenerationConfig | None" = None,
tools: dict[str, "ToolExecutable"] | None = None,
tool_choice: str | dict[str, Any] | None = None,
tools: list[Any] | None = None,
tool_choice: "str | dict[str, Any] | ToolChoice | None" = None,
) -> RequestData:
"""准备高级请求 - OpenAI兼容格式"""
url = self.get_api_url(model, self.get_chat_endpoint(model))
@@ -494,28 +457,44 @@ class OpenAICompatAdapter(BaseAdapter):
"X-Title": "Zhenxun Bot",
}
)
openai_messages = self.convert_messages_to_openai_format(messages)
from .components.openai_components import OpenAIMessageConverter
converter = OpenAIMessageConverter()
openai_messages = converter.convert_messages(messages)
body = {
"model": model.model_name,
"messages": openai_messages,
}
openai_tools: list[dict[str, Any]] | None = None
executables: list[Any] = []
if tools:
for tool in tools:
if hasattr(tool, "get_definition"):
executables.append(tool)
if executables:
import asyncio
from zhenxun.utils.pydantic_compat import model_dump
definition_tasks = [
executable.get_definition() for executable in tools.values()
executable.get_definition() for executable in executables
]
openai_tools = await asyncio.gather(*definition_tasks)
if openai_tools:
body["tools"] = [
tool_defs = []
if definition_tasks:
tool_defs = await asyncio.gather(*definition_tasks)
if tool_defs:
openai_tools = [
{"type": "function", "function": model_dump(tool)}
for tool in openai_tools
for tool in tool_defs
]
if openai_tools:
body["tools"] = openai_tools
if tool_choice:
body["tool_choice"] = tool_choice
@@ -528,20 +507,21 @@ class OpenAICompatAdapter(BaseAdapter):
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
"""解析响应 - 直接使用基类的 OpenAI 格式解析"""
"""解析响应 - 直接使用组件化 ResponseParser"""
_ = model, is_advanced
return self.parse_openai_response(response_json)
from .components.openai_components import OpenAIResponseParser
parser = OpenAIResponseParser()
return parser.parse(response_json)
def prepare_embedding_request(
self,
model: "LLMModel",
api_key: str,
texts: list[str],
task_type: "EmbeddingTaskType | str",
**kwargs: Any,
config: "LLMEmbeddingConfig",
) -> RequestData:
"""准备嵌入请求 - OpenAI兼容格式"""
_ = task_type
url = self.get_api_url(model, self.get_embedding_endpoint(model))
headers = self.get_base_headers(api_key)
@@ -550,8 +530,14 @@ class OpenAICompatAdapter(BaseAdapter):
"input": texts,
}
if kwargs:
body.update(kwargs)
if config.output_dimensionality:
body["dimensions"] = config.output_dimensionality
if config.task_type:
body["task"] = config.task_type
if config.encoding_format and config.encoding_format != "float":
body["encoding_format"] = config.encoding_format
return RequestData(url=url, headers=headers, body=body)
zhenxun/services/llm/adapters/components/__init__.py
+1
View File
@@ -0,0 +1 @@
zhenxun/services/llm/adapters/components/gemini_components.py
+606
View File
@@ -0,0 +1,606 @@
import base64
import json
from pathlib import Path
from typing import Any
from zhenxun.services.llm.adapters.base import ResponseData, process_image_data
from zhenxun.services.llm.adapters.components.interfaces import (
ConfigMapper,
MessageConverter,
ResponseParser,
ToolSerializer,
)
from zhenxun.services.llm.config.generation import (
ImageAspectRatio,
LLMGenerationConfig,
ReasoningEffort,
ResponseFormat,
)
from zhenxun.services.llm.config.providers import get_gemini_safety_threshold
from zhenxun.services.llm.types import (
CodeExecutionOutcome,
LLMContentPart,
LLMMessage,
)
from zhenxun.services.llm.types.capabilities import ModelCapabilities
from zhenxun.services.llm.types.exceptions import LLMErrorCode, LLMException
from zhenxun.services.llm.types.models import (
LLMGroundingAttribution,
LLMGroundingMetadata,
LLMToolCall,
LLMToolFunction,
ModelDetail,
ToolDefinition,
)
from zhenxun.services.llm.utils import (
resolve_json_schema_refs,
sanitize_schema_for_llm,
)
from zhenxun.services.log import logger
from zhenxun.utils.http_utils import AsyncHttpx
from zhenxun.utils.pydantic_compat import model_copy, model_dump
class GeminiConfigMapper(ConfigMapper):
def map_config(
self,
config: LLMGenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
params: dict[str, Any] = {}
if config.core:
if config.core.temperature is not None:
params["temperature"] = config.core.temperature
if config.core.max_tokens is not None:
params["maxOutputTokens"] = config.core.max_tokens
if config.core.top_k is not None:
params["topK"] = config.core.top_k
if config.core.top_p is not None:
params["topP"] = config.core.top_p
if config.output:
if config.output.response_format == ResponseFormat.JSON:
params["responseMimeType"] = "application/json"
if config.output.response_schema:
params["responseJsonSchema"] = config.output.response_schema
elif config.output.response_mime_type is not None:
params["responseMimeType"] = config.output.response_mime_type
if (
config.output.response_schema is not None
and "responseJsonSchema" not in params
):
params["responseJsonSchema"] = config.output.response_schema
if config.output.response_modalities:
params["responseModalities"] = config.output.response_modalities
if config.tool_config:
fc_config: dict[str, Any] = {"mode": config.tool_config.mode}
if (
config.tool_config.allowed_function_names
and config.tool_config.mode == "ANY"
):
builtins = {"code_execution", "google_search", "google_map"}
user_funcs = [
name
for name in config.tool_config.allowed_function_names
if name not in builtins
]
if user_funcs:
fc_config["allowedFunctionNames"] = user_funcs
params["toolConfig"] = {"functionCallingConfig": fc_config}
if config.reasoning:
thinking_config = params.setdefault("thinkingConfig", {})
if config.reasoning.budget_tokens is not None:
if (
config.reasoning.budget_tokens <= 0
or config.reasoning.budget_tokens >= 1
):
budget_value = int(config.reasoning.budget_tokens)
else:
budget_value = int(config.reasoning.budget_tokens * 32768)
thinking_config["thinkingBudget"] = budget_value
elif config.reasoning.effort:
if config.reasoning.effort == ReasoningEffort.MEDIUM:
thinking_config["thinkingLevel"] = "HIGH"
else:
thinking_config["thinkingLevel"] = config.reasoning.effort.value
if config.reasoning.show_thoughts is not None:
thinking_config["includeThoughts"] = config.reasoning.show_thoughts
elif capabilities and capabilities.reasoning_visibility == "visible":
thinking_config["includeThoughts"] = True
if config.visual:
image_config: dict[str, Any] = {}
if config.visual.aspect_ratio is not None:
ar_value = (
config.visual.aspect_ratio.value
if isinstance(config.visual.aspect_ratio, ImageAspectRatio)
else config.visual.aspect_ratio
)
image_config["aspectRatio"] = ar_value
if config.visual.resolution:
image_config["imageSize"] = config.visual.resolution
if image_config:
params["imageConfig"] = image_config
if config.visual.media_resolution:
media_value = config.visual.media_resolution.upper()
if not media_value.startswith("MEDIA_RESOLUTION_"):
media_value = f"MEDIA_RESOLUTION_{media_value}"
params["mediaResolution"] = media_value
if config.custom_params:
mapped_custom = config.custom_params.copy()
if "max_tokens" in mapped_custom:
mapped_custom["maxOutputTokens"] = mapped_custom.pop("max_tokens")
if "top_k" in mapped_custom:
mapped_custom["topK"] = mapped_custom.pop("top_k")
if "top_p" in mapped_custom:
mapped_custom["topP"] = mapped_custom.pop("top_p")
for key in (
"code_execution_timeout",
"grounding_config",
"dynamic_threshold",
"user_location",
"reflexion_retries",
):
mapped_custom.pop(key, None)
for unsupported in [
"frequency_penalty",
"presence_penalty",
"repetition_penalty",
]:
if unsupported in mapped_custom:
mapped_custom.pop(unsupported)
params.update(mapped_custom)
safety_settings: list[dict[str, Any]] = []
if config.safety and config.safety.safety_settings:
for category, threshold in config.safety.safety_settings.items():
safety_settings.append({"category": category, "threshold": threshold})
else:
threshold = get_gemini_safety_threshold()
for category in [
"HARM_CATEGORY_HARASSMENT",
"HARM_CATEGORY_HATE_SPEECH",
"HARM_CATEGORY_SEXUALLY_EXPLICIT",
"HARM_CATEGORY_DANGEROUS_CONTENT",
]:
safety_settings.append({"category": category, "threshold": threshold})
if safety_settings:
params["safetySettings"] = safety_settings
return params
class GeminiMessageConverter(MessageConverter):
async def convert_part(self, part: LLMContentPart) -> dict[str, Any]:
"""将单个内容部分转换为 Gemini API 格式"""
def _get_gemini_resolution_dict() -> dict[str, Any]:
if part.media_resolution:
value = part.media_resolution.upper()
if not value.startswith("MEDIA_RESOLUTION_"):
value = f"MEDIA_RESOLUTION_{value}"
return {"media_resolution": {"level": value}}
return {}
if part.type == "text":
return {"text": part.text}
if part.type == "thought":
return {"text": part.thought_text, "thought": True}
if part.type == "image":
if not part.image_source:
raise ValueError("图像类型的内容必须包含image_source")
if part.is_image_base64():
base64_info = part.get_base64_data()
if base64_info:
mime_type, data = base64_info
payload = {"inlineData": {"mimeType": mime_type, "data": data}}
payload.update(_get_gemini_resolution_dict())
return payload
raise ValueError(f"无法解析Base64图像数据: {part.image_source[:50]}...")
if part.is_image_url():
logger.debug(f"正在为Gemini下载并编码URL图片: {part.image_source}")
try:
image_bytes = await AsyncHttpx.get_content(part.image_source)
mime_type = part.mime_type or "image/jpeg"
base64_data = base64.b64encode(image_bytes).decode("utf-8")
payload = {
"inlineData": {"mimeType": mime_type, "data": base64_data}
}
payload.update(_get_gemini_resolution_dict())
return payload
except Exception as e:
logger.error(f"下载或编码URL图片失败: {e}", e=e)
raise ValueError(f"无法处理图片URL: {e}")
raise ValueError(f"不支持的图像源格式: {part.image_source[:50]}...")
if part.type == "video":
if not part.video_source:
raise ValueError("视频类型的内容必须包含video_source")
if part.video_source.startswith("data:"):
try:
header, data = part.video_source.split(",", 1)
mime_type = header.split(";")[0].replace("data:", "")
payload = {"inlineData": {"mimeType": mime_type, "data": data}}
payload.update(_get_gemini_resolution_dict())
return payload
except (ValueError, IndexError):
raise ValueError(
f"无法解析Base64视频数据: {part.video_source[:50]}..."
)
raise ValueError(
"Gemini API 的视频处理需要通过 File API 上传，不支持直接 URL"
)
if part.type == "audio":
if not part.audio_source:
raise ValueError("音频类型的内容必须包含audio_source")
if part.audio_source.startswith("data:"):
try:
header, data = part.audio_source.split(",", 1)
mime_type = header.split(";")[0].replace("data:", "")
payload = {"inlineData": {"mimeType": mime_type, "data": data}}
payload.update(_get_gemini_resolution_dict())
return payload
except (ValueError, IndexError):
raise ValueError(
f"无法解析Base64音频数据: {part.audio_source[:50]}..."
)
raise ValueError(
"Gemini API 的音频处理需要通过 File API 上传，不支持直接 URL"
)
if part.type == "file":
if part.file_uri:
payload = {
"fileData": {"mimeType": part.mime_type, "fileUri": part.file_uri}
}
payload.update(_get_gemini_resolution_dict())
return payload
if part.file_source:
file_name = (
part.metadata.get("name", "file") if part.metadata else "file"
)
return {"text": f"[文件: {file_name}]\n{part.file_source}"}
raise ValueError("文件类型的内容必须包含file_uri或file_source")
raise ValueError(f"不支持的内容类型: {part.type}")
async def convert_messages_async(
self, messages: list[LLMMessage]
) -> list[dict[str, Any]]:
gemini_contents: list[dict[str, Any]] = []
for msg in messages:
current_parts: list[dict[str, Any]] = []
if msg.role == "system":
continue
elif msg.role == "user":
if isinstance(msg.content, str):
current_parts.append({"text": msg.content})
elif isinstance(msg.content, list):
for part_obj in msg.content:
current_parts.append(await self.convert_part(part_obj))
gemini_contents.append({"role": "user", "parts": current_parts})
elif msg.role == "assistant" or msg.role == "model":
if isinstance(msg.content, str) and msg.content:
current_parts.append({"text": msg.content})
elif isinstance(msg.content, list):
for part_obj in msg.content:
part_dict = await self.convert_part(part_obj)
if "executableCode" in part_dict:
part_dict["executable_code"] = part_dict.pop(
"executableCode"
)
if "codeExecutionResult" in part_dict:
part_dict["code_execution_result"] = part_dict.pop(
"codeExecutionResult"
)
if (
part_obj.metadata
and "thought_signature" in part_obj.metadata
):
part_dict["thoughtSignature"] = part_obj.metadata[
"thought_signature"
]
current_parts.append(part_dict)
if msg.tool_calls:
for call in msg.tool_calls:
fc_part = {
"functionCall": {
"name": call.function.name,
"args": json.loads(call.function.arguments),
}
}
if call.thought_signature:
fc_part["thoughtSignature"] = call.thought_signature
current_parts.append(fc_part)
if current_parts:
gemini_contents.append({"role": "model", "parts": current_parts})
elif msg.role == "tool":
if not msg.name:
raise ValueError("Gemini 工具消息必须包含 'name' 字段（函数名）。")
try:
content_str = (
msg.content
if isinstance(msg.content, str)
else str(msg.content)
)
tool_result_obj = json.loads(content_str)
except json.JSONDecodeError:
content_str = (
msg.content
if isinstance(msg.content, str)
else str(msg.content)
)
tool_result_obj = {"raw_output": content_str}
if isinstance(tool_result_obj, list):
final_response_payload = {"result": tool_result_obj}
elif not isinstance(tool_result_obj, dict):
final_response_payload = {"result": tool_result_obj}
else:
final_response_payload = tool_result_obj
current_parts.append(
{
"functionResponse": {
"name": msg.name,
"response": final_response_payload,
}
}
)
if gemini_contents and gemini_contents[-1]["role"] == "function":
gemini_contents[-1]["parts"].extend(current_parts)
else:
gemini_contents.append({"role": "function", "parts": current_parts})
return gemini_contents
def convert_messages(self, messages: list[LLMMessage]) -> list[dict[str, Any]]:
raise NotImplementedError("Use convert_messages_async for Gemini")
class GeminiToolSerializer(ToolSerializer):
def serialize_tools(self, tools: list[ToolDefinition]) -> list[dict[str, Any]]:
function_declarations: list[dict[str, Any]] = []
for tool_def in tools:
tool_copy = model_copy(tool_def)
tool_copy.parameters = resolve_json_schema_refs(tool_copy.parameters)
tool_copy.parameters = sanitize_schema_for_llm(
tool_copy.parameters, api_type="gemini"
)
function_declarations.append(model_dump(tool_copy))
return function_declarations
class GeminiResponseParser(ResponseParser):
def validate_response(self, response_json: dict[str, Any]) -> None:
if error := response_json.get("error"):
code = error.get("code")
message = error.get("message", "")
status = error.get("status")
details = error.get("details", [])
if code == 429 or status == "RESOURCE_EXHAUSTED":
is_quota = any(
d.get("reason") in ("QUOTA_EXCEEDED", "SERVICE_DISABLED")
for d in details
if isinstance(d, dict)
)
if is_quota or "quota" in message.lower():
raise LLMException(
f"Gemini配额耗尽: {message}",
code=LLMErrorCode.API_QUOTA_EXCEEDED,
details=error,
)
raise LLMException(
f"Gemini速率限制: {message}",
code=LLMErrorCode.API_RATE_LIMITED,
details=error,
)
if code == 400 or status in ("INVALID_ARGUMENT", "FAILED_PRECONDITION"):
raise LLMException(
f"Gemini参数错误: {message}",
code=LLMErrorCode.INVALID_PARAMETER,
details=error,
recoverable=False,
)
if prompt_feedback := response_json.get("promptFeedback"):
if block_reason := prompt_feedback.get("blockReason"):
raise LLMException(
f"内容被安全过滤: {block_reason}",
code=LLMErrorCode.CONTENT_FILTERED,
details={
"block_reason": block_reason,
"safety_ratings": prompt_feedback.get("safetyRatings"),
},
)
def parse(self, response_json: dict[str, Any]) -> ResponseData:
self.validate_response(response_json)
if "image_generation" in response_json and isinstance(
response_json["image_generation"], dict
):
candidates_source = response_json["image_generation"]
else:
candidates_source = response_json
candidates = candidates_source.get("candidates", [])
usage_info = response_json.get("usageMetadata")
if not candidates:
return ResponseData(text="", raw_response=response_json)
candidate = candidates[0]
thought_signature: str | None = None
content_data = candidate.get("content", {})
parts = content_data.get("parts", [])
text_content = ""
images_payload: list[bytes | Path] = []
parsed_tool_calls: list[LLMToolCall] | None = None
parsed_code_executions: list[dict[str, Any]] = []
content_parts: list[LLMContentPart] = []
thought_summary_parts: list[str] = []
answer_parts = []
for part in parts:
part_signature = part.get("thoughtSignature")
if part_signature and thought_signature is None:
thought_signature = part_signature
part_metadata: dict[str, Any] | None = None
if part_signature:
part_metadata = {"thought_signature": part_signature}
if part.get("thought") is True:
t_text = part.get("text", "")
thought_summary_parts.append(t_text)
content_parts.append(LLMContentPart.thought_part(t_text))
elif "text" in part:
answer_parts.append(part["text"])
c_part = LLMContentPart(
type="text", text=part["text"], metadata=part_metadata
)
content_parts.append(c_part)
elif "thoughtSummary" in part:
thought_summary_parts.append(part["thoughtSummary"])
content_parts.append(
LLMContentPart.thought_part(part["thoughtSummary"])
)
elif "inlineData" in part:
inline_data = part["inlineData"]
if "data" in inline_data:
decoded = base64.b64decode(inline_data["data"])
images_payload.append(process_image_data(decoded))
elif "functionCall" in part:
if parsed_tool_calls is None:
parsed_tool_calls = []
fc_data = part["functionCall"]
fc_sig = part_signature
try:
call_id = f"call_gemini_{len(parsed_tool_calls)}"
parsed_tool_calls.append(
LLMToolCall(
id=call_id,
thought_signature=fc_sig,
function=LLMToolFunction(
name=fc_data["name"],
arguments=json.dumps(fc_data["args"]),
),
)
)
except Exception as e:
logger.warning(
f"解析Gemini functionCall时出错: {fc_data}, 错误: {e}"
)
elif "executableCode" in part:
exec_code = part["executableCode"]
lang = exec_code.get("language", "PYTHON")
code = exec_code.get("code", "")
content_parts.append(LLMContentPart.executable_code_part(lang, code))
answer_parts.append(f"\n[生成代码 ({lang})]:\n```python\n{code}\n```\n")
elif "codeExecutionResult" in part:
result = part["codeExecutionResult"]
outcome = result.get("outcome", CodeExecutionOutcome.OUTCOME_UNKNOWN)
output = result.get("output", "")
content_parts.append(
LLMContentPart.execution_result_part(outcome, output)
)
parsed_code_executions.append(result)
if outcome == CodeExecutionOutcome.OUTCOME_OK:
answer_parts.append(f"\n[代码执行结果]:\n```\n{output}\n```\n")
else:
answer_parts.append(f"\n[代码执行失败 ({outcome})]:\n{output}\n")
full_answer = "".join(answer_parts).strip()
text_content = full_answer
final_thought_text = (
"\n\n".join(thought_summary_parts).strip()
if thought_summary_parts
else None
)
grounding_metadata_obj = None
if grounding_data := candidate.get("groundingMetadata"):
try:
sep_content = None
sep_field = grounding_data.get("searchEntryPoint")
if isinstance(sep_field, dict):
sep_content = sep_field.get("renderedContent")
attributions = []
if chunks := grounding_data.get("groundingChunks"):
for chunk in chunks:
if web := chunk.get("web"):
attributions.append(
LLMGroundingAttribution(
title=web.get("title"),
uri=web.get("uri"),
snippet=web.get("snippet"),
confidence_score=None,
)
)
grounding_metadata_obj = LLMGroundingMetadata(
web_search_queries=grounding_data.get("webSearchQueries"),
grounding_attributions=attributions or None,
search_suggestions=grounding_data.get("searchSuggestions"),
search_entry_point=sep_content,
map_widget_token=grounding_data.get("googleMapsWidgetContextToken"),
)
except Exception as e:
logger.warning(f"无法解析Grounding元数据: {grounding_data}, {e}")
return ResponseData(
text=text_content,
tool_calls=parsed_tool_calls,
code_executions=parsed_code_executions if parsed_code_executions else None,
content_parts=content_parts if content_parts else None,
images=images_payload if images_payload else None,
usage_info=usage_info,
raw_response=response_json,
grounding_metadata=grounding_metadata_obj,
thought_text=final_thought_text,
thought_signature=thought_signature,
)
zhenxun/services/llm/adapters/components/interfaces.py
+43
View File
@@ -0,0 +1,43 @@
from abc import ABC, abstractmethod
from typing import Any
from zhenxun.services.llm.adapters.base import ResponseData
from zhenxun.services.llm.config.generation import LLMGenerationConfig
from zhenxun.services.llm.types import LLMMessage
from zhenxun.services.llm.types.capabilities import ModelCapabilities
from zhenxun.services.llm.types.models import ModelDetail, ToolDefinition
class ConfigMapper(ABC):
@abstractmethod
def map_config(
self,
config: LLMGenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
"""将通用生成配置转换为特定 API 的参数字典"""
...
class MessageConverter(ABC):
@abstractmethod
def convert_messages(
self, messages: list[LLMMessage]
) -> list[dict[str, Any]] | dict[str, Any]:
"""将通用消息列表转换为特定 API 的消息格式"""
...
class ToolSerializer(ABC):
@abstractmethod
def serialize_tools(self, tools: list[ToolDefinition]) -> Any:
"""将通用工具定义转换为特定 API 的工具格式"""
...
class ResponseParser(ABC):
@abstractmethod
def parse(self, response_json: dict[str, Any]) -> ResponseData:
"""将特定 API 的响应解析为通用响应数据"""
...
zhenxun/services/llm/adapters/components/openai_components.py
+347
View File
@@ -0,0 +1,347 @@
import base64
import binascii
import json
from pathlib import Path
from typing import Any
from zhenxun.services.llm.adapters.base import ResponseData, process_image_data
from zhenxun.services.llm.adapters.components.interfaces import (
ConfigMapper,
MessageConverter,
ResponseParser,
ToolSerializer,
)
from zhenxun.services.llm.config.generation import (
ImageAspectRatio,
LLMGenerationConfig,
ResponseFormat,
StructuredOutputStrategy,
)
from zhenxun.services.llm.types import LLMMessage
from zhenxun.services.llm.types.capabilities import ModelCapabilities
from zhenxun.services.llm.types.exceptions import LLMErrorCode, LLMException
from zhenxun.services.llm.types.models import (
LLMToolCall,
LLMToolFunction,
ModelDetail,
ToolDefinition,
)
from zhenxun.services.llm.utils import sanitize_schema_for_llm
from zhenxun.services.log import logger
from zhenxun.utils.pydantic_compat import model_dump
class OpenAIConfigMapper(ConfigMapper):
def __init__(self, api_type: str = "openai"):
self.api_type = api_type
def map_config(
self,
config: LLMGenerationConfig,
model_detail: ModelDetail | None = None,
capabilities: ModelCapabilities | None = None,
) -> dict[str, Any]:
params: dict[str, Any] = {}
strategy = config.output.structured_output_strategy if config.output else None
if strategy is None:
strategy = (
StructuredOutputStrategy.TOOL_CALL
if self.api_type == "deepseek"
else StructuredOutputStrategy.NATIVE
)
if config.core:
if config.core.temperature is not None:
params["temperature"] = config.core.temperature
if config.core.max_tokens is not None:
params["max_tokens"] = config.core.max_tokens
if config.core.top_k is not None:
params["top_k"] = config.core.top_k
if config.core.top_p is not None:
params["top_p"] = config.core.top_p
if config.core.frequency_penalty is not None:
params["frequency_penalty"] = config.core.frequency_penalty
if config.core.presence_penalty is not None:
params["presence_penalty"] = config.core.presence_penalty
if config.core.stop is not None:
params["stop"] = config.core.stop
if config.core.repetition_penalty is not None:
if self.api_type == "openai":
logger.warning("OpenAI官方API不支持repetition_penalty参数，已忽略")
else:
params["repetition_penalty"] = config.core.repetition_penalty
if config.reasoning and config.reasoning.effort:
params["reasoning_effort"] = config.reasoning.effort.value.lower()
if config.output:
if isinstance(config.output.response_format, dict):
params["response_format"] = config.output.response_format
elif (
config.output.response_format == ResponseFormat.JSON
and strategy == StructuredOutputStrategy.NATIVE
):
if config.output.response_schema:
sanitized = sanitize_schema_for_llm(
config.output.response_schema, api_type="openai"
)
params["response_format"] = {
"type": "json_schema",
"json_schema": {
"name": "structured_response",
"schema": sanitized,
"strict": True,
},
}
else:
params["response_format"] = {"type": "json_object"}
if config.tool_config:
mode = config.tool_config.mode
if mode == "NONE":
params["tool_choice"] = "none"
elif mode == "AUTO":
params["tool_choice"] = "auto"
elif mode == "ANY":
params["tool_choice"] = "required"
if config.visual and config.visual.aspect_ratio:
size_map = {
ImageAspectRatio.SQUARE: "1024x1024",
ImageAspectRatio.LANDSCAPE_16_9: "1792x1024",
ImageAspectRatio.PORTRAIT_9_16: "1024x1792",
}
ar = config.visual.aspect_ratio
if isinstance(ar, ImageAspectRatio):
mapped_size = size_map.get(ar)
if mapped_size:
params["size"] = mapped_size
elif isinstance(ar, str):
params["size"] = ar
if config.custom_params:
mapped_custom = config.custom_params.copy()
if "repetition_penalty" in mapped_custom and self.api_type == "openai":
mapped_custom.pop("repetition_penalty")
if "stop" in mapped_custom:
stop_value = mapped_custom["stop"]
if isinstance(stop_value, str):
mapped_custom["stop"] = [stop_value]
params.update(mapped_custom)
return params
class OpenAIMessageConverter(MessageConverter):
def convert_messages(self, messages: list[LLMMessage]) -> list[dict[str, Any]]:
openai_messages: list[dict[str, Any]] = []
for msg in messages:
openai_msg: dict[str, Any] = {"role": msg.role}
if msg.role == "tool":
openai_msg["tool_call_id"] = msg.tool_call_id
openai_msg["name"] = msg.name
openai_msg["content"] = msg.content
else:
if isinstance(msg.content, str):
openai_msg["content"] = msg.content
else:
content_parts = []
for part in msg.content:
if part.type == "text":
content_parts.append({"type": "text", "text": part.text})
elif part.type == "image":
content_parts.append(
{
"type": "image_url",
"image_url": {"url": part.image_source},
}
)
openai_msg["content"] = content_parts
if msg.role == "assistant" and msg.tool_calls:
assistant_tool_calls = []
for call in msg.tool_calls:
assistant_tool_calls.append(
{
"id": call.id,
"type": "function",
"function": {
"name": call.function.name,
"arguments": call.function.arguments,
},
}
)
openai_msg["tool_calls"] = assistant_tool_calls
if msg.name and msg.role != "tool":
openai_msg["name"] = msg.name
openai_messages.append(openai_msg)
return openai_messages
class OpenAIToolSerializer(ToolSerializer):
def serialize_tools(
self, tools: list[ToolDefinition]
) -> list[dict[str, Any]] | None:
if not tools:
return None
openai_tools = []
for tool in tools:
tool_dict = model_dump(tool)
parameters = tool_dict.get("parameters")
if parameters:
tool_dict["parameters"] = sanitize_schema_for_llm(
parameters, api_type="openai"
)
tool_dict["strict"] = True
openai_tools.append({"type": "function", "function": tool_dict})
return openai_tools
class OpenAIResponseParser(ResponseParser):
def validate_response(self, response_json: dict[str, Any]) -> None:
if response_json.get("error"):
error_info = response_json["error"]
if isinstance(error_info, dict):
error_message = error_info.get("message", "未知错误")
error_code = error_info.get("code", "unknown")
error_code_mapping = {
"invalid_api_key": LLMErrorCode.API_KEY_INVALID,
"authentication_failed": LLMErrorCode.API_KEY_INVALID,
"insufficient_quota": LLMErrorCode.API_QUOTA_EXCEEDED,
"rate_limit_exceeded": LLMErrorCode.API_RATE_LIMITED,
"quota_exceeded": LLMErrorCode.API_RATE_LIMITED,
"model_not_found": LLMErrorCode.MODEL_NOT_FOUND,
"invalid_model": LLMErrorCode.MODEL_NOT_FOUND,
"context_length_exceeded": LLMErrorCode.CONTEXT_LENGTH_EXCEEDED,
"max_tokens_exceeded": LLMErrorCode.CONTEXT_LENGTH_EXCEEDED,
"invalid_request_error": LLMErrorCode.INVALID_PARAMETER,
"invalid_parameter": LLMErrorCode.INVALID_PARAMETER,
}
llm_error_code = error_code_mapping.get(
error_code, LLMErrorCode.API_RESPONSE_INVALID
)
else:
error_message = str(error_info)
error_code = "unknown"
llm_error_code = LLMErrorCode.API_RESPONSE_INVALID
raise LLMException(
f"API请求失败: {error_message}",
code=llm_error_code,
details={"api_error": error_info, "error_code": error_code},
)
def parse(self, response_json: dict[str, Any]) -> ResponseData:
self.validate_response(response_json)
choices = response_json.get("choices", [])
if not choices:
return ResponseData(text="", raw_response=response_json)
choice = choices[0]
message = choice.get("message", {})
content = message.get("content", "")
reasoning_content = message.get("reasoning_content", None)
refusal = message.get("refusal")
if refusal:
raise LLMException(
f"模型拒绝生成请求: {refusal}",
code=LLMErrorCode.CONTENT_FILTERED,
details={"refusal": refusal},
recoverable=False,
)
if content:
content = content.strip()
images_payload: list[bytes | Path] = []
if content and content.startswith("{") and content.endswith("}"):
try:
content_json = json.loads(content)
if "b64_json" in content_json:
b64_str = content_json["b64_json"]
if isinstance(b64_str, str) and b64_str.startswith("data:"):
b64_str = b64_str.split(",", 1)[1]
decoded = base64.b64decode(b64_str)
images_payload.append(process_image_data(decoded))
content = "[图片已生成]"
elif "data" in content_json and isinstance(content_json["data"], str):
b64_str = content_json["data"]
if b64_str.startswith("data:"):
b64_str = b64_str.split(",", 1)[1]
decoded = base64.b64decode(b64_str)
images_payload.append(process_image_data(decoded))
content = "[图片已生成]"
except (json.JSONDecodeError, KeyError, binascii.Error):
pass
elif (
"images" in message
and isinstance(message["images"], list)
and message["images"]
):
for image_info in message["images"]:
if image_info.get("type") == "image_url":
image_url_obj = image_info.get("image_url", {})
url_str = image_url_obj.get("url", "")
if url_str.startswith("data:image"):
try:
b64_data = url_str.split(",", 1)[1]
decoded = base64.b64decode(b64_data)
images_payload.append(process_image_data(decoded))
except (IndexError, binascii.Error) as e:
logger.warning(f"解析OpenRouter Base64图片数据失败: {e}")
if images_payload:
content = content if content else "[图片已生成]"
parsed_tool_calls: list[LLMToolCall] | None = None
if message_tool_calls := message.get("tool_calls"):
parsed_tool_calls = []
for tc_data in message_tool_calls:
try:
if tc_data.get("type") == "function":
parsed_tool_calls.append(
LLMToolCall(
id=tc_data["id"],
function=LLMToolFunction(
name=tc_data["function"]["name"],
arguments=tc_data["function"]["arguments"],
),
)
)
except KeyError as e:
logger.warning(
f"解析OpenAI工具调用数据时缺少键: {tc_data}, 错误: {e}"
)
except Exception as e:
logger.warning(
f"解析OpenAI工具调用数据时出错: {tc_data}, 错误: {e}"
)
if not parsed_tool_calls:
parsed_tool_calls = None
final_text = content if content is not None else ""
if not final_text and parsed_tool_calls:
final_text = f"请求调用 {len(parsed_tool_calls)} 个工具。"
usage_info = response_json.get("usage")
return ResponseData(
text=final_text,
tool_calls=parsed_tool_calls,
usage_info=usage_info,
images=images_payload if images_payload else None,
raw_response=response_json,
thought_text=reasoning_content,
)
zhenxun/services/llm/adapters/factory.py
+110 -3
View File
@@ -2,10 +2,17 @@
LLM 适配器工厂类
"""
from typing import ClassVar
import fnmatch
from typing import TYPE_CHECKING, Any, ClassVar
from ..types.exceptions import LLMErrorCode, LLMException
from .base import BaseAdapter
from ..types.models import ToolChoice
from .base import BaseAdapter, RequestData, ResponseData
if TYPE_CHECKING:
from ..config.generation import LLMEmbeddingConfig, LLMGenerationConfig
from ..service import LLMModel
from ..types import LLMMessage
class LLMAdapterFactory:
@@ -21,10 +28,13 @@ class LLMAdapterFactory:
return
from .gemini import GeminiAdapter
from .openai import OpenAIAdapter
from .openai import DeepSeekAdapter, OpenAIAdapter, OpenAIImageAdapter
cls.register_adapter(OpenAIAdapter())
cls.register_adapter(DeepSeekAdapter())
cls.register_adapter(GeminiAdapter())
cls.register_adapter(SmartAdapter())
cls.register_adapter(OpenAIImageAdapter())
@classmethod
def register_adapter(cls, adapter: BaseAdapter) -> None:
@@ -74,3 +84,100 @@ def get_adapter_for_api_type(api_type: str) -> BaseAdapter:
def register_adapter(adapter: BaseAdapter) -> None:
"""注册新的适配器"""
LLMAdapterFactory.register_adapter(adapter)
class SmartAdapter(BaseAdapter):
"""
智能路由适配器。
本身不处理序列化，而是根据规则委托给 OpenAIAdapter 或 GeminiAdapter。
"""
@property
def log_sanitization_context(self) -> str:
return "openai_request"
_ROUTING_RULES: ClassVar[list[tuple[str, str]]] = [
("*nano-banana*", "gemini"),
("*gemini*", "gemini"),
]
_DEFAULT_API_TYPE: ClassVar[str] = "openai"
def __init__(self):
self._adapter_cache: dict[str, BaseAdapter] = {}
@property
def api_type(self) -> str:
return "smart"
@property
def supported_api_types(self) -> list[str]:
return ["smart"]
def _get_delegate_adapter(self, model: "LLMModel") -> BaseAdapter:
"""
核心路由逻辑：决定使用哪个适配器 (带缓存)
"""
if model.model_detail.api_type:
return get_adapter_for_api_type(model.model_detail.api_type)
model_name = model.model_name
if model_name in self._adapter_cache:
return self._adapter_cache[model_name]
target_api_type = self._DEFAULT_API_TYPE
model_name_lower = model_name.lower()
for pattern, api_type in self._ROUTING_RULES:
if fnmatch.fnmatch(model_name_lower, pattern):
target_api_type = api_type
break
adapter = get_adapter_for_api_type(target_api_type)
self._adapter_cache[model_name] = adapter
return adapter
async def prepare_advanced_request(
self,
model: "LLMModel",
api_key: str,
messages: list["LLMMessage"],
config: "LLMGenerationConfig | None" = None,
tools: list[Any] | None = None,
tool_choice: "str | dict[str, Any] | ToolChoice | None" = None,
) -> RequestData:
adapter = self._get_delegate_adapter(model)
return await adapter.prepare_advanced_request(
model, api_key, messages, config, tools, tool_choice
)
def parse_response(
self,
model: "LLMModel",
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
adapter = self._get_delegate_adapter(model)
return adapter.parse_response(model, response_json, is_advanced)
def prepare_embedding_request(
self,
model: "LLMModel",
api_key: str,
texts: list[str],
config: "LLMEmbeddingConfig",
) -> RequestData:
adapter = self._get_delegate_adapter(model)
return adapter.prepare_embedding_request(model, api_key, texts, config)
def parse_embedding_response(
self, response_json: dict[str, Any]
) -> list[list[float]]:
return get_adapter_for_api_type("openai").parse_embedding_response(
response_json
)
def convert_generation_config(
self, config: "LLMGenerationConfig", model: "LLMModel"
) -> dict[str, Any]:
adapter = self._get_delegate_adapter(model)
return adapter.convert_generation_config(config, model)
zhenxun/services/llm/adapters/gemini.py
+151 -430
View File
@@ -2,27 +2,35 @@
Gemini API 适配器
"""
import base64
from typing import TYPE_CHECKING, Any
from zhenxun.services.log import logger
from ..config.generation import ResponseFormat
from ..types import LLMContentPart
from ..types.exceptions import LLMErrorCode, LLMException
from ..utils import sanitize_schema_for_llm
from ..types.models import BasePlatformTool, ToolChoice
from .base import BaseAdapter, RequestData, ResponseData
from .components.gemini_components import (
GeminiConfigMapper,
GeminiMessageConverter,
GeminiResponseParser,
GeminiToolSerializer,
)
if TYPE_CHECKING:
from ..config.generation import LLMGenerationConfig
from ..config.generation import LLMEmbeddingConfig, LLMGenerationConfig
from ..service import LLMModel
from ..types.content import LLMMessage
from ..types.enums import EmbeddingTaskType
from ..types.models import LLMToolCall
from ..types.protocols import ToolExecutable
from ..types import LLMMessage
class GeminiAdapter(BaseAdapter):
"""Gemini API 适配器"""
@property
def log_sanitization_context(self) -> str:
return "gemini_request"
@property
def api_type(self) -> str:
return "gemini"
@@ -47,110 +55,75 @@ class GeminiAdapter(BaseAdapter):
api_key: str,
messages: list["LLMMessage"],
config: "LLMGenerationConfig | None" = None,
tools: dict[str, "ToolExecutable"] | None = None,
tool_choice: str | dict[str, Any] | None = None,
tools: list[Any] | None = None,
tool_choice: str | dict[str, Any] | ToolChoice | None = None,
) -> RequestData:
"""准备高级请求"""
effective_config = config if config is not None else model._generation_config
if tools:
from ..types.models import GeminiUrlContext
context_urls: list[str] = []
for tool in tools:
if isinstance(tool, GeminiUrlContext):
context_urls.extend(tool.urls)
if context_urls and messages:
last_msg = messages[-1]
if last_msg.role == "user":
url_text = "\n\n[Context URLs]:\n" + "\n".join(context_urls)
if isinstance(last_msg.content, str):
last_msg.content += url_text
elif isinstance(last_msg.content, list):
last_msg.content.append(LLMContentPart.text_part(url_text))
has_function_tools = False
if tools:
has_function_tools = any(hasattr(tool, "get_definition") for tool in tools)
is_structured = False
if effective_config and effective_config.output:
if (
effective_config.output.response_schema
or effective_config.output.response_format == ResponseFormat.JSON
or effective_config.output.response_mime_type == "application/json"
):
is_structured = True
if (has_function_tools or is_structured) and effective_config:
if effective_config.reasoning is None:
from ..config.generation import ReasoningConfig
effective_config.reasoning = ReasoningConfig()
if (
effective_config.reasoning.budget_tokens is None
and effective_config.reasoning.effort is None
):
reason_desc = "工具调用" if has_function_tools else "结构化输出"
logger.debug(
f"检测到{reason_desc}，自动为模型 {model.model_name} 开启思维链增强"
)
effective_config.reasoning.budget_tokens = -1
endpoint = self._get_gemini_endpoint(model, effective_config)
url = self.get_api_url(model, endpoint)
headers = self.get_base_headers(api_key)
gemini_contents: list[dict[str, Any]] = []
converter = GeminiMessageConverter()
system_instruction_parts: list[dict[str, Any]] | None = None
for msg in messages:
current_parts: list[dict[str, Any]] = []
if msg.role == "system":
if isinstance(msg.content, str):
system_instruction_parts = [{"text": msg.content}]
elif isinstance(msg.content, list):
system_instruction_parts = [
await part.convert_for_api_async("gemini")
for part in msg.content
await converter.convert_part(part) for part in msg.content
]
continue
elif msg.role == "user":
if isinstance(msg.content, str):
current_parts.append({"text": msg.content})
elif isinstance(msg.content, list):
for part_obj in msg.content:
current_parts.append(
await part_obj.convert_for_api_async("gemini")
)
gemini_contents.append({"role": "user", "parts": current_parts})
elif msg.role == "assistant" or msg.role == "model":
if isinstance(msg.content, str) and msg.content:
current_parts.append({"text": msg.content})
elif isinstance(msg.content, list):
for part_obj in msg.content:
current_parts.append(
await part_obj.convert_for_api_async("gemini")
)
if msg.tool_calls:
import json
for call in msg.tool_calls:
current_parts.append(
{
"functionCall": {
"name": call.function.name,
"args": json.loads(call.function.arguments),
}
}
)
if current_parts:
gemini_contents.append({"role": "model", "parts": current_parts})
elif msg.role == "tool":
if not msg.name:
raise ValueError("Gemini 工具消息必须包含 'name' 字段（函数名）。")
import json
try:
content_str = (
msg.content
if isinstance(msg.content, str)
else str(msg.content)
)
tool_result_obj = json.loads(content_str)
except json.JSONDecodeError:
content_str = (
msg.content
if isinstance(msg.content, str)
else str(msg.content)
)
logger.warning(
f"工具 {msg.name} 的结果不是有效的 JSON: {content_str}. "
f"包装为原始字符串。"
)
tool_result_obj = {"raw_output": content_str}
if isinstance(tool_result_obj, list):
logger.debug(
f"工具 '{msg.name}' 的返回结果是列表，"
f"正在为Gemini API包装为JSON对象。"
)
final_response_payload = {"result": tool_result_obj}
elif not isinstance(tool_result_obj, dict):
final_response_payload = {"result": tool_result_obj}
else:
final_response_payload = tool_result_obj
current_parts.append(
{
"functionResponse": {
"name": msg.name,
"response": final_response_payload,
}
}
)
gemini_contents.append({"role": "function", "parts": current_parts})
gemini_contents = await converter.convert_messages_async(messages)
body: dict[str, Any] = {"contents": gemini_contents}
@@ -158,75 +131,78 @@ class GeminiAdapter(BaseAdapter):
body["systemInstruction"] = {"parts": system_instruction_parts}
all_tools_for_request = []
has_user_functions = False
if tools:
import asyncio
from ..types.protocols import ToolExecutable
from zhenxun.utils.pydantic_compat import model_dump
function_tools: list[ToolExecutable] = []
gemini_tools_dict: dict[str, Any] = {}
definition_tasks = [
executable.get_definition() for executable in tools.values()
]
tool_definitions = await asyncio.gather(*definition_tasks)
for tool in tools:
if isinstance(tool, BasePlatformTool):
declaration = tool.get_tool_declaration()
if declaration:
gemini_tools_dict.update(declaration)
elif hasattr(tool, "get_definition"):
function_tools.append(tool)
function_declarations = []
for tool_def in tool_definitions:
tool_def.parameters = sanitize_schema_for_llm(
tool_def.parameters, api_type="gemini"
)
function_declarations.append(model_dump(tool_def))
if function_tools:
import asyncio
if function_declarations:
all_tools_for_request.append(
{"functionDeclarations": function_declarations}
)
definition_tasks = [
executable.get_definition() for executable in function_tools
]
tool_definitions = await asyncio.gather(*definition_tasks)
if effective_config:
if getattr(effective_config, "enable_grounding", False):
has_explicit_gs_tool = any(
"googleSearch" in tool_item for tool_item in all_tools_for_request
)
if not has_explicit_gs_tool:
all_tools_for_request.append({"googleSearch": {}})
logger.debug("隐式启用 Google Search 工具进行信息来源关联。")
serializer = GeminiToolSerializer()
function_declarations = serializer.serialize_tools(tool_definitions)
if getattr(effective_config, "enable_code_execution", False):
has_explicit_ce_tool = any(
"codeExecution" in tool_item for tool_item in all_tools_for_request
)
if not has_explicit_ce_tool:
all_tools_for_request.append({"codeExecution": {}})
logger.debug("隐式启用代码执行工具。")
if function_declarations:
gemini_tools_dict["functionDeclarations"] = function_declarations
has_user_functions = True
if gemini_tools_dict:
all_tools_for_request.append(gemini_tools_dict)
if all_tools_for_request:
body["tools"] = all_tools_for_request
final_tool_choice = tool_choice
if final_tool_choice is None and effective_config:
final_tool_choice = getattr(effective_config, "tool_choice", None)
tool_config_updates: dict[str, Any] = {}
if (
effective_config
and effective_config.custom_params
and "user_location" in effective_config.custom_params
):
tool_config_updates["retrievalConfig"] = {
"latLng": effective_config.custom_params["user_location"]
}
if final_tool_choice:
if isinstance(final_tool_choice, str):
mode_upper = final_tool_choice.upper()
if mode_upper in ["AUTO", "NONE", "ANY"]:
body["toolConfig"] = {"functionCallingConfig": {"mode": mode_upper}}
else:
body["toolConfig"] = self._convert_tool_choice_to_gemini(
final_tool_choice
)
else:
body["toolConfig"] = self._convert_tool_choice_to_gemini(
final_tool_choice
if tool_config_updates:
body.setdefault("toolConfig", {}).update(tool_config_updates)
converted_params: dict[str, Any] = {}
if effective_config:
converted_params = self.convert_generation_config(effective_config, model)
if converted_params:
if "toolConfig" in converted_params:
tool_config_payload = converted_params.pop("toolConfig")
fc_config = tool_config_payload.get("functionCallingConfig")
should_apply_fc = has_user_functions or (
fc_config and fc_config.get("mode") == "NONE"
)
if should_apply_fc:
body.setdefault("toolConfig", {}).update(tool_config_payload)
elif fc_config and fc_config.get("mode") != "AUTO":
logger.debug(
"Gemini: 忽略针对纯内置工具的 functionCallingConfig (API限制)"
)
final_generation_config = self._build_gemini_generation_config(
model, effective_config
)
if final_generation_config:
body["generationConfig"] = final_generation_config
if "safetySettings" in converted_params:
body["safetySettings"] = converted_params.pop("safetySettings")
safety_settings = self._build_safety_settings(effective_config)
if safety_settings:
body["safetySettings"] = safety_settings
if converted_params:
body["generationConfig"] = converted_params
return RequestData(url=url, headers=headers, body=body)
@@ -242,317 +218,56 @@ class GeminiAdapter(BaseAdapter):
def _get_gemini_endpoint(
self, model: "LLMModel", config: "LLMGenerationConfig | None" = None
) -> str:
"""根据配置选择Gemini API端点"""
if config:
if getattr(config, "enable_code_execution", False):
return f"/v1beta/models/{model.model_name}:generateContent"
if getattr(config, "enable_grounding", False):
return f"/v1beta/models/{model.model_name}:generateContent"
"""返回Gemini generateContent 端点"""
return f"/v1beta/models/{model.model_name}:generateContent"
def _convert_tool_choice_to_gemini(
self, tool_choice_value: str | dict[str, Any]
) -> dict[str, Any]:
"""转换工具选择策略为Gemini格式"""
if isinstance(tool_choice_value, str):
mode_upper = tool_choice_value.upper()
if mode_upper in ["AUTO", "NONE", "ANY"]:
return {"functionCallingConfig": {"mode": mode_upper}}
else:
logger.warning(
f"不支持的 tool_choice 字符串值: '{tool_choice_value}'。"
f"回退到 AUTO。"
)
return {"functionCallingConfig": {"mode": "AUTO"}}
elif isinstance(tool_choice_value, dict):
if (
tool_choice_value.get("type") == "function"
and "function" in tool_choice_value
):
func_name = tool_choice_value["function"].get("name")
if func_name:
return {
"functionCallingConfig": {
"mode": "ANY",
"allowedFunctionNames": [func_name],
}
}
else:
logger.warning(
f"tool_choice dict 中的函数名无效: {tool_choice_value}。"
f"回退到 AUTO。"
)
return {"functionCallingConfig": {"mode": "AUTO"}}
elif "functionCallingConfig" in tool_choice_value:
return {
"functionCallingConfig": tool_choice_value["functionCallingConfig"]
}
else:
logger.warning(
f"不支持的 tool_choice dict 值: {tool_choice_value}。回退到 AUTO。"
)
return {"functionCallingConfig": {"mode": "AUTO"}}
logger.warning(
f"tool_choice 的类型无效: {type(tool_choice_value)}。回退到 AUTO。"
)
return {"functionCallingConfig": {"mode": "AUTO"}}
def _build_gemini_generation_config(
self, model: "LLMModel", config: "LLMGenerationConfig | None" = None
) -> dict[str, Any]:
"""构建Gemini生成配置"""
effective_config = config if config is not None else model._generation_config
if not effective_config:
return {}
generation_config = effective_config.to_api_params(
api_type="gemini", model_name=model.model_name
)
if generation_config:
param_keys = list(generation_config.keys())
logger.debug(
f"构建Gemini生成配置完成，包含 {len(generation_config)} 个参数: "
f"{param_keys}"
)
return generation_config
def _build_safety_settings(
self, config: "LLMGenerationConfig | None" = None
) -> list[dict[str, Any]] | None:
"""构建安全设置"""
if not config:
return None
safety_settings = []
safety_categories = [
"HARM_CATEGORY_HARASSMENT",
"HARM_CATEGORY_HATE_SPEECH",
"HARM_CATEGORY_SEXUALLY_EXPLICIT",
"HARM_CATEGORY_DANGEROUS_CONTENT",
]
custom_safety_settings = getattr(config, "safety_settings", None)
if custom_safety_settings:
for category, threshold in custom_safety_settings.items():
safety_settings.append({"category": category, "threshold": threshold})
else:
from ..config.providers import get_gemini_safety_threshold
threshold = get_gemini_safety_threshold()
for category in safety_categories:
safety_settings.append({"category": category, "threshold": threshold})
return safety_settings if safety_settings else None
def validate_response(self, response_json: dict[str, Any]) -> None:
"""验证 Gemini API 响应，增加对 promptFeedback 的检查"""
super().validate_response(response_json)
if prompt_feedback := response_json.get("promptFeedback"):
if block_reason := prompt_feedback.get("blockReason"):
logger.warning(
f"Gemini 内容因 promptFeedback 被安全过滤: {block_reason}"
)
raise LLMException(
f"内容被安全过滤: {block_reason}",
code=LLMErrorCode.CONTENT_FILTERED,
details={
"block_reason": block_reason,
"safety_ratings": prompt_feedback.get("safetyRatings"),
},
)
def parse_response(
self,
model: "LLMModel",
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
"""解析API响应"""
return self._parse_response(model, response_json, is_advanced)
def _parse_response(
self,
model: "LLMModel",
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
"""解析 Gemini API 响应"""
_ = is_advanced
self.validate_response(response_json)
try:
if "image_generation" in response_json and isinstance(
response_json["image_generation"], dict
):
candidates_source = response_json["image_generation"]
else:
candidates_source = response_json
candidates = candidates_source.get("candidates", [])
usage_info = response_json.get("usageMetadata")
if not candidates:
logger.debug("Gemini响应中没有candidates。")
return ResponseData(text="", raw_response=response_json)
candidate = candidates[0]
if candidate.get("finishReason") in [
"RECITATION",
"OTHER",
] and not candidate.get("content"):
logger.warning(
f"Gemini candidate finished with reason "
f"'{candidate.get('finishReason')}' and no content."
)
return ResponseData(
text="",
raw_response=response_json,
usage_info=response_json.get("usageMetadata"),
)
content_data = candidate.get("content", {})
parts = content_data.get("parts", [])
text_content = ""
images_bytes: list[bytes] = []
parsed_tool_calls: list["LLMToolCall"] | None = None
thought_summary_parts = []
answer_parts = []
for part in parts:
if "text" in part:
answer_parts.append(part["text"])
elif "thought" in part:
thought_summary_parts.append(part["thought"])
elif "thoughtSummary" in part:
thought_summary_parts.append(part["thoughtSummary"])
elif "inlineData" in part:
inline_data = part["inlineData"]
if "data" in inline_data:
images_bytes.append(base64.b64decode(inline_data["data"]))
elif "functionCall" in part:
if parsed_tool_calls is None:
parsed_tool_calls = []
fc_data = part["functionCall"]
try:
import json
from ..types.models import LLMToolCall, LLMToolFunction
call_id = f"call_{model.provider_name}_{len(parsed_tool_calls)}"
parsed_tool_calls.append(
LLMToolCall(
id=call_id,
function=LLMToolFunction(
name=fc_data["name"],
arguments=json.dumps(fc_data["args"]),
),
)
)
except KeyError as e:
logger.warning(
f"解析Gemini functionCall时缺少键: {fc_data}, 错误: {e}"
)
except Exception as e:
logger.warning(
f"解析Gemini functionCall时出错: {fc_data}, 错误: {e}"
)
elif "codeExecutionResult" in part:
result = part["codeExecutionResult"]
if result.get("outcome") == "OK":
output = result.get("output", "")
answer_parts.append(f"\n[代码执行结果]:\n```\n{output}\n```\n")
else:
answer_parts.append(
f"\n[代码执行失败]: {result.get('outcome', 'UNKNOWN')}\n"
)
if thought_summary_parts:
full_thought_summary = "\n".join(thought_summary_parts).strip()
full_answer = "".join(answer_parts).strip()
formatted_parts = []
if full_thought_summary:
formatted_parts.append(f"🤔 **思考过程**\n\n{full_thought_summary}")
if full_answer:
separator = "\n\n---\n\n" if full_thought_summary else ""
formatted_parts.append(f"{separator}✅ **回答**\n\n{full_answer}")
text_content = "".join(formatted_parts)
else:
text_content = "".join(answer_parts)
usage_info = response_json.get("usageMetadata")
grounding_metadata_obj = None
if grounding_data := candidate.get("groundingMetadata"):
try:
from ..types.models import LLMGroundingMetadata
grounding_metadata_obj = LLMGroundingMetadata(**grounding_data)
except Exception as e:
logger.warning(f"无法解析Grounding元数据: {grounding_data}, {e}")
return ResponseData(
text=text_content,
tool_calls=parsed_tool_calls,
images=images_bytes if images_bytes else None,
usage_info=usage_info,
raw_response=response_json,
grounding_metadata=grounding_metadata_obj,
)
except Exception as e:
logger.error(f"解析 Gemini 响应失败: {e}", e=e)
raise LLMException(
f"解析API响应失败: {e}",
code=LLMErrorCode.RESPONSE_PARSE_ERROR,
cause=e,
)
_ = model, is_advanced
parser = GeminiResponseParser()
return parser.parse(response_json)
def prepare_embedding_request(
self,
model: "LLMModel",
api_key: str,
texts: list[str],
task_type: "EmbeddingTaskType | str",
**kwargs: Any,
config: "LLMEmbeddingConfig",
) -> RequestData:
"""准备文本嵌入请求"""
api_model_name = model.model_name
if not api_model_name.startswith("models/"):
api_model_name = f"models/{api_model_name}"
url = self.get_api_url(model, f"/{api_model_name}:batchEmbedContents")
if not model.api_base:
raise LLMException(
f"模型 {model.model_name} 的 api_base 未设置",
code=LLMErrorCode.CONFIGURATION_ERROR,
)
base_url = model.api_base.rstrip("/")
url = f"{base_url}/v1beta/{api_model_name}:batchEmbedContents"
headers = self.get_base_headers(api_key)
requests_payload = []
for text_content in texts:
safe_text = text_content if text_content else " "
request_item: dict[str, Any] = {
"content": {"parts": [{"text": text_content}]},
"model": api_model_name,
"content": {"parts": [{"text": safe_text}]},
}
from ..types.enums import EmbeddingTaskType
if task_type and task_type != EmbeddingTaskType.RETRIEVAL_DOCUMENT:
request_item["task_type"] = str(task_type).upper()
if title := kwargs.get("title"):
request_item["title"] = title
if output_dimensionality := kwargs.get("output_dimensionality"):
request_item["output_dimensionality"] = output_dimensionality
if config.task_type:
request_item["task_type"] = str(config.task_type).upper()
if config.title:
request_item["title"] = config.title
if config.output_dimensionality:
request_item["output_dimensionality"] = config.output_dimensionality
requests_payload.append(request_item)
@@ -601,3 +316,9 @@ class GeminiAdapter(BaseAdapter):
code=LLMErrorCode.RESPONSE_PARSE_ERROR,
details=response_json,
)
def convert_generation_config(
self, config: "LLMGenerationConfig", model: "LLMModel"
) -> dict[str, Any]:
mapper = GeminiConfigMapper()
return mapper.map_config(config, model.model_detail, model.capabilities)
zhenxun/services/llm/adapters/openai.py
+561 -7
View File
@@ -1,15 +1,181 @@
"""
OpenAI API 适配器
支持 OpenAI、DeepSeek、智谱AI 和其他 OpenAI 兼容的 API 服务。
支持 OpenAI、智谱AI 等 OpenAI 兼容的 API 服务。
"""
from typing import TYPE_CHECKING
from abc import ABC, abstractmethod
import base64
from pathlib import Path
from typing import TYPE_CHECKING, Any
from .base import OpenAICompatAdapter
import json_repair
from zhenxun.services.llm.config.generation import ImageAspectRatio
from zhenxun.services.llm.types.exceptions import LLMErrorCode, LLMException
from zhenxun.services.log import logger
from zhenxun.utils.http_utils import AsyncHttpx
from ..types import StructuredOutputStrategy
from ..types.models import ToolChoice
from ..utils import sanitize_schema_for_llm
from .base import (
BaseAdapter,
OpenAICompatAdapter,
RequestData,
ResponseData,
process_image_data,
)
from .components.openai_components import (
OpenAIConfigMapper,
OpenAIMessageConverter,
OpenAIResponseParser,
OpenAIToolSerializer,
)
if TYPE_CHECKING:
from ..config.generation import LLMEmbeddingConfig, LLMGenerationConfig
from ..service import LLMModel
from ..types import LLMMessage
class APIProtocol(ABC):
"""API 协议策略基类"""
@abstractmethod
def build_request_body(
self,
model: "LLMModel",
messages: list["LLMMessage"],
tools: list[dict[str, Any]] | None,
tool_choice: Any,
) -> dict[str, Any]:
"""构建不同协议下的请求体"""
pass
@abstractmethod
def parse_response(self, response_json: dict[str, Any]) -> ResponseData:
"""解析不同协议下的响应"""
pass
class StandardProtocol(APIProtocol):
"""标准 OpenAI 协议策略"""
def __init__(self, adapter: "OpenAICompatAdapter"):
self.adapter = adapter
def build_request_body(
self,
model: "LLMModel",
messages: list["LLMMessage"],
tools: list[dict[str, Any]] | None,
tool_choice: Any,
) -> dict[str, Any]:
converter = OpenAIMessageConverter()
openai_messages = converter.convert_messages(messages)
body: dict[str, Any] = {
"model": model.model_name,
"messages": openai_messages,
}
if tools:
body["tools"] = tools
if tool_choice:
body["tool_choice"] = tool_choice
return body
def parse_response(self, response_json: dict[str, Any]) -> ResponseData:
parser = OpenAIResponseParser()
return parser.parse(response_json)
class ResponsesProtocol(APIProtocol):
"""/v1/responses 新版协议策略"""
def __init__(self, adapter: "OpenAICompatAdapter"):
self.adapter = adapter
def build_request_body(
self,
model: "LLMModel",
messages: list["LLMMessage"],
tools: list[dict[str, Any]] | None,
tool_choice: Any,
) -> dict[str, Any]:
input_items: list[dict[str, Any]] = []
for msg in messages:
role = msg.role
content_list: list[dict[str, Any]] = []
raw_contents = (
msg.content if isinstance(msg.content, list) else [msg.content]
)
for part in raw_contents:
if part is None:
continue
if isinstance(part, str):
content_list.append({"type": "input_text", "text": part})
continue
if hasattr(part, "type"):
part_type = getattr(part, "type", None)
if part_type == "text":
content_list.append(
{"type": "input_text", "text": getattr(part, "text", "")}
)
elif part_type == "image":
content_list.append(
{
"type": "input_image",
"image_url": getattr(part, "image_source", ""),
}
)
continue
if isinstance(part, dict):
part_type = part.get("type")
if part_type == "text":
content_list.append(
{"type": "input_text", "text": part.get("text", "")}
)
elif part_type in {"image", "image_url"}:
image_src = part.get("image_url") or part.get(
"image_source", ""
)
content_list.append(
{
"type": "input_image",
"image_url": image_src,
}
)
input_items.append({"role": role, "content": content_list})
body: dict[str, Any] = {
"model": model.model_name,
"input": input_items,
}
if tools:
body["tools"] = tools
if tool_choice:
body["tool_choice"] = tool_choice
return body
def parse_response(self, response_json: dict[str, Any]) -> ResponseData:
self.adapter.validate_response(response_json)
text_content = ""
for item in response_json.get("output", []):
if item.get("type") == "message" and item.get("role") == "assistant":
for content_item in item.get("content", []):
if content_item.get("type") == "output_text":
text_content += content_item.get("text", "")
return ResponseData(
text=text_content,
usage_info=response_json.get("usage"),
raw_response=response_json,
)
class OpenAIAdapter(OpenAICompatAdapter):
@@ -23,23 +189,411 @@ class OpenAIAdapter(OpenAICompatAdapter):
def supported_api_types(self) -> list[str]:
return [
"openai",
"deepseek",
"zhipu",
"general_openai_compat",
"ark",
"openrouter",
"openai_responses",
]
def get_chat_endpoint(self, model: "LLMModel") -> str:
"""返回聊天完成端点"""
if model.api_type == "ark":
if model.model_detail.endpoint:
return model.model_detail.endpoint
current_api_type = model.model_detail.api_type or model.api_type
if current_api_type == "openai_responses":
return "/v1/responses"
if current_api_type == "ark":
return "/api/v3/chat/completions"
if model.api_type == "zhipu":
if current_api_type == "zhipu":
return "/api/paas/v4/chat/completions"
return "/v1/chat/completions"
def _get_protocol_strategy(self, model: "LLMModel") -> APIProtocol:
"""根据 API 类型获取对应的处理策略"""
current_api_type = model.model_detail.api_type or model.api_type
if current_api_type == "openai_responses":
return ResponsesProtocol(self)
return StandardProtocol(self)
def get_embedding_endpoint(self, model: "LLMModel") -> str:
"""根据API类型返回嵌入端点"""
if model.api_type == "zhipu":
return "/v4/embeddings"
return "/v1/embeddings"
def convert_generation_config(
self, config: "LLMGenerationConfig", model: "LLMModel"
) -> dict[str, Any]:
mapper = OpenAIConfigMapper(api_type=self.api_type)
return mapper.map_config(config, model.model_detail, model.capabilities)
async def prepare_advanced_request(
self,
model: "LLMModel",
api_key: str,
messages: list["LLMMessage"],
config: "LLMGenerationConfig | None" = None,
tools: list[Any] | None = None,
tool_choice: str | dict[str, Any] | ToolChoice | None = None,
) -> "RequestData":
"""根据不同协议策略构建高级请求"""
url = self.get_api_url(model, self.get_chat_endpoint(model))
headers = self.get_base_headers(api_key)
if model.api_type == "openrouter":
headers.update(
{
"HTTP-Referer": "https://github.com/zhenxun-org/zhenxun_bot",
"X-Title": "Zhenxun Bot",
}
)
default_config = getattr(model, "_generation_config", None)
effective_config = config if config is not None else default_config
structured_strategy = (
effective_config.output.structured_output_strategy
if effective_config and effective_config.output
else None
)
if structured_strategy is None:
structured_strategy = StructuredOutputStrategy.NATIVE
openai_tools: list[dict[str, Any]] | None = None
executables: list[Any] = []
if tools:
if isinstance(tools, dict):
executables = list(tools.values())
else:
for tool in tools:
if hasattr(tool, "get_definition"):
executables.append(tool)
definition_tasks = [executable.get_definition() for executable in executables]
tool_defs: list[Any] = []
if definition_tasks:
import asyncio
tool_defs = await asyncio.gather(*definition_tasks)
if tool_defs:
serializer = OpenAIToolSerializer()
openai_tools = serializer.serialize_tools(tool_defs)
final_tool_choice = tool_choice
if final_tool_choice is None:
if (
effective_config
and effective_config.tool_config
and effective_config.tool_config.mode == "ANY"
):
allowed = effective_config.tool_config.allowed_function_names
if allowed:
if len(allowed) == 1:
final_tool_choice = {
"type": "function",
"function": {"name": allowed[0]},
}
else:
logger.warning(
"OpenAI API 不支持多个 allowed_function_names，降级为"
" required。"
)
final_tool_choice = "required"
else:
final_tool_choice = "required"
if (
structured_strategy == StructuredOutputStrategy.TOOL_CALL
and effective_config
and effective_config.output
and effective_config.output.response_schema
):
sanitized_schema = sanitize_schema_for_llm(
effective_config.output.response_schema, api_type="openai"
)
structured_tool = {
"type": "function",
"function": {
"name": "return_structured_response",
"description": "Return the final structured response.",
"parameters": sanitized_schema,
"strict": True if model.api_type != "deepseek" else False,
},
}
if openai_tools is None:
openai_tools = []
openai_tools.append(structured_tool)
final_tool_choice = {
"type": "function",
"function": {"name": "return_structured_response"},
}
protocol_strategy = self._get_protocol_strategy(model)
body = protocol_strategy.build_request_body(
model=model,
messages=messages,
tools=openai_tools,
tool_choice=final_tool_choice,
)
body = self.apply_config_override(model, body, config)
if final_tool_choice is not None:
body["tool_choice"] = final_tool_choice
response_format = body.get("response_format", {})
inject_prompt = (
structured_strategy == StructuredOutputStrategy.NATIVE
and isinstance(response_format, dict)
and response_format.get("type") == "json_object"
)
if inject_prompt:
messages_list = body.get("messages", [])
has_json_keyword = False
for msg in messages_list:
content = msg.get("content")
if isinstance(content, str) and "json" in content.lower():
has_json_keyword = True
break
if isinstance(content, list):
for part in content:
if (
isinstance(part, dict)
and part.get("type") == "text"
and "json" in part.get("text", "").lower()
):
has_json_keyword = True
break
if has_json_keyword:
break
if not has_json_keyword:
injection_text = (
"请务必输出合法的 JSON 格式，避免额外的文本、Markdown 或解释。"
)
system_msg = next(
(m for m in messages_list if m.get("role") == "system"), None
)
if system_msg:
if isinstance(system_msg.get("content"), str):
system_msg["content"] += " " + injection_text
elif isinstance(system_msg.get("content"), list):
system_msg["content"].append(
{"type": "text", "text": injection_text}
)
else:
messages_list.insert(
0, {"role": "system", "content": injection_text}
)
body["messages"] = messages_list
return RequestData(url=url, headers=headers, body=body)
def parse_response(
self,
model: "LLMModel",
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
"""解析响应 - 使用策略模式委托处理"""
_ = is_advanced
protocol_strategy = self._get_protocol_strategy(model)
response_data = protocol_strategy.parse_response(response_json)
if response_data.tool_calls:
target_tool = next(
(
tc
for tc in response_data.tool_calls
if tc.function.name == "return_structured_response"
),
None,
)
if target_tool:
response_data.text = json_repair.repair_json(
target_tool.function.arguments
)
remaining = [
tc
for tc in response_data.tool_calls
if tc.function.name != "return_structured_response"
]
response_data.tool_calls = remaining or None
return response_data
class DeepSeekAdapter(OpenAIAdapter):
"""DeepSeek 专用适配器 (基于 OpenAI 协议)"""
@property
def api_type(self) -> str:
return "deepseek"
@property
def supported_api_types(self) -> list[str]:
return ["deepseek"]
class OpenAIImageAdapter(BaseAdapter):
"""OpenAI 图像生成/编辑适配器"""
@property
def api_type(self) -> str:
return "openai_image"
@property
def log_sanitization_context(self) -> str:
return "openai_request"
@property
def supported_api_types(self) -> list[str]:
return ["openai_image", "nano_banana"]
async def prepare_advanced_request(
self,
model: "LLMModel",
api_key: str,
messages: list["LLMMessage"],
config: "LLMGenerationConfig | None" = None,
tools: list[Any] | None = None,
tool_choice: "str | dict[str, Any] | ToolChoice | None" = None,
) -> RequestData:
_ = tools, tool_choice
effective_config = config if config is not None else model._generation_config
headers = self.get_base_headers(api_key)
prompt = ""
images_bytes_list: list[bytes] = []
for msg in reversed(messages):
if msg.role != "user":
continue
if isinstance(msg.content, str):
prompt = msg.content
elif isinstance(msg.content, list):
for part in msg.content:
if part.type == "text" and not prompt:
prompt = part.text
elif part.type == "image":
if part.is_image_base64():
if b64_data := part.get_base64_data():
_, b64_str = b64_data
images_bytes_list.append(base64.b64decode(b64_str))
elif part.is_image_url() and part.image_source:
images_bytes_list.append(
await AsyncHttpx.get_content(part.image_source)
)
if prompt:
break
if not prompt and not images_bytes_list:
raise LLMException(
"图像生成需要提供 Prompt",
code=LLMErrorCode.CONFIGURATION_ERROR,
)
body: dict[str, Any] = {
"model": model.model_name,
"prompt": prompt,
"response_format": "b64_json",
}
if effective_config:
if effective_config.visual:
if effective_config.visual.aspect_ratio:
ar = effective_config.visual.aspect_ratio
size_map = {
ImageAspectRatio.SQUARE: "1024x1024",
ImageAspectRatio.LANDSCAPE_16_9: "1792x1024",
ImageAspectRatio.PORTRAIT_9_16: "1024x1792",
}
if isinstance(ar, ImageAspectRatio) and ar in size_map:
body["size"] = size_map[ar]
body["aspect_ratio"] = ar.value
elif isinstance(ar, str):
if "x" in ar:
body["size"] = ar
else:
body["aspect_ratio"] = ar
if effective_config.visual.resolution:
res_val = effective_config.visual.resolution
if not isinstance(res_val, str):
res_val = getattr(res_val, "value", res_val)
body["image_size"] = res_val
if effective_config.custom_params:
body.update(effective_config.custom_params)
if images_bytes_list:
b64_images = []
for img_bytes in images_bytes_list:
b64_str = base64.b64encode(img_bytes).decode("utf-8")
b64_images.append(b64_str)
body["image"] = b64_images
endpoint = "/v1/images/generations"
url = self.get_api_url(model, endpoint)
return RequestData(url=url, headers=headers, body=body)
def parse_response(
self,
model: "LLMModel",
response_json: dict[str, Any],
is_advanced: bool = False,
) -> ResponseData:
_ = model, is_advanced
self.validate_response(response_json)
images_data: list[bytes | Path] = []
data_list = response_json.get("data", [])
for item in data_list:
if "b64_json" in item:
try:
b64_str = item["b64_json"]
if b64_str.startswith("data:"):
b64_str = b64_str.split(",", 1)[1]
img = base64.b64decode(b64_str)
images_data.append(process_image_data(img))
except Exception as exc:
logger.error(f"Base64 解码失败: {exc}")
elif "url" in item:
logger.warning(
f"API 返回了 URL 而不是 Base64: {item.get('url', 'unknown')}"
)
text_summary = (
f"已生成 {len(images_data)} 张图片。"
if images_data
else "图像生成接口调用成功，但未解析到图片数据。"
)
return ResponseData(
text=text_summary,
images=images_data if images_data else None,
raw_response=response_json,
)
def prepare_embedding_request(
self,
model: "LLMModel",
api_key: str,
texts: list[str],
config: "LLMEmbeddingConfig",
) -> RequestData:
raise NotImplementedError("OpenAIImageAdapter 不支持 Embedding")
def parse_embedding_response(
self, response_json: dict[str, Any]
) -> list[list[float]]:
raise NotImplementedError("OpenAIImageAdapter 不支持 Embedding")
def convert_generation_config(
self, config: "LLMGenerationConfig", model: "LLMModel"
) -> dict[str, Any]:
_ = config, model
return {}