mirror of
https://github.com/zhenxun-org/zhenxun_bot.git
synced 2026-10-18 02:30:08 +08:00
🎉🍾♻️refactor(core): 重构核心鉴权与缓存机制，引入异步发送队列以提升性能 🚀 (#2088)
* 添加测试插件 * ✨ feat(auth): 添加缓存机制以优化用户和插件数据查询性能 * 🗑️ chore(help): 删除笨蛋检测插件代码 * ✨ feat(bot): 添加对扩展插件的加载支持 * ✨ feat(auth): 优化权限检查和缓存机制，增加用户和插件数据的并行查询 * ✨ feat(cache): 引入运行时缓存机制，优化用户和插件的ban记录管理 * ✨ feat(auth): 更新is_ban函数文档，添加参数和返回值说明 * ``` feat(auth): 使用内存缓存优化权限验证性能 - 移除数据库查询超时控制，改用 LevelUserMemoryCache、BotMemoryCache 和 PluginLimitMemoryCache 进行缓存查询 - 优化 auth_admin、auth_bot、auth_limit 权限验证逻辑，提升响应速度 - 添加 background 参数支持异步发送权限不足提示消息 - 移除 asyncio 依赖，简化代码结构 fix(auth): 修复限制通知频率控制问题 - 实现限制通知冷却机制，避免重复发送相同限制消息 - 添加 AUTH_LIMIT_NOTICE_CD 配置项，默认值为 2 秒 - 使用 FreqLimiter 控制限制通知发送频率 refactor(models): 增强模型数据变更时的缓存同步 - 在 BotConsole、GroupConsole、LevelUser、PluginLimit 模型的 create、update_or_create、save、delete 方法中自动更新对应缓存 - 确保数据库和内存缓存数据一致性 docs(ban_console): 修正文档注释并优化日志信息 - 修正 BanConsole 类中方法的文档字符串，使用标准参数和返回值格式 - 优化调试日志信息，使描述更加清晰准确 ``` * ✨ feat(auth): 更新Limit类以支持PluginLimitSnapshot，优化限制信息处理 * ✨ feat(bot_manage): 优化Bot控制台初始化逻辑，处理IntegrityError异常 * ✨ feat(mmm1): 新增消息推送功能，支持私聊和群聊事件处理 * ✨ feat(group_member_update): 优化群组成员更新逻辑，增加活动跟踪和消息记录功能 * ✨ feat(mmm1): 删除冗余的消息推送功能代码 * ✨ feat(auth): 优化权限检查逻辑，增加模块阻止功能和缓存处理 * ✨ feat(auth): 优化权限检查逻辑，增加快速ban检测和前置检查功能 * ✨ feat(chat_history): 增强消息处理规则，添加时间间隔限制以防止重复消息 ✨ feat(data_source): 引入异步获取群成员信息的功能，优化用户信息更新逻辑 * ✨ feat(send_queue): 添加异步发送队列以优化API调用和速率限制 * ✨ feat(auth): 添加缓存就绪检查以优化权限处理逻辑 * ✨ feat(plugins): 移除不必要的插件加载以简化插件管理 * ✨ feat(group_console): 优化群组获取逻辑，添加缓存检查以提升性能 ✨ 只接收缓存完成之后时间的消息 * ✨ feat(auth): 添加异步任务管理和超载检测，优化权限处理逻辑 ✨ feat(chat_history): 修改规则函数为异步，提升消息处理效率 ✨ feat(group_handle): 增加安全获取群组信息的异步方法，添加超时处理 ✨ feat(record_request): 引入安全获取群组信息的异步方法，优化群邀请处理 ✨ feat(ban_memory_cache): 增强禁言内存缓存，添加负缓存机制 ✨ feat(message_load): 新增消息负载检测功能，优化任务调度 ✨ feat(scheduler): 在调度器中集成消息压力检测，优化任务执行 ✨ feat(send_queue): 引入异步任务管理，优化发送队列处理 * ✨ feat(auth): 添加对 LevelUserSnapshot 和 BotSnapshot 的支持，优化权限检查逻辑 ✨ 格式化 * ✨ feat(db_context): 增强 get_or_create 方法，处理并发创建冲突并回退查询已存在记录 * ✨ feat(bot_manage): 增强 init_bot_console 方法，处理并发创建冲突并回退查询已存在的 bot 数据 * 🚨 auto fix by pre-commit hooks * ✨ feat(runtime_cache): 优化消息处理逻辑，支持 bytes 和 bytearray 类型的联合判断 * ✨ style(runtime_cache): 格式化代码，优化多行表达式的可读性 * 🚨 auto fix by pre-commit hooks * ✨ refactor: 优化类型注解，改进群组成员更新逻辑和平台处理 ✨ 格式化 * 🚨 auto fix by pre-commit hooks * ✨ refactor: 优化代码格式，增强可读性并修复类型注解 * 🚨 auto fix by pre-commit hooks * ✨ refactor: 更新文档注释，增强is_ban函数的可读性 * ✨ refactor: 优化代码结构，移除冗余函数，增强可读性并改进任务调度逻辑 * ✨ refactor: 调整定时任务时间，优化渲染服务的初始化逻辑，增强代码可读性 --------- Co-authored-by: ATTomatoo <1126160939@qq.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
ManyManyTomato
co-authored by
ATTomatoo
pre-commit-ci[bot]
ATTomatoo
pre-commit-ci[bot]
parent
c9f0a8b9d9
commit
837330e30a
@@ -1,34 +1,159 @@
|
||||
import asyncio
|
||||
import time
|
||||
|
||||
from nonebot import get_driver
|
||||
from nonebot.adapters import Bot, Event
|
||||
from nonebot.exception import IgnoredException
|
||||
from nonebot.matcher import Matcher
|
||||
from nonebot.message import run_postprocessor, run_preprocessor
|
||||
from nonebot.message import event_preprocessor, run_postprocessor, run_preprocessor
|
||||
from nonebot_plugin_alconna import UniMsg
|
||||
from nonebot_plugin_uninfo import Uninfo
|
||||
|
||||
from zhenxun.services.cache.runtime_cache import is_cache_ready
|
||||
from zhenxun.services.log import logger
|
||||
from zhenxun.services.message_load import is_overloaded, signal_overload
|
||||
from zhenxun.utils.utils import get_entity_ids
|
||||
|
||||
from .auth.config import LOGGER_COMMAND
|
||||
from .auth_checker import LimitManager, auth
|
||||
from .auth.exception import SkipPluginException
|
||||
from .auth_checker import (
|
||||
LimitManager,
|
||||
_get_event_cache,
|
||||
auth,
|
||||
auth_ban_fast,
|
||||
auth_precheck,
|
||||
route_precheck,
|
||||
)
|
||||
|
||||
_SKIP_AUTH_PLUGINS = {"chat_history", "chat_message"}
|
||||
_BOT_CONNECT_TS: float | None = None
|
||||
_AUTH_QUEUE_MAXSIZE = 200
|
||||
_AUTH_QUEUE_HIGH_WATER = 160
|
||||
_AUTH_OVERLOAD_WINDOW = 5.0
|
||||
_AUTH_QUEUE: asyncio.Queue[tuple[Matcher, Event, Bot, Uninfo, UniMsg]] = asyncio.Queue(
|
||||
maxsize=_AUTH_QUEUE_MAXSIZE
|
||||
)
|
||||
_AUTH_QUEUE_STARTED = False
|
||||
_AUTH_WORKERS: list[asyncio.Task] = []
|
||||
_LAST_DROP_LOG = 0.0
|
||||
|
||||
driver = get_driver()
|
||||
|
||||
|
||||
@driver.on_bot_connect
|
||||
async def _mark_bot_connected(bot: Bot):
|
||||
del bot
|
||||
global _BOT_CONNECT_TS
|
||||
_BOT_CONNECT_TS = time.time()
|
||||
|
||||
|
||||
async def _auth_worker(worker_id: int) -> None:
|
||||
while True:
|
||||
matcher, event, bot, session, message = await _AUTH_QUEUE.get()
|
||||
try:
|
||||
await auth(
|
||||
matcher,
|
||||
event,
|
||||
bot,
|
||||
session,
|
||||
message,
|
||||
skip_ban=True,
|
||||
)
|
||||
except IgnoredException:
|
||||
pass
|
||||
except Exception as exc:
|
||||
if not is_overloaded():
|
||||
logger.error("async auth failed", LOGGER_COMMAND, e=exc)
|
||||
finally:
|
||||
_AUTH_QUEUE.task_done()
|
||||
|
||||
|
||||
@driver.on_startup
|
||||
async def _start_auth_queue():
|
||||
global _AUTH_QUEUE_STARTED
|
||||
if _AUTH_QUEUE_STARTED:
|
||||
return
|
||||
_AUTH_QUEUE_STARTED = True
|
||||
worker_count = max(1, min(6, _AUTH_QUEUE_MAXSIZE // 50))
|
||||
for idx in range(worker_count):
|
||||
_AUTH_WORKERS.append(asyncio.create_task(_auth_worker(idx)))
|
||||
|
||||
|
||||
def _skip_auth_for_plugin(matcher: Matcher) -> bool:
|
||||
if not matcher.plugin:
|
||||
return False
|
||||
name = (matcher.plugin.name or "").lower()
|
||||
if name in _SKIP_AUTH_PLUGINS:
|
||||
return True
|
||||
module_name = getattr(matcher.plugin, "module_name", "") or ""
|
||||
return "chat_history" in module_name
|
||||
|
||||
|
||||
@event_preprocessor
|
||||
async def _drop_message_before_cache_ready(event: Event):
|
||||
if event.get_type() != "message":
|
||||
return
|
||||
if not is_cache_ready():
|
||||
raise IgnoredException("cache not ready ignore")
|
||||
if _BOT_CONNECT_TS is not None:
|
||||
event_ts = getattr(event, "time", None)
|
||||
if event_ts is not None and event_ts < _BOT_CONNECT_TS:
|
||||
raise IgnoredException("drop backlog message")
|
||||
|
||||
|
||||
# # 权限检测
|
||||
@run_preprocessor
|
||||
async def _(matcher: Matcher, event: Event, bot: Bot, session: Uninfo, message: UniMsg):
|
||||
async def _auth_preprocessor(
|
||||
matcher: Matcher, event: Event, bot: Bot, session: Uninfo, message: UniMsg
|
||||
):
|
||||
if event.get_type() == "message" and not is_cache_ready():
|
||||
raise IgnoredException("cache not ready ignore")
|
||||
if _skip_auth_for_plugin(matcher):
|
||||
return
|
||||
start_time = time.time()
|
||||
await auth(
|
||||
matcher,
|
||||
event,
|
||||
bot,
|
||||
session,
|
||||
message,
|
||||
)
|
||||
logger.debug(f"权限检测耗时：{time.time() - start_time}秒", LOGGER_COMMAND)
|
||||
entity = get_entity_ids(session)
|
||||
event_cache = _get_event_cache(event, session, entity)
|
||||
if await route_precheck(matcher, event, session, message):
|
||||
return
|
||||
try:
|
||||
await auth_ban_fast(matcher, event, bot, session)
|
||||
except SkipPluginException as exc:
|
||||
logger.info(str(exc), LOGGER_COMMAND, session=session)
|
||||
raise IgnoredException("ban fast ignore") from exc
|
||||
try:
|
||||
await auth_precheck(matcher, event, bot, session, message)
|
||||
except SkipPluginException as exc:
|
||||
logger.info(str(exc), LOGGER_COMMAND, session=session)
|
||||
raise IgnoredException("precheck ignore") from exc
|
||||
|
||||
if event_cache is not None and event_cache.get("route_skip") is True:
|
||||
if not is_overloaded():
|
||||
logger.debug("route miss skip auth task", LOGGER_COMMAND)
|
||||
return
|
||||
|
||||
try:
|
||||
_AUTH_QUEUE.put_nowait((matcher, event, bot, session, message))
|
||||
except asyncio.QueueFull:
|
||||
signal_overload(_AUTH_OVERLOAD_WINDOW)
|
||||
now = time.monotonic()
|
||||
global _LAST_DROP_LOG
|
||||
if now - _LAST_DROP_LOG > 1.0:
|
||||
_LAST_DROP_LOG = now
|
||||
logger.warning("auth queue full, skip auth task", LOGGER_COMMAND)
|
||||
return
|
||||
if _AUTH_QUEUE.qsize() >= _AUTH_QUEUE_HIGH_WATER:
|
||||
signal_overload(_AUTH_OVERLOAD_WINDOW)
|
||||
now = time.monotonic()
|
||||
last_log = getattr(_auth_preprocessor, "_last_log", 0.0)
|
||||
if now - last_log > 1.0 and not is_overloaded():
|
||||
setattr(_auth_preprocessor, "_last_log", now)
|
||||
logger.debug(
|
||||
f"auth check cost: {time.time() - start_time:.3f}s",
|
||||
LOGGER_COMMAND,
|
||||
)
|
||||
|
||||
|
||||
# 解除命令block阻塞
|
||||
@run_postprocessor
|
||||
async def _(matcher: Matcher, session: Uninfo):
|
||||
async def _unblock_after_matcher(matcher: Matcher, session: Uninfo):
|
||||
user_id = session.user.id
|
||||
group_id = None
|
||||
channel_id = None
|
||||
|
||||
Reference in New Issue
Block a user