Diuse/zhenxun_bot
1
0
Fork 0
mirror of https://github.com/zhenxun-org/zhenxun_bot.git synced 2026-10-18 18:50:05 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

🎉🍾♻️refactor(core): 重构核心鉴权与缓存机制，引入异步发送队列以提升性能 🚀 (#2088)

Browse Source 
* 添加测试插件

* ✨ feat(auth): 添加缓存机制以优化用户和插件数据查询性能

* 🗑️ chore(help): 删除笨蛋检测插件代码

* ✨ feat(bot): 添加对扩展插件的加载支持

* ✨ feat(auth): 优化权限检查和缓存机制，增加用户和插件数据的并行查询

* ✨ feat(cache): 引入运行时缓存机制，优化用户和插件的ban记录管理

* ✨ feat(auth): 更新is_ban函数文档，添加参数和返回值说明

* ```
feat(auth): 使用内存缓存优化权限验证性能

- 移除数据库查询超时控制，改用 LevelUserMemoryCache、BotMemoryCache
  和 PluginLimitMemoryCache 进行缓存查询
- 优化 auth_admin、auth_bot、auth_limit 权限验证逻辑，提升响应速度
- 添加 background 参数支持异步发送权限不足提示消息
- 移除 asyncio 依赖，简化代码结构

fix(auth): 修复限制通知频率控制问题

- 实现限制通知冷却机制，避免重复发送相同限制消息
- 添加 AUTH_LIMIT_NOTICE_CD 配置项，默认值为 2 秒
- 使用 FreqLimiter 控制限制通知发送频率

refactor(models): 增强模型数据变更时的缓存同步

- 在 BotConsole、GroupConsole、LevelUser、PluginLimit 模型的
  create、update_or_create、save、delete 方法中自动更新对应缓存
- 确保数据库和内存缓存数据一致性

docs(ban_console): 修正文档注释并优化日志信息

- 修正 BanConsole 类中方法的文档字符串，使用标准参数和返回值格式
- 优化调试日志信息，使描述更加清晰准确
```

* ✨ feat(auth): 更新Limit类以支持PluginLimitSnapshot，优化限制信息处理

* ✨ feat(bot_manage): 优化Bot控制台初始化逻辑，处理IntegrityError异常

* ✨ feat(mmm1): 新增消息推送功能，支持私聊和群聊事件处理

* ✨ feat(group_member_update): 优化群组成员更新逻辑，增加活动跟踪和消息记录功能

* ✨ feat(mmm1): 删除冗余的消息推送功能代码

* ✨ feat(auth): 优化权限检查逻辑，增加模块阻止功能和缓存处理

* ✨ feat(auth): 优化权限检查逻辑，增加快速ban检测和前置检查功能

* ✨ feat(chat_history): 增强消息处理规则，添加时间间隔限制以防止重复消息
✨ feat(data_source): 引入异步获取群成员信息的功能，优化用户信息更新逻辑

* ✨ feat(send_queue): 添加异步发送队列以优化API调用和速率限制

* ✨ feat(auth): 添加缓存就绪检查以优化权限处理逻辑

* ✨ feat(plugins): 移除不必要的插件加载以简化插件管理

* ✨ feat(group_console): 优化群组获取逻辑，添加缓存检查以提升性能
✨ 只接收缓存完成之后时间的消息

* ✨ feat(auth): 添加异步任务管理和超载检测，优化权限处理逻辑
✨ feat(chat_history): 修改规则函数为异步，提升消息处理效率
✨ feat(group_handle): 增加安全获取群组信息的异步方法，添加超时处理
✨ feat(record_request): 引入安全获取群组信息的异步方法，优化群邀请处理
✨ feat(ban_memory_cache): 增强禁言内存缓存，添加负缓存机制
✨ feat(message_load): 新增消息负载检测功能，优化任务调度
✨ feat(scheduler): 在调度器中集成消息压力检测，优化任务执行
✨ feat(send_queue): 引入异步任务管理，优化发送队列处理

* ✨ feat(auth): 添加对 LevelUserSnapshot 和 BotSnapshot 的支持，优化权限检查逻辑
✨ 格式化

* ✨ feat(db_context): 增强 get_or_create 方法，处理并发创建冲突并回退查询已存在记录

* ✨ feat(bot_manage): 增强 init_bot_console 方法，处理并发创建冲突并回退查询已存在的 bot 数据

* 🚨 auto fix by pre-commit hooks

* ✨ feat(runtime_cache): 优化消息处理逻辑，支持 bytes 和 bytearray 类型的联合判断

* ✨ style(runtime_cache): 格式化代码，优化多行表达式的可读性

* 🚨 auto fix by pre-commit hooks

* ✨ refactor: 优化类型注解，改进群组成员更新逻辑和平台处理
✨ 格式化

* 🚨 auto fix by pre-commit hooks

* ✨ refactor: 优化代码格式，增强可读性并修复类型注解

* 🚨 auto fix by pre-commit hooks

* ✨ refactor: 更新文档注释，增强is_ban函数的可读性

* ✨ refactor: 优化代码结构，移除冗余函数，增强可读性并改进任务调度逻辑

* ✨ refactor: 调整定时任务时间，优化渲染服务的初始化逻辑，增强代码可读性

---------

Co-authored-by: ATTomatoo <1126160939@qq.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
ManyManyTomato
2026-01-28 10:15:24 +08:00
committed by GitHub
co-authored by ATTomatoo pre-commit-ci[bot]
parent c9f0a8b9d9
commit 837330e30a
41 changed files with 3656 additions and 763 deletions
Download Patch File Download Diff File Expand all files Collapse all files
zhenxun/models/ban_console.py
+21 -24
View File
@@ -4,6 +4,7 @@ from typing_extensions import Self
from tortoise import fields
from zhenxun.services.cache.runtime_cache import BanMemoryCache
from zhenxun.services.data_access import DataAccess
from zhenxun.services.db_context import Model
from zhenxun.services.log import logger
@@ -42,6 +43,18 @@ class BanConsole(Model):
enable_lock: ClassVar[list[DbLockType]] = [DbLockType.CREATE, DbLockType.UPSERT]
"""开启锁"""
@classmethod
async def create(cls, *args, **kwargs) -> Self:
result = await super().create(*args, **kwargs)
await BanMemoryCache.upsert_from_model(result)
return result
async def delete(self, *args, **kwargs):
user_id = self.user_id
group_id = self.group_id
await super().delete(*args, **kwargs)
await BanMemoryCache.remove(user_id, group_id)
@classmethod
async def _get_data(cls, user_id: str | None, group_id: str | None) -> Self | None:
"""获取数据
@@ -82,14 +95,10 @@ class BanConsole(Model):
返回:
bool: 权限判断，能否unban
"""
user = await cls._get_data(user_id, group_id)
if user:
logger.debug(
f"检测用户被ban等级，user_level: {user.ban_level}，level: {level}",
target=f"{group_id}:{user_id}",
)
return user.ban_level <= level
return False
logger.debug("检测用户被ban等级", target=f"{group_id}:{user_id}")
if not BanMemoryCache.is_loaded():
return False
return BanMemoryCache.check_ban_level(user_id, group_id, level)
@classmethod
async def check_ban_time(
@@ -104,17 +113,9 @@ class BanConsole(Model):
int: ban剩余时长，-1时为永久ban，0表示未被ban
"""
logger.debug("获取用户ban时长", target=f"{group_id}:{user_id}")
user = await cls._get_data(user_id, group_id)
if not user and user_id:
user = await cls._get_data(user_id, None)
if user:
if user.duration == -1:
return -1
_time = time.time() - (user.ban_time + user.duration)
if _time < 0:
return int(abs(_time))
await user.delete()
return 0
if not BanMemoryCache.is_loaded():
return 0
return BanMemoryCache.remaining_time(user_id, group_id)
@classmethod
async def is_ban(cls, user_id: str | None, group_id: str | None = None) -> bool:
@@ -127,11 +128,7 @@ class BanConsole(Model):
bool: 是否被ban
"""
logger.debug("检测是否被ban", target=f"{group_id}:{user_id}")
if await cls.check_ban_time(user_id, group_id):
return True
else:
await cls.unban(user_id, group_id)
return False
return (await cls.check_ban_time(user_id, group_id)) != 0
@classmethod
async def ban(
zhenxun/models/bot_console.py
+24
View File
@@ -2,6 +2,7 @@ from typing import Literal, overload
from tortoise import fields
from zhenxun.services.cache.runtime_cache import BotMemoryCache
from zhenxun.services.db_context import Model
from zhenxun.utils.enum import CacheType
@@ -160,8 +161,10 @@ class BotConsole(Model):
affected_rows = await cls.filter(bot_id=bot_id).update(status=status)
if not affected_rows:
raise ValueError(f"未找到 bot_id: {bot_id}")
await BotMemoryCache.update_status(bot_id, status)
else:
await cls.all().update(status=status)
await BotMemoryCache.refresh()
@overload
@classmethod
@@ -434,6 +437,27 @@ class BotConsole(Model):
bot_data, _ = await cls.get_or_create(bot_id=bot_id)
return cls.format(task_name) in bot_data.block_tasks
@classmethod
async def create(cls, *args, **kwargs):
result = await super().create(*args, **kwargs)
await BotMemoryCache.upsert_from_model(result)
return result
@classmethod
async def update_or_create(cls, *args, **kwargs):
result = await super().update_or_create(*args, **kwargs)
await BotMemoryCache.upsert_from_model(result[0])
return result
async def save(self, *args, **kwargs):
await super().save(*args, **kwargs)
await BotMemoryCache.upsert_from_model(self)
async def delete(self, *args, **kwargs):
bot_id = self.bot_id
await super().delete(*args, **kwargs)
await BotMemoryCache.remove(bot_id)
@classmethod
async def _run_script(cls):
return [
zhenxun/models/group_console.py
+78 -101
View File
@@ -1,4 +1,4 @@
from typing import Any, ClassVar, cast, overload
from typing import TYPE_CHECKING, Any, ClassVar, cast, overload
from typing_extensions import Self
from tortoise import fields
@@ -7,10 +7,14 @@ from tortoise.backends.base.client import BaseDBAsyncClient
from zhenxun.models.plugin_info import PluginInfo
from zhenxun.models.task_info import TaskInfo
from zhenxun.services.cache import CacheRoot
from zhenxun.services.cache.runtime_cache import GroupMemoryCache
from zhenxun.services.data_access import DataAccess
from zhenxun.services.db_context import Model
from zhenxun.utils.enum import CacheType, DbLockType, PluginType
if TYPE_CHECKING:
from zhenxun.services.cache.runtime_cache import GroupSnapshot
def add_disable_marker(name: str) -> str:
"""添加模块禁用标记符
@@ -155,6 +159,7 @@ class GroupConsole(Model):
# 更新缓存
await cls._update_cache(group)
await GroupMemoryCache.upsert_from_model(group)
return group
@@ -210,6 +215,7 @@ class GroupConsole(Model):
# 更新缓存
if is_create:
await cls._update_cache(group)
await GroupMemoryCache.upsert_from_model(group)
return group, is_create
@@ -235,26 +241,37 @@ class GroupConsole(Model):
# 更新缓存
await cls._update_cache(group)
await GroupMemoryCache.upsert_from_model(group)
return group, is_create
async def save(self, *args, **kwargs):
await super().save(*args, **kwargs)
await GroupMemoryCache.upsert_from_model(self)
async def delete(self, *args, **kwargs):
group_id = self.group_id
channel_id = self.channel_id
await super().delete(*args, **kwargs)
await GroupMemoryCache.remove(group_id, channel_id)
@classmethod
async def get_group(
cls,
group_id: str,
channel_id: str | None = None,
clean_duplicates: bool = True,
) -> "GroupSnapshot | None":
return GroupMemoryCache.get_if_ready(group_id, channel_id)
@classmethod
async def get_group_db(
cls,
group_id: str,
channel_id: str | None = None,
clean_duplicates: bool = True,
) -> Self | None:
"""获取群组
参数:
group_id: 群组id
channel_id: 频道id
clean_duplicates: 是否删除重复的记录，仅保留最新的
返回:
Self: GroupConsole
"""
"""获取群组（数据库）"""
dao = DataAccess(cls)
if channel_id:
return await dao.safe_get_or_none(
@@ -270,49 +287,32 @@ class GroupConsole(Model):
@classmethod
async def is_super_group(cls, group_id: str) -> bool:
"""是否超级用户指定群
参数:
group_id: 群组id
返回:
bool: 是否超级用户指定群
"""
return group.is_super if (group := await cls.get_group(group_id)) else False
group = GroupMemoryCache.get_if_ready(group_id, None)
return bool(group and group.is_super)
@classmethod
async def is_superuser_block_plugin(cls, group_id: str, module: str) -> bool:
"""查看群组是否超级用户禁用功能
参数:
group_id: 群组id
module: 模块名称
返回:
bool: 是否禁用被动
"""
return await cls.exists(
group_id=group_id,
superuser_block_plugin__contains=add_disable_marker(module),
group = GroupMemoryCache.get_if_ready(group_id, None)
if not group:
return False
return bool(
group.superuser_block_plugin_set
and module in group.superuser_block_plugin_set
)
@classmethod
async def is_block_plugin(cls, group_id: str, module: str) -> bool:
"""查看群组是否禁用插件
参数:
group_id: 群组id
plugin: 插件名称
返回:
bool: 是否禁用插件
"""
module = add_disable_marker(module)
return await cls.exists(
group_id=group_id, block_plugin__contains=module
) or await cls.exists(
group_id=group_id, superuser_block_plugin__contains=module
)
group = GroupMemoryCache.get_if_ready(group_id, None)
if not group:
return False
if group.block_plugin_set and module in group.block_plugin_set:
return True
if (
group.superuser_block_plugin_set
and module in group.superuser_block_plugin_set
):
return True
return False
@classmethod
async def set_block_plugin(
@@ -396,70 +396,47 @@ class GroupConsole(Model):
async def is_normal_block_plugin(
cls, group_id: str, module: str, channel_id: str | None = None
) -> bool:
"""查看群组是否禁用功能
参数:
group_id: 群组id
module: 模块名称
channel_id: 频道id
返回:
bool: 是否禁用被动
"""
return await cls.exists(
group_id=group_id,
channel_id=channel_id,
block_plugin__contains=f"<{module},",
)
group = GroupMemoryCache.get_if_ready(group_id, channel_id)
if not group:
return False
return bool(group.block_plugin_set and module in group.block_plugin_set)
@classmethod
async def is_superuser_block_task(cls, group_id: str, task: str) -> bool:
"""查看群组是否超级用户禁用被动
参数:
group_id: 群组id
task: 模块名称
返回:
bool: 是否禁用被动
"""
return await cls.exists(
group_id=group_id,
superuser_block_task__contains=add_disable_marker(task),
group = GroupMemoryCache.get_if_ready(group_id, None)
if not group:
return False
return bool(
group.superuser_block_task_set and task in group.superuser_block_task_set
)
@classmethod
async def is_block_task(
cls, group_id: str, task: str, channel_id: str | None = None
) -> bool:
"""查看群组是否禁用被动
参数:
group_id: 群组id
task: 任务模块
channel_id: 频道id
返回:
bool: 是否禁用被动
"""
task = add_disable_marker(task)
if not channel_id:
return await cls.exists(
group_id=group_id,
channel_id__isnull=True,
block_task__contains=task,
) or await cls.exists(
group_id=group_id,
channel_id__isnull=True,
superuser_block_task__contains=task,
)
return await cls.exists(
group_id=group_id, channel_id=channel_id, block_task__contains=task
) or await cls.exists(
group_id=group_id,
channel_id__isnull=True,
superuser_block_task__contains=task,
)
group = GroupMemoryCache.get_if_ready(group_id, None)
if not group:
return False
if group.block_task_set and task in group.block_task_set:
return True
if (
group.superuser_block_task_set
and task in group.superuser_block_task_set
):
return True
return False
group = GroupMemoryCache.get_if_ready(group_id, channel_id)
if group and group.block_task_set and task in group.block_task_set:
return True
super_group = GroupMemoryCache.get_if_ready(group_id, None)
if (
super_group
and super_group.superuser_block_task_set
and task in super_group.superuser_block_task_set
):
return True
return False
@classmethod
async def set_block_task(
zhenxun/models/level_user.py
+23
View File
@@ -1,5 +1,6 @@
from tortoise import fields
from zhenxun.services.cache.runtime_cache import LevelUserMemoryCache
from zhenxun.services.db_context import Model
from zhenxun.utils.enum import CacheType
@@ -124,6 +125,28 @@ class LevelUser(Model):
return user.group_flag == 1
return False
@classmethod
async def create(cls, *args, **kwargs):
result = await super().create(*args, **kwargs)
await LevelUserMemoryCache.upsert_from_model(result)
return result
@classmethod
async def update_or_create(cls, *args, **kwargs):
result = await super().update_or_create(*args, **kwargs)
await LevelUserMemoryCache.upsert_from_model(result[0])
return result
async def save(self, *args, **kwargs):
await super().save(*args, **kwargs)
await LevelUserMemoryCache.upsert_from_model(self)
async def delete(self, *args, **kwargs):
user_id = self.user_id
group_id = self.group_id
await super().delete(*args, **kwargs)
await LevelUserMemoryCache.remove(user_id, group_id)
@classmethod
async def _run_script(cls):
return [
zhenxun/models/plugin_info.py
+1
View File
@@ -109,4 +109,5 @@ class PluginInfo(Model):
"ALTER TABLE plugin_info ADD COLUMN is_show boolean DEFAULT true;",
"ALTER TABLE plugin_info ADD COLUMN ignore_prompt boolean DEFAULT false;",
"ALTER TABLE plugin_info ADD COLUMN impression float DEFAULT 0;",
"CREATE INDEX idx_plugin_info_module ON plugin_info(module);",
]
zhenxun/models/plugin_limit.py
+22
View File
@@ -1,5 +1,6 @@
from tortoise import fields
from zhenxun.services.cache.runtime_cache import PluginLimitMemoryCache
from zhenxun.services.db_context import Model
from zhenxun.utils.enum import LimitCheckType, LimitWatchType, PluginLimitType
@@ -38,3 +39,24 @@ class PluginLimit(Model):
class Meta: # pyright: ignore [reportIncompatibleVariableOverride]
table = "plugin_limit"
table_description = "插件限制"
@classmethod
async def create(cls, *args, **kwargs):
result = await super().create(*args, **kwargs)
await PluginLimitMemoryCache.upsert_from_model(result)
return result
@classmethod
async def update_or_create(cls, *args, **kwargs):
result = await super().update_or_create(*args, **kwargs)
await PluginLimitMemoryCache.upsert_from_model(result[0])
return result
async def save(self, *args, **kwargs):
await super().save(*args, **kwargs)
await PluginLimitMemoryCache.upsert_from_model(self)
async def delete(self, *args, **kwargs):
limit_id = self.id
await super().delete(*args, **kwargs)
await PluginLimitMemoryCache.remove_by_id(limit_id)
zhenxun/models/user_console.py
+39 -28
View File
@@ -1,4 +1,8 @@
import asyncio
from typing import ClassVar
from tortoise import fields
from tortoise.exceptions import IntegrityError
from zhenxun.models.goods_info import GoodsInfo
from zhenxun.services.db_context import Model
@@ -36,6 +40,29 @@ class UserConsole(Model):
cache_key_field = "user_id"
"""缓存键字段"""
_uid_counter: ClassVar[int | None] = None
_uid_lock: ClassVar[asyncio.Lock] = asyncio.Lock()
@classmethod
async def get_or_create_user(
cls, user_id: str, platform: str | None = None
) -> tuple["UserConsole", bool]:
for attempt in range(2):
try:
return await cls.get_or_create(
user_id=user_id,
defaults={"platform": platform, "uid": await cls.get_new_uid()},
)
except IntegrityError:
async with cls._uid_lock:
cls._uid_counter = None
if attempt >= 1:
raise
return await cls.get_or_create(
user_id=user_id,
defaults={"platform": platform, "uid": await cls.get_new_uid()},
)
@classmethod
async def get_user(cls, user_id: str, platform: str | None = None) -> "UserConsole":
"""获取用户
@@ -47,15 +74,8 @@ class UserConsole(Model):
返回:
UserConsole: UserConsole
"""
if not await cls.exists(user_id=user_id):
await cls.create(
user_id=user_id, platform=platform, uid=await cls.get_new_uid()
)
# user, _ = await UserConsole.get_or_create(
# user_id=user_id,
# defaults={"platform": platform, "uid": await cls.get_new_uid()},
# )
return await cls.get(user_id=user_id)
user, _ = await cls.get_or_create_user(user_id=user_id, platform=platform)
return user
@classmethod
async def get_new_uid(cls) -> int:
@@ -64,9 +84,12 @@ class UserConsole(Model):
返回:
int: 最新uid
"""
if user := await cls.annotate().order_by("-uid").first():
return user.uid + 1
return 1
async with cls._uid_lock:
if cls._uid_counter is None:
user = await cls.annotate().order_by("-uid").first()
cls._uid_counter = user.uid if user else 0
cls._uid_counter += 1
return cls._uid_counter
@classmethod
async def add_gold(
@@ -80,10 +103,7 @@ class UserConsole(Model):
source: 来源
platform: 平台.
"""
user, _ = await cls.get_or_create(
user_id=user_id,
defaults={"platform": platform, "uid": await cls.get_new_uid()},
)
user, _ = await cls.get_or_create_user(user_id=user_id, platform=platform)
user.gold += gold
await user.save(update_fields=["gold"])
await UserGoldLog.create(
@@ -111,10 +131,7 @@ class UserConsole(Model):
异常:
InsufficientGold: 金币不足
"""
user, _ = await cls.get_or_create(
user_id=user_id,
defaults={"platform": platform, "uid": await cls.get_new_uid()},
)
user, _ = await cls.get_or_create_user(user_id=user_id, platform=platform)
if user.gold < gold:
raise InsufficientGold()
user.gold -= gold
@@ -135,10 +152,7 @@ class UserConsole(Model):
num: 道具数量.
platform: 平台.
"""
user, _ = await cls.get_or_create(
user_id=user_id,
defaults={"platform": platform, "uid": await cls.get_new_uid()},
)
user, _ = await cls.get_or_create_user(user_id=user_id, platform=platform)
if goods_uuid not in user.props:
user.props[goods_uuid] = 0
user.props[goods_uuid] += num
@@ -172,10 +186,7 @@ class UserConsole(Model):
num: 道具数量.
platform: 平台.
"""
user, _ = await cls.get_or_create(
user_id=user_id,
defaults={"platform": platform, "uid": await cls.get_new_uid()},
)
user, _ = await cls.get_or_create_user(user_id=user_id, platform=platform)
if goods_uuid not in user.props or user.props[goods_uuid] < num:
raise GoodsNotFound("未找到商品或道具数量不足...")