diff --git a/Moncter/Moncter/.env.example b/Moncter/Moncter/.env.example deleted file mode 100644 index 7ea7304e..00000000 --- a/Moncter/Moncter/.env.example +++ /dev/null @@ -1,24 +0,0 @@ -# 存客宝标签系统 - 环境变量配置示例 -# 复制此文件为 .env 并修改相应的配置值，不要提交到版本控制系统 - -# ============================================ -# 加密配置 -# ============================================ - -# AES 加密密钥（至少32字符，建议使用随机生成的强密钥） -# 生产环境请务必修改此密钥，并妥善保管 -ENCRYPTION_AES_KEY=your-32-byte-secret-key-here-12345678 - -# 哈希盐值（用于身份证哈希，增强安全性） -# 生产环境请务必修改此盐值 -ENCRYPTION_HASH_SALT=your-hash-salt-here-change-in-production - -# ============================================ -# 应用配置 -# ============================================ - -# 应用环境（development/production） -APP_ENV=development - -# 应用调试模式（true/false） -APP_DEBUG=true \ No newline at end of file diff --git a/Moncter/Moncter/.gitignore b/Moncter/Moncter/.gitignore deleted file mode 100644 index 516299c3..00000000 --- a/Moncter/Moncter/.gitignore +++ /dev/null @@ -1,8 +0,0 @@ -/runtime -/.idea -/.vscode -/vendor -*.log -.env -/tests/tmp -/tests/.phpunit.result.cache diff --git a/Moncter/Moncter/LICENSE b/Moncter/Moncter/LICENSE deleted file mode 100644 index 2c662929..00000000 --- a/Moncter/Moncter/LICENSE +++ /dev/null @@ -1,21 +0,0 @@ -MIT License - -Copyright (c) 2021 walkor and contributors (see https://github.com/walkor/webman/contributors) - -Permission is hereby granted, free of charge, to any person obtaining a copy -of this software and associated documentation files (the "Software"), to deal -in the Software without restriction, including without limitation the rights -to use, copy, modify, merge, publish, distribute, sublicense, and/or sell -copies of the Software, and to permit persons to whom the Software is -furnished to do so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE. diff --git a/Moncter/Moncter/MCP/MCP服务器使用说明.md b/Moncter/Moncter/MCP/MCP服务器使用说明.md deleted file mode 100644 index 1dd2aa4d..00000000 --- a/Moncter/Moncter/MCP/MCP服务器使用说明.md +++ /dev/null @@ -1,233 +0,0 @@ -# Moncter MCP 服务器使用说明 - -## 概述 - -Moncter MCP Server 是一个基于 Model Context Protocol (MCP) 的服务器，允许通过 MCP 协议来管理 Moncter 系统的数据采集任务和标签任务。 - -## 安装步骤 - -### 1. 安装依赖 - -```bash -cd MCP/moncter-mcp-server -npm install -``` - -### 2. 编译 TypeScript - -```bash -npm run build -``` - -### 3. 配置 MCP - -编辑 `MCP/mcp.json` 文件，确保 Moncter MCP 服务器配置正确： - -```json -{ - "mcpServers": { - "Moncter": { - "command": "node", - "args": ["./MCP/moncter-mcp-server/dist/index.js"], - "cwd": "E:/Cunkebao/Cunkebao02/Moncter", - "env": { - "MONCTER_API_URL": "http://127.0.0.1:8787" - } - } - } -} -``` - -**注意**：`cwd` 路径需要根据实际项目路径修改。 - -## 可用的 MCP 工具 - -### 1. create_data_collection_task - -创建数据采集任务。 - -**参数**： -- `name` (string, 必需): 任务名称 -- `description` (string, 可选): 任务描述 -- `data_source_id` (string, 必需): 数据源ID -- `database` (string, 必需): 数据库名称 -- `collection` (string, 可选): 集合名称（单集合模式） -- `collections` (array, 可选): 集合列表（多集合模式） -- `mode` (string, 必需): 采集模式（batch/realtime） -- `field_mappings` (array, 可选): 字段映射配置 -- `schedule` (object, 可选): 调度配置 - -**示例**： -```json -{ - "name": "create_data_collection_task", - "arguments": { - "name": "订单数据采集", - "data_source_id": "data_source_id_123", - "database": "KR_商城", - "collection": "21年贝蒂喜订单整合", - "mode": "realtime" - } -} -``` - -### 2. create_tag_task - -创建标签计算任务。 - -**参数**： -- `name` (string, 必需): 任务名称 -- `task_type` (string, 必需): 任务类型（full/incremental/specific） -- `target_tag_ids` (array, 必需): 要计算的标签ID列表 -- `user_scope` (object, 可选): 用户范围配置 -- `schedule` (object, 可选): 调度配置 -- `config` (object, 可选): 高级配置 - -**示例**： -```json -{ - "name": "create_tag_task", - "arguments": { - "name": "高价值用户标签计算", - "task_type": "full", - "target_tag_ids": ["tag_id_1", "tag_id_2"], - "user_scope": { - "type": "all" - } - } -} -``` - -### 3. list_data_collection_tasks - -获取数据采集任务列表。 - -**参数**： -- `page` (number, 可选): 页码 -- `page_size` (number, 可选): 每页数量 - -### 4. list_tag_tasks - -获取标签任务列表。 - -**参数**： -- `page` (number, 可选): 页码 -- `page_size` (number, 可选): 每页数量 - -### 5. get_data_sources - -获取数据源列表。 - -**参数**： -- `type` (string, 可选): 数据源类型筛选 -- `status` (number, 可选): 状态筛选（1=启用，0=禁用） - -### 6. get_tag_definitions - -获取标签定义列表。 - -**参数**： -- `status` (number, 可选): 状态筛选（1=启用，0=禁用） - -### 7. start_data_collection_task - -启动数据采集任务。 - -**参数**： -- `task_id` (string, 必需): 任务ID - -### 8. start_tag_task - -启动标签任务。 - -**参数**： -- `task_id` (string, 必需): 任务ID - -## 环境变量 - -- `MONCTER_API_URL`: 后端API基础URL（默认: http://127.0.0.1:8787） - -## 使用场景 - -### 场景1：通过 AI 助手创建数据采集任务 - -你可以通过支持 MCP 的 AI 助手（如 Claude Desktop）来创建任务： - -1. 告诉 AI："创建一个实时监听的数据采集任务，从数据源 X 的数据库 Y 的集合 Z 采集数据" -2. AI 会调用 `create_data_collection_task` 工具 -3. 任务创建成功后，AI 会告诉你任务ID和状态 - -### 场景2：批量创建标签任务 - -通过 MCP 工具批量创建多个标签计算任务： - -1. 列出所有标签定义：`get_tag_definitions` -2. 为每个标签创建计算任务：`create_tag_task` -3. 启动所有任务：`start_tag_task` - -### 场景3：任务管理 - -通过 MCP 工具查询和管理任务： - -1. 列出所有任务：`list_data_collection_tasks` / `list_tag_tasks` -2. 查看任务详情和状态 -3. 启动/暂停/停止任务 - -## 开发调试 - -### 开发模式 - -```bash -cd MCP/moncter-mcp-server -npm run dev -``` - -### 测试 MCP 服务器 - -可以使用 MCP Inspector 或其他 MCP 客户端工具来测试服务器： - -```bash -# 如果安装了 @modelcontextprotocol/inspector -npx @modelcontextprotocol/inspector node dist/index.js -``` - -## 故障排除 - -### 问题1：服务器无法启动 - -- 检查 Node.js 版本（需要 >= 18） -- 检查是否已安装依赖：`npm install` -- 检查是否已编译：`npm run build` - -### 问题2：无法连接到后端API - -- 检查 `MONCTER_API_URL` 环境变量是否正确 -- 检查后端服务是否运行在指定端口 -- 检查防火墙和网络连接 - -### 问题3：工具调用失败 - -- 检查后端API接口是否正常 -- 检查参数是否正确 -- 查看服务器日志输出 - -## 扩展开发 - -要添加新的 MCP 工具： - -1. 在 `src/index.ts` 的 `ListToolsRequestSchema` handler 中添加新工具定义 -2. 在 `CallToolRequestSchema` handler 中添加工具处理逻辑 -3. 重新编译：`npm run build` -4. 重启 MCP 服务器 - -## 注意事项 - -1. **API URL 配置**：确保 `MONCTER_API_URL` 指向正确的后端服务地址 -2. **路径配置**：`mcp.json` 中的 `cwd` 和 `args` 路径需要根据实际项目路径调整 -3. **权限**：MCP 工具调用会直接操作后端API，请确保权限控制 -4. **错误处理**：工具调用失败时会返回错误信息，请检查返回内容 - ---- - -**更新时间**：2025-01-24 - diff --git a/Moncter/Moncter/MCP/README.md b/Moncter/Moncter/MCP/README.md deleted file mode 100644 index 072190bc..00000000 --- a/Moncter/Moncter/MCP/README.md +++ /dev/null @@ -1,70 +0,0 @@ -# Moncter MCP 集成 - -这个目录包含 Moncter 系统的 MCP (Model Context Protocol) 服务器实现。 - -## 目录结构 - -``` -MCP/ -├── mcp.json # MCP 服务器配置文件（需要配置路径） -├── mcp.json.example # MCP 配置文件示例 -├── moncter-mcp-server/ # Moncter MCP 服务器源代码 -│ ├── src/ -│ │ └── index.ts # 服务器主文件 -│ ├── package.json # Node.js 依赖配置 -│ ├── tsconfig.json # TypeScript 配置 -│ ├── install.sh # Linux/Mac 安装脚本 -│ ├── install.bat # Windows 安装脚本 -│ └── README.md # 服务器详细文档 -├── 快速开始.md # 快速开始指南 -└── MCP服务器使用说明.md # 详细使用说明 -``` - -## 快速开始 - -1. **安装 MCP Server** - - ```bash - cd MCP/moncter-mcp-server - npm install - npm run build - ``` - -2. **配置 MCP 客户端** - - 编辑 `MCP/mcp.json`，将 `YOUR_PROJECT_PATH` 替换为实际的项目路径。 - -3. **启动后端服务** - - ```bash - php start.php start - ``` - -4. **使用 MCP 工具** - - 在支持 MCP 的 AI 客户端（如 Claude Desktop）中使用 Moncter MCP 服务器提供的工具。 - -## 可用的 MCP 工具 - -- `create_data_collection_task` - 创建数据采集任务 -- `create_tag_task` - 创建标签计算任务 -- `list_data_collection_tasks` - 获取数据采集任务列表 -- `list_tag_tasks` - 获取标签任务列表 -- `get_data_sources` - 获取数据源列表 -- `get_tag_definitions` - 获取标签定义列表 -- `start_data_collection_task` - 启动数据采集任务 -- `start_tag_task` - 启动标签任务 - -## 文档 - -- [快速开始指南](./快速开始.md) - 安装和配置步骤 -- [详细使用说明](./MCP服务器使用说明.md) - 完整的工具说明和使用示例 -- [服务器 README](./moncter-mcp-server/README.md) - 服务器开发文档 - -## 注意事项 - -1. 确保 Node.js 版本 >= 18 -2. 确保后端服务运行在配置的端口（默认 8787） -3. 配置文件中的路径需要根据实际情况修改 -4. MCP 工具调用会直接操作后端API，请确保权限控制 - diff --git a/Moncter/Moncter/MCP/mcp.json b/Moncter/Moncter/MCP/mcp.json deleted file mode 100644 index f44cf4b9..00000000 --- a/Moncter/Moncter/MCP/mcp.json +++ /dev/null @@ -1,26 +0,0 @@ -{ - "mcpServers": { - "MongoDB_ckb": { - "command": "npx", - "args": ["-y", "mongodb-mcp-server@1.2.0", "--readOnly"], - "env": { - "MDB_MCP_CONNECTION_STRING": "mongodb://ckb:123456@192.168.1.106:27017/ckb" - } - }, - "MongoDB_KR": { - "command": "npx", - "args": ["-y", "mongodb-mcp-server@1.2.0", "--readOnly"], - "env": { - "MDB_MCP_CONNECTION_STRING": "mongodb://admin:key123456@192.168.2.16:27017/admin" - } - }, - "Moncter": { - "command": "node", - "args": ["./MCP/moncter-mcp-server/dist/index.js"], - "cwd": "E:/Cunkebao/Cunkebao02/Moncter", - "env": { - "MONCTER_API_URL": "http://127.0.0.1:8787" - } - } - } -} \ No newline at end of file diff --git a/Moncter/Moncter/MCP/mcp.json.example b/Moncter/Moncter/MCP/mcp.json.example deleted file mode 100644 index ac79f5af..00000000 --- a/Moncter/Moncter/MCP/mcp.json.example +++ /dev/null @@ -1,27 +0,0 @@ -{ - "mcpServers": { - "MongoDB_ckb": { - "command": "npx", - "args": ["-y", "mongodb-mcp-server@1.2.0", "--readOnly"], - "env": { - "MDB_MCP_CONNECTION_STRING": "mongodb://ckb:123456@192.168.1.106:27017/ckb" - } - }, - "MongoDB_KR": { - "command": "npx", - "args": ["-y", "mongodb-mcp-server@1.2.0", "--readOnly"], - "env": { - "MDB_MCP_CONNECTION_STRING": "mongodb://admin:key123456@192.168.2.16:27017/admin" - } - }, - "Moncter": { - "command": "node", - "args": ["./MCP/moncter-mcp-server/dist/index.js"], - "cwd": "YOUR_PROJECT_PATH", - "env": { - "MONCTER_API_URL": "http://127.0.0.1:8787" - } - } - } -} - diff --git a/Moncter/Moncter/MCP/moncter-mcp-server/.gitignore b/Moncter/Moncter/MCP/moncter-mcp-server/.gitignore deleted file mode 100644 index b365c4e3..00000000 --- a/Moncter/Moncter/MCP/moncter-mcp-server/.gitignore +++ /dev/null @@ -1,7 +0,0 @@ -node_modules/ -dist/ -*.log -.DS_Store -.env -*.tsbuildinfo - diff --git a/Moncter/Moncter/MCP/moncter-mcp-server/MCP接口对比分析.md b/Moncter/Moncter/MCP/moncter-mcp-server/MCP接口对比分析.md deleted file mode 100644 index 2485213a..00000000 --- a/Moncter/Moncter/MCP/moncter-mcp-server/MCP接口对比分析.md +++ /dev/null @@ -1,194 +0,0 @@ -# MCP服务器接口对比分析 - -## 问题分析 - -对比标签引擎的MCP服务器和实际的采集任务接口，发现以下差异： - ---- - -## 一、后端接口要求（DataCollectionTaskController） - -### 必填字段（从Controller验证逻辑看）： -```php -$requiredFields = ['name', 'data_source_id', 'database', 'target_data_source_id', 'target_database', 'target_collection']; -``` - -### 可选但支持的字段： -- `target_type` - 目标类型（consumption_record 或 generic） -- `mode` - 采集模式（batch 或 realtime） -- `collection` - 单集合模式 -- `collections` - 多集合模式（与collection二选一） -- `multi_collection` - 是否多集合模式 -- `field_mappings` - 字段映射（单集合模式） -- `collection_field_mappings` - 字段映射（多集合模式） -- `lookups` - 连表查询配置（单集合模式） -- `collection_lookups` - 连表查询配置（多集合模式） -- `filter_conditions` - 过滤条件 -- `schedule` - 调度配置 -- `description` - 任务描述 - ---- - -## 二、前端TaskForm实际使用的字段 - -根据 `TaskShow/src/views/DataCollection/TaskForm.vue`，前端表单包含： - -```typescript -{ - name: string - description: string - data_source_id: string - database: string - collection?: string // 单集合模式 - collections?: string[] // 多集合模式 - multi_collection: boolean - target_type: string // 'consumption_record' | 'generic' - mode: 'batch' | 'realtime' - field_mappings: FieldMapping[] - collection_field_mappings?: Record - lookups?: LookupConfig[] - collection_lookups?: Record - filter_conditions?: FilterCondition[] - schedule: { - enabled: boolean - cron?: string - } -} -``` - -**注意**：前端**没有** `target_data_source_id`, `target_database`, `target_collection` 字段！ - ---- - -## 三、MCP服务器当前支持的字段 - -根据 `MCP/moncter-mcp-server/src/index.ts`，当前支持： - -```typescript -{ - name: string - description: string - data_source_id: string - database: string - collection?: string - collections?: string[] - mode: 'batch' | 'realtime' - field_mappings: FieldMapping[] - schedule: { - enabled: boolean - cron?: string - } -} -``` - ---- - -## 四、字段差异对比 - -### ❌ MCP服务器缺少的字段： - -1. **`target_type`** - 目标类型（重要！） - - 前端使用：`'consumption_record'` 或 `'generic'` - - 用途：决定使用哪个Handler处理数据 - - 优先级：**高** - -2. **`multi_collection`** - 多集合模式标识 - - 前端使用：`boolean` - - 用途：区分单集合和多集合模式 - - 优先级：**中** - -3. **`filter_conditions`** - 过滤条件 - - 前端使用：`FilterCondition[]` - - 用途：数据采集的过滤条件 - - 优先级：**中** - -4. **`lookups`** - 连表查询配置（单集合模式） - - 前端使用：`LookupConfig[]` - - 用途：MongoDB $lookup 连表查询 - - 优先级：**低** - -5. **`collection_lookups`** - 连表查询配置（多集合模式） - - 前端使用：`Record` - - 用途：多集合模式下的连表查询 - - 优先级：**低** - -6. **`collection_field_mappings`** - 字段映射（多集合模式） - - 前端使用：`Record` - - 用途：多集合模式下每个集合的字段映射 - - 优先级：**中** - -### ⚠️ 后端Controller要求的字段（但前端和Service都没使用）： - -1. **`target_data_source_id`** - 目标数据源ID -2. **`target_database`** - 目标数据库 -3. **`target_collection`** - 目标集合 - -**分析**： -- Controller的验证逻辑要求这些字段必填 -- 但Service的`createTask`方法并没有使用这些字段 -- 前端TaskForm也没有这些字段 -- 对于`consumption_record`类型，Handler会自动处理存储，不需要指定目标 -- 对于`generic`类型，可能需要这些字段 - -**结论**：这些字段可能是历史遗留，或者仅对`generic`类型必需，对`consumption_record`类型不是必需的。需要确认后端逻辑。 - ---- - -## 五、建议的修复方案 - -### 方案1：更新MCP服务器，添加缺失字段 - -**优先添加的字段**： -1. ✅ `target_type` - **必需** -2. ✅ `multi_collection` - **推荐** -3. ✅ `filter_conditions` - **推荐** -4. ✅ `collection_field_mappings` - **推荐**（支持多集合模式） -5. ⚠️ `lookups` - 可选 -6. ⚠️ `collection_lookups` - 可选 - -**暂不处理**： -- `target_data_source_id`, `target_database`, `target_collection` - 需要先确认后端逻辑 - -### 方案2：确认后端接口的实际要求 - -需要确认： -1. `target_data_source_id`, `target_database`, `target_collection` 是否真的必需？ -2. 对于`consumption_record`类型，这些字段是否可选？ -3. 如果必需，MCP服务器需要添加这些字段 - ---- - -## 六、当前状态总结 - -### ✅ MCP服务器已支持的字段： -- name -- description -- data_source_id -- database -- collection / collections -- mode -- field_mappings -- schedule - -### ❌ MCP服务器缺少的字段（按优先级）： -1. **target_type** - ⚠️ 重要！决定Handler类型 -2. **multi_collection** - 区分单/多集合模式 -3. **filter_conditions** - 数据过滤条件 -4. **collection_field_mappings** - 多集合模式的字段映射 -5. **lookups** - 连表查询（可选） -6. **collection_lookups** - 多集合连表查询（可选） - -### ⚠️ 需要确认的字段： -- target_data_source_id -- target_database -- target_collection - ---- - -## 七、下一步行动 - -1. ✅ 对比分析完成 -2. ⏳ 更新MCP服务器，添加缺失字段 -3. ⏳ 确认后端接口对target相关字段的要求 -4. ⏳ 测试更新后的MCP服务器 - diff --git a/Moncter/Moncter/MCP/moncter-mcp-server/MCP服务器同步更新说明.md b/Moncter/Moncter/MCP/moncter-mcp-server/MCP服务器同步更新说明.md deleted file mode 100644 index d144db11..00000000 --- a/Moncter/Moncter/MCP/moncter-mcp-server/MCP服务器同步更新说明.md +++ /dev/null @@ -1,342 +0,0 @@ -# MCP服务器同步更新说明 - -## 更新日期 -2025年12月 - -## 更新背景 - -根据最新的 `TaskForm.vue` 界面变更，MCP服务器需要同步更新以匹配最新的界面逻辑。 - ---- - -## 主要变更 - -### ✅ 移除的字段 - -根据最新的TaskForm.vue，以下字段已经从界面中移除，MCP服务器也已移除： - -1. **`target_data_source_id`** - 目标数据源ID -2. **`target_database`** - 目标数据库 -3. **`target_collection`** - 目标集合 - -**原因**： -- 对于 `consumption_record` 类型，Handler会自动处理存储到标签引擎数据库，不需要指定目标 -- 对于 `generic` 类型，如果需要指定目标，应该在Handler配置或业务逻辑中处理 -- 界面已经简化，不再要求用户配置这些字段 - -### ✅ 保留和优化的字段 - -#### 核心字段（必填） - -1. **`name`** - 任务名称 -2. **`data_source_id`** - 源数据源ID -3. **`database`** - 源数据库名称 -4. **`mode`** - 采集模式（`batch` 或 `realtime`） -5. **`target_type`** - 目标类型（`consumption_record` 或 `generic`） - -#### 集合配置字段 - -6. **`collection`** - 源集合名称（单集合模式） -7. **`collections`** - 源集合列表（多集合模式） -8. **`multi_collection`** - 是否启用多集合模式 - -**说明**：`collection` 和 `collections` 二选一，由 `multi_collection` 字段决定使用哪个。 - -#### 字段映射字段 - -9. **`field_mappings`** - 字段映射配置（单集合模式） - - 格式：`[{ source_field, target_field, transform? }]` - - 转换函数：`parse_amount`, `parse_datetime`, `parse_phone` - -10. **`collection_field_mappings`** - 字段映射配置（多集合模式） - - 格式：`{ "collection_name": [FieldMapping] }` - - 每个集合可配置独立的字段映射 - -#### 查询配置字段 - -11. **`lookups`** - MongoDB $lookup连表查询配置（单集合模式） - - 格式：`[{ from, local_field, foreign_field, as, unwrap?, preserve_null? }]` - -12. **`collection_lookups`** - MongoDB $lookup连表查询配置（多集合模式） - - 格式：`{ "collection_name": [LookupConfig] }` - -13. **`filter_conditions`** - 过滤条件 - - 格式：`[{ field, operator, value }]` - - 运算符：`eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `nin` - -#### 调度配置字段 - -14. **`schedule`** - 调度配置（批量模式使用） - - `enabled`: 是否启用调度 - - `cron`: Cron表达式 - ---- - -## 字段使用指南 - -### 1. 消费记录采集任务（consumption_record） - -**适用场景**：采集订单、交易等消费记录数据 - -**特点**： -- Handler会自动转换数据格式 -- 通过手机号/身份证自动解析用户ID -- 自动时间分片存储（按月分表） -- 存储到标签引擎数据库 - -**示例**： -```json -{ - "name": "订单数据采集", - "description": "从KR商城采集订单数据", - "data_source_id": "source_123", - "database": "KR_商城", - "collection": "21年贝蒂喜订单整合", - "multi_collection": false, - "target_type": "consumption_record", - "mode": "batch", - "field_mappings": [ - { - "source_field": "联系手机", - "target_field": "phone_number", - "transform": "parse_phone" - }, - { - "source_field": "买家实际支付金额", - "target_field": "actual_amount", - "transform": "parse_amount" - }, - { - "source_field": "订单付款时间", - "target_field": "consume_time", - "transform": "parse_datetime" - }, - { - "source_field": "店铺名称", - "target_field": "store_name" - } - ], - "filter_conditions": [ - { - "field": "买家实际支付金额", - "operator": "ne", - "value": "0" - }, - { - "field": "订单付款时间", - "operator": "ne", - "value": null - } - ], - "schedule": { - "enabled": true, - "cron": "0 2 * * *" - } -} -``` - -### 2. 通用集合采集任务（generic） - -**适用场景**：采集任意数据并存储到指定集合 - -**特点**： -- 需要自定义字段映射 -- 需要指定目标存储（在Handler配置中） - -**示例**： -```json -{ - "name": "通用数据采集", - "description": "采集用户数据", - "data_source_id": "source_123", - "database": "user_db", - "collection": "users", - "multi_collection": false, - "target_type": "generic", - "mode": "realtime", - "field_mappings": [ - { - "source_field": "user_name", - "target_field": "name" - }, - { - "source_field": "user_email", - "target_field": "email" - } - ], - "schedule": { - "enabled": false - } -} -``` - -### 3. 多集合模式 - -**适用场景**：同时从多个集合采集数据 - -**示例**： -```json -{ - "name": "多集合数据采集", - "data_source_id": "source_123", - "database": "multi_db", - "multi_collection": true, - "collections": ["collection1", "collection2"], - "target_type": "generic", - "mode": "batch", - "collection_field_mappings": { - "collection1": [ - { - "source_field": "field1", - "target_field": "target1" - } - ], - "collection2": [ - { - "source_field": "field2", - "target_field": "target2" - } - ] - }, - "collection_lookups": { - "collection1": [ - { - "from": "related_collection", - "local_field": "related_id", - "foreign_field": "_id", - "as": "related_data", - "unwrap": false, - "preserve_null": true - } - ] - }, - "schedule": { - "enabled": true, - "cron": "0 3 * * *" - } -} -``` - -### 4. 连表查询配置 - -**适用场景**：需要从其他集合关联数据 - -**示例**： -```json -{ - "lookups": [ - { - "from": "user_info", - "local_field": "user_id", - "foreign_field": "_id", - "as": "user_info", - "unwrap": true, - "preserve_null": false - } - ] -} -``` - -**说明**： -- `unwrap: true` - 解构后可直接使用 `user_info.mobile` -- `unwrap: false` - 返回数组 `user_info[0].mobile` - ---- - -## 与界面的对应关系 - -| MCP字段 | TaskForm字段 | 界面位置 | 说明 | -|---------|-------------|---------|------| -| name | form.name | 步骤1：基本信息 | 任务名称 | -| description | form.description | 步骤1：基本信息 | 任务描述 | -| mode | form.mode | 步骤1：基本信息 | 采集模式 | -| target_type | form.target_type | 步骤2：Handler配置 | 数据处理方式 | -| data_source_id | form.data_source_id | 步骤3：源数据配置 | 数据源 | -| database | form.database | 步骤3：源数据配置 | 数据库 | -| collection | form.collection | 步骤3：源数据配置 | 集合（单集合） | -| collections | form.collections | 步骤3：源数据配置 | 集合列表（多集合） | -| multi_collection | form.multi_collection | 步骤3：源数据配置 | 多集合模式开关 | -| lookups | form.lookups | 步骤3：连表查询 | 连表查询配置 | -| filter_conditions | form.filter_conditions | 步骤3：过滤条件 | 过滤条件 | -| field_mappings | form.field_mappings | 步骤4：字段映射 | 字段映射（单集合） | -| collection_field_mappings | form.collection_field_mappings | 步骤4：字段映射 | 字段映射（多集合） | -| schedule | form.schedule | 步骤5：调度配置 | 调度配置 | - ---- - -## 验证清单 - -✅ MCP服务器已更新，完全匹配最新的TaskForm.vue界面逻辑： - -- [x] 移除了 `target_data_source_id`, `target_database`, `target_collection` 字段 -- [x] 保留了所有界面使用的字段 -- [x] 更新了字段描述，使其更清晰准确 -- [x] 明确了字段的使用场景和条件要求 -- [x] 支持单集合和多集合模式 -- [x] 支持连表查询和过滤条件 -- [x] 支持字段映射和转换函数 - ---- - -## 注意事项 - -1. **target_type是必填的**：必须明确指定是 `consumption_record` 还是 `generic` -2. **collection和collections二选一**：根据 `multi_collection` 决定使用哪个 -3. **字段映射方式**： - - 单集合模式使用 `field_mappings` - - 多集合模式使用 `collection_field_mappings` -4. **连表查询方式**： - - 单集合模式使用 `lookups` - - 多集合模式使用 `collection_lookups` -5. **调度配置**：仅在 `mode=batch` 时使用 - ---- - -## 测试建议 - -1. **测试消费记录采集任务创建** - ```json - { - "name": "测试任务", - "data_source_id": "test_source", - "database": "test_db", - "collection": "test_collection", - "target_type": "consumption_record", - "mode": "batch", - "field_mappings": [...], - "schedule": {"enabled": true, "cron": "0 2 * * *"} - } - ``` - -2. **测试多集合模式** - ```json - { - "name": "多集合测试", - "data_source_id": "test_source", - "database": "test_db", - "multi_collection": true, - "collections": ["coll1", "coll2"], - "target_type": "generic", - "mode": "batch", - "collection_field_mappings": {...} - } - ``` - -3. **测试连表查询** - ```json - { - "lookups": [{ - "from": "related", - "local_field": "id", - "foreign_field": "_id", - "as": "related_data" - }] - } - ``` - ---- - -## 总结 - -MCP服务器已成功同步更新，完全匹配最新的TaskForm.vue界面逻辑。所有字段定义、使用方式和验证规则都与界面保持一致。 - diff --git a/Moncter/Moncter/MCP/moncter-mcp-server/MCP服务器更新说明.md b/Moncter/Moncter/MCP/moncter-mcp-server/MCP服务器更新说明.md deleted file mode 100644 index 8c636155..00000000 --- a/Moncter/Moncter/MCP/moncter-mcp-server/MCP服务器更新说明.md +++ /dev/null @@ -1,246 +0,0 @@ -# MCP服务器更新说明 - -## 更新日期 -2025年12月 - -## 更新内容 - -### ✅ 已添加的字段 - -更新了 `create_data_collection_task` 工具，添加了以下缺失的字段： - -1. **`target_type`** ⭐ **重要** - - 类型：`'consumption_record' | 'generic'` - - 必填：是 - - 说明：决定使用哪个Handler处理数据 - - 用途： - - `consumption_record`: 使用ConsumptionCollectionHandler，自动处理消费记录格式转换和存储 - - `generic`: 使用GenericCollectionHandler，支持自定义字段映射和目标存储 - -2. **`multi_collection`** - - 类型：`boolean` - - 必填：否 - - 说明：是否启用多集合模式 - -3. **`target_data_source_id`** - - 类型：`string` - - 必填：否（但后端Controller验证要求必填，见注意事项） - - 说明：目标数据源ID（通用Handler需要） - -4. **`target_database`** - - 类型：`string` - - 必填：否（但后端Controller验证要求必填，见注意事项） - - 说明：目标数据库（通用Handler需要） - -5. **`target_collection`** - - 类型：`string` - - 必填：否（但后端Controller验证要求必填，见注意事项） - - 说明：目标集合（通用Handler需要） - -6. **`collection_field_mappings`** - - 类型：`object` - - 必填：否 - - 说明：多集合模式下的字段映射，格式：`{ "collection_name": [FieldMapping] }` - -7. **`lookups`** - - 类型：`array` - - 必填：否 - - 说明：单集合模式下的MongoDB $lookup连表查询配置 - - 结构： - ```typescript - { - from: string, // 关联集合名 - local_field: string, // 主集合字段 - foreign_field: string, // 关联集合字段 - as: string, // 结果字段名 - unwrap?: boolean, // 是否解构 - preserve_null?: boolean // 是否保留空值 - } - ``` - -8. **`collection_lookups`** - - 类型：`object` - - 必填：否 - - 说明：多集合模式下的连表查询配置，格式：`{ "collection_name": [LookupConfig] }` - -9. **`filter_conditions`** - - 类型：`array` - - 必填：否 - - 说明：数据采集的过滤条件 - - 结构： - ```typescript - { - field: string, - operator: 'eq' | 'ne' | 'gt' | 'gte' | 'lt' | 'lte' | 'in' | 'nin', - value: any - } - ``` - -### 📝 更新的字段说明 - -- **`field_mappings`**: 添加了更详细的说明，明确是单集合模式的字段映射 - -### ✅ 更新的必填字段 - -- 新增 `target_type` 为必填字段（这是最重要的字段，决定Handler类型） - ---- - -## ⚠️ 注意事项 - -### 1. 后端Controller验证问题 - -**问题**： -后端 `DataCollectionTaskController::create()` 方法要求以下字段必填： -- `target_data_source_id` -- `target_database` -- `target_collection` - -**但实际情况**： -- 前端 `TaskForm.vue` 没有这些字段 -- Service的 `createTask()` 方法也没有使用这些字段 -- 对于 `consumption_record` 类型，Handler会自动处理存储，不需要指定目标 - -**可能的原因**： -1. Controller的验证逻辑过于严格 -2. 这些字段仅对 `generic` 类型必需 -3. 后端代码不一致（Controller和Service不同步） - -**建议**： -1. 如果使用 `consumption_record` 类型，可能需要传递空值或默认值 -2. 如果使用 `generic` 类型，必须提供这些字段 -3. 建议后端修改验证逻辑，根据 `target_type` 动态验证必填字段 - -### 2. 字段使用建议 - -**对于 consumption_record 类型**： -```json -{ - "name": "订单采集任务", - "data_source_id": "source_123", - "database": "KR_商城", - "collection": "21年贝蒂喜订单整合", - "target_type": "consumption_record", - "mode": "batch", - "field_mappings": [ - { - "source_field": "联系手机", - "target_field": "phone_number", - "transform": "parse_phone" - }, - { - "source_field": "买家实际支付金额", - "target_field": "actual_amount", - "transform": "parse_amount" - } - ], - "filter_conditions": [ - { - "field": "买家实际支付金额", - "operator": "ne", - "value": "0" - } - ], - "schedule": { - "enabled": true, - "cron": "0 2 * * *" - } -} -``` - -**对于 generic 类型**： -```json -{ - "name": "通用数据采集", - "data_source_id": "source_123", - "database": "KR_商城", - "collection": "some_collection", - "target_type": "generic", - "target_data_source_id": "target_source_123", - "target_database": "target_db", - "target_collection": "target_collection", - "mode": "batch", - "field_mappings": [ - { - "source_field": "source_field1", - "target_field": "target_field1" - } - ], - "schedule": { - "enabled": false - } -} -``` - ---- - -## 📊 字段对比表 - -| 字段 | MCP服务器（更新前） | MCP服务器（更新后） | 前端 | 后端Controller | 优先级 | -|------|-------------------|-------------------|------|---------------|--------| -| name | ✅ | ✅ | ✅ | ✅ 必填 | 高 | -| data_source_id | ✅ | ✅ | ✅ | ✅ 必填 | 高 | -| database | ✅ | ✅ | ✅ | ✅ 必填 | 高 | -| collection/collections | ✅ | ✅ | ✅ | ✅ 必填 | 高 | -| mode | ✅ | ✅ | ✅ | ✅ | 高 | -| field_mappings | ✅ | ✅ | ✅ | ✅ | 中 | -| schedule | ✅ | ✅ | ✅ | ✅ | 中 | -| target_type | ❌ | ✅ **新增** | ✅ | ✅ | **高** ⭐ | -| multi_collection | ❌ | ✅ **新增** | ✅ | ✅ | 中 | -| filter_conditions | ❌ | ✅ **新增** | ✅ | ✅ | 中 | -| collection_field_mappings | ❌ | ✅ **新增** | ✅ | ✅ | 中 | -| lookups | ❌ | ✅ **新增** | ✅ | ✅ | 低 | -| collection_lookups | ❌ | ✅ **新增** | ✅ | ✅ | 低 | -| target_data_source_id | ❌ | ✅ **新增** | ❌ | ✅ 必填 | ⚠️ | -| target_database | ❌ | ✅ **新增** | ❌ | ✅ 必填 | ⚠️ | -| target_collection | ❌ | ✅ **新增** | ❌ | ✅ 必填 | ⚠️ | - ---- - -## ✅ 更新后的状态 - -### 完全支持的字段： -- ✅ 基本字段（name, description, data_source_id, database, collection/collections） -- ✅ 模式配置（mode, multi_collection） -- ✅ 目标配置（target_type, target_data_source_id, target_database, target_collection） -- ✅ 字段映射（field_mappings, collection_field_mappings） -- ✅ 查询配置（lookups, collection_lookups, filter_conditions） -- ✅ 调度配置（schedule） - -### ⚠️ 需要注意的问题： -- 后端Controller要求 `target_data_source_id`, `target_database`, `target_collection` 必填，但前端和Service都没有使用 -- 建议在使用MCP创建任务时，对于 `consumption_record` 类型，可能需要传递这些字段的空值或默认值，或者后端需要修改验证逻辑 - ---- - -## 🔄 下一步建议 - -1. **后端验证逻辑优化**： - - 根据 `target_type` 动态验证必填字段 - - 对于 `consumption_record` 类型，不需要 `target_*` 字段 - - 对于 `generic` 类型，需要 `target_*` 字段 - -2. **测试验证**： - - 测试使用MCP创建 `consumption_record` 类型的任务 - - 测试使用MCP创建 `generic` 类型的任务 - - 验证所有新增字段是否正确传递 - -3. **文档更新**： - - 更新MCP使用文档，说明不同 `target_type` 的字段要求 - - 添加使用示例 - ---- - -## 📝 总结 - -MCP服务器已经更新，**基本符合**当前的采集任务接口要求。主要添加了： - -1. ✅ **target_type** - 最重要的字段，决定Handler类型 -2. ✅ **multi_collection** - 支持多集合模式 -3. ✅ **filter_conditions** - 支持数据过滤 -4. ✅ **lookups/collection_lookups** - 支持连表查询 -5. ✅ **collection_field_mappings** - 支持多集合字段映射 -6. ✅ **target_* 字段** - 虽然前端没有，但后端Controller要求，已添加 - -**剩余问题**：后端Controller的验证逻辑可能与实际使用不一致，需要确认或调整。 - diff --git a/Moncter/Moncter/MCP/moncter-mcp-server/README.md b/Moncter/Moncter/MCP/moncter-mcp-server/README.md deleted file mode 100644 index 7e394f00..00000000 --- a/Moncter/Moncter/MCP/moncter-mcp-server/README.md +++ /dev/null @@ -1,122 +0,0 @@ -# Moncter MCP Server - -Moncter MCP Server 是一个 Model Context Protocol (MCP) 服务器，用于通过 MCP 协议管理 Moncter 系统的数据采集任务和标签任务。 - -## 功能 - -提供以下 MCP 工具： - -1. **create_data_collection_task** - 创建数据采集任务 -2. **create_tag_task** - 创建标签计算任务 -3. **list_data_collection_tasks** - 获取数据采集任务列表 -4. **list_tag_tasks** - 获取标签任务列表 -5. **get_data_sources** - 获取数据源列表 -6. **get_tag_definitions** - 获取标签定义列表 -7. **start_data_collection_task** - 启动数据采集任务 -8. **start_tag_task** - 启动标签任务 - -## 安装 - -```bash -cd MCP/moncter-mcp-server -npm install -npm run build -``` - -## 配置 - -在 `mcp.json` 中配置服务器： - -```json -{ - "mcpServers": { - "Moncter": { - "command": "node", - "args": ["E:/Cunkebao/Cunkebao02/Moncter/MCP/moncter-mcp-server/dist/index.js"], - "env": { - "MONCTER_API_URL": "http://127.0.0.1:8787" - } - } - } -} -``` - -或者使用 npm 方式（如果全局安装）： - -```json -{ - "mcpServers": { - "Moncter": { - "command": "node", - "args": ["./MCP/moncter-mcp-server/dist/index.js"], - "cwd": "E:/Cunkebao/Cunkebao02/Moncter", - "env": { - "MONCTER_API_URL": "http://127.0.0.1:8787" - } - } - } -} -``` - -## 环境变量 - -- `MONCTER_API_URL`: 后端API基础URL（默认: http://127.0.0.1:8787） - -## 使用示例 - -### 创建数据采集任务 - -```json -{ - "name": "create_data_collection_task", - "arguments": { - "name": "订单数据采集", - "description": "从KR商城采集订单数据", - "data_source_id": "your_data_source_id", - "database": "KR_商城", - "collection": "21年贝蒂喜订单整合", - "mode": "realtime", - "field_mappings": [ - { - "source_field": "订单号", - "target_field": "order_no" - } - ] - } -} -``` - -### 创建标签任务 - -```json -{ - "name": "create_tag_task", - "arguments": { - "name": "高价值用户标签计算", - "description": "计算高价值用户标签", - "task_type": "full", - "target_tag_ids": ["tag_id_1", "tag_id_2"], - "user_scope": { - "type": "all" - }, - "schedule": { - "enabled": true, - "cron": "0 2 * * *" - } - } -} -``` - -## 开发 - -```bash -# 开发模式（使用 tsx） -npm run dev - -# 编译 -npm run build - -# 运行 -npm start -``` - diff --git a/Moncter/Moncter/MCP/moncter-mcp-server/install.bat b/Moncter/Moncter/MCP/moncter-mcp-server/install.bat deleted file mode 100644 index 5a077917..00000000 --- a/Moncter/Moncter/MCP/moncter-mcp-server/install.bat +++ /dev/null @@ -1,39 +0,0 @@ -@echo off -REM Moncter MCP Server 安装脚本 (Windows) - -echo 正在安装 Moncter MCP Server... - -REM 检查 Node.js -where node >nul 2>nul -if %ERRORLEVEL% NEQ 0 ( - echo 错误: 未找到 Node.js，请先安装 Node.js (^>= 18^) - exit /b 1 -) - -REM 安装依赖 -echo 安装依赖... -call npm install -if %ERRORLEVEL% NEQ 0 ( - echo 错误: npm install 失败 - exit /b 1 -) - -REM 编译 TypeScript -echo 编译 TypeScript... -call npm run build -if %ERRORLEVEL% NEQ 0 ( - echo 错误: 编译失败 - exit /b 1 -) - -echo. -echo ✅ Moncter MCP Server 安装成功！ -echo. -echo 使用说明： -echo 1. 确保后端服务运行在 http://127.0.0.1:8787 -echo 2. 配置 MCP 客户端，添加 Moncter MCP 服务器 -echo 3. 查看 MCP/MCP服务器使用说明.md 了解详细用法 -echo. - -pause - diff --git a/Moncter/Moncter/MCP/moncter-mcp-server/install.sh b/Moncter/Moncter/MCP/moncter-mcp-server/install.sh deleted file mode 100644 index 17f9daba..00000000 --- a/Moncter/Moncter/MCP/moncter-mcp-server/install.sh +++ /dev/null @@ -1,38 +0,0 @@ -#!/bin/bash - -# Moncter MCP Server 安装脚本 - -echo "正在安装 Moncter MCP Server..." - -# 检查 Node.js -if ! command -v node &> /dev/null; then - echo "错误: 未找到 Node.js，请先安装 Node.js (>= 18)" - exit 1 -fi - -NODE_VERSION=$(node -v | cut -d'v' -f2 | cut -d'.' -f1) -if [ "$NODE_VERSION" -lt 18 ]; then - echo "错误: Node.js 版本过低，需要 >= 18" - exit 1 -fi - -# 安装依赖 -echo "安装依赖..." -npm install - -# 编译 TypeScript -echo "编译 TypeScript..." -npm run build - -if [ $? -eq 0 ]; then - echo "✅ Moncter MCP Server 安装成功！" - echo "" - echo "使用说明：" - echo "1. 确保后端服务运行在 http://127.0.0.1:8787" - echo "2. 配置 MCP 客户端，添加 Moncter MCP 服务器" - echo "3. 查看 MCP/MCP服务器使用说明.md 了解详细用法" -else - echo "❌ 编译失败，请检查错误信息" - exit 1 -fi - diff --git a/Moncter/Moncter/MCP/moncter-mcp-server/package-lock.json b/Moncter/Moncter/MCP/moncter-mcp-server/package-lock.json deleted file mode 100644 index 3697e071..00000000 --- a/Moncter/Moncter/MCP/moncter-mcp-server/package-lock.json +++ /dev/null @@ -1,1103 +0,0 @@ -{ - "name": "moncter-mcp-server", - "version": "1.0.0", - "lockfileVersion": 3, - "requires": true, - "packages": { - "": { - "name": "moncter-mcp-server", - "version": "1.0.0", - "license": "MIT", - "dependencies": { - "@modelcontextprotocol/sdk": "^0.5.0", - "node-fetch": "^3.3.2" - }, - "devDependencies": { - "@types/node": "^20.10.0", - "@types/node-fetch": "^2.6.11", - "tsx": "^4.7.0", - "typescript": "^5.3.3" - } - }, - "node_modules/@esbuild/aix-ppc64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/aix-ppc64/-/aix-ppc64-0.27.2.tgz", - "integrity": "sha512-GZMB+a0mOMZs4MpDbj8RJp4cw+w1WV5NYD6xzgvzUJ5Ek2jerwfO2eADyI6ExDSUED+1X8aMbegahsJi+8mgpw==", - "cpu": [ - "ppc64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "aix" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/android-arm": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/android-arm/-/android-arm-0.27.2.tgz", - "integrity": "sha512-DVNI8jlPa7Ujbr1yjU2PfUSRtAUZPG9I1RwW4F4xFB1Imiu2on0ADiI/c3td+KmDtVKNbi+nffGDQMfcIMkwIA==", - "cpu": [ - "arm" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "android" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/android-arm64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/android-arm64/-/android-arm64-0.27.2.tgz", - "integrity": "sha512-pvz8ZZ7ot/RBphf8fv60ljmaoydPU12VuXHImtAs0XhLLw+EXBi2BLe3OYSBslR4rryHvweW5gmkKFwTiFy6KA==", - "cpu": [ - "arm64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "android" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/android-x64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/android-x64/-/android-x64-0.27.2.tgz", - "integrity": "sha512-z8Ank4Byh4TJJOh4wpz8g2vDy75zFL0TlZlkUkEwYXuPSgX8yzep596n6mT7905kA9uHZsf/o2OJZubl2l3M7A==", - "cpu": [ - "x64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "android" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/darwin-arm64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/darwin-arm64/-/darwin-arm64-0.27.2.tgz", - "integrity": "sha512-davCD2Zc80nzDVRwXTcQP/28fiJbcOwvdolL0sOiOsbwBa72kegmVU0Wrh1MYrbuCL98Omp5dVhQFWRKR2ZAlg==", - "cpu": [ - "arm64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "darwin" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/darwin-x64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/darwin-x64/-/darwin-x64-0.27.2.tgz", - "integrity": "sha512-ZxtijOmlQCBWGwbVmwOF/UCzuGIbUkqB1faQRf5akQmxRJ1ujusWsb3CVfk/9iZKr2L5SMU5wPBi1UWbvL+VQA==", - "cpu": [ - "x64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "darwin" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/freebsd-arm64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/freebsd-arm64/-/freebsd-arm64-0.27.2.tgz", - "integrity": "sha512-lS/9CN+rgqQ9czogxlMcBMGd+l8Q3Nj1MFQwBZJyoEKI50XGxwuzznYdwcav6lpOGv5BqaZXqvBSiB/kJ5op+g==", - "cpu": [ - "arm64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "freebsd" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/freebsd-x64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/freebsd-x64/-/freebsd-x64-0.27.2.tgz", - "integrity": "sha512-tAfqtNYb4YgPnJlEFu4c212HYjQWSO/w/h/lQaBK7RbwGIkBOuNKQI9tqWzx7Wtp7bTPaGC6MJvWI608P3wXYA==", - "cpu": [ - "x64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "freebsd" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/linux-arm": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/linux-arm/-/linux-arm-0.27.2.tgz", - "integrity": "sha512-vWfq4GaIMP9AIe4yj1ZUW18RDhx6EPQKjwe7n8BbIecFtCQG4CfHGaHuh7fdfq+y3LIA2vGS/o9ZBGVxIDi9hw==", - "cpu": [ - "arm" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "linux" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/linux-arm64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/linux-arm64/-/linux-arm64-0.27.2.tgz", - "integrity": "sha512-hYxN8pr66NsCCiRFkHUAsxylNOcAQaxSSkHMMjcpx0si13t1LHFphxJZUiGwojB1a/Hd5OiPIqDdXONia6bhTw==", - "cpu": [ - "arm64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "linux" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/linux-ia32": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/linux-ia32/-/linux-ia32-0.27.2.tgz", - "integrity": "sha512-MJt5BRRSScPDwG2hLelYhAAKh9imjHK5+NE/tvnRLbIqUWa+0E9N4WNMjmp/kXXPHZGqPLxggwVhz7QP8CTR8w==", - "cpu": [ - "ia32" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "linux" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/linux-loong64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/linux-loong64/-/linux-loong64-0.27.2.tgz", - "integrity": "sha512-lugyF1atnAT463aO6KPshVCJK5NgRnU4yb3FUumyVz+cGvZbontBgzeGFO1nF+dPueHD367a2ZXe1NtUkAjOtg==", - "cpu": [ - "loong64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "linux" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/linux-mips64el": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/linux-mips64el/-/linux-mips64el-0.27.2.tgz", - "integrity": "sha512-nlP2I6ArEBewvJ2gjrrkESEZkB5mIoaTswuqNFRv/WYd+ATtUpe9Y09RnJvgvdag7he0OWgEZWhviS1OTOKixw==", - "cpu": [ - "mips64el" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "linux" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/linux-ppc64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/linux-ppc64/-/linux-ppc64-0.27.2.tgz", - "integrity": "sha512-C92gnpey7tUQONqg1n6dKVbx3vphKtTHJaNG2Ok9lGwbZil6DrfyecMsp9CrmXGQJmZ7iiVXvvZH6Ml5hL6XdQ==", - "cpu": [ - "ppc64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "linux" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/linux-riscv64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/linux-riscv64/-/linux-riscv64-0.27.2.tgz", - "integrity": "sha512-B5BOmojNtUyN8AXlK0QJyvjEZkWwy/FKvakkTDCziX95AowLZKR6aCDhG7LeF7uMCXEJqwa8Bejz5LTPYm8AvA==", - "cpu": [ - "riscv64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "linux" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/linux-s390x": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/linux-s390x/-/linux-s390x-0.27.2.tgz", - "integrity": "sha512-p4bm9+wsPwup5Z8f4EpfN63qNagQ47Ua2znaqGH6bqLlmJ4bx97Y9JdqxgGZ6Y8xVTixUnEkoKSHcpRlDnNr5w==", - "cpu": [ - "s390x" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "linux" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/linux-x64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/linux-x64/-/linux-x64-0.27.2.tgz", - "integrity": "sha512-uwp2Tip5aPmH+NRUwTcfLb+W32WXjpFejTIOWZFw/v7/KnpCDKG66u4DLcurQpiYTiYwQ9B7KOeMJvLCu/OvbA==", - "cpu": [ - "x64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "linux" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/netbsd-arm64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/netbsd-arm64/-/netbsd-arm64-0.27.2.tgz", - "integrity": "sha512-Kj6DiBlwXrPsCRDeRvGAUb/LNrBASrfqAIok+xB0LxK8CHqxZ037viF13ugfsIpePH93mX7xfJp97cyDuTZ3cw==", - "cpu": [ - "arm64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "netbsd" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/netbsd-x64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/netbsd-x64/-/netbsd-x64-0.27.2.tgz", - "integrity": "sha512-HwGDZ0VLVBY3Y+Nw0JexZy9o/nUAWq9MlV7cahpaXKW6TOzfVno3y3/M8Ga8u8Yr7GldLOov27xiCnqRZf0tCA==", - "cpu": [ - "x64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "netbsd" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/openbsd-arm64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/openbsd-arm64/-/openbsd-arm64-0.27.2.tgz", - "integrity": "sha512-DNIHH2BPQ5551A7oSHD0CKbwIA/Ox7+78/AWkbS5QoRzaqlev2uFayfSxq68EkonB+IKjiuxBFoV8ESJy8bOHA==", - "cpu": [ - "arm64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "openbsd" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/openbsd-x64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/openbsd-x64/-/openbsd-x64-0.27.2.tgz", - "integrity": "sha512-/it7w9Nb7+0KFIzjalNJVR5bOzA9Vay+yIPLVHfIQYG/j+j9VTH84aNB8ExGKPU4AzfaEvN9/V4HV+F+vo8OEg==", - "cpu": [ - "x64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "openbsd" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/openharmony-arm64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/openharmony-arm64/-/openharmony-arm64-0.27.2.tgz", - "integrity": "sha512-LRBbCmiU51IXfeXk59csuX/aSaToeG7w48nMwA6049Y4J4+VbWALAuXcs+qcD04rHDuSCSRKdmY63sruDS5qag==", - "cpu": [ - "arm64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "openharmony" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/sunos-x64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/sunos-x64/-/sunos-x64-0.27.2.tgz", - "integrity": "sha512-kMtx1yqJHTmqaqHPAzKCAkDaKsffmXkPHThSfRwZGyuqyIeBvf08KSsYXl+abf5HDAPMJIPnbBfXvP2ZC2TfHg==", - "cpu": [ - "x64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "sunos" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/win32-arm64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/win32-arm64/-/win32-arm64-0.27.2.tgz", - "integrity": "sha512-Yaf78O/B3Kkh+nKABUF++bvJv5Ijoy9AN1ww904rOXZFLWVc5OLOfL56W+C8F9xn5JQZa3UX6m+IktJnIb1Jjg==", - "cpu": [ - "arm64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "win32" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/win32-ia32": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/win32-ia32/-/win32-ia32-0.27.2.tgz", - "integrity": "sha512-Iuws0kxo4yusk7sw70Xa2E2imZU5HoixzxfGCdxwBdhiDgt9vX9VUCBhqcwY7/uh//78A1hMkkROMJq9l27oLQ==", - "cpu": [ - "ia32" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "win32" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@esbuild/win32-x64": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/@esbuild/win32-x64/-/win32-x64-0.27.2.tgz", - "integrity": "sha512-sRdU18mcKf7F+YgheI/zGf5alZatMUTKj/jNS6l744f9u3WFu4v7twcUI9vu4mknF4Y9aDlblIie0IM+5xxaqQ==", - "cpu": [ - "x64" - ], - "dev": true, - "license": "MIT", - "optional": true, - "os": [ - "win32" - ], - "engines": { - "node": ">=18" - } - }, - "node_modules/@modelcontextprotocol/sdk": { - "version": "0.5.0", - "resolved": "https://registry.npmmirror.com/@modelcontextprotocol/sdk/-/sdk-0.5.0.tgz", - "integrity": "sha512-RXgulUX6ewvxjAG0kOpLMEdXXWkzWgaoCGaA2CwNW7cQCIphjpJhjpHSiaPdVCnisjRF/0Cm9KWHUuIoeiAblQ==", - "license": "MIT", - "dependencies": { - "content-type": "^1.0.5", - "raw-body": "^3.0.0", - "zod": "^3.23.8" - } - }, - "node_modules/@types/node": { - "version": "20.19.27", - "resolved": "https://registry.npmmirror.com/@types/node/-/node-20.19.27.tgz", - "integrity": "sha512-N2clP5pJhB2YnZJ3PIHFk5RkygRX5WO/5f0WC08tp0wd+sv0rsJk3MqWn3CbNmT2J505a5336jaQj4ph1AdMug==", - "dev": true, - "license": "MIT", - "dependencies": { - "undici-types": "~6.21.0" - } - }, - "node_modules/@types/node-fetch": { - "version": "2.6.13", - "resolved": "https://registry.npmmirror.com/@types/node-fetch/-/node-fetch-2.6.13.tgz", - "integrity": "sha512-QGpRVpzSaUs30JBSGPjOg4Uveu384erbHBoT1zeONvyCfwQxIkUshLAOqN/k9EjGviPRmWTTe6aH2qySWKTVSw==", - "dev": true, - "license": "MIT", - "dependencies": { - "@types/node": "*", - "form-data": "^4.0.4" - } - }, - "node_modules/asynckit": { - "version": "0.4.0", - "resolved": "https://registry.npmmirror.com/asynckit/-/asynckit-0.4.0.tgz", - "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==", - "dev": true, - "license": "MIT" - }, - "node_modules/bytes": { - "version": "3.1.2", - "resolved": "https://registry.npmmirror.com/bytes/-/bytes-3.1.2.tgz", - "integrity": "sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg==", - "license": "MIT", - "engines": { - "node": ">= 0.8" - } - }, - "node_modules/call-bind-apply-helpers": { - "version": "1.0.2", - "resolved": "https://registry.npmmirror.com/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz", - "integrity": "sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==", - "dev": true, - "license": "MIT", - "dependencies": { - "es-errors": "^1.3.0", - "function-bind": "^1.1.2" - }, - "engines": { - "node": ">= 0.4" - } - }, - "node_modules/combined-stream": { - "version": "1.0.8", - "resolved": "https://registry.npmmirror.com/combined-stream/-/combined-stream-1.0.8.tgz", - "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==", - "dev": true, - "license": "MIT", - "dependencies": { - "delayed-stream": "~1.0.0" - }, - "engines": { - "node": ">= 0.8" - } - }, - "node_modules/content-type": { - "version": "1.0.5", - "resolved": "https://registry.npmmirror.com/content-type/-/content-type-1.0.5.tgz", - "integrity": "sha512-nTjqfcBFEipKdXCv4YDQWCfmcLZKm81ldF0pAopTvyrFGVbcR6P/VAAd5G7N+0tTr8QqiU0tFadD6FK4NtJwOA==", - "license": "MIT", - "engines": { - "node": ">= 0.6" - } - }, - "node_modules/data-uri-to-buffer": { - "version": "4.0.1", - "resolved": "https://registry.npmmirror.com/data-uri-to-buffer/-/data-uri-to-buffer-4.0.1.tgz", - "integrity": "sha512-0R9ikRb668HB7QDxT1vkpuUBtqc53YyAwMwGeUFKRojY/NWKvdZ+9UYtRfGmhqNbRkTSVpMbmyhXipFFv2cb/A==", - "license": "MIT", - "engines": { - "node": ">= 12" - } - }, - "node_modules/delayed-stream": { - "version": "1.0.0", - "resolved": "https://registry.npmmirror.com/delayed-stream/-/delayed-stream-1.0.0.tgz", - "integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==", - "dev": true, - "license": "MIT", - "engines": { - "node": ">=0.4.0" - } - }, - "node_modules/depd": { - "version": "2.0.0", - "resolved": "https://registry.npmmirror.com/depd/-/depd-2.0.0.tgz", - "integrity": "sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw==", - "license": "MIT", - "engines": { - "node": ">= 0.8" - } - }, - "node_modules/dunder-proto": { - "version": "1.0.1", - "resolved": "https://registry.npmmirror.com/dunder-proto/-/dunder-proto-1.0.1.tgz", - "integrity": "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==", - "dev": true, - "license": "MIT", - "dependencies": { - "call-bind-apply-helpers": "^1.0.1", - "es-errors": "^1.3.0", - "gopd": "^1.2.0" - }, - "engines": { - "node": ">= 0.4" - } - }, - "node_modules/es-define-property": { - "version": "1.0.1", - "resolved": "https://registry.npmmirror.com/es-define-property/-/es-define-property-1.0.1.tgz", - "integrity": "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==", - "dev": true, - "license": "MIT", - "engines": { - "node": ">= 0.4" - } - }, - "node_modules/es-errors": { - "version": "1.3.0", - "resolved": "https://registry.npmmirror.com/es-errors/-/es-errors-1.3.0.tgz", - "integrity": "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw==", - "dev": true, - "license": "MIT", - "engines": { - "node": ">= 0.4" - } - }, - "node_modules/es-object-atoms": { - "version": "1.1.1", - "resolved": "https://registry.npmmirror.com/es-object-atoms/-/es-object-atoms-1.1.1.tgz", - "integrity": "sha512-FGgH2h8zKNim9ljj7dankFPcICIK9Cp5bm+c2gQSYePhpaG5+esrLODihIorn+Pe6FGJzWhXQotPv73jTaldXA==", - "dev": true, - "license": "MIT", - "dependencies": { - "es-errors": "^1.3.0" - }, - "engines": { - "node": ">= 0.4" - } - }, - "node_modules/es-set-tostringtag": { - "version": "2.1.0", - "resolved": "https://registry.npmmirror.com/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz", - "integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==", - "dev": true, - "license": "MIT", - "dependencies": { - "es-errors": "^1.3.0", - "get-intrinsic": "^1.2.6", - "has-tostringtag": "^1.0.2", - "hasown": "^2.0.2" - }, - "engines": { - "node": ">= 0.4" - } - }, - "node_modules/esbuild": { - "version": "0.27.2", - "resolved": "https://registry.npmmirror.com/esbuild/-/esbuild-0.27.2.tgz", - "integrity": "sha512-HyNQImnsOC7X9PMNaCIeAm4ISCQXs5a5YasTXVliKv4uuBo1dKrG0A+uQS8M5eXjVMnLg3WgXaKvprHlFJQffw==", - "dev": true, - "hasInstallScript": true, - "license": "MIT", - "bin": { - "esbuild": "bin/esbuild" - }, - "engines": { - "node": ">=18" - }, - "optionalDependencies": { - "@esbuild/aix-ppc64": "0.27.2", - "@esbuild/android-arm": "0.27.2", - "@esbuild/android-arm64": "0.27.2", - "@esbuild/android-x64": "0.27.2", - "@esbuild/darwin-arm64": "0.27.2", - "@esbuild/darwin-x64": "0.27.2", - "@esbuild/freebsd-arm64": "0.27.2", - "@esbuild/freebsd-x64": "0.27.2", - "@esbuild/linux-arm": "0.27.2", - "@esbuild/linux-arm64": "0.27.2", - "@esbuild/linux-ia32": "0.27.2", - "@esbuild/linux-loong64": "0.27.2", - "@esbuild/linux-mips64el": "0.27.2", - "@esbuild/linux-ppc64": "0.27.2", - "@esbuild/linux-riscv64": "0.27.2", - "@esbuild/linux-s390x": "0.27.2", - "@esbuild/linux-x64": "0.27.2", - "@esbuild/netbsd-arm64": "0.27.2", - "@esbuild/netbsd-x64": "0.27.2", - "@esbuild/openbsd-arm64": "0.27.2", - "@esbuild/openbsd-x64": "0.27.2", - "@esbuild/openharmony-arm64": "0.27.2", - "@esbuild/sunos-x64": "0.27.2", - "@esbuild/win32-arm64": "0.27.2", - "@esbuild/win32-ia32": "0.27.2", - "@esbuild/win32-x64": "0.27.2" - } - }, - "node_modules/fetch-blob": { - "version": "3.2.0", - "resolved": "https://registry.npmmirror.com/fetch-blob/-/fetch-blob-3.2.0.tgz", - "integrity": "sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ==", - "funding": [ - { - "type": "github", - "url": "https://github.com/sponsors/jimmywarting" - }, - { - "type": "paypal", - "url": "https://paypal.me/jimmywarting" - } - ], - "license": "MIT", - "dependencies": { - "node-domexception": "^1.0.0", - "web-streams-polyfill": "^3.0.3" - }, - "engines": { - "node": "^12.20 || >= 14.13" - } - }, - "node_modules/form-data": { - "version": "4.0.5", - "resolved": "https://registry.npmmirror.com/form-data/-/form-data-4.0.5.tgz", - "integrity": "sha512-8RipRLol37bNs2bhoV67fiTEvdTrbMUYcFTiy3+wuuOnUog2QBHCZWXDRijWQfAkhBj2Uf5UnVaiWwA5vdd82w==", - "dev": true, - "license": "MIT", - "dependencies": { - "asynckit": "^0.4.0", - "combined-stream": "^1.0.8", - "es-set-tostringtag": "^2.1.0", - "hasown": "^2.0.2", - "mime-types": "^2.1.12" - }, - "engines": { - "node": ">= 6" - } - }, - "node_modules/formdata-polyfill": { - "version": "4.0.10", - "resolved": "https://registry.npmmirror.com/formdata-polyfill/-/formdata-polyfill-4.0.10.tgz", - "integrity": "sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g==", - "license": "MIT", - "dependencies": { - "fetch-blob": "^3.1.2" - }, - "engines": { - "node": ">=12.20.0" - } - }, - "node_modules/fsevents": { - "version": "2.3.3", - "resolved": "https://registry.npmmirror.com/fsevents/-/fsevents-2.3.3.tgz", - "integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==", - "dev": true, - "hasInstallScript": true, - "license": "MIT", - "optional": true, - "os": [ - "darwin" - ], - "engines": { - "node": "^8.16.0 || ^10.6.0 || >=11.0.0" - } - }, - "node_modules/function-bind": { - "version": "1.1.2", - "resolved": "https://registry.npmmirror.com/function-bind/-/function-bind-1.1.2.tgz", - "integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==", - "dev": true, - "license": "MIT", - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, - "node_modules/get-intrinsic": { - "version": "1.3.0", - "resolved": "https://registry.npmmirror.com/get-intrinsic/-/get-intrinsic-1.3.0.tgz", - "integrity": "sha512-9fSjSaos/fRIVIp+xSJlE6lfwhES7LNtKaCBIamHsjr2na1BiABJPo0mOjjz8GJDURarmCPGqaiVg5mfjb98CQ==", - "dev": true, - "license": "MIT", - "dependencies": { - "call-bind-apply-helpers": "^1.0.2", - "es-define-property": "^1.0.1", - "es-errors": "^1.3.0", - "es-object-atoms": "^1.1.1", - "function-bind": "^1.1.2", - "get-proto": "^1.0.1", - "gopd": "^1.2.0", - "has-symbols": "^1.1.0", - "hasown": "^2.0.2", - "math-intrinsics": "^1.1.0" - }, - "engines": { - "node": ">= 0.4" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, - "node_modules/get-proto": { - "version": "1.0.1", - "resolved": "https://registry.npmmirror.com/get-proto/-/get-proto-1.0.1.tgz", - "integrity": "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==", - "dev": true, - "license": "MIT", - "dependencies": { - "dunder-proto": "^1.0.1", - "es-object-atoms": "^1.0.0" - }, - "engines": { - "node": ">= 0.4" - } - }, - "node_modules/get-tsconfig": { - "version": "4.13.0", - "resolved": "https://registry.npmmirror.com/get-tsconfig/-/get-tsconfig-4.13.0.tgz", - "integrity": "sha512-1VKTZJCwBrvbd+Wn3AOgQP/2Av+TfTCOlE4AcRJE72W1ksZXbAx8PPBR9RzgTeSPzlPMHrbANMH3LbltH73wxQ==", - "dev": true, - "license": "MIT", - "dependencies": { - "resolve-pkg-maps": "^1.0.0" - }, - "funding": { - "url": "https://github.com/privatenumber/get-tsconfig?sponsor=1" - } - }, - "node_modules/gopd": { - "version": "1.2.0", - "resolved": "https://registry.npmmirror.com/gopd/-/gopd-1.2.0.tgz", - "integrity": "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==", - "dev": true, - "license": "MIT", - "engines": { - "node": ">= 0.4" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, - "node_modules/has-symbols": { - "version": "1.1.0", - "resolved": "https://registry.npmmirror.com/has-symbols/-/has-symbols-1.1.0.tgz", - "integrity": "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==", - "dev": true, - "license": "MIT", - "engines": { - "node": ">= 0.4" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, - "node_modules/has-tostringtag": { - "version": "1.0.2", - "resolved": "https://registry.npmmirror.com/has-tostringtag/-/has-tostringtag-1.0.2.tgz", - "integrity": "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==", - "dev": true, - "license": "MIT", - "dependencies": { - "has-symbols": "^1.0.3" - }, - "engines": { - "node": ">= 0.4" - }, - "funding": { - "url": "https://github.com/sponsors/ljharb" - } - }, - "node_modules/hasown": { - "version": "2.0.2", - "resolved": "https://registry.npmmirror.com/hasown/-/hasown-2.0.2.tgz", - "integrity": "sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ==", - "dev": true, - "license": "MIT", - "dependencies": { - "function-bind": "^1.1.2" - }, - "engines": { - "node": ">= 0.4" - } - }, - "node_modules/http-errors": { - "version": "2.0.1", - "resolved": "https://registry.npmmirror.com/http-errors/-/http-errors-2.0.1.tgz", - "integrity": "sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ==", - "license": "MIT", - "dependencies": { - "depd": "~2.0.0", - "inherits": "~2.0.4", - "setprototypeof": "~1.2.0", - "statuses": "~2.0.2", - "toidentifier": "~1.0.1" - }, - "engines": { - "node": ">= 0.8" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/express" - } - }, - "node_modules/iconv-lite": { - "version": "0.7.1", - "resolved": "https://registry.npmmirror.com/iconv-lite/-/iconv-lite-0.7.1.tgz", - "integrity": "sha512-2Tth85cXwGFHfvRgZWszZSvdo+0Xsqmw8k8ZwxScfcBneNUraK+dxRxRm24nszx80Y0TVio8kKLt5sLE7ZCLlw==", - "license": "MIT", - "dependencies": { - "safer-buffer": ">= 2.1.2 < 3.0.0" - }, - "engines": { - "node": ">=0.10.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/express" - } - }, - "node_modules/inherits": { - "version": "2.0.4", - "resolved": "https://registry.npmmirror.com/inherits/-/inherits-2.0.4.tgz", - "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==", - "license": "ISC" - }, - "node_modules/math-intrinsics": { - "version": "1.1.0", - "resolved": "https://registry.npmmirror.com/math-intrinsics/-/math-intrinsics-1.1.0.tgz", - "integrity": "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==", - "dev": true, - "license": "MIT", - "engines": { - "node": ">= 0.4" - } - }, - "node_modules/mime-db": { - "version": "1.52.0", - "resolved": "https://registry.npmmirror.com/mime-db/-/mime-db-1.52.0.tgz", - "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==", - "dev": true, - "license": "MIT", - "engines": { - "node": ">= 0.6" - } - }, - "node_modules/mime-types": { - "version": "2.1.35", - "resolved": "https://registry.npmmirror.com/mime-types/-/mime-types-2.1.35.tgz", - "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==", - "dev": true, - "license": "MIT", - "dependencies": { - "mime-db": "1.52.0" - }, - "engines": { - "node": ">= 0.6" - } - }, - "node_modules/node-domexception": { - "version": "1.0.0", - "resolved": "https://registry.npmmirror.com/node-domexception/-/node-domexception-1.0.0.tgz", - "integrity": "sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ==", - "deprecated": "Use your platform's native DOMException instead", - "funding": [ - { - "type": "github", - "url": "https://github.com/sponsors/jimmywarting" - }, - { - "type": "github", - "url": "https://paypal.me/jimmywarting" - } - ], - "license": "MIT", - "engines": { - "node": ">=10.5.0" - } - }, - "node_modules/node-fetch": { - "version": "3.3.2", - "resolved": "https://registry.npmmirror.com/node-fetch/-/node-fetch-3.3.2.tgz", - "integrity": "sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA==", - "license": "MIT", - "dependencies": { - "data-uri-to-buffer": "^4.0.0", - "fetch-blob": "^3.1.4", - "formdata-polyfill": "^4.0.10" - }, - "engines": { - "node": "^12.20.0 || ^14.13.1 || >=16.0.0" - }, - "funding": { - "type": "opencollective", - "url": "https://opencollective.com/node-fetch" - } - }, - "node_modules/raw-body": { - "version": "3.0.2", - "resolved": "https://registry.npmmirror.com/raw-body/-/raw-body-3.0.2.tgz", - "integrity": "sha512-K5zQjDllxWkf7Z5xJdV0/B0WTNqx6vxG70zJE4N0kBs4LovmEYWJzQGxC9bS9RAKu3bgM40lrd5zoLJ12MQ5BA==", - "license": "MIT", - "dependencies": { - "bytes": "~3.1.2", - "http-errors": "~2.0.1", - "iconv-lite": "~0.7.0", - "unpipe": "~1.0.0" - }, - "engines": { - "node": ">= 0.10" - } - }, - "node_modules/resolve-pkg-maps": { - "version": "1.0.0", - "resolved": "https://registry.npmmirror.com/resolve-pkg-maps/-/resolve-pkg-maps-1.0.0.tgz", - "integrity": "sha512-seS2Tj26TBVOC2NIc2rOe2y2ZO7efxITtLZcGSOnHHNOQ7CkiUBfw0Iw2ck6xkIhPwLhKNLS8BO+hEpngQlqzw==", - "dev": true, - "license": "MIT", - "funding": { - "url": "https://github.com/privatenumber/resolve-pkg-maps?sponsor=1" - } - }, - "node_modules/safer-buffer": { - "version": "2.1.2", - "resolved": "https://registry.npmmirror.com/safer-buffer/-/safer-buffer-2.1.2.tgz", - "integrity": "sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg==", - "license": "MIT" - }, - "node_modules/setprototypeof": { - "version": "1.2.0", - "resolved": "https://registry.npmmirror.com/setprototypeof/-/setprototypeof-1.2.0.tgz", - "integrity": "sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw==", - "license": "ISC" - }, - "node_modules/statuses": { - "version": "2.0.2", - "resolved": "https://registry.npmmirror.com/statuses/-/statuses-2.0.2.tgz", - "integrity": "sha512-DvEy55V3DB7uknRo+4iOGT5fP1slR8wQohVdknigZPMpMstaKJQWhwiYBACJE3Ul2pTnATihhBYnRhZQHGBiRw==", - "license": "MIT", - "engines": { - "node": ">= 0.8" - } - }, - "node_modules/toidentifier": { - "version": "1.0.1", - "resolved": "https://registry.npmmirror.com/toidentifier/-/toidentifier-1.0.1.tgz", - "integrity": "sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA==", - "license": "MIT", - "engines": { - "node": ">=0.6" - } - }, - "node_modules/tsx": { - "version": "4.21.0", - "resolved": "https://registry.npmmirror.com/tsx/-/tsx-4.21.0.tgz", - "integrity": "sha512-5C1sg4USs1lfG0GFb2RLXsdpXqBSEhAaA/0kPL01wxzpMqLILNxIxIOKiILz+cdg/pLnOUxFYOR5yhHU666wbw==", - "dev": true, - "license": "MIT", - "dependencies": { - "esbuild": "~0.27.0", - "get-tsconfig": "^4.7.5" - }, - "bin": { - "tsx": "dist/cli.mjs" - }, - "engines": { - "node": ">=18.0.0" - }, - "optionalDependencies": { - "fsevents": "~2.3.3" - } - }, - "node_modules/typescript": { - "version": "5.9.3", - "resolved": "https://registry.npmmirror.com/typescript/-/typescript-5.9.3.tgz", - "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", - "dev": true, - "license": "Apache-2.0", - "bin": { - "tsc": "bin/tsc", - "tsserver": "bin/tsserver" - }, - "engines": { - "node": ">=14.17" - } - }, - "node_modules/undici-types": { - "version": "6.21.0", - "resolved": "https://registry.npmmirror.com/undici-types/-/undici-types-6.21.0.tgz", - "integrity": "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ==", - "dev": true, - "license": "MIT" - }, - "node_modules/unpipe": { - "version": "1.0.0", - "resolved": "https://registry.npmmirror.com/unpipe/-/unpipe-1.0.0.tgz", - "integrity": "sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ==", - "license": "MIT", - "engines": { - "node": ">= 0.8" - } - }, - "node_modules/web-streams-polyfill": { - "version": "3.3.3", - "resolved": "https://registry.npmmirror.com/web-streams-polyfill/-/web-streams-polyfill-3.3.3.tgz", - "integrity": "sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw==", - "license": "MIT", - "engines": { - "node": ">= 8" - } - }, - "node_modules/zod": { - "version": "3.25.76", - "resolved": "https://registry.npmmirror.com/zod/-/zod-3.25.76.tgz", - "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", - "license": "MIT", - "funding": { - "url": "https://github.com/sponsors/colinhacks" - } - } - } -} diff --git a/Moncter/Moncter/MCP/moncter-mcp-server/package.json b/Moncter/Moncter/MCP/moncter-mcp-server/package.json deleted file mode 100644 index 6a6c17fc..00000000 --- a/Moncter/Moncter/MCP/moncter-mcp-server/package.json +++ /dev/null @@ -1,26 +0,0 @@ -{ - "name": "moncter-mcp-server", - "version": "1.0.0", - "description": "MCP Server for Moncter - Data Collection and Tag Task Management", - "main": "dist/index.js", - "type": "module", - "scripts": { - "build": "tsc", - "start": "node dist/index.js", - "dev": "tsx src/index.ts" - }, - "keywords": ["mcp", "moncter", "data-collection", "tag-task"], - "author": "", - "license": "MIT", - "dependencies": { - "@modelcontextprotocol/sdk": "^0.5.0", - "node-fetch": "^3.3.2" - }, - "devDependencies": { - "@types/node": "^20.10.0", - "@types/node-fetch": "^2.6.11", - "tsx": "^4.7.0", - "typescript": "^5.3.3" - } -} - diff --git a/Moncter/Moncter/MCP/moncter-mcp-server/src/index.ts b/Moncter/Moncter/MCP/moncter-mcp-server/src/index.ts deleted file mode 100644 index 473de05a..00000000 --- a/Moncter/Moncter/MCP/moncter-mcp-server/src/index.ts +++ /dev/null @@ -1,595 +0,0 @@ -#!/usr/bin/env node - -import { Server } from '@modelcontextprotocol/sdk/server/index.js'; -import { StdioServerTransport } from '@modelcontextprotocol/sdk/server/stdio.js'; -import { - CallToolRequestSchema, - ListToolsRequestSchema, -} from '@modelcontextprotocol/sdk/types.js'; - -// 获取后端API基础URL（从环境变量或默认值） -const API_BASE_URL = process.env.MONCTER_API_URL || 'http://127.0.0.1:8787'; - -/** - * HTTP请求辅助函数 - */ -async function apiRequest( - method: string, - endpoint: string, - data?: any -): Promise { - const url = `${API_BASE_URL}${endpoint}`; - const options: RequestInit = { - method, - headers: { - 'Content-Type': 'application/json', - }, - }; - - if (data && (method === 'POST' || method === 'PUT')) { - options.body = JSON.stringify(data); - } - - try { - const response = await fetch(url, options); - const result = await response.json() as any; - - if (result.code !== 0 && result.code !== undefined) { - throw new Error(result.message || 'API请求失败'); - } - - return result.data || result; - } catch (error: any) { - throw new Error(`API请求错误: ${error.message}`); - } -} - -/** - * 创建MCP服务器 - */ -const server = new Server( - { - name: 'moncter-mcp-server', - version: '1.0.0', - }, - { - capabilities: { - tools: {}, - }, - } -); - -// 列出可用工具 -server.setRequestHandler(ListToolsRequestSchema, async () => { - return { - tools: [ - { - name: 'create_data_collection_task', - description: '创建数据采集任务。用于从数据源采集数据并转换为消费记录或其他格式。支持单集合和多集合模式，支持连表查询和过滤条件。', - inputSchema: { - type: 'object', - properties: { - name: { - type: 'string', - description: '任务名称（必填）', - }, - description: { - type: 'string', - description: '任务描述（可选）', - }, - data_source_id: { - type: 'string', - description: '源数据源ID（必填）', - }, - database: { - type: 'string', - description: '源数据库名称（必填）', - }, - collection: { - type: 'string', - description: '源集合名称（单集合模式，与collections二选一）', - }, - collections: { - type: 'array', - items: { type: 'string' }, - description: '源集合列表（多集合模式，与collection二选一）', - }, - multi_collection: { - type: 'boolean', - description: '是否启用多集合模式。true=使用collections字段，false=使用collection字段', - }, - target_type: { - type: 'string', - enum: ['consumption_record', 'generic'], - description: '目标类型（必填）：consumption_record=消费记录处理（自动转换格式，通过手机号解析用户ID，时间分片存储到标签引擎数据库），generic=通用集合处理（需要自定义字段映射和目标存储配置）', - }, - mode: { - type: 'string', - enum: ['batch', 'realtime'], - description: '采集模式（必填）：batch=批量采集（定时执行），realtime=实时监听（持续监听数据变化）', - }, - field_mappings: { - type: 'array', - description: '字段映射配置（单集合模式使用），将源字段映射到目标字段', - items: { - type: 'object', - properties: { - source_field: { - type: 'string', - description: '源字段名（查询结果中的字段）' - }, - target_field: { - type: 'string', - description: '目标字段名（Handler需要的字段名）' - }, - transform: { - type: 'string', - enum: ['parse_amount', 'parse_datetime', 'parse_phone'], - description: '转换函数（可选）：parse_amount=解析金额，parse_datetime=解析日期时间，parse_phone=解析手机号' - }, - }, - required: ['source_field', 'target_field'], - }, - }, - collection_field_mappings: { - type: 'object', - description: '字段映射配置（多集合模式使用），格式：{ "collection_name": [FieldMapping] }，每个集合可配置独立的字段映射', - additionalProperties: { - type: 'array', - items: { - type: 'object', - properties: { - source_field: { type: 'string' }, - target_field: { type: 'string' }, - transform: { type: 'string' }, - }, - }, - }, - }, - lookups: { - type: 'array', - description: 'MongoDB $lookup连表查询配置（单集合模式使用，可选），可以从其他集合关联数据', - items: { - type: 'object', - properties: { - from: { - type: 'string', - description: '关联集合名' - }, - local_field: { - type: 'string', - description: '主集合字段（用于关联的字段）' - }, - foreign_field: { - type: 'string', - description: '关联集合字段（被关联集合的字段，通常是_id）' - }, - as: { - type: 'string', - description: '结果字段名（关联结果存储的字段名）' - }, - unwrap: { - type: 'boolean', - description: '是否解构（true=解构后可直接使用user_info.mobile，false=返回数组）' - }, - preserve_null: { - type: 'boolean', - description: '是否保留空值（当关联不到数据时是否保留）' - }, - }, - required: ['from', 'local_field', 'foreign_field', 'as'], - }, - }, - collection_lookups: { - type: 'object', - description: 'MongoDB $lookup连表查询配置（多集合模式使用，可选），格式：{ "collection_name": [LookupConfig] }，每个集合可配置独立的连表查询', - additionalProperties: { - type: 'array', - items: { - type: 'object', - properties: { - from: { type: 'string' }, - local_field: { type: 'string' }, - foreign_field: { type: 'string' }, - as: { type: 'string' }, - unwrap: { type: 'boolean' }, - preserve_null: { type: 'boolean' }, - }, - }, - }, - }, - filter_conditions: { - type: 'array', - description: '过滤条件（可选），只采集满足条件的数据', - items: { - type: 'object', - properties: { - field: { - type: 'string', - description: '字段名' - }, - operator: { - type: 'string', - enum: ['eq', 'ne', 'gt', 'gte', 'lt', 'lte', 'in', 'nin'], - description: '运算符：eq=等于，ne=不等于，gt=大于，gte=大于等于，lt=小于，lte=小于等于，in=在列表中，nin=不在列表中' - }, - value: { - type: ['string', 'number', 'boolean', 'array'], - description: '值（可以是字符串、数字、布尔值或数组）' - }, - }, - required: ['field', 'operator', 'value'], - }, - }, - schedule: { - type: 'object', - description: '调度配置（批量模式batch时使用）', - properties: { - enabled: { - type: 'boolean', - description: '是否启用调度（批量模式时可启用Cron定时执行）' - }, - cron: { - type: 'string', - description: 'Cron表达式（格式：分 时 日 月 周，例如：0 2 * * * 表示每天凌晨2点执行）' - }, - }, - }, - }, - required: ['name', 'data_source_id', 'database', 'mode', 'target_type'], - }, - }, - { - name: 'create_tag_task', - description: '创建标签计算任务。用于批量计算用户标签。', - inputSchema: { - type: 'object', - properties: { - name: { - type: 'string', - description: '任务名称', - }, - description: { - type: 'string', - description: '任务描述', - }, - task_type: { - type: 'string', - enum: ['full', 'incremental', 'specific'], - description: '任务类型：full=全量计算，incremental=增量计算，specific=指定用户', - }, - target_tag_ids: { - type: 'array', - items: { type: 'string' }, - description: '要计算的标签ID列表', - }, - user_scope: { - type: 'object', - properties: { - type: { - type: 'string', - enum: ['all', 'list', 'filter'], - description: '用户范围类型', - }, - user_ids: { - type: 'array', - items: { type: 'string' }, - description: '用户ID列表（当type=list时）', - }, - filter_conditions: { - type: 'array', - description: '筛选条件（当type=filter时）', - items: { - type: 'object', - properties: { - field: { type: 'string' }, - operator: { type: 'string' }, - value: { type: 'string' }, - }, - }, - }, - }, - }, - schedule: { - type: 'object', - properties: { - enabled: { type: 'boolean' }, - cron: { type: 'string' }, - }, - }, - config: { - type: 'object', - properties: { - concurrency: { type: 'number' }, - batch_size: { type: 'number' }, - error_handling: { - type: 'string', - enum: ['skip', 'stop', 'retry'], - }, - }, - }, - }, - required: ['name', 'task_type', 'target_tag_ids'], - }, - }, - { - name: 'list_data_collection_tasks', - description: '获取数据采集任务列表', - inputSchema: { - type: 'object', - properties: { - page: { type: 'number', description: '页码' }, - page_size: { type: 'number', description: '每页数量' }, - }, - }, - }, - { - name: 'list_tag_tasks', - description: '获取标签任务列表', - inputSchema: { - type: 'object', - properties: { - page: { type: 'number', description: '页码' }, - page_size: { type: 'number', description: '每页数量' }, - }, - }, - }, - { - name: 'get_data_sources', - description: '获取数据源列表', - inputSchema: { - type: 'object', - properties: { - type: { type: 'string', description: '数据源类型筛选' }, - status: { type: 'number', description: '状态筛选：1=启用，0=禁用' }, - }, - }, - }, - { - name: 'get_tag_definitions', - description: '获取标签定义列表', - inputSchema: { - type: 'object', - properties: { - status: { type: 'number', description: '状态筛选：1=启用，0=禁用' }, - }, - }, - }, - { - name: 'start_data_collection_task', - description: '启动数据采集任务', - inputSchema: { - type: 'object', - properties: { - task_id: { - type: 'string', - description: '任务ID', - }, - }, - required: ['task_id'], - }, - }, - { - name: 'start_tag_task', - description: '启动标签任务', - inputSchema: { - type: 'object', - properties: { - task_id: { - type: 'string', - description: '任务ID', - }, - }, - required: ['task_id'], - }, - }, - ], - }; -}); - -// 处理工具调用 -server.setRequestHandler(CallToolRequestSchema, async (request) => { - const { name, arguments: args } = request.params; - - try { - switch (name) { - case 'create_data_collection_task': { - const result = await apiRequest( - 'POST', - '/api/data-collection-tasks', - args - ); - return { - content: [ - { - type: 'text', - text: JSON.stringify( - { - success: true, - message: '数据采集任务创建成功', - data: result, - }, - null, - 2 - ), - }, - ], - }; - } - - case 'create_tag_task': { - const result = await apiRequest('POST', '/api/tag-tasks', args); - return { - content: [ - { - type: 'text', - text: JSON.stringify( - { - success: true, - message: '标签任务创建成功', - data: result, - }, - null, - 2 - ), - }, - ], - }; - } - - case 'list_data_collection_tasks': { - const params = new URLSearchParams(); - if (args?.page) params.append('page', String(args.page)); - if (args?.page_size) params.append('page_size', String(args.page_size)); - const query = params.toString(); - const endpoint = `/api/data-collection-tasks${query ? `?${query}` : ''}`; - const result = await apiRequest('GET', endpoint); - return { - content: [ - { - type: 'text', - text: JSON.stringify(result, null, 2), - }, - ], - }; - } - - case 'list_tag_tasks': { - const params = new URLSearchParams(); - if (args?.page) params.append('page', String(args.page)); - if (args?.page_size) params.append('page_size', String(args.page_size)); - const query = params.toString(); - const endpoint = `/api/tag-tasks${query ? `?${query}` : ''}`; - const result = await apiRequest('GET', endpoint); - return { - content: [ - { - type: 'text', - text: JSON.stringify(result, null, 2), - }, - ], - }; - } - - case 'get_data_sources': { - const params = new URLSearchParams(); - if (args?.type) params.append('type', String(args.type)); - if (args?.status !== undefined) params.append('status', String(args.status)); - const query = params.toString(); - const endpoint = `/api/data-sources${query ? `?${query}` : ''}`; - const result = await apiRequest('GET', endpoint); - return { - content: [ - { - type: 'text', - text: JSON.stringify(result, null, 2), - }, - ], - }; - } - - case 'get_tag_definitions': { - const params = new URLSearchParams(); - if (args?.status !== undefined) params.append('status', String(args.status)); - const query = params.toString(); - const endpoint = `/api/tag-definitions${query ? `?${query}` : ''}`; - const result = await apiRequest('GET', endpoint); - return { - content: [ - { - type: 'text', - text: JSON.stringify(result, null, 2), - }, - ], - }; - } - - case 'start_data_collection_task': { - if (!args?.task_id) { - throw new Error('缺少必需参数: task_id'); - } - const result = await apiRequest( - 'POST', - `/api/data-collection-tasks/${args.task_id}/start`, - {} - ); - return { - content: [ - { - type: 'text', - text: JSON.stringify( - { - success: true, - message: '数据采集任务启动成功', - data: result, - }, - null, - 2 - ), - }, - ], - }; - } - - case 'start_tag_task': { - if (!args?.task_id) { - throw new Error('缺少必需参数: task_id'); - } - const result = await apiRequest( - 'POST', - `/api/tag-tasks/${args.task_id}/start`, - {} - ); - return { - content: [ - { - type: 'text', - text: JSON.stringify( - { - success: true, - message: '标签任务启动成功', - data: result, - }, - null, - 2 - ), - }, - ], - }; - } - - default: - throw new Error(`未知的工具: ${name}`); - } - } catch (error: any) { - return { - content: [ - { - type: 'text', - text: JSON.stringify( - { - success: false, - error: error.message, - }, - null, - 2 - ), - }, - ], - isError: true, - }; - } -}); - -// 启动服务器 -async function main() { - const transport = new StdioServerTransport(); - await server.connect(transport); - console.error('Moncter MCP Server running on stdio'); -} - -main().catch((error) => { - console.error('服务器启动失败:', error); - process.exit(1); -}); - diff --git a/Moncter/Moncter/MCP/moncter-mcp-server/tsconfig.json b/Moncter/Moncter/MCP/moncter-mcp-server/tsconfig.json deleted file mode 100644 index 65824748..00000000 --- a/Moncter/Moncter/MCP/moncter-mcp-server/tsconfig.json +++ /dev/null @@ -1,20 +0,0 @@ -{ - "compilerOptions": { - "target": "ES2022", - "module": "ES2022", - "lib": ["ES2022"], - "moduleResolution": "node", - "outDir": "./dist", - "rootDir": "./src", - "strict": true, - "esModuleInterop": true, - "skipLibCheck": true, - "forceConsistentCasingInFileNames": true, - "resolveJsonModule": true, - "declaration": true, - "declarationMap": true, - "sourceMap": true - }, - "include": ["src/**/*"], - "exclude": ["node_modules", "dist"] -} diff --git a/Moncter/Moncter/MCP/实现总结.md b/Moncter/Moncter/MCP/实现总结.md deleted file mode 100644 index 6231f78c..00000000 --- a/Moncter/Moncter/MCP/实现总结.md +++ /dev/null @@ -1,164 +0,0 @@ -# Moncter MCP 服务器实现总结 - -## 一、实现概述 - -已成功为 Moncter 系统创建了一个 MCP (Model Context Protocol) 服务器，允许通过 MCP 协议来管理数据采集任务和标签任务。 - -## 二、已创建的文件 - -### 1. 核心代码 - -- `MCP/moncter-mcp-server/src/index.ts` - MCP 服务器主文件 -- `MCP/moncter-mcp-server/package.json` - Node.js 依赖配置 -- `MCP/moncter-mcp-server/tsconfig.json` - TypeScript 配置 - -### 2. 配置文件 - -- `MCP/mcp.json` - MCP 服务器配置（已更新，添加了 Moncter 服务器） -- `MCP/mcp.json.example` - 配置示例文件 - -### 3. 安装脚本 - -- `MCP/moncter-mcp-server/install.sh` - Linux/Mac 安装脚本 -- `MCP/moncter-mcp-server/install.bat` - Windows 安装脚本 - -### 4. 文档 - -- `MCP/README.md` - MCP 目录说明 -- `MCP/快速开始.md` - 快速开始指南 -- `MCP/MCP服务器使用说明.md` - 详细使用说明 -- `MCP/moncter-mcp-server/README.md` - 服务器开发文档 - -## 三、实现的 MCP 工具 - -### 1. 数据采集任务管理 - -- ✅ `create_data_collection_task` - 创建数据采集任务 -- ✅ `list_data_collection_tasks` - 获取数据采集任务列表 -- ✅ `start_data_collection_task` - 启动数据采集任务 - -### 2. 标签任务管理 - -- ✅ `create_tag_task` - 创建标签计算任务 -- ✅ `list_tag_tasks` - 获取标签任务列表 -- ✅ `start_tag_task` - 启动标签任务 - -### 3. 辅助工具 - -- ✅ `get_data_sources` - 获取数据源列表 -- ✅ `get_tag_definitions` - 获取标签定义列表 - -## 四、技术实现 - -### 架构设计 - -``` -MCP Client (Claude Desktop, etc.) - ↓ (stdio/stdin) -Moncter MCP Server (Node.js) - ↓ (HTTP REST API) -Moncter Backend (PHP/Webman) - ↓ -MongoDB / Redis / RabbitMQ -``` - -### 关键技术 - -- **MCP SDK**: 使用 `@modelcontextprotocol/sdk` 实现 MCP 协议 -- **HTTP 客户端**: 使用原生 `fetch` API（Node.js 18+） -- **TypeScript**: 类型安全的实现 -- **Stdio Transport**: 通过标准输入输出与 MCP 客户端通信 - -## 五、安装和使用步骤 - -### 1. 安装 - -```bash -cd MCP/moncter-mcp-server -npm install -npm run build -``` - -### 2. 配置 - -编辑 `MCP/mcp.json`，确保路径正确： - -```json -{ - "mcpServers": { - "Moncter": { - "command": "node", - "args": ["./MCP/moncter-mcp-server/dist/index.js"], - "cwd": "YOUR_PROJECT_PATH", - "env": { - "MONCTER_API_URL": "http://127.0.0.1:8787" - } - } - } -} -``` - -### 3. 使用 - -在支持 MCP 的客户端（如 Claude Desktop）中： -- 配置 MCP 服务器（引用 `mcp.json`） -- 重启客户端 -- 通过对话使用工具："创建一个数据采集任务..." - -## 六、功能特点 - -1. **完整的任务管理**: 支持创建、查询、启动数据采集任务和标签任务 -2. **参数验证**: 通过 JSON Schema 验证工具参数 -3. **错误处理**: 完善的错误处理和错误信息返回 -4. **类型安全**: 使用 TypeScript 确保类型安全 -5. **易于扩展**: 可以轻松添加新的 MCP 工具 - -## 七、后续扩展建议 - -### 可以添加的工具 - -1. **任务管理工具**: - - `update_data_collection_task` - 更新数据采集任务 - - `delete_data_collection_task` - 删除数据采集任务 - - `pause_data_collection_task` - 暂停数据采集任务 - - `stop_data_collection_task` - 停止数据采集任务 - - 类似的标签任务管理工具 - -2. **数据源管理工具**: - - `create_data_source` - 创建数据源 - - `update_data_source` - 更新数据源 - - `test_data_source_connection` - 测试数据源连接 - -3. **标签定义管理工具**: - - `create_tag_definition` - 创建标签定义 - - `update_tag_definition` - 更新标签定义 - -4. **查询工具**: - - `get_task_detail` - 获取任务详情 - - `get_task_progress` - 获取任务进度 - - `get_task_executions` - 获取任务执行记录 - -5. **批量操作工具**: - - `batch_create_tasks` - 批量创建任务 - - `batch_start_tasks` - 批量启动任务 - -## 八、注意事项 - -1. **Node.js 版本**: 需要 Node.js >= 18（使用原生 fetch API） -2. **后端服务**: 确保后端服务运行在配置的端口 -3. **路径配置**: MCP 配置中的路径需要根据实际情况修改 -4. **权限控制**: MCP 工具直接调用后端API，需要考虑权限控制 -5. **错误处理**: 工具调用失败时会返回错误信息，便于调试 - -## 九、测试建议 - -1. **单元测试**: 为各个工具函数编写单元测试 -2. **集成测试**: 测试与后端API的集成 -3. **端到端测试**: 使用 MCP Inspector 进行端到端测试 -4. **错误场景测试**: 测试各种错误场景的处理 - ---- - -**实现完成时间**: 2025-01-24 -**版本**: 1.0.0 - diff --git a/Moncter/Moncter/MCP/快速开始.md b/Moncter/Moncter/MCP/快速开始.md deleted file mode 100644 index cae31b21..00000000 --- a/Moncter/Moncter/MCP/快速开始.md +++ /dev/null @@ -1,212 +0,0 @@ -# Moncter MCP Server 快速开始 - -## 一、安装 MCP Server - -### Windows - -```bash -cd MCP/moncter-mcp-server -install.bat -``` - -### Linux/Mac - -```bash -cd MCP/moncter-mcp-server -chmod +x install.sh -./install.sh -``` - -### 手动安装 - -```bash -cd MCP/moncter-mcp-server -npm install -npm run build -``` - -## 二、配置 MCP 客户端 - -### 方式1：使用相对路径（推荐） - -编辑 `MCP/mcp.json`，使用相对于项目根目录的路径： - -```json -{ - "mcpServers": { - "Moncter": { - "command": "node", - "args": ["./MCP/moncter-mcp-server/dist/index.js"], - "cwd": "E:/Cunkebao/Cunkebao02/Moncter", - "env": { - "MONCTER_API_URL": "http://127.0.0.1:8787" - } - } - } -} -``` - -**注意**：`cwd` 需要修改为你的实际项目路径。 - -### 方式2：使用绝对路径 - -```json -{ - "mcpServers": { - "Moncter": { - "command": "node", - "args": ["E:/Cunkebao/Cunkebao02/Moncter/MCP/moncter-mcp-server/dist/index.js"], - "env": { - "MONCTER_API_URL": "http://127.0.0.1:8787" - } - } - } -} -``` - -### 方式3：使用 npx（如果发布到npm） - -```json -{ - "mcpServers": { - "Moncter": { - "command": "npx", - "args": ["-y", "moncter-mcp-server"], - "env": { - "MONCTER_API_URL": "http://127.0.0.1:8787" - } - } - } -} -``` - -## 三、确保后端服务运行 - -确保 Moncter 后端服务正在运行： - -```bash -# 检查服务状态 -php start.php status - -# 如果未运行，启动服务 -php start.php start -``` - -默认端口：`8787` - -## 四、测试 MCP Server - -### 在 Claude Desktop 中使用 - -1. 打开 Claude Desktop -2. 在设置中添加 MCP 服务器配置（引用 `mcp.json`） -3. 重启 Claude Desktop -4. 在对话中尝试："列出所有数据源" - -### 使用 MCP Inspector 测试 - -```bash -# 安装 MCP Inspector -npm install -g @modelcontextprotocol/inspector - -# 测试服务器 -cd MCP/moncter-mcp-server -npx @modelcontextprotocol/inspector node dist/index.js -``` - -## 五、使用示例 - -### 示例1：创建数据采集任务 - -对 AI 说： -> "创建一个实时监听的数据采集任务，名称为'订单采集'，从数据源'data_source_123'的数据库'KR_商城'的集合'21年贝蒂喜订单整合'采集数据" - -AI 会调用 `create_data_collection_task` 工具来创建任务。 - -### 示例2：创建标签任务 - -对 AI 说： -> "创建一个全量标签计算任务，名称为'高价值用户标签'，计算所有标签，每天凌晨2点执行" - -AI 会调用 `create_tag_task` 工具来创建任务。 - -### 示例3：查询数据源 - -对 AI 说： -> "列出所有启用的数据源" - -AI 会调用 `get_data_sources` 工具。 - -## 六、可用的工具列表 - -| 工具名称 | 功能 | 主要参数 | -|---------|------|---------| -| `create_data_collection_task` | 创建数据采集任务 | name, data_source_id, database, collection, mode | -| `create_tag_task` | 创建标签任务 | name, task_type, target_tag_ids | -| `list_data_collection_tasks` | 列出数据采集任务 | page, page_size | -| `list_tag_tasks` | 列出标签任务 | page, page_size | -| `get_data_sources` | 获取数据源列表 | type, status | -| `get_tag_definitions` | 获取标签定义列表 | status | -| `start_data_collection_task` | 启动数据采集任务 | task_id | -| `start_tag_task` | 启动标签任务 | task_id | - -详细参数说明请查看 `MCP/MCP服务器使用说明.md`。 - -## 七、故障排除 - -### 问题1：找不到模块 - -**错误**：`Cannot find module '@modelcontextprotocol/sdk'` - -**解决**： -```bash -cd MCP/moncter-mcp-server -npm install -``` - -### 问题2：编译失败 - -**错误**：TypeScript 编译错误 - -**解决**： -- 检查 Node.js 版本（需要 >= 18） -- 检查 TypeScript 版本 -- 运行 `npm install` 重新安装依赖 - -### 问题3：无法连接到后端 - -**错误**：`API请求错误: connect ECONNREFUSED` - -**解决**： -1. 检查后端服务是否运行：`php start.php status` -2. 检查 `MONCTER_API_URL` 环境变量是否正确 -3. 检查端口是否被占用：`netstat -ano | findstr :8787` - -### 问题4：路径错误 - -**错误**：`Cannot find module` 或路径相关错误 - -**解决**： -- 检查 `mcp.json` 中的路径是否正确 -- 使用绝对路径而不是相对路径 -- 确保 `cwd` 设置正确 - -## 八、开发调试 - -### 查看日志 - -MCP 服务器的错误日志会输出到 stderr，可以在 MCP 客户端中查看。 - -### 本地测试 - -```bash -cd MCP/moncter-mcp-server -npm run dev -``` - -然后使用 MCP Inspector 连接测试。 - ---- - -**需要帮助？** 查看 `MCP/MCP服务器使用说明.md` 获取详细文档。 - diff --git a/Moncter/Moncter/MCP/数据库账户密码.md b/Moncter/Moncter/MCP/数据库账户密码.md deleted file mode 100644 index 6d14bc26..00000000 --- a/Moncter/Moncter/MCP/数据库账户密码.md +++ /dev/null @@ -1,15 +0,0 @@ -# 账户密码 -``` -HOST 192.168.1.106 -PORT 27017 -ACCOUNT：ckb -DBNAME：ckb -PASSWROD：123456 -``` - -================== -# 环境准备 -需要安装一个全局环境否则无法运行 -node版本为22.12.0 - -npm i mongodb-mcp-server -g \ No newline at end of file diff --git a/Moncter/Moncter/README.md b/Moncter/Moncter/README.md deleted file mode 100644 index 4031784b..00000000 --- a/Moncter/Moncter/README.md +++ /dev/null @@ -1,70 +0,0 @@ -
-

webman

- -基于workerman开发的超高性能PHP框架 - - -

学习

- - - -
- -

赞助商

- -

特别赞助

- - - - -

铂金赞助

- - - - -
- - -
- -

请作者喝咖啡

- - - -
-如果您觉得webman对您有所帮助，欢迎捐赠。 - - -
- - -
-

LICENSE

-The webman is open-sourced software licensed under the MIT. -
- -
- - diff --git a/Moncter/Moncter/app/command/BatchUpdateTags.php b/Moncter/Moncter/app/command/BatchUpdateTags.php deleted file mode 100644 index 24703a5c..00000000 --- a/Moncter/Moncter/app/command/BatchUpdateTags.php +++ /dev/null @@ -1,163 +0,0 @@ -addOption( - 'frequency', - 'f', - InputOption::VALUE_OPTIONAL, - '更新频率：daily/weekly/monthly', - 'daily' - ); - - $this->addOption( - 'limit', - 'l', - InputOption::VALUE_OPTIONAL, - '每次处理的最大用户数', - 1000 - ); - - $this->addOption( - 'user-ids', - 'u', - InputOption::VALUE_OPTIONAL, - '指定用户ID列表（逗号分隔），如果提供则只更新这些用户', - null - ); - } - - protected function execute(InputInterface $input, OutputInterface $output): int - { - $frequency = $input->getOption('frequency'); - $limit = (int)$input->getOption('limit'); - $userIdsOption = $input->getOption('user-ids'); - - $output->writeln("开始批量更新标签..."); - $output->writeln("更新频率: {$frequency}"); - $output->writeln("处理限制: {$limit}"); - - try { - // 获取需要更新的标签定义 - $tagDefinitionRepo = new TagDefinitionRepository(); - $tagDefinitions = $tagDefinitionRepo->newQuery() - ->where('status', 0) // 只获取启用的标签 - ->where('update_frequency', $frequency) // 匹配更新频率 - ->get(); - - if ($tagDefinitions->isEmpty()) { - $output->writeln("没有找到需要更新的标签定义（频率: {$frequency}）"); - return Command::SUCCESS; - } - - $tagIds = $tagDefinitions->pluck('tag_id')->toArray(); - $output->writeln("找到 " . count($tagIds) . " 个标签需要更新"); - - // 获取需要更新的用户列表 - $userProfileRepo = new UserProfileRepository(); - if ($userIdsOption !== null) { - // 指定用户ID列表 - $userIds = array_filter(array_map('trim', explode(',', $userIdsOption))); - } else { - // 获取所有有效用户（限制数量） - $users = $userProfileRepo->newQuery() - ->where('status', 0) - ->limit($limit) - ->get(); - $userIds = $users->pluck('user_id')->toArray(); - } - - if (empty($userIds)) { - $output->writeln("没有找到需要更新的用户"); - return Command::SUCCESS; - } - - $output->writeln("找到 " . count($userIds) . " 个用户需要更新"); - - // 创建 TagService 实例 - $tagService = new TagService( - new TagDefinitionRepository(), - new UserProfileRepository(), - new UserTagRepository(), - new TagHistoryRepository(), - new SimpleRuleEngine() - ); - - // 批量更新标签 - $successCount = 0; - $errorCount = 0; - $startTime = microtime(true); - - foreach ($userIds as $index => $userId) { - try { - $tagService->calculateTags($userId, $tagIds); - $successCount++; - - if (($index + 1) % 100 === 0) { - $output->writeln("已处理: " . ($index + 1) . " / " . count($userIds)); - } - } catch (\Throwable $e) { - $errorCount++; - LoggerHelper::logError($e, [ - 'component' => 'BatchUpdateTags', - 'user_id' => $userId, - 'frequency' => $frequency, - ]); - } - } - - $duration = microtime(true) - $startTime; - - $output->writeln("批量更新完成！"); - $output->writeln("成功: {$successCount}"); - $output->writeln("失败: {$errorCount}"); - $output->writeln("耗时: " . round($duration, 2) . " 秒"); - - LoggerHelper::logBusiness('batch_tag_update_completed', [ - 'frequency' => $frequency, - 'tag_count' => count($tagIds), - 'user_count' => count($userIds), - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'duration' => $duration, - ]); - - return Command::SUCCESS; - } catch (\Throwable $e) { - $output->writeln("批量更新失败: " . $e->getMessage()); - LoggerHelper::logError($e, [ - 'component' => 'BatchUpdateTags', - 'frequency' => $frequency, - ]); - return Command::FAILURE; - } - } -} - diff --git a/Moncter/Moncter/app/command/InitTags.php b/Moncter/Moncter/app/command/InitTags.php deleted file mode 100644 index 973c2093..00000000 --- a/Moncter/Moncter/app/command/InitTags.php +++ /dev/null @@ -1,28 +0,0 @@ -initBasicTags(); - - echo "标签初始化完成！\n"; - } -} - diff --git a/Moncter/Moncter/app/controller/ConsumptionController.php b/Moncter/Moncter/app/controller/ConsumptionController.php deleted file mode 100644 index ba882e49..00000000 --- a/Moncter/Moncter/app/controller/ConsumptionController.php +++ /dev/null @@ -1,110 +0,0 @@ - $request->getRealIp(), - 'user_agent' => $request->header('user-agent'), - ]); - - // 获取 JSON 请求体 - $rawBody = $request->rawBody(); - - if (empty($rawBody)) { - return ApiResponseHelper::error('请求体为空，请确保 Content-Type 为 application/json 并发送有效的 JSON 数据', 400); - } - - $payload = json_decode($rawBody, true); - - if (json_last_error() !== JSON_ERROR_NONE) { - return ApiResponseHelper::error('JSON 格式错误: ' . json_last_error_msg(), 400); - } - - // 验证用户标识：必须提供 user_id、phone_number 或 id_card 之一 - // 注意：如果手机号和身份证号都为空（但提供了user_id），仍然可以处理 - $phoneNumber = trim($payload['phone_number'] ?? ''); - $idCard = trim($payload['id_card'] ?? ''); - if (empty($payload['user_id']) && empty($phoneNumber) && empty($idCard)) { - throw new \InvalidArgumentException('缺少用户标识：必须提供 user_id、phone_number 或 id_card 之一'); - } - - // 简单手动组装 Service 依赖，后续可接入容器配置 - $tagService = new \app\service\TagService( - new \app\repository\TagDefinitionRepository(), - new \app\repository\UserProfileRepository(), - new \app\repository\UserTagRepository(), - new \app\repository\TagHistoryRepository(), - new \app\service\TagRuleEngine\SimpleRuleEngine() - ); - - $identifierService = new \app\service\IdentifierService( - new UserProfileRepository(), - new \app\service\UserPhoneService( - new \app\repository\UserPhoneRelationRepository() - ) - ); - - $service = new ConsumptionService( - new ConsumptionRecordRepository(), - new UserProfileRepository(), - $identifierService, - $tagService - ); - - $result = $service->createRecord($payload); - - // 如果返回 null，说明手机号和身份证号都为空，跳过该记录 - if ($result === null) { - LoggerHelper::logBusiness('consumption_record_skipped_no_identifier', [ - 'reason' => 'phone_number and id_card are both empty', - 'consume_time' => $payload['consume_time'] ?? null, - ]); - return ApiResponseHelper::error('记录已跳过：手机号和身份证号都为空', 400, 400); - } - - // 记录业务日志 - LoggerHelper::logBusiness('consumption_record_created', [ - 'user_id' => $result['user_id'] ?? null, - 'record_id' => $result['record_id'] ?? null, - 'amount' => $payload['amount'] ?? null, - 'phone_number' => $payload['phone_number'] ?? null, - 'id_card_provided' => !empty($payload['id_card']), - ]); - - $duration = microtime(true) - $startTime; - LoggerHelper::logPerformance('consumption_record_create', $duration, [ - 'user_id' => $result['user_id'] ?? null, - ]); - - return ApiResponseHelper::success($result); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } -} - - diff --git a/Moncter/Moncter/app/controller/DataCollectionTaskController.php b/Moncter/Moncter/app/controller/DataCollectionTaskController.php deleted file mode 100644 index 6a6beb55..00000000 --- a/Moncter/Moncter/app/controller/DataCollectionTaskController.php +++ /dev/null @@ -1,850 +0,0 @@ -post(); - - // 验证必填字段 - $requiredFields = ['name', 'data_source_id', 'database', 'target_type']; - foreach ($requiredFields as $field) { - if (empty($data[$field])) { - return ApiResponseHelper::error("缺少必填字段: {$field}", 400); - } - } - - // 验证目标类型 - if (!in_array($data['target_type'], ['consumption_record', 'generic'])) { - return ApiResponseHelper::error("目标类型必须是 consumption_record 或 generic", 400); - } - - // 如果是通用Handler，需要目标数据源配置（后端会自动处理consumption_record的配置） - if ($data['target_type'] === 'generic') { - $genericRequiredFields = ['target_data_source_id', 'target_database', 'target_collection']; - foreach ($genericRequiredFields as $field) { - if (empty($data[$field])) { - return ApiResponseHelper::error("通用Handler缺少必填字段: {$field}", 400); - } - } - } - - // 验证模式 - if (isset($data['mode']) && !in_array($data['mode'], ['batch', 'realtime'])) { - return ApiResponseHelper::error("模式必须是 batch 或 realtime", 400); - } - - // 验证集合配置 - if (empty($data['collection']) && empty($data['collections'])) { - return ApiResponseHelper::error("必须指定 collection 或 collections", 400); - } - - $service = $this->getService(); - $task = $service->createTask($data); - - return ApiResponseHelper::success($task, '任务创建成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 更新任务 - * - * PUT /api/data-collection-tasks/{task_id} - */ - public function update(Request $request): Response - { - try { - // 从请求路径中解析 task_id - $path = $request->path(); - if (preg_match('#/api/data-collection-tasks/([^/]+)#', $path, $matches)) { - $taskId = $matches[1]; - } else { - $taskId = $request->get('task_id'); - if (!$taskId) { - throw new \InvalidArgumentException('缺少 task_id 参数'); - } - } - - $data = $request->post(); - - $service = $this->getService(); - $result = $service->updateTask($taskId, $data); - - if ($result) { - return ApiResponseHelper::success(null, '任务更新成功'); - } else { - return ApiResponseHelper::error('任务更新失败', 500); - } - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 删除任务 - * - * DELETE /api/data-collection-tasks/{task_id} - */ - public function delete(Request $request): Response - { - try { - // 从请求路径中解析 task_id - $path = $request->path(); - if (preg_match('#/api/data-collection-tasks/([^/]+)#', $path, $matches)) { - $taskId = $matches[1]; - } else { - $taskId = $request->get('task_id'); - if (!$taskId) { - throw new \InvalidArgumentException('缺少 task_id 参数'); - } - } - - $service = $this->getService(); - $result = $service->deleteTask($taskId); - - if ($result) { - return ApiResponseHelper::success(null, '任务删除成功'); - } else { - return ApiResponseHelper::error('任务删除失败', 500); - } - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 启动任务 - * - * POST /api/data-collection-tasks/{task_id}/start - */ - public function start(Request $request): Response - { - try { - // 从请求路径中解析 task_id - $path = $request->path(); - if (preg_match('#/api/data-collection-tasks/([^/]+)/start#', $path, $matches)) { - $taskId = $matches[1]; - } else { - $taskId = $request->get('task_id'); - if (!$taskId) { - throw new \InvalidArgumentException('缺少 task_id 参数'); - } - } - - $service = $this->getService(); - $result = $service->startTask($taskId); - - if ($result) { - return ApiResponseHelper::success(null, '任务启动成功'); - } else { - return ApiResponseHelper::error('任务启动失败', 500); - } - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 暂停任务 - * - * POST /api/data-collection-tasks/{task_id}/pause - */ - public function pause(Request $request): Response - { - try { - // 从请求路径中解析 task_id - $path = $request->path(); - if (preg_match('#/api/data-collection-tasks/([^/]+)/pause#', $path, $matches)) { - $taskId = $matches[1]; - } else { - $taskId = $request->get('task_id'); - if (!$taskId) { - throw new \InvalidArgumentException('缺少 task_id 参数'); - } - } - - $service = $this->getService(); - $result = $service->pauseTask($taskId); - - if ($result) { - return ApiResponseHelper::success(null, '任务暂停成功'); - } else { - return ApiResponseHelper::error('任务暂停失败', 500); - } - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 停止任务 - * - * POST /api/data-collection-tasks/{task_id}/stop - */ - public function stop(Request $request): Response - { - try { - // 从请求路径中解析 task_id - $path = $request->path(); - if (preg_match('#/api/data-collection-tasks/([^/]+)/stop#', $path, $matches)) { - $taskId = $matches[1]; - } else { - $taskId = $request->get('task_id'); - if (!$taskId) { - throw new \InvalidArgumentException('缺少 task_id 参数'); - } - } - - $service = $this->getService(); - $result = $service->stopTask($taskId); - - if ($result) { - return ApiResponseHelper::success(null, '任务停止成功'); - } else { - return ApiResponseHelper::error('任务停止失败', 500); - } - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取任务列表 - * - * GET /api/data-collection-tasks - */ - public function list(Request $request): Response - { - try { - // 只收集非空的筛选条件 - $filters = []; - if ($request->get('status') !== null && $request->get('status') !== '') { - $filters['status'] = $request->get('status'); - } - if ($request->get('data_source_id') !== null && $request->get('data_source_id') !== '') { - $filters['data_source_id'] = $request->get('data_source_id'); - } - if ($request->get('name') !== null && $request->get('name') !== '') { - $filters['name'] = $request->get('name'); - } - - $page = (int)($request->get('page', 1)); - $pageSize = (int)($request->get('page_size', 20)); - - $service = $this->getService(); - $result = $service->getTaskList($filters, $page, $pageSize); - - return ApiResponseHelper::success($result, '查询成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取任务详情 - * - * GET /api/data-collection-tasks/{task_id} - */ - public function detail(Request $request): Response - { - try { - // 从请求路径中解析 task_id - $path = $request->path(); - if (preg_match('#/api/data-collection-tasks/([^/]+)$#', $path, $matches)) { - $taskId = $matches[1]; - } else { - $taskId = $request->get('task_id'); - if (!$taskId) { - throw new \InvalidArgumentException('缺少 task_id 参数'); - } - } - - $service = $this->getService(); - $task = $service->getTask($taskId); - - if ($task === null) { - return ApiResponseHelper::error('任务不存在', 404); - } - - return ApiResponseHelper::success($task, '查询成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取任务进度 - * - * GET /api/data-collection-tasks/{task_id}/progress - */ - public function progress(Request $request): Response - { - try { - // 从请求路径中解析 task_id - $path = $request->path(); - if (preg_match('#/api/data-collection-tasks/([^/]+)/progress#', $path, $matches)) { - $taskId = $matches[1]; - } else { - $taskId = $request->get('task_id'); - if (!$taskId) { - throw new \InvalidArgumentException('缺少 task_id 参数'); - } - } - - $service = $this->getService(); - $task = $service->getTask($taskId); - - if ($task === null) { - return ApiResponseHelper::error('任务不存在', 404); - } - - $progress = $task['progress'] ?? []; - - return ApiResponseHelper::success($progress, '查询成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取数据源列表 - * - * GET /api/data-collection-tasks/data-sources - */ - public function getDataSources(Request $request): Response - { - try { - // 优先使用数据库中的数据源，如果没有则使用配置文件 - $service = new \app\service\DataSourceService(new \app\repository\DataSourceRepository()); - $result = $service->getDataSourceList(['status' => 1]); - - if (!empty($result['list'])) { - // 使用数据库中的数据源 - $list = array_map(function ($ds) { - return [ - 'id' => $ds['data_source_id'], - 'name' => $ds['name'] ?? $ds['data_source_id'], // 添加名称字段 - 'type' => $ds['type'] ?? 'unknown', - 'host' => $ds['host'] ?? '', - 'port' => $ds['port'] ?? 0, - 'database' => $ds['database'] ?? '', - ]; - }, $result['list']); - } - // 注意：现在数据源配置统一从数据库读取，不再使用config('data_sources') - - // 如果数据库中没有数据源，返回空列表 - if (!isset($list)) { - $list = []; - } - - return ApiResponseHelper::success($list, '查询成功'); - } catch (\MongoDB\Driver\Exception\Exception $e) { - // MongoDB 连接错误，返回友好提示 - $errorMessage = '无法连接到 MongoDB 数据库，请检查数据库服务是否正常运行。错误详情：' . $e->getMessage(); - return ApiResponseHelper::error($errorMessage, 500); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取数据源的数据库列表 - * - * GET /api/data-collection-tasks/data-sources/{data_source_id}/databases - */ - public function getDatabases(Request $request, string $data_source_id): Response - { - try { - // 从数据库获取数据源配置 - $service = new \app\service\DataSourceService(new \app\repository\DataSourceRepository()); - $dataSourceConfig = $service->getDataSourceConfig($data_source_id); - - if (!$dataSourceConfig) { - return ApiResponseHelper::error('数据源不存在', 404); - } - - $dataSource = $dataSourceConfig; - - // 如果是MongoDB，连接并获取数据库列表 - if ($dataSource['type'] === 'mongodb') { - $client = $this->getMongoClient($dataSource); - $databases = $client->listDatabases(); - - $list = []; - foreach ($databases as $database) { - $dbName = $database->getName(); - // 同时返回原始名称和base64编码的ID（URL友好） - $list[] = [ - 'name' => $dbName, - 'id' => base64_encode($dbName), // URL友好的标识符 - ]; - } - - return ApiResponseHelper::success($list, '查询成功'); - } - - return ApiResponseHelper::error('不支持的数据源类型', 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取数据库的集合列表 - * - * GET /api/data-collection-tasks/data-sources/{data_source_id}/databases/{database}/collections - */ - public function getCollections(Request $request, string $data_source_id, string $database): Response - { - try { - // 解码数据库名称（支持base64编码和URL编码） - $database = $this->decodeName($database); - - // 从数据库获取数据源配置 - $service = new \app\service\DataSourceService(new \app\repository\DataSourceRepository()); - $dataSourceConfig = $service->getDataSourceConfig($data_source_id); - - if (!$dataSourceConfig) { - return ApiResponseHelper::error('数据源不存在', 404); - } - - $dataSource = $dataSourceConfig; - - // 如果是MongoDB，连接并获取集合列表 - if ($dataSource['type'] === 'mongodb') { - $client = $this->getMongoClient($dataSource); - $db = $client->selectDatabase($database); - $collections = $db->listCollections(); - - $list = []; - foreach ($collections as $collection) { - $collName = $collection->getName(); - // 同时返回原始名称和base64编码的ID（URL友好） - $list[] = [ - 'name' => $collName, - 'id' => base64_encode($collName), // URL友好的标识符 - ]; - } - - return ApiResponseHelper::success($list, '查询成功'); - } - - return ApiResponseHelper::error('不支持的数据源类型', 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取Handler的目标字段列表 - * - * GET /api/data-collection-tasks/handlers/{handler_type}/target-fields - */ - public function getHandlerTargetFields(Request $request, string $handler_type): Response - { - try { - $fields = []; - - switch ($handler_type) { - case 'consumption_record': - // 消费记录Handler的目标字段列表 - // 包含原始输入字段（推荐）和转换后字段（可选） - // Handler会自动进行转换：phone_number/id_card -> user_id, store_name -> store_id - $fields = [ - // 用户标识字段（原始输入，推荐使用） - ['name' => 'phone_number', 'label' => '手机号', 'type' => 'string', 'required' => false, 'description' => '手机号，Handler会自动解析为user_id', 'is_original' => true], - ['name' => 'id_card', 'label' => '身份证', 'type' => 'string', 'required' => false, 'description' => '身份证号，Handler会自动解析为user_id', 'is_original' => true], - // 用户ID（转换后字段，由Handler自动生成，不需要映射） - ['name' => 'user_id', 'label' => '用户ID', 'type' => 'string', 'required' => false, 'description' => '用户ID，由Handler通过phone_number/id_card自动解析生成，无需映射', 'is_original' => false, 'no_mapping' => true], - - // 门店标识字段（原始输入，推荐使用） - ['name' => 'store_name', 'label' => '门店名称', 'type' => 'string', 'required' => false, 'description' => '门店名称，Handler会自动转换为store_id', 'is_original' => true], - // 门店ID（转换后字段，由Handler自动生成，不需要映射） - ['name' => 'store_id', 'label' => '门店ID', 'type' => 'string', 'required' => false, 'description' => '门店ID，由Handler通过store_name自动转换生成，无需映射', 'is_original' => false, 'no_mapping' => true], - - // 订单标识字段（用于去重） - ['name' => 'source_order_id', 'label' => '原始订单ID', 'type' => 'string', 'required' => false, 'description' => '原始订单ID，配合店铺名称做去重唯一标识（建议配置）', 'is_original' => true], - // 注意：order_no 由系统自动生成（自动递增），不需要映射 - - // 金额和时间字段（直接字段） - ['name' => 'amount', 'label' => '消费金额', 'type' => 'float', 'required' => true, 'description' => '消费金额（必填）', 'is_original' => true], - ['name' => 'actual_amount', 'label' => '实际金额', 'type' => 'float', 'required' => true, 'description' => '实际支付金额（必填）', 'is_original' => true], - ['name' => 'consume_time', 'label' => '消费时间', 'type' => 'datetime', 'required' => true, 'description' => '消费时间，用于时间分片存储（必填）', 'is_original' => true], - - // 其他可选字段 - ['name' => 'currency', 'label' => '币种', 'type' => 'string', 'required' => false, 'description' => '币种，默认CNY（人民币）', 'is_original' => true, 'fixed_options' => true, 'options' => [['value' => 'CNY', 'label' => '人民币(CNY)'], ['value' => 'USD', 'label' => '美元(USD)']], 'default_value' => 'CNY'], - ['name' => 'status', 'label' => '记录状态', 'type' => 'int', 'required' => false, 'description' => '记录状态：0-正常，1-异常，2-已删除。默认0。需要配置源状态值到标准状态值的映射', 'is_original' => true, 'value_mapping' => true, 'target_values' => [['value' => 0, 'label' => '正常(0)'], ['value' => 1, 'label' => '异常(1)'], ['value' => 2, 'label' => '已删除(2)']], 'default_value' => 0], - ]; - break; - - case 'generic': - // 通用Handler - 没有固定的字段列表，由用户自定义 - $fields = []; - break; - - default: - return ApiResponseHelper::error("未知的Handler类型: {$handler_type}", 400); - } - - return ApiResponseHelper::success($fields, '查询成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取集合的字段列表（采样） - * - * GET /api/data-collection-tasks/data-sources/{data_source_id}/databases/{database}/collections/{collection}/fields - */ - public function getFields(Request $request, string $data_source_id, string $database, string $collection): Response - { - try { - // 解码数据库名称和集合名称（支持base64编码和URL编码） - $database = $this->decodeName($database); - $collection = $this->decodeName($collection); - - // 从数据库获取数据源配置 - $service = new \app\service\DataSourceService(new \app\repository\DataSourceRepository()); - $dataSourceConfig = $service->getDataSourceConfig($data_source_id); - - if (!$dataSourceConfig) { - return ApiResponseHelper::error('数据源不存在', 404); - } - - $dataSource = $dataSourceConfig; - - // 如果是MongoDB，采样获取字段 - if ($dataSource['type'] === 'mongodb') { - $client = $this->getMongoClient($dataSource); - $db = $client->selectDatabase($database); - $coll = $db->selectCollection($collection); - - // 采样一条数据 - $sample = $coll->findOne([]); - - if ($sample) { - $fields = []; - $this->extractFields($sample, '', $fields); - - return ApiResponseHelper::success($fields, '查询成功'); - } else { - return ApiResponseHelper::success([], '集合为空，无法获取字段'); - } - } - - return ApiResponseHelper::error('不支持的数据源类型', 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 递归提取字段 - */ - private function extractFields($data, string $prefix, array &$fields): void - { - if (is_array($data) || is_object($data)) { - foreach ($data as $key => $value) { - $fieldName = $prefix ? "{$prefix}.{$key}" : $key; - - if (is_array($value) || is_object($value)) { - if (empty($value)) { - $fields[] = [ - 'name' => $fieldName, - 'type' => 'array', - ]; - } else { - $this->extractFields($value, $fieldName, $fields); - } - } else { - $fields[] = [ - 'name' => $fieldName, - 'type' => gettype($value), - ]; - } - } - } - } - - /** - * 预览查询结果（包含lookup） - * - * POST /api/data-collection-tasks/preview-query - */ - public function previewQuery(Request $request): Response - { - try { - $data = $request->post(); - - $dataSourceId = $data['data_source_id'] ?? ''; - $database = $data['database'] ?? ''; - $collection = $data['collection'] ?? ''; - $lookups = $data['lookups'] ?? []; - $filterConditions = $data['filter_conditions'] ?? []; - $limit = (int)($data['limit'] ?? 5); // 默认预览5条 - - if (empty($dataSourceId) || empty($database) || empty($collection)) { - return ApiResponseHelper::error('缺少必要参数：data_source_id, database, collection', 400); - } - - // 获取数据源配置 - $service = new \app\service\DataSourceService(new \app\repository\DataSourceRepository()); - $dataSourceConfig = $service->getDataSourceConfig($dataSourceId); - - if (!$dataSourceConfig) { - return ApiResponseHelper::error('数据源不存在', 404); - } - - if ($dataSourceConfig['type'] !== 'mongodb') { - return ApiResponseHelper::error('目前只支持MongoDB数据源预览', 400); - } - - // 连接MongoDB - $client = $this->getMongoClient($dataSourceConfig); - $db = $client->selectDatabase($database); - $coll = $db->selectCollection($collection); - - // 构建聚合管道 - $pipeline = []; - - // 1. 添加过滤条件（$match）- 必须在最前面 - $filter = $this->buildFilterForPreview($filterConditions); - if (!empty($filter)) { - $pipeline[] = ['$match' => $filter]; - } - - // 2. 添加lookup查询 - foreach ($lookups as $lookup) { - if (empty($lookup['from']) || empty($lookup['local_field']) || empty($lookup['foreign_field'])) { - continue; - } - - $lookupStage = [ - '$lookup' => [ - 'from' => $lookup['from'], - 'localField' => $lookup['local_field'], - 'foreignField' => $lookup['foreign_field'], - 'as' => $lookup['as'] ?? 'joined' - ] - ]; - - $pipeline[] = $lookupStage; - - // 如果配置了解构 - if (!empty($lookup['unwrap'])) { - $pipeline[] = [ - '$unwind' => [ - 'path' => '$' . ($lookup['as'] ?? 'joined'), - 'preserveNullAndEmptyArrays' => !empty($lookup['preserve_null']) - ] - ]; - } - } - - // 3. 限制返回数量 - $pipeline[] = ['$limit' => $limit]; - - // 执行聚合查询 - $cursor = $coll->aggregate($pipeline); - $results = []; - $fields = []; - - foreach ($cursor as $doc) { - $docArray = $this->convertMongoDocumentToArray($doc); - $results[] = $docArray; - - // 提取字段 - $this->extractFields($docArray, '', $fields); - } - - // 去重字段 - $uniqueFields = []; - $fieldMap = []; - foreach ($fields as $field) { - if (!isset($fieldMap[$field['name']])) { - $fieldMap[$field['name']] = true; - $uniqueFields[] = $field; - } - } - - return ApiResponseHelper::success([ - 'fields' => $uniqueFields, - 'data' => $results, - 'count' => count($results) - ], '预览成功'); - - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 将MongoDB文档转换为数组 - */ - private function convertMongoDocumentToArray($document): array - { - if (is_array($document)) { - return $document; - } - - if (is_object($document)) { - $array = []; - foreach ($document as $key => $value) { - if ($value instanceof \MongoDB\BSON\UTCDateTime) { - $array[$key] = $value->toDateTime()->format('Y-m-d H:i:s'); - } elseif (is_object($value) && method_exists($value, '__toString')) { - $array[$key] = (string)$value; - } elseif (is_array($value) || is_object($value)) { - $array[$key] = $this->convertMongoDocumentToArray($value); - } else { - $array[$key] = $value; - } - } - return $array; - } - - return []; - } - - /** - * 构建过滤条件（用于预览查询） - * - * @param array $filterConditions 过滤条件列表 - * @return array MongoDB查询过滤器 - */ - private function buildFilterForPreview(array $filterConditions): array - { - $filter = []; - - foreach ($filterConditions as $condition) { - $field = $condition['field'] ?? ''; - $operator = $condition['operator'] ?? 'eq'; - $value = $condition['value'] ?? null; - - if (empty($field)) { - continue; - } - - // 处理值的类型转换 - if ($value !== null && $value !== '') { - // 尝试转换为数字（如果是数字字符串） - if (is_numeric($value)) { - // 判断是整数还是浮点数 - if (strpos($value, '.') !== false) { - $value = (float)$value; - } else { - $value = (int)$value; - } - } - } - - switch ($operator) { - case 'eq': - $filter[$field] = $value; - break; - case 'ne': - $filter[$field] = ['$ne' => $value]; - break; - case 'gt': - $filter[$field] = ['$gt' => $value]; - break; - case 'gte': - $filter[$field] = ['$gte' => $value]; - break; - case 'lt': - $filter[$field] = ['$lt' => $value]; - break; - case 'lte': - $filter[$field] = ['$lte' => $value]; - break; - case 'in': - // in操作符的值应该是数组 - $valueArray = is_array($value) ? $value : explode(',', (string)$value); - $filter[$field] = ['$in' => $valueArray]; - break; - case 'nin': - // nin操作符的值应该是数组 - $valueArray = is_array($value) ? $value : explode(',', (string)$value); - $filter[$field] = ['$nin' => $valueArray]; - break; - } - } - - return $filter; - } - - /** - * 解码数据库或集合名称（支持base64编码和URL编码） - * - * @param string $name 编码后的名称 - * @return string 解码后的名称 - */ - private function decodeName(string $name): string - { - // 尝试base64解码（如果前端使用的是编码后的ID） - // 检查是否可能是base64编码（只包含base64字符且长度合理） - if (preg_match('/^[A-Za-z0-9+\/]*={0,2}$/', $name) && strlen($name) > 0) { - $decoded = @base64_decode($name, true); - if ($decoded !== false && $decoded !== '') { - // 解码成功，使用解码后的值 - return $decoded; - } - } - - // 不是base64格式或解码失败，使用URL解码（处理中文等特殊字符） - return rawurldecode($name); - } - - /** - * 获取MongoDB客户端 - */ - private function getMongoClient(array $config): \MongoDB\Client - { - $host = $config['host'] ?? ''; - $port = (int)($config['port'] ?? 27017); - $username = $config['username'] ?? ''; - $password = $config['password'] ?? ''; - $authSource = $config['auth_source'] ?? 'admin'; - - if (!empty($username) && !empty($password)) { - $dsn = "mongodb://{$username}:{$password}@{$host}:{$port}/{$authSource}"; - } else { - $dsn = "mongodb://{$host}:{$port}"; - } - - return new \MongoDB\Client($dsn, $config['options'] ?? []); - } -} - diff --git a/Moncter/Moncter/app/controller/DataSourceController.php b/Moncter/Moncter/app/controller/DataSourceController.php deleted file mode 100644 index 3f1ef36e..00000000 --- a/Moncter/Moncter/app/controller/DataSourceController.php +++ /dev/null @@ -1,173 +0,0 @@ -getService(); - $filters = [ - 'type' => $request->get('type'), - 'status' => $request->get('status'), - 'name' => $request->get('name'), - 'page' => $request->get('page', 1), - 'page_size' => $request->get('page_size', 20), - ]; - - $result = $service->getDataSourceList($filters); - - return ApiResponseHelper::success([ - 'data_sources' => $result['list'], - 'total' => $result['total'], - 'page' => (int)$filters['page'], - 'page_size' => (int)$filters['page_size'], - ], '查询成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取数据源详情 - * - * GET /api/data-sources/{data_source_id} - */ - public function detail(Request $request, string $data_source_id): Response - { - try { - $service = $this->getService(); - $dataSource = $service->getDataSourceDetail($data_source_id); - - if (!$dataSource) { - return ApiResponseHelper::error('数据源不存在', 404); - } - - return ApiResponseHelper::success($dataSource, '查询成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 创建数据源 - * - * POST /api/data-sources - */ - public function create(Request $request): Response - { - try { - $data = $request->post(); - - $service = $this->getService(); - $dataSource = $service->createDataSource($data); - - // 不返回密码 - $result = $dataSource->toArray(); - unset($result['password']); - - return ApiResponseHelper::success($result, '创建成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 更新数据源 - * - * PUT /api/data-sources/{data_source_id} - */ - public function update(Request $request, string $data_source_id): Response - { - try { - $data = $request->post(); - - $service = $this->getService(); - $result = $service->updateDataSource($data_source_id, $data); - - if ($result) { - return ApiResponseHelper::success(null, '更新成功'); - } else { - return ApiResponseHelper::error('更新失败', 500); - } - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 删除数据源 - * - * DELETE /api/data-sources/{data_source_id} - */ - public function delete(Request $request, string $data_source_id): Response - { - try { - $service = $this->getService(); - $result = $service->deleteDataSource($data_source_id); - - if ($result) { - return ApiResponseHelper::success(null, '删除成功'); - } else { - return ApiResponseHelper::error('删除失败', 500); - } - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 测试数据源连接 - * - * POST /api/data-sources/test-connection - */ - public function testConnection(Request $request): Response - { - try { - $data = $request->post(); - - // 验证必填字段 - $requiredFields = ['type', 'host', 'port', 'database']; - foreach ($requiredFields as $field) { - if (empty($data[$field])) { - return ApiResponseHelper::error("缺少必填字段: {$field}", 400); - } - } - - $service = $this->getService(); - $connected = $service->testConnection($data); - - if ($connected) { - return ApiResponseHelper::success(['connected' => true], '连接成功'); - } else { - return ApiResponseHelper::error('连接失败，请检查配置', 400); - } - } catch (\Throwable $e) { - return ApiResponseHelper::error('连接测试失败: ' . $e->getMessage(), 400); - } - } -} - diff --git a/Moncter/Moncter/app/controller/DatabaseSyncController.php b/Moncter/Moncter/app/controller/DatabaseSyncController.php deleted file mode 100644 index c1ae814b..00000000 --- a/Moncter/Moncter/app/controller/DatabaseSyncController.php +++ /dev/null @@ -1,455 +0,0 @@ -404 Not Found

看板页面不存在

', 404) - ->withHeader('Content-Type', 'text/html; charset=utf-8'); - } - - $html = file_get_contents($htmlPath); - return response($html)->withHeader('Content-Type', 'text/html; charset=utf-8'); - } - - /** - * 获取同步进度 - * - * GET /api/database-sync/progress - */ - public function progress(Request $request): Response - { - try { - // 创建 DatabaseSyncService 实例（传递最小配置，仅用于读取进度） - // 注意：数据库同步功能已迁移到 data_collection_tasks.php，这里仅用于查询进度 - $minimalConfig = [ - 'source' => ['host' => '', 'port' => 27017], // 占位符，不会实际连接 - 'target' => ['host' => '', 'port' => 27017], // 占位符，不会实际连接 - 'sync' => [], - 'monitoring' => [], - ]; - $syncService = new DatabaseSyncService($minimalConfig); - // 加载最新进度 - $syncService->loadProgress(); - $progress = $syncService->getProgress(); - - // 获取多进程状态信息 - $workerStatus = $this->getWorkerStatus(); - $progress['worker_status'] = $workerStatus; - - // 获取数据库连接状态 - $connectionStatus = $this->getConnectionStatus(); - $progress['connection_status'] = $connectionStatus; - - // 获取数据库列表信息（已完成和待同步） - $databaseList = $this->getDatabaseList($syncService); - $progress['database_list'] = $databaseList; - - // 检查进度文件最后修改时间 - $runtimePath = function_exists('runtime_path') ? runtime_path() : (config('app.runtime_path', base_path() . DIRECTORY_SEPARATOR . 'runtime')); - $progressFile = $runtimePath . DIRECTORY_SEPARATOR . 'database_sync_progress.json'; - if (file_exists($progressFile)) { - $fileTime = filemtime($progressFile); - $progress['progress_file_last_modified'] = date('Y-m-d H:i:s', $fileTime); - $progress['progress_file_age_seconds'] = time() - $fileTime; - } else { - $progress['progress_file_last_modified'] = null; - $progress['progress_file_age_seconds'] = null; - } - - // 如果状态是idle且没有开始时间，尝试检查是否真的在运行 - if ($progress['status'] === 'idle' && $progress['time']['start_time'] === null) { - if (!file_exists($progressFile)) { - $progress['hint'] = '请执行: php start.php status 查看 data_sync_scheduler 进程是否运行（数据库同步任务由 data_sync_scheduler 管理）'; - } - } - - return ApiResponseHelper::success($progress, '同步进度查询成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取 Worker 进程状态信息 - * - * @return array Worker 状态信息 - */ - private function getWorkerStatus(): array - { - $status = [ - 'total_workers' => 0, - 'active_workers' => 0, - 'workers' => [], - ]; - - try { - // 从配置中获取 Worker 数量 - $processConfig = config('process.data_sync_scheduler', []); - $totalWorkers = (int)($processConfig['count'] ?? 10); - $status['total_workers'] = $totalWorkers; - - // 检查进度文件中的 checkpoints，推断每个 Worker 处理的数据库 - $runtimePath = function_exists('runtime_path') ? runtime_path() : (config('app.runtime_path', base_path() . DIRECTORY_SEPARATOR . 'runtime')); - $progressFile = $runtimePath . DIRECTORY_SEPARATOR . 'database_sync_progress.json'; - - if (file_exists($progressFile)) { - // 使用文件锁读取，避免并发问题 - $fp = fopen($progressFile, 'r'); - if ($fp && flock($fp, LOCK_SH)) { - try { - $content = stream_get_contents($fp); - $progressData = json_decode($content, true); - - if ($progressData && isset($progressData['checkpoints'])) { - $checkpoints = $progressData['checkpoints']; - $databases = array_keys($checkpoints); - - // 根据数据库分配推断每个 Worker 的状态（使用取模分配） - foreach ($databases as $index => $database) { - $workerId = $index % $totalWorkers; - if (!isset($status['workers'][$workerId])) { - $status['workers'][$workerId] = [ - 'worker_id' => $workerId, - 'databases' => [], - 'collections' => 0, - 'documents_processed' => 0, - 'status' => 'active', - ]; - } - - $status['workers'][$workerId]['databases'][] = $database; - - // 统计该 Worker 处理的集合和文档数 - if (isset($checkpoints[$database]) && is_array($checkpoints[$database])) { - foreach ($checkpoints[$database] as $collection => $checkpoint) { - $status['workers'][$workerId]['collections']++; - if (isset($checkpoint['processed'])) { - $status['workers'][$workerId]['documents_processed'] += (int)$checkpoint['processed']; - } - } - } - } - - $status['active_workers'] = count($status['workers']); - - // 将 workers 数组转换为索引数组（便于前端遍历） - $status['workers'] = array_values($status['workers']); - } - } finally { - flock($fp, LOCK_UN); - fclose($fp); - } - } else { - if ($fp) { - fclose($fp); - } - } - } - - // 如果没有活动的 Worker，但配置了 Worker 数量，显示所有 Worker（等待状态） - if ($status['active_workers'] === 0 && $status['total_workers'] > 0) { - // 创建所有 Worker 的占位信息 - for ($i = 0; $i < $totalWorkers; $i++) { - $status['workers'][] = [ - 'worker_id' => $i, - 'databases' => [], - 'collections' => 0, - 'documents_processed' => 0, - 'status' => 'waiting', // 等待状态 - ]; - } - $status['message'] = '所有 Worker 处于等待状态，同步尚未开始或进度文件为空'; - } elseif ($status['total_workers'] === 0) { - $status['message'] = '未配置 Worker 数量，请检查 config/process.php'; - } - - } catch (\Throwable $e) { - $status['error'] = '获取 Worker 状态失败: ' . $e->getMessage(); - } - - return $status; - } - - /** - * 获取同步统计信息 - * - * GET /api/database-sync/stats - */ - public function stats(Request $request): Response - { - try { - // 创建 DatabaseSyncService 实例（传递最小配置，仅用于读取统计） - $minimalConfig = [ - 'source' => ['host' => '', 'port' => 27017], - 'target' => ['host' => '', 'port' => 27017], - ]; - $syncService = new DatabaseSyncService($minimalConfig); - $stats = $syncService->getStats(); - - return ApiResponseHelper::success($stats, '统计信息查询成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 重置同步进度 - * - * POST /api/database-sync/reset - */ - public function reset(Request $request): Response - { - try { - // 创建 DatabaseSyncService 实例（传递最小配置，仅用于重置进度） - $minimalConfig = [ - 'source' => ['host' => '', 'port' => 27017], - 'target' => ['host' => '', 'port' => 27017], - ]; - $syncService = new DatabaseSyncService($minimalConfig); - $syncService->resetProgress(); - - return ApiResponseHelper::success(null, '同步进度已重置'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 跳过错误数据库，继续同步 - * - * POST /api/database-sync/skip-error - */ - public function skipError(Request $request): Response - { - try { - // 创建 DatabaseSyncService 实例（传递最小配置，仅用于跳过错误） - $minimalConfig = [ - 'source' => ['host' => '', 'port' => 27017], - 'target' => ['host' => '', 'port' => 27017], - ]; - $syncService = new DatabaseSyncService($minimalConfig); - $syncService->loadProgress(); - - $skipped = $syncService->skipErrorDatabase(); - - if ($skipped) { - return ApiResponseHelper::success(null, '已跳过错误数据库，将继续同步下一个数据库'); - } else { - return ApiResponseHelper::error('当前没有错误数据库需要跳过', 400); - } - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取数据库连接状态 - * - * @return array 连接状态信息 - */ - private function getConnectionStatus(): array - { - $status = [ - 'source' => [ - 'connected' => false, - 'host' => '', - 'port' => 0, - 'error' => null, - ], - 'target' => [ - 'connected' => false, - 'host' => '', - 'port' => 0, - 'error' => null, - ], - ]; - - try { - // 从数据库获取数据源配置 - $dataSourceService = new \app\service\DataSourceService(new \app\repository\DataSourceRepository()); - - // 检查源数据库连接 - $sourceDataSourceId = 'kr_mongodb'; // 默认源数据库ID，可以从任务配置中获取 - $sourceConfig = $dataSourceService->getDataSourceConfigById($sourceDataSourceId); - - if ($sourceConfig) { - $status['source']['host'] = $sourceConfig['host'] ?? ''; - $status['source']['port'] = (int)($sourceConfig['port'] ?? 27017); - - // 尝试连接源数据库 - try { - $sourceDsn = $this->buildDsn($sourceConfig); - $sourceClient = new \MongoDB\Client($sourceDsn, $sourceConfig['options'] ?? []); - // 执行一个简单的命令来测试连接 - $sourceClient->selectDatabase('admin')->command(['ping' => 1]); - $status['source']['connected'] = true; - } catch (\Throwable $e) { - $status['source']['connected'] = false; - $status['source']['error'] = $e->getMessage(); - } - } else { - $status['source']['error'] = '源数据库配置不存在'; - } - - // 检查目标数据库连接 - $targetDataSourceId = 'sync_mongodb'; // 默认目标数据库ID，可以从任务配置中获取 - $targetConfig = $dataSourceService->getDataSourceConfigById($targetDataSourceId); - - if ($targetConfig) { - $status['target']['host'] = $targetConfig['host'] ?? ''; - $status['target']['port'] = (int)($targetConfig['port'] ?? 27017); - - // 尝试连接目标数据库 - try { - $targetDsn = $this->buildDsn($targetConfig); - $targetClient = new \MongoDB\Client($targetDsn, $targetConfig['options'] ?? []); - // 执行一个简单的命令来测试连接 - $targetClient->selectDatabase('admin')->command(['ping' => 1]); - $status['target']['connected'] = true; - } catch (\Throwable $e) { - $status['target']['connected'] = false; - $status['target']['error'] = $e->getMessage(); - } - } else { - $status['target']['error'] = '目标数据库配置不存在'; - } - - } catch (\Throwable $e) { - $status['error'] = '检查连接状态失败: ' . $e->getMessage(); - } - - return $status; - } - - /** - * 构建 MongoDB DSN - * - * @param array $config 数据库配置 - * @return string DSN 字符串 - */ - private function buildDsn(array $config): string - { - $host = $config['host'] ?? ''; - $port = (int)($config['port'] ?? 27017); - - $dsn = 'mongodb://'; - if (!empty($config['username']) && !empty($config['password'])) { - $dsn .= urlencode($config['username']) . ':' . urlencode($config['password']) . '@'; - } - $dsn .= $host . ':' . $port; - if (!empty($config['auth_source'])) { - $dsn .= '/?authSource=' . urlencode($config['auth_source']); - } - return $dsn; - } - - /** - * 获取数据库列表信息（已完成和待同步） - * - * @param DatabaseSyncService $syncService 同步服务实例 - * @return array 数据库列表信息 - */ - private function getDatabaseList(DatabaseSyncService $syncService): array - { - $list = [ - 'completed' => [], - 'pending' => [], - 'processing' => [], - ]; - - try { - $runtimePath = function_exists('runtime_path') ? runtime_path() : (config('app.runtime_path', base_path() . DIRECTORY_SEPARATOR . 'runtime')); - $progressFile = $runtimePath . DIRECTORY_SEPARATOR . 'database_sync_progress.json'; - - if (file_exists($progressFile)) { - $fp = fopen($progressFile, 'r'); - if ($fp && flock($fp, LOCK_SH)) { - try { - $content = stream_get_contents($fp); - $progressData = json_decode($content, true); - - if ($progressData) { - $checkpoints = $progressData['checkpoints'] ?? []; - $collectionsSnapshot = $progressData['collections_snapshot'] ?? []; - $currentDatabase = $progressData['current_database'] ?? null; - - // 获取所有数据库名称 - $allDatabases = array_keys($collectionsSnapshot); - - foreach ($allDatabases as $database) { - $dbCheckpoints = $checkpoints[$database] ?? []; - - // 检查该数据库的所有集合是否都已完成 - $collections = $collectionsSnapshot[$database] ?? []; - $allCompleted = true; - $hasData = false; - - foreach ($collections as $collection) { - $checkpoint = $dbCheckpoints[$collection] ?? null; - if ($checkpoint) { - $hasData = true; - if (!($checkpoint['completed'] ?? false)) { - $allCompleted = false; - break; - } - } else { - $allCompleted = false; - } - } - - if ($database === $currentDatabase) { - $list['processing'][] = [ - 'name' => $database, - 'collections' => count($collections), - 'collections_completed' => count(array_filter($dbCheckpoints, fn($cp) => $cp['completed'] ?? false)), - ]; - } elseif ($allCompleted && $hasData) { - $list['completed'][] = [ - 'name' => $database, - 'collections' => count($collections), - ]; - } else { - $list['pending'][] = [ - 'name' => $database, - 'collections' => count($collections), - ]; - } - } - } - } finally { - flock($fp, LOCK_UN); - fclose($fp); - } - } else { - if ($fp) { - fclose($fp); - } - } - } - } catch (\Throwable $e) { - // 忽略错误，返回空列表 - } - - return $list; - } -} diff --git a/Moncter/Moncter/app/controller/IndexController.php b/Moncter/Moncter/app/controller/IndexController.php deleted file mode 100644 index 688d17ba..00000000 --- a/Moncter/Moncter/app/controller/IndexController.php +++ /dev/null @@ -1,154 +0,0 @@ - 'webman']); - } - - public function json(Request $request) - { - return json(['code' => 0, 'msg' => 'ok']); - } - - /** - * 测试 MongoDB 数据库连接 - * GET /api/test/db - */ - public function testDb(Request $request) - { - $result = [ - 'code' => 0, - 'msg' => 'ok', - 'data' => [ - 'config' => [], - 'connection' => [], - 'test_query' => [], - ], - ]; - - try { - // 读取数据库配置 - $dbConfig = config('database', []); - $mongoConfig = $dbConfig['connections']['mongodb'] ?? null; - - if (!$mongoConfig) { - throw new \Exception('MongoDB 配置不存在'); - } - - $result['data']['config'] = [ - 'driver' => $mongoConfig['driver'] ?? 'unknown', - 'database' => $mongoConfig['database'] ?? 'unknown', - 'dsn' => $mongoConfig['dsn'] ?? 'unknown', - 'has_username' => !empty($mongoConfig['username']), - 'has_password' => !empty($mongoConfig['password']), - ]; - - // 尝试使用 MongoDB 客户端直接连接 - try { - // 构建包含认证信息的 DSN（如果配置了用户名和密码） - $dsn = $mongoConfig['dsn']; - if (!empty($mongoConfig['username']) && !empty($mongoConfig['password'])) { - // 如果 DSN 中不包含认证信息，则添加 - if (strpos($dsn, '@') === false) { - // 从 mongodb://host:port 格式转换为 mongodb://username:password@host:port/database - $dsn = str_replace( - 'mongodb://', - 'mongodb://' . urlencode($mongoConfig['username']) . ':' . urlencode($mongoConfig['password']) . '@', - $dsn - ); - // 添加数据库名和认证源 - $dsn .= '/' . $mongoConfig['database']; - if (!empty($mongoConfig['options']['authSource'])) { - $dsn .= '?authSource=' . urlencode($mongoConfig['options']['authSource']); - } - } - } - - // 过滤掉空字符串的选项（MongoDB 客户端不允许空字符串） - $options = array_filter($mongoConfig['options'] ?? [], function ($value) { - return $value !== ''; - }); - - $client = new \MongoDB\Client( - $dsn, - $options - ); - - // 尝试执行 ping 命令 - $adminDb = $client->selectDatabase('admin'); - $pingResult = $adminDb->command(['ping' => 1])->toArray(); - - $result['data']['connection'] = [ - 'status' => 'connected', - 'ping' => 'ok', - 'server_info' => $client->getManager()->getServers(), - ]; - - // 尝试选择目标数据库并列出集合 - $targetDb = $client->selectDatabase($mongoConfig['database']); - $collections = $targetDb->listCollections(); - $collectionNames = []; - foreach ($collections as $collection) { - $collectionNames[] = $collection->getName(); - } - - $result['data']['test_query'] = [ - 'database' => $mongoConfig['database'], - 'collections_count' => count($collectionNames), - 'collections' => $collectionNames, - ]; - - } catch (\MongoDB\Driver\Exception\Exception $e) { - $result['data']['connection'] = [ - 'status' => 'failed', - 'error' => $e->getMessage(), - 'code' => $e->getCode(), - ]; - $result['code'] = 500; - $result['msg'] = 'MongoDB 连接失败'; - } - - // 尝试使用 Repository 查询（如果连接成功） - if ($result['data']['connection']['status'] === 'connected') { - try { - $userRepo = new \app\repository\UserProfileRepository(); - $count = $userRepo->newQuery()->count(); - $result['data']['repository_test'] = [ - 'status' => 'ok', - 'user_profile_count' => $count, - ]; - } catch (\Throwable $e) { - $result['data']['repository_test'] = [ - 'status' => 'failed', - 'error' => $e->getMessage(), - ]; - } - } - - } catch (\Throwable $e) { - $result = [ - 'code' => 500, - 'msg' => '测试失败: ' . $e->getMessage(), - 'data' => [ - 'error' => $e->getMessage(), - 'file' => $e->getFile(), - 'line' => $e->getLine(), - ], - ]; - } - - return json($result); - } - -} diff --git a/Moncter/Moncter/app/controller/PersonMergeController.php b/Moncter/Moncter/app/controller/PersonMergeController.php deleted file mode 100644 index 6173e0cd..00000000 --- a/Moncter/Moncter/app/controller/PersonMergeController.php +++ /dev/null @@ -1,169 +0,0 @@ -rawBody(); - - if (empty($rawBody)) { - return ApiResponseHelper::error('请求体为空，请确保 Content-Type 为 application/json 并发送有效的 JSON 数据', 400); - } - - $body = json_decode($rawBody, true); - - if (json_last_error() !== JSON_ERROR_NONE) { - return ApiResponseHelper::error('JSON 格式错误: ' . json_last_error_msg(), 400); - } - - // 验证必填字段 - if (empty($body['phone_number'])) { - throw new \InvalidArgumentException('缺少必填字段：phone_number'); - } - if (empty($body['id_card'])) { - throw new \InvalidArgumentException('缺少必填字段：id_card'); - } - - $phoneNumber = (string)$body['phone_number']; - $idCard = (string)$body['id_card']; - - // 创建服务实例 - $mergeService = new PersonMergeService( - new UserProfileRepository(), - new UserTagRepository(), - new UserPhoneService( - new UserPhoneRelationRepository() - ), - new TagService( - new TagDefinitionRepository(), - new UserProfileRepository(), - new UserTagRepository(), - new TagHistoryRepository(), - new SimpleRuleEngine() - ) - ); - - // 执行合并 - $formalUserId = $mergeService->mergePhoneToIdCard($phoneNumber, $idCard); - - LoggerHelper::logBusiness('person_merge_phone_to_id_card', [ - 'phone_number' => $phoneNumber, - 'id_card_provided' => true, - 'formal_user_id' => $formalUserId, - ]); - - return ApiResponseHelper::success([ - 'phone_number' => $phoneNumber, - 'formal_user_id' => $formalUserId, - 'message' => '身份合并成功，标签已重新计算', - ]); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 合并临时人到正式人 - * - * POST /api/person-merge/temporary-to-formal - */ - public function mergeTemporaryToFormal(Request $request): Response - { - try { - LoggerHelper::logRequest('POST', '/api/person-merge/temporary-to-formal'); - - $rawBody = $request->rawBody(); - - if (empty($rawBody)) { - return ApiResponseHelper::error('请求体为空，请确保 Content-Type 为 application/json 并发送有效的 JSON 数据', 400); - } - - $body = json_decode($rawBody, true); - - if (json_last_error() !== JSON_ERROR_NONE) { - return ApiResponseHelper::error('JSON 格式错误: ' . json_last_error_msg(), 400); - } - - // 验证必填字段 - if (empty($body['user_id'])) { - throw new \InvalidArgumentException('缺少必填字段：user_id'); - } - if (empty($body['id_card'])) { - throw new \InvalidArgumentException('缺少必填字段：id_card'); - } - - $tempUserId = (string)$body['user_id']; - $idCard = (string)$body['id_card']; - - // 创建服务实例 - $mergeService = new PersonMergeService( - new UserProfileRepository(), - new UserTagRepository(), - new UserPhoneService( - new UserPhoneRelationRepository() - ), - new TagService( - new TagDefinitionRepository(), - new UserProfileRepository(), - new UserTagRepository(), - new TagHistoryRepository(), - new SimpleRuleEngine() - ) - ); - - // 执行合并 - $formalUserId = $mergeService->mergeTemporaryToFormal($tempUserId, $idCard); - - LoggerHelper::logBusiness('person_merge_temporary_to_formal', [ - 'temp_user_id' => $tempUserId, - 'formal_user_id' => $formalUserId, - ]); - - return ApiResponseHelper::success([ - 'temp_user_id' => $tempUserId, - 'formal_user_id' => $formalUserId, - 'message' => '临时人已转为正式人，标签已重新计算', - ]); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } -} - diff --git a/Moncter/Moncter/app/controller/TagCohortController.php b/Moncter/Moncter/app/controller/TagCohortController.php deleted file mode 100644 index 217c0076..00000000 --- a/Moncter/Moncter/app/controller/TagCohortController.php +++ /dev/null @@ -1,308 +0,0 @@ -get('page') ?? 1); - $pageSize = (int)($request->get('page_size') ?? 20); - - if ($page < 1) { - $page = 1; - } - if ($pageSize < 1 || $pageSize > 100) { - $pageSize = 20; - } - - $cohortRepo = new TagCohortRepository(); - - $total = $cohortRepo->newQuery()->count(); - - $cohorts = $cohortRepo->newQuery() - ->orderBy('created_at', 'desc') - ->skip(($page - 1) * $pageSize) - ->take($pageSize) - ->get(); - - $result = []; - foreach ($cohorts as $cohort) { - $result[] = [ - 'cohort_id' => $cohort->cohort_id, - 'name' => $cohort->name, - 'user_count' => $cohort->user_count ?? 0, - 'created_at' => $cohort->created_at ? $cohort->created_at->format('Y-m-d H:i:s') : null, - ]; - } - - LoggerHelper::logBusiness('get_tag_cohort_list', [ - 'total' => $total, - 'page' => $page, - ]); - - return ApiResponseHelper::success([ - 'cohorts' => $result, - 'total' => $total, - 'page' => $page, - 'page_size' => $pageSize, - ]); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取人群快照详情 - * - * GET /api/tag-cohorts/{cohort_id} - */ - public function detail(Request $request, string $cohortId): Response - { - try { - LoggerHelper::logRequest('GET', "/api/tag-cohorts/{$cohortId}"); - - $cohortRepo = new TagCohortRepository(); - $cohort = $cohortRepo->newQuery()->where('cohort_id', $cohortId)->first(); - - if (!$cohort) { - return ApiResponseHelper::error('人群快照不存在', 404, 404); - } - - $result = [ - 'cohort_id' => $cohort->cohort_id, - 'name' => $cohort->name, - 'description' => $cohort->description ?? '', - 'conditions' => $cohort->conditions ?? [], - 'logic' => $cohort->logic ?? 'AND', - 'user_ids' => $cohort->user_ids ?? [], - 'user_count' => $cohort->user_count ?? 0, - 'created_at' => $cohort->created_at ? $cohort->created_at->format('Y-m-d H:i:s') : null, - ]; - - LoggerHelper::logBusiness('get_tag_cohort_detail', [ - 'cohort_id' => $cohortId, - ]); - - return ApiResponseHelper::success($result); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 创建人群快照 - * - * POST /api/tag-cohorts - */ - public function create(Request $request): Response - { - try { - LoggerHelper::logRequest('POST', '/api/tag-cohorts'); - - $rawBody = $request->rawBody(); - - if (empty($rawBody)) { - return ApiResponseHelper::error('请求体为空，请确保 Content-Type 为 application/json 并发送有效的 JSON 数据', 400); - } - - $body = json_decode($rawBody, true); - - if (json_last_error() !== JSON_ERROR_NONE) { - return ApiResponseHelper::error('JSON 格式错误: ' . json_last_error_msg(), 400); - } - - // 验证必填字段 - if (empty($body['name'])) { - throw new \InvalidArgumentException('缺少必填字段：name'); - } - - if (empty($body['conditions']) || !is_array($body['conditions'])) { - throw new \InvalidArgumentException('缺少必填字段：conditions（必须为数组）'); - } - - if (empty($body['user_ids']) || !is_array($body['user_ids'])) { - throw new \InvalidArgumentException('缺少必填字段：user_ids（必须为数组）'); - } - - // 使用标签筛选服务获取用户列表 - $tagService = new TagService( - new TagDefinitionRepository(), - new UserProfileRepository(), - new UserTagRepository(), - new TagHistoryRepository(), - new SimpleRuleEngine() - ); - - $conditions = $body['conditions']; - $logic = $body['logic'] ?? 'AND'; - - // 筛选用户（获取所有用户，不包含用户信息） - $filterResult = $tagService->filterUsersByTags( - $conditions, - $logic, - 1, - 10000, // 最多获取10000个用户 - false - ); - - // 从返回结果中提取用户ID - $userIds = []; - if (isset($filterResult['users']) && is_array($filterResult['users'])) { - foreach ($filterResult['users'] as $user) { - if (isset($user['user_id'])) { - $userIds[] = $user['user_id']; - } elseif (is_string($user)) { - // 如果直接返回的是用户ID字符串 - $userIds[] = $user; - } - } - } - - // 如果用户提供了 user_ids，使用提供的列表（优先级更高） - if (!empty($body['user_ids']) && is_array($body['user_ids'])) { - $userIds = $body['user_ids']; - } - - // 创建人群快照 - $cohortRepo = new TagCohortRepository(); - $cohort = new TagCohortRepository(); - $cohort->cohort_id = Uuid::uuid4()->toString(); - $cohort->name = $body['name']; - $cohort->description = $body['description'] ?? ''; - $cohort->conditions = $conditions; - $cohort->logic = $logic; - $cohort->user_ids = $userIds; - $cohort->user_count = count($userIds); - $cohort->created_by = $body['created_by'] ?? 'system'; - $cohort->created_at = new \DateTime(); - $cohort->updated_at = new \DateTime(); - $cohort->save(); - - LoggerHelper::logBusiness('create_tag_cohort', [ - 'cohort_id' => $cohort->cohort_id, - 'name' => $cohort->name, - 'user_count' => $cohort->user_count, - ]); - - return ApiResponseHelper::success([ - 'cohort_id' => $cohort->cohort_id, - 'name' => $cohort->name, - 'user_count' => $cohort->user_count, - ], '人群快照创建成功'); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 删除人群快照 - * - * DELETE /api/tag-cohorts/{cohort_id} - */ - public function delete(Request $request, string $cohortId): Response - { - try { - LoggerHelper::logRequest('DELETE', "/api/tag-cohorts/{$cohortId}"); - - $cohortRepo = new TagCohortRepository(); - $cohort = $cohortRepo->newQuery()->where('cohort_id', $cohortId)->first(); - - if (!$cohort) { - return ApiResponseHelper::error('人群快照不存在', 404, 404); - } - - $cohort->delete(); - - LoggerHelper::logBusiness('delete_tag_cohort', [ - 'cohort_id' => $cohortId, - ]); - - return ApiResponseHelper::success(null, '人群快照删除成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 导出人群快照 - * - * POST /api/tag-cohorts/{cohort_id}/export - */ - public function export(Request $request, string $cohortId): Response - { - try { - LoggerHelper::logRequest('POST', "/api/tag-cohorts/{$cohortId}/export"); - - $cohortRepo = new TagCohortRepository(); - $cohort = $cohortRepo->newQuery()->where('cohort_id', $cohortId)->first(); - - if (!$cohort) { - return ApiResponseHelper::error('人群快照不存在', 404, 404); - } - - $userIds = $cohort->user_ids ?? []; - $userProfileRepo = new UserProfileRepository(); - - // 获取用户信息 - $users = []; - foreach ($userIds as $userId) { - $user = $userProfileRepo->findByUserId($userId); - if ($user) { - $users[] = [ - 'user_id' => $user->user_id, - 'phone' => $user->phone ?? '', - 'name' => $user->name ?? '', - ]; - } - } - - // 生成 CSV 内容 - $csvContent = "用户ID,手机号,姓名\n"; - foreach ($users as $user) { - $csvContent .= sprintf( - "%s,%s,%s\n", - $user['user_id'], - $user['phone'], - $user['name'] - ); - } - - LoggerHelper::logBusiness('export_tag_cohort', [ - 'cohort_id' => $cohortId, - 'user_count' => count($users), - ]); - - // 返回 CSV 文件 - return response($csvContent) - ->header('Content-Type', 'text/csv; charset=utf-8') - ->header('Content-Disposition', "attachment; filename=\"cohort_{$cohortId}.csv\""); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } -} - diff --git a/Moncter/Moncter/app/controller/TagController.php b/Moncter/Moncter/app/controller/TagController.php deleted file mode 100644 index 630fafc8..00000000 --- a/Moncter/Moncter/app/controller/TagController.php +++ /dev/null @@ -1,521 +0,0 @@ -path(); - if (preg_match('#/api/users/([^/]+)/tags#', $path, $matches)) { - $userId = $matches[1]; - } else { - // 如果路径解析失败，尝试从查询参数获取 - $userId = $request->get('user_id'); - if (!$userId) { - throw new \InvalidArgumentException('缺少 user_id 参数'); - } - } - - LoggerHelper::logRequest('GET', $path, ['user_id' => $userId]); - - $userTagRepo = new UserTagRepository(); - $tagDefRepo = new TagDefinitionRepository(); - - // 查询用户的所有标签 - $userTags = $userTagRepo->newQuery() - ->where('user_id', $userId) - ->get(); - - // 关联标签定义信息 - $result = []; - foreach ($userTags as $userTag) { - $tagDef = $tagDefRepo->newQuery() - ->where('tag_id', $userTag->tag_id) - ->first(); - - $result[] = [ - 'tag_id' => $userTag->tag_id, - 'tag_code' => $tagDef ? $tagDef->tag_code : null, - 'tag_name' => $tagDef ? $tagDef->tag_name : null, - 'category' => $tagDef ? $tagDef->category : null, - 'tag_value' => $userTag->tag_value, - 'tag_value_type' => $userTag->tag_value_type, - 'confidence' => $userTag->confidence, - 'effective_time' => $userTag->effective_time, - 'expire_time' => $userTag->expire_time, - 'update_time' => $userTag->update_time, - ]; - } - - LoggerHelper::logBusiness('get_user_tags', [ - 'user_id' => $userId, - 'tag_count' => count($result), - ]); - - return ApiResponseHelper::success([ - 'user_id' => $userId, - 'tags' => $result, - 'count' => count($result), - ]); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 更新/计算用户标签 - * - * PUT /api/users/{user_id}/tags - */ - public function calculate(Request $request): Response - { - $startTime = microtime(true); - - try { - // 从请求路径中解析 user_id - $path = $request->path(); - if (preg_match('#/api/users/([^/]+)/tags#', $path, $matches)) { - $userId = $matches[1]; - } else { - // 如果路径解析失败，尝试从查询参数获取 - $userId = $request->get('user_id'); - if (!$userId) { - throw new \InvalidArgumentException('缺少 user_id 参数'); - } - } - - LoggerHelper::logRequest('PUT', $path, ['user_id' => $userId]); - - $tagService = new TagService( - new TagDefinitionRepository(), - new UserProfileRepository(), - new UserTagRepository(), - new TagHistoryRepository(), - new SimpleRuleEngine() - ); - - $tags = $tagService->calculateTags($userId); - - $duration = microtime(true) - $startTime; - LoggerHelper::logBusiness('calculate_tags', [ - 'user_id' => $userId, - 'updated_count' => count($tags), - ], 'info'); - LoggerHelper::logPerformance('tag_calculation', $duration, [ - 'user_id' => $userId, - 'tag_count' => count($tags), - ]); - - return ApiResponseHelper::success([ - 'user_id' => $userId, - 'updated_tags' => $tags, - 'count' => count($tags), - ]); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 删除用户的指定标签 - * - * DELETE /api/users/{user_id}/tags/{tag_id} - */ - public function destroy(Request $request): Response - { - try { - // 从请求路径中解析 user_id 和 tag_id - $path = $request->path(); - if (preg_match('#/api/users/([^/]+)/tags/([^/]+)#', $path, $matches)) { - $userId = $matches[1]; - $tagId = $matches[2]; - } else { - throw new \InvalidArgumentException('缺少 user_id 或 tag_id 参数'); - } - - LoggerHelper::logRequest('DELETE', $path, ['user_id' => $userId, 'tag_id' => $tagId]); - - $tagService = new TagService( - new TagDefinitionRepository(), - new UserProfileRepository(), - new UserTagRepository(), - new TagHistoryRepository(), - new SimpleRuleEngine() - ); - - $deleted = $tagService->deleteUserTag($userId, $tagId); - - if (!$deleted) { - return ApiResponseHelper::error('标签不存在', 404, 404); - } - - LoggerHelper::logBusiness('tag_deleted', [ - 'user_id' => $userId, - 'tag_id' => $tagId, - ]); - - return ApiResponseHelper::success(null, '标签删除成功'); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 根据标签筛选用户 - * - * POST /api/tags/filter - */ - public function filter(Request $request): Response - { - try { - LoggerHelper::logRequest('POST', '/api/tags/filter'); - - $rawBody = $request->rawBody(); - - if (empty($rawBody)) { - return ApiResponseHelper::error('请求体为空，请确保 Content-Type 为 application/json 并发送有效的 JSON 数据', 400); - } - - $body = json_decode($rawBody, true); - - if (json_last_error() !== JSON_ERROR_NONE) { - return ApiResponseHelper::error('JSON 格式错误: ' . json_last_error_msg(), 400); - } - - // 验证必填字段 - if (empty($body['tag_conditions']) || !is_array($body['tag_conditions'])) { - throw new \InvalidArgumentException('缺少必填字段：tag_conditions（必须为数组）'); - } - - // 验证条件格式 - foreach ($body['tag_conditions'] as $condition) { - if (!isset($condition['tag_code']) || !isset($condition['operator']) || !isset($condition['value'])) { - throw new \InvalidArgumentException('每个条件必须包含 tag_code、operator 和 value 字段'); - } - } - - $tagService = new TagService( - new TagDefinitionRepository(), - new UserProfileRepository(), - new UserTagRepository(), - new TagHistoryRepository(), - new SimpleRuleEngine() - ); - - $conditions = $body['tag_conditions']; - $logic = $body['logic'] ?? 'AND'; - $page = (int)($body['page'] ?? 1); - $pageSize = (int)($body['page_size'] ?? 20); - $includeUserInfo = (bool)($body['include_user_info'] ?? false); - - if ($page < 1) { - $page = 1; - } - if ($pageSize < 1 || $pageSize > 100) { - $pageSize = 20; - } - - $result = $tagService->filterUsersByTags( - $conditions, - $logic, - $page, - $pageSize, - $includeUserInfo - ); - - // 对返回的用户信息进行脱敏处理 - if ($includeUserInfo && isset($result['users']) && is_array($result['users'])) { - foreach ($result['users'] as &$user) { - $user = DataMaskingHelper::maskArray($user, ['phone', 'email']); - } - unset($user); - } - - LoggerHelper::logBusiness('filter_users_by_tags', [ - 'conditions_count' => count($conditions), - 'logic' => $logic, - 'result_count' => $result['total'] ?? 0, - ]); - - return ApiResponseHelper::success($result); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 批量初始化标签定义 - * - * POST /api/tag-definitions/batch - */ - public function init(Request $request): Response - { - try { - LoggerHelper::logRequest('POST', '/api/tag-definitions/batch'); - - $initService = new \app\service\TagInitService( - new TagDefinitionRepository() - ); - - $initService->initBasicTags(); - - LoggerHelper::logBusiness('init_tags', []); - - return ApiResponseHelper::success([ - 'message' => '标签初始化完成', - ]); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取标签统计信息 - * - * GET /api/tags/statistics - */ - public function statistics(Request $request): Response - { - try { - LoggerHelper::logRequest('GET', '/api/tags/statistics'); - - $tagId = $request->get('tag_id'); - $startDate = $request->get('start_date'); - $endDate = $request->get('end_date'); - - $userTagRepo = new UserTagRepository(); - $tagDefRepo = new TagDefinitionRepository(); - - $result = [ - 'value_distribution' => [], - 'trend_data' => [], - 'coverage_stats' => [], - ]; - - // 如果指定了 tag_id，统计该标签的值分布 - if ($tagId) { - $tagDef = $tagDefRepo->newQuery()->where('tag_id', $tagId)->first(); - if ($tagDef) { - // 统计标签值分布 - $userTags = $userTagRepo->newQuery() - ->where('tag_id', $tagId) - ->get(['tag_value']); - - $valueCounts = []; - foreach ($userTags as $userTag) { - $value = (string)$userTag->tag_value; - if (!isset($valueCounts[$value])) { - $valueCounts[$value] = 0; - } - $valueCounts[$value]++; - } - - // 按数量排序 - arsort($valueCounts); - $valueCounts = array_slice($valueCounts, 0, 20, true); - - foreach ($valueCounts as $value => $count) { - $result['value_distribution'][] = [ - 'value' => $value, - 'count' => $count - ]; - } - - // 统计标签覆盖度 - $totalUsers = $userTagRepo->newQuery() - ->distinct('user_id') - ->count(); - - $taggedUsers = $userTagRepo->newQuery() - ->where('tag_id', $tagId) - ->distinct('user_id') - ->count(); - - $result['coverage_stats'][] = [ - 'tag_id' => $tagId, - 'tag_name' => $tagDef->tag_name ?? '', - 'total_users' => $totalUsers, - 'tagged_users' => $taggedUsers, - 'coverage_rate' => $totalUsers > 0 ? round($taggedUsers / $totalUsers * 100, 2) : 0 - ]; - } - } else { - // 统计所有标签的覆盖度 - $tagDefs = $tagDefRepo->newQuery()->where('status', 1)->get(); - $totalUsers = $userTagRepo->newQuery()->distinct('user_id')->count(); - - foreach ($tagDefs as $tagDef) { - $taggedUsers = $userTagRepo->newQuery() - ->where('tag_id', $tagDef->tag_id) - ->distinct('user_id') - ->count(); - - $result['coverage_stats'][] = [ - 'tag_id' => $tagDef->tag_id, - 'tag_name' => $tagDef->tag_name ?? '', - 'total_users' => $totalUsers, - 'tagged_users' => $taggedUsers, - 'coverage_rate' => $totalUsers > 0 ? round($taggedUsers / $totalUsers * 100, 2) : 0 - ]; - } - } - - // 趋势数据（如果有时间范围） - if ($startDate && $endDate) { - $historyRepo = new TagHistoryRepository(); - $start = new \DateTime($startDate); - $end = new \DateTime($endDate); - - // 按日期统计标签变更次数 - $trendData = []; - $current = clone $start; - while ($current <= $end) { - $dateStr = $current->format('Y-m-d'); - $nextDay = clone $current; - $nextDay->modify('+1 day'); - - $count = $historyRepo->newQuery() - ->where('change_time', '>=', $current) - ->where('change_time', '<', $nextDay) - ->count(); - - $trendData[] = [ - 'date' => $dateStr, - 'count' => $count - ]; - - $current->modify('+1 day'); - } - - $result['trend_data'] = $trendData; - } - - LoggerHelper::logBusiness('get_tag_statistics', [ - 'tag_id' => $tagId, - 'start_date' => $startDate, - 'end_date' => $endDate, - ]); - - return ApiResponseHelper::success($result); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取标签历史记录 - * - * GET /api/tags/history - */ - public function history(Request $request): Response - { - try { - LoggerHelper::logRequest('GET', '/api/tags/history'); - - $userId = $request->get('user_id'); - $tagId = $request->get('tag_id'); - $startDate = $request->get('start_date'); - $endDate = $request->get('end_date'); - $page = (int)($request->get('page') ?? 1); - $pageSize = (int)($request->get('page_size') ?? 20); - - if ($page < 1) { - $page = 1; - } - if ($pageSize < 1 || $pageSize > 100) { - $pageSize = 20; - } - - $historyRepo = new TagHistoryRepository(); - $tagDefRepo = new TagDefinitionRepository(); - - $query = $historyRepo->newQuery(); - - if ($userId) { - $query->where('user_id', $userId); - } - - if ($tagId) { - $query->where('tag_id', $tagId); - } - - if ($startDate) { - $query->where('change_time', '>=', new \DateTime($startDate)); - } - - if ($endDate) { - $endDateTime = new \DateTime($endDate); - $endDateTime->modify('+1 day'); - $query->where('change_time', '<', $endDateTime); - } - - $total = $query->count(); - - $histories = $query->orderBy('change_time', 'desc') - ->skip(($page - 1) * $pageSize) - ->take($pageSize) - ->get(); - - $items = []; - foreach ($histories as $history) { - $tagDef = $tagDefRepo->newQuery()->where('tag_id', $history->tag_id)->first(); - - $items[] = [ - 'user_id' => $history->user_id, - 'tag_id' => $history->tag_id, - 'tag_name' => $tagDef ? $tagDef->tag_name : null, - 'old_value' => $history->old_value, - 'new_value' => $history->new_value, - 'change_reason' => $history->change_reason, - 'change_time' => $history->change_time ? $history->change_time->format('Y-m-d H:i:s') : null, - 'operator' => $history->operator, - ]; - } - - LoggerHelper::logBusiness('get_tag_history', [ - 'user_id' => $userId, - 'tag_id' => $tagId, - 'total' => $total, - ]); - - return ApiResponseHelper::success([ - 'items' => $items, - 'total' => $total, - 'page' => $page, - 'page_size' => $pageSize, - ]); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - -} - diff --git a/Moncter/Moncter/app/controller/TagDefinitionController.php b/Moncter/Moncter/app/controller/TagDefinitionController.php deleted file mode 100644 index def47f88..00000000 --- a/Moncter/Moncter/app/controller/TagDefinitionController.php +++ /dev/null @@ -1,155 +0,0 @@ -query(); - - // 筛选条件 - if ($request->get('category')) { - $query->where('category', $request->get('category')); - } - if ($request->get('status')) { - $query->where('status', $request->get('status')); - } - if ($request->get('name')) { - $query->where('tag_name', 'like', '%' . $request->get('name') . '%'); - } - - $page = (int)($request->get('page', 1)); - $pageSize = (int)($request->get('page_size', 20)); - - $total = $query->count(); - $definitions = $query->orderBy('created_at', 'desc') - ->skip(($page - 1) * $pageSize) - ->take($pageSize) - ->get() - ->toArray(); - - return ApiResponseHelper::success([ - 'definitions' => $definitions, - 'total' => $total, - 'page' => $page, - 'page_size' => $pageSize, - 'total_pages' => ceil($total / $pageSize), - ], '查询成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取标签定义详情 - * - * GET /api/tag-definitions/{tag_id} - */ - public function detail(Request $request, string $tagId): Response - { - try { - $repo = new TagDefinitionRepository(); - $definition = $repo->find($tagId); - - if (!$definition) { - return ApiResponseHelper::error('标签定义不存在', 404); - } - - return ApiResponseHelper::success($definition->toArray(), '查询成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 创建标签定义 - * - * POST /api/tag-definitions - */ - public function create(Request $request): Response - { - try { - $data = $request->post(); - - $requiredFields = ['tag_code', 'tag_name', 'category']; - foreach ($requiredFields as $field) { - if (empty($data[$field])) { - return ApiResponseHelper::error("缺少必填字段: {$field}", 400); - } - } - - $repo = new TagDefinitionRepository(); - $definition = $repo->create($data); - - return ApiResponseHelper::success($definition->toArray(), '创建成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 更新标签定义 - * - * PUT /api/tag-definitions/{tag_id} - */ - public function update(Request $request, string $tagId): Response - { - try { - $data = $request->post(); - - $repo = new TagDefinitionRepository(); - $definition = $repo->find($tagId); - - if (!$definition) { - return ApiResponseHelper::error('标签定义不存在', 404); - } - - $definition->fill($data); - $definition->save(); - - return ApiResponseHelper::success($definition->toArray(), '更新成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 删除标签定义 - * - * DELETE /api/tag-definitions/{tag_id} - */ - public function delete(Request $request, string $tagId): Response - { - try { - $repo = new TagDefinitionRepository(); - $definition = $repo->find($tagId); - - if (!$definition) { - return ApiResponseHelper::error('标签定义不存在', 404); - } - - $definition->delete(); - - return ApiResponseHelper::success(null, '删除成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } -} - diff --git a/Moncter/Moncter/app/controller/TagTaskController.php b/Moncter/Moncter/app/controller/TagTaskController.php deleted file mode 100644 index b8bb81c1..00000000 --- a/Moncter/Moncter/app/controller/TagTaskController.php +++ /dev/null @@ -1,227 +0,0 @@ -post(); - - $requiredFields = ['name', 'task_type']; - foreach ($requiredFields as $field) { - if (empty($data[$field])) { - return ApiResponseHelper::error("缺少必填字段: {$field}", 400); - } - } - - $service = $this->getService(); - $task = $service->createTask($data); - - return ApiResponseHelper::success($task, '任务创建成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 更新任务 - * - * PUT /api/tag-tasks/{task_id} - */ - public function update(Request $request, string $taskId): Response - { - try { - $data = $request->post(); - - $service = $this->getService(); - $result = $service->updateTask($taskId, $data); - - return ApiResponseHelper::success(null, '任务更新成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 删除任务 - * - * DELETE /api/tag-tasks/{task_id} - */ - public function delete(Request $request, string $taskId): Response - { - try { - $service = $this->getService(); - $result = $service->deleteTask($taskId); - - return ApiResponseHelper::success(null, '任务删除成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 启动任务 - * - * POST /api/tag-tasks/{task_id}/start - */ - public function start(Request $request, string $taskId): Response - { - try { - $service = $this->getService(); - $result = $service->startTask($taskId); - - return ApiResponseHelper::success(null, '任务启动成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 暂停任务 - * - * POST /api/tag-tasks/{task_id}/pause - */ - public function pause(Request $request, string $taskId): Response - { - try { - $service = $this->getService(); - $result = $service->pauseTask($taskId); - - return ApiResponseHelper::success(null, '任务暂停成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 停止任务 - * - * POST /api/tag-tasks/{task_id}/stop - */ - public function stop(Request $request, string $taskId): Response - { - try { - $service = $this->getService(); - $result = $service->stopTask($taskId); - - return ApiResponseHelper::success(null, '任务停止成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取任务列表 - * - * GET /api/tag-tasks - */ - public function list(Request $request): Response - { - try { - $filters = [ - 'status' => $request->get('status'), - 'task_type' => $request->get('task_type'), - 'name' => $request->get('name'), - ]; - - $page = (int)($request->get('page', 1)); - $pageSize = (int)($request->get('page_size', 20)); - - $service = $this->getService(); - $result = $service->getTaskList($filters, $page, $pageSize); - - return ApiResponseHelper::success($result, '查询成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取任务详情 - * - * GET /api/tag-tasks/{task_id} - */ - public function detail(Request $request, string $taskId): Response - { - try { - $service = $this->getService(); - $task = $service->getTask($taskId); - - if ($task === null) { - return ApiResponseHelper::error('任务不存在', 404); - } - - return ApiResponseHelper::success($task, '查询成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取任务执行记录 - * - * GET /api/tag-tasks/{task_id}/executions - */ - public function executions(Request $request, string $taskId): Response - { - try { - $page = (int)($request->get('page', 1)); - $pageSize = (int)($request->get('page_size', 20)); - - $service = $this->getService(); - $result = $service->getExecutions($taskId, $page, $pageSize); - - return ApiResponseHelper::success($result, '查询成功'); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 获取服务实例 - */ - private function getService(): TagTaskService - { - return new TagTaskService( - new TagTaskRepository(), - new TagTaskExecutionRepository(), - new UserProfileRepository(), - new TagService( - new TagDefinitionRepository(), - new UserProfileRepository(), - new UserTagRepository(), - new TagHistoryRepository(), - new SimpleRuleEngine() - ) - ); - } -} - diff --git a/Moncter/Moncter/app/controller/UserController.php b/Moncter/Moncter/app/controller/UserController.php deleted file mode 100644 index 9835c82a..00000000 --- a/Moncter/Moncter/app/controller/UserController.php +++ /dev/null @@ -1,557 +0,0 @@ -rawBody(); - - // 调试：记录原始请求体 - if (empty($rawBody)) { - return ApiResponseHelper::error('请求体为空，请确保 Content-Type 为 application/json 并发送有效的 JSON 数据', 400); - } - - $body = json_decode($rawBody, true); - if (json_last_error() !== JSON_ERROR_NONE) { - $errorMsg = '请求体必须是有效的 JSON 格式'; - $jsonError = json_last_error_msg(); - if ($jsonError) { - $errorMsg .= ': ' . $jsonError; - } - // 开发环境输出更多调试信息 - if (getenv('APP_DEBUG') === 'true') { - $errorMsg .= ' (原始请求体: ' . substr($rawBody, 0, 200) . ')'; - } - return ApiResponseHelper::error($errorMsg, 400); - } - - $userService = new UserService(new UserProfileRepository()); - $result = $userService->createUser($body); - - return ApiResponseHelper::success($result, '用户创建成功'); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 查询用户信息 - * - * GET /api/users/{user_id}?decrypt_id_card=1 - * - * @param Request $request - * @return Response - */ - public function show(Request $request): Response - { - try { - // 从请求路径中解析 user_id - $path = $request->path(); - if (preg_match('#/api/users/([^/]+)#', $path, $matches)) { - $userId = $matches[1]; - } else { - $userId = $request->get('user_id'); - if (!$userId) { - throw new \InvalidArgumentException('缺少 user_id 参数'); - } - } - - LoggerHelper::logRequest('GET', $path, ['user_id' => $userId]); - - // 检查是否需要解密身份证（需要权限控制，这里简单用参数控制） - $decryptIdCard = (bool)$request->get('decrypt_id_card', false); - - $userService = new UserService(new UserProfileRepository()); - $user = $userService->getUserById($userId, $decryptIdCard); - - if (!$user) { - return ApiResponseHelper::error('用户不存在', 404, 404); - } - - // 如果不需要解密身份证，对敏感字段进行脱敏 - if (!$decryptIdCard) { - $user = DataMaskingHelper::maskArray($user, ['phone', 'email']); - } - - LoggerHelper::logBusiness('get_user_info', [ - 'user_id' => $userId, - 'decrypt_id_card' => $decryptIdCard, - ]); - - return ApiResponseHelper::success($user); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 更新用户信息 - * - * PUT /api/users/{user_id} - * - * 请求体示例： - * { - * "name": "张三", - * "phone": "13800138000", - * "email": "zhangsan@example.com", - * "gender": 1, - * "birthday": "1990-01-01", - * "address": "北京市朝阳区", - * "status": 0 - * } - */ - public function update(Request $request): Response - { - try { - // 从请求路径中解析 user_id - $path = $request->path(); - if (preg_match('#/api/users/([^/]+)#', $path, $matches)) { - $userId = $matches[1]; - } else { - $userId = $request->get('user_id'); - if (!$userId) { - throw new \InvalidArgumentException('缺少 user_id 参数'); - } - } - - LoggerHelper::logRequest('PUT', $path, ['user_id' => $userId]); - - $rawBody = $request->rawBody(); - - // 调试：记录原始请求体 - if (empty($rawBody)) { - return ApiResponseHelper::error('请求体为空，请确保 Content-Type 为 application/json 并发送有效的 JSON 数据', 400); - } - - $body = json_decode($rawBody, true); - if (json_last_error() !== JSON_ERROR_NONE) { - $errorMsg = '请求体必须是有效的 JSON 格式'; - $jsonError = json_last_error_msg(); - if ($jsonError) { - $errorMsg .= ': ' . $jsonError; - } - // 开发环境输出更多调试信息 - if (getenv('APP_DEBUG') === 'true') { - $errorMsg .= ' (原始请求体: ' . substr($rawBody, 0, 200) . ')'; - } - return ApiResponseHelper::error($errorMsg, 400); - } - - if (empty($body)) { - return ApiResponseHelper::error('请求体不能为空', 400); - } - - $userService = new UserService(new UserProfileRepository()); - $result = $userService->updateUser($userId, $body); - - // 脱敏处理 - $result = DataMaskingHelper::maskArray($result, ['phone', 'email']); - - return ApiResponseHelper::success($result, '用户更新成功'); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 解密身份证号 - * - * GET /api/users/{user_id}/decrypt-id-card - */ - public function decryptIdCard(Request $request): Response - { - try { - // 从请求路径中解析 user_id - $path = $request->path(); - if (preg_match('#/api/users/([^/]+)/decrypt-id-card#', $path, $matches)) { - $userId = $matches[1]; - } else { - $userId = $request->get('user_id'); - if (!$userId) { - throw new \InvalidArgumentException('缺少 user_id 参数'); - } - } - - LoggerHelper::logRequest('GET', $path, ['user_id' => $userId]); - - $userService = new UserService(new UserProfileRepository()); - $user = $userService->getUserById($userId, true); // 强制解密 - - if (!$user) { - return ApiResponseHelper::error('用户不存在', 404, 404); - } - - LoggerHelper::logBusiness('decrypt_id_card', [ - 'user_id' => $userId, - ]); - - return ApiResponseHelper::success([ - 'user_id' => $user['user_id'], - 'id_card' => $user['id_card'] ?? '' - ]); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 删除用户（软删除） - * - * DELETE /api/users/{user_id} - */ - public function destroy(Request $request): Response - { - try { - // 从请求路径中解析 user_id - $path = $request->path(); - if (preg_match('#/api/users/([^/]+)#', $path, $matches)) { - $userId = $matches[1]; - } else { - $userId = $request->get('user_id'); - if (!$userId) { - throw new \InvalidArgumentException('缺少 user_id 参数'); - } - } - - LoggerHelper::logRequest('DELETE', $path, ['user_id' => $userId]); - - $userService = new UserService(new UserProfileRepository()); - $userService->deleteUser($userId); - - return ApiResponseHelper::success(null, '用户删除成功'); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } - - /** - * 搜索用户（支持多种搜索条件组合） - * - * POST /api/users/search - * - * 支持以下搜索方式： - * 1. 基础字段搜索：姓名、手机号、邮箱、身份证号等 - * 2. 标签筛选：根据用户标签筛选 - * 3. 组合搜索：基础字段 + 标签筛选 - * - * 请求体示例1（姓名模糊搜索）： - * { - * "name": "张三", - * "page": 1, - * "page_size": 20 - * } - * - * 请求体示例2（组合搜索：姓名 + 手机号）： - * { - * "name": "张", - * "phone": "138", - * "page": 1, - * "page_size": 20 - * } - * - * 请求体示例3（根据标签筛选）： - * { - * "tag_conditions": [ - * { - * "tag_code": "high_consumer", - * "operator": "=", - * "value": "high" - * } - * ], - * "logic": "AND", - * "page": 1, - * "page_size": 20 - * } - * - * 请求体示例4（组合搜索：基础字段 + 标签）： - * { - * "name": "张", - * "min_total_amount": 1000, - * "tag_conditions": [ - * { - * "tag_code": "active_user", - * "operator": "=", - * "value": "active" - * } - * ], - * "page": 1, - * "page_size": 20 - * } - */ - public function search(Request $request): Response - { - try { - LoggerHelper::logRequest('POST', '/api/users/search'); - - $rawBody = $request->rawBody(); - - // 调试：记录原始请求体 - if (empty($rawBody)) { - return ApiResponseHelper::error('请求体为空，请确保 Content-Type 为 application/json 并发送有效的 JSON 数据', 400); - } - - $body = json_decode($rawBody, true); - if (json_last_error() !== JSON_ERROR_NONE) { - $errorMsg = '请求体必须是有效的 JSON 格式'; - $jsonError = json_last_error_msg(); - if ($jsonError) { - $errorMsg .= ': ' . $jsonError; - } - // 开发环境输出更多调试信息 - if (getenv('APP_DEBUG') === 'true') { - $errorMsg .= ' (原始请求体: ' . substr($rawBody, 0, 200) . ')'; - } - return ApiResponseHelper::error($errorMsg, 400); - } - - $page = (int)($body['page'] ?? 1); - $pageSize = (int)($body['page_size'] ?? 20); - - if ($page < 1) { - $page = 1; - } - if ($pageSize < 1 || $pageSize > 100) { - $pageSize = 20; - } - - $userService = new UserService(new UserProfileRepository()); - - // 情况1：仅根据身份证号查找（返回单个用户，不分页） - if (!empty($body['id_card']) && empty($body['tag_conditions']) && empty($body['name']) && empty($body['phone']) && empty($body['email'])) { - $user = $userService->findUserByIdCard($body['id_card']); - - if (!$user) { - return ApiResponseHelper::error('未找到该身份证号对应的用户', 404, 404); - } - - // 脱敏处理 - $user = DataMaskingHelper::maskArray($user, ['phone', 'email']); - - LoggerHelper::logBusiness('search_user_by_id_card', [ - 'found' => true, - ]); - - return ApiResponseHelper::success($user); - } - - // 情况2：根据标签筛选用户（可能结合基础字段搜索） - if (!empty($body['tag_conditions'])) { - $tagService = new \app\service\TagService( - new \app\repository\TagDefinitionRepository(), - new UserProfileRepository(), - new \app\repository\UserTagRepository(), - new \app\repository\TagHistoryRepository(), - new \app\service\TagRuleEngine\SimpleRuleEngine() - ); - - $conditions = $body['tag_conditions']; - $logic = $body['logic'] ?? 'AND'; - $includeUserInfo = true; // 标签筛选需要用户信息 - - // 验证条件格式 - foreach ($conditions as $condition) { - if (!isset($condition['tag_code']) || !isset($condition['operator']) || !isset($condition['value'])) { - throw new \InvalidArgumentException('每个条件必须包含 tag_code、operator 和 value 字段'); - } - } - - // 先根据标签筛选用户 - $tagResult = $tagService->filterUsersByTags( - $conditions, - $logic, - 1, // 先获取所有符合条件的用户ID - 10000, // 临时设置大值，获取所有用户ID - true - ); - - $userIds = array_column($tagResult['users'], 'user_id'); - - if (empty($userIds)) { - return ApiResponseHelper::success([ - 'users' => [], - 'total' => 0, - 'page' => $page, - 'page_size' => $pageSize, - 'total_pages' => 0, - ]); - } - - // 如果有基础字段搜索条件，进一步筛选 - $baseConditions = []; - if (!empty($body['name'])) { - $baseConditions['name'] = $body['name']; - } - if (!empty($body['phone'])) { - $baseConditions['phone'] = $body['phone']; - $baseConditions['phone_exact'] = $body['phone_exact'] ?? false; - } - if (!empty($body['email'])) { - $baseConditions['email'] = $body['email']; - $baseConditions['email_exact'] = $body['email_exact'] ?? false; - } - if (isset($body['gender']) && $body['gender'] !== '') { - $baseConditions['gender'] = $body['gender']; - } - if (isset($body['status']) && $body['status'] !== '') { - $baseConditions['status'] = $body['status']; - } - if (isset($body['min_total_amount'])) { - $baseConditions['min_total_amount'] = $body['min_total_amount']; - } - if (isset($body['max_total_amount'])) { - $baseConditions['max_total_amount'] = $body['max_total_amount']; - } - if (isset($body['min_total_count'])) { - $baseConditions['min_total_count'] = $body['min_total_count']; - } - if (isset($body['max_total_count'])) { - $baseConditions['max_total_count'] = $body['max_total_count']; - } - - // 如果有基础字段条件，需要进一步筛选 - if (!empty($baseConditions)) { - $baseConditions['user_ids'] = $userIds; // 限制在标签筛选的用户范围内 - $result = $userService->searchUsers($baseConditions, $page, $pageSize); - } else { - // 没有基础字段条件，直接使用标签筛选结果并分页 - $total = count($userIds); - $offset = ($page - 1) * $pageSize; - $pagedUserIds = array_slice($userIds, $offset, $pageSize); - - // 获取用户详细信息 - $users = []; - foreach ($pagedUserIds as $userId) { - $user = $userService->getUserById($userId, false); - if ($user) { - $users[] = $user; - } - } - - $result = [ - 'users' => $users, - 'total' => $total, - 'page' => $page, - 'page_size' => $pageSize, - 'total_pages' => (int)ceil($total / $pageSize), - ]; - } - - // 对返回的用户信息进行脱敏处理 - if (isset($result['users']) && is_array($result['users'])) { - foreach ($result['users'] as &$user) { - $user = DataMaskingHelper::maskArray($user, ['phone', 'email']); - } - unset($user); - } - - LoggerHelper::logBusiness('search_users_by_tags', [ - 'conditions_count' => count($conditions), - 'base_conditions' => !empty($baseConditions), - 'result_count' => $result['total'] ?? 0, - ]); - - return ApiResponseHelper::success($result); - } - - // 情况3：仅基础字段搜索（无标签条件） - $baseConditions = []; - if (!empty($body['name'])) { - $baseConditions['name'] = $body['name']; - } - if (!empty($body['phone'])) { - $baseConditions['phone'] = $body['phone']; - $baseConditions['phone_exact'] = $body['phone_exact'] ?? false; - } - if (!empty($body['email'])) { - $baseConditions['email'] = $body['email']; - $baseConditions['email_exact'] = $body['email_exact'] ?? false; - } - if (!empty($body['id_card'])) { - $baseConditions['id_card'] = $body['id_card']; - } - if (isset($body['gender']) && $body['gender'] !== '') { - $baseConditions['gender'] = $body['gender']; - } - if (isset($body['status']) && $body['status'] !== '') { - $baseConditions['status'] = $body['status']; - } - if (isset($body['min_total_amount'])) { - $baseConditions['min_total_amount'] = $body['min_total_amount']; - } - if (isset($body['max_total_amount'])) { - $baseConditions['max_total_amount'] = $body['max_total_amount']; - } - if (isset($body['min_total_count'])) { - $baseConditions['min_total_count'] = $body['min_total_count']; - } - if (isset($body['max_total_count'])) { - $baseConditions['max_total_count'] = $body['max_total_count']; - } - - if (empty($baseConditions)) { - return ApiResponseHelper::error('请提供至少一个搜索条件', 400); - } - - $result = $userService->searchUsers($baseConditions, $page, $pageSize); - - // 对返回的用户信息进行脱敏处理 - if (isset($result['users']) && is_array($result['users'])) { - foreach ($result['users'] as &$user) { - $user = DataMaskingHelper::maskArray($user, ['phone', 'email']); - } - unset($user); - } - - LoggerHelper::logBusiness('search_users_by_base_fields', [ - 'conditions' => array_keys($baseConditions), - 'result_count' => $result['total'] ?? 0, - ]); - - return ApiResponseHelper::success($result); - } catch (\InvalidArgumentException $e) { - return ApiResponseHelper::error($e->getMessage(), 400); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } - } -} diff --git a/Moncter/Moncter/app/middleware/StaticFile.php b/Moncter/Moncter/app/middleware/StaticFile.php deleted file mode 100644 index fa8dbf71..00000000 --- a/Moncter/Moncter/app/middleware/StaticFile.php +++ /dev/null @@ -1,42 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -namespace app\middleware; - -use Webman\MiddlewareInterface; -use Webman\Http\Response; -use Webman\Http\Request; - -/** - * Class StaticFile - * @package app\middleware - */ -class StaticFile implements MiddlewareInterface -{ - public function process(Request $request, callable $handler): Response - { - // Access to files beginning with. Is prohibited - if (strpos($request->path(), '/.') !== false) { - return response('

403 forbidden

', 403); - } - /** @var Response $response */ - $response = $handler($request); - // Add cross domain HTTP header - /*$response->withHeaders([ - 'Access-Control-Allow-Origin' => '*', - 'Access-Control-Allow-Credentials' => 'true', - ]);*/ - return $response; - } -} diff --git a/Moncter/Moncter/app/model/Test.php b/Moncter/Moncter/app/model/Test.php deleted file mode 100644 index 92d70e38..00000000 --- a/Moncter/Moncter/app/model/Test.php +++ /dev/null @@ -1,29 +0,0 @@ - 'string', - 'age' => 'integer', - 'created_at' => 'datetime', - 'updated_at' => 'datetime', - 'tags' => 'array', // 支持数组类型（MongoDB 原生支持数组） - ]; - - // 自动维护时间戳（created_at/updated_at，默认启用） - // 若不需要可关闭：public $timestamps = false; - - // 自定义时间戳字段名（可选） - // const CREATED_AT = 'create_time'; - // const UPDATED_AT = 'update_time'; -} \ No newline at end of file diff --git a/Moncter/Moncter/app/process/DataSyncScheduler.php b/Moncter/Moncter/app/process/DataSyncScheduler.php deleted file mode 100644 index 5fe716e6..00000000 --- a/Moncter/Moncter/app/process/DataSyncScheduler.php +++ /dev/null @@ -1,695 +0,0 @@ -taskService = new DataCollectionTaskService( - new DataCollectionTaskRepository() - ); - - // 初始化数据源服务 - $this->dataSourceService = new DataSourceService( - new DataSourceRepository() - ); - - // 初始化标签任务服务（避免在多个方法中重复实例化） - $this->tagTaskService = new \app\service\TagTaskService( - new \app\repository\TagTaskRepository(), - new \app\repository\TagTaskExecutionRepository(), - new \app\repository\UserProfileRepository(), - new \app\service\TagService( - new \app\repository\TagDefinitionRepository(), - new \app\repository\UserProfileRepository(), - new \app\repository\UserTagRepository(), - new \app\repository\TagHistoryRepository(), - new \app\service\TagRuleEngine\SimpleRuleEngine() - ) - ); - - // 加载任务采集配置（配置文件中的任务） - $taskConfig = config('data_collection_tasks', []); - $this->tasks = $taskConfig['tasks'] ?? []; - $this->globalConfig = $taskConfig['global'] ?? []; - - // 从数据库加载数据源配置（替代config('data_sources')） - $this->loadDataSourcesConfig(); - - LoggerHelper::logBusiness('data_collection_scheduler_started', [ - 'worker_id' => $worker->id, - 'total_workers' => $worker->count, - 'config_task_count' => count($this->tasks), - 'data_source_count' => count($this->dataSourcesConfig), - ]); - - // 加载配置文件中的任务 - $this->loadConfigTasks($worker); - - // 加载数据库中的动态任务 - $this->loadDatabaseTasks($worker); - - // 每30秒刷新一次数据库任务列表（检查新任务、状态变更等） - Timer::add(30, function () use ($worker) { - $this->refreshDatabaseTasks($worker); - }); - } - - /** - * 从数据库加载数据源配置 - */ - private function loadDataSourcesConfig(): void - { - try { - $this->dataSourcesConfig = $this->dataSourceService->getAllEnabledDataSources(); - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DataSyncScheduler', - 'action' => 'loadDataSourcesConfig', - ]); - // 如果加载失败，使用空数组，避免后续错误 - $this->dataSourcesConfig = []; - } - } - - /** - * 加载配置文件中的任务 - */ - private function loadConfigTasks(Worker $worker): void - { - // 为每个启用的任务设置定时任务 - foreach ($this->tasks as $taskId => $taskConfig) { - if (!($taskConfig['enabled'] ?? true)) { - continue; - } - - // 检查是否应该由当前 Worker 处理（分片分配） - if (!$this->shouldHandleTask($taskId, $taskConfig, $worker)) { - continue; - } - - // 获取调度配置 - $schedule = $taskConfig['schedule'] ?? []; - - // 如果调度被禁用，直接启动任务（持续运行） - if (!($schedule['enabled'] ?? true)) { - // 使用 Timer 延迟启动，避免阻塞 Worker 启动 - Timer::add(0, function () use ($taskId, $taskConfig, $worker) { - $this->executeTask($taskId, $taskConfig, $worker); - }, [], false); - continue; - } - - $cronExpression = $schedule['cron'] ?? '*/5 * * * *'; // 默认每5分钟 - - // 创建定时任务 - $this->scheduleTask($taskId, $taskConfig, $cronExpression, $worker); - } - } - - /** - * 加载数据库中的动态任务 - */ - private function loadDatabaseTasks(Worker $worker): void - { - try { - // 加载数据采集任务 - $dataCollectionTaskService = new \app\service\DataCollectionTaskService( - new \app\repository\DataCollectionTaskRepository() - ); - $runningCollectionTasks = $dataCollectionTaskService->getRunningTasks(); - - \Workerman\Worker::safeEcho("[DataSyncScheduler] 从数据库加载到 " . count($runningCollectionTasks) . " 个运行中的任务 (worker_id={$worker->id})\n"); - - foreach ($runningCollectionTasks as $task) { - $taskId = $task['task_id']; - $taskName = $task['name'] ?? $taskId; - - if (!$this->shouldHandleDatabaseTask($taskId, $worker)) { - \Workerman\Worker::safeEcho("[DataSyncScheduler] 跳过任务 [{$taskName}]，由其他 Worker 处理 (worker_id={$worker->id})\n"); - continue; - } - - \Workerman\Worker::safeEcho("[DataSyncScheduler] ✓ 准备启动任务: [{$taskName}] task_id={$taskId} (worker_id={$worker->id})\n"); - - $taskConfig = $this->convertDatabaseTaskToConfig($task); - - // 使用统一的任务启动方法 - $this->startTask($taskId, $taskConfig, $worker); - } - - // 加载标签任务（使用已初始化的服务） - $runningTagTasks = $this->tagTaskService->getTaskList(['status' => 'running'], 1, 1000); - - foreach ($runningTagTasks['tasks'] as $task) { - $taskId = $task['task_id']; - - if (!$this->shouldHandleDatabaseTask($taskId, $worker)) { - continue; - } - - \Workerman\Worker::safeEcho("[DataSyncScheduler] ✓ 准备启动标签任务: task_id={$taskId} (worker_id={$worker->id})\n"); - - $taskConfig = $this->convertDatabaseTaskToConfig($task); - - // 标签任务通常是批量执行，根据调度配置执行 - $schedule = $taskConfig['schedule'] ?? []; - if (!($schedule['enabled'] ?? true)) { - // 立即执行一次 - \Workerman\Worker::safeEcho("[DataSyncScheduler] → 立即执行标签任务\n"); - Timer::add(0, function () use ($taskId, $taskConfig, $worker) { - $this->executeTask($taskId, $taskConfig, $worker); - }, [], false); - } else { - $cronExpression = $schedule['cron'] ?? '0 2 * * *'; // 默认每天凌晨2点 - \Workerman\Worker::safeEcho("[DataSyncScheduler] → 定时执行标签任务，Cron: {$cronExpression}\n"); - $this->scheduleTask($taskId, $taskConfig, $cronExpression, $worker); - } - } - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DataSyncScheduler', - 'action' => 'loadDatabaseTasks', - ]); - } - } - - /** - * 刷新数据库任务列表 - */ - private function refreshDatabaseTasks(Worker $worker): void - { - try { - // 刷新数据采集任务 - $dataCollectionTaskService = new \app\service\DataCollectionTaskService( - new \app\repository\DataCollectionTaskRepository() - ); - $runningCollectionTasks = $dataCollectionTaskService->getRunningTasks(); - $collectionTaskIds = array_column($runningCollectionTasks, 'task_id'); - - foreach ($runningCollectionTasks as $task) { - $taskId = $task['task_id']; - - if (isset($this->runningTasks[$taskId])) { - \Workerman\Worker::safeEcho("[DataSyncScheduler] 任务 {$taskId} 已在运行中，跳过\n"); - continue; - } - - \Workerman\Worker::safeEcho("[DataSyncScheduler] 检测到新的运行中任务: {$taskId}\n"); - - $taskConfig = $this->convertDatabaseTaskToConfig($task); - - // 使用统一的任务启动方法 - $this->startTask($taskId, $taskConfig, $worker); - } - - // 刷新标签任务（使用已初始化的服务） - $runningTagTasks = $this->tagTaskService->getTaskList(['status' => 'running'], 1, 1000); - $tagTaskIds = array_column($runningTagTasks['tasks'], 'task_id'); - - foreach ($runningTagTasks['tasks'] as $task) { - $taskId = $task['task_id']; - - if (isset($this->runningTasks[$taskId])) { - continue; - } - - if (RedisHelper::exists("tag_task:{$taskId}:start")) { - RedisHelper::del("tag_task:{$taskId}:start"); - - $taskConfig = $this->convertDatabaseTaskToConfig($task); - - Timer::add(0, function () use ($taskId, $taskConfig, $worker) { - $this->executeTask($taskId, $taskConfig, $worker); - }, [], false); - $this->runningTasks[$taskId] = true; - } - } - - // 检查是否有任务需要停止或暂停 - $allTaskIds = array_merge($collectionTaskIds, $tagTaskIds); - foreach (array_keys($this->runningTasks) as $taskId) { - if (!in_array($taskId, $allTaskIds)) { - // 任务不在运行列表中了，移除 - unset($this->runningTasks[$taskId]); - } elseif (RedisHelper::exists("data_collection_task:{$taskId}:stop") || - RedisHelper::exists("tag_task:{$taskId}:stop")) { - // 检查停止标志 - RedisHelper::del("data_collection_task:{$taskId}:stop"); - RedisHelper::del("tag_task:{$taskId}:stop"); - unset($this->runningTasks[$taskId]); - \Workerman\Worker::safeEcho("[DataSyncScheduler] 检测到停止信号，任务 {$taskId} 已停止\n"); - } elseif (RedisHelper::exists("data_collection_task:{$taskId}:pause")) { - // 检查暂停标志 - RedisHelper::del("data_collection_task:{$taskId}:pause"); - unset($this->runningTasks[$taskId]); - \Workerman\Worker::safeEcho("[DataSyncScheduler] 检测到暂停信号，任务 {$taskId} 已暂停（正在执行的任务会在下次检查时停止）\n"); - } - } - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DataSyncScheduler', - 'action' => 'refreshDatabaseTasks', - ]); - } - } - - /** - * 判断是否应该处理数据库任务 - */ - private function shouldHandleDatabaseTask(string $taskId, Worker $worker): bool - { - // 简单策略：按 Worker ID 取模分配 - // 可以根据需要调整分配策略 - return ($worker->id % $worker->count) === (hexdec(substr($taskId, 0, 8)) % $worker->count); - } - - /** - * 将数据库任务转换为配置格式 - */ - private function convertDatabaseTaskToConfig(array $task): array - { - // 判断任务类型：数据采集任务还是标签任务 - // 如果任务有 data_source_id，说明是数据采集任务 - // 如果任务有 target_tag_ids，说明是标签任务 - if (isset($task['data_source_id']) && !empty($task['data_source_id'])) { - // 数据采集任务 - // 根据 target_type 选择不同的 Handler - $targetType = $task['target_type'] ?? 'generic'; - $handlerClass = \app\service\DataCollection\Handler\GenericCollectionHandler::class; - - if ($targetType === 'consumption_record') { - $handlerClass = \app\service\DataCollection\Handler\ConsumptionCollectionHandler::class; - } - - $config = [ - 'task_id' => $task['task_id'], - 'name' => $task['name'] ?? '', - 'data_source_id' => $task['data_source_id'], - 'data_source' => $task['data_source_id'], - 'database' => $task['database'] ?? '', - 'collection' => $task['collection'] ?? null, - 'collections' => $task['collections'] ?? null, - 'target_type' => $targetType, - 'target_data_source_id' => $task['target_data_source_id'] ?? null, - 'target_database' => $task['target_database'] ?? null, - 'target_collection' => $task['target_collection'] ?? null, - 'mode' => $task['mode'] ?? 'batch', - 'field_mappings' => $task['field_mappings'] ?? [], - 'collection_field_mappings' => $task['collection_field_mappings'] ?? [], - 'lookups' => $task['lookups'] ?? [], - 'collection_lookups' => $task['collection_lookups'] ?? [], - 'filter_conditions' => $task['filter_conditions'] ?? [], - 'schedule' => $task['schedule'] ?? [ - 'enabled' => false, - 'cron' => null, - ], - 'handler_class' => $handlerClass, - 'batch_size' => $task['batch_size'] ?? 1000, - ]; - - // 对于consumption_record类型的任务，添加source_type字段（如果存在） - if ($targetType === 'consumption_record' && isset($task['source_type'])) { - $config['source_type'] = $task['source_type']; - } - - return $config; - } elseif (isset($task['target_tag_ids']) || isset($task['task_type'])) { - // 标签任务 - return [ - 'task_id' => $task['task_id'], - 'name' => $task['name'] ?? '', - 'task_type' => $task['task_type'] ?? 'full', - 'target_tag_ids' => $task['target_tag_ids'] ?? [], - 'user_scope' => $task['user_scope'] ?? ['type' => 'all'], - 'schedule' => $task['schedule'] ?? [ - 'enabled' => false, - 'cron' => null, - ], - 'config' => $task['config'] ?? [ - 'concurrency' => 10, - 'batch_size' => 100, - 'error_handling' => 'skip', - ], - 'handler_class' => \app\service\DataCollection\Handler\TagTaskHandler::class, - ]; - } else { - throw new \InvalidArgumentException("无法识别任务类型: {$task['task_id']}"); - } - } - - /** - * 判断当前 Worker 是否应该处理该任务（分片分配） - * - * @param string $taskId 任务ID - * @param array $taskConfig 任务配置 - * @param Worker $worker Worker 实例 - * @return bool 是否应该处理 - */ - private function shouldHandleTask(string $taskId, array $taskConfig, Worker $worker): bool - { - $sharding = $taskConfig['sharding'] ?? []; - $strategy = $sharding['strategy'] ?? 'none'; - - // 如果不需要分片，所有 Worker 都处理（但通过分布式锁保证只有一个执行） - if ($strategy === 'none') { - return true; - } - - // by_database 策略：所有 Worker 都处理，但每个 Worker 处理不同的数据库（在 Handler 中分配） - if ($strategy === 'by_database') { - return true; // 所有 Worker 都处理，数据库分配在 Handler 中进行 - } - - // 其他分片策略：按 Worker ID 取模分配 - $shardCount = $sharding['shard_count'] ?? 1; - if ($shardCount <= 1) { - // 如果 shard_count <= 1，所有 Worker 都处理 - return true; - } - - $assignedShardId = $worker->id % $shardCount; - // 对于分片策略，只处理分配给当前 Worker 的分片 - return $assignedShardId === ($worker->id % $shardCount); - } - - /** - * 为任务设置定时任务 - * - * @param string $taskId 任务ID - * @param array $taskConfig 任务配置 - * @param string $cronExpression Cron 表达式 - * @param Worker $worker Worker 实例 - * @return void - */ - private function scheduleTask(string $taskId, array $taskConfig, string $cronExpression, Worker $worker): void - { - try { - $cron = CronExpression::factory($cronExpression); - - // 计算下次执行时间 - $nextRunTime = $cron->getNextRunDate()->getTimestamp(); - $now = time(); - $delay = max(0, $nextRunTime - $now); - - LoggerHelper::logBusiness('data_collection_task_scheduled', [ - 'task_id' => $taskId, - 'task_name' => $taskConfig['name'] ?? $taskId, - 'cron' => $cronExpression, - 'next_run_time' => date('Y-m-d H:i:s', $nextRunTime), - 'delay' => $delay, - ]); - - // 设置定时器（延迟执行第一次，然后每60秒检查一次） - Timer::add(60, function () use ($taskId, $taskConfig, $cron, $worker) { - $now = time(); - $nextRunTime = $cron->getNextRunDate()->getTimestamp(); - - // 如果到了执行时间（允许1秒误差） - if ($nextRunTime <= $now + 1) { - $this->executeTask($taskId, $taskConfig, $worker); - } - }, [], false, $delay); - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DataSyncScheduler', - 'action' => 'scheduleTask', - 'task_id' => $taskId, - 'cron' => $cronExpression, - ]); - } - } - - /** - * 执行采集任务 - * - * @param string $taskId 任务ID - * @param array $taskConfig 任务配置 - * @param Worker $worker Worker 实例 - * @return void - */ - private function executeTask(string $taskId, array $taskConfig, Worker $worker): void - { - // 在执行前，检查任务状态，如果已经是 completed，就不应该再执行 - try { - $task = $this->taskService->getTask($taskId); - if ($task && isset($task['status'])) { - if ($task['status'] === 'completed') { - \Workerman\Worker::safeEcho("[DataSyncScheduler] ⚠️ 任务已完成，跳过执行: task_id={$taskId}\n"); - LoggerHelper::logBusiness('data_collection_skipped_completed', [ - 'task_id' => $taskId, - 'worker_id' => $worker->id, - ]); - return; - } - if (in_array($task['status'], ['stopped', 'paused', 'error'])) { - \Workerman\Worker::safeEcho("[DataSyncScheduler] ⚠️ 任务状态为 {$task['status']}，跳过执行: task_id={$taskId}\n"); - LoggerHelper::logBusiness('data_collection_skipped_status', [ - 'task_id' => $taskId, - 'status' => $task['status'], - 'worker_id' => $worker->id, - ]); - return; - } - } - } catch (\Throwable $e) { - // 如果查询任务状态失败，记录日志但继续执行（避免因为查询失败而阻塞任务） - LoggerHelper::logError($e, [ - 'component' => 'DataSyncScheduler', - 'action' => 'executeTask', - 'task_id' => $taskId, - 'message' => '检查任务状态失败，继续执行任务', - ]); - } - - $sharding = $taskConfig['sharding'] ?? []; - $strategy = $sharding['strategy'] ?? 'none'; - - // 对于 by_database 策略，不使用全局锁，让所有 Worker 并行执行 - // 每个 Worker 会在 Handler 中分配不同的数据库 - $useLock = ($strategy !== 'by_database'); - - if ($useLock) { - $lockKey = "data_collection:{$taskId}"; - $lockConfig = $this->globalConfig['distributed_lock'] ?? []; - $ttl = $lockConfig['ttl'] ?? 300; - $retryTimes = $lockConfig['retry_times'] ?? 3; - $retryDelay = $lockConfig['retry_delay'] ?? 1000; - - // 尝试获取分布式锁 - if (!RedisHelper::acquireLock($lockKey, $ttl, $retryTimes, $retryDelay)) { - LoggerHelper::logBusiness('data_collection_skipped_locked', [ - 'task_id' => $taskId, - 'worker_id' => $worker->id, - ]); - return; - } - } - - try { - LoggerHelper::logBusiness('data_collection_started', [ - 'task_id' => $taskId, - 'task_name' => $taskConfig['name'] ?? $taskId, - 'worker_id' => $worker->id, - ]); - // 控制台提示，标记具体哪个采集/同步任务被当前 Worker 拉起，方便排查任务是否真正执行 - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤1-任务启动】执行采集任务：task_id={$taskId}, task_name=" . ($taskConfig['name'] ?? $taskId) . ", worker_id={$worker->id}\n"); - - // 判断任务类型：数据采集任务需要数据源适配器，标签任务不需要 - $handlerClass = $taskConfig['handler_class'] ?? ''; - $isTagTask = strpos($handlerClass, 'TagTaskHandler') !== false; - - $adapter = null; - if (!$isTagTask) { - // 数据采集任务：需要数据源适配器 - // 支持两种配置方式： - // 1. 单数据源：data_source（用于普通采集任务） - // 2. 多数据源：source_data_source 和 target_data_source（用于数据库同步任务） - // 3. 动态任务：data_source_id（从数据库读取的任务） - $dataSourceId = $taskConfig['data_source'] ?? $taskConfig['data_source_id'] ?? ''; - $sourceDataSourceId = $taskConfig['source_data_source'] ?? ''; - - // 如果配置了 source_data_source，说明是多数据源任务（如数据库同步） - // 这种情况下，只需要创建源数据源适配器，目标数据源由 Handler 内部处理 - if (!empty($sourceDataSourceId)) { - $dataSourceId = $sourceDataSourceId; - } - - // 从缓存或数据库获取数据源配置 - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤2-查询数据源配置】开始查询数据源配置: data_source_id={$dataSourceId}\n"); - $dataSourceConfig = $this->dataSourcesConfig[$dataSourceId] ?? null; - - // 如果缓存中没有，尝试从数据库加载 - if (empty($dataSourceConfig)) { - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤2-查询数据源配置】缓存中没有，从数据库加载: data_source_id={$dataSourceId}\n"); - $dataSourceConfig = $this->dataSourceService->getDataSourceConfigById($dataSourceId); - if ($dataSourceConfig) { - // 更新缓存（使用原始 dataSourceId 作为 key） - $this->dataSourcesConfig[$dataSourceId] = $dataSourceConfig; - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤2-查询数据源配置】✓ 数据源配置加载成功: host={$dataSourceConfig['host']}, port={$dataSourceConfig['port']}, database={$dataSourceConfig['database']}\n"); - } else { - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤2-查询数据源配置】✗ 数据源配置不存在: data_source_id={$dataSourceId}\n"); - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤2-查询数据源配置】提示：请检查 data_sources 表中是否存在该数据源，或检查 name 字段是否匹配\n"); - } - } else { - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤2-查询数据源配置】✓ 从缓存获取数据源配置: host={$dataSourceConfig['host']}, port={$dataSourceConfig['port']}\n"); - } - - if (empty($dataSourceConfig)) { - throw new \InvalidArgumentException("数据源配置不存在: {$dataSourceId}"); - } - - // 创建数据源适配器 - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤3-创建数据源适配器】开始创建适配器: type={$dataSourceConfig['type']}\n"); - $adapter = DataSourceAdapterFactory::create( - $dataSourceConfig['type'], - $dataSourceConfig - ); - - // 确保适配器已连接 - if (!$adapter->isConnected()) { - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤3-创建数据源适配器】适配器未连接，尝试连接...\n"); - if (!$adapter->connect($dataSourceConfig)) { - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤3-创建数据源适配器】✗ 连接失败: data_source_id={$dataSourceId}\n"); - throw new \RuntimeException("无法连接到数据源: {$dataSourceId}"); - } - } - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤3-创建数据源适配器】✓ 数据源适配器创建并连接成功\n"); - } - - // 创建业务处理类（处理逻辑在业务代码中） - $handlerClass = $taskConfig['handler_class'] ?? ''; - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤4-创建Handler】开始创建Handler: handler_class={$handlerClass}\n"); - if (empty($handlerClass) || !class_exists($handlerClass)) { - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤4-创建Handler】✗ Handler类不存在: {$handlerClass}\n"); - throw new \InvalidArgumentException("处理类不存在或未配置: {$handlerClass}"); - } - - // 实例化处理类 - $handler = new $handlerClass(); - - // 检查处理类是否实现了采集接口 - if (!method_exists($handler, 'collect')) { - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤4-创建Handler】✗ Handler未实现collect方法: {$handlerClass}\n"); - throw new \InvalidArgumentException("处理类必须实现 collect 方法: {$handlerClass}"); - } - - // 添加任务ID到配置中 - $taskConfig['task_id'] = $taskId; - - // 添加 Worker 信息到配置中（用于多进程数据库分配） - $taskConfig['worker_id'] = $worker->id; - $taskConfig['worker_count'] = $worker->count; - - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤4-创建Handler】✓ Handler创建成功，开始调用collect方法\n"); - // 调用业务处理类的采集方法 - try { - $handler->collect($adapter, $taskConfig); - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤4-创建Handler】✓ Handler.collect()方法执行完成\n"); - } catch (\Throwable $collectException) { - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤4-创建Handler】✗ Handler.collect()方法执行失败: " . $collectException->getMessage() . "\n"); - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【步骤4-创建Handler】异常堆栈: " . $collectException->getTraceAsString() . "\n"); - throw $collectException; // 重新抛出异常，让外层catch处理 - } - - LoggerHelper::logBusiness('data_collection_completed', [ - 'task_id' => $taskId, - 'task_name' => $taskConfig['name'] ?? $taskId, - ]); - } catch (\Throwable $e) { - // 在控制台输出异常信息，方便排查 - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【异常捕获】任务执行失败: task_id={$taskId}, error=" . $e->getMessage() . "\n"); - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【异常捕获】异常文件: " . $e->getFile() . ":" . $e->getLine() . "\n"); - \Workerman\Worker::safeEcho("[DataSyncScheduler] 【异常捕获】异常堆栈: " . $e->getTraceAsString() . "\n"); - - LoggerHelper::logError($e, [ - 'component' => 'DataSyncScheduler', - 'action' => 'executeTask', - 'task_id' => $taskId, - 'worker_id' => $worker->id, - ]); - } finally { - // 释放锁（仅在使用锁的情况下） - if (isset($useLock) && $useLock && isset($lockKey)) { - RedisHelper::releaseLock($lockKey); - } - } - } - - /** - * 统一的任务启动方法 - * - * @param string $taskId 任务ID - * @param array $taskConfig 任务配置 - * @param Worker $worker Worker 实例 - * @return void - */ - private function startTask(string $taskId, array $taskConfig, Worker $worker): void - { - $taskName = $taskConfig['name'] ?? $taskId; - - if ($taskConfig['mode'] === 'realtime') { - // 实时模式：立即启动，持续运行 - \Workerman\Worker::safeEcho("[DataSyncScheduler] → 实时模式任务: [{$taskName}]\n"); - Timer::add(0, function () use ($taskId, $taskConfig, $worker) { - $this->executeTask($taskId, $taskConfig, $worker); - }, [], false); - $this->runningTasks[$taskId] = true; - } else { - // 批量模式：根据调度配置执行 - \Workerman\Worker::safeEcho("[DataSyncScheduler] → 批量模式任务: [{$taskName}]\n"); - $schedule = $taskConfig['schedule'] ?? []; - if (!($schedule['enabled'] ?? true)) { - // 调度被禁用，立即执行一次 - \Workerman\Worker::safeEcho("[DataSyncScheduler] → 立即执行（调度已禁用）\n"); - Timer::add(0, function () use ($taskId, $taskConfig, $worker) { - $this->executeTask($taskId, $taskConfig, $worker); - }, [], false); - } else { - // 使用 Cron 表达式定时执行 - $cronExpression = $schedule['cron'] ?? '*/5 * * * *'; - \Workerman\Worker::safeEcho("[DataSyncScheduler] → 定时执行，Cron: {$cronExpression}\n"); - $this->scheduleTask($taskId, $taskConfig, $cronExpression, $worker); - } - } - } - - public function onWorkerStop(Worker $worker): void - { - LoggerHelper::logBusiness('data_collection_scheduler_stopped', [ - 'worker_id' => $worker->id, - ]); - } -} diff --git a/Moncter/Moncter/app/process/DataSyncWorker.php b/Moncter/Moncter/app/process/DataSyncWorker.php deleted file mode 100644 index 1a5276e3..00000000 --- a/Moncter/Moncter/app/process/DataSyncWorker.php +++ /dev/null @@ -1,212 +0,0 @@ -config = config('queue.connections.rabbitmq', []); - - // 初始化 DataSyncService - $this->dataSyncService = new DataSyncService( - new ConsumptionRecordRepository(), - new UserProfileRepository() - ); - - try { - // 建立 RabbitMQ 连接 - $this->connection = new AMQPStreamConnection( - $this->config['host'], - $this->config['port'], - $this->config['user'], - $this->config['password'], - $this->config['vhost'], - false, // insist - 'AMQPLAIN', // login_method - null, // login_response - 'en_US', // locale - $this->config['timeout'] ?? 10.0, // connection_timeout - $this->config['timeout'] ?? 10.0, // read_write_timeout - null, // context - false, // keepalive - $this->config['heartbeat'] ?? 0 // heartbeat - ); - - $this->channel = $this->connection->channel(); - - // 声明队列（确保队列存在） - $queueConfig = $this->config['queues']['data_sync']; - $this->channel->queue_declare( - $queueConfig['name'], - false, // passive - $queueConfig['durable'], - false, // exclusive - $queueConfig['auto_delete'], - false, // nowait - $queueConfig['arguments'] ?? [] - ); - - // 设置 QoS（批量处理） - $consumerConfig = config('queue.consumer.data_sync', []); - $this->channel->basic_qos( - null, // prefetch_size - $consumerConfig['prefetch_count'] ?? 10, // prefetch_count（每次处理10条消息） - false // global - ); - - // 注册消费者回调 - $this->channel->basic_consume( - $queueConfig['name'], - '', // consumer_tag - false, // no_local - false, // no_ack - 设为 false，手动确认 - false, // exclusive - false, // nowait - [$this, 'processMessage'] - ); - - LoggerHelper::logBusiness('data_sync_worker_started', [ - 'worker_id' => $worker->id, - ]); - - // 循环消费消息 - while ($this->channel->is_consuming()) { - $this->channel->wait(); - } - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DataSyncWorker', - 'action' => 'onWorkerStart', - 'worker_id' => $worker->id, - ]); - $this->closeConnection(); - } - } - - /** - * 处理 RabbitMQ 消息 - * - * @param AMQPMessage $msg 消息对象 - * @return void - */ - public function processMessage(AMQPMessage $msg): void - { - $payload = json_decode($msg->getBody(), true); - $sourceId = $payload['source_id'] ?? 'unknown'; - $dataCount = count($payload['data'] ?? []); - - if (empty($payload['data'])) { - LoggerHelper::logBusiness('data_sync_message_empty', [ - 'source_id' => $sourceId, - ]); - $msg->ack(); // 确认消息，不再重试 - return; - } - - LoggerHelper::logBusiness('processing_data_sync_message', [ - 'source_id' => $sourceId, - 'data_count' => $dataCount, - 'worker_id' => Worker::getCurrentWorker()->id, - ]); - - try { - // 调用数据同步服务 - $result = $this->dataSyncService->syncData($payload); - - if ($result['success']) { - $msg->ack(); // 成功处理，发送 ACK - LoggerHelper::logBusiness('data_sync_message_processed', [ - 'source_id' => $sourceId, - 'synced_count' => $result['synced_count'], - 'skipped_count' => $result['skipped_count'], - ]); - } else { - // 处理失败，但不重试（避免重复数据） - $msg->ack(); - LoggerHelper::logError(new \RuntimeException("数据同步失败"), [ - 'component' => 'DataSyncWorker', - 'action' => 'processMessage', - 'source_id' => $sourceId, - 'result' => $result, - ]); - } - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DataSyncWorker', - 'action' => 'processMessage', - 'source_id' => $sourceId, - 'payload' => $payload, - 'worker_id' => Worker::getCurrentWorker()->id, - ]); - - // 业务错误不重试，直接 ACK（避免重复数据） - // 系统错误（如数据库连接断开）可以考虑重试，这里简化处理为直接 ACK - $msg->ack(); - // 如果需要重试，可以 basic_reject($msg->delivery_info['delivery_tag'], true); - // 或者推送到死信队列 - } - } - - /** - * Worker 停止时关闭连接 - * - * @param Worker $worker Worker 实例 - * @return void - */ - public function onWorkerStop(Worker $worker): void - { - LoggerHelper::logBusiness('data_sync_worker_stopping', [ - 'worker_id' => $worker->id, - ]); - $this->closeConnection(); - } - - /** - * 关闭 RabbitMQ 连接和通道 - * - * @return void - */ - private function closeConnection(): void - { - try { - if ($this->channel) { - $this->channel->close(); - } - if ($this->connection && $this->connection->isConnected()) { - $this->connection->close(); - } - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DataSyncWorker', - 'action' => 'closeConnection', - ]); - } finally { - $this->channel = null; - $this->connection = null; - } - } -} - diff --git a/Moncter/Moncter/app/process/Http.php b/Moncter/Moncter/app/process/Http.php deleted file mode 100644 index f462c3a4..00000000 --- a/Moncter/Moncter/app/process/Http.php +++ /dev/null @@ -1,10 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -namespace app\process; - -use FilesystemIterator; -use RecursiveDirectoryIterator; -use RecursiveIteratorIterator; -use SplFileInfo; -use Workerman\Timer; -use Workerman\Worker; - -/** - * Class FileMonitor - * @package process - */ -class Monitor -{ - /** - * @var array - */ - protected array $paths = []; - - /** - * @var array - */ - protected array $extensions = []; - - /** - * @var array - */ - protected array $loadedFiles = []; - - /** - * @var int - */ - protected int $ppid = 0; - - /** - * Pause monitor - * @return void - */ - public static function pause(): void - { - file_put_contents(static::lockFile(), time()); - } - - /** - * Resume monitor - * @return void - */ - public static function resume(): void - { - clearstatcache(); - if (is_file(static::lockFile())) { - unlink(static::lockFile()); - } - } - - /** - * Whether monitor is paused - * @return bool - */ - public static function isPaused(): bool - { - clearstatcache(); - return file_exists(static::lockFile()); - } - - /** - * Lock file - * @return string - */ - protected static function lockFile(): string - { - return runtime_path('monitor.lock'); - } - - /** - * FileMonitor constructor. - * @param $monitorDir - * @param $monitorExtensions - * @param array $options - */ - public function __construct($monitorDir, $monitorExtensions, array $options = []) - { - $this->ppid = function_exists('posix_getppid') ? posix_getppid() : 0; - static::resume(); - $this->paths = (array)$monitorDir; - $this->extensions = $monitorExtensions; - foreach (get_included_files() as $index => $file) { - $this->loadedFiles[$file] = $index; - if (strpos($file, 'webman-framework/src/support/App.php')) { - break; - } - } - if (!Worker::getAllWorkers()) { - return; - } - $disableFunctions = explode(',', ini_get('disable_functions')); - if (in_array('exec', $disableFunctions, true)) { - echo "\nMonitor file change turned off because exec() has been disabled by disable_functions setting in " . PHP_CONFIG_FILE_PATH . "/php.ini\n"; - } else { - if ($options['enable_file_monitor'] ?? true) { - Timer::add(1, function () { - $this->checkAllFilesChange(); - }); - } - } - - $memoryLimit = $this->getMemoryLimit($options['memory_limit'] ?? null); - if ($memoryLimit && ($options['enable_memory_monitor'] ?? true)) { - Timer::add(60, [$this, 'checkMemory'], [$memoryLimit]); - } - } - - /** - * @param $monitorDir - * @return bool - */ - public function checkFilesChange($monitorDir): bool - { - static $lastMtime, $tooManyFilesCheck; - if (!$lastMtime) { - $lastMtime = time(); - } - clearstatcache(); - if (!is_dir($monitorDir)) { - if (!is_file($monitorDir)) { - return false; - } - $iterator = [new SplFileInfo($monitorDir)]; - } else { - // recursive traversal directory - $dirIterator = new RecursiveDirectoryIterator($monitorDir, FilesystemIterator::SKIP_DOTS | FilesystemIterator::FOLLOW_SYMLINKS); - $iterator = new RecursiveIteratorIterator($dirIterator); - } - $count = 0; - foreach ($iterator as $file) { - $count ++; - /** var SplFileInfo $file */ - if (is_dir($file->getRealPath())) { - continue; - } - // check mtime - if (in_array($file->getExtension(), $this->extensions, true) && $lastMtime < $file->getMTime()) { - $lastMtime = $file->getMTime(); - if (DIRECTORY_SEPARATOR === '/' && isset($this->loadedFiles[$file->getRealPath()])) { - echo "$file updated but cannot be reloaded because only auto-loaded files support reload.\n"; - continue; - } - $var = 0; - exec('"'.PHP_BINARY . '" -l ' . $file, $out, $var); - if ($var) { - continue; - } - // send SIGUSR1 signal to master process for reload - if (DIRECTORY_SEPARATOR === '/') { - if ($masterPid = $this->getMasterPid()) { - echo $file . " updated and reload\n"; - posix_kill($masterPid, SIGUSR1); - } else { - echo "Master process has gone away and can not reload\n"; - } - return true; - } - echo $file . " updated and reload\n"; - return true; - } - } - if (!$tooManyFilesCheck && $count > 1000) { - echo "Monitor: There are too many files ($count files) in $monitorDir which makes file monitoring very slow\n"; - $tooManyFilesCheck = 1; - } - return false; - } - - /** - * @return int - */ - public function getMasterPid(): int - { - if ($this->ppid === 0) { - return 0; - } - if (function_exists('posix_kill') && !posix_kill($this->ppid, 0)) { - echo "Master process has gone away\n"; - return $this->ppid = 0; - } - if (PHP_OS_FAMILY !== 'Linux') { - return $this->ppid; - } - $cmdline = "/proc/$this->ppid/cmdline"; - if (!is_readable($cmdline) || !($content = file_get_contents($cmdline)) || (!str_contains($content, 'WorkerMan') && !str_contains($content, 'php'))) { - // Process not exist - $this->ppid = 0; - } - return $this->ppid; - } - - /** - * @return bool - */ - public function checkAllFilesChange(): bool - { - if (static::isPaused()) { - return false; - } - foreach ($this->paths as $path) { - if ($this->checkFilesChange($path)) { - return true; - } - } - return false; - } - - /** - * @param $memoryLimit - * @return void - */ - public function checkMemory($memoryLimit): void - { - if (static::isPaused() || $memoryLimit <= 0) { - return; - } - $masterPid = $this->getMasterPid(); - if ($masterPid <= 0) { - echo "Master process has gone away\n"; - return; - } - - $childrenFile = "/proc/$masterPid/task/$masterPid/children"; - if (!is_file($childrenFile) || !($children = file_get_contents($childrenFile))) { - return; - } - foreach (explode(' ', $children) as $pid) { - $pid = (int)$pid; - $statusFile = "/proc/$pid/status"; - if (!is_file($statusFile) || !($status = file_get_contents($statusFile))) { - continue; - } - $mem = 0; - if (preg_match('/VmRSS\s*?:\s*?(\d+?)\s*?kB/', $status, $match)) { - $mem = $match[1]; - } - $mem = (int)($mem / 1024); - if ($mem >= $memoryLimit) { - posix_kill($pid, SIGINT); - } - } - } - - /** - * Get memory limit - * @param $memoryLimit - * @return int - */ - protected function getMemoryLimit($memoryLimit): int - { - if ($memoryLimit === 0) { - return 0; - } - $usePhpIni = false; - if (!$memoryLimit) { - $memoryLimit = ini_get('memory_limit'); - $usePhpIni = true; - } - - if ($memoryLimit == -1) { - return 0; - } - $unit = strtolower($memoryLimit[strlen($memoryLimit) - 1]); - $memoryLimit = (int)$memoryLimit; - if ($unit === 'g') { - $memoryLimit = 1024 * $memoryLimit; - } else if ($unit === 'k') { - $memoryLimit = ($memoryLimit / 1024); - } else if ($unit === 'm') { - $memoryLimit = (int)($memoryLimit); - } else if ($unit === 't') { - $memoryLimit = (1024 * 1024 * $memoryLimit); - } else { - $memoryLimit = ($memoryLimit / (1024 * 1024)); - } - if ($memoryLimit < 50) { - $memoryLimit = 50; - } - if ($usePhpIni) { - $memoryLimit = (0.8 * $memoryLimit); - } - return (int)$memoryLimit; - } - -} diff --git a/Moncter/Moncter/app/process/TagCalculationWorker.php b/Moncter/Moncter/app/process/TagCalculationWorker.php deleted file mode 100644 index 3153e2cd..00000000 --- a/Moncter/Moncter/app/process/TagCalculationWorker.php +++ /dev/null @@ -1,294 +0,0 @@ -config = config('queue.connections.rabbitmq', []); - - // 使用 Timer 延迟初始化，避免阻塞 Worker 启动 - \Workerman\Timer::add(0, function () use ($worker) { - $this->initConnection($worker); - }, [], false); - } - - /** - * 初始化 RabbitMQ 连接 - * - * @param Worker $worker Worker 实例 - * @return void - */ - private function initConnection(Worker $worker): void - { - static $retryCount = 0; - $maxRetries = 10; - $retryInterval = 5; - - try { - // 清理之前的连接 - $this->closeConnection(); - - // 建立 RabbitMQ 连接 - $this->connection = new AMQPStreamConnection( - $this->config['host'], - $this->config['port'], - $this->config['user'], - $this->config['password'], - $this->config['vhost'], - false, // insist - 'AMQPLAIN', // login_method - null, // login_response - 'en_US', // locale - $this->config['timeout'] ?? 10.0, // connection_timeout - $this->config['timeout'] ?? 10.0, // read_write_timeout - null, // context - false, // keepalive - $this->config['heartbeat'] ?? 0 // heartbeat - ); - - $this->channel = $this->connection->channel(); - - // 声明队列（确保队列存在） - $queueConfig = $this->config['queues']['tag_calculation']; - $this->channel->queue_declare( - $queueConfig['name'], - false, // passive - $queueConfig['durable'], - false, // exclusive - $queueConfig['auto_delete'], - false, // nowait - $queueConfig['arguments'] ?? [] - ); - - // 设置 QoS（每次只处理一条消息） - $consumerConfig = config('queue.consumer.tag_calculation', []); - $this->channel->basic_qos( - null, // prefetch_size - $consumerConfig['prefetch_count'] ?? 1, // prefetch_count - false // global - ); - - // 开始消费消息 - $this->channel->basic_consume( - $queueConfig['name'], - '', // consumer_tag - false, // no_local - $consumerConfig['no_ack'] ?? false, // no_ack - false, // exclusive - false, // nowait - [$this, 'processMessage'] // callback - ); - - LoggerHelper::logBusiness('tag_calculation_worker_started', [ - 'queue' => $queueConfig['name'], - 'worker_id' => $worker->id, - ]); - - // 重置重试计数 - $retryCount = 0; - - // 监听消息（使用 Timer 定期检查，避免阻塞） - \Workerman\Timer::add(0.1, function () use ($worker) { - if ($this->channel && $this->channel->is_consuming()) { - try { - // 非阻塞方式检查消息 - $this->channel->wait(null, false, 0); // timeout 0，非阻塞 - } catch (\PhpAmqpLib\Exception\AMQPTimeoutException $e) { - // 超时是正常的，继续等待 - return; - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'TagCalculationWorker', - 'action' => 'channel_wait', - ]); - // 连接断开，重新初始化（使用当前 Worker） - $currentWorker = \Workerman\Worker::getCurrentWorker(); - if ($currentWorker) { - $this->initConnection($currentWorker); - } - } - } - }, [], true); // 持续执行 - - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'TagCalculationWorker', - 'action' => 'initConnection', - 'retry_count' => $retryCount, - ]); - - // 重试连接（最多重试10次） - if ($retryCount < $maxRetries) { - $retryCount++; - \Workerman\Timer::add($retryInterval, function () use ($worker) { - $this->initConnection($worker); - }, [], false); - } else { - LoggerHelper::logError(new \RuntimeException("RabbitMQ 连接失败，已达到最大重试次数"), [ - 'component' => 'TagCalculationWorker', - 'action' => 'initConnection', - 'max_retries' => $maxRetries, - ]); - } - } - } - - /** - * 关闭连接 - * - * @return void - */ - private function closeConnection(): void - { - try { - if ($this->channel !== null) { - $this->channel->close(); - $this->channel = null; - } - if ($this->connection !== null && $this->connection->isConnected()) { - $this->connection->close(); - $this->connection = null; - } - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'TagCalculationWorker', - 'action' => 'closeConnection', - ]); - } - } - - /** - * 处理消息 - * - * @param AMQPMessage $message - */ - public function processMessage(AMQPMessage $message): void - { - $startTime = microtime(true); - $deliveryTag = $message->getDeliveryTag(); - - try { - // 解析消息 - $body = $message->getBody(); - $data = json_decode($body, true); - - if (json_last_error() !== JSON_ERROR_NONE) { - throw new \InvalidArgumentException('消息格式错误: ' . json_last_error_msg()); - } - - // 验证必要字段 - if (empty($data['user_id'])) { - throw new \InvalidArgumentException('消息缺少 user_id 字段'); - } - - $userId = (string)$data['user_id']; - $tagIds = $data['tag_ids'] ?? null; - $triggerType = $data['trigger_type'] ?? 'consumption_record'; - $recordId = $data['record_id'] ?? null; - - LoggerHelper::logBusiness('tag_calculation_message_received', [ - 'user_id' => $userId, - 'tag_ids' => $tagIds, - 'trigger_type' => $triggerType, - 'record_id' => $recordId, - ]); - - // 创建 TagService 实例 - $tagService = new TagService( - new TagDefinitionRepository(), - new UserProfileRepository(), - new UserTagRepository(), - new TagHistoryRepository(), - new SimpleRuleEngine() - ); - - // 执行标签计算 - $tags = $tagService->calculateTags($userId, $tagIds); - - $duration = microtime(true) - $startTime; - LoggerHelper::logBusiness('tag_calculation_completed', [ - 'user_id' => $userId, - 'updated_count' => count($tags), - 'duration' => $duration, - ]); - LoggerHelper::logPerformance('tag_calculation_async', $duration, [ - 'user_id' => $userId, - 'tag_count' => count($tags), - ]); - - // 确认消息（只有在 no_ack = false 时才需要） - $consumerConfig = config('queue.consumer.tag_calculation', []); - if (!($consumerConfig['no_ack'] ?? false)) { - $message->ack(); - } - } catch (\Throwable $e) { - $duration = microtime(true) - $startTime; - - LoggerHelper::logError($e, [ - 'component' => 'TagCalculationWorker', - 'action' => 'processMessage', - 'delivery_tag' => $deliveryTag, - 'duration' => $duration, - ]); - - // 根据错误类型决定是否重试 - // 如果是业务逻辑错误（如用户不存在），直接确认消息，不重试 - // 如果是系统错误（如数据库连接失败），可以重试 - if ($e instanceof \InvalidArgumentException) { - // 业务逻辑错误，确认消息，不重试 - $consumerConfig = config('queue.consumer.tag_calculation', []); - if (!($consumerConfig['no_ack'] ?? false)) { - $message->ack(); - } - } else { - // 系统错误，拒绝消息并重新入队（重试） - $message->nack(false, true); // requeue = true - } - } - } - - public function onWorkerStop(Worker $worker): void - { - $this->closeConnection(); - - LoggerHelper::logBusiness('tag_calculation_worker_stopped', [ - 'worker_id' => $worker->id, - ]); - } -} - diff --git a/Moncter/Moncter/app/repository/ConsumptionRecordRepository.php b/Moncter/Moncter/app/repository/ConsumptionRecordRepository.php deleted file mode 100644 index 9b705f7f..00000000 --- a/Moncter/Moncter/app/repository/ConsumptionRecordRepository.php +++ /dev/null @@ -1,90 +0,0 @@ - - */ - protected $fillable = [ - 'record_id', - 'user_id', - 'consume_time', - 'amount', - 'actual_amount', - 'currency', - 'store_id', - 'status', - 'create_time', - ]; - - /** - * 字段类型转换 - * - * @var array - */ - protected $casts = [ - 'amount' => 'float', - 'actual_amount'=> 'float', - 'consume_time' => 'datetime', - 'create_time' => 'datetime', - 'status' => 'int', - ]; - - /** - * 禁用 Laravel 默认时间戳 - * - * @var bool - */ - public $timestamps = false; -} - - diff --git a/Moncter/Moncter/app/repository/DataCollectionTaskRepository.php b/Moncter/Moncter/app/repository/DataCollectionTaskRepository.php deleted file mode 100644 index b2051ac3..00000000 --- a/Moncter/Moncter/app/repository/DataCollectionTaskRepository.php +++ /dev/null @@ -1,112 +0,0 @@ - - */ - protected $fillable = [ - 'task_id', - 'name', - 'description', - 'data_source_id', // 源数据源ID - 'database', // 源数据库 - 'collection', // 源集合（单集合模式） - 'collections', // 源集合列表（多集合模式） - 'target_data_source_id', // 目标数据源ID - 'target_database', // 目标数据库 - 'target_collection', // 目标集合 - 'target_type', // 目标类型：consumption_record（消费记录）、generic（通用集合）等 - 'mode', // batch: 批量采集, realtime: 实时监听 - 'field_mappings', // 字段映射配置（单集合模式） - 'collection_field_mappings', // 字段映射配置（多集合模式） - 'lookups', // 连表查询配置（单集合模式） - 'collection_lookups', // 连表查询配置（多集合模式） - 'filter_conditions', // 过滤条件 - 'schedule', // 调度配置 - 'status', // pending: 待启动, running: 运行中, paused: 已暂停, stopped: 已停止, error: 错误 - 'progress', // 进度信息 - 'statistics', // 统计信息 - 'created_by', - 'created_at', - 'updated_at', - ]; - - /** - * 字段类型转换 - * - * @var array - * - * 注意：MongoDB 原生支持数组类型，不需要对数组字段进行 cast - * 如果进行 cast，当数据已经是数组时，Laravel 可能会尝试使用 Json cast 导致错误 - */ - protected $casts = [ - // 数组字段不进行 cast，让 MongoDB 直接处理 - // 'field_mappings' => 'array', - // 'collection_field_mappings' => 'array', - // 'lookups' => 'array', - // 'collection_lookups' => 'array', - // 'filter_conditions' => 'array', - // 'schedule' => 'array', - // 'progress' => 'array', - // 'statistics' => 'array', - // 'collections' => 'array', - 'created_at' => 'datetime', - 'updated_at' => 'datetime', - ]; - - /** - * 启用 Laravel 默认时间戳 - * - * @var bool - */ - public $timestamps = true; -} - diff --git a/Moncter/Moncter/app/repository/DataSourceRepository.php b/Moncter/Moncter/app/repository/DataSourceRepository.php deleted file mode 100644 index 05f82f68..00000000 --- a/Moncter/Moncter/app/repository/DataSourceRepository.php +++ /dev/null @@ -1,99 +0,0 @@ - 'int', - 'options' => 'array', - 'status' => 'int', - 'is_tag_engine' => 'bool', - 'created_at' => 'datetime', - 'updated_at' => 'datetime', - ]; - - public $timestamps = true; - - const CREATED_AT = 'created_at'; - const UPDATED_AT = 'updated_at'; - - /** - * 转换为配置格式（兼容原有的config格式） - * - * @return array - */ - public function toConfigArray(): array - { - $config = [ - 'type' => $this->type, - 'host' => $this->host, - 'port' => $this->port, - 'database' => $this->database, - ]; - - if ($this->username) { - $config['username'] = $this->username; - } - - if ($this->password) { - $config['password'] = $this->password; - } - - if ($this->auth_source) { - $config['auth_source'] = $this->auth_source; - } - - if ($this->options) { - $config['options'] = $this->options; - } - - return $config; - } -} - diff --git a/Moncter/Moncter/app/repository/StoreRepository.php b/Moncter/Moncter/app/repository/StoreRepository.php deleted file mode 100644 index 44b46a97..00000000 --- a/Moncter/Moncter/app/repository/StoreRepository.php +++ /dev/null @@ -1,123 +0,0 @@ - - */ - protected $fillable = [ - 'store_id', - 'store_code', - 'store_name', - 'store_type', - 'store_level', - 'industry_id', - 'industry_detail_id', - 'store_address', - 'store_province', - 'store_city', - 'store_district', - 'store_business_area', - 'store_longitude', - 'store_latitude', - 'store_phone', - 'status', - 'create_time', - 'update_time', - ]; - - /** - * 字段类型转换 - * - * @var array - */ - protected $casts = [ - 'store_longitude' => 'float', - 'store_latitude' => 'float', - 'status' => 'int', - 'create_time' => 'datetime', - 'update_time' => 'datetime', - ]; - - /** - * 禁用 Laravel 默认时间戳 - * - * @var bool - */ - public $timestamps = false; - - /** - * 根据门店名称查找门店 - * - * @param string $storeName 门店名称 - * @return StoreRepository|null - */ - public function findByStoreName(string $storeName): ?StoreRepository - { - return $this->newQuery() - ->where('store_name', $storeName) - ->where('status', 0) // 只查询正常状态的门店 - ->first(); - } - - /** - * 根据门店编码查找门店 - * - * @param string $storeCode 门店编码 - * @return StoreRepository|null - */ - public function findByStoreCode(string $storeCode): ?StoreRepository - { - return $this->newQuery() - ->where('store_code', $storeCode) - ->first(); - } -} - diff --git a/Moncter/Moncter/app/repository/TagCohortRepository.php b/Moncter/Moncter/app/repository/TagCohortRepository.php deleted file mode 100644 index 4bc45799..00000000 --- a/Moncter/Moncter/app/repository/TagCohortRepository.php +++ /dev/null @@ -1,57 +0,0 @@ - 'array', - 'user_ids' => 'array', - 'user_count' => 'int', - 'created_at' => 'datetime', - 'updated_at' => 'datetime', - ]; - - public $timestamps = false; -} - diff --git a/Moncter/Moncter/app/repository/TagDefinitionRepository.php b/Moncter/Moncter/app/repository/TagDefinitionRepository.php deleted file mode 100644 index 3f9d9b4e..00000000 --- a/Moncter/Moncter/app/repository/TagDefinitionRepository.php +++ /dev/null @@ -1,64 +0,0 @@ - 'array', - 'dependencies' => 'array', - 'priority' => 'int', - 'status' => 'int', - 'version' => 'int', - 'create_time' => 'datetime', - 'update_time' => 'datetime', - ]; - - public $timestamps = false; -} - - diff --git a/Moncter/Moncter/app/repository/TagHistoryRepository.php b/Moncter/Moncter/app/repository/TagHistoryRepository.php deleted file mode 100644 index 9f24ed5e..00000000 --- a/Moncter/Moncter/app/repository/TagHistoryRepository.php +++ /dev/null @@ -1,52 +0,0 @@ - 'datetime', - ]; - - public $timestamps = false; -} - - diff --git a/Moncter/Moncter/app/repository/TagTaskExecutionRepository.php b/Moncter/Moncter/app/repository/TagTaskExecutionRepository.php deleted file mode 100644 index a364592c..00000000 --- a/Moncter/Moncter/app/repository/TagTaskExecutionRepository.php +++ /dev/null @@ -1,86 +0,0 @@ - - */ - protected $fillable = [ - 'execution_id', - 'task_id', - 'started_at', - 'finished_at', - 'status', // running, completed, failed, cancelled - 'processed_users', - 'success_count', - 'error_count', - 'error_message', - 'created_at', - ]; - - /** - * 字段类型转换 - * - * @var array - */ - protected $casts = [ - 'started_at' => 'datetime', - 'finished_at' => 'datetime', - 'created_at' => 'datetime', - ]; - - /** - * 启用 Laravel 默认时间戳 - * - * @var bool - */ - public $timestamps = true; -} - diff --git a/Moncter/Moncter/app/repository/TagTaskRepository.php b/Moncter/Moncter/app/repository/TagTaskRepository.php deleted file mode 100644 index b883f4c6..00000000 --- a/Moncter/Moncter/app/repository/TagTaskRepository.php +++ /dev/null @@ -1,95 +0,0 @@ - - */ - protected $fillable = [ - 'task_id', - 'name', - 'description', - 'task_type', // full: 全量计算, incremental: 增量计算, specific: 指定用户 - 'target_tag_ids', // 目标标签ID列表 - 'user_scope', // 用户范围配置 - 'schedule', // 调度配置 - 'config', // 高级配置 - 'status', // pending, running, paused, stopped, completed, error - 'progress', // 进度信息 - 'statistics', // 统计信息 - 'created_by', - 'created_at', - 'updated_at', - ]; - - /** - * 字段类型转换 - * - * @var array - */ - protected $casts = [ - 'target_tag_ids' => 'array', - 'user_scope' => 'array', - 'schedule' => 'array', - 'config' => 'array', - 'progress' => 'array', - 'statistics' => 'array', - 'created_at' => 'datetime', - 'updated_at' => 'datetime', - ]; - - /** - * 启用 Laravel 默认时间戳 - * - * @var bool - */ - public $timestamps = true; -} - diff --git a/Moncter/Moncter/app/repository/UserPhoneRelationRepository.php b/Moncter/Moncter/app/repository/UserPhoneRelationRepository.php deleted file mode 100644 index a214ee20..00000000 --- a/Moncter/Moncter/app/repository/UserPhoneRelationRepository.php +++ /dev/null @@ -1,168 +0,0 @@ - - */ - protected $fillable = [ - 'relation_id', - 'phone_number', - 'phone_hash', - 'user_id', - 'effective_time', - 'expire_time', - 'is_active', - 'type', - 'is_verified', - 'source', - 'create_time', - 'update_time', - ]; - - /** - * 字段类型转换 - * - * @var array - */ - protected $casts = [ - 'effective_time' => 'datetime', - 'expire_time' => 'datetime', - 'is_active' => 'boolean', - 'is_verified' => 'boolean', - 'create_time' => 'datetime', - 'update_time' => 'datetime', - ]; - - /** - * 禁用 Laravel 默认的 created_at/updated_at - * - * 我们使用 create_time / update_time 字段。 - * - * @var bool - */ - public $timestamps = false; - - /** - * 根据 relation_id 获取关联记录 - * - * @param string $relationId - * @return self|null - */ - public function findByRelationId(string $relationId): ?self - { - /** @var Builder $query */ - $query = static::query(); - return $query->where('relation_id', $relationId)->first(); - } - - /** - * 根据手机号哈希查找当前有效的关联 - * - * @param string $phoneHash - * @param \DateTimeInterface|null $atTime 查询时间点（默认为当前时间） - * @return self|null - */ - public function findActiveByPhoneHash(string $phoneHash, ?\DateTimeInterface $atTime = null): ?self - { - $queryTime = $atTime ?? new \DateTimeImmutable('now'); - - /** @var Builder $query */ - $query = static::query(); - return $query->where('phone_hash', $phoneHash) - ->where('effective_time', '<=', $queryTime) - ->where(function($q) use ($queryTime) { - $q->whereNull('expire_time') - ->orWhere('expire_time', '>=', $queryTime); - }) - ->where('is_active', true) - ->orderBy('effective_time', 'desc') - ->first(); - } - - /** - * 根据用户ID查找所有手机号关联（当前有效） - * - * @param string $userId - * @param bool $includeHistory 是否包含历史记录 - * @return array - */ - public function findByUserId(string $userId, bool $includeHistory = false): array - { - /** @var Builder $query */ - $query = static::query(); - $query->where('user_id', $userId); - - if (!$includeHistory) { - $query->where('is_active', true) - ->whereNull('expire_time'); - } - - return $query->orderBy('effective_time', 'desc')->get()->all(); - } - - /** - * 根据手机号哈希查找所有历史关联记录 - * - * @param string $phoneHash - * @return array - */ - public function findHistoryByPhoneHash(string $phoneHash): array - { - /** @var Builder $query */ - $query = static::query(); - return $query->where('phone_hash', $phoneHash) - ->orderBy('effective_time', 'desc') - ->get() - ->all(); - } -} - diff --git a/Moncter/Moncter/app/repository/UserProfileRepository.php b/Moncter/Moncter/app/repository/UserProfileRepository.php deleted file mode 100644 index 3ab7ca8f..00000000 --- a/Moncter/Moncter/app/repository/UserProfileRepository.php +++ /dev/null @@ -1,227 +0,0 @@ - - */ - protected $fillable = [ - 'user_id', - 'id_card_hash', - 'id_card_encrypted', - 'id_card_type', - 'name', - 'phone', - 'address', - 'email', - 'gender', - 'birthday', - 'total_amount', - 'total_count', - 'last_consume_time', - 'tags_update_time', - 'is_temporary', // 是否为临时人（true=临时人，false=正式人） - 'merged_from_user_id', // 如果是从临时人合并而来，记录原user_id - 'status', - 'create_time', - 'update_time', - ]; - - /** - * 字段类型转换 - * - * @var array - */ - protected $casts = [ - 'total_amount' => 'float', - 'total_count' => 'int', - 'last_consume_time' => 'datetime', - 'tags_update_time' => 'datetime', - 'birthday' => 'datetime', - 'is_temporary' => 'bool', - 'status' => 'int', - 'create_time' => 'datetime', - 'update_time' => 'datetime', - ]; - - /** - * 禁用 Laravel 默认的 created_at/updated_at - * - * 我们使用 create_time / update_time 字段。 - * - * @var bool - */ - public $timestamps = false; - - /** - * 根据 user_id 获取用户记录（不存在时返回 null） - */ - public function findByUserId(string $userId): ?self - { - /** @var Builder $query */ - $query = static::query(); - return $query->where('user_id', $userId)->first(); - } - - /** - * 创建或更新用户的基础统计信息 - * - * 仅用于标签系统第一阶段：更新总金额、总次数、最后消费时间。 - * - * @param string $userId - * @param float $amount 本次消费金额 - * @param \DateTimeInterface $consumeTime 消费时间 - */ - public function increaseStats(string $userId, float $amount, \DateTimeInterface $consumeTime): self - { - $now = new \DateTimeImmutable('now'); - - /** @var self|null $user */ - $user = $this->findByUserId($userId); - - if (!$user) { - $user = new self([ - 'user_id' => $userId, - 'total_amount' => $amount, - 'total_count' => 1, - 'last_consume_time'=> $consumeTime, - 'is_temporary' => true, // 默认创建为临时人 - 'status' => 0, - 'create_time' => $now, - 'update_time' => $now, - ]); - } else { - $user->total_amount = (float)$user->total_amount + $amount; - $user->total_count = (int)$user->total_count + 1; - // 只在消费时间更晚时更新 - if (!$user->last_consume_time || $consumeTime > $user->last_consume_time) { - $user->last_consume_time = $consumeTime; - } - $user->update_time = $now; - } - - $user->save(); - - return $user; - } - - /** - * 根据身份证哈希查找用户 - * - * @param string $idCardHash 身份证哈希值 - * @return self|null - */ - public function findByIdCardHash(string $idCardHash): ?self - { - return $this->newQuery() - ->where('id_card_hash', $idCardHash) - ->where('status', 0) - ->first(); - } - - /** - * 查找所有临时人 - * - * @return \Illuminate\Database\Eloquent\Collection - */ - public function findTemporaryUsers() - { - return $this->newQuery() - ->where('is_temporary', true) - ->where('status', 0) - ->get(); - } - - /** - * 标记用户为正式人 - * - * @param string $userId - * @param string|null $idCardHash 身份证哈希 - * @param string|null $idCardEncrypted 加密的身份证 - * @param string|null $idCard 原始身份证号（用于提取基础信息，可选） - * @return bool - */ - public function markAsFormal(string $userId, ?string $idCardHash = null, ?string $idCardEncrypted = null, ?string $idCard = null): bool - { - $user = $this->findByUserId($userId); - if (!$user) { - return false; - } - - $user->is_temporary = false; - if ($idCardHash !== null) { - $user->id_card_hash = $idCardHash; - } - if ($idCardEncrypted !== null) { - $user->id_card_encrypted = $idCardEncrypted; - } - - // 如果有原始身份证号，自动提取基础信息（如果字段为空才更新） - if ($idCard !== null && !empty($idCard)) { - $idCardInfo = \app\utils\IdCardHelper::extractInfo($idCard); - if ($idCardInfo['birthday'] !== null && $user->birthday === null) { - $user->birthday = $idCardInfo['birthday']; - } - // 只有当性别解析成功且当前值为 null 时才更新（0 也被认为是未设置） - if ($idCardInfo['gender'] > 0 && ($user->gender === null || $user->gender === 0)) { - $user->gender = $idCardInfo['gender']; - } - } - - $user->update_time = new \DateTimeImmutable('now'); - return $user->save(); - } -} - - diff --git a/Moncter/Moncter/app/repository/UserTagRepository.php b/Moncter/Moncter/app/repository/UserTagRepository.php deleted file mode 100644 index 20d1cba3..00000000 --- a/Moncter/Moncter/app/repository/UserTagRepository.php +++ /dev/null @@ -1,62 +0,0 @@ - 'string', - 'confidence' => 'float', - 'effective_time' => 'datetime', - 'expire_time' => 'datetime', - 'create_time' => 'datetime', - 'update_time' => 'datetime', - ]; - - public $timestamps = false; -} - - diff --git a/Moncter/Moncter/app/service/ConsumptionService.php b/Moncter/Moncter/app/service/ConsumptionService.php deleted file mode 100644 index 65d10786..00000000 --- a/Moncter/Moncter/app/service/ConsumptionService.php +++ /dev/null @@ -1,282 +0,0 @@ - $payload - * @return array|null 如果手机号和身份证号都为空，返回null（跳过该记录） - */ - public function createRecord(array $payload): ?array - { - // 基础必填字段校验 - foreach (['amount', 'actual_amount', 'store_id', 'consume_time'] as $field) { - if (!isset($payload[$field]) || $payload[$field] === '') { - throw new \InvalidArgumentException("缺少必填字段：{$field}"); - } - } - - $amount = (float)$payload['amount']; - $actual = (float)$payload['actual_amount']; - $storeId = (string)$payload['store_id']; - $consumeTime = new \DateTimeImmutable((string)$payload['consume_time']); - - // 解析用户ID：优先使用user_id，如果没有则通过手机号/身份证解析 - $userId = null; - if (!empty($payload['user_id'])) { - $userId = (string)$payload['user_id']; - } else { - // 通过手机号或身份证解析用户ID - $phoneNumber = trim($payload['phone_number'] ?? ''); - $idCard = trim($payload['id_card'] ?? ''); - - // 如果手机号和身份证号都为空，直接跳过该记录 - if (empty($phoneNumber) && empty($idCard)) { - LoggerHelper::logBusiness('consumption_record_skipped_no_identifier', [ - 'reason' => 'phone_number and id_card are both empty', - 'consume_time' => $consumeTime->format('Y-m-d H:i:s'), - ]); - return null; - } - - // 传入 consume_time 作为查询时间点 - $userId = $this->identifierService->resolvePersonId($phoneNumber, $idCard, $consumeTime); - - // 如果同时提供了手机号和身份证，检查是否需要合并 - if (!empty($phoneNumber) && !empty($idCard)) { - $userId = $this->handleMergeIfNeeded($phoneNumber, $idCard, $userId, $consumeTime); - } - } - - $now = new \DateTimeImmutable('now'); - $recordId = UuidGenerator::uuid4()->toString(); - - // 写入消费记录 - $record = new ConsumptionRecordRepository(); - $record->record_id = $recordId; - $record->user_id = $userId; - $record->consume_time = $consumeTime; - $record->amount = $amount; - $record->actual_amount = $actual; - $record->currency = $payload['currency'] ?? 'CNY'; - $record->store_id = $storeId; - $record->status = 0; - $record->create_time = $now; - $record->save(); - - // 更新用户统计信息 - $user = $this->userProfileRepository->increaseStats($userId, $actual, $consumeTime); - - // 触发标签计算（异步方式） - $tags = []; - $useAsync = getenv('TAG_CALCULATION_ASYNC') !== 'false'; // 默认使用异步，可通过环境变量关闭 - - if ($useAsync) { - // 异步方式：推送到消息队列 - try { - $success = QueueService::pushTagCalculation([ - 'user_id' => $userId, - 'tag_ids' => null, // null 表示计算所有 real_time 标签 - 'trigger_type' => 'consumption_record', - 'record_id' => $recordId, - 'timestamp' => time(), - ]); - - if ($success) { - LoggerHelper::logBusiness('tag_calculation_queued', [ - 'user_id' => $userId, - 'record_id' => $recordId, - ]); - } else { - // 如果推送失败，降级到同步调用 - LoggerHelper::logBusiness('tag_calculation_queue_failed_fallback', [ - 'user_id' => $userId, - 'record_id' => $recordId, - ]); - $useAsync = false; - } - } catch (\Throwable $e) { - // 如果队列服务异常，降级到同步调用 - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionService', - 'action' => 'pushTagCalculation', - 'user_id' => $userId, - ]); - $useAsync = false; - } - } - - // 同步方式（降级方案或配置关闭异步时使用） - if (!$useAsync && $this->tagService) { - try { - $tags = $this->tagService->calculateTags($userId); - } catch (\Throwable $e) { - // 标签计算失败不影响消费记录写入，只记录错误 - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionService', - 'action' => 'calculateTags', - 'user_id' => $userId, - ]); - } - } - - return [ - 'record_id' => $recordId, - 'user_id' => $userId, - 'user' => [ - 'total_amount' => $user->total_amount ?? 0, - 'total_count' => $user->total_count ?? 0, - 'last_consume_time' => $user->last_consume_time, - ], - 'tags' => $tags, // 异步模式下为空数组，同步模式下包含标签信息 - 'tag_calculation_mode' => $useAsync ? 'async' : 'sync', - ]; - } - - /** - * 当手机号和身份证号同时出现时，检查是否需要合并用户 - * - * @param string $phoneNumber 手机号 - * @param string $idCard 身份证号 - * @param string $currentUserId 当前解析出的用户ID - * @param \DateTimeInterface $consumeTime 消费时间 - * @return string 最终使用的用户ID - */ - private function handleMergeIfNeeded( - string $phoneNumber, - string $idCard, - string $currentUserId, - \DateTimeInterface $consumeTime - ): string { - // 通过身份证查找用户 - $userIdByIdCard = $this->identifierService->resolvePersonIdByIdCard($idCard); - - // 在消费时间点查询手机号关联（使用反射或公共方法） - $userPhoneService = new \app\service\UserPhoneService( - new \app\repository\UserPhoneRelationRepository() - ); - $userIdByPhone = $userPhoneService->findUserByPhone($phoneNumber, $consumeTime); - - // 如果身份证找到用户A，手机号关联到用户B，且A≠B - if ($userIdByIdCard && $userIdByPhone && $userIdByIdCard !== $userIdByPhone) { - // 检查用户B是否为临时用户 - $userB = $this->userProfileRepository->findByUserId($userIdByPhone); - $userA = $this->userProfileRepository->findByUserId($userIdByIdCard); - - if ($userB && $userB->is_temporary) { - // 情况1：用户B是临时用户 → 合并到正式用户A - // 需要合并服务，动态创建 - $tagService = $this->tagService ?? new TagService( - new \app\repository\TagDefinitionRepository(), - $this->userProfileRepository, - new \app\repository\UserTagRepository(), - new \app\repository\TagHistoryRepository(), - new \app\service\TagRuleEngine\SimpleRuleEngine() - ); - - $mergeService = new PersonMergeService( - $this->userProfileRepository, - new \app\repository\UserTagRepository(), - $userPhoneService, - $tagService - ); - - // 合并临时用户B到正式用户A - $mergeService->mergeUsers($userIdByPhone, $userIdByIdCard); - - // 将旧的手机关联标记为过期（使用消费时间作为过期时间） - $userPhoneService->removePhoneFromUser($userIdByPhone, $phoneNumber, $consumeTime); - - // 建立新的手机关联到用户A（使用消费时间作为生效时间） - $userPhoneService->addPhoneToUser($userIdByIdCard, $phoneNumber, [ - 'source' => 'merge_after_id_card_binding', - 'effective_time' => $consumeTime, - 'type' => 'personal', - ]); - - LoggerHelper::logBusiness('auto_merge_triggered', [ - 'phone_number' => $phoneNumber, - 'source_user_id' => $userIdByPhone, - 'target_user_id' => $userIdByIdCard, - 'consume_time' => $consumeTime->format('Y-m-d H:i:s'), - 'reason' => 'temporary_user_merge', - ]); - - return $userIdByIdCard; - } elseif ($userA && !$userA->is_temporary && $userB && !$userB->is_temporary) { - // 情况2：两者都是正式用户（如酒店预订代订场景） - // 策略：以身份证为准，消费记录归属到身份证用户，但手机号关联保持不变 - // 原因：手机号和身份证同时出现时，身份证更可信；但手机号可能是代订，不应自动转移 - - // 检查手机号在消费时间点是否已经关联到用户A（可能之前已经转移过） - $phoneRelationAtTime = $userPhoneService->findUserByPhone($phoneNumber, $consumeTime); - - if ($phoneRelationAtTime !== $userIdByIdCard) { - // 手机号在消费时间点还未关联到身份证用户 - // 记录异常情况，但不强制转移手机号（可能是代订场景） - LoggerHelper::logBusiness('phone_id_card_mismatch_formal_users', [ - 'phone_number' => $phoneNumber, - 'phone_user_id' => $userIdByPhone, - 'id_card_user_id' => $userIdByIdCard, - 'consume_time' => $consumeTime->format('Y-m-d H:i:s'), - 'decision' => 'use_id_card_user', - 'note' => '正式用户冲突，以身份证为准（可能是代订场景）', - ]); - } - - // 以身份证用户为准，返回身份证用户ID - return $userIdByIdCard; - } else { - // 其他情况（理论上不应该发生），记录日志并返回身份证用户 - LoggerHelper::logBusiness('phone_id_card_mismatch_unknown', [ - 'phone_number' => $phoneNumber, - 'phone_user_id' => $userIdByPhone, - 'id_card_user_id' => $userIdByIdCard, - 'phone_user_is_temporary' => $userB ? $userB->is_temporary : 'unknown', - 'id_card_user_is_temporary' => $userA ? $userA->is_temporary : 'unknown', - 'consume_time' => $consumeTime->format('Y-m-d H:i:s'), - ]); - - // 默认返回身份证用户（更可信） - return $userIdByIdCard; - } - } - - return $currentUserId; - } - -} - - diff --git a/Moncter/Moncter/app/service/DataCollection/Handler/BaseCollectionHandler.php b/Moncter/Moncter/app/service/DataCollection/Handler/BaseCollectionHandler.php deleted file mode 100644 index aa8a6422..00000000 --- a/Moncter/Moncter/app/service/DataCollection/Handler/BaseCollectionHandler.php +++ /dev/null @@ -1,115 +0,0 @@ -dataSourceService = new DataSourceService( - new DataSourceRepository() - ); - - // 初始化公共服务（避免在子类中重复实例化） - $this->identifierService = new \app\service\IdentifierService( - new \app\repository\UserProfileRepository(), - new \app\service\UserPhoneService( - new \app\repository\UserPhoneRelationRepository() - ) - ); - - $this->consumptionService = new \app\service\ConsumptionService( - new \app\repository\ConsumptionRecordRepository(), - new \app\repository\UserProfileRepository(), - $this->identifierService - ); - - $this->storeService = new \app\service\StoreService( - new \app\repository\StoreRepository() - ); - } - - /** - * 获取 MongoDB 客户端 - * - * @param array $taskConfig 任务配置 - * @return Client MongoDB 客户端实例 - * @throws \InvalidArgumentException 如果数据源配置不存在 - */ - protected function getMongoClient(array $taskConfig): Client - { - $dataSourceId = $taskConfig['data_source_id'] - ?? $taskConfig['data_source'] - ?? 'sync_mongodb'; - - $dataSourceConfig = $this->dataSourceService->getDataSourceConfigById($dataSourceId); - - if (empty($dataSourceConfig)) { - throw new \InvalidArgumentException("数据源配置不存在: {$dataSourceId}"); - } - - return MongoDBHelper::createClient($dataSourceConfig); - } - - /** - * 连接到目标数据源 - * - * @param string $targetDataSourceId 目标数据源ID - * @param string|null $targetDatabase 目标数据库名（可选，默认使用数据源配置中的数据库） - * @return array{client: Client, database: \MongoDB\Database, dbName: string, config: array} 连接信息 - * @throws \InvalidArgumentException 如果目标数据源配置不存在 - */ - protected function connectToTargetDataSource( - string $targetDataSourceId, - ?string $targetDatabase = null - ): array { - $targetDataSourceConfig = $this->dataSourceService->getDataSourceConfigById($targetDataSourceId); - - if (empty($targetDataSourceConfig)) { - throw new \InvalidArgumentException("目标数据源配置不存在: {$targetDataSourceId}"); - } - - $client = MongoDBHelper::createClient($targetDataSourceConfig); - $dbName = $targetDatabase ?? $targetDataSourceConfig['database'] ?? 'ckb'; - $database = $client->selectDatabase($dbName); - - return [ - 'client' => $client, - 'database' => $database, - 'dbName' => $dbName, - 'config' => $targetDataSourceConfig, - ]; - } - - /** - * 采集数据（抽象方法，由子类实现） - * - * @param mixed $adapter 数据源适配器 - * @param array $taskConfig 任务配置 - * @return void - */ - abstract public function collect($adapter, array $taskConfig): void; -} - diff --git a/Moncter/Moncter/app/service/DataCollection/Handler/ConsumptionCollectionHandler.php b/Moncter/Moncter/app/service/DataCollection/Handler/ConsumptionCollectionHandler.php deleted file mode 100644 index b4fd4037..00000000 --- a/Moncter/Moncter/app/service/DataCollection/Handler/ConsumptionCollectionHandler.php +++ /dev/null @@ -1,1760 +0,0 @@ -taskService = new \app\service\DataCollectionTaskService( - new \app\repository\DataCollectionTaskRepository() - ); - } - - /** - * 采集消费记录 - * - * @param \app\service\DataSource\DataSourceAdapterInterface $adapter 数据源适配器 - * @param array $taskConfig 任务配置 - * @return void - */ - public function collect($adapter, array $taskConfig): void - { - $this->taskConfig = $taskConfig; - $taskId = $taskConfig['task_id'] ?? ''; - $taskName = $taskConfig['name'] ?? '消费记录采集'; - $sourceType = $taskConfig['source_type'] ?? 'kr_mall'; // kr_mall, kr_finance - $mode = $taskConfig['mode'] ?? 'batch'; // batch: 批量采集, realtime: 实时监听 - - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤5-Handler开始】任务ID={$taskId}, 任务名称={$taskName}, 数据源类型={$sourceType}, 模式={$mode}\n"); - LoggerHelper::logBusiness('consumption_collection_started', [ - 'task_id' => $taskId, - 'task_name' => $taskName, - 'source_type' => $sourceType, - 'mode' => $mode, - ]); - - try { - // 根据模式执行不同的采集逻辑 - if ($mode === 'realtime') { - // 实时监听模式 - switch ($sourceType) { - case 'kr_mall': - $this->watchKrMallCollection($taskConfig); - break; - case 'kr_finance': - $this->watchKrFinanceCollections($taskConfig); - break; - default: - throw new \InvalidArgumentException("不支持的数据源类型: {$sourceType}"); - } - } else { - // 批量采集模式 - switch ($sourceType) { - case 'kr_mall': - $this->collectFromKrMall($adapter, $taskConfig); - break; - case 'kr_finance': - $this->collectFromKrFinance($adapter, $taskConfig); - break; - default: - throw new \InvalidArgumentException("不支持的数据源类型: {$sourceType}"); - } - - LoggerHelper::logBusiness('consumption_collection_completed', [ - 'task_id' => $taskId, - 'task_name' => $taskName, - ]); - } - - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'collect', - 'task_id' => $taskId, - ]); - throw $e; - } - } - - /** - * 从KR_商城数据库采集订单数据 - * - * @param mixed $adapter 数据源适配器 - * @param array $taskConfig 任务配置 - * @return void - */ - private function collectFromKrMall($adapter, array $taskConfig): void - { - $taskId = $taskConfig['task_id'] ?? ''; - $databaseName = $taskConfig['database'] ?? 'KR_商城'; - $collectionName = $taskConfig['collection'] ?? '21年贝蒂喜订单整合'; - $lastSyncTime = $taskConfig['last_sync_time'] ?? null; - $batchSize = $taskConfig['batch_size'] ?? 1000; - - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤6-连接源数据库】开始连接: database={$databaseName}, collection={$collectionName}\n"); - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤6-连接源数据库】任务配置来源: task_id={$taskId}\n"); - LoggerHelper::logBusiness('kr_mall_collection_start', [ - 'database' => $databaseName, - 'collection' => $collectionName, - ]); - - // 获取MongoDB客户端和数据库 - $client = $this->getMongoClient($taskConfig); - $database = $client->selectDatabase($databaseName); - $collection = $database->selectCollection($collectionName); - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤6-连接源数据库】✓ 源数据库连接成功\n"); - - // 构建查询条件（如果有上次同步时间，只查询新数据） - $filter = []; - if ($lastSyncTime !== null) { - $lastSyncTimestamp = is_numeric($lastSyncTime) ? (int)$lastSyncTime : strtotime($lastSyncTime); - $lastSyncDate = new \MongoDB\BSON\UTCDateTime($lastSyncTimestamp * 1000); - $filter['订单创建时间'] = ['$gt' => $lastSyncDate]; - } - - // 获取总数（用于计算进度） - $totalCount = $collection->countDocuments($filter); - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤7-统计总数】总记录数: {$totalCount}\n"); - - // 计算进度更新间隔（根据总数动态调整） - $updateInterval = $this->calculateProgressUpdateInterval($totalCount); - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤7-统计总数】进度更新间隔: 每 {$updateInterval} 条更新一次\n"); - - // 更新进度：开始采集 - // 注意：任务状态已经在 startTask 方法中设置为 running，这里不需要再次更新状态 - // 只需要更新进度信息（start_time, total_count等） - if (!empty($taskId)) { - $this->updateProgress($taskId, [ - // 不更新 status，因为 startTask 已经设置为 running - 'start_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - 'total_count' => $totalCount, - 'processed_count' => 0, - 'success_count' => 0, - 'error_count' => 0, - 'percentage' => 0, - ]); - } - - // 分页查询 - $offset = 0; - $processedCount = 0; - $successCount = 0; - $errorCount = 0; - $lastUpdateCount = 0; // 记录上次更新的处理数量 - $isCompleted = false; // 标记是否已完成 - - do { - $cursor = $collection->find( - $filter, - [ - 'limit' => $batchSize, - 'skip' => $offset, - 'sort' => ['订单创建时间' => 1], - ] - ); - - $batch = []; - foreach ($cursor as $doc) { - $batch[] = $this->convertMongoDocumentToArray($doc); - } - - if (empty($batch)) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤8-查询数据】批次为空，结束查询\n"); - break; - } - - $batchCount = count($batch); - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤8-查询数据】查询到 {$batchCount} 条数据，offset={$offset}\n"); - - // 获取任务ID - $taskId = $taskConfig['task_id'] ?? ''; - - // 检查任务状态（在批次处理前） - if (!empty($taskId) && !$this->checkTaskStatus($taskId)) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ⚠️ 任务已暂停或停止，停止采集\n"); - break; - } - - // 如果已完成，不再处理 - if ($isCompleted) { - break; - } - - // 处理批量数据 - foreach ($batch as $index => $orderData) { - // 每10条检查一次任务状态 - if (!empty($taskId) && ($index + 1) % 10 === 0) { - if (!$this->checkTaskStatus($taskId)) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ⚠️ 任务已暂停或停止，停止处理剩余数据\n"); - break 2; // 跳出两层循环（foreach 和 do-while） - } - } - - // 检查是否已达到总数（在每条处理前检查，避免超出） - // 注意：检查在递增之前，如果 processedCount == totalCount - 1，会继续处理一条 - // 然后 processedCount 变成 totalCount，下次循环时会 break - if ($totalCount > 0 && $processedCount >= $totalCount) { - $isCompleted = true; - break; // 跳出当前批次处理循环 - } - - $processedCount++; - $orderNo = $orderData['订单编号'] ?? 'unknown'; - - try { - // 每10条输出一次简要进度 - if (($index + 1) % 10 === 0) { - // \Workerman\Worker::safeEcho(" ⏳ 批量处理进度: {$processedCount} / {$batchCount} (本批次) | 总成功: {$successCount} | 总失败: {$errorCount}\n"); - } - - $this->processKrMallOrder($orderData, $taskConfig); - $successCount++; - } catch (\Exception $e) { - $errorCount++; - $errorMsg = $e->getMessage(); - // \Workerman\Worker::safeEcho(" ❌ [订单编号: {$orderNo}] 处理失败: {$errorMsg}\n"); - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'processKrMallOrder', - 'order_no' => $orderNo, - ]); - } - } - - // 批次处理完成，根据更新间隔决定是否更新进度 - if (!empty($taskId) && $totalCount > 0) { - // 只有当处理数量达到更新间隔时才更新进度 - if (($processedCount - $lastUpdateCount) >= $updateInterval || $processedCount >= $totalCount) { - $percentage = round(($processedCount / $totalCount) * 100, 2); - - // 检查是否达到100% - if ($processedCount >= $totalCount) { - // 进度达到100%，停止采集并更新状态为已完成 - $this->updateProgress($taskId, [ - 'status' => 'completed', - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'percentage' => 100, - 'end_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]); - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ✅ 采集完成，进度已达到100%，已停止采集\n"); - $isCompleted = true; // 标记为已完成 - } else { - $this->updateProgress($taskId, [ - 'total_count' => $totalCount, // 确保每次更新都包含 total_count - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'percentage' => $percentage, - ]); - } - $lastUpdateCount = $processedCount; - } - } - - // 批次处理完成，输出统计 - if ($batchCount > 0) { - // \Workerman\Worker::safeEcho(" 📊 本批次完成: 总数={$batchCount}, 成功={$successCount}, 失败={$errorCount}\n"); - } - - $offset += $batchSize; - - LoggerHelper::logBusiness('kr_mall_collection_batch_processed', [ - 'processed' => $processedCount, - 'success' => $successCount, - 'error' => $errorCount, - 'offset' => $offset, - ]); - - } while (count($batch) === $batchSize && !$isCompleted); - - // 更新进度：采集完成（如果循环正常结束，也更新状态为已完成） - if (!empty($taskId)) { - $percentage = $totalCount > 0 ? round(($processedCount / $totalCount) * 100, 2) : 100; - // 获取当前任务状态 - $task = $this->taskService->getTask($taskId); - if ($task) { - // 只有在任务状态不是 completed、paused、stopped 时，才更新为 completed - // 如果任务被暂停或停止，不应该更新为 completed - if ($task['status'] === 'completed') { - // 已经是 completed，不需要更新 - } elseif (in_array($task['status'], ['paused', 'stopped'])) { - // 任务被暂停或停止，只更新进度，不更新状态 - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ⚠️ 任务已被暂停或停止，不更新为completed状态\n"); - $this->updateProgress($taskId, [ - 'total_count' => $totalCount, - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'percentage' => $percentage, - ]); - } else { - // 任务正常完成，更新状态为 completed - $this->updateProgress($taskId, [ - 'status' => 'completed', - 'total_count' => $totalCount, - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'percentage' => 100, // 完成时强制设置为100% - 'end_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]); - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ✅ 采集任务完成，状态已更新为completed\n"); - } - } - } - - LoggerHelper::logBusiness('kr_mall_collection_completed', [ - 'total_processed' => $processedCount, - 'total_success' => $successCount, - 'total_error' => $errorCount, - ]); - } - - /** - * 处理KR_商城订单数据 - * - * @param array $orderData 订单数据 - * @param array $taskConfig 任务配置 - * @return void - */ - private function processKrMallOrder(array $orderData, array $taskConfig): void - { - $orderNo = $orderData['订单编号'] ?? 'unknown'; - - // 1. 提取手机号（优先使用支付宝账号，其次是收货人电话） - $phoneNumber = $this->extractPhoneNumber($orderData); - if (empty($phoneNumber)) { - LoggerHelper::logBusiness('consumption_collection_skip_no_phone', [ - 'order_no' => $orderNo, - 'reason' => '无法提取手机号', - ]); - // \Workerman\Worker::safeEcho(" ⚠️ [订单编号: {$orderNo}] 跳过：无法提取手机号\n"); - return; // 跳过无法提取手机号的记录 - } - - // 2. 字段映射和转换 - $consumeRecord = $this->transformKrMallOrder($orderData, $phoneNumber, $taskConfig); - - // 3. 写入消费记录（会在saveConsumptionRecord中输出详细的流水信息） - $this->saveConsumptionRecord($consumeRecord); - } - - /** - * 转换KR_商城订单数据为标准消费记录格式 - * - * @param array $orderData 订单数据 - * @param string $phoneNumber 手机号 - * @param array $taskConfig 任务配置 - * @return array 标准消费记录数据 - */ - private function transformKrMallOrder(array $orderData, string $phoneNumber, array $taskConfig): array - { - // 首先应用字段映射（从任务配置中读取字段映射） - $fieldMappings = $taskConfig['field_mappings'] ?? []; - - // 调试：输出字段映射配置和源数据字段 - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤9-字段映射】字段映射配置数量: " . count($fieldMappings) . "\n"); - if (!empty($fieldMappings)) { - foreach ($fieldMappings as $idx => $mapping) { - $targetField = $mapping['target_field'] ?? ''; - $sourceField = $mapping['source_field'] ?? ''; - if ($targetField === 'store_name') { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤9-字段映射】找到store_name映射: target={$targetField}, source={$sourceField}\n"); - } - } - } - - // 调试：输出源数据中的字段名（用于排查） - $sourceFields = array_keys($orderData); - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤9-字段映射】源数据字段列表: " . implode(', ', array_slice($sourceFields, 0, 20)) . (count($sourceFields) > 20 ? '...' : '') . "\n"); - - $mappedData = $this->applyFieldMappings($orderData, $fieldMappings); - - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤9-字段映射】应用字段映射完成，映射字段数: " . count($mappedData) . ", 映射后的字段: " . implode(', ', array_keys($mappedData)) . "\n"); - - // 从映射后的数据中获取字段值（如果没有映射，使用默认值或从源数据获取） - // 消费时间：从字段映射中获取，如果没有则尝试从源数据获取 - $consumeTimeStr = $mappedData['consume_time'] ?? null; - if (empty($consumeTimeStr)) { - // 后备方案：尝试从源数据中获取（向后兼容） - $consumeTimeStr = $orderData['订单付款时间'] ?? $orderData['订单创建时间'] ?? null; - } - $consumeTime = $this->parseDateTime($consumeTimeStr); - if ($consumeTime === null) { - throw new \InvalidArgumentException('无法解析消费时间'); - } - - // 金额：从字段映射中获取 - $totalAmount = $this->parseAmount($mappedData['amount'] ?? '0'); - $actualAmount = $this->parseAmount($mappedData['actual_amount'] ?? $totalAmount); - $discountAmount = $totalAmount - $actualAmount; - - // 积分抵扣：从字段映射中获取（如果有） - $pointsDeduction = 0; - if (isset($mappedData['points_deduction']) && !empty($mappedData['points_deduction'])) { - $pointsDeduction = $this->parseAmount($mappedData['points_deduction']); - } - - // 门店名称：从字段映射中获取，优先保存原始门店名称 - $storeName = $mappedData['store_name'] ?? null; - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤9-字段映射】映射后的store_name值: " . ($storeName ?? 'null') . "\n"); - - if (empty($storeName)) { - // 后备方案：尝试从源数据中获取（向后兼容） - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤9-字段映射】store_name为空，尝试从源数据中查找\n"); - // 尝试多个可能的字段名 - $possibleStoreNameFields = ['新零售成交门店昵称', '门店名称', '店铺名称', '门店名', '店铺名', 'store_name', 'storeName', '门店', '店铺']; - foreach ($possibleStoreNameFields as $fieldName) { - if (isset($orderData[$fieldName]) && !empty($orderData[$fieldName])) { - $storeName = $orderData[$fieldName]; - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤9-字段映射】从源数据中找到门店名称: {$fieldName} = {$storeName}\n"); - break; - } - } - if (empty($storeName)) { - $storeName = 'KR_商城_在线店铺'; // 默认值 - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤9-字段映射】未找到门店名称字段，使用默认值: {$storeName}\n"); - } - } - - // 门店ID：通过门店服务获取或创建（即使失败也不影响门店名称的保存） - $storeId = $mappedData['store_id'] ?? null; - if (empty($storeId) && !empty($storeName)) { - try { - $source = $taskConfig['data_source_id'] ?? $taskConfig['name'] ?? 'KR_商城'; - $storeId = $this->storeService->getOrCreateStoreByName($storeName, $source); - } catch (\Throwable $e) { - // 店铺ID获取失败不影响门店名称的保存，只记录日志 - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'transformKrMallOrder', - 'message' => '获取店铺ID失败，但会继续保存门店名称', - 'store_name' => $storeName, - ]); - } - } - - // 支付方式：从字段映射中获取 - $paymentMethodCode = $mappedData['payment_method_code'] ?? null; - if (empty($paymentMethodCode)) { - // 后备方案：从源数据解析（向后兼容） - $paymentMethodCode = $this->parsePaymentMethod($orderData['支付详情'] ?? ''); - } - - // 支付状态：从字段映射中获取，如果没有则从订单状态解析 - $paymentStatus = $mappedData['payment_status'] ?? null; - if ($paymentStatus === null) { - // 后备方案：从源数据解析（向后兼容） - $paymentStatus = $this->parsePaymentStatus($orderData['订单状态'] ?? ''); - } - - // 消费渠道：从字段映射中获取 - $consumeChannel = $mappedData['consume_channel'] ?? null; - if (empty($consumeChannel)) { - // 后备方案：从源数据解析（向后兼容） - $consumeChannel = $this->parseConsumeChannel($orderData['是否手机订单'] ?? ''); - } - - // 消费时段 - $consumePeriod = $this->parseConsumePeriod($consumeTime); - - // 原始订单ID：从字段映射中获取 - $sourceOrderId = $mappedData['source_order_id'] ?? null; - if (empty($sourceOrderId)) { - // 后备方案：从源数据获取（向后兼容） - $sourceOrderId = $orderData['订单编号'] ?? null; - } - - // 币种：从字段映射中获取，默认为CNY - $currency = $mappedData['currency'] ?? 'CNY'; - - // 状态：从字段映射中获取，默认为0（正常） - $status = $mappedData['status'] ?? 0; - - // 支付单号：从字段映射中获取 - $paymentTransactionId = $mappedData['payment_transaction_id'] ?? null; - if (empty($paymentTransactionId)) { - // 后备方案：从源数据获取（向后兼容） - $paymentTransactionId = $orderData['支付单号'] ?? null; - } - - return [ - 'phone_number' => $phoneNumber, // 传递手机号，让ConsumptionService解析user_id - 'consume_time' => $consumeTime->format('Y-m-d H:i:s'), - 'amount' => $totalAmount, - 'actual_amount' => $actualAmount, - 'discount_amount' => $discountAmount > 0 ? $discountAmount : null, - 'points_deduction' => $pointsDeduction > 0 ? $pointsDeduction : null, - 'currency' => $currency, - 'store_id' => $storeId, - 'store_name' => $storeName, // 保存门店名称，用于去重和展示 - 'payment_method_code' => $paymentMethodCode, - 'payment_channel' => $paymentMethodCode === 'alipay' ? '支付宝' : '其他', - 'payment_transaction_id' => $paymentTransactionId, - 'payment_status' => $paymentStatus, - 'consume_channel' => $consumeChannel, - 'consume_period' => $consumePeriod, - 'is_workday' => $this->isWorkday($consumeTime) ? 1 : 0, - 'source_order_id' => $sourceOrderId, // 原始订单ID，用于去重 - 'status' => $status, - ]; - } - - /** - * 从KR数据库采集金融贷款数据 - * - * @param mixed $adapter 数据源适配器 - * @param array $taskConfig 任务配置 - * @return void - */ - private function collectFromKrFinance($adapter, array $taskConfig): void - { - $taskId = $taskConfig['task_id'] ?? ''; - $databaseName = $taskConfig['database'] ?? 'KR'; - $collections = $taskConfig['collections'] ?? [ - '金融客户_厦门_A级用户', - '金融客户_厦门_B级用户', - '金融客户_厦门_C级用户', - '金融客户_厦门_D级用户', - '金融客户_厦门_E级用户', - '厦门用户资产2025年9月_优化版', - ]; - - LoggerHelper::logBusiness('kr_finance_collection_start', [ - 'database' => $databaseName, - 'collections' => $collections, - ]); - - $client = $this->getMongoClient($taskConfig); - $database = $client->selectDatabase($databaseName); - - // 计算总数（遍历所有集合） - $totalCount = 0; - foreach ($collections as $collectionName) { - $collection = $database->selectCollection($collectionName); - $totalCount += $collection->countDocuments([ - 'loan_amount' => ['$exists' => true, '$ne' => null, '$ne' => ''], - ]); - } - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤7-统计总数】总记录数: {$totalCount}\n"); - - // 计算进度更新间隔（根据总数动态调整） - $updateInterval = $this->calculateProgressUpdateInterval($totalCount); - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤7-统计总数】进度更新间隔: 每 {$updateInterval} 条更新一次\n"); - - // 更新进度：开始采集 - if (!empty($taskId)) { - $this->updateProgress($taskId, [ - 'status' => 'running', - 'start_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - 'total_count' => $totalCount, - 'processed_count' => 0, - 'success_count' => 0, - 'error_count' => 0, - 'percentage' => 0, - ]); - } - - $processedCount = 0; - $successCount = 0; - $errorCount = 0; - $lastUpdateCount = 0; // 记录上次更新的处理数量 - $isCompleted = false; // 标记是否已完成 - - foreach ($collections as $collectionName) { - // 如果已完成，不再处理 - if ($isCompleted) { - break; - } - - try { - $collection = $database->selectCollection($collectionName); - - // 查询有loan_amount的记录 - $cursor = $collection->find([ - 'loan_amount' => ['$exists' => true, '$ne' => null, '$ne' => ''], - ]); - - foreach ($cursor as $doc) { - // 检查任务状态 - if (!empty($taskId) && !$this->checkTaskStatus($taskId)) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ⚠️ 任务已暂停或停止，停止采集\n"); - $isCompleted = true; - break 2; // 跳出两层循环 - } - - // 检查是否已达到总数（在每条处理前检查，避免超出） - if ($totalCount > 0 && $processedCount >= $totalCount) { - $isCompleted = true; - break 2; // 跳出两层循环 - } - - $processedCount++; - try { - $financeData = $this->convertMongoDocumentToArray($doc); - $this->processKrFinanceRecord($financeData, $collectionName, $taskConfig); - $successCount++; - - // 根据更新间隔更新进度 - if (!empty($taskId)) { - // 如果totalCount为0，也要更新进度（显示已处理数量） - if ($totalCount == 0 || ($processedCount - $lastUpdateCount) >= $updateInterval || $processedCount >= $totalCount) { - if ($totalCount > 0) { - $percentage = round(($processedCount / $totalCount) * 100, 2); - } else { - $percentage = 0; // 总数未知时，百分比为0 - } - - // 检查是否达到100% - if ($totalCount > 0 && $processedCount >= $totalCount) { - // 进度达到100%，停止采集并更新状态为已完成 - $this->updateProgress($taskId, [ - 'status' => 'completed', - 'total_count' => $totalCount, - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'percentage' => 100, - 'end_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]); - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ✅ 采集完成，进度已达到100%，已停止采集\n"); - $isCompleted = true; // 标记为已完成 - break 2; // 跳出两层循环 - } else { - $this->updateProgress($taskId, [ - 'total_count' => $totalCount, - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'percentage' => $percentage, - ]); - } - $lastUpdateCount = $processedCount; - } - } - } catch (\Exception $e) { - $errorCount++; - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'processKrFinanceRecord', - 'collection' => $collectionName, - ]); - } - } - } catch (\Exception $e) { - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'collectFromKrFinance', - 'collection' => $collectionName, - ]); - } - } - - // 更新进度：采集完成（如果循环正常结束，也更新状态为已完成） - if (!empty($taskId)) { - $percentage = $totalCount > 0 ? round(($processedCount / $totalCount) * 100, 2) : 100; - // 获取当前任务状态 - $task = $this->taskService->getTask($taskId); - if ($task) { - // 只有在任务状态不是 completed、paused、stopped 时，才更新为 completed - // 如果任务被暂停或停止，不应该更新为 completed - if ($task['status'] === 'completed') { - // 已经是 completed，不需要更新 - } elseif (in_array($task['status'], ['paused', 'stopped'])) { - // 任务被暂停或停止，只更新进度，不更新状态 - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ⚠️ 任务已被暂停或停止，不更新为completed状态\n"); - $this->updateProgress($taskId, [ - 'total_count' => $totalCount, - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'percentage' => $percentage, - ]); - } else { - // 任务正常完成，更新状态为 completed - $this->updateProgress($taskId, [ - 'status' => 'completed', - 'total_count' => $totalCount, - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'percentage' => 100, // 完成时强制设置为100% - 'end_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]); - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ✅ 采集任务完成，状态已更新为completed\n"); - } - } - } - - LoggerHelper::logBusiness('kr_finance_collection_completed', [ - 'total_processed' => $processedCount, - 'total_success' => $successCount, - 'total_error' => $errorCount, - ]); - } - - /** - * 处理KR金融记录数据 - * - * @param array $financeData 金融数据 - * @param string $collectionName 集合名称 - * @param array $taskConfig 任务配置 - * @return void - */ - private function processKrFinanceRecord(array $financeData, string $collectionName, array $taskConfig): void - { - // 1. 提取手机号 - $phoneNumber = $financeData['mobile'] ?? null; - if (empty($phoneNumber)) { - LoggerHelper::logBusiness('consumption_collection_skip_no_phone', [ - 'collection' => $collectionName, - 'reason' => '无法提取手机号', - ]); - return; // 跳过无法提取手机号的记录 - } - - // 2. 字段映射和转换 - $consumeRecord = $this->transformKrFinanceRecord($financeData, $phoneNumber, $collectionName, $taskConfig); - - // 3. 写入消费记录 - $this->saveConsumptionRecord($consumeRecord); - } - - /** - * 转换KR金融记录数据为标准消费记录格式 - * - * @param array $financeData 金融数据 - * @param string $phoneNumber 手机号 - * @param string $collectionName 集合名称 - * @param array $taskConfig 任务配置 - * @return array 标准消费记录数据 - */ - private function transformKrFinanceRecord( - array $financeData, - string $phoneNumber, - string $collectionName, - array $taskConfig - ): array { - // 首先应用字段映射（从任务配置中读取字段映射） - $fieldMappings = $taskConfig['field_mappings'] ?? []; - $mappedData = $this->applyFieldMappings($financeData, $fieldMappings); - - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤9-字段映射】应用字段映射完成，映射字段数: " . count($mappedData) . "\n"); - - // 从映射后的数据中获取字段值 - // 贷款金额作为消费金额：从字段映射中获取 - $loanAmount = $this->parseAmount($mappedData['amount'] ?? '0'); - if ($loanAmount <= 0) { - // 后备方案：从源数据获取（向后兼容） - $loanAmount = $this->parseAmount($financeData['loan_amount'] ?? '0'); - if ($loanAmount <= 0) { - throw new \InvalidArgumentException('贷款金额无效'); - } - } - - // 消费时间：从字段映射中获取 - $consumeTimeStr = $mappedData['consume_time'] ?? null; - if (empty($consumeTimeStr)) { - // 后备方案：从源数据获取（向后兼容） - $consumeTimeStr = $financeData['借款日期'] ?? null; - } - $consumeTime = $this->parseDateTime($consumeTimeStr); - if ($consumeTime === null) { - $consumeTime = new \DateTimeImmutable('now'); - } - - // 门店名称：从字段映射中获取 - $storeName = $mappedData['store_name'] ?? "未知门店"; - - - // 门店ID：通过门店服务获取或创建（即使失败也不影响门店名称的保存） - $storeId = $mappedData['store_id'] ?? null; - if (empty($storeId) && !empty($storeName)) { - try { - $source = $taskConfig['data_source_id'] ?? $taskConfig['name'] ?? 'KR_金融'; - $storeId = $this->storeService->getOrCreateStoreByName($storeName, $source); - } catch (\Throwable $e) { - // 店铺ID获取失败不影响门店名称的保存，只记录日志 - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'transformKrFinanceRecord', - 'message' => '获取店铺ID失败，但会继续保存门店名称', - 'store_name' => $storeName, - ]); - } - } - - // 消费时段 - $consumePeriod = $this->parseConsumePeriod($consumeTime); - - // 币种：从字段映射中获取，默认为CNY - $currency = $mappedData['currency'] ?? 'CNY'; - - // 状态：从字段映射中获取，默认为0（正常） - $status = $mappedData['status'] ?? 0; - - // 支付方式：从字段映射中获取，默认为finance_loan - $paymentMethodCode = $mappedData['payment_method_code'] ?? 'finance_loan'; - - // 支付渠道：从字段映射中获取，默认为金融 - $paymentChannel = $mappedData['payment_channel'] ?? '金融'; - - // 支付状态：从字段映射中获取，默认为0（成功） - $paymentStatus = $mappedData['payment_status'] ?? 0; - - // 消费渠道：从字段映射中获取，默认为线下 - $consumeChannel = $mappedData['consume_channel'] ?? '线下'; - - return [ - 'phone_number' => $phoneNumber, // 传递手机号，让ConsumptionService解析user_id - 'consume_time' => $consumeTime->format('Y-m-d H:i:s'), - 'amount' => $loanAmount, - 'actual_amount' => $loanAmount, // 金融贷款，实际金额等于贷款金额 - 'currency' => $currency, - 'store_id' => $storeId, - 'store_name' => $storeName, // 保存门店名称，用于去重和展示 - 'payment_method_code' => $paymentMethodCode, - 'payment_channel' => $paymentChannel, - 'payment_status' => $paymentStatus, - 'consume_channel' => $consumeChannel, - 'consume_period' => $consumePeriod, - 'is_workday' => $this->isWorkday($consumeTime) ? 1 : 0, - 'status' => $status, - ]; - } - - /** - * 保存消费记录 - * - * @param array $recordData 记录数据 - * @return void - */ - private function saveConsumptionRecord(array $recordData): void - { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤11-保存数据】开始保存消费记录\n"); - // 根据任务配置保存到目标数据源 - $targetDataSourceId = $this->taskConfig['target_data_source_id'] ?? null; - $targetDatabase = $this->taskConfig['target_database'] ?? null; - $targetCollection = $this->taskConfig['target_collection'] ?? 'consumption_records'; - - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤11-保存数据】目标配置: data_source_id={$targetDataSourceId}, database={$targetDatabase}, collection={$targetCollection}\n"); - - if (empty($targetDataSourceId)) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤11-保存数据】使用默认ConsumptionService（向后兼容）\n"); - // 如果没有配置目标数据源，使用默认的 ConsumptionService（向后兼容） - $result = $this->consumptionService->createRecord($recordData); - // 如果返回 null，说明手机号和身份证号都为空，跳过该记录 - if ($result === null) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤11-保存数据】⚠️ 跳过记录：手机号和身份证号都为空\n"); - return; - } - return; - } - - // 连接到目标数据源 - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤12-连接目标数据源】开始查询目标数据源配置: data_source_id={$targetDataSourceId}\n"); - $connectionInfo = $this->connectToTargetDataSource($targetDataSourceId, $targetDatabase); - $targetDataSourceConfig = $connectionInfo['config']; - $dbName = $connectionInfo['dbName']; - $database = $connectionInfo['database']; - - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤12-连接目标数据源】✓ 目标数据源配置查询成功: host={$targetDataSourceConfig['host']}, port={$targetDataSourceConfig['port']}\n"); - - // 根据消费时间确定月份集合（使用 Trait 方法） - $collectionName = $this->getMonthlyCollectionName( - $targetCollection, - $recordData['consume_time'] ?? null - ); - - // 解析用户ID（如果提供了手机号或身份证） - if (empty($recordData['user_id']) && (!empty($recordData['phone_number']) || !empty($recordData['id_card']))) { - // 解析 consume_time 作为查询时间点 - $consumeTime = null; - if (isset($recordData['consume_time'])) { - if (is_string($recordData['consume_time'])) { - $consumeTime = new \DateTimeImmutable($recordData['consume_time']); - } elseif ($recordData['consume_time'] instanceof \MongoDB\BSON\UTCDateTime) { - $timestamp = $recordData['consume_time']->toDateTime()->getTimestamp(); - $consumeTime = new \DateTimeImmutable('@' . $timestamp); - } - } - - $userId = $this->identifierService->resolvePersonId( - $recordData['phone_number'] ?? null, - $recordData['id_card'] ?? null, - $consumeTime - ); - $recordData['user_id'] = $userId; - } - - // 转换时间字段为 MongoDB UTCDateTime（在去重检查前转换，用于查询） - $consumeTimeForQuery = null; - if (isset($recordData['consume_time'])) { - if (is_string($recordData['consume_time'])) { - $consumeTimeForQuery = new \MongoDB\BSON\UTCDateTime(strtotime($recordData['consume_time']) * 1000); - } elseif ($recordData['consume_time'] instanceof \MongoDB\BSON\UTCDateTime) { - $consumeTimeForQuery = $recordData['consume_time']; - } - } - $recordData['consume_time'] = $consumeTimeForQuery ?? new \MongoDB\BSON\UTCDateTime(time() * 1000); - - if (empty($recordData['create_time'])) { - $recordData['create_time'] = new \MongoDB\BSON\UTCDateTime(time() * 1000); - } elseif (is_string($recordData['create_time'])) { - $recordData['create_time'] = new \MongoDB\BSON\UTCDateTime(strtotime($recordData['create_time']) * 1000); - } - - // 写入数据 - $collection = $database->selectCollection($collectionName); - - // 获取门店名称（用于去重） - // 优先使用从源数据映射的门店名称，无论店铺表查询结果如何 - $storeName = $recordData['store_name'] ?? null; - - // 如果源数据中没有 store_name 但有 store_id，尝试从店铺表获取门店名称（作为后备方案） - // 但即使查询失败，也要确保 store_name 字段被保存（可能为 null） - if (empty($storeName) && !empty($recordData['store_id'])) { - try { - $store = $this->storeService->getStoreById($recordData['store_id']); - if ($store && $store->store_name) { - $storeName = $store->store_name; - } - } catch (\Throwable $e) { - // 从店铺表反查失败不影响数据保存，只记录日志 - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'saveConsumptionRecord', - 'message' => '从店铺表反查门店名称失败，将保存null值', - 'store_id' => $recordData['store_id'] ?? null, - ]); - } - } - - // 确保 store_name 字段被保存到 recordData 中（即使为 null 也要保存，保持数据结构一致） - $recordData['store_name'] = $storeName; - - // 基于业务唯一标识检查重复（防止重复插入） - // 方案：使用 store_name + source_order_id 作为唯一标识 - // 注意：order_no 是系统自动生成的（自动递增），不参与去重判断 - $duplicateQuery = null; - $duplicateIdentifier = null; - $sourceOrderId = $recordData['source_order_id'] ?? null; - - if (!empty($storeName) && !empty($sourceOrderId)) { - // 使用门店名称 + 原始订单ID作为唯一标识 - $duplicateQuery = [ - 'store_name' => $storeName, - 'source_order_id' => $sourceOrderId, - ]; - $duplicateIdentifier = "store_name={$storeName}, source_order_id={$sourceOrderId}"; - } - - // 如果找到了唯一标识，检查是否已存在 - if ($duplicateQuery) { - $existingRecord = $collection->findOne($duplicateQuery); - if ($existingRecord) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ⚠️ 记录已存在，跳过插入: {$duplicateIdentifier}, collection={$collectionName}\n"); - LoggerHelper::logBusiness('consumption_record_duplicate_skipped', [ - 'duplicate_identifier' => $duplicateIdentifier, - 'target_collection' => $collectionName, - ]); - return; // 跳过重复记录 - } - } - - // 生成 record_id（如果还没有） - // 使用门店名称 + 原始订单ID生成稳定的 record_id - if (empty($recordData['record_id'])) { - if (!empty($storeName) && !empty($sourceOrderId)) { - // 使用门店名称 + 原始订单ID生成稳定的 record_id - $uniqueKey = "{$storeName}|{$sourceOrderId}"; - $recordData['record_id'] = 'store_source_' . md5($uniqueKey); - } else { - // 如果都没有，生成 UUID（这种情况应该很少） - $recordData['record_id'] = UuidGenerator::uuid4()->toString(); - } - } - - // 生成 order_no（系统自动生成，自动递增） - // 注意：order_no 不参与去重判断，仅用于展示和查询 - // 使用计数器集合来生成唯一的 order_no（在去重检查之后，只有实际插入的记录才生成 order_no） - if (empty($recordData['order_no'])) { - try { - // 使用计数器集合来生成唯一的 order_no - $counterCollection = $database->selectCollection($collectionName . '_counter'); - - // 原子性地递增计数器 - $counterResult = $counterCollection->findOneAndUpdate( - ['_id' => 'order_no'], - ['$inc' => ['seq' => 1], '$setOnInsert' => ['_id' => 'order_no', 'seq' => 1]], - ['upsert' => true, 'returnDocument' => 1] // 1 = RETURN_DOCUMENT_AFTER - ); - - $nextOrderNo = $counterResult['seq'] ?? 1; - $recordData['order_no'] = (string)$nextOrderNo; - } catch (\Throwable $e) { - // 如果计数器操作失败，回退到查询最大值的方案 - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'saveConsumptionRecord', - 'message' => '使用计数器生成order_no失败，回退到查询最大值方案', - ]); - - try { - $maxOrderNo = $collection->findOne( - [], - ['sort' => ['order_no' => -1], 'projection' => ['order_no' => 1]] - ); - $nextOrderNo = 1; - if ($maxOrderNo && isset($maxOrderNo['order_no']) && is_numeric($maxOrderNo['order_no'])) { - $nextOrderNo = (int)$maxOrderNo['order_no'] + 1; - } - $recordData['order_no'] = (string)$nextOrderNo; - } catch (\Throwable $e2) { - // 如果查询也失败，使用时间戳作为备选方案 - $recordData['order_no'] = (string)(time() * 1000 + mt_rand(1000, 9999)); - LoggerHelper::logError($e2, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'saveConsumptionRecord', - 'message' => '查询最大order_no也失败，使用时间戳作为备选', - ]); - } - } - } - - // 格式化输出流水信息 - $timestamp = date('Y-m-d H:i:s'); - $recordId = $recordData['record_id'] ?? 'null'; - $userId = $recordData['user_id'] ?? 'null'; - $amount = $recordData['amount'] ?? 0; - $actualAmount = $recordData['actual_amount'] ?? $amount; - $consumeTime = isset($recordData['consume_time']) && $recordData['consume_time'] instanceof \MongoDB\BSON\UTCDateTime - ? $recordData['consume_time']->toDateTime()->format('Y-m-d H:i:s') - : ($recordData['consume_time'] ?? 'null'); - $storeId = $recordData['store_id'] ?? 'null'; - $phoneNumber = $recordData['phone_number'] ?? 'null'; - - // 确保 storeName 变量已定义（从 recordData 中获取，如果之前已赋值） - $storeNameOutput = $recordData['store_name'] ?? 'null'; - - // 输出详细的插入流水信息（输出到终端） - // $output = "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n" - // . "📝 [{$timestamp}] 消费记录插入流水\n" - // . " ├─ 记录ID: {$recordId}\n" - // . " ├─ 用户ID: {$userId}\n" - // . " ├─ 手机号: {$phoneNumber}\n" - // . " ├─ 消费时间: {$consumeTime}\n" - // . " ├─ 消费金额: ¥" . number_format($amount, 2) . "\n" - // . " ├─ 实际金额: ¥" . number_format($actualAmount, 2) . "\n" - // . " ├─ 店铺ID: {$storeId}\n" - // . " ├─ 门店名称: {$storeNameOutput}\n" - // . " ├─ 目标数据库: {$dbName}\n" - // . " └─ 目标集合: {$collectionName}\n"; - - // // \Workerman\Worker::safeEcho($output); - - $result = $collection->insertOne($recordData); - $insertedId = $result->getInsertedId(); - - $successOutput = " ✅ 插入成功 | MongoDB ID: " . (is_object($insertedId) ? (string)$insertedId : json_encode($insertedId)) . "\n" - . "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n"; - - // \Workerman\Worker::safeEcho($successOutput); - - LoggerHelper::logBusiness('consumption_record_saved_to_target', [ - 'target_data_source_id' => $targetDataSourceId, - 'target_database' => $dbName, - 'target_collection' => $collectionName, - 'record_id' => $recordData['record_id'], - 'inserted_id' => (string)$insertedId, - ]); - - // 更新用户统计信息（使用默认连接，因为用户数据在主数据库） - // 如果身份证和手机号都是空的（没有user_id），则不更新用户主表 - if (empty($recordData['user_id'])) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ⚠️ 身份证和手机号都为空，跳过用户主表更新\n"); - LoggerHelper::logBusiness('consumption_record_skip_user_update_no_identifier', [ - 'record_id' => $recordData['record_id'] ?? null, - 'phone_number' => $recordData['phone_number'] ?? null, - 'id_card' => isset($recordData['id_card']) ? '***' : null, // 不记录敏感信息 - ]); - return; - } - - try { - $userProfileRepo = new \app\repository\UserProfileRepository(); - $consumeTime = isset($recordData['consume_time']) && $recordData['consume_time'] instanceof \MongoDB\BSON\UTCDateTime - ? \DateTimeImmutable::createFromMutable($recordData['consume_time']->toDateTime()) - : new \DateTimeImmutable(); - $user = $userProfileRepo->increaseStats( - $recordData['user_id'], - $recordData['actual_amount'] ?? $recordData['amount'] ?? 0, - $consumeTime - ); - } catch (\Exception $e) { - // 更新用户统计失败不影响数据保存，只记录日志 - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'saveConsumptionRecord', - 'message' => '更新用户统计失败', - ]); - } - } - - /** - * 应用字段映射 - * - * @param array $sourceData 源数据 - * @param array $fieldMappings 字段映射配置 - * @return array 映射后的数据 - */ - private function applyFieldMappings(array $sourceData, array $fieldMappings): array - { - $mappedData = []; - - foreach ($fieldMappings as $mapping) { - $sourceField = $mapping['source_field'] ?? ''; - $targetField = $mapping['target_field'] ?? ''; - $transform = $mapping['transform'] ?? null; - - // 如果源字段或目标字段为空，跳过该映射 - if (empty($sourceField) || empty($targetField)) { - continue; - } - - // 从源数据中获取值（支持嵌套字段，如 "user.name"） - $value = $this->getNestedValue($sourceData, $sourceField); - - // 调试：输出store_name字段的映射详情 - if ($targetField === 'store_name') { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 【步骤9-字段映射】字段映射详情: target={$targetField}, source={$sourceField}, value=" . ($value ?? 'null') . "\n"); - } - - // 应用转换函数 - if ($transform && is_callable($transform)) { - $value = $transform($value); - } elseif ($transform && is_string($transform)) { - $value = $this->applyTransform($value, $transform); - } - - $mappedData[$targetField] = $value; - } - - return $mappedData; - } - - /** - * 获取嵌套字段值 - * - * @param array $data 数据 - * @param string $fieldPath 字段路径（支持嵌套，如 "user.name"） - * @return mixed 字段值 - */ - private function getNestedValue(array $data, string $fieldPath) - { - $parts = explode('.', $fieldPath); - $value = $data; - - foreach ($parts as $part) { - if (is_array($value) && isset($value[$part])) { - $value = $value[$part]; - } elseif (is_object($value) && isset($value->$part)) { - $value = $value->$part; - } else { - return null; - } - } - - return $value; - } - - /** - * 应用转换函数 - * - * @param mixed $value 原始值 - * @param string $transform 转换函数名称 - * @return mixed 转换后的值 - */ - private function applyTransform($value, string $transform) - { - switch ($transform) { - case 'parse_amount': - return $this->parseAmount($value); - case 'parse_datetime': - return is_string($value) ? $value : ($value instanceof \DateTimeImmutable ? $value->format('Y-m-d H:i:s') : (string)$value); - case 'parse_phone': - return $this->extractPhoneNumberFromValue($value); - default: - return $value; - } - } - - /** - * 从值中提取手机号 - * - * @param mixed $value 值 - * @return string|null 手机号 - */ - private function extractPhoneNumberFromValue($value): ?string - { - if (empty($value)) { - return null; - } - $phone = trim((string)$value); - - // 先过滤非数字字符 - $cleanedPhone = $this->filterPhoneNumber($phone); - - // 验证过滤后的手机号 - if ($this->isValidPhone($cleanedPhone)) { - // 返回过滤后的手机号 - return $cleanedPhone; - } - - return null; - } - - /** - * 从订单数据中提取手机号 - * - * @param array $orderData 订单数据 - * @return string|null 手机号 - */ - private function extractPhoneNumber(array $orderData): ?string - { - // 优先使用支付宝账号（通常是手机号） - if (!empty($orderData['买家支付宝账号'])) { - $phone = trim($orderData['买家支付宝账号']); - // 先过滤非数字字符 - $cleanedPhone = $this->filterPhoneNumber($phone); - if (!empty($cleanedPhone) && $this->isValidPhone($cleanedPhone)) { - return $cleanedPhone; - } - } - - // 其次使用联系电话 - if (!empty($orderData['联系电话'])) { - $phone = trim($orderData['联系电话']); - // 先过滤非数字字符 - $cleanedPhone = $this->filterPhoneNumber($phone); - if (!empty($cleanedPhone) && $this->isValidPhone($cleanedPhone)) { - return $cleanedPhone; - } - } - - return null; - } - - /** - * 提取手机号（订单数据专用） - * - * 注意：这个方法保留在此类中，因为它处理的是订单数据的特定字段 - * 通用的手机号提取逻辑在 Trait 中 - */ - - - /** - * 解析支付方式 - * - * @param string $paymentDetail 支付详情 - * @return string 支付方式编码 - */ - private function parsePaymentMethod(string $paymentDetail): string - { - $detail = strtolower($paymentDetail); - - if (strpos($detail, '支付宝') !== false || strpos($detail, 'alipay') !== false) { - return 'alipay'; - } - if (strpos($detail, '微信') !== false || strpos($detail, 'wechat') !== false || strpos($detail, 'weixin') !== false) { - return 'wechat'; - } - if (strpos($detail, '银行卡') !== false || strpos($detail, 'card') !== false) { - return 'bank_card'; - } - - return 'other'; - } - - /** - * 解析支付状态 - * - * @param string $orderStatus 订单状态 - * @return int 支付状态：0-成功，1-失败，2-退款 - */ - private function parsePaymentStatus(string $orderStatus): int - { - $status = strtolower($orderStatus); - - if (strpos($status, '退款') !== false || strpos($status, 'refund') !== false) { - return 2; // 退款 - } - if (strpos($status, '失败') !== false || strpos($status, 'fail') !== false) { - return 1; // 失败 - } - - return 0; // 成功 - } - - /** - * 解析消费渠道 - * - * @param string $isMobileOrder 是否手机订单 - * @return string 消费渠道 - */ - private function parseConsumeChannel(string $isMobileOrder): string - { - if (strpos($isMobileOrder, '是') !== false || strpos(strtolower($isMobileOrder), 'true') !== false || $isMobileOrder === '1') { - return '线上_移动端'; - } - return '线上_PC端'; - } - - /** - * 解析消费时段 - * - * @param \DateTimeImmutable $dateTime 日期时间 - * @return string 消费时段 - */ - private function parseConsumePeriod(\DateTimeImmutable $dateTime): string - { - $hour = (int)$dateTime->format('H'); - - if ($hour >= 6 && $hour < 12) { - return '上午'; - } elseif ($hour >= 12 && $hour < 14) { - return '中午'; - } elseif ($hour >= 14 && $hour < 18) { - return '下午'; - } elseif ($hour >= 18 && $hour < 22) { - return '晚上'; - } else { - return '深夜'; - } - } - - /** - * 判断是否为工作日 - * - * @param \DateTimeImmutable $dateTime 日期时间 - * @return bool 是否为工作日 - */ - private function isWorkday(\DateTimeImmutable $dateTime): bool - { - $dayOfWeek = (int)$dateTime->format('w'); // 0=Sunday, 6=Saturday - return $dayOfWeek >= 1 && $dayOfWeek <= 5; - } - - - /** - * 从集合名称中提取用户等级 - * - * @param string $collectionName 集合名称 - * @return string 用户等级 - */ - private function extractUserLevel(string $collectionName): string - { - if (preg_match('/[ABCEDS]级用户/', $collectionName, $matches)) { - return $matches[0]; - } - return '未知'; - } - - - /** - * 实时监听KR商城集合变化 - * - * @param array $taskConfig 任务配置 - * @return void - */ - private function watchKrMallCollection(array $taskConfig): void - { - $databaseName = $taskConfig['database'] ?? 'KR_商城'; - $collectionName = $taskConfig['collection'] ?? '21年贝蒂喜订单整合'; - - LoggerHelper::logBusiness('kr_mall_realtime_watch_start', [ - 'database' => $databaseName, - 'collection' => $collectionName, - ]); - - $client = $this->getMongoClient($taskConfig); - $database = $client->selectDatabase($databaseName); - $collection = $database->selectCollection($collectionName); - - // 创建Change Stream监听集合变化 - $changeStream = $collection->watch( - [], - [ - 'fullDocument' => 'updateLookup', - 'batchSize' => 100, - 'maxAwaitTimeMS' => 1000, - ] - ); - - LoggerHelper::logBusiness('kr_mall_realtime_watch_ready', [ - 'database' => $databaseName, - 'collection' => $collectionName, - ]); - - // 处理变更事件 - foreach ($changeStream as $change) { - try { - $operationType = $change['operationType'] ?? ''; - - // 只处理插入和更新操作 - if ($operationType === 'insert' || $operationType === 'update') { - $document = $change['fullDocument'] ?? null; - - if ($document === null) { - // 如果是更新操作但没有fullDocument，需要查询完整文档 - if ($operationType === 'update') { - $documentId = $change['documentKey']['_id'] ?? null; - if ($documentId !== null) { - $document = $collection->findOne(['_id' => $documentId]); - } - } - } - - if ($document !== null) { - $orderData = $this->convertMongoDocumentToArray($document); - $orderNo = $orderData['订单编号'] ?? 'unknown'; - - try { - // \Workerman\Worker::safeEcho(" 🔔 [实时监听] 检测到变更: operation={$operationType}, 订单编号={$orderNo}\n"); - $this->processKrMallOrder($orderData, $taskConfig); - - LoggerHelper::logBusiness('kr_mall_realtime_record_processed', [ - 'operation' => $operationType, - 'order_no' => $orderNo, - ]); - } catch (\Exception $e) { - $errorMsg = $e->getMessage(); - // \Workerman\Worker::safeEcho(" ❌ [实时监听] 处理失败: 订单编号={$orderNo}, 错误={$errorMsg}\n"); - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'processKrMallOrder_realtime', - 'operation' => $operationType, - 'order_no' => $orderNo, - ]); - } - } - } - } catch (\Exception $e) { - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'watchKrMallCollection', - 'change' => $change, - ]); - } - } - } - - /** - * 实时监听KR金融集合变化 - * - * @param array $taskConfig 任务配置 - * @return void - */ - private function watchKrFinanceCollections(array $taskConfig): void - { - $databaseName = $taskConfig['database'] ?? 'KR'; - $collections = $taskConfig['collections'] ?? [ - '金融客户_厦门_A级用户', - '金融客户_厦门_B级用户', - '金融客户_厦门_C级用户', - '金融客户_厦门_D级用户', - '金融客户_厦门_E级用户', - '厦门用户资产2025年9月_优化版', - ]; - - LoggerHelper::logBusiness('kr_finance_realtime_watch_start', [ - 'database' => $databaseName, - 'collections' => $collections, - ]); - - $client = $this->getMongoClient($taskConfig); - $database = $client->selectDatabase($databaseName); - - // 使用数据库级别的Change Stream监听所有集合 - $changeStream = $database->watch( - [], - [ - 'fullDocument' => 'updateLookup', - 'batchSize' => 100, - 'maxAwaitTimeMS' => 1000, - ] - ); - - LoggerHelper::logBusiness('kr_finance_realtime_watch_ready', [ - 'database' => $databaseName, - ]); - - // 处理变更事件 - foreach ($changeStream as $change) { - try { - $collectionName = $change['ns']['coll'] ?? ''; - - // 只处理配置的集合 - if (!in_array($collectionName, $collections)) { - continue; - } - - $operationType = $change['operationType'] ?? ''; - - // 只处理插入和更新操作，且必须有loan_amount字段 - if ($operationType === 'insert' || $operationType === 'update') { - $document = $change['fullDocument'] ?? null; - - if ($document === null && $operationType === 'update') { - // 如果是更新操作但没有fullDocument，需要查询完整文档 - $documentId = $change['documentKey']['_id'] ?? null; - if ($documentId !== null) { - $collection = $database->selectCollection($collectionName); - $document = $collection->findOne(['_id' => $documentId]); - } - } - - if ($document !== null) { - $docArray = $this->convertMongoDocumentToArray($document); - - // 检查是否有loan_amount字段 - if (isset($docArray['loan_amount']) && !empty($docArray['loan_amount'])) { - try { - $this->processKrFinanceRecord($docArray, $collectionName, $taskConfig); - - LoggerHelper::logBusiness('kr_finance_realtime_record_processed', [ - 'operation' => $operationType, - 'collection' => $collectionName, - 'mobile' => $docArray['mobile'] ?? 'unknown', - ]); - } catch (\Exception $e) { - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'processKrFinanceRecord_realtime', - 'operation' => $operationType, - 'collection' => $collectionName, - ]); - } - } - } - } - } catch (\Exception $e) { - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'watchKrFinanceCollections', - 'change' => $change, - ]); - } - } - } - - /** - * 检查任务状态（是否应该继续执行） - * - * @param string $taskId 任务ID - * @return bool true=继续执行, false=暂停/停止 - */ - private function checkTaskStatus(string $taskId): bool - { - // 检查Redis标志 - if (\app\utils\RedisHelper::exists("data_collection_task:{$taskId}:pause")) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 检测到暂停标志，任务 {$taskId} 暂停\n"); - return false; - } - if (\app\utils\RedisHelper::exists("data_collection_task:{$taskId}:stop")) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 检测到停止标志，任务 {$taskId} 停止\n"); - return false; - } - - // 检查数据库状态 - $task = $this->taskService->getTask($taskId); - if ($task && in_array($task['status'], ['paused', 'stopped', 'error'])) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 检测到任务状态为 {$task['status']}，任务 {$taskId} 停止\n"); - return false; - } - - return true; - } - - /** - * 根据总记录数计算合适的进度更新间隔 - * - * @param int $totalCount 总记录数 - * @return int 更新间隔（每处理多少条记录更新一次） - */ - private function calculateProgressUpdateInterval(int $totalCount): int - { - // 根据总数动态调整更新间隔，确保既不会太频繁也不会太慢 - // 策略：大约每1%更新一次，但限制在合理范围内 - - if ($totalCount <= 0) { - return 50; // 默认50条 - } - - // 计算1%的数量 - $onePercent = max(1, (int)($totalCount * 0.01)); - - // 根据总数范围调整： - // - 小于1000条：每50条更新（保证至少更新20次） - // - 1000-10000条：每1%更新（约10-100条） - // - 10000-100000条：每1%更新（约100-1000条） - // - 100000-1000000条：每1%更新（约1000-10000条），但最多5000条 - // - 大于1000000条：每5000条更新（避免更新太频繁） - - if ($totalCount < 1000) { - return 50; - } elseif ($totalCount < 10000) { - return max(50, min(500, $onePercent)); - } elseif ($totalCount < 100000) { - return max(100, min(1000, $onePercent)); - } elseif ($totalCount < 1000000) { - return max(500, min(5000, $onePercent)); - } else { - return 5000; // 大数据量固定5000条更新一次 - } - } - - /** - * 更新任务进度 - * - * @param string $taskId 任务ID - * @param array $progress 进度信息（可以包含status字段来更新任务状态） - * @return void - */ - private function updateProgress(string $taskId, array $progress): void - { - try { - $task = $this->taskService->getTask($taskId); - if (!$task) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ⚠️ 更新进度失败：任务不存在 task_id={$taskId}\n"); - return; - } - - $currentProgress = $task['progress'] ?? []; - - // 保护已完成任务的进度：如果任务已完成且百分比为100%，且没有明确指定要更新百分比，则保护当前进度 - $isCompleted = $task['status'] === 'completed'; - $currentPercentage = $currentProgress['percentage'] ?? 0; - $shouldProtectProgress = $isCompleted && $currentPercentage === 100 && !isset($progress['percentage']); - - // 检查是否需要更新任务状态 - $updateTaskStatus = false; - $newStatus = null; - if (isset($progress['status'])) { - $newStatus = $progress['status']; - unset($progress['status']); // 从progress中移除，单独处理 - - // 如果当前任务状态是 completed，不允许再更新为 running（防止循环） - // 只有用户手动重新启动任务时（通过 startTask），才会从 completed 变为 running - if ($newStatus === 'running' && $task['status'] === 'completed') { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ⚠️ 任务已完成，不允许更新为 running，跳过状态更新\n"); - LoggerHelper::logBusiness('task_status_update_skipped_completed', [ - 'task_id' => $taskId, - 'current_status' => $task['status'], - 'attempted_status' => $newStatus, - ]); - } else { - $updateTaskStatus = true; - } - } - - // 合并进度信息 - foreach ($progress as $key => $value) { - $currentProgress[$key] = $value; - } - - // 确保 percentage 字段存在且正确计算（基于已采集条数/总条数） - if (isset($currentProgress['processed_count']) && isset($currentProgress['total_count'])) { - if ($currentProgress['total_count'] > 0) { - // 进度 = 已采集条数 / 总条数 * 100 - $calculatedPercentage = round( - ($currentProgress['processed_count'] / $currentProgress['total_count']) * 100, - 2 - ); - // 确保不超过100% - $calculatedPercentage = min(100, $calculatedPercentage); - - // 如果任务已完成且当前百分比为100%，且计算出的百分比小于100%，保持100% - // 否则使用计算出的百分比 - if ($isCompleted && $currentPercentage === 100 && $calculatedPercentage < 100) { - $currentProgress['percentage'] = 100; // 保护已完成任务的100%进度 - } else { - $currentProgress['percentage'] = $calculatedPercentage; - } - } else { - // 如果 total_count 为 0，但任务已完成且百分比为100%，保持100% - // 否则设置为0（表示重新开始） - if ($isCompleted && $currentPercentage === 100) { - $currentProgress['percentage'] = 100; // 保持100% - } else { - $currentProgress['percentage'] = 0; - } - } - } elseif ($shouldProtectProgress) { - // 如果没有传入 processed_count 或 total_count，但应该保护进度，保持当前百分比 - $currentProgress['percentage'] = $currentPercentage; - } elseif (!isset($currentProgress['percentage'])) { - // 如果没有传入 percentage，且不需要保护，保持当前百分比（如果存在） - $currentProgress['percentage'] = $currentPercentage; - } - - // 输出进度更新日志（用于调试） - $processedCount = $currentProgress['processed_count'] ?? 0; - $totalCount = $currentProgress['total_count'] ?? 0; - $percentage = $currentProgress['percentage'] ?? 0; - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] 📊 更新进度: processed={$processedCount}/{$totalCount}, percentage={$percentage}%\n"); - - // 更新进度到数据库 - $result = $this->taskService->updateProgress($taskId, $currentProgress); - if ($result) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ✅ 进度已保存到数据库\n"); - } else { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ⚠️ 进度保存到数据库失败\n"); - } - - // 如果指定了状态，更新任务状态（例如：completed） - if ($updateTaskStatus && $newStatus !== null) { - $this->taskService->updateTask($taskId, ['status' => $newStatus]); - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ✅ 任务状态已更新为: {$newStatus}\n"); - } - } catch (\Exception $e) { - // \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ❌ 更新进度异常: " . $e->getMessage() . "\n"); - LoggerHelper::logError($e, [ - 'component' => 'ConsumptionCollectionHandler', - 'action' => 'updateProgress', - 'task_id' => $taskId, - ]); - } - } -} - diff --git a/Moncter/Moncter/app/service/DataCollection/Handler/DatabaseSyncHandler.php b/Moncter/Moncter/app/service/DataCollection/Handler/DatabaseSyncHandler.php deleted file mode 100644 index 5df5943a..00000000 --- a/Moncter/Moncter/app/service/DataCollection/Handler/DatabaseSyncHandler.php +++ /dev/null @@ -1,427 +0,0 @@ - $taskConfig 任务配置 - * @return void - */ - public function collect(DataSourceAdapterInterface $adapter, array $taskConfig): void - { - $this->taskConfig = $taskConfig; - $taskId = $taskConfig['task_id'] ?? ''; - $taskName = $taskConfig['name'] ?? '数据库同步'; - - LoggerHelper::logBusiness('database_sync_collection_started', [ - 'task_id' => $taskId, - 'task_name' => $taskName, - ]); - // 控制台直接输出一条提示，方便在启动时观察数据库同步任务是否真正开始执行 - error_log("[DatabaseSyncHandler] 数据库同步任务已启动：task_id={$taskId}, task_name={$taskName}"); - - try { - // 创建 DatabaseSyncService（使用任务配置中的源和目标数据源） - $this->syncService = $this->createSyncService($taskConfig); - - // 获取要同步的数据库列表 - $databases = $this->getDatabasesToSync($taskConfig); - - if (empty($databases)) { - LoggerHelper::logBusiness('database_sync_no_databases', [ - 'task_id' => $taskId, - 'message' => '没有找到要同步的数据库', - ]); - return; - } - - // 启动进度日志定时器（定期输出同步进度） - $this->startProgressTimer($taskConfig); - - // 是否执行全量同步（从业务配置中获取） - $businessConfig = $this->getBusinessConfig(); - $fullSyncEnabled = $businessConfig['change_stream']['full_sync_on_start'] ?? false; - - if ($fullSyncEnabled) { - // 执行全量同步 - $this->performFullSync($databases, $taskConfig); - } - - // 启动增量同步监听（Change Streams） - $this->startIncrementalSync($databases, $taskConfig); - - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DatabaseSyncHandler', - 'action' => 'collect', - 'task_id' => $taskId, - ]); - throw $e; - } - } - - /** - * 获取业务配置（从独立配置文件或使用默认值） - * - * @return array 业务配置 - */ - private function getBusinessConfig(): array - { - // 可以从独立配置文件读取，或使用默认值 - // 这里使用默认值，业务逻辑统一在代码中管理 - return [ - // 数据库同步配置 - 'databases' => [], // 空数组表示同步所有数据库 - 'exclude_databases' => ['admin', 'local', 'config'], // 排除的系统数据库 - 'exclude_collections' => ['system.profile', 'system.js'], // 排除的系统集合 - - // Change Streams 配置 - 'change_stream' => [ - 'batch_size' => 100, - 'max_await_time_ms' => 1000, - 'full_sync_on_start' => true, // 首次启动时是否执行全量同步 - 'full_sync_batch_size' => 1000, - ], - - // 重试配置 - 'retry' => [ - 'max_connect_retries' => 10, - 'retry_interval' => 5, - 'max_sync_retries' => 3, - 'sync_retry_interval' => 2, - ], - - // 性能配置 - 'performance' => [ - 'concurrent_databases' => 5, - 'concurrent_collections' => 10, - 'batch_write_size' => 5000, - // 为了让断点续传逻辑简单可靠，这里关闭集合级并行同步 - // 后续如果需要再做更复杂的分片断点策略，可以重新打开 - 'enable_parallel_sync' => false, - 'max_parallel_tasks_per_collection' => 4, - 'documents_per_task' => 100000, - ], - - // 监控配置 - 'monitoring' => [ - 'log_sync' => true, - 'log_detail' => false, - 'stats_interval' => 10, // 每10秒输出一次进度日志 - ], - ]; - } - - /** - * 创建 DatabaseSyncService 实例 - * - * @param array $taskConfig 任务配置 - * @return DatabaseSyncService - */ - private function createSyncService(array $taskConfig): DatabaseSyncService - { - // 从数据库获取源和目标数据源配置 - $dataSourceService = new DataSourceService(new DataSourceRepository()); - $sourceDataSourceId = $taskConfig['source_data_source'] ?? 'kr_mongodb'; - $targetDataSourceId = $taskConfig['target_data_source'] ?? 'sync_mongodb'; - - $sourceConfig = $dataSourceService->getDataSourceConfigById($sourceDataSourceId); - $targetConfig = $dataSourceService->getDataSourceConfigById($targetDataSourceId); - - if (empty($sourceConfig) || empty($targetConfig)) { - throw new \InvalidArgumentException("数据源配置不存在: source={$sourceDataSourceId}, target={$targetDataSourceId}"); - } - - // 获取业务配置（统一在代码中管理） - $businessConfig = $this->getBusinessConfig(); - - // 构建同步配置 - $syncConfig = [ - 'enabled' => true, - 'source' => [ - 'host' => $sourceConfig['host'], - 'port' => $sourceConfig['port'], - 'username' => $sourceConfig['username'] ?? '', - 'password' => $sourceConfig['password'] ?? '', - 'auth_source' => $sourceConfig['auth_source'] ?? 'admin', - 'options' => array_merge([ - 'connectTimeoutMS' => 10000, - 'socketTimeoutMS' => 30000, - 'serverSelectionTimeoutMS' => 10000, - 'heartbeatFrequencyMS' => 10000, - ], $sourceConfig['options'] ?? []), - ], - 'target' => [ - 'host' => $targetConfig['host'], - 'port' => $targetConfig['port'], - 'username' => $targetConfig['username'] ?? '', - 'password' => $targetConfig['password'] ?? '', - 'auth_source' => $targetConfig['auth_source'] ?? 'admin', - 'options' => array_merge([ - 'connectTimeoutMS' => 10000, - 'socketTimeoutMS' => 30000, - 'serverSelectionTimeoutMS' => 10000, - ], $targetConfig['options'] ?? []), - ], - 'sync' => [ - 'databases' => $businessConfig['databases'], - 'exclude_databases' => $businessConfig['exclude_databases'], - 'exclude_collections' => $businessConfig['exclude_collections'], - 'change_stream' => $businessConfig['change_stream'], - 'retry' => $businessConfig['retry'], - 'performance' => $businessConfig['performance'], - ], - 'monitoring' => $businessConfig['monitoring'], - ]; - - // 直接传递配置给 DatabaseSyncService 构造函数 - return new DatabaseSyncService($syncConfig); - } - - /** - * 获取要同步的数据库列表 - * - * @param array $taskConfig 任务配置 - * @return array 数据库名称列表 - */ - private function getDatabasesToSync(array $taskConfig): array - { - return $this->syncService->getDatabasesToSync(); - } - - /** - * 执行全量同步（支持多进程数据库级并行） - * - * @param array $databases 数据库列表 - * @param array $taskConfig 任务配置 - * @return void - */ - private function performFullSync(array $databases, array $taskConfig): void - { - // 获取 Worker 信息（用于多进程分配） - $workerId = $taskConfig['worker_id'] ?? 0; - $workerCount = $taskConfig['worker_count'] ?? 1; - - // 分配数据库给当前 Worker（负载均衡算法） - $assignedDatabases = $this->assignDatabasesToWorker($databases, $workerId, $workerCount); - - LoggerHelper::logBusiness('database_sync_full_sync_start', [ - 'worker_id' => $workerId, - 'worker_count' => $workerCount, - 'total_databases' => count($databases), - 'assigned_databases' => $assignedDatabases, - 'assigned_count' => count($assignedDatabases), - ]); - - foreach ($assignedDatabases as $databaseName) { - try { - $this->syncService->fullSyncDatabase($databaseName); - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DatabaseSyncHandler', - 'action' => 'performFullSync', - 'database' => $databaseName, - 'worker_id' => $workerId, - ]); - // 继续同步其他数据库 - } - } - - LoggerHelper::logBusiness('database_sync_full_sync_completed', [ - 'worker_id' => $workerId, - 'databases' => $assignedDatabases, - ]); - } - - /** - * 分配数据库给当前 Worker（负载均衡算法） - * - * 策略： - * 1. 按数据库大小排序（小库优先，提升完成感） - * 2. 使用贪心算法：每次分配给当前负载最小的 Worker - * 3. 考虑 Worker 当前处理的数据库数量 - * - * @param array $databases 数据库列表（已按大小排序） - * @param int $workerId 当前 Worker ID - * @param int $workerCount Worker 总数 - * @return array 分配给当前 Worker 的数据库列表 - */ - private function assignDatabasesToWorker(array $databases, int $workerId, int $workerCount): array - { - // 如果只有一个 Worker，返回所有数据库 - if ($workerCount <= 1) { - return $databases; - } - - // 方案A：简单取模分配（快速实现） - // 适用于数据库数量较多且大小相近的场景 - $assignedDatabases = []; - foreach ($databases as $index => $databaseName) { - if ($index % $workerCount === $workerId) { - $assignedDatabases[] = $databaseName; - } - } - - // 方案B：负载均衡分配（推荐，但需要数据库大小信息） - // 由于 getDatabasesToSync 已经按大小排序，简单取模即可实现较好的负载均衡 - // 如果后续需要更精确的负载均衡，可以从 DatabaseSyncService 获取数据库大小信息 - - return $assignedDatabases; - } - - /** - * 启动增量同步监听（支持多进程数据库级并行） - * - * @param array $databases 数据库列表 - * @param array $taskConfig 任务配置 - * @return void - */ - private function startIncrementalSync(array $databases, array $taskConfig): void - { - // 获取 Worker 信息（用于多进程分配） - $workerId = $taskConfig['worker_id'] ?? 0; - $workerCount = $taskConfig['worker_count'] ?? 1; - - // 分配数据库给当前 Worker（与全量同步使用相同的分配策略） - $assignedDatabases = $this->assignDatabasesToWorker($databases, $workerId, $workerCount); - - LoggerHelper::logBusiness('database_sync_incremental_sync_start', [ - 'worker_id' => $workerId, - 'worker_count' => $workerCount, - 'total_databases' => count($databases), - 'assigned_databases' => $assignedDatabases, - 'assigned_count' => count($assignedDatabases), - ]); - - // 为分配给当前 Worker 的数据库启动监听（在后台进程中） - foreach ($assignedDatabases as $databaseName) { - // 使用 Timer 在后台启动监听，避免阻塞 - \Workerman\Timer::add(0, function () use ($databaseName) { - try { - $this->syncService->watchDatabase($databaseName); - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DatabaseSyncHandler', - 'action' => 'startIncrementalSync', - 'database' => $databaseName, - ]); - - // 重试逻辑（从业务配置中获取） - $businessConfig = $this->getBusinessConfig(); - $retryConfig = $businessConfig['retry'] ?? []; - $maxRetries = $retryConfig['max_connect_retries'] ?? 10; - $retryInterval = $retryConfig['retry_interval'] ?? 5; - - static $retryCount = []; - if (!isset($retryCount[$databaseName])) { - $retryCount[$databaseName] = 0; - } - - if ($retryCount[$databaseName] < $maxRetries) { - $retryCount[$databaseName]++; - \Workerman\Timer::add($retryInterval, function () use ($databaseName) { - $this->startIncrementalSync([$databaseName], $this->taskConfig); - }, [], false); - } - } - }, [], false); - } - } - - /** - * 启动进度日志定时器 - * - * @param array $taskConfig 任务配置 - * @return void - */ - private function startProgressTimer(array $taskConfig): void - { - $businessConfig = $this->getBusinessConfig(); - $statsInterval = $businessConfig['monitoring']['stats_interval'] ?? 10; // 默认10秒输出一次进度 - - // 使用 Workerman Timer 定期输出进度 - $this->progressTimerId = \Workerman\Timer::add($statsInterval, function () use ($taskConfig) { - try { - // 重新加载最新进度（从文件读取） - $this->syncService->loadProgress(); - $progress = $this->syncService->getProgress(); - $stats = $this->syncService->getStats(); - - // 输出格式化的进度信息 - $progressInfo = [ - 'task_id' => $taskConfig['task_id'] ?? '', - 'task_name' => $taskConfig['name'] ?? '数据库同步', - 'status' => $progress['status'], - 'progress_percent' => $progress['progress_percent'] . '%', - 'current_database' => $progress['current_database'] ?? '无', - 'current_collection' => $progress['current_collection'] ?? '无', - 'databases' => "{$progress['databases']['completed']}/{$progress['databases']['total']}", - 'collections' => "{$progress['collections']['completed']}/{$progress['collections']['total']}", - 'documents' => "{$progress['documents']['processed']}/{$progress['documents']['total']}", - 'documents_inserted' => $stats['documents_inserted'], - 'documents_updated' => $stats['documents_updated'], - 'documents_deleted' => $stats['documents_deleted'], - 'errors' => $stats['errors'], - 'elapsed_time' => round($progress['time']['elapsed_seconds'], 2) . 's', - 'estimated_remaining' => $progress['time']['estimated_remaining_seconds'] - ? round($progress['time']['estimated_remaining_seconds'], 2) . 's' - : '计算中...', - ]; - - // 输出到日志 - LoggerHelper::logBusiness('database_sync_progress_report', $progressInfo); - - // 如果状态是错误，输出错误信息 - if ($progress['status'] === 'error' && isset($progress['last_error'])) { - LoggerHelper::logBusiness('database_sync_error_info', [ - 'error_message' => $progress['last_error']['message'] ?? '未知错误', - 'error_database' => $progress['error_database'] ?? '未知', - 'error_collection' => $progress['last_error']['collection'] ?? '未知', - ]); - } - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DatabaseSyncHandler', - 'action' => 'startProgressTimer', - ]); - } - }); - } - - /** - * 停止进度日志定时器 - * - * @return void - */ - public function stopProgressTimer(): void - { - if ($this->progressTimerId > 0) { - \Workerman\Timer::del($this->progressTimerId); - $this->progressTimerId = 0; - } - } -} - diff --git a/Moncter/Moncter/app/service/DataCollection/Handler/GenericCollectionHandler.php b/Moncter/Moncter/app/service/DataCollection/Handler/GenericCollectionHandler.php deleted file mode 100644 index 02791913..00000000 --- a/Moncter/Moncter/app/service/DataCollection/Handler/GenericCollectionHandler.php +++ /dev/null @@ -1,1360 +0,0 @@ -taskService = new DataCollectionTaskService( - new \app\repository\DataCollectionTaskRepository() - ); - } - - /** - * 采集数据 - * - * @param \app\service\DataSource\DataSourceAdapterInterface $adapter 数据源适配器 - * @param array $taskConfig 任务配置 - * @return void - */ - public function collect($adapter, array $taskConfig): void - { - $this->taskConfig = $taskConfig; - $taskId = $taskConfig['task_id'] ?? ''; - $taskName = $taskConfig['name'] ?? '通用采集任务'; - $mode = $taskConfig['mode'] ?? 'batch'; // batch: 批量采集, realtime: 实时监听 - - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤5-Handler开始】任务ID={$taskId}, 任务名称={$taskName}, 模式={$mode}\n"); - LoggerHelper::logBusiness('generic_collection_started', [ - 'task_id' => $taskId, - 'task_name' => $taskName, - 'mode' => $mode, - ]); - - try { - // 检查任务状态（从Redis或数据库） - if (!$this->checkTaskStatus($taskId)) { - LoggerHelper::logBusiness('generic_collection_skipped', [ - 'task_id' => $taskId, - 'reason' => '任务已暂停或停止', - ]); - return; - } - - // 根据模式执行不同的采集逻辑 - if ($mode === 'realtime') { - $this->watchCollection($taskConfig); - } else { - $this->collectBatch($adapter, $taskConfig); - } - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'GenericCollectionHandler', - 'action' => 'collect', - 'task_id' => $taskId, - ]); - - // 更新任务状态为错误 - $this->taskService->updateTask($taskId, [ - 'status' => 'error', - 'progress.status' => 'error', - 'progress.last_error' => $e->getMessage(), - ]); - - throw $e; - } - } - - /** - * 批量采集 - */ - private function collectBatch($adapter, array $taskConfig): void - { - $taskId = $taskConfig['task_id'] ?? ''; - $database = $taskConfig['database'] ?? ''; - $collection = $taskConfig['collection'] ?? null; - $collections = $taskConfig['collections'] ?? null; - $fieldMappings = $taskConfig['field_mappings'] ?? []; - $filterConditions = $taskConfig['filter_conditions'] ?? []; - $batchSize = $taskConfig['batch_size'] ?? 1000; - - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤6-连接源数据库】开始连接源数据库: database={$database}\n"); - $client = $this->getMongoClient($taskConfig); - $db = $client->selectDatabase($database); - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤6-连接源数据库】✓ 源数据库连接成功: database={$database}\n"); - - // 确定要处理的集合列表 - $targetCollections = []; - if ($collection) { - $targetCollections[] = $collection; - } elseif ($collections && is_array($collections)) { - $targetCollections = $collections; - } else { - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤6-连接源数据库】✗ 未指定collection或collections\n"); - throw new \InvalidArgumentException('必须指定 collection 或 collections'); - } - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤6-连接源数据库】要处理的集合: " . json_encode($targetCollections, JSON_UNESCAPED_UNICODE) . "\n"); - - // 先计算总记录数（用于进度计算和更新间隔） - $totalCount = 0; - $filter = $this->buildFilter($filterConditions); - foreach ($targetCollections as $collName) { - $coll = $db->selectCollection($collName); - $collTotal = $coll->countDocuments($filter); - $totalCount += $collTotal; - } - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤6-连接源数据库】总记录数: {$totalCount}\n"); - - // 计算进度更新间隔（根据总数动态调整） - $updateInterval = $this->calculateProgressUpdateInterval($totalCount); - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤6-连接源数据库】进度更新间隔: 每 {$updateInterval} 条更新一次\n"); - - // 更新进度：开始 - $this->updateProgress($taskId, [ - 'status' => 'running', - 'start_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - 'total_count' => $totalCount, - 'processed_count' => 0, - 'success_count' => 0, - 'error_count' => 0, - 'percentage' => 0, - ]); - - $processedCount = 0; - $successCount = 0; - $errorCount = 0; - $lastUpdateCount = 0; // 记录上次更新的处理数量 - - foreach ($targetCollections as $collName) { - if (!$this->checkTaskStatus($taskId)) { - break; // 任务已暂停或停止 - } - - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤7-处理集合】开始处理集合: collection={$collName}\n"); - $coll = $db->selectCollection($collName); - - // 获取该集合的字段映射（优先使用集合级映射） - $collectionFieldMappings = $this->getFieldMappingsForCollection($collName, $taskConfig); - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤7-处理集合】字段映射数量: " . count($collectionFieldMappings) . "\n"); - - // 获取该集合的连表查询配置 - $collectionLookups = $this->getLookupsForCollection($collName, $taskConfig); - - // 如果有连表查询，使用聚合管道 - if (!empty($collectionLookups)) { - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤7-处理集合】使用连表查询模式\n"); - $result = $this->collectWithLookup($coll, $collName, $collectionFieldMappings, $collectionLookups, $filterConditions, $taskConfig, $taskId); - $processedCount += $result['processed']; - $successCount += $result['success']; - $errorCount += $result['error']; - } else { - // 普通查询 - // 构建查询条件 - $filter = $this->buildFilter($filterConditions); - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤7-处理集合】查询条件: " . json_encode($filter, JSON_UNESCAPED_UNICODE) . "\n"); - - // 获取当前集合的总数（用于日志） - $collTotalCount = $coll->countDocuments($filter); - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤7-处理集合】集合总文档数: {$collTotalCount}\n"); - - // 分页查询 - $offset = 0; - do { - if (!$this->checkTaskStatus($taskId)) { - break; // 任务已暂停或停止 - } - - $cursor = $coll->find( - $filter, - [ - 'limit' => $batchSize, - 'skip' => $offset, - ] - ); - - $batch = []; - foreach ($cursor as $doc) { - $batch[] = $this->convertMongoDocumentToArray($doc); - } - - if (empty($batch)) { - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤8-查询数据】批次为空，结束查询\n"); - break; - } - - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤8-查询数据】查询到 {$batchSize} 条数据，offset={$offset}\n"); - - // 处理批量数据 - foreach ($batch as $index => $docData) { - // 检查是否已达到总数（在每条处理前检查，避免超出） - if ($totalCount > 0 && $processedCount >= $totalCount) { - break 2; // 跳出两层循环（foreach 和 do-while） - } - - $processedCount++; - try { - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤9-处理文档】开始处理第 {$processedCount} 条文档 (批次内第 " . ($index + 1) . " 条)\n"); - $this->processDocument($docData, $collectionFieldMappings, $taskConfig); - $successCount++; - if (($index + 1) % 100 === 0) { - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤9-处理文档】已处理 {$successCount} 条成功\n"); - } - } catch (\Exception $e) { - $errorCount++; - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤9-处理文档】✗ 处理文档失败: " . $e->getMessage() . "\n"); - LoggerHelper::logError($e, [ - 'component' => 'GenericCollectionHandler', - 'action' => 'processDocument', - 'task_id' => $taskId, - 'collection' => $collName, - ]); - } - } - - // 根据更新间隔决定是否更新进度 - if ($totalCount == 0 || ($processedCount - $lastUpdateCount) >= $updateInterval || $processedCount >= $totalCount) { - $percentage = $totalCount > 0 ? round(($processedCount / $totalCount) * 100, 2) : 0; - - // 检查是否达到100% - if ($totalCount > 0 && $processedCount >= $totalCount) { - // 进度达到100%，停止采集并更新状态为已完成 - $this->updateProgress($taskId, [ - 'status' => 'completed', - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'total_count' => $totalCount, - 'percentage' => 100, - 'end_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]); - \Workerman\Worker::safeEcho("[GenericCollectionHandler] ✅ 采集完成，进度已达到100%，已停止采集\n"); - break 2; // 跳出两层循环（foreach 和 do-while） - } else { - $this->updateProgress($taskId, [ - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'total_count' => $totalCount, - 'percentage' => $percentage, - ]); - } - $lastUpdateCount = $processedCount; - } - - $offset += $batchSize; - - } while (count($batch) === $batchSize && $processedCount < $totalCount); - } - } - - // 更新进度：完成（如果循环正常结束，也更新状态为已完成） - $task = $this->taskService->getTask($taskId); - if ($task) { - // 只有在任务状态不是 completed、paused、stopped 时，才更新为 completed - // 如果任务被暂停或停止，不应该更新为 completed - if ($task['status'] === 'completed') { - // 已经是 completed，不需要更新 - } elseif (in_array($task['status'], ['paused', 'stopped'])) { - // 任务被暂停或停止，只更新进度，不更新状态 - \Workerman\Worker::safeEcho("[GenericCollectionHandler] ⚠️ 任务已被暂停或停止，不更新为completed状态\n"); - $percentage = $totalCount > 0 ? round(($processedCount / $totalCount) * 100, 2) : 0; - $this->updateProgress($taskId, [ - 'total_count' => $totalCount, - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'percentage' => $percentage, - ]); - } else { - // 任务正常完成，更新状态为 completed - $this->updateProgress($taskId, [ - 'status' => 'completed', - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'total_count' => $totalCount, - 'percentage' => 100, // 完成时强制设置为100% - 'end_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]); - \Workerman\Worker::safeEcho("[GenericCollectionHandler] ✅ 采集任务完成，状态已更新为completed\n"); - } - } - - LoggerHelper::logBusiness('generic_collection_completed', [ - 'task_id' => $taskId, - 'processed' => $processedCount, - 'success' => $successCount, - 'error' => $errorCount, - ]); - } - - /** - * 实时监听 - */ - private function watchCollection(array $taskConfig): void - { - $taskId = $taskConfig['task_id'] ?? ''; - $database = $taskConfig['database'] ?? ''; - $collection = $taskConfig['collection'] ?? null; - $collections = $taskConfig['collections'] ?? null; - $fieldMappings = $taskConfig['field_mappings'] ?? []; - - // 更新进度：开始 - $this->updateProgress($taskId, [ - 'status' => 'running', - 'start_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]); - - $client = $this->getMongoClient($taskConfig); - $db = $client->selectDatabase($database); - - // 如果指定了单个集合，监听该集合 - if ($collection) { - $coll = $db->selectCollection($collection); - $this->watchSingleCollection($coll, $fieldMappings, $taskConfig); - } elseif ($collections && is_array($collections)) { - // 如果指定了多个集合，监听数据库级别（然后过滤） - $this->watchMultipleCollections($db, $collections, $fieldMappings, $taskConfig); - } else { - throw new \InvalidArgumentException('实时模式必须指定 collection 或 collections'); - } - } - - /** - * 监听单个集合 - */ - private function watchSingleCollection(Collection $collection, array $fieldMappings, array $taskConfig): void - { - $taskId = $taskConfig['task_id'] ?? ''; - - $changeStream = $collection->watch( - [], - [ - 'fullDocument' => 'updateLookup', - 'batchSize' => 100, - 'maxAwaitTimeMS' => 1000, - ] - ); - - LoggerHelper::logBusiness('generic_collection_watch_ready', [ - 'task_id' => $taskId, - 'collection' => $collection->getCollectionName(), - ]); - - foreach ($changeStream as $change) { - if (!$this->checkTaskStatus($taskId)) { - break; // 任务已暂停或停止 - } - - try { - $operationType = $change['operationType'] ?? ''; - - if ($operationType === 'insert' || $operationType === 'update') { - $document = $change['fullDocument'] ?? null; - - if ($document === null && $operationType === 'update') { - $documentId = $change['documentKey']['_id'] ?? null; - if ($documentId !== null) { - $document = $collection->findOne(['_id' => $documentId]); - } - } - - if ($document !== null) { - $docData = $this->convertMongoDocumentToArray($document); - $this->processDocument($docData, $fieldMappings, $taskConfig); - - // 更新进度 - $this->updateProgress($taskId, [ - 'processed_count' => ['$inc' => 1], - 'success_count' => ['$inc' => 1], - ]); - } - } - } catch (\Exception $e) { - LoggerHelper::logError($e, [ - 'component' => 'GenericCollectionHandler', - 'action' => 'watchSingleCollection', - 'task_id' => $taskId, - ]); - - // 更新错误计数 - $this->updateProgress($taskId, [ - 'error_count' => ['$inc' => 1], - ]); - } - } - } - - /** - * 监听多个集合 - */ - private function watchMultipleCollections(Database $database, array $collections, array $fieldMappings, array $taskConfig): void - { - $taskId = $taskConfig['task_id'] ?? ''; - - $changeStream = $database->watch( - [], - [ - 'fullDocument' => 'updateLookup', - 'batchSize' => 100, - 'maxAwaitTimeMS' => 1000, - ] - ); - - LoggerHelper::logBusiness('generic_collection_watch_ready', [ - 'task_id' => $taskId, - 'collections' => $collections, - ]); - - foreach ($changeStream as $change) { - if (!$this->checkTaskStatus($taskId)) { - break; // 任务已暂停或停止 - } - - try { - $collectionName = $change['ns']['coll'] ?? ''; - - // 只处理配置的集合 - if (!in_array($collectionName, $collections)) { - continue; - } - - $operationType = $change['operationType'] ?? ''; - - if ($operationType === 'insert' || $operationType === 'update') { - $document = $change['fullDocument'] ?? null; - - if ($document === null && $operationType === 'update') { - $documentId = $change['documentKey']['_id'] ?? null; - if ($documentId !== null) { - $collection = $database->selectCollection($collectionName); - $document = $collection->findOne(['_id' => $documentId]); - } - } - - if ($document !== null) { - $docData = $this->convertMongoDocumentToArray($document); - $this->processDocument($docData, $fieldMappings, $taskConfig); - - // 更新进度 - $this->updateProgress($taskId, [ - 'processed_count' => ['$inc' => 1], - 'success_count' => ['$inc' => 1], - ]); - } - } - } catch (\Exception $e) { - LoggerHelper::logError($e, [ - 'component' => 'GenericCollectionHandler', - 'action' => 'watchMultipleCollections', - 'task_id' => $taskId, - ]); - - // 更新错误计数 - $this->updateProgress($taskId, [ - 'error_count' => ['$inc' => 1], - ]); - } - } - } - - /** - * 处理文档 - */ - private function processDocument(array $docData, array $fieldMappings, array $taskConfig): void - { - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤10-字段映射】开始应用字段映射，源字段数量: " . count(array_keys($docData)) . "\n"); - // 应用字段映射 - $mappedData = $this->applyFieldMappings($docData, $fieldMappings); - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤10-字段映射】✓ 字段映射完成，目标字段数量: " . count(array_keys($mappedData)) . "\n"); - - // 提取用户标识（优先使用user_id，否则使用phone_number或id_card） - $userId = $mappedData['user_id'] ?? null; - $phoneNumber = $mappedData['phone_number'] ?? null; - $idCard = $mappedData['id_card'] ?? null; - - // 如果既没有user_id，也没有phone_number和id_card，则跳过 - if (empty($userId) && empty($phoneNumber) && empty($idCard)) { - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤10-字段映射】✗ 跳过：缺少用户标识\n"); - LoggerHelper::logBusiness('generic_collection_skip_no_user_identifier', [ - 'task_id' => $taskConfig['task_id'] ?? '', - ]); - return; - } - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤10-字段映射】用户标识: user_id=" . ($userId ?? 'null') . ", phone=" . ($phoneNumber ?? 'null') . "\n"); - - // 店铺名称：优先保存从源数据映射的店铺名称（无论店铺表查询结果如何） - $storeName = $mappedData['store_name'] ?? null; - - // 调试：输出映射后的店铺名称（用于排查问题） - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤10-字段映射】映射后的店铺名称: " . ($storeName ?? 'null') . "\n"); - - // 如果映射后没有店铺名称，尝试从源数据中查找可能的店铺名称字段（作为后备方案） - if (empty($storeName)) { - // 常见的店铺名称字段名 - $possibleStoreNameFields = ['store_name', '门店名称', '店铺名称', '门店名', '店铺名', 'storeName', '门店', '店铺', '新零售成交门店昵称']; - foreach ($possibleStoreNameFields as $fieldName) { - $value = $docData[$fieldName] ?? null; - if (!empty($value)) { - $storeName = $value; - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤10-字段映射】从源数据中提取店铺名称: {$fieldName} = {$storeName}\n"); - break; - } - } - } - - // 处理门店ID：如果提供了store_name但没有store_id，则通过门店服务获取或创建 - // 注意：即使获取store_id失败，也要保存store_name - $storeId = $mappedData['store_id'] ?? null; - if (empty($storeId) && !empty($storeName)) { - try { - $source = $taskConfig['data_source_id'] ?? $taskConfig['name'] ?? 'unknown'; - $storeId = $this->storeService->getOrCreateStoreByName( - $storeName, - $source - ); - } catch (\Throwable $e) { - // 店铺ID获取失败不影响店铺名称的保存，只记录日志 - LoggerHelper::logError($e, [ - 'component' => 'GenericCollectionHandler', - 'action' => 'processDocument', - 'message' => '获取店铺ID失败，但会继续保存店铺名称', - 'store_name' => $storeName, - ]); - } - } - - // 构建消费记录数据 - $recordData = [ - 'consume_time' => $mappedData['consume_time'] ?? date('Y-m-d H:i:s'), - 'amount' => $mappedData['amount'] ?? 0, - 'actual_amount' => $mappedData['actual_amount'] ?? $mappedData['amount'] ?? 0, - 'currency' => $mappedData['currency'] ?? 'CNY', - 'status' => $mappedData['status'] ?? 0, - ]; - - // 添加用户标识（优先使用user_id，否则使用phone_number或id_card） - if (!empty($userId)) { - $recordData['user_id'] = $userId; - } elseif (!empty($phoneNumber)) { - $recordData['phone_number'] = $phoneNumber; - } elseif (!empty($idCard)) { - $recordData['id_card'] = $idCard; - } - - // 添加门店ID（如果已转换或已提供） - if (!empty($storeId)) { - $recordData['store_id'] = $storeId; - } - - // 添加店铺名称（优先保存从源数据映射的店铺名称） - if (!empty($storeName)) { - $recordData['store_name'] = $storeName; - } - - // 根据任务配置保存到目标数据源 - $targetType = $taskConfig['target_type'] ?? 'generic'; - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤11-保存数据】开始保存数据到目标数据源: target_type={$targetType}\n"); - - if ($targetType === 'consumption_record') { - // 消费记录类型：使用 ConsumptionService（它会写入到目标数据源） - // 但需要确保 ConsumptionService 使用正确的数据源连接 - $this->saveToTargetDataSource($recordData, $taskConfig, 'consumption_record'); - } else { - // 通用类型：直接保存到指定的目标数据源、数据库、集合 - $this->saveToTargetDataSource($recordData, $taskConfig, 'generic'); - } - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤11-保存数据】✓ 数据保存完成\n"); - } - - /** - * 保存数据到目标数据源 - * - * @param array $data 要保存的数据 - * @param array $taskConfig 任务配置 - * @param string $targetType 目标类型（consumption_record 或 generic） - * @return void - */ - private function saveToTargetDataSource(array $data, array $taskConfig, string $targetType): void - { - $targetDataSourceId = $taskConfig['target_data_source_id'] ?? null; - $targetDatabase = $taskConfig['target_database'] ?? null; - $targetCollection = $taskConfig['target_collection'] ?? null; - - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤12-连接目标数据源】目标数据源ID={$targetDataSourceId}, 目标数据库={$targetDatabase}, 目标集合={$targetCollection}\n"); - - if (empty($targetDataSourceId)) { - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤12-连接目标数据源】✗ 缺少target_data_source_id\n"); - throw new \InvalidArgumentException('任务配置中缺少 target_data_source_id'); - } - - // 连接到目标数据源 - // \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤12-连接目标数据源】开始查询目标数据源配置\n"); - $connectionInfo = $this->connectToTargetDataSource($targetDataSourceId, $targetDatabase); - $targetDataSourceConfig = $connectionInfo['config']; - $dbName = $connectionInfo['dbName']; - $database = $connectionInfo['database']; - - // \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤12-连接目标数据源】✓ 目标数据源配置查询成功: host={$targetDataSourceConfig['host']}, port={$targetDataSourceConfig['port']}\n"); - - // 确定目标集合 - if ($targetType === 'consumption_record') { - // 消费记录类型：集合名为 consumption_records（按月份分表） - $collectionName = 'consumption_records'; - - // 如果有 consume_time，根据时间确定月份集合 - if (isset($data['consume_time'])) { - try { - $consumeTime = new \DateTimeImmutable($data['consume_time']); - $monthSuffix = $consumeTime->format('Ym'); - $collectionName = "consumption_records_{$monthSuffix}"; - } catch (\Exception $e) { - // 如果解析失败，使用当前月份 - $collectionName = 'consumption_records_' . date('Ym'); - } - } else { - $collectionName = 'consumption_records_' . date('Ym'); - } - - // 对于消费记录，需要先通过 ConsumptionService 处理（解析用户ID等） - // 但需要确保它写入到正确的数据源 - // 这里我们直接写入，但需要先解析用户ID - if (empty($data['user_id']) && (!empty($data['phone_number']) || !empty($data['id_card']))) { - // 需要解析用户ID - $userId = $this->identifierService->resolvePersonId( - $data['phone_number'] ?? null, - $data['id_card'] ?? null - ); - $data['user_id'] = $userId; - } - - // 确保有 record_id - if (empty($data['record_id'])) { - $data['record_id'] = \Ramsey\Uuid\Uuid::uuid4()->toString(); - } - - // 转换时间字段 - if (isset($data['consume_time']) && is_string($data['consume_time'])) { - $data['consume_time'] = new \MongoDB\BSON\UTCDateTime(strtotime($data['consume_time']) * 1000); - } - if (empty($data['create_time'])) { - $data['create_time'] = new \MongoDB\BSON\UTCDateTime(time() * 1000); - } elseif (is_string($data['create_time'])) { - $data['create_time'] = new \MongoDB\BSON\UTCDateTime(strtotime($data['create_time']) * 1000); - } - } else { - // 通用类型：使用任务配置中的目标集合 - if (empty($targetCollection)) { - throw new \InvalidArgumentException('通用类型任务必须配置 target_collection'); - } - $collectionName = $targetCollection; - } - - // 写入数据 - $collection = $database->selectCollection($collectionName); - - // 对于消费记录类型，基于业务唯一标识检查是否已存在（防止重复插入） - if ($targetType === 'consumption_record') { - // 转换 consume_time 为 UTCDateTime（如果存在） - $consumeTimeForQuery = null; - if (isset($data['consume_time'])) { - if (is_string($data['consume_time'])) { - $consumeTimeForQuery = new \MongoDB\BSON\UTCDateTime(strtotime($data['consume_time']) * 1000); - } elseif ($data['consume_time'] instanceof \MongoDB\BSON\UTCDateTime) { - $consumeTimeForQuery = $data['consume_time']; - } - } - - // 获取店铺名称（用于去重） - // 优先使用从源数据映射的店铺名称，无论店铺表查询结果如何 - $storeName = $data['store_name'] ?? null; - - // 如果源数据中没有 store_name 但有 store_id，尝试从店铺表获取店铺名称（作为后备方案） - // 但即使查询失败，也要确保 store_name 字段被保存（可能为 null） - if (empty($storeName) && !empty($data['store_id'])) { - try { - $store = $this->storeService->getStoreById($data['store_id']); - if ($store && $store->store_name) { - $storeName = $store->store_name; - } - } catch (\Throwable $e) { - // 从店铺表反查失败不影响数据保存，只记录日志 - LoggerHelper::logError($e, [ - 'component' => 'GenericCollectionHandler', - 'action' => 'saveToTargetDataSource', - 'message' => '从店铺表反查店铺名称失败，将保存null值', - 'store_id' => $data['store_id'] ?? null, - ]); - } - } - - // 确保 store_name 字段被保存到 data 中（即使为 null 也要保存，保持数据结构一致） - $data['store_name'] = $storeName; - - // 基于业务唯一标识检查重复（防止重复插入） - // 方案：使用 store_name + source_order_id 作为唯一标识 - // 注意：order_no 是系统自动生成的（自动递增），不参与去重判断 - $duplicateQuery = null; - $duplicateIdentifier = null; - $sourceOrderId = $data['source_order_id'] ?? null; - - if (!empty($storeName) && !empty($sourceOrderId)) { - // 使用店铺名称 + 原始订单ID作为唯一标识 - $duplicateQuery = [ - 'store_name' => $storeName, - 'source_order_id' => $sourceOrderId, - ]; - $duplicateIdentifier = "store_name={$storeName}, source_order_id={$sourceOrderId}"; - } - - // 如果找到了唯一标识，检查是否已存在 - if ($duplicateQuery) { - $existingRecord = $collection->findOne($duplicateQuery); - if ($existingRecord) { - \Workerman\Worker::safeEcho("[GenericCollectionHandler] ⚠️ 消费记录已存在，跳过插入: {$duplicateIdentifier}, collection={$collectionName}\n"); - LoggerHelper::logBusiness('generic_collection_duplicate_skipped', [ - 'task_id' => $taskId, - 'duplicate_identifier' => $duplicateIdentifier, - 'target_collection' => $collectionName, - ]); - return; // 跳过重复记录 - } - } - - // 生成 record_id（如果还没有） - // 使用店铺名称 + 原始订单ID生成稳定的 record_id - if (empty($data['record_id'])) { - if (!empty($storeName) && !empty($sourceOrderId)) { - // 使用店铺名称 + 原始订单ID生成稳定的 record_id - $uniqueKey = "{$storeName}|{$sourceOrderId}"; - $data['record_id'] = 'store_source_' . md5($uniqueKey); - } else { - // 如果都没有，生成 UUID - $data['record_id'] = \Ramsey\Uuid\Uuid::uuid4()->toString(); - } - } - - // 生成 order_no（系统自动生成，自动递增） - // 注意：order_no 不参与去重判断，仅用于展示和查询 - // 使用计数器集合来生成唯一的 order_no（在去重检查之后，只有实际插入的记录才生成 order_no） - if (empty($data['order_no'])) { - try { - // 使用计数器集合来生成唯一的 order_no - $counterCollection = $database->selectCollection($collectionName . '_counter'); - - // 原子性地递增计数器 - $counterResult = $counterCollection->findOneAndUpdate( - ['_id' => 'order_no'], - ['$inc' => ['seq' => 1], '$setOnInsert' => ['_id' => 'order_no', 'seq' => 1]], - ['upsert' => true, 'returnDocument' => 1] // 1 = RETURN_DOCUMENT_AFTER - ); - - $nextOrderNo = $counterResult['seq'] ?? 1; - $data['order_no'] = (string)$nextOrderNo; - } catch (\Throwable $e) { - // 如果计数器操作失败，回退到查询最大值的方案 - LoggerHelper::logError($e, [ - 'component' => 'GenericCollectionHandler', - 'action' => 'saveToTargetDataSource', - 'message' => '使用计数器生成order_no失败，回退到查询最大值方案', - ]); - - try { - $maxOrderNo = $collection->findOne( - [], - ['sort' => ['order_no' => -1], 'projection' => ['order_no' => 1]] - ); - $nextOrderNo = 1; - if ($maxOrderNo && isset($maxOrderNo['order_no']) && is_numeric($maxOrderNo['order_no'])) { - $nextOrderNo = (int)$maxOrderNo['order_no'] + 1; - } - $data['order_no'] = (string)$nextOrderNo; - } catch (\Throwable $e2) { - // 如果查询也失败，使用时间戳作为备选方案 - $data['order_no'] = (string)(time() * 1000 + mt_rand(1000, 9999)); - LoggerHelper::logError($e2, [ - 'component' => 'GenericCollectionHandler', - 'action' => 'saveToTargetDataSource', - 'message' => '查询最大order_no也失败，使用时间戳作为备选', - ]); - } - } - } - - // 确保 consume_time 是 UTCDateTime 类型 - if ($consumeTimeForQuery) { - $data['consume_time'] = $consumeTimeForQuery; - } - } - - // 格式化输出流水信息 - $timestamp = date('Y-m-d H:i:s'); - $taskId = $taskConfig['task_id'] ?? 'unknown'; - $recordId = $data['record_id'] ?? $data['_id'] ?? 'auto'; - $userId = $data['user_id'] ?? 'null'; - $phoneNumber = $data['phone_number'] ?? $data['phone'] ?? 'null'; - - // 提取关键字段用于显示（最多显示10个字段） - $keyFields = []; - $fieldCount = 0; - foreach ($data as $key => $value) { - if ($fieldCount >= 10) break; - if (in_array($key, ['_id', 'record_id', 'user_id', 'phone_number', 'phone'])) continue; - if (is_array($value) || is_object($value)) { - $keyFields[] = "{$key}: " . json_encode($value, JSON_UNESCAPED_UNICODE); - } else { - $keyFields[] = "{$key}: {$value}"; - } - $fieldCount++; - } - $keyFieldsStr = !empty($keyFields) ? implode(', ', $keyFields) : '(无其他字段)'; - - // 输出详细的插入流水信息（输出到终端） - $output = "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n" - . "📝 [{$timestamp}] 通用数据插入流水 | 任务ID: {$taskId}\n" - . " ├─ 记录ID: {$recordId}\n" - . " ├─ 用户ID: {$userId}\n" - . " ├─ 手机号: {$phoneNumber}\n" - . " ├─ 关键字段: {$keyFieldsStr}\n" - . " ├─ 目标数据库: {$dbName}\n" - . " └─ 目标集合: {$collectionName}\n"; - - \Workerman\Worker::safeEcho($output); - - $result = $collection->insertOne($data); - $insertedId = $result->getInsertedId(); - - $successOutput = " ✅ 插入成功 | MongoDB ID: " . (is_object($insertedId) ? (string)$insertedId : json_encode($insertedId)) . "\n" - . "━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n"; - - \Workerman\Worker::safeEcho($successOutput); - - LoggerHelper::logBusiness('generic_collection_data_saved', [ - 'task_id' => $taskConfig['task_id'] ?? '', - 'target_data_source_id' => $targetDataSourceId, - 'target_database' => $dbName, - 'target_collection' => $collectionName, - 'inserted_id' => (string)$insertedId, - ]); - } - - /** - * 应用字段映射 - * - * 注意：如果源字段或目标字段为空（空字符串、null、未设置），则跳过该映射 - * 这样用户可以清除源字段选择，表示不需要映射该目标字段 - */ - private function applyFieldMappings(array $sourceData, array $fieldMappings): array - { - $mappedData = []; - - foreach ($fieldMappings as $mapping) { - $sourceField = $mapping['source_field'] ?? ''; - $targetField = $mapping['target_field'] ?? ''; - $transform = $mapping['transform'] ?? null; - - // 如果源字段或目标字段为空，跳过该映射（兼容用户清除源字段选择的情况） - if (empty($sourceField) || empty($targetField)) { - continue; - } - - // 从源数据中获取值（支持嵌套字段，如 "user.name"） - $value = $this->getNestedValue($sourceData, $sourceField); - - // 调试：输出字段映射详情（仅对关键字段） - if ($targetField === 'store_name') { - \Workerman\Worker::safeEcho("[GenericCollectionHandler] 【通用-步骤10-字段映射】字段映射详情: target={$targetField}, source={$sourceField}, value=" . ($value ?? 'null') . "\n"); - } - - // 应用转换函数 - if ($transform && is_callable($transform)) { - $value = $transform($value); - } elseif ($transform && is_string($transform)) { - $value = $this->applyTransform($value, $transform); - } - - $mappedData[$targetField] = $value; - } - - return $mappedData; - } - - /** - * 获取嵌套字段值 - */ - private function getNestedValue(array $data, string $fieldPath) - { - $parts = explode('.', $fieldPath); - $value = $data; - - foreach ($parts as $part) { - if (is_array($value) && isset($value[$part])) { - $value = $value[$part]; - } elseif (is_object($value) && isset($value->$part)) { - $value = $value->$part; - } else { - return null; - } - } - - return $value; - } - - /** - * 应用转换函数 - */ - private function applyTransform($value, string $transform) - { - switch ($transform) { - case 'parse_amount': - return $this->parseAmount($value); - case 'parse_datetime': - return $this->parseDateTimeToString($value); - case 'parse_phone': - return $this->extractPhoneNumber(['phone' => $value]); - default: - return $value; - } - } - - /** - * 提取手机号（通用数据专用） - * - * 注意：这个方法保留在此类中，因为它处理的是通用数据的字段名 - * 订单数据的手机号提取在 ConsumptionCollectionHandler 中 - */ - private function extractPhoneNumber(array $data): ?string - { - // 尝试多个可能的字段名 - $phoneFields = ['phone_number', 'phone', 'mobile', 'tel', 'contact_phone']; - - foreach ($phoneFields as $field) { - if (isset($data[$field])) { - $phone = trim((string)$data[$field]); - // 先过滤非数字字符 - $cleanedPhone = $this->filterPhoneNumber($phone); - if (!empty($cleanedPhone) && $this->isValidPhone($cleanedPhone)) { - // 返回过滤后的手机号 - return $cleanedPhone; - } - } - } - - return null; - } - - /** - * 解析日期时间为字符串（用于通用数据保存） - * - * 注意：这个方法与 Trait 中的 parseDateTime 不同，它返回字符串格式 - */ - private function parseDateTimeToString($dateTimeStr): string - { - if (empty($dateTimeStr)) { - return date('Y-m-d H:i:s'); - } - - $dateTime = $this->parseDateTime($dateTimeStr); - if ($dateTime === null) { - return date('Y-m-d H:i:s'); - } - - return $dateTime->format('Y-m-d H:i:s'); - } - - /** - * 构建过滤条件 - */ - private function buildFilter(array $filterConditions): array - { - $filter = []; - - foreach ($filterConditions as $condition) { - $field = $condition['field'] ?? ''; - $operator = $condition['operator'] ?? 'eq'; - $value = $condition['value'] ?? null; - - if (empty($field)) { - continue; - } - - switch ($operator) { - case 'eq': - $filter[$field] = $value; - break; - case 'ne': - $filter[$field] = ['$ne' => $value]; - break; - case 'gt': - $filter[$field] = ['$gt' => $value]; - break; - case 'gte': - $filter[$field] = ['$gte' => $value]; - break; - case 'lt': - $filter[$field] = ['$lt' => $value]; - break; - case 'lte': - $filter[$field] = ['$lte' => $value]; - break; - case 'in': - $filter[$field] = ['$in' => $value]; - break; - case 'nin': - $filter[$field] = ['$nin' => $value]; - break; - } - } - - return $filter; - } - - /** - * 检查任务状态 - */ - private function checkTaskStatus(string $taskId): bool - { - // 检查Redis标志 - if (\app\utils\RedisHelper::exists("data_collection_task:{$taskId}:pause")) { - return false; - } - if (\app\utils\RedisHelper::exists("data_collection_task:{$taskId}:stop")) { - return false; - } - - // 检查数据库状态 - $task = $this->taskService->getTask($taskId); - if ($task && in_array($task['status'], ['paused', 'stopped', 'error'])) { - return false; - } - - return true; - } - - /** - * 更新进度 - */ - /** - * 根据总记录数计算合适的进度更新间隔 - * - * @param int $totalCount 总记录数 - * @return int 更新间隔（每处理多少条记录更新一次） - */ - private function calculateProgressUpdateInterval(int $totalCount): int - { - // 根据总数动态调整更新间隔，确保既不会太频繁也不会太慢 - // 策略：大约每1%更新一次，但限制在合理范围内 - - if ($totalCount <= 0) { - return 50; // 默认50条 - } - - // 计算1%的数量 - $onePercent = max(1, (int)($totalCount * 0.01)); - - // 根据总数范围调整： - // - 小于1000条：每50条更新（保证至少更新20次） - // - 1000-10000条：每1%更新（约10-100条） - // - 10000-100000条：每1%更新（约100-1000条） - // - 100000-1000000条：每1%更新（约1000-10000条），但最多5000条 - // - 大于1000000条：每5000条更新（避免更新太频繁） - - if ($totalCount < 1000) { - return 50; - } elseif ($totalCount < 10000) { - return max(50, min(500, $onePercent)); - } elseif ($totalCount < 100000) { - return max(100, min(1000, $onePercent)); - } elseif ($totalCount < 1000000) { - return max(500, min(5000, $onePercent)); - } else { - return 5000; // 大数据量固定5000条更新一次 - } - } - - private function updateProgress(string $taskId, array $progress): void - { - try { - $task = $this->taskService->getTask($taskId); - if (!$task) { - return; - } - - $currentProgress = $task['progress'] ?? []; - - // 检查是否需要更新任务状态 - $updateTaskStatus = false; - $newStatus = null; - if (isset($progress['status'])) { - $updateTaskStatus = true; - $newStatus = $progress['status']; - unset($progress['status']); // 从progress中移除，单独处理 - } - - // 处理增量更新 - foreach ($progress as $key => $value) { - if (is_array($value) && isset($value['$inc'])) { - $currentProgress[$key] = ($currentProgress[$key] ?? 0) + $value['$inc']; - } else { - $currentProgress[$key] = $value; - } - } - - // 确保 percentage 字段存在且正确计算（基于已采集条数/总条数） - if (isset($currentProgress['processed_count']) && isset($currentProgress['total_count'])) { - if ($currentProgress['total_count'] > 0) { - // 进度 = 已采集条数 / 总条数 * 100 - $currentProgress['percentage'] = round( - ($currentProgress['processed_count'] / $currentProgress['total_count']) * 100, - 2 - ); - // 确保不超过100% - $currentProgress['percentage'] = min(100, $currentProgress['percentage']); - } else { - $currentProgress['percentage'] = 0; - } - } - - // 更新进度到数据库 - $this->taskService->updateProgress($taskId, $currentProgress); - - // 如果指定了状态，更新任务状态（例如：completed） - if ($updateTaskStatus && $newStatus !== null) { - $this->taskService->updateTask($taskId, ['status' => $newStatus]); - } - } catch (\Exception $e) { - LoggerHelper::logError($e, [ - 'component' => 'GenericCollectionHandler', - 'action' => 'updateProgress', - 'task_id' => $taskId, - ]); - } - } - - - /** - * 获取集合的字段映射（优先使用集合级映射，否则使用全局映射） - * - * @param string $collectionName 集合名称 - * @param array $taskConfig 任务配置 - * @return array 字段映射配置 - */ - private function getFieldMappingsForCollection(string $collectionName, array $taskConfig): array - { - // 优先使用集合级映射 - $collectionMappings = $taskConfig['collection_field_mappings'][$collectionName] ?? null; - if ($collectionMappings !== null && is_array($collectionMappings)) { - return $collectionMappings; - } - - // 回退到全局映射 - return $taskConfig['field_mappings'] ?? []; - } - - /** - * 获取集合的连表查询配置（优先使用集合级配置，否则使用全局配置） - * - * @param string $collectionName 集合名称 - * @param array $taskConfig 任务配置 - * @return array 连表查询配置 - */ - private function getLookupsForCollection(string $collectionName, array $taskConfig): array - { - // 优先使用集合级连表查询配置 - $collectionLookups = $taskConfig['collection_lookups'][$collectionName] ?? null; - if ($collectionLookups !== null && is_array($collectionLookups)) { - return $collectionLookups; - } - - // 回退到全局连表查询配置（单集合模式） - return $taskConfig['lookups'] ?? []; - } - - /** - * 使用连表查询采集数据 - * - * @param Collection $collection MongoDB集合对象 - * @param string $collectionName 集合名称 - * @param array $fieldMappings 字段映射配置 - * @param array $lookups 连表查询配置 - * @param array $filterConditions 过滤条件 - * @param array $taskConfig 任务配置 - * @param string $taskId 任务ID - * @return array{processed: int, success: int, error: int} 处理结果统计 - */ - private function collectWithLookup( - Collection $collection, - string $collectionName, - array $fieldMappings, - array $lookups, - array $filterConditions, - array $taskConfig, - string $taskId - ): array { - $processedCount = 0; - $successCount = 0; - $errorCount = 0; - $batchSize = $taskConfig['batch_size'] ?? 1000; - - // 构建聚合管道 - $pipeline = $this->buildAggregationPipeline($filterConditions, $lookups); - - LoggerHelper::logBusiness('generic_collection_lookup_start', [ - 'task_id' => $taskId, - 'collection' => $collectionName, - 'lookups' => $lookups, - ]); - - $offset = 0; - do { - if (!$this->checkTaskStatus($taskId)) { - break; // 任务已暂停或停止 - } - - // 构建分页管道 - $pagedPipeline = $pipeline; - if ($offset > 0) { - $pagedPipeline[] = ['$skip' => $offset]; - } - $pagedPipeline[] = ['$limit' => $batchSize]; - - // 执行聚合查询 - $cursor = $collection->aggregate($pagedPipeline); - - $batch = []; - foreach ($cursor as $doc) { - $batch[] = $this->convertMongoDocumentToArray($doc); - } - - if (empty($batch)) { - break; - } - - // 处理批量数据 - foreach ($batch as $docData) { - $processedCount++; - try { - $this->processDocument($docData, $fieldMappings, $taskConfig); - $successCount++; - } catch (\Exception $e) { - $errorCount++; - LoggerHelper::logError($e, [ - 'component' => 'GenericCollectionHandler', - 'action' => 'processDocument_lookup', - 'task_id' => $taskId, - 'collection' => $collectionName, - ]); - } - } - - // 更新进度 - $this->updateProgress($taskId, [ - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - ]); - - $offset += $batchSize; - - } while (count($batch) === $batchSize); - - LoggerHelper::logBusiness('generic_collection_lookup_completed', [ - 'task_id' => $taskId, - 'collection' => $collectionName, - 'processed' => $processedCount, - 'success' => $successCount, - 'error' => $errorCount, - ]); - - return [ - 'processed' => $processedCount, - 'success' => $successCount, - 'error' => $errorCount, - ]; - } - - /** - * 构建MongoDB聚合管道（支持连表查询） - * - * @param array $filterConditions 过滤条件 - * @param array $lookups 连表查询配置 - * @return array MongoDB聚合管道 - */ - private function buildAggregationPipeline(array $filterConditions, array $lookups): array - { - $pipeline = []; - - // 1. 匹配条件（$match） - $filter = $this->buildFilter($filterConditions); - if (!empty($filter)) { - $pipeline[] = ['$match' => $filter]; - } - - // 2. 连表查询（$lookup） - foreach ($lookups as $lookup) { - $from = $lookup['from'] ?? ''; - $localField = $lookup['local_field'] ?? ''; - $foreignField = $lookup['foreign_field'] ?? ''; - $as = $lookup['as'] ?? 'joined'; - - if (empty($from) || empty($localField) || empty($foreignField)) { - LoggerHelper::logBusiness('generic_collection_lookup_invalid', [ - 'lookup' => $lookup, - ]); - continue; - } - - // 构建 $lookup 阶段 - $lookupStage = [ - '$lookup' => [ - 'from' => $from, - 'localField' => $localField, - 'foreignField' => $foreignField, - 'as' => $as, - ], - ]; - $pipeline[] = $lookupStage; - - // 如果配置了解构（unwrap），添加 $unwind 阶段 - if ($lookup['unwrap'] ?? false) { - $pipeline[] = [ - '$unwind' => [ - 'path' => '$' . $as, - 'preserveNullAndEmptyArrays' => $lookup['preserve_null'] ?? true, // 默认保留没有关联的记录 - ], - ]; - } - } - - return $pipeline; - } -} - diff --git a/Moncter/Moncter/app/service/DataCollection/Handler/TagTaskHandler.php b/Moncter/Moncter/app/service/DataCollection/Handler/TagTaskHandler.php deleted file mode 100644 index 27112ba2..00000000 --- a/Moncter/Moncter/app/service/DataCollection/Handler/TagTaskHandler.php +++ /dev/null @@ -1,74 +0,0 @@ - $taskConfig 任务配置 - * @return void - */ - public function collect($adapter, array $taskConfig): void - { - $taskId = $taskConfig['task_id'] ?? ''; - $taskName = $taskConfig['name'] ?? '标签任务'; - - LoggerHelper::logBusiness('tag_task_handler_started', [ - 'task_id' => $taskId, - 'task_name' => $taskName, - ]); - - try { - // 创建TagTaskService实例 - $tagTaskService = new TagTaskService( - new TagTaskRepository(), - new TagTaskExecutionRepository(), - new UserProfileRepository(), - new TagService( - new TagDefinitionRepository(), - new UserProfileRepository(), - new UserTagRepository(), - new TagHistoryRepository(), - new SimpleRuleEngine() - ) - ); - - // 执行任务 - $tagTaskService->executeTask($taskId); - - LoggerHelper::logBusiness('tag_task_handler_completed', [ - 'task_id' => $taskId, - 'task_name' => $taskName, - ]); - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'TagTaskHandler', - 'action' => 'collect', - 'task_id' => $taskId, - ]); - throw $e; - } - } -} - diff --git a/Moncter/Moncter/app/service/DataCollection/Handler/Trait/DataCollectionHelperTrait.php b/Moncter/Moncter/app/service/DataCollection/Handler/Trait/DataCollectionHelperTrait.php deleted file mode 100644 index 44ecfd6c..00000000 --- a/Moncter/Moncter/app/service/DataCollection/Handler/Trait/DataCollectionHelperTrait.php +++ /dev/null @@ -1,172 +0,0 @@ - 数组格式的数据 - */ - protected function convertMongoDocumentToArray($document): array - { - if (is_array($document)) { - return $document; - } - - if (is_object($document) && method_exists($document, 'toArray')) { - return $document->toArray(); - } - - return json_decode(json_encode($document), true) ?? []; - } - - /** - * 解析日期时间字符串 - * - * @param mixed $dateTimeStr 日期时间字符串或对象 - * @return \DateTimeImmutable|null 解析后的日期时间对象 - */ - protected function parseDateTime($dateTimeStr): ?\DateTimeImmutable - { - if (empty($dateTimeStr)) { - return null; - } - - // 如果是 MongoDB 的 UTCDateTime 对象 - if ($dateTimeStr instanceof UTCDateTime) { - return \DateTimeImmutable::createFromMutable($dateTimeStr->toDateTime()); - } - - // 如果是 DateTime 对象 - if ($dateTimeStr instanceof \DateTime || $dateTimeStr instanceof \DateTimeImmutable) { - if ($dateTimeStr instanceof \DateTime) { - return \DateTimeImmutable::createFromMutable($dateTimeStr); - } - return $dateTimeStr; - } - - // 尝试解析字符串 - try { - return new \DateTimeImmutable((string)$dateTimeStr); - } catch (\Exception $e) { - \app\utils\LoggerHelper::logBusiness('datetime_parse_failed', [ - 'input' => $dateTimeStr, - 'error' => $e->getMessage(), - ]); - return null; - } - } - - /** - * 解析金额 - * - * @param mixed $amount 金额字符串或数字 - * @return float 解析后的金额 - */ - protected function parseAmount($amount): float - { - if (is_numeric($amount)) { - return (float)$amount; - } - - if (is_string($amount)) { - // 移除所有非数字字符（除了小数点） - $cleaned = preg_replace('/[^\d.]/', '', $amount); - return (float)$cleaned; - } - - return 0.0; - } - - /** - * 过滤手机号中的非数字字符 - * - * @param string $phoneNumber 原始手机号 - * @return string 过滤后的手机号（只包含数字） - */ - protected function filterPhoneNumber(string $phoneNumber): string - { - // 移除所有非数字字符 - return preg_replace('/\D/', '', $phoneNumber); - } - - /** - * 验证手机号格式 - * - * @param string $phone 手机号（已经过滤过非数字字符） - * @return bool 是否有效（11位数字，1开头） - */ - protected function isValidPhone(string $phone): bool - { - // 如果为空，直接返回 false - if (empty($phone)) { - return false; - } - - // 中国大陆手机号：11位数字，以1开头 - return preg_match('/^1[3-9]\d{9}$/', $phone) === 1; - } - - /** - * 根据消费时间生成月份集合名 - * - * @param string $baseCollectionName 基础集合名 - * @param mixed $dateTimeStr 日期时间字符串或对象 - * @return string 带月份后缀的集合名（如：consumption_records_202512） - */ - protected function getMonthlyCollectionName(string $baseCollectionName, $dateTimeStr = null): string - { - $consumeTime = $this->parseDateTime($dateTimeStr); - if ($consumeTime === null) { - $consumeTime = new \DateTimeImmutable(); - } - $monthSuffix = $consumeTime->format('Ym'); - return "{$baseCollectionName}_{$monthSuffix}"; - } - - /** - * 转换为 MongoDB UTCDateTime - * - * @param mixed $dateTimeStr 日期时间字符串或对象 - * @return UTCDateTime|null MongoDB UTCDateTime 对象 - */ - protected function convertToUTCDateTime($dateTimeStr): ?UTCDateTime - { - if (empty($dateTimeStr)) { - return null; - } - - // 如果已经是 UTCDateTime，直接返回 - if ($dateTimeStr instanceof UTCDateTime) { - return $dateTimeStr; - } - - // 如果是 DateTime 对象 - if ($dateTimeStr instanceof \DateTime || $dateTimeStr instanceof \DateTimeImmutable) { - return new UTCDateTime($dateTimeStr->getTimestamp() * 1000); - } - - // 尝试解析字符串 - try { - $dateTime = new \DateTimeImmutable((string)$dateTimeStr); - return new UTCDateTime($dateTime->getTimestamp() * 1000); - } catch (\Exception $e) { - \app\utils\LoggerHelper::logBusiness('convert_to_utcdatetime_failed', [ - 'input' => $dateTimeStr, - 'error' => $e->getMessage(), - ]); - return null; - } - } -} - diff --git a/Moncter/Moncter/app/service/DataCollectionTaskService.php b/Moncter/Moncter/app/service/DataCollectionTaskService.php deleted file mode 100644 index 1c89a9b1..00000000 --- a/Moncter/Moncter/app/service/DataCollectionTaskService.php +++ /dev/null @@ -1,660 +0,0 @@ - $taskData 任务数据 - * @return array 创建的任务信息 - */ - public function createTask(array $taskData): array - { - // 生成任务ID - $taskId = UuidGenerator::uuid4()->toString(); - - // 根据Handler类型自动处理目标数据源配置 - $targetType = $taskData['target_type'] ?? ''; - $targetDataSourceId = $taskData['target_data_source_id'] ?? ''; - $targetDatabase = $taskData['target_database'] ?? ''; - $targetCollection = $taskData['target_collection'] ?? ''; - - if ($targetType === 'consumption_record') { - // 消费记录Handler：自动使用标签数据库配置 - $dataSourceService = new \app\service\DataSourceService(new \app\repository\DataSourceRepository()); - $dataSources = $dataSourceService->getDataSourceList(['status' => 1]); - - // 查找标签数据库数据源（通过名称或ID匹配） - $tagDataSource = null; - foreach ($dataSources['list'] ?? [] as $ds) { - $dsName = strtolower($ds['name'] ?? ''); - $dsId = strtolower($ds['data_source_id'] ?? ''); - if ($dsId === 'tag_mongodb' || - $dsName === 'tag_mongodb' || - stripos($dsName, '标签') !== false || - stripos($dsName, 'tag') !== false) { - $tagDataSource = $ds; - break; - } - } - - if ($tagDataSource) { - $targetDataSourceId = $tagDataSource['data_source_id']; - $targetDatabase = $tagDataSource['database'] ?? 'ckb'; - $targetCollection = 'consumption_records'; // 消费记录Handler会自动按时间分表 - } else { - // 如果找不到，使用默认值 - $targetDataSourceId = 'tag_mongodb'; // 尝试使用配置key作为ID - $targetDatabase = 'ckb'; - $targetCollection = 'consumption_records'; - } - } elseif ($targetType === 'generic') { - // 通用Handler：验证用户是否提供了配置 - if (empty($targetDataSourceId) || empty($targetDatabase) || empty($targetCollection)) { - throw new \InvalidArgumentException('通用Handler必须配置目标数据源、目标数据库和目标集合'); - } - } - - // 构建任务文档 - $task = [ - 'task_id' => $taskId, - 'name' => $taskData['name'] ?? '未命名任务', - 'description' => $taskData['description'] ?? '', - 'data_source_id' => $taskData['data_source_id'] ?? '', - 'database' => $taskData['database'] ?? '', - 'collection' => $taskData['collection'] ?? null, - 'collections' => $taskData['collections'] ?? null, - 'target_type' => $targetType, - 'target_data_source_id' => $targetDataSourceId, - 'target_database' => $targetDatabase, - 'target_collection' => $targetCollection, - 'mode' => $taskData['mode'] ?? 'batch', // batch: 批量采集, realtime: 实时监听 - 'field_mappings' => $this->cleanFieldMappings($taskData['field_mappings'] ?? []), - 'collection_field_mappings' => $taskData['collection_field_mappings'] ?? [], - 'lookups' => $taskData['lookups'] ?? [], - 'collection_lookups' => $taskData['collection_lookups'] ?? [], - 'filter_conditions' => $taskData['filter_conditions'] ?? [], - 'schedule' => $taskData['schedule'] ?? [ - 'enabled' => false, - 'cron' => null, - ], - 'status' => 'pending', // pending: 待启动, running: 运行中, paused: 已暂停, stopped: 已停止, error: 错误 - 'progress' => [ - 'status' => 'idle', // idle, running, paused, completed, error - 'processed_count' => 0, - 'success_count' => 0, - 'error_count' => 0, - 'total_count' => 0, - 'percentage' => 0, - 'start_time' => null, - 'end_time' => null, - 'last_sync_time' => null, - ], - 'statistics' => [ - 'total_processed' => 0, - 'total_success' => 0, - 'total_error' => 0, - 'last_run_time' => null, - ], - 'created_by' => $taskData['created_by'] ?? 'system', - ]; - - // 保存到数据库（使用原生MongoDB客户端，明确指定集合名） - // 注意：MongoDB Laravel的Model在数据中包含collection字段时，可能会误用该字段作为集合名 - // 因此使用原生客户端明确指定集合名为data_collection_tasks - $dbConfig = config('database.connections.mongodb'); - - // 使用 MongoDBHelper 创建客户端（统一DSN构建逻辑） - $client = \app\utils\MongoDBHelper::createClient([ - 'host' => parse_url($dbConfig['dsn'], PHP_URL_HOST) ?? '192.168.1.106', - 'port' => parse_url($dbConfig['dsn'], PHP_URL_PORT) ?? 27017, - 'username' => $dbConfig['username'] ?? '', - 'password' => $dbConfig['password'] ?? '', - 'auth_source' => $dbConfig['options']['authSource'] ?? 'admin', - ], array_filter($dbConfig['options'] ?? [], function ($value) { - return $value !== '' && $value !== null; - })); - - $database = $client->selectDatabase($dbConfig['database']); - $collection = $database->selectCollection('data_collection_tasks'); - - // 添加时间戳 - $task['created_at'] = new \MongoDB\BSON\UTCDateTime(time() * 1000); - $task['updated_at'] = new \MongoDB\BSON\UTCDateTime(time() * 1000); - - // 插入文档 - $result = $collection->insertOne($task); - - // 验证插入成功 - if ($result->getInsertedCount() !== 1) { - throw new \RuntimeException("任务创建失败：未能插入到数据库"); - } - - // 如果任务状态是 running，立即设置 Redis 启动标志，让调度器启动采集进程 - if ($task['status'] === 'running') { - try { - \app\utils\RedisHelper::set("data_collection_task:{$taskId}:start", '1', 3600); // 1小时过期 - LoggerHelper::logBusiness('data_collection_task_start_flag_set', [ - 'task_id' => $taskId, - 'task_name' => $task['name'], - ]); - } catch (\Throwable $e) { - // Redis 设置失败不影响任务创建，只记录日志 - LoggerHelper::logError($e, [ - 'component' => 'DataCollectionTaskService', - 'action' => 'createTask', - 'task_id' => $taskId, - 'message' => '设置启动标志失败', - ]); - } - } - - LoggerHelper::logBusiness('data_collection_task_created', [ - 'task_id' => $taskId, - 'task_name' => $task['name'], - ]); - - return $task; - } - - /** - * 清理字段映射数据，移除无效的映射项 - * - * @param array $fieldMappings 原始字段映射数组 - * @return array 清理后的字段映射数组 - */ - private function cleanFieldMappings(array $fieldMappings): array - { - $cleaned = []; - foreach ($fieldMappings as $mapping) { - // 如果缺少target_field，跳过该项 - if (empty($mapping['target_field'])) { - continue; - } - - // 清理状态值映射中的源状态值（移除多余的引号） - if (isset($mapping['value_mapping']) && is_array($mapping['value_mapping'])) { - foreach ($mapping['value_mapping'] as &$vm) { - if (isset($vm['source_value'])) { - // 移除字符串两端的单引号或双引号 - $vm['source_value'] = trim($vm['source_value'], "'\""); - } - } - unset($vm); // 解除引用 - } - - $cleaned[] = $mapping; - } - return $cleaned; - } - - /** - * 更新任务 - * - * @param string $taskId 任务ID - * @param array $taskData 任务数据 - * @return bool 是否更新成功 - */ - public function updateTask(string $taskId, array $taskData): bool - { - // 使用where查询，因为主键是task_id而不是_id - $task = $this->taskRepository->where('task_id', $taskId)->first(); - - if (!$task) { - throw new \InvalidArgumentException("任务不存在: {$taskId}"); - } - - // 如果任务正在运行，完全禁止编辑（与前端逻辑保持一致） - if ($task->status === 'running') { - throw new \RuntimeException("运行中的任务不允许编辑，请先停止任务: {$taskId}"); - } - - // timestamps会自动处理updated_at - - $result = $this->taskRepository->where('task_id', $taskId)->update($taskData); - - LoggerHelper::logBusiness('data_collection_task_updated', [ - 'task_id' => $taskId, - 'updated_fields' => array_keys($taskData), - ]); - - return $result > 0; - } - - /** - * 删除任务 - * - * 如果任务正在运行或已暂停，会先停止任务再删除 - * - * @param string $taskId 任务ID - * @return bool 是否删除成功 - */ - public function deleteTask(string $taskId): bool - { - // 使用where查询，因为主键是task_id而不是_id - $task = $this->taskRepository->where('task_id', $taskId)->first(); - - if (!$task) { - throw new \InvalidArgumentException("任务不存在: {$taskId}"); - } - - // 如果任务正在运行或已暂停，先停止 - if (in_array($task->status, ['running', 'paused'])) { - $this->stopTask($taskId); - } - - $result = $this->taskRepository->where('task_id', $taskId)->delete(); - - LoggerHelper::logBusiness('data_collection_task_deleted', [ - 'task_id' => $taskId, - 'previous_status' => $task->status, - ]); - - return $result > 0; - } - - /** - * 启动任务 - * - * 允许从以下状态启动： - * - pending (待启动) -> running - * - paused (已暂停) -> running (恢复) - * - stopped (已停止) -> running (重新启动) - * - completed (已完成) -> running (重新启动) - * - error (错误) -> running (重新启动) - * - * @param string $taskId 任务ID - * @return bool 是否启动成功 - */ - public function startTask(string $taskId): bool - { - // 使用where查询，因为主键是task_id而不是_id - $task = $this->taskRepository->where('task_id', $taskId)->first(); - - if (!$task) { - throw new \InvalidArgumentException("任务不存在: {$taskId}"); - } - - // 只允许从特定状态启动 - $allowedStatuses = ['pending', 'paused', 'stopped', 'completed', 'error']; - if (!in_array($task->status, $allowedStatuses)) { - if ($task->status === 'running') { - throw new \RuntimeException("任务已在运行中: {$taskId}"); - } - throw new \RuntimeException("任务当前状态不允许启动: {$taskId} (当前状态: {$task->status})"); - } - - // 如果是从 paused, stopped, completed, error 状态启动（重新启动），需要重置进度 - $progress = $task->progress ?? []; - if (in_array($task->status, ['paused', 'stopped', 'completed', 'error'])) { - // 重新启动时，重置进度（保留总数为0，表示重新开始） - $progress = [ - 'status' => 'running', - 'processed_count' => 0, - 'success_count' => 0, - 'error_count' => 0, - 'total_count' => 0, // 总数量会在采集开始时设置 - 'percentage' => 0, - 'start_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - 'end_time' => null, - 'last_sync_time' => null, - ]; - } else { - // 从 pending 状态启动，初始化进度 - $progress['status'] = 'running'; - $progress['start_time'] = new \MongoDB\BSON\UTCDateTime(time() * 1000); - } - - $this->taskRepository->where('task_id', $taskId)->update([ - 'status' => 'running', - 'progress' => $progress, - ]); - - // 清除之前的暂停和停止标志（如果存在） - RedisHelper::del("data_collection_task:{$taskId}:pause"); - RedisHelper::del("data_collection_task:{$taskId}:stop"); - - // 设置Redis标志，通知调度器启动任务 - RedisHelper::set("data_collection_task:{$taskId}:start", '1', 3600); - - LoggerHelper::logBusiness('data_collection_task_started', [ - 'task_id' => $taskId, - 'previous_status' => $task->status, - ]); - - return true; - } - - /** - * 暂停任务 - * - * @param string $taskId 任务ID - * @return bool 是否暂停成功 - */ - public function pauseTask(string $taskId): bool - { - // 使用where查询，因为主键是task_id而不是_id - $task = $this->taskRepository->where('task_id', $taskId)->first(); - - if (!$task) { - throw new \InvalidArgumentException("任务不存在: {$taskId}"); - } - - if ($task->status !== 'running') { - throw new \RuntimeException("任务未在运行中: {$taskId}"); - } - - // 更新任务状态 - // 注意：需要使用完整的数组来更新嵌套字段，timestamps会自动处理updated_at - $progress = $task->progress ?? []; - $progress['status'] = 'paused'; - - $this->taskRepository->where('task_id', $taskId)->update([ - 'status' => 'paused', - 'progress' => $progress, - ]); - - // 设置Redis标志，通知调度器暂停任务 - RedisHelper::set("data_collection_task:{$taskId}:pause", '1', 3600); - - LoggerHelper::logBusiness('data_collection_task_paused', [ - 'task_id' => $taskId, - ]); - - return true; - } - - /** - * 停止任务 - * - * 只允许从以下状态停止： - * - running (运行中) -> stopped - * - paused (已暂停) -> stopped - * - * @param string $taskId 任务ID - * @return bool 是否停止成功 - */ - public function stopTask(string $taskId): bool - { - // 使用where查询，因为主键是task_id而不是_id - $task = $this->taskRepository->where('task_id', $taskId)->first(); - - if (!$task) { - throw new \InvalidArgumentException("任务不存在: {$taskId}"); - } - - // 只允许从 running 或 paused 状态停止 - if (!in_array($task->status, ['running', 'paused'])) { - throw new \RuntimeException("任务当前状态不允许停止: {$taskId} (当前状态: {$task->status})"); - } - - // 停止任务时，保持当前进度，不重置（只更新状态） - $currentProgress = $task->progress ?? []; - $progress = [ - 'status' => 'idle', // idle, running, paused, completed, error - 'processed_count' => $currentProgress['processed_count'] ?? 0, - 'success_count' => $currentProgress['success_count'] ?? 0, - 'error_count' => $currentProgress['error_count'] ?? 0, - 'total_count' => $currentProgress['total_count'] ?? 0, - 'percentage' => $currentProgress['percentage'] ?? 0, // 保持当前进度百分比 - 'start_time' => $currentProgress['start_time'] ?? null, - 'end_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), // 记录停止时间 - 'last_sync_time' => $currentProgress['last_sync_time'] ?? null, - ]; - - $this->taskRepository->where('task_id', $taskId)->update([ - 'status' => 'stopped', - 'progress' => $progress, - ]); - - // 设置Redis标志，通知调度器停止任务 - RedisHelper::set("data_collection_task:{$taskId}:stop", '1', 3600); - - // 如果任务之前是 paused，也需要清除暂停标志 - if ($task->status === 'paused') { - RedisHelper::del("data_collection_task:{$taskId}:pause"); - } - - LoggerHelper::logBusiness('data_collection_task_stopped', [ - 'task_id' => $taskId, - 'previous_status' => $task->status, - 'progress_reset' => true, - ]); - - return true; - } - - /** - * 获取任务列表 - * - * @param array $filters 过滤条件 - * @param int $page 页码 - * @param int $pageSize 每页数量 - * @return array 任务列表 - */ - public function getTaskList(array $filters = [], int $page = 1, int $pageSize = 20): array - { - $query = $this->taskRepository->query(); - - // 应用过滤条件（只处理非空值，如果筛选条件为空则返回所有任务） - if (!empty($filters['status']) && $filters['status'] !== '') { - $query->where('status', $filters['status']); - } - if (!empty($filters['data_source_id']) && $filters['data_source_id'] !== '') { - $query->where('data_source_id', $filters['data_source_id']); - } - if (!empty($filters['name']) && $filters['name'] !== '') { - // MongoDB 使用正则表达式进行模糊查询 - $namePattern = preg_quote($filters['name'], '/'); - $query->where('name', 'regex', "/{$namePattern}/i"); - } - - // 分页 - $total = $query->count(); - $taskModels = $query->orderBy('created_at', 'desc') - ->skip(($page - 1) * $pageSize) - ->take($pageSize) - ->get(); - - // 手动转换为数组，避免 cast 机制对数组字段的错误处理 - $tasks = []; - foreach ($taskModels as $model) { - $task = $model->getAttributes(); - // 使用统一的日期字段处理方法 - $task = $this->normalizeDateFields($task); - $tasks[] = $task; - } - - return [ - 'tasks' => $tasks, - 'total' => $total, - 'page' => $page, - 'page_size' => $pageSize, - 'total_pages' => ceil($total / $pageSize), - ]; - } - - /** - * 获取任务详情 - * - * @param string $taskId 任务ID - * @return array|null 任务详情 - */ - public function getTask(string $taskId): ?array - { - // 使用where查询，因为主键是task_id而不是_id - $task = $this->taskRepository->where('task_id', $taskId)->first(); - - if (!$task) { - return null; - } - - // 手动转换为数组，避免 cast 机制对数组字段的错误处理 - $taskArray = $task->getAttributes(); - // 使用统一的日期字段处理方法 - $taskArray = $this->normalizeDateFields($taskArray); - - return $taskArray; - } - - /** - * 更新任务进度 - * - * @param string $taskId 任务ID - * @param array $progress 进度信息 - * @return bool 是否更新成功 - */ - public function updateProgress(string $taskId, array $progress): bool - { - $updateData = [ - 'progress' => $progress, - ]; - - // 如果进度包含统计信息，也更新统计 - // 注意：这里的统计应该是累加的，但进度字段（processed_count等）应该直接设置 - if (isset($progress['success_count']) || isset($progress['error_count'])) { - // 使用where查询，因为主键是task_id而不是_id - $task = $this->taskRepository->where('task_id', $taskId)->first(); - if ($task) { - $statistics = $task->statistics ?? []; - // 统计信息使用增量更新（累加本次运行的数据） - // 但这里需要判断是增量还是绝对值，如果是绝对值则应该直接设置 - // 由于进度更新传入的是绝对值，所以这里应该直接使用最新值而不是累加 - if (isset($progress['processed_count'])) { - $statistics['total_processed'] = $progress['processed_count']; - } - if (isset($progress['success_count'])) { - $statistics['total_success'] = $progress['success_count']; - } - if (isset($progress['error_count'])) { - $statistics['total_error'] = $progress['error_count']; - } - $statistics['last_run_time'] = new \MongoDB\BSON\UTCDateTime(time() * 1000); - - $updateData['statistics'] = $statistics; - } - } - - // 使用 where()->update() 更新文档 - // 注意：MongoDB Laravel Eloquent 的 update() 返回匹配的文档数量（通常是1或0） - $result = $this->taskRepository->where('task_id', $taskId)->update($updateData); - - // 添加日志以便调试 - if ($result === false || $result === 0) { - \Workerman\Worker::safeEcho("[DataCollectionTaskService] ⚠️ 更新进度失败: task_id={$taskId}, result={$result}\n"); - } else { - \Workerman\Worker::safeEcho("[DataCollectionTaskService] ✅ 更新进度成功: task_id={$taskId}, 匹配文档数={$result}\n"); - } - - return $result > 0; - } - - /** - * 统一处理日期字段，转换为 ISO 8601 字符串格式 - * - * @param array $task 任务数据 - * @return array 处理后的任务数据 - */ - private function normalizeDateFields(array $task): array - { - foreach (['created_at', 'updated_at'] as $dateField) { - if (isset($task[$dateField])) { - if ($task[$dateField] instanceof \MongoDB\BSON\UTCDateTime) { - $task[$dateField] = $task[$dateField]->toDateTime()->format('Y-m-d\TH:i:s.000\Z'); - } elseif ($task[$dateField] instanceof \DateTime || $task[$dateField] instanceof \DateTimeInterface) { - $task[$dateField] = $task[$dateField]->format('Y-m-d\TH:i:s.000\Z'); - } elseif (is_array($task[$dateField]) && isset($task[$dateField]['$date'])) { - // 处理 JSON 编码后的日期格式 - $dateValue = $task[$dateField]['$date']; - if (is_numeric($dateValue)) { - // 如果是数字，假设是毫秒时间戳 - $timestamp = $dateValue / 1000; - $task[$dateField] = date('Y-m-d\TH:i:s.000\Z', (int)$timestamp); - } elseif (is_array($dateValue) && isset($dateValue['$numberLong'])) { - // MongoDB 扩展 JSON 格式：{"$date": {"$numberLong": "1640000000000"}} - $timestamp = intval($dateValue['$numberLong']) / 1000; - $task[$dateField] = date('Y-m-d\TH:i:s.000\Z', (int)$timestamp); - } else { - // 其他格式，尝试解析或保持原样 - $task[$dateField] = is_string($dateValue) ? $dateValue : json_encode($dateValue); - } - } - } - } - return $task; - } - - /** - * 获取所有运行中的任务 - * - * @return array> 运行中的任务列表 - */ - public function getRunningTasks(): array - { - // 使用原生 MongoDB 查询，避免 Model 的 cast 机制导致数组字段被错误处理 - $dbConfig = config('database.connections.mongodb'); - - // 使用 MongoDBHelper 创建客户端（统一DSN构建逻辑） - $client = \app\utils\MongoDBHelper::createClient([ - 'host' => parse_url($dbConfig['dsn'], PHP_URL_HOST) ?? '192.168.1.106', - 'port' => parse_url($dbConfig['dsn'], PHP_URL_PORT) ?? 27017, - 'username' => $dbConfig['username'] ?? '', - 'password' => $dbConfig['password'] ?? '', - 'auth_source' => $dbConfig['options']['authSource'] ?? 'admin', - ], array_filter($dbConfig['options'] ?? [], function ($value) { - return $value !== '' && $value !== null; - })); - - $database = $client->selectDatabase($dbConfig['database']); - $collection = $database->selectCollection('data_collection_tasks'); - - // 查询所有运行中的任务 - $cursor = $collection->find(['status' => 'running']); - - $tasks = []; - foreach ($cursor as $document) { - // MongoDB BSONDocument 需要转换为数组 - if ($document instanceof \MongoDB\Model\BSONDocument) { - $task = json_decode(json_encode($document), true); - } elseif (is_array($document)) { - $task = $document; - } else { - // 其他类型，尝试转换为数组 - $task = (array)$document; - } - - // 处理 MongoDB 的 _id 字段 - if (isset($task['_id'])) { - if (is_object($task['_id'])) { - $task['_id'] = (string)$task['_id']; - } - } - // 使用统一的日期字段处理方法 - $task = $this->normalizeDateFields($task); - $tasks[] = $task; - } - - return $tasks; - } -} - diff --git a/Moncter/Moncter/app/service/DataSource/Adapter/MongoDBAdapter.php b/Moncter/Moncter/app/service/DataSource/Adapter/MongoDBAdapter.php deleted file mode 100644 index b4f50858..00000000 --- a/Moncter/Moncter/app/service/DataSource/Adapter/MongoDBAdapter.php +++ /dev/null @@ -1,309 +0,0 @@ - $config 数据源配置 - * @return bool 是否连接成功 - */ - public function connect(array $config): bool - { - try { - $host = $config['host'] ?? '127.0.0.1'; - $port = (int)($config['port'] ?? 27017); - $this->databaseName = $config['database'] ?? ''; - $username = $config['username'] ?? ''; - $password = $config['password'] ?? ''; - $authSource = $config['auth_source'] ?? $this->databaseName; - - // 构建 DSN - $dsn = "mongodb://"; - if (!empty($username) && !empty($password)) { - $dsn .= urlencode($username) . ':' . urlencode($password) . '@'; - } - $dsn .= "{$host}:{$port}"; - if (!empty($this->databaseName)) { - $dsn .= "/{$this->databaseName}"; - } - if (!empty($authSource)) { - $dsn .= "?authSource=" . urlencode($authSource); - } - - // MongoDB 连接选项 - $options = []; - if (isset($config['options'])) { - $options = array_filter($config['options'], function ($value) { - return $value !== '' && $value !== null; - }); - } - - // 设置超时选项 - if (!isset($options['connectTimeoutMS'])) { - $options['connectTimeoutMS'] = ($config['timeout'] ?? 10) * 1000; - } - if (!isset($options['socketTimeoutMS'])) { - $options['socketTimeoutMS'] = ($config['timeout'] ?? 10) * 1000; - } - - $this->client = new Client($dsn, $options); - - // 选择数据库 - if (!empty($this->databaseName)) { - $this->database = $this->client->selectDatabase($this->databaseName); - } - - // 测试连接 - $this->client->getManager()->selectServer(); - - LoggerHelper::logBusiness('mongodb_adapter_connected', [ - 'host' => $host, - 'port' => $port, - 'database' => $this->databaseName, - ]); - - return true; - } catch (MongoDBException $e) { - LoggerHelper::logError($e, [ - 'component' => 'MongoDBAdapter', - 'action' => 'connect', - 'config' => array_merge($config, ['password' => '***']), // 隐藏密码 - ]); - return false; - } - } - - /** - * 关闭数据库连接 - * - * @return void - */ - public function disconnect(): void - { - if ($this->client !== null) { - $this->client = null; - $this->database = null; - LoggerHelper::logBusiness('mongodb_adapter_disconnected', []); - } - } - - /** - * 测试连接是否有效 - * - * @return bool 连接是否有效 - */ - public function isConnected(): bool - { - if ($this->client === null) { - return false; - } - - try { - // 执行 ping 命令测试连接 - $adminDb = $this->client->selectDatabase('admin'); - $adminDb->command(['ping' => 1]); - return true; - } catch (MongoDBException $e) { - LoggerHelper::logError($e, [ - 'component' => 'MongoDBAdapter', - 'action' => 'isConnected', - ]); - return false; - } - } - - /** - * 执行查询（返回多条记录） - * - * 注意：对于 MongoDB，$sql 参数表示集合名称，$params 是一个包含 'filter' 和 'options' 的数组 - * - * @param string $sql 集合名称（MongoDB 中相当于表名） - * @param array $params 查询参数，格式：['filter' => [...], 'options' => [...]] - * @return array> 查询结果数组 - */ - public function query(string $sql, array $params = []): array - { - if ($this->database === null) { - throw new \RuntimeException('数据库连接未建立或未选择数据库'); - } - - try { - $collection = $sql; // $sql 参数在 MongoDB 中表示集合名 - $filter = $params['filter'] ?? []; - $options = $params['options'] ?? []; - - $cursor = $this->database->selectCollection($collection)->find($filter, $options); - $results = []; - - foreach ($cursor as $document) { - $results[] = $this->convertMongoDocumentToArray($document); - } - - LoggerHelper::logBusiness('mongodb_query_executed', [ - 'collection' => $collection, - 'filter' => $filter, - 'result_count' => count($results), - ]); - - return $results; - } catch (MongoDBException $e) { - LoggerHelper::logError($e, [ - 'component' => 'MongoDBAdapter', - 'action' => 'query', - 'collection' => $sql, - 'params' => $params, - ]); - throw $e; - } - } - - /** - * 执行查询（返回单条记录） - * - * 注意：对于 MongoDB，$sql 参数表示集合名称，$params 是一个包含 'filter' 和 'options' 的数组 - * - * @param string $sql 集合名称 - * @param array $params 查询参数，格式：['filter' => [...], 'options' => [...]] - * @return array|null 查询结果（单条记录）或 null - */ - public function queryOne(string $sql, array $params = []): ?array - { - if ($this->database === null) { - throw new \RuntimeException('数据库连接未建立或未选择数据库'); - } - - try { - $collection = $sql; // $sql 参数在 MongoDB 中表示集合名 - $filter = $params['filter'] ?? []; - $options = $params['options'] ?? []; - - $document = $this->database->selectCollection($collection)->findOne($filter, $options); - - if ($document === null) { - return null; - } - - LoggerHelper::logBusiness('mongodb_query_one_executed', [ - 'collection' => $collection, - 'filter' => $filter, - 'has_result' => true, - ]); - - return $this->convertMongoDocumentToArray($document); - } catch (MongoDBException $e) { - LoggerHelper::logError($e, [ - 'component' => 'MongoDBAdapter', - 'action' => 'queryOne', - 'collection' => $sql, - 'params' => $params, - ]); - throw $e; - } - } - - /** - * 批量查询（分页查询，用于大数据量场景） - * - * 注意：对于 MongoDB，$sql 参数表示集合名称，$params 是一个包含 'filter' 和 'options' 的数组 - * - * @param string $sql 集合名称 - * @param array $params 查询参数，格式：['filter' => [...], 'options' => [...]] - * @param int $offset 偏移量 - * @param int $limit 每页数量 - * @return array> 查询结果数组 - */ - public function queryBatch(string $sql, array $params = [], int $offset = 0, int $limit = 1000): array - { - if ($this->database === null) { - throw new \RuntimeException('数据库连接未建立或未选择数据库'); - } - - try { - $collection = $sql; // $sql 参数在 MongoDB 中表示集合名 - $filter = $params['filter'] ?? []; - $options = $params['options'] ?? []; - - // 设置分页选项 - $options['skip'] = $offset; - $options['limit'] = $limit; - - $cursor = $this->database->selectCollection($collection)->find($filter, $options); - $results = []; - - foreach ($cursor as $document) { - $results[] = $this->convertMongoDocumentToArray($document); - } - - LoggerHelper::logBusiness('mongodb_query_batch_executed', [ - 'collection' => $collection, - 'offset' => $offset, - 'limit' => $limit, - 'result_count' => count($results), - ]); - - return $results; - } catch (MongoDBException $e) { - LoggerHelper::logError($e, [ - 'component' => 'MongoDBAdapter', - 'action' => 'queryBatch', - 'collection' => $sql, - 'params' => $params, - 'offset' => $offset, - 'limit' => $limit, - ]); - throw $e; - } - } - - /** - * 获取数据源类型 - * - * @return string 数据源类型 - */ - public function getType(): string - { - return $this->type; - } - - /** - * 将 MongoDB 文档转换为数组 - * - * @param mixed $document MongoDB 文档对象 - * @return array 数组格式的数据 - */ - private function convertMongoDocumentToArray($document): array - { - if (is_array($document)) { - return $document; - } - - // MongoDB\BSON\Document 或 MongoDB\Model\BSONDocument - if (method_exists($document, 'toArray')) { - return $document->toArray(); - } - - // 转换为数组 - return json_decode(json_encode($document), true) ?? []; - } -} - diff --git a/Moncter/Moncter/app/service/DataSource/Adapter/MySQLAdapter.php b/Moncter/Moncter/app/service/DataSource/Adapter/MySQLAdapter.php deleted file mode 100644 index 966ef98a..00000000 --- a/Moncter/Moncter/app/service/DataSource/Adapter/MySQLAdapter.php +++ /dev/null @@ -1,234 +0,0 @@ - $config 数据源配置 - * @return bool 是否连接成功 - */ - public function connect(array $config): bool - { - try { - $host = $config['host'] ?? '127.0.0.1'; - $port = $config['port'] ?? 3306; - $database = $config['database'] ?? ''; - $username = $config['username'] ?? ''; - $password = $config['password'] ?? ''; - $charset = $config['charset'] ?? 'utf8mb4'; - - // 构建 DSN - $dsn = "mysql:host={$host};port={$port};dbname={$database};charset={$charset}"; - - // PDO 选项 - $options = [ - PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION, - PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC, - PDO::ATTR_EMULATE_PREPARES => false, // 禁用预处理语句模拟 - PDO::ATTR_PERSISTENT => $config['persistent'] ?? false, // 是否持久连接 - PDO::ATTR_TIMEOUT => $config['timeout'] ?? 10, // 连接超时 - ]; - - $this->connection = new PDO($dsn, $username, $password, $options); - - LoggerHelper::logBusiness('mysql_adapter_connected', [ - 'host' => $host, - 'port' => $port, - 'database' => $database, - ]); - - return true; - } catch (PDOException $e) { - LoggerHelper::logError($e, [ - 'component' => 'MySQLAdapter', - 'action' => 'connect', - 'config' => array_merge($config, ['password' => '***']), // 隐藏密码 - ]); - return false; - } - } - - /** - * 关闭数据库连接 - * - * @return void - */ - public function disconnect(): void - { - if ($this->connection !== null) { - $this->connection = null; - LoggerHelper::logBusiness('mysql_adapter_disconnected', []); - } - } - - /** - * 测试连接是否有效 - * - * @return bool 连接是否有效 - */ - public function isConnected(): bool - { - if ($this->connection === null) { - return false; - } - - try { - // 执行简单查询测试连接 - $this->connection->query('SELECT 1'); - return true; - } catch (PDOException $e) { - LoggerHelper::logError($e, [ - 'component' => 'MySQLAdapter', - 'action' => 'isConnected', - ]); - return false; - } - } - - /** - * 执行查询（返回多条记录） - * - * @param string $sql SQL 查询语句 - * @param array $params 查询参数（绑定参数） - * @return array> 查询结果数组 - */ - public function query(string $sql, array $params = []): array - { - if ($this->connection === null) { - throw new \RuntimeException('数据库连接未建立'); - } - - try { - $stmt = $this->connection->prepare($sql); - $stmt->execute($params); - $results = $stmt->fetchAll(PDO::FETCH_ASSOC); - - LoggerHelper::logBusiness('mysql_query_executed', [ - 'sql' => $sql, - 'params_count' => count($params), - 'result_count' => count($results), - ]); - - return $results; - } catch (PDOException $e) { - LoggerHelper::logError($e, [ - 'component' => 'MySQLAdapter', - 'action' => 'query', - 'sql' => $sql, - 'params' => $params, - ]); - throw $e; - } - } - - /** - * 执行查询（返回单条记录） - * - * @param string $sql SQL 查询语句 - * @param array $params 查询参数 - * @return array|null 查询结果（单条记录）或 null - */ - public function queryOne(string $sql, array $params = []): ?array - { - if ($this->connection === null) { - throw new \RuntimeException('数据库连接未建立'); - } - - try { - $stmt = $this->connection->prepare($sql); - $stmt->execute($params); - $result = $stmt->fetch(PDO::FETCH_ASSOC); - - LoggerHelper::logBusiness('mysql_query_one_executed', [ - 'sql' => $sql, - 'params_count' => count($params), - 'has_result' => $result !== false, - ]); - - return $result !== false ? $result : null; - } catch (PDOException $e) { - LoggerHelper::logError($e, [ - 'component' => 'MySQLAdapter', - 'action' => 'queryOne', - 'sql' => $sql, - 'params' => $params, - ]); - throw $e; - } - } - - /** - * 批量查询（分页查询，用于大数据量场景） - * - * @param string $sql SQL 查询语句（需要包含 LIMIT 和 OFFSET，或由适配器自动添加） - * @param array $params 查询参数 - * @param int $offset 偏移量 - * @param int $limit 每页数量 - * @return array> 查询结果数组 - */ - public function queryBatch(string $sql, array $params = [], int $offset = 0, int $limit = 1000): array - { - if ($this->connection === null) { - throw new \RuntimeException('数据库连接未建立'); - } - - try { - // 如果 SQL 中已包含 LIMIT，则直接使用；否则自动添加 - if (stripos($sql, 'LIMIT') === false) { - $sql .= " LIMIT {$limit} OFFSET {$offset}"; - } - - $stmt = $this->connection->prepare($sql); - $stmt->execute($params); - $results = $stmt->fetchAll(PDO::FETCH_ASSOC); - - LoggerHelper::logBusiness('mysql_query_batch_executed', [ - 'sql' => $sql, - 'offset' => $offset, - 'limit' => $limit, - 'result_count' => count($results), - ]); - - return $results; - } catch (PDOException $e) { - LoggerHelper::logError($e, [ - 'component' => 'MySQLAdapter', - 'action' => 'queryBatch', - 'sql' => $sql, - 'params' => $params, - 'offset' => $offset, - 'limit' => $limit, - ]); - throw $e; - } - } - - /** - * 获取数据源类型 - * - * @return string 数据源类型 - */ - public function getType(): string - { - return $this->type; - } -} - diff --git a/Moncter/Moncter/app/service/DataSource/DataSourceAdapterFactory.php b/Moncter/Moncter/app/service/DataSource/DataSourceAdapterFactory.php deleted file mode 100644 index 0935f65a..00000000 --- a/Moncter/Moncter/app/service/DataSource/DataSourceAdapterFactory.php +++ /dev/null @@ -1,116 +0,0 @@ - - */ - private static array $instances = []; - - /** - * 创建数据源适配器 - * - * @param string $type 数据源类型（mysql、postgresql、mongodb 等） - * @param array $config 数据源配置 - * @return DataSourceAdapterInterface 适配器实例 - * @throws \InvalidArgumentException 不支持的数据源类型 - */ - public static function create(string $type, array $config): DataSourceAdapterInterface - { - // 生成缓存键（基于类型和配置） - $cacheKey = self::generateCacheKey($type, $config); - - // 如果已存在实例，直接返回 - if (isset(self::$instances[$cacheKey])) { - $adapter = self::$instances[$cacheKey]; - // 检查连接是否有效 - if ($adapter->isConnected()) { - return $adapter; - } - // 连接已断开，重新创建 - unset(self::$instances[$cacheKey]); - } - - // 根据类型创建适配器 - $adapter = match (strtolower($type)) { - 'mysql' => new MySQLAdapter(), - 'mongodb' => new MongoDBAdapter(), - // 'postgresql' => new PostgreSQLAdapter(), - default => throw new \InvalidArgumentException("不支持的数据源类型: {$type}"), - }; - - // 建立连接 - if (!$adapter->connect($config)) { - throw new \RuntimeException("无法连接到数据源: {$type}"); - } - - // 缓存实例 - self::$instances[$cacheKey] = $adapter; - - LoggerHelper::logBusiness('data_source_adapter_created', [ - 'type' => $type, - 'cache_key' => $cacheKey, - ]); - - return $adapter; - } - - /** - * 生成缓存键 - * - * @param string $type 数据源类型 - * @param array $config 数据源配置 - * @return string 缓存键 - */ - private static function generateCacheKey(string $type, array $config): string - { - // 基于类型、主机、端口、数据库名生成唯一键 - $host = $config['host'] ?? 'unknown'; - $port = $config['port'] ?? 'unknown'; - $database = $config['database'] ?? 'unknown'; - return md5("{$type}:{$host}:{$port}:{$database}"); - } - - /** - * 清除所有适配器实例（用于测试或重新连接） - * - * @return void - */ - public static function clearInstances(): void - { - foreach (self::$instances as $adapter) { - try { - $adapter->disconnect(); - } catch (\Throwable $e) { - LoggerHelper::logError($e, ['component' => 'DataSourceAdapterFactory', 'action' => 'clearInstances']); - } - } - self::$instances = []; - } - - /** - * 获取所有已创建的适配器实例 - * - * @return array - */ - public static function getInstances(): array - { - return self::$instances; - } -} - diff --git a/Moncter/Moncter/app/service/DataSource/DataSourceAdapterInterface.php b/Moncter/Moncter/app/service/DataSource/DataSourceAdapterInterface.php deleted file mode 100644 index 2d160585..00000000 --- a/Moncter/Moncter/app/service/DataSource/DataSourceAdapterInterface.php +++ /dev/null @@ -1,73 +0,0 @@ - $config 数据源配置 - * @return bool 是否连接成功 - */ - public function connect(array $config): bool; - - /** - * 关闭数据库连接 - * - * @return void - */ - public function disconnect(): void; - - /** - * 测试连接是否有效 - * - * @return bool 连接是否有效 - */ - public function isConnected(): bool; - - /** - * 执行查询（返回多条记录） - * - * @param string $sql SQL 查询语句（或 MongoDB 查询条件） - * @param array $params 查询参数（绑定参数或 MongoDB 查询选项） - * @return array> 查询结果数组 - */ - public function query(string $sql, array $params = []): array; - - /** - * 执行查询（返回单条记录） - * - * @param string $sql SQL 查询语句（或 MongoDB 查询条件） - * @param array $params 查询参数 - * @return array|null 查询结果（单条记录）或 null - */ - public function queryOne(string $sql, array $params = []): ?array; - - /** - * 批量查询（分页查询，用于大数据量场景） - * - * @param string $sql SQL 查询语句 - * @param array $params 查询参数 - * @param int $offset 偏移量 - * @param int $limit 每页数量 - * @return array> 查询结果数组 - */ - public function queryBatch(string $sql, array $params = [], int $offset = 0, int $limit = 1000): array; - - /** - * 获取数据源类型 - * - * @return string 数据源类型（mysql、postgresql、mongodb 等） - */ - public function getType(): string; -} - diff --git a/Moncter/Moncter/app/service/DataSource/PollingStrategyFactory.php b/Moncter/Moncter/app/service/DataSource/PollingStrategyFactory.php deleted file mode 100644 index bb79b1e8..00000000 --- a/Moncter/Moncter/app/service/DataSource/PollingStrategyFactory.php +++ /dev/null @@ -1,68 +0,0 @@ - $strategyConfig 策略配置（字符串为策略类名，数组包含 class 和 config） - * @return PollingStrategyInterface 策略实例 - * @throws \InvalidArgumentException 无效的策略配置 - */ - public static function create(string|array $strategyConfig): PollingStrategyInterface - { - // 如果配置是字符串，则作为策略类名 - if (is_string($strategyConfig)) { - $className = $strategyConfig; - $strategyConfig = ['class' => $className]; - } - - // 获取策略类名 - $className = $strategyConfig['class'] ?? null; - if (!$className) { - // 如果没有指定策略，使用默认策略 - $className = DefaultConsumptionStrategy::class; - } - - // 验证类是否存在 - if (!class_exists($className)) { - throw new \InvalidArgumentException("策略类不存在: {$className}"); - } - - // 验证类是否实现了接口 - if (!is_subclass_of($className, PollingStrategyInterface::class)) { - throw new \InvalidArgumentException("策略类必须实现 PollingStrategyInterface: {$className}"); - } - - // 创建策略实例 - try { - $strategy = new $className(); - - LoggerHelper::logBusiness('polling_strategy_created', [ - 'class' => $className, - ]); - - return $strategy; - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'PollingStrategyFactory', - 'action' => 'create', - 'class' => $className, - ]); - throw new \RuntimeException("无法创建策略实例: {$className}", 0, $e); - } - } -} - diff --git a/Moncter/Moncter/app/service/DataSource/PollingStrategyInterface.php b/Moncter/Moncter/app/service/DataSource/PollingStrategyInterface.php deleted file mode 100644 index fda21115..00000000 --- a/Moncter/Moncter/app/service/DataSource/PollingStrategyInterface.php +++ /dev/null @@ -1,54 +0,0 @@ - $config 数据源配置 - * @param array $lastSyncInfo 上次同步信息（包含 last_sync_time、last_sync_id 等） - * @return array> 查询结果数组（原始数据） - */ - public function poll( - DataSourceAdapterInterface $adapter, - array $config, - array $lastSyncInfo = [] - ): array; - - /** - * 数据转换 - * - * @param array> $rawData 原始数据 - * @param array $config 数据源配置 - * @return array> 转换后的数据（标准格式） - */ - public function transform(array $rawData, array $config): array; - - /** - * 数据验证 - * - * @param array $record 单条记录 - * @param array $config 数据源配置 - * @return bool 是否通过验证 - */ - public function validate(array $record, array $config): bool; - - /** - * 获取策略名称 - * - * @return string 策略名称 - */ - public function getName(): string; -} - diff --git a/Moncter/Moncter/app/service/DataSource/Strategy/DefaultConsumptionStrategy.php b/Moncter/Moncter/app/service/DataSource/Strategy/DefaultConsumptionStrategy.php deleted file mode 100644 index dddf79c2..00000000 --- a/Moncter/Moncter/app/service/DataSource/Strategy/DefaultConsumptionStrategy.php +++ /dev/null @@ -1,197 +0,0 @@ - $config 数据源配置 - * @param array $lastSyncInfo 上次同步信息 - * @return array> 查询结果数组 - */ - public function poll( - DataSourceAdapterInterface $adapter, - array $config, - array $lastSyncInfo = [] - ): array { - // 从配置中获取表名和查询条件 - $tableName = $config['table'] ?? 'consumption_records'; - $lastSyncTime = $lastSyncInfo['last_sync_time'] ?? null; - $lastSyncId = $lastSyncInfo['last_sync_id'] ?? null; - - // 构建 SQL 查询（增量查询） - $sql = "SELECT * FROM `{$tableName}` WHERE 1=1"; - $params = []; - - // 如果有上次同步时间，只查询新增或更新的记录 - if ($lastSyncTime !== null) { - $sql .= " AND (`created_at` > :last_sync_time OR `updated_at` > :last_sync_time)"; - $params[':last_sync_time'] = $lastSyncTime; - } - - // 如果有上次同步ID，用于去重（可选） - if ($lastSyncId !== null) { - $sql .= " AND `id` > :last_sync_id"; - $params[':last_sync_id'] = $lastSyncId; - } - - // 按创建时间排序 - $sql .= " ORDER BY `created_at` ASC, `id` ASC"; - - // 执行查询（批量查询，每次最多1000条） - $limit = $config['batch_size'] ?? 1000; - $offset = 0; - $allResults = []; - - do { - $batchSql = $sql . " LIMIT {$limit} OFFSET {$offset}"; - $results = $adapter->queryBatch($batchSql, $params, $offset, $limit); - - if (empty($results)) { - break; - } - - $allResults = array_merge($allResults, $results); - $offset += $limit; - - // 防止无限循环（最多查询10万条） - if (count($allResults) >= 100000) { - LoggerHelper::logBusiness('polling_batch_limit_reached', [ - 'table' => $tableName, - 'count' => count($allResults), - ]); - break; - } - } while (count($results) === $limit); - - LoggerHelper::logBusiness('polling_query_completed', [ - 'table' => $tableName, - 'result_count' => count($allResults), - 'last_sync_time' => $lastSyncTime, - ]); - - return $allResults; - } - - /** - * 数据转换 - * - * @param array> $rawData 原始数据 - * @param array $config 数据源配置 - * @return array> 转换后的数据 - */ - public function transform(array $rawData, array $config): array - { - // 字段映射配置（从外部数据库字段映射到标准字段） - $fieldMapping = $config['field_mapping'] ?? [ - // 默认映射（如果外部数据库字段名与标准字段名一致，则无需映射） - 'id' => 'id', - 'user_id' => 'user_id', - 'amount' => 'amount', - 'store_id' => 'store_id', - 'product_id' => 'product_id', - 'consume_time' => 'consume_time', - 'created_at' => 'created_at', - ]; - - $transformedData = []; - - foreach ($rawData as $record) { - $transformed = []; - - // 应用字段映射 - foreach ($fieldMapping as $standardField => $sourceField) { - if (isset($record[$sourceField])) { - $transformed[$standardField] = $record[$sourceField]; - } - } - - // 确保必要字段存在 - if (!empty($transformed)) { - $transformedData[] = $transformed; - } - } - - LoggerHelper::logBusiness('polling_transform_completed', [ - 'input_count' => count($rawData), - 'output_count' => count($transformedData), - ]); - - return $transformedData; - } - - /** - * 数据验证 - * - * @param array $record 单条记录 - * @param array $config 数据源配置 - * @return bool 是否通过验证 - */ - public function validate(array $record, array $config): bool - { - // 必填字段验证 - $requiredFields = $config['required_fields'] ?? ['user_id', 'amount', 'consume_time']; - - foreach ($requiredFields as $field) { - if (!isset($record[$field]) || $record[$field] === null || $record[$field] === '') { - LoggerHelper::logBusiness('polling_validation_failed', [ - 'reason' => "缺少必填字段: {$field}", - 'record' => $record, - ]); - return false; - } - } - - // 金额验证（必须为正数） - if (isset($record['amount'])) { - $amount = (float)$record['amount']; - if ($amount <= 0) { - LoggerHelper::logBusiness('polling_validation_failed', [ - 'reason' => '金额必须大于0', - 'amount' => $amount, - ]); - return false; - } - } - - // 时间格式验证（可选） - if (isset($record['consume_time'])) { - $time = strtotime($record['consume_time']); - if ($time === false) { - LoggerHelper::logBusiness('polling_validation_failed', [ - 'reason' => '时间格式无效', - 'consume_time' => $record['consume_time'], - ]); - return false; - } - } - - return true; - } - - /** - * 获取策略名称 - * - * @return string 策略名称 - */ - public function getName(): string - { - return 'default_consumption'; - } -} - diff --git a/Moncter/Moncter/app/service/DataSource/Strategy/MongoDBConsumptionStrategy.php b/Moncter/Moncter/app/service/DataSource/Strategy/MongoDBConsumptionStrategy.php deleted file mode 100644 index d6cc4951..00000000 --- a/Moncter/Moncter/app/service/DataSource/Strategy/MongoDBConsumptionStrategy.php +++ /dev/null @@ -1,225 +0,0 @@ - $config 数据源配置 - * @param array $lastSyncInfo 上次同步信息 - * @return array> 查询结果数组 - */ - public function poll( - DataSourceAdapterInterface $adapter, - array $config, - array $lastSyncInfo = [] - ): array { - // 从配置中获取集合名和查询条件 - $collectionName = $config['collection'] ?? 'consumption_records'; - $lastSyncTime = $lastSyncInfo['last_sync_time'] ?? null; - $lastSyncId = $lastSyncInfo['last_sync_id'] ?? null; - - // 构建 MongoDB 查询过滤器 - $filter = []; - - // 如果有上次同步时间，只查询新增或更新的记录 - if ($lastSyncTime !== null) { - $lastSyncTimestamp = is_numeric($lastSyncTime) ? (int)$lastSyncTime : strtotime($lastSyncTime); - $lastSyncDate = new \MongoDB\BSON\UTCDateTime($lastSyncTimestamp * 1000); - - $filter['$or'] = [ - ['created_at' => ['$gt' => $lastSyncDate]], - ['updated_at' => ['$gt' => $lastSyncDate]], - ]; - } - - // 如果有上次同步ID，用于去重（可选） - if ($lastSyncId !== null) { - $filter['_id'] = ['$gt' => $lastSyncId]; - } - - // 查询选项 - $options = [ - 'sort' => ['created_at' => 1, '_id' => 1], // 按创建时间和ID排序 - ]; - - // 执行查询（批量查询，每次最多1000条） - $limit = $config['batch_size'] ?? 1000; - $offset = 0; - $allResults = []; - - do { - // MongoDB 适配器的 queryBatch 方法签名：queryBatch(string $sql, array $params = [], int $offset = 0, int $limit = 1000) - // 对于 MongoDB，$sql 是集合名，$params 包含 'filter' 和 'options' - $results = $adapter->queryBatch($collectionName, [ - 'filter' => $filter, - 'options' => $options, - ], $offset, $limit); - - if (empty($results)) { - break; - } - - $allResults = array_merge($allResults, $results); - $offset += $limit; - - // 防止无限循环（最多查询10万条） - if (count($allResults) >= 100000) { - LoggerHelper::logBusiness('polling_batch_limit_reached', [ - 'collection' => $collectionName, - 'count' => count($allResults), - ]); - break; - } - } while (count($results) === $limit); - - LoggerHelper::logBusiness('polling_query_completed', [ - 'collection' => $collectionName, - 'result_count' => count($allResults), - 'last_sync_time' => $lastSyncTime, - ]); - - return $allResults; - } - - /** - * 数据转换 - * - * @param array> $rawData 原始数据 - * @param array $config 数据源配置 - * @return array> 转换后的数据 - */ - public function transform(array $rawData, array $config): array - { - // 字段映射配置（从外部数据库字段映射到标准字段） - $fieldMapping = $config['field_mapping'] ?? [ - // 默认映射（MongoDB 使用 _id，需要转换为 id） - '_id' => 'id', - 'user_id' => 'user_id', - 'amount' => 'amount', - 'store_id' => 'store_id', - 'product_id' => 'product_id', - 'consume_time' => 'consume_time', - 'created_at' => 'created_at', - ]; - - $transformedData = []; - - foreach ($rawData as $record) { - $transformed = []; - - // 处理 MongoDB 的 _id 字段（转换为字符串） - if (isset($record['_id'])) { - if (is_object($record['_id']) && method_exists($record['_id'], '__toString')) { - $record['id'] = (string)$record['_id']; - } else { - $record['id'] = (string)$record['_id']; - } - } - - // 处理 MongoDB 的日期字段（UTCDateTime 转换为字符串） - foreach (['created_at', 'updated_at', 'consume_time'] as $dateField) { - if (isset($record[$dateField])) { - if (is_object($record[$dateField]) && method_exists($record[$dateField], 'toDateTime')) { - $record[$dateField] = $record[$dateField]->toDateTime()->format('Y-m-d H:i:s'); - } elseif (is_object($record[$dateField]) && method_exists($record[$dateField], '__toString')) { - $record[$dateField] = (string)$record[$dateField]; - } - } - } - - // 应用字段映射 - foreach ($fieldMapping as $standardField => $sourceField) { - if (isset($record[$sourceField])) { - $transformed[$standardField] = $record[$sourceField]; - } - } - - // 确保必要字段存在 - if (!empty($transformed)) { - $transformedData[] = $transformed; - } - } - - LoggerHelper::logBusiness('polling_transform_completed', [ - 'input_count' => count($rawData), - 'output_count' => count($transformedData), - ]); - - return $transformedData; - } - - /** - * 数据验证 - * - * @param array $record 单条记录 - * @param array $config 数据源配置 - * @return bool 是否通过验证 - */ - public function validate(array $record, array $config): bool - { - // 必填字段验证 - $requiredFields = $config['required_fields'] ?? ['user_id', 'amount', 'consume_time']; - - foreach ($requiredFields as $field) { - if (!isset($record[$field]) || $record[$field] === null || $record[$field] === '') { - LoggerHelper::logBusiness('polling_validation_failed', [ - 'reason' => "缺少必填字段: {$field}", - 'record' => $record, - ]); - return false; - } - } - - // 金额验证（必须为正数） - if (isset($record['amount'])) { - $amount = (float)$record['amount']; - if ($amount <= 0) { - LoggerHelper::logBusiness('polling_validation_failed', [ - 'reason' => '金额必须大于0', - 'amount' => $amount, - ]); - return false; - } - } - - // 时间格式验证（可选） - if (isset($record['consume_time'])) { - $time = strtotime($record['consume_time']); - if ($time === false) { - LoggerHelper::logBusiness('polling_validation_failed', [ - 'reason' => '时间格式无效', - 'consume_time' => $record['consume_time'], - ]); - return false; - } - } - - return true; - } - - /** - * 获取策略名称 - * - * @return string 策略名称 - */ - public function getName(): string - { - return 'mongodb_consumption'; - } -} - diff --git a/Moncter/Moncter/app/service/DataSourceService.php b/Moncter/Moncter/app/service/DataSourceService.php deleted file mode 100644 index b27396e3..00000000 --- a/Moncter/Moncter/app/service/DataSourceService.php +++ /dev/null @@ -1,498 +0,0 @@ - $data - * @return DataSourceRepository - * @throws \Exception - */ - public function createDataSource(array $data): DataSourceRepository - { - // 生成ID - if (empty($data['data_source_id'])) { - $data['data_source_id'] = UuidGenerator::uuid4()->toString(); - } - - // 验证必填字段 - $requiredFields = ['name', 'type', 'host', 'port', 'database']; - foreach ($requiredFields as $field) { - if (empty($data[$field])) { - throw new \InvalidArgumentException("缺少必填字段: {$field}"); - } - } - - // 验证类型 - $allowedTypes = ['mongodb', 'mysql', 'postgresql']; - if (!in_array(strtolower($data['type']), $allowedTypes)) { - throw new \InvalidArgumentException("不支持的数据源类型: {$data['type']}"); - } - - // 验证ID唯一性 - $existing = $this->repository->newQuery() - ->where('data_source_id', $data['data_source_id']) - ->first(); - - if ($existing) { - throw new \InvalidArgumentException("数据源ID已存在: {$data['data_source_id']}"); - } - - // 验证名称唯一性 - $existingByName = $this->repository->newQuery() - ->where('name', $data['name']) - ->first(); - - if ($existingByName) { - throw new \InvalidArgumentException("数据源名称已存在: {$data['name']}"); - } - - // 设置默认值 - $data['status'] = $data['status'] ?? 1; // 1:启用, 0:禁用 - $data['options'] = $data['options'] ?? []; - $data['is_tag_engine'] = $data['is_tag_engine'] ?? false; // 默认不是标签引擎数据库 - - // 创建数据源 - $dataSource = new DataSourceRepository($data); - $dataSource->save(); - - // 如果设置为标签引擎数据库，自动将其他数据源设置为 false（确保只有一个） - if (!empty($data['is_tag_engine'])) { - // 将所有其他数据源的 is_tag_engine 设置为 false - $this->repository->newQuery() - ->where('data_source_id', '!=', $dataSource->data_source_id) - ->update(['is_tag_engine' => false]); - - LoggerHelper::logBusiness('tag_engine_set', [ - 'data_source_id' => $dataSource->data_source_id, - 'action' => 'create', - ]); - } - - LoggerHelper::logBusiness('data_source_created', [ - 'data_source_id' => $dataSource->data_source_id, - 'name' => $dataSource->name, - 'type' => $dataSource->type, - ]); - - return $dataSource; - } - - /** - * 更新数据源 - * - * @param string $dataSourceId - * @param array $data - * @return bool - */ - public function updateDataSource(string $dataSourceId, array $data): bool - { - $dataSource = $this->repository->find($dataSourceId); - - if (!$dataSource) { - throw new \InvalidArgumentException("数据源不存在: {$dataSourceId}"); - } - - // 如果更新名称，验证唯一性 - if (isset($data['name']) && $data['name'] !== $dataSource->name) { - $existing = $this->repository->newQuery() - ->where('name', $data['name']) - ->where('data_source_id', '!=', $dataSourceId) - ->first(); - - if ($existing) { - throw new \InvalidArgumentException("数据源名称已存在: {$data['name']}"); - } - } - - // 如果设置为标签引擎数据库，自动将其他数据源设置为 false（确保只有一个） - if (isset($data['is_tag_engine']) && !empty($data['is_tag_engine'])) { - // 将所有其他数据源的 is_tag_engine 设置为 false - $this->repository->newQuery() - ->where('data_source_id', '!=', $dataSourceId) - ->update(['is_tag_engine' => false]); - - LoggerHelper::logBusiness('tag_engine_set', [ - 'data_source_id' => $dataSourceId, - 'action' => 'update', - ]); - } - - // 更新数据 - $dataSource->fill($data); - $result = $dataSource->save(); - - if ($result) { - LoggerHelper::logBusiness('data_source_updated', [ - 'data_source_id' => $dataSourceId, - ]); - } - - return $result; - } - - /** - * 删除数据源 - * - * @param string $dataSourceId - * @return bool - */ - public function deleteDataSource(string $dataSourceId): bool - { - $dataSource = $this->repository->find($dataSourceId); - - if (!$dataSource) { - throw new \InvalidArgumentException("数据源不存在: {$dataSourceId}"); - } - - // TODO: 检查是否有任务在使用此数据源 - // 可以查询 DataCollectionTask 中是否有引用此数据源 - - $result = $dataSource->delete(); - - if ($result) { - LoggerHelper::logBusiness('data_source_deleted', [ - 'data_source_id' => $dataSourceId, - ]); - } - - return $result; - } - - /** - * 获取数据源列表 - * - * @param array $filters - * @return array{list: array, total: int} - */ - public function getDataSourceList(array $filters = []): array - { - try { - $query = $this->repository->newQuery(); - - // 筛选条件 - if (isset($filters['type'])) { - $query->where('type', $filters['type']); - } - - if (isset($filters['status'])) { - $query->where('status', $filters['status']); - } - - if (isset($filters['name'])) { - $query->where('name', 'like', '%' . $filters['name'] . '%'); - } - - // 排序 - $query->orderBy('created_at', 'desc'); - - // 分页 - $page = (int)($filters['page'] ?? 1); - $pageSize = (int)($filters['page_size'] ?? 20); - - $total = $query->count(); - $list = $query->skip(($page - 1) * $pageSize) - ->take($pageSize) - ->get() - ->map(function ($item) { - // 不返回密码 - $data = $item->toArray(); - unset($data['password']); - return $data; - }) - ->toArray(); - - return [ - 'list' => $list, - 'total' => $total, - ]; - } catch (\MongoDB\Driver\Exception\Exception $e) { - // MongoDB 连接错误 - LoggerHelper::logError($e, [ - 'component' => 'DataSourceService', - 'action' => 'getDataSourceList', - ]); - throw new \RuntimeException('无法连接到 MongoDB 数据库，请检查数据库服务是否正常运行', 500, $e); - } catch (\Exception $e) { - LoggerHelper::logError($e, [ - 'component' => 'DataSourceService', - 'action' => 'getDataSourceList', - ]); - throw $e; - } - } - - /** - * 获取数据源详情（不包含密码） - * - * @param string $dataSourceId - * @return array|null - */ - public function getDataSourceDetail(string $dataSourceId): ?array - { - $dataSource = $this->repository->find($dataSourceId); - - if (!$dataSource) { - return null; - } - - $data = $dataSource->toArray(); - unset($data['password']); - - return $data; - } - - /** - * 获取数据源详情（包含密码，用于连接） - * - * @param string $dataSourceId - * @return array|null - */ - public function getDataSourceConfig(string $dataSourceId): ?array - { - $dataSource = $this->repository->find($dataSourceId); - - if (!$dataSource) { - return null; - } - - if ($dataSource->status != 1) { - throw new \RuntimeException("数据源已禁用: {$dataSourceId}"); - } - - return $dataSource->toConfigArray(); - } - - /** - * 测试数据源连接 - * - * @param array $config - * @return bool - */ - public function testConnection(array $config): bool - { - try { - $type = strtolower($config['type'] ?? ''); - - // MongoDB特殊处理 - if ($type === 'mongodb') { - // 使用 MongoDBHelper 创建客户端（统一DSN构建逻辑） - $client = \app\utils\MongoDBHelper::createClient($config, [ - 'connectTimeoutMS' => 3000, - 'socketTimeoutMS' => 5000, - ]); - - // 尝试列出数据库来测试连接 - $client->listDatabases(); - return true; - } - - // 其他类型使用适配器 - $adapter = DataSourceAdapterFactory::create($type, $config); - $connected = $adapter->isConnected(); - $adapter->disconnect(); - return $connected; - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DataSourceService', - 'action' => 'testConnection', - ]); - return false; - } - } - - /** - * 获取所有启用的数据源（用于替代config('data_sources')，从数据库读取） - * - * @return array 以data_source_id为key的配置数组 - */ - public function getAllEnabledDataSources(): array - { - $dataSources = $this->repository->newQuery() - ->where('status', 1) - ->get(); - - $result = []; - foreach ($dataSources as $ds) { - $result[$ds->data_source_id] = $ds->toConfigArray(); - } - - return $result; - } - - /** - * 根据数据源ID获取配置（从数据库读取） - * - * 支持两种查询方式： - * 1. 通过 data_source_id (UUID) 查询 - * 2. 通过 name 字段查询（兼容配置文件中的 key，如 sync_mongodb, tag_mongodb） - * - * @param string $dataSourceId 数据源ID或名称 - * @return array|null 数据源配置，不存在或禁用时返回null - */ - public function getDataSourceConfigById(string $dataSourceId): ?array - { - // \Workerman\Worker::safeEcho("[DataSourceService] 查询数据源配置: data_source_id={$dataSourceId}\n"); - - // 先尝试通过 data_source_id 查询（UUID 格式） - $dataSource = $this->repository->newQuery() - ->where('data_source_id', $dataSourceId) - ->where('status', 1) - ->first(); - - if ($dataSource) { - // \Workerman\Worker::safeEcho("[DataSourceService] ✓ 通过 data_source_id 查询成功: name={$dataSource->name}\n"); - return $dataSource->toConfigArray(); - } - - // 如果通过 data_source_id 查不到，尝试通过 name 字段查询（兼容配置文件中的 key） - // \Workerman\Worker::safeEcho("[DataSourceService] 通过 data_source_id 未找到，尝试通过 name 查询\n"); - - // 处理配置文件中的常见 key 映射 - // 注意：这些映射需要根据实际数据库中的 name 字段值来调整 - $nameMapping = [ - 'sync_mongodb' => '本地大数据库', // 根据实际数据库中的名称调整 - 'tag_mongodb' => '主数据库', // 标签引擎数据库（is_tag_engine=true） - 'kr_mongodb' => '卡若的主机', // 卡若数据库 - ]; - - $searchName = $nameMapping[$dataSourceId] ?? null; - - if ($searchName) { - // \Workerman\Worker::safeEcho("[DataSourceService] 使用映射名称查询: {$dataSourceId} -> {$searchName}\n"); - // 使用映射的名称查询 - $dataSource = $this->repository->newQuery() - ->where('name', $searchName) - ->where('status', 1) - ->first(); - - if ($dataSource) { - // \Workerman\Worker::safeEcho("[DataSourceService] ✓ 通过映射名称查询成功: name={$dataSource->name}, data_source_id={$dataSource->data_source_id}\n"); - return $dataSource->toConfigArray(); - } - } - - // 如果还是查不到，尝试直接使用 dataSourceId 作为 name 查询 - // \Workerman\Worker::safeEcho("[DataSourceService] 尝试直接使用 dataSourceId 作为 name 查询: {$dataSourceId}\n"); - $dataSource = $this->repository->newQuery() - ->where('name', $dataSourceId) - ->where('status', 1) - ->first(); - - if ($dataSource) { - // \Workerman\Worker::safeEcho("[DataSourceService] ✓ 通过 name 直接查询成功: name={$dataSource->name}\n"); - return $dataSource->toConfigArray(); - } - - // 如果还是查不到，对于 tag_mongodb，尝试查询 is_tag_engine=true 的数据源 - if ($dataSourceId === 'tag_mongodb') { - // \Workerman\Worker::safeEcho("[DataSourceService] 对于 tag_mongodb，尝试查询 is_tag_engine=true 的数据源\n"); - $dataSource = $this->repository->newQuery() - ->where('is_tag_engine', true) - ->where('status', 1) - ->first(); - - if ($dataSource) { - // \Workerman\Worker::safeEcho("[DataSourceService] ✓ 通过 is_tag_engine 查询成功: name={$dataSource->name}, data_source_id={$dataSource->data_source_id}\n"); - return $dataSource->toConfigArray(); - } - } - - // \Workerman\Worker::safeEcho("[DataSourceService] ✗ 未找到数据源配置: data_source_id={$dataSourceId}\n"); - return null; - } - - /** - * 获取标签引擎数据库配置（is_tag_engine = true的数据源） - * - * @return array|null 标签引擎数据库配置，未找到时返回null - */ - public function getTagEngineDataSourceConfig(): ?array - { - $dataSource = $this->repository->newQuery() - ->where('is_tag_engine', true) - ->where('status', 1) - ->first(); - - if (!$dataSource) { - return null; - } - - return $dataSource->toConfigArray(); - } - - /** - * 获取标签引擎数据库的data_source_id - * - * @return string|null 标签引擎数据库的data_source_id，未找到时返回null - */ - public function getTagEngineDataSourceId(): ?string - { - $dataSource = $this->repository->newQuery() - ->where('is_tag_engine', true) - ->where('status', 1) - ->first(); - - return $dataSource ? $dataSource->data_source_id : null; - } - - /** - * 验证标签引擎数据库配置是否存在 - * - * @return bool 是否存在标签引擎数据库 - */ - public function hasTagEngineDataSource(): bool - { - $count = $this->repository->newQuery() - ->where('is_tag_engine', true) - ->where('status', 1) - ->count(); - - return $count > 0; - } - - /** - * 获取所有标签引擎数据库（理论上应该只有一个，但允许有多个） - * - * @return array 标签引擎数据库列表 - */ - public function getAllTagEngineDataSources(): array - { - $dataSources = $this->repository->newQuery() - ->where('is_tag_engine', true) - ->where('status', 1) - ->get(); - - $result = []; - foreach ($dataSources as $ds) { - $data = $ds->toArray(); - unset($data['password']); // 不返回密码 - $result[] = $data; - } - - return $result; - } -} - diff --git a/Moncter/Moncter/app/service/DataSyncService.php b/Moncter/Moncter/app/service/DataSyncService.php deleted file mode 100644 index 7c1ee3eb..00000000 --- a/Moncter/Moncter/app/service/DataSyncService.php +++ /dev/null @@ -1,242 +0,0 @@ - $messageData 消息数据（包含 source_id、data 等） - * @return array 同步结果 - */ - public function syncData(array $messageData): array - { - $sourceId = $messageData['source_id'] ?? 'unknown'; - $data = $messageData['data'] ?? []; - $count = count($data); - - if (empty($data)) { - LoggerHelper::logBusiness('data_sync_empty', [ - 'source_id' => $sourceId, - ]); - return [ - 'success' => true, - 'synced_count' => 0, - 'skipped_count' => 0, - ]; - } - - LoggerHelper::logBusiness('data_sync_service_started', [ - 'source_id' => $sourceId, - 'data_count' => $count, - ]); - - $syncedCount = 0; - $skippedCount = 0; - $userIds = []; - - // 批量写入消费记录 - foreach ($data as $record) { - try { - // 数据验证 - if (!$this->validateRecord($record)) { - $skippedCount++; - continue; - } - - // 确保有 record_id - if (empty($record['record_id'])) { - $record['record_id'] = (string)Uuid::uuid4(); - } - - // 写入消费记录（使用 Eloquent Model 方式） - $consumptionRecord = new ConsumptionRecordRepository(); - $consumptionRecord->record_id = $record['record_id'] ?? (string)Uuid::uuid4(); - $consumptionRecord->user_id = $record['user_id']; - $consumptionRecord->consume_time = new \DateTimeImmutable($record['consume_time']); - $consumptionRecord->amount = (float)($record['amount'] ?? 0); - $consumptionRecord->actual_amount = (float)($record['actual_amount'] ?? $record['amount'] ?? 0); - $consumptionRecord->currency = $record['currency'] ?? 'CNY'; - $consumptionRecord->store_id = $record['store_id'] ?? ''; - $consumptionRecord->status = $record['status'] ?? 0; - $consumptionRecord->create_time = new \DateTimeImmutable('now'); - $consumptionRecord->save(); - $syncedCount++; - - // 收集用户ID（用于后续批量更新统计） - $userId = $record['user_id'] ?? null; - if ($userId) { - $userIds[] = $userId; - } - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DataSyncService', - 'action' => 'syncData', - 'source_id' => $sourceId, - 'record' => $record, - ]); - $skippedCount++; - } - } - - // 批量更新用户统计（去重） - $uniqueUserIds = array_unique($userIds); - foreach ($uniqueUserIds as $userId) { - try { - $this->updateUserStatistics($userId); - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DataSyncService', - 'action' => 'updateUserStatistics', - 'user_id' => $userId, - ]); - } - } - - // 触发标签计算（为每个用户推送消息） - $tagCalculationCount = 0; - foreach ($uniqueUserIds as $userId) { - try { - $message = [ - 'user_id' => $userId, - 'tag_ids' => null, // null 表示计算所有 real_time 标签 - 'trigger_type' => 'data_sync', - 'source_id' => $sourceId, - 'timestamp' => time(), - ]; - - if (QueueService::pushTagCalculation($message)) { - $tagCalculationCount++; - } - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'DataSyncService', - 'action' => 'triggerTagCalculation', - 'user_id' => $userId, - ]); - } - } - - $result = [ - 'success' => true, - 'synced_count' => $syncedCount, - 'skipped_count' => $skippedCount, - 'user_count' => count($uniqueUserIds), - 'tag_calculation_triggered' => $tagCalculationCount, - ]; - - LoggerHelper::logBusiness('data_sync_service_completed', array_merge([ - 'source_id' => $sourceId, - ], $result)); - - return $result; - } - - /** - * 验证记录 - * - * @param array $record 记录数据 - * @return bool 是否通过验证 - */ - private function validateRecord(array $record): bool - { - // 必填字段验证 - $requiredFields = ['user_id', 'amount', 'consume_time']; - foreach ($requiredFields as $field) { - if (!isset($record[$field]) || $record[$field] === null || $record[$field] === '') { - return false; - } - } - - // 金额验证 - $amount = (float)($record['amount'] ?? 0); - if ($amount <= 0) { - return false; - } - - // 时间格式验证 - $consumeTime = $record['consume_time'] ?? ''; - if (strtotime($consumeTime) === false) { - return false; - } - - return true; - } - - /** - * 更新用户统计 - * - * @param string $userId 用户ID - * @return void - */ - private function updateUserStatistics(string $userId): void - { - // 获取用户的所有消费记录（用于重新计算统计） - // 这里简化处理，只更新最近的数据 - // 实际场景中，可以增量更新或全量重新计算 - - // 查询用户最近的消费记录（使用 Eloquent 查询） - $records = ConsumptionRecordRepository::where('user_id', $userId) - ->orderBy('consume_time', 'desc') - ->limit(1000) - ->get() - ->toArray(); - - if (empty($records)) { - return; - } - - // 计算统计值 - $totalAmount = 0; - $totalCount = count($records); - $lastConsumeTime = null; - - foreach ($records as $record) { - $amount = (float)($record['amount'] ?? 0); - $totalAmount += $amount; - - $consumeTime = $record['consume_time'] ?? null; - if ($consumeTime) { - $time = strtotime($consumeTime); - if ($time && ($lastConsumeTime === null || $time > $lastConsumeTime)) { - $lastConsumeTime = $time; - } - } - } - - // 更新用户档案（使用 increaseStats 方法，但这里需要全量更新） - // 简化处理：直接更新统计字段 - $user = $this->userProfileRepository->findByUserId($userId); - if ($user) { - $user->total_amount = $totalAmount; - $user->total_count = $totalCount; - if ($lastConsumeTime) { - $user->last_consume_time = new \DateTimeImmutable('@' . $lastConsumeTime); - } - $user->save(); - } - } -} - diff --git a/Moncter/Moncter/app/service/DatabaseSyncService.php b/Moncter/Moncter/app/service/DatabaseSyncService.php deleted file mode 100644 index 8bf83292..00000000 --- a/Moncter/Moncter/app/service/DatabaseSyncService.php +++ /dev/null @@ -1,1417 +0,0 @@ - 0, - 'collections' => 0, - 'documents_inserted' => 0, - 'documents_updated' => 0, - 'documents_deleted' => 0, - 'errors' => 0, - 'last_sync_time' => null, - ]; - // 同步进度信息 - private array $progress = [ - 'status' => 'idle', // idle, full_sync, incremental_sync, error - 'current_database' => null, - 'current_collection' => null, - 'databases_total' => 0, - 'databases_completed' => 0, - 'collections_total' => 0, - 'collections_completed' => 0, - // 文档级进度（行数） - 'documents_total' => 0, - 'documents_processed' => 0, - // 数据量级进度（基于 collStats / dbStats 估算的字节数） - 'bytes_total' => 0, - // 已经清空过的目标数据库列表，避免重复清空影响断点续传 - 'cleared_databases' => [], - // 源端数据库的集合快照（用于检测“同名库但结构已变更/被重建”的情况） - // 结构示例：'collections_snapshot' => ['KR' => ['coll1', 'coll2', ...]] - 'collections_snapshot' => [], - // 在历史进度中出现过，但当前源库已不存在的数据库（用于给出提醒） - 'orphan_databases' => [], - // 断点续传检查点：按数据库/集合记录最后一个处理的 _id 和已处理数量 - // 结构示例： - // 'checkpoints' => [ - // 'KR_腾讯' => [ - // '某集合名' => [ - // 'last_id' => 'xxx', - // 'processed' => 123, - // 'completed' => false, - // ], - // ], - // ], - 'checkpoints' => [], - // bytes_processed 不单独持久化，在 getProgress 中按 documents 比例动态估算 - 'start_time' => null, - 'current_database_start_time' => null, - 'estimated_time_remaining' => null, - 'last_error' => null, // 记录最后一次错误信息 - 'error_database' => null, // 出错的数据库名称 - ]; - - /** - * 构造函数 - * - * @param array|null $config 配置数组，必须包含 'source' 和 'target' 数据库配置 - * 如果为 null 或配置无效，将跳过数据库连接初始化（仅用于读取进度文件） - * 注意：config('database_sync') 已废弃，必须通过 DatabaseSyncHandler 传递配置 - * - * @throws \InvalidArgumentException 如果配置为 null 且无效 - */ - public function __construct(?array $config = null) - { - if ($config === null) { - throw new \InvalidArgumentException( - 'DatabaseSyncService 必须传递配置参数。' . - 'config(\'database_sync\') 已废弃，请使用 DatabaseSyncHandler 传递配置。' - ); - } - $this->config = $config; - - try { - // 只有在配置有效时才初始化数据库连接（用于查询进度时可能不需要连接） - if ($this->hasValidConfig()) { - $this->initClients(); - LoggerHelper::logBusiness('database_sync_service_initialized', [ - 'source' => $this->config['source']['host'] . ':' . $this->config['source']['port'], - 'target' => $this->config['target']['host'] . ':' . $this->config['target']['port'], - ]); - } - $this->loadProgress(); - } catch (\Exception $e) { - LoggerHelper::logError($e, [ - 'action' => 'database_sync_service_init_error', - ]); - throw $e; - } - } - - /** - * 检查配置是否有效（用于判断是否需要初始化数据库连接） - * - * @return bool - */ - private function hasValidConfig(): bool - { - $sourceHost = $this->config['source']['host'] ?? ''; - $sourcePort = $this->config['source']['port'] ?? 0; - $targetHost = $this->config['target']['host'] ?? ''; - $targetPort = $this->config['target']['port'] ?? 0; - - return !empty($sourceHost) && $sourcePort > 0 && !empty($targetHost) && $targetPort > 0; - } - - /** - * 初始化数据库连接 - */ - private function initClients(): void - { - // 源数据库连接 - $sourceConfig = $this->config['source']; - $sourceDsn = $this->buildDsn($sourceConfig); - $this->sourceClient = new Client($sourceDsn, $sourceConfig['options']); - - // 目标数据库连接 - $targetConfig = $this->config['target']; - $targetDsn = $this->buildDsn($targetConfig); - $this->targetClient = new Client($targetDsn, $targetConfig['options']); - - LoggerHelper::logBusiness('database_sync_clients_initialized', [ - 'source' => $sourceConfig['host'] . ':' . $sourceConfig['port'], - 'target' => $targetConfig['host'] . ':' . $targetConfig['port'], - ]); - } - - /** - * 构建 MongoDB DSN - */ - private function buildDsn(array $config): string - { - // 验证必需的配置项 - $host = $config['host'] ?? ''; - $port = $config['port'] ?? 0; - - if (empty($host)) { - throw new \InvalidArgumentException( - 'MongoDB host 配置为空。请设置环境变量 DB_SYNC_SOURCE_HOST 和 DB_SYNC_TARGET_HOST' - ); - } - - if (empty($port) || $port <= 0) { - throw new \InvalidArgumentException( - 'MongoDB port 配置无效。请设置环境变量 DB_SYNC_SOURCE_PORT 和 DB_SYNC_TARGET_PORT' - ); - } - - $dsn = 'mongodb://'; - if (!empty($config['username']) && !empty($config['password'])) { - $dsn .= urlencode($config['username']) . ':' . urlencode($config['password']) . '@'; - } - $dsn .= $host . ':' . $port; - if (!empty($config['auth_source'])) { - $dsn .= '/?authSource=' . urlencode($config['auth_source']); - } - return $dsn; - } - - /** - * 获取要同步的数据库列表 - */ - public function getDatabasesToSync(): array - { - try { - $databases = $this->sourceClient->listDatabases(); - $databasesToSync = []; - // 记录每个数据库的大致大小，用于排序（小库优先同步） - $databaseSizes = []; - $excludeDatabases = $this->config['sync']['exclude_databases'] ?? []; - - $currentDbNames = []; - foreach ($databases as $databaseInfo) { - $dbName = (string)$databaseInfo->getName(); - $currentDbNames[] = $dbName; - - // 记录源端数据库的大致大小（单位：字节），用于后续排序 - try { - $sizeOnDisk = method_exists($databaseInfo, 'getSizeOnDisk') - ? (int)$databaseInfo->getSizeOnDisk() - : 0; - $databaseSizes[$dbName] = $sizeOnDisk; - } catch (\Throwable $e) { - $databaseSizes[$dbName] = 0; - } - - // 排除系统数据库 - if (in_array($dbName, $excludeDatabases)) { - continue; - } - - // 如果指定了要同步的数据库列表，只同步列表中的 - $syncDatabases = $this->config['sync']['databases'] ?? []; - if (!empty($syncDatabases) && !in_array($dbName, $syncDatabases)) { - continue; - } - - $databasesToSync[] = $dbName; - } - - // 检测历史进度中曾经同步过，但当前源库已不存在的“孤儿数据库” - $knownDbNames = array_keys($this->progress['collections_snapshot'] ?? []); - $orphanDatabases = $this->progress['orphan_databases'] ?? []; - foreach ($knownDbNames as $knownDb) { - if (!in_array($knownDb, $currentDbNames, true) && !in_array($knownDb, $orphanDatabases, true)) { - $orphanDatabases[] = $knownDb; - LoggerHelper::logBusiness('database_sync_source_database_missing', [ - 'database' => $knownDb, - ], 'warning'); - } - } - $this->progress['orphan_databases'] = $orphanDatabases; - - // 更新进度信息 - // 根据数据库大小排序：小的优先同步，便于尽快完成更多库，提高“完成感” - usort($databasesToSync, function (string $a, string $b) use ($databaseSizes): int { - $sizeA = $databaseSizes[$a] ?? PHP_INT_MAX; - $sizeB = $databaseSizes[$b] ?? PHP_INT_MAX; - if ($sizeA === $sizeB) { - return strcmp($a, $b); - } - return $sizeA <=> $sizeB; - }); - - $this->progress['databases_total'] = count($databasesToSync); - // 如果是首次获取数据库列表（start_time 为空），才重置 completed 计数， - // 避免在进程中途多次调用时把已完成的统计清零。 - if ($this->progress['start_time'] === null) { - $this->progress['databases_completed'] = 0; - } - if ($this->progress['start_time'] === null) { - $this->progress['start_time'] = microtime(true); - } - $this->saveProgress(); - - return $databasesToSync; - } catch (MongoDBException $e) { - LoggerHelper::logError($e, [ - 'action' => 'database_sync_list_databases_error', - ]); - return []; - } - } - - /** - * 确保目标数据库存在，如果不存在则创建 - */ - private function ensureTargetDatabaseExists(string $databaseName): void - { - try { - // 检查目标数据库是否存在 - $targetDatabases = $this->targetClient->listDatabases(); - $databaseExists = false; - - foreach ($targetDatabases as $dbInfo) { - if ($dbInfo->getName() === $databaseName) { - $databaseExists = true; - break; - } - } - - // 如果数据库不存在，创建一个临时集合并插入一条记录来触发数据库创建 - if (!$databaseExists) { - $targetDb = $this->targetClient->selectDatabase($databaseName); - $tempCollection = $targetDb->selectCollection('__temp_sync_init__'); - - // 插入一条临时记录来创建数据库 - $tempCollection->insertOne(['_created' => new \MongoDB\BSON\UTCDateTime()]); - - // 删除临时集合 - $tempCollection->drop(); - - LoggerHelper::logBusiness('database_sync_database_created', [ - 'database' => $databaseName, - 'target' => $this->config['target']['host'] . ':' . $this->config['target']['port'], - ]); - } - } catch (MongoDBException $e) { - LoggerHelper::logError($e, [ - 'action' => 'database_sync_ensure_database_error', - 'database' => $databaseName, - ]); - // 不抛出异常，继续执行同步（MongoDB 会在第一次插入时自动创建数据库） - } - } - - /** - * 清空目标数据库（用于全量同步前的初始化） - * - * 注意： - * - 仅在首次同步该数据库时调用（通过 progress.cleared_databases 控制） - * - 后续断点续传时不会再次清空，避免丢失已同步的数据 - */ - private function clearTargetDatabase(string $databaseName): void - { - try { - $targetDb = $this->targetClient->selectDatabase($databaseName); - $targetDb->drop(); - - LoggerHelper::logBusiness('database_sync_target_database_cleared', [ - 'database' => $databaseName, - 'target' => $this->config['target']['host'] . ':' . $this->config['target']['port'], - ]); - } catch (MongoDBException $e) { - LoggerHelper::logError($e, [ - 'action' => 'database_sync_clear_target_error', - 'database' => $databaseName, - ]); - // 清空失败属于严重问题，这里抛出异常，避免在脏数据基础上继续同步 - throw $e; - } - } - - /** - * 全量同步数据库 - */ - public function fullSyncDatabase(string $databaseName): bool - { - try { - // 更新进度状态 - if ($this->progress['start_time'] === null) { - $this->progress['start_time'] = microtime(true); - } - $this->progress['status'] = 'full_sync'; - $this->progress['current_database'] = $databaseName; - $this->progress['current_database_start_time'] = microtime(true); - $this->saveProgress(); - - LoggerHelper::logBusiness('database_sync_database_start', [ - 'database' => $databaseName, - 'status' => 'full_sync', - ]); - - // 确保目标数据库存在 - $this->ensureTargetDatabaseExists($databaseName); - - // 如果尚未清空过该目标数据库，则执行一次清空（适用于你当前“目标库可以清空”的场景） - $clearedDatabases = $this->progress['cleared_databases'] ?? []; - if (!in_array($databaseName, $clearedDatabases, true)) { - $this->clearTargetDatabase($databaseName); - $clearedDatabases[] = $databaseName; - $this->progress['cleared_databases'] = $clearedDatabases; - $this->saveProgress(); - } - - $sourceDb = $this->sourceClient->selectDatabase($databaseName); - $targetDb = $this->targetClient->selectDatabase($databaseName); - - // 获取所有集合 - $collections = $sourceDb->listCollections(); - $batchSize = $this->config['sync']['change_stream']['full_sync_batch_size'] ?? 1000; - $excludeCollections = $this->config['sync']['exclude_collections'] ?? []; - - // 统计集合总数，同时预估总文档数和总数据量（用于更精确的进度估算） - $collectionList = []; - $totalDocuments = 0; - $totalBytes = 0; - - foreach ($collections as $collectionInfo) { - $collectionName = $collectionInfo->getName(); - if (in_array($collectionName, $excludeCollections)) { - continue; - } - - $collectionList[] = $collectionName; - - try { - // 使用 collStats 获取集合的文档数和大小 - $statsCursor = $sourceDb->command(['collStats' => $collectionName]); - $statsArray = $statsCursor->toArray(); - $collStats = $statsArray[0] ?? []; - - $collCount = (int)($collStats['count'] ?? 0); - $collSizeBytes = (int)($collStats['size'] ?? 0); - - $totalDocuments += $collCount; - $totalBytes += $collSizeBytes; - } catch (MongoDBException $e) { - // 单个集合统计失败不影响整体同步，只记录日志 - LoggerHelper::logError($e, [ - 'action' => 'database_sync_collstats_error', - 'database' => $databaseName, - 'collection' => $collectionName, - ]); - } - } - - // 按名称排序，便于与历史快照稳定对比 - sort($collectionList); - - // 检测同名数据库结构是否发生重大变化（例如：被删除后重建） - $collectionsSnapshot = $this->progress['collections_snapshot'] ?? []; - $previousSnapshot = $collectionsSnapshot[$databaseName] ?? null; - if ($previousSnapshot !== null && $previousSnapshot !== $collectionList) { - // 源库结构变化：为了避免旧 checkpoint 导致数据不一致，将该库视为“新库”，重新清空目标并丢弃旧断点 - LoggerHelper::logBusiness('database_sync_source_schema_changed', [ - 'database' => $databaseName, - 'previous_collections' => $previousSnapshot, - 'current_collections' => $collectionList, - ], 'warning'); - - // 重新清空目标库 - $this->clearTargetDatabase($databaseName); - // 丢弃该库的旧断点 - unset($this->progress['checkpoints'][$databaseName]); - // 标记为已清空 - $clearedDatabases = $this->progress['cleared_databases'] ?? []; - if (!in_array($databaseName, $clearedDatabases, true)) { - $clearedDatabases[] = $databaseName; - } - $this->progress['cleared_databases'] = $clearedDatabases; - } - - // 记录当前集合快照 - $collectionsSnapshot[$databaseName] = $collectionList; - $this->progress['collections_snapshot'] = $collectionsSnapshot; - - $this->progress['collections_total'] = count($collectionList); - $this->progress['collections_completed'] = 0; - // 为整个数据库预先写入总文档数和总数据量（按库维度估算进度） - if ($totalDocuments > 0) { - $this->progress['documents_total'] = $totalDocuments; - } - if ($totalBytes > 0) { - $this->progress['bytes_total'] = $totalBytes; - } - // 每次开始全量同步时重置已处理文档数 - $this->progress['documents_processed'] = 0; - $this->saveProgress(); - - // 根据配置决定是否并行同步集合 - $enableParallel = $this->config['sync']['performance']['enable_parallel_sync'] ?? true; - $concurrentCollections = $this->config['sync']['performance']['concurrent_collections'] ?? 10; - - if ($enableParallel && count($collectionList) > 1) { - // 并行同步多个集合 - $this->syncCollectionsParallel($sourceDb, $targetDb, $collectionList, $databaseName, $batchSize, $concurrentCollections); - } else { - // 顺序同步集合 - foreach ($collectionList as $collectionName) { - $this->syncCollection($sourceDb, $targetDb, $collectionName, $databaseName, $batchSize); - } - } - - $this->stats['databases']++; - $this->progress['databases_completed']++; - $this->progress['current_database'] = null; - $this->progress['current_collection'] = null; - $this->saveProgress(); - - return true; - } catch (MongoDBException $e) { - // 记录错误信息，但不停止整个同步流程 - $errorMessage = $e->getMessage(); - $this->progress['status'] = 'error'; - $this->progress['last_error'] = [ - 'message' => $errorMessage, - 'database' => $databaseName, - 'collection' => $this->progress['current_collection'], - 'file' => $e->getFile(), - 'line' => $e->getLine(), - 'time' => date('Y-m-d H:i:s'), - ]; - $this->progress['error_database'] = $databaseName; - $this->saveProgress(); - - LoggerHelper::logError($e, [ - 'action' => 'database_sync_full_sync_error', - 'database' => $databaseName, - 'collection' => $this->progress['current_collection'], - ]); - $this->stats['errors']++; - - // 不返回 false，让调用者决定是否继续同步其他数据库 - // 这样可以跳过有问题的数据库，继续同步其他数据库 - return false; - } - } - - /** - * 同步单个集合（支持大数据量分片） - * - * 错误隔离：集合级错误不会影响其他集合的同步 - */ - private function syncCollection(Database $sourceDb, Database $targetDb, string $collectionName, string $databaseName, int $batchSize): void - { - $this->progress['current_collection'] = $collectionName; - $this->saveProgress(); - - LoggerHelper::logBusiness('database_sync_full_sync_collection_start', [ - 'database' => $databaseName, - 'collection' => $collectionName, - ]); - - try { - $sourceCollection = $sourceDb->selectCollection($collectionName); - $targetCollection = $targetDb->selectCollection($collectionName); - - // 统计当前集合文档总数（用于分片和日志），但不再覆盖全局 documents_total， - // 全库的总文档数在 fullSyncDatabase 中基于 collStats 预估 - $totalDocuments = $sourceCollection->countDocuments([]); - - // 检查是否需要分片处理（大数据量） - $documentsPerTask = $this->config['sync']['performance']['documents_per_task'] ?? 100000; - $enableParallel = $this->config['sync']['performance']['enable_parallel_sync'] ?? true; - $maxParallelTasks = $this->config['sync']['performance']['max_parallel_tasks_per_collection'] ?? 4; - - if ($enableParallel && $totalDocuments > $documentsPerTask && $maxParallelTasks > 1) { - // 大数据量集合，使用分片并行处理 - $this->syncCollectionParallel($sourceCollection, $targetCollection, $collectionName, $databaseName, $batchSize, $totalDocuments, $maxParallelTasks); - } else { - // 小数据量集合，直接同步 - $this->syncCollectionSequential($sourceCollection, $targetCollection, $collectionName, $databaseName, $batchSize); - } - - LoggerHelper::logBusiness('database_sync_full_sync_collection_complete', [ - 'database' => $databaseName, - 'collection' => $collectionName, - 'count' => $totalDocuments, - ]); - - $this->stats['collections']++; - $this->progress['collections_completed']++; - } catch (\Throwable $e) { - // 集合级错误隔离：记录错误但继续同步其他集合 - LoggerHelper::logError($e, [ - 'action' => 'database_sync_collection_error', - 'database' => $databaseName, - 'collection' => $collectionName, - ]); - - $this->stats['errors']++; - - // 记录集合级错误到进度文件 - $this->progress['last_error'] = [ - 'message' => $e->getMessage(), - 'database' => $databaseName, - 'collection' => $collectionName, - 'file' => $e->getFile(), - 'line' => $e->getLine(), - 'time' => date('Y-m-d H:i:s'), - ]; - - // 仍然标记集合为已完成（跳过），继续同步其他集合 - $this->stats['collections']++; - $this->progress['collections_completed']++; - } finally { - $this->progress['current_collection'] = null; - $this->saveProgress(); - } - } - - /** - * 顺序同步集合（小数据量） - */ - private function syncCollectionSequential(Collection $sourceCollection, Collection $targetCollection, string $collectionName, string $databaseName, int $batchSize): void - { - // 从断点读取上次同步位置（基于 _id 断点） - $checkpoint = $this->progress['checkpoints'][$databaseName][$collectionName] ?? null; - $lastId = $checkpoint['last_id'] ?? null; - - $filter = []; - if ($lastId) { - try { - $filter['_id'] = ['$gt' => new \MongoDB\BSON\ObjectId($lastId)]; - } catch (\Throwable $e) { - // 如果 last_id 无法解析为 ObjectId，则退回全量同步 - LoggerHelper::logError($e, [ - 'action' => 'database_sync_invalid_checkpoint_id', - 'database' => $databaseName, - 'collection' => $collectionName, - 'last_id' => $lastId, - ]); - $filter = []; - } - } - - $options = [ - 'batchSize' => $batchSize, - // 确保按 _id 递增，便于基于 _id 做断点续传 - 'sort' => ['_id' => 1], - ]; - - $cursor = $sourceCollection->find($filter, $options); - $batch = []; - $lastProgressLogTime = time(); - - foreach ($cursor as $document) { - $batch[] = $document; - - if (count($batch) >= $batchSize) { - $this->batchInsert($targetCollection, $batch); - $batchCount = count($batch); - $this->progress['documents_processed'] += $batchCount; - - // 记录本批次最后一个文档的 _id 作为断点 - $lastDoc = end($batch); - if (isset($lastDoc['_id'])) { - $this->progress['checkpoints'][$databaseName][$collectionName] = [ - 'last_id' => (string)$lastDoc['_id'], - 'processed' => ($this->progress['checkpoints'][$databaseName][$collectionName]['processed'] ?? 0) + $batchCount, - 'completed' => false, - ]; - } - - $batch = []; - - // 每5秒输出一次进度 - if (time() - $lastProgressLogTime >= 5) { - $this->logProgress(); - $lastProgressLogTime = time(); - } - $this->saveProgress(); - } - } - - // 处理剩余数据 - if (!empty($batch)) { - $this->batchInsert($targetCollection, $batch); - $batchCount = count($batch); - $this->progress['documents_processed'] += $batchCount; - - $lastDoc = end($batch); - if (isset($lastDoc['_id'])) { - $this->progress['checkpoints'][$databaseName][$collectionName] = [ - 'last_id' => (string)$lastDoc['_id'], - 'processed' => ($this->progress['checkpoints'][$databaseName][$collectionName]['processed'] ?? 0) + $batchCount, - 'completed' => true, - ]; - } - - $this->saveProgress(); - } - } - - /** - * 并行同步集合（大数据量，使用分片） - */ - private function syncCollectionParallel(Collection $sourceCollection, Collection $targetCollection, string $collectionName, string $databaseName, int $batchSize, int $totalDocuments, int $maxParallelTasks): void - { - // 计算每个任务处理的文档数 - $documentsPerTask = (int)ceil($totalDocuments / $maxParallelTasks); - - LoggerHelper::logBusiness('database_sync_collection_parallel_start', [ - 'database' => $databaseName, - 'collection' => $collectionName, - 'total_documents' => $totalDocuments, - 'parallel_tasks' => $maxParallelTasks, - 'documents_per_task' => $documentsPerTask, - ]); - - // 创建任务列表 - $tasks = []; - for ($i = 0; $i < $maxParallelTasks; $i++) { - $skip = $i * $documentsPerTask; - $limit = min($documentsPerTask, $totalDocuments - $skip); - - if ($limit <= 0) { - break; - } - - $tasks[] = [ - 'skip' => $skip, - 'limit' => $limit, - 'task_id' => $i + 1, - ]; - } - - // 使用 Workerman 的协程或进程并行执行 - $this->executeParallelTasks($sourceCollection, $targetCollection, $tasks, $batchSize); - } - - /** - * 执行并行任务 - * - * 注意：由于 Workerman Coroutine 可能存在类加载冲突问题，这里使用顺序执行 - * MongoDB 操作本身已经很快，顺序执行也能保证良好的性能 - */ - private function executeParallelTasks(Collection $sourceCollection, Collection $targetCollection, array $tasks, int $batchSize): void - { - // 顺序执行任务（避免协程类加载冲突） - foreach ($tasks as $task) { - $this->syncCollectionChunk($sourceCollection, $targetCollection, $task['skip'], $task['limit'], $batchSize); - } - } - - /** - * 同步集合的一个分片 - */ - private function syncCollectionChunk(Collection $sourceCollection, Collection $targetCollection, int $skip, int $limit, int $batchSize): void - { - $cursor = $sourceCollection->find([], [ - 'skip' => $skip, - 'limit' => $limit, - 'batchSize' => $batchSize, - ]); - - $batch = []; - $count = 0; - - foreach ($cursor as $document) { - $batch[] = $document; - $count++; - - if (count($batch) >= $batchSize) { - $this->batchInsert($targetCollection, $batch); - $this->progress['documents_processed'] += count($batch); - $batch = []; - $this->saveProgress(); - } - } - - // 处理剩余数据 - if (!empty($batch)) { - $this->batchInsert($targetCollection, $batch); - $this->progress['documents_processed'] += count($batch); - $this->saveProgress(); - } - } - - /** - * 并行同步多个集合 - */ - private function syncCollectionsParallel(Database $sourceDb, Database $targetDb, array $collectionList, string $databaseName, int $batchSize, int $concurrentCollections): void - { - // 将集合列表分成多个批次 - $chunks = array_chunk($collectionList, $concurrentCollections); - - foreach ($chunks as $chunk) { - // 顺序同步当前批次（避免协程类加载冲突） - // 注意：虽然配置了并发集合数，但由于协程存在类加载问题，这里使用顺序执行 - // MongoDB 操作本身已经很快，顺序执行也能保证良好的性能 - foreach ($chunk as $collectionName) { - $this->syncCollection($sourceDb, $targetDb, $collectionName, $databaseName, $batchSize); - } - } - } - - /** - * 批量插入文档（支持重试机制） - * - * 错误隔离：文档级错误不会影响其他批次的同步 - */ - private function batchInsert(Collection $collection, array $documents): void - { - if (empty($documents)) { - return; - } - - $maxRetries = $this->config['sync']['retry']['max_sync_retries'] ?? 3; - $retryDelay = $this->config['sync']['retry']['sync_retry_interval'] ?? 2; - $retryCount = 0; - - while ($retryCount <= $maxRetries) { - try { - // 使用 bulkWrite 进行批量写入 - $operations = []; - foreach ($documents as $doc) { - $operations[] = [ - 'insertOne' => [$doc], - ]; - } - - $collection->bulkWrite($operations, ['ordered' => false]); - $this->stats['documents_inserted'] += count($documents); - return; // 成功，退出重试循环 - } catch (MongoDBException $e) { - $retryCount++; - - if ($retryCount > $maxRetries) { - // 超过最大重试次数，记录错误但继续处理下一批 - LoggerHelper::logError($e, [ - 'action' => 'database_sync_batch_insert_error', - 'collection' => $collection->getCollectionName(), - 'count' => count($documents), - 'retry_count' => $retryCount - 1, - ]); - - $this->stats['errors']++; - - // 对于文档级错误，不抛出异常，继续处理下一批 - // 这样可以保证即使某些文档失败，也能继续同步其他文档 - return; - } - - // 指数退避重试 - $delay = $retryDelay * pow(2, $retryCount - 1); - LoggerHelper::logBusiness('database_sync_batch_insert_retry', [ - 'collection' => $collection->getCollectionName(), - 'retry_count' => $retryCount, - 'max_retries' => $maxRetries, - 'delay' => $delay, - ]); - - // 等待后重试 - sleep($delay); - } - } - } - - /** - * 监听数据库变化并同步 - * - * 注意：此方法会阻塞，需要在独立进程中运行 - */ - public function watchDatabase(string $databaseName): void - { - try { - // 确保目标数据库存在 - $this->ensureTargetDatabaseExists($databaseName); - - $sourceDb = $this->sourceClient->selectDatabase($databaseName); - $targetDb = $this->targetClient->selectDatabase($databaseName); - - $batchSize = $this->config['sync']['change_stream']['batch_size'] ?? 100; - $maxAwaitTimeMs = $this->config['sync']['change_stream']['max_await_time_ms'] ?? 1000; - $excludeCollections = $this->config['sync']['exclude_collections'] ?? []; - - // 使用数据库级别的 Change Stream（MongoDB 4.0+） - // 这样可以监听整个数据库的所有集合变化 - $changeStream = $sourceDb->watch( - [], - [ - 'fullDocument' => 'updateLookup', - 'batchSize' => $batchSize, - 'maxAwaitTimeMS' => $maxAwaitTimeMs, - ] - ); - - LoggerHelper::logBusiness('database_sync_watch_database_start', [ - 'database' => $databaseName, - ]); - - // 处理变更事件 - foreach ($changeStream as $change) { - $collectionName = $change['ns']['coll'] ?? ''; - - // 排除系统集合 - if (in_array($collectionName, $excludeCollections)) { - continue; - } - - // 获取目标集合 - $targetCollection = $targetDb->selectCollection($collectionName); - - // 处理变更 - $this->processChange($targetCollection, $change); - $this->stats['last_sync_time'] = time(); - $this->progress['status'] = 'incremental_sync'; - $this->saveProgress(); - } - } catch (MongoDBException $e) { - LoggerHelper::logError($e, [ - 'action' => 'database_sync_watch_database_error', - 'database' => $databaseName, - ]); - throw $e; - } - } - - - /** - * 处理变更事件 - */ - private function processChange(Collection $targetCollection, $change): void - { - try { - $operationType = $change['operationType'] ?? ''; - - switch ($operationType) { - case 'insert': - $this->handleInsert($targetCollection, $change); - break; - case 'update': - case 'replace': - $this->handleUpdate($targetCollection, $change); - break; - case 'delete': - $this->handleDelete($targetCollection, $change); - break; - default: - LoggerHelper::logBusiness('database_sync_unknown_operation', [ - 'operation' => $operationType, - 'collection' => $targetCollection->getCollectionName(), - ]); - } - } catch (MongoDBException $e) { - LoggerHelper::logError($e, [ - 'action' => 'database_sync_process_change_error', - 'collection' => $targetCollection->getCollectionName(), - 'operation' => $change['operationType'] ?? 'unknown', - ]); - $this->stats['errors']++; - } - } - - /** - * 处理插入操作 - */ - private function handleInsert(Collection $targetCollection, array $change): void - { - $document = $change['fullDocument'] ?? null; - if ($document) { - $targetCollection->insertOne($document); - $this->stats['documents_inserted']++; - - if ($this->config['monitoring']['log_detail'] ?? false) { - LoggerHelper::logBusiness('database_sync_insert', [ - 'collection' => $targetCollection->getCollectionName(), - 'document_id' => (string)($document['_id'] ?? ''), - ]); - } - } - } - - /** - * 处理更新操作 - */ - private function handleUpdate(Collection $targetCollection, array $change): void - { - $documentId = $change['documentKey']['_id'] ?? null; - $fullDocument = $change['fullDocument'] ?? null; - - if ($documentId) { - if ($fullDocument) { - // 使用完整文档替换 - $targetCollection->replaceOne( - ['_id' => $documentId], - $fullDocument, - ['upsert' => true] - ); - } else { - // 使用更新操作 - $updateDescription = $change['updateDescription'] ?? []; - $updatedFields = $updateDescription['updatedFields'] ?? []; - $removedFields = $updateDescription['removedFields'] ?? []; - - $update = []; - if (!empty($updatedFields)) { - $update['$set'] = $updatedFields; - } - if (!empty($removedFields)) { - $update['$unset'] = array_fill_keys($removedFields, ''); - } - - if (!empty($update)) { - $targetCollection->updateOne( - ['_id' => $documentId], - $update, - ['upsert' => true] - ); - } - } - $this->stats['documents_updated']++; - - if ($this->config['monitoring']['log_detail'] ?? false) { - LoggerHelper::logBusiness('database_sync_update', [ - 'collection' => $targetCollection->getCollectionName(), - 'document_id' => (string)$documentId, - ]); - } - } - } - - /** - * 处理删除操作 - */ - private function handleDelete(Collection $targetCollection, array $change): void - { - $documentId = $change['documentKey']['_id'] ?? null; - if ($documentId) { - $targetCollection->deleteOne(['_id' => $documentId]); - $this->stats['documents_deleted']++; - - if ($this->config['monitoring']['log_detail'] ?? false) { - LoggerHelper::logBusiness('database_sync_delete', [ - 'collection' => $targetCollection->getCollectionName(), - 'document_id' => (string)$documentId, - ]); - } - } - } - - /** - * 获取同步统计信息 - */ - public function getStats(): array - { - return $this->stats; - } - - /** - * 获取同步进度信息 - */ - public function getProgress(): array - { - // 计算进度百分比（优先使用文档级进度，更准确） - $progressPercent = 0; - - // 方法1：基于文档数计算（最准确） - if ($this->progress['documents_total'] > 0 && $this->progress['documents_processed'] > 0) { - $docProgress = ($this->progress['documents_processed'] / $this->progress['documents_total']) * 100; - $progressPercent = round($docProgress, 2); - } - // 方法2：基于数据库和集合计算（备用） - elseif ($this->progress['databases_total'] > 0) { - $dbProgress = ($this->progress['databases_completed'] / $this->progress['databases_total']) * 100; - - // 如果当前正在处理某个数据库，考虑集合进度 - if ($this->progress['collections_total'] > 0 && $this->progress['current_database']) { - $collectionProgress = ($this->progress['collections_completed'] / $this->progress['collections_total']) * 100; - // 当前数据库的进度 = 已完成数据库数 + 当前数据库的集合进度 - $dbProgress = ($this->progress['databases_completed'] + ($collectionProgress / 100)) / $this->progress['databases_total'] * 100; - } - - $progressPercent = round($dbProgress, 2); - } - - // 确保进度在 0-100 之间 - $progressPercent = max(0, min(100, $progressPercent)); - - // 计算已用时间 - $elapsedTime = null; - if ($this->progress['start_time']) { - $elapsedTime = round(microtime(true) - $this->progress['start_time'], 2); - } - - // 基于文档数和预估总数据量，计算按“数据量”的同步进度（字节级） - $bytesTotal = (int)($this->progress['bytes_total'] ?? 0); - $bytesProcessed = 0; - if ($bytesTotal > 0 && $this->progress['documents_total'] > 0) { - $ratio = $this->progress['documents_processed'] / max(1, $this->progress['documents_total']); - if ($ratio > 1) { - $ratio = 1; - } elseif ($ratio < 0) { - $ratio = 0; - } - $bytesProcessed = (int)round($bytesTotal * $ratio); - } - - // 计算预计剩余时间 - $estimatedRemaining = null; - if ($progressPercent > 0 && $elapsedTime) { - $totalEstimatedTime = $elapsedTime / ($progressPercent / 100); - $estimatedRemaining = round($totalEstimatedTime - $elapsedTime, 2); - } - - return [ - 'status' => $this->progress['status'], - 'progress_percent' => $progressPercent, - 'current_database' => $this->progress['current_database'], - 'current_collection' => $this->progress['current_collection'], - 'databases' => [ - 'total' => $this->progress['databases_total'], - 'completed' => $this->progress['databases_completed'], - 'remaining' => $this->progress['databases_total'] - $this->progress['databases_completed'], - ], - 'collections' => [ - 'total' => $this->progress['collections_total'], - 'completed' => $this->progress['collections_completed'], - 'remaining' => $this->progress['collections_total'] - $this->progress['collections_completed'], - ], - 'documents' => [ - 'total' => $this->progress['documents_total'], - 'processed' => $this->progress['documents_processed'], - 'remaining' => max(0, $this->progress['documents_total'] - $this->progress['documents_processed']), - ], - 'bytes' => [ - 'total' => $bytesTotal, - 'processed' => $bytesProcessed, - 'remaining' => max(0, $bytesTotal - $bytesProcessed), - ], - 'time' => [ - 'elapsed_seconds' => $elapsedTime, - 'estimated_remaining_seconds' => $estimatedRemaining, - 'start_time' => $this->progress['start_time'] ? date('Y-m-d H:i:s', (int)$this->progress['start_time']) : null, - ], - 'stats' => $this->stats, - 'last_error' => $this->progress['last_error'] ?? null, - 'error_database' => $this->progress['error_database'] ?? null, - ]; - } - - /** - * 重置进度并清除错误状态（用于恢复同步） - */ - public function resetProgress(): void - { - $this->resetStats(); - LoggerHelper::logBusiness('database_sync_progress_reset', []); - } - - /** - * 跳过当前错误数据库，继续同步下一个 - */ - public function skipErrorDatabase(): bool - { - if ($this->progress['status'] === 'error' && $this->progress['error_database']) { - $errorDb = $this->progress['error_database']; - - // 标记该数据库为已完成（跳过） - $this->stats['databases']++; - $this->progress['databases_completed']++; - - // 清除错误状态 - $this->progress['status'] = 'full_sync'; - $this->progress['current_database'] = null; - $this->progress['current_collection'] = null; - $this->progress['error_database'] = null; - $this->progress['last_error'] = null; - - $this->saveProgress(); - - LoggerHelper::logBusiness('database_sync_skip_error_database', [ - 'database' => $errorDb, - ]); - - return true; - } - return false; - } - - /** - * 获取运行时目录路径 - */ - private function getRuntimePath(): string - { - if (function_exists('runtime_path')) { - $path = runtime_path(); - } else { - $basePath = function_exists('base_path') ? base_path() : __DIR__ . '/../../'; - $path = config('app.runtime_path', $basePath . DIRECTORY_SEPARATOR . 'runtime'); - } - if (!is_dir($path)) { - mkdir($path, 0777, true); - } - return $path; - } - - /** - * 保存进度到文件（用于多进程共享） - * - * 使用文件锁（LOCK_EX）保证多进程写入的原子性，避免并发冲突 - */ - private function saveProgress(): void - { - $progressFile = $this->getRuntimePath() . DIRECTORY_SEPARATOR . 'database_sync_progress.json'; - - // 使用文件锁保证原子性写入 - $fp = fopen($progressFile, 'c+'); // 'c+' 模式：如果文件不存在则创建，如果存在则打开用于读写 - if ($fp === false) { - LoggerHelper::logError(new \RuntimeException("无法打开进度文件: {$progressFile}"), [ - 'action' => 'database_sync_save_progress_error', - ]); - return; - } - - // 获取独占锁（LOCK_EX），阻塞直到获取锁 - if (flock($fp, LOCK_EX)) { - try { - // 读取现有进度（如果存在），智能合并更新 - $existingContent = stream_get_contents($fp); - if ($existingContent) { - $existingProgress = json_decode($existingContent, true); - if ($existingProgress && is_array($existingProgress)) { - // 智能合并策略： - // 1. 保留全局统计信息（databases_total, collections_total 等） - // 2. 合并 checkpoints（每个进程只更新自己负责的数据库） - // 3. 合并 cleared_databases（避免重复清空） - // 4. 合并 collections_snapshot（保留所有数据库的快照） - // 5. 更新当前进程的进度信息 - - // 保留全局统计（取最大值，确保不丢失） - $this->progress['databases_total'] = max( - $this->progress['databases_total'] ?? 0, - $existingProgress['databases_total'] ?? 0 - ); - $this->progress['collections_total'] = max( - $this->progress['collections_total'] ?? 0, - $existingProgress['collections_total'] ?? 0 - ); - $this->progress['documents_total'] = max( - $this->progress['documents_total'] ?? 0, - $existingProgress['documents_total'] ?? 0 - ); - $this->progress['bytes_total'] = max( - $this->progress['bytes_total'] ?? 0, - $existingProgress['bytes_total'] ?? 0 - ); - - // 对于 completed 计数，需要累加（多进程场景） - // 但由于每个进程只处理部分数据库，直接累加会导致重复计数 - // 因此采用基于 checkpoints 重新计算的方式 - // 这里先保留现有值，在 getProgress 中基于 checkpoints 重新计算 - // 为了简化，这里采用取最大值的方式（每个进程只更新自己完成的部分） - // 注意：这种方式在多进程场景下可能不够精确，但可以避免重复计数 - $this->progress['databases_completed'] = max( - $this->progress['databases_completed'] ?? 0, - $existingProgress['databases_completed'] ?? 0 - ); - $this->progress['collections_completed'] = max( - $this->progress['collections_completed'] ?? 0, - $existingProgress['collections_completed'] ?? 0 - ); - $this->progress['documents_processed'] = max( - $this->progress['documents_processed'] ?? 0, - $existingProgress['documents_processed'] ?? 0 - ); - - // 合并 checkpoints（每个进程只更新自己负责的数据库） - if (isset($existingProgress['checkpoints']) && is_array($existingProgress['checkpoints'])) { - if (!isset($this->progress['checkpoints'])) { - $this->progress['checkpoints'] = []; - } - $this->progress['checkpoints'] = array_merge( - $existingProgress['checkpoints'], - $this->progress['checkpoints'] - ); - } - - // 合并 cleared_databases（避免重复清空） - if (isset($existingProgress['cleared_databases']) && is_array($existingProgress['cleared_databases'])) { - if (!isset($this->progress['cleared_databases'])) { - $this->progress['cleared_databases'] = []; - } - $this->progress['cleared_databases'] = array_unique(array_merge( - $existingProgress['cleared_databases'], - $this->progress['cleared_databases'] - )); - } - - // 合并 collections_snapshot（保留所有数据库的快照） - if (isset($existingProgress['collections_snapshot']) && is_array($existingProgress['collections_snapshot'])) { - if (!isset($this->progress['collections_snapshot'])) { - $this->progress['collections_snapshot'] = []; - } - $this->progress['collections_snapshot'] = array_merge( - $existingProgress['collections_snapshot'], - $this->progress['collections_snapshot'] - ); - } - - // 合并 orphan_databases - if (isset($existingProgress['orphan_databases']) && is_array($existingProgress['orphan_databases'])) { - if (!isset($this->progress['orphan_databases'])) { - $this->progress['orphan_databases'] = []; - } - $this->progress['orphan_databases'] = array_unique(array_merge( - $existingProgress['orphan_databases'], - $this->progress['orphan_databases'] - )); - } - - // 保留最早的 start_time - if (isset($existingProgress['start_time']) && $existingProgress['start_time'] > 0) { - if (!isset($this->progress['start_time']) || $this->progress['start_time'] === null) { - $this->progress['start_time'] = $existingProgress['start_time']; - } else { - $this->progress['start_time'] = min( - $this->progress['start_time'], - $existingProgress['start_time'] - ); - } - } - } - } - - // 清空文件并写入合并后的进度 - ftruncate($fp, 0); - rewind($fp); - fwrite($fp, json_encode($this->progress, JSON_PRETTY_PRINT | JSON_UNESCAPED_UNICODE)); - fflush($fp); // 确保立即写入磁盘 - } finally { - // 释放锁 - flock($fp, LOCK_UN); - } - } else { - LoggerHelper::logError(new \RuntimeException("无法获取进度文件锁: {$progressFile}"), [ - 'action' => 'database_sync_save_progress_lock_error', - ]); - } - - fclose($fp); - } - - /** - * 设置进度状态（公开方法，供外部调用） - */ - public function setProgressStatus(string $status): void - { - $this->progress['status'] = $status; - if ($status !== 'idle' && $this->progress['start_time'] === null) { - $this->progress['start_time'] = microtime(true); - } - $this->saveProgress(); - } - - /** - * 从文件加载进度（使用文件锁保证读取一致性） - */ - public function loadProgress(): void - { - $progressFile = $this->getRuntimePath() . DIRECTORY_SEPARATOR . 'database_sync_progress.json'; - if (!file_exists($progressFile)) { - return; - } - - // 使用文件锁保证读取一致性 - $fp = fopen($progressFile, 'r'); - if ($fp === false) { - LoggerHelper::logError(new \RuntimeException("无法打开进度文件: {$progressFile}"), [ - 'action' => 'database_sync_load_progress_error', - ]); - return; - } - - // 获取共享锁（LOCK_SH），允许多个进程同时读取 - if (flock($fp, LOCK_SH)) { - try { - $content = stream_get_contents($fp); - if ($content) { - $loaded = json_decode($content, true); - if ($loaded && is_array($loaded)) { - // 合并进度：保留现有字段，更新加载的字段 - $this->progress = array_merge($this->progress, $loaded); - } - } - } finally { - // 释放锁 - flock($fp, LOCK_UN); - } - } else { - LoggerHelper::logError(new \RuntimeException("无法获取进度文件锁: {$progressFile}"), [ - 'action' => 'database_sync_load_progress_lock_error', - ]); - } - - fclose($fp); - } - - /** - * 输出进度日志 - */ - private function logProgress(): void - { - $progress = $this->getProgress(); - LoggerHelper::logBusiness('database_sync_progress', [ - 'status' => $progress['status'], - 'progress_percent' => $progress['progress_percent'] . '%', - 'current_database' => $progress['current_database'], - 'current_collection' => $progress['current_collection'], - 'databases' => "{$progress['databases']['completed']}/{$progress['databases']['total']}", - 'collections' => "{$progress['collections']['completed']}/{$progress['collections']['total']}", - 'documents' => "{$progress['documents']['processed']}/{$progress['documents']['total']}", - 'elapsed_time' => $progress['time']['elapsed_seconds'] . 's', - 'estimated_remaining' => $progress['time']['estimated_remaining_seconds'] ? $progress['time']['estimated_remaining_seconds'] . 's' : 'calculating...', - ]); - } - - /** - * 重置统计信息 - */ - public function resetStats(): void - { - $this->stats = [ - 'databases' => 0, - 'collections' => 0, - 'documents_inserted' => 0, - 'documents_updated' => 0, - 'documents_deleted' => 0, - 'errors' => 0, - 'last_sync_time' => null, - ]; - $this->progress = [ - 'status' => 'idle', - 'current_database' => null, - 'current_collection' => null, - 'databases_total' => 0, - 'databases_completed' => 0, - 'collections_total' => 0, - 'collections_completed' => 0, - 'documents_total' => 0, - 'documents_processed' => 0, - 'bytes_total' => 0, - 'cleared_databases' => [], - 'collections_snapshot' => [], - 'orphan_databases' => [], - 'start_time' => null, - 'current_database_start_time' => null, - 'estimated_time_remaining' => null, - 'last_error' => null, - 'error_database' => null, - ]; - $this->saveProgress(); - } -} - diff --git a/Moncter/Moncter/app/service/IdentifierService.php b/Moncter/Moncter/app/service/IdentifierService.php deleted file mode 100644 index 08eac2a0..00000000 --- a/Moncter/Moncter/app/service/IdentifierService.php +++ /dev/null @@ -1,312 +0,0 @@ -createTemporaryPerson(null, $atTime); - LoggerHelper::logBusiness('temporary_person_created_no_phone', [ - 'user_id' => $userId, - 'note' => '手机号为空，创建无手机号的临时用户', - ]); - return $userId; - } - - // 1. 先查询手机号关联表（使用指定的时间点） - $userId = $this->userPhoneService->findUserByPhone($trimmedPhone, $atTime); - - if ($userId !== null) { - LoggerHelper::logBusiness('person_resolved_by_phone', [ - 'phone_number' => $trimmedPhone, - 'user_id' => $userId, - 'source' => 'existing_relation', - 'at_time' => $atTime ? $atTime->format('Y-m-d H:i:s') : null, - ]); - return $userId; - } - - // 2. 如果找不到，创建临时人（使用atTime作为生效时间） - $userId = $this->createTemporaryPerson($trimmedPhone, $atTime); - - LoggerHelper::logBusiness('temporary_person_created', [ - 'phone_number' => $trimmedPhone, - 'user_id' => $userId, - 'effective_time' => $atTime ? $atTime->format('Y-m-d H:i:s') : null, - ]); - - return $userId; - } - - /** - * 根据身份证解析用户ID（person_id） - * - * @param string $idCard 身份证号 - * @return string|null user_id（person_id），如果不存在返回null - */ - public function resolvePersonIdByIdCard(string $idCard): ?string - { - $idCardHash = EncryptionHelper::hash($idCard); - $user = $this->userProfileRepository->findByIdCardHash($idCardHash); - - if ($user) { - LoggerHelper::logBusiness('person_resolved_by_id_card', [ - 'id_card_hash' => $idCardHash, - 'user_id' => $user->user_id, - ]); - return $user->user_id; - } - - return null; - } - - /** - * 绑定身份证到用户（将临时人转为正式人，或创建正式人） - * - * @param string $userId 用户ID - * @param string $idCard 身份证号 - * @return bool 是否成功 - * @throws \InvalidArgumentException - */ - public function bindIdCardToPerson(string $userId, string $idCard): bool - { - $idCardHash = EncryptionHelper::hash($idCard); - $idCardEncrypted = EncryptionHelper::encrypt($idCard); - - // 检查该身份证是否已被其他用户使用 - $existingUser = $this->userProfileRepository->findByIdCardHash($idCardHash); - if ($existingUser && $existingUser->user_id !== $userId) { - throw new \InvalidArgumentException("身份证号已被其他用户使用，user_id: {$existingUser->user_id}"); - } - - // 更新用户信息 - $user = $this->userProfileRepository->findByUserId($userId); - if (!$user) { - throw new \InvalidArgumentException("用户不存在: {$userId}"); - } - - // 如果用户已经是正式人且身份证匹配，无需更新 - if (!$user->is_temporary && $user->id_card_hash === $idCardHash) { - return true; - } - - // 更新身份证信息并标记为正式人 - $user->id_card_hash = $idCardHash; - $user->id_card_encrypted = $idCardEncrypted; - $user->id_card_type = '身份证'; - $user->is_temporary = false; - - // 从身份证号中自动提取基础信息（如果字段为空才更新） - $idCardInfo = IdCardHelper::extractInfo($idCard); - if ($idCardInfo['birthday'] !== null && $user->birthday === null) { - $user->birthday = $idCardInfo['birthday']; - } - // 只有当性别解析成功且当前值为 null 时才更新（0 也被认为是未设置） - if ($idCardInfo['gender'] > 0 && ($user->gender === null || $user->gender === 0)) { - $user->gender = $idCardInfo['gender']; - } - - $user->update_time = new \DateTimeImmutable('now'); - $user->save(); - - LoggerHelper::logBusiness('id_card_bound_to_person', [ - 'user_id' => $userId, - 'id_card_hash' => $idCardHash, - 'was_temporary' => $user->is_temporary ?? true, - ]); - - return true; - } - - /** - * 创建临时人 - * - * @param string|null $phoneNumber 手机号（可选，用于建立关联） - * @param \DateTimeInterface|null $effectiveTime 生效时间（用于手机关联，默认当前时间） - * @return string user_id - */ - private function createTemporaryPerson(?string $phoneNumber = null, ?\DateTimeInterface $effectiveTime = null): string - { - $now = new \DateTimeImmutable('now'); - $userId = UuidGenerator::uuid4()->toString(); - - // 创建临时人记录 - $user = new UserProfileRepository(); - $user->user_id = $userId; - $user->is_temporary = true; - $user->status = 0; - $user->total_amount = 0; - $user->total_count = 0; - $user->create_time = $now; - $user->update_time = $now; - $user->save(); - - // 如果有手机号，建立关联（使用effectiveTime作为生效时间） - // 检查手机号不为空（null 或空字符串都跳过） - if ($phoneNumber !== null && trim($phoneNumber) !== '') { - try { - $trimmedPhone = trim($phoneNumber); - $this->userPhoneService->addPhoneToUser($userId, $trimmedPhone, [ - 'source' => 'auto_created', - 'type' => 'personal', - 'effective_time' => $effectiveTime ?? $now, - ]); - - LoggerHelper::logBusiness('phone_relation_created_success', [ - 'user_id' => $userId, - 'phone_number' => $trimmedPhone, - 'effective_time' => ($effectiveTime ?? $now)->format('Y-m-d H:i:s'), - ]); - } catch (\Throwable $e) { - // 手机号关联失败不影响用户创建，只记录详细的错误日志 - LoggerHelper::logError($e, [ - 'component' => 'IdentifierService', - 'action' => 'createTemporaryPerson', - 'user_id' => $userId, - 'phone_number' => $phoneNumber, - 'phone_number_length' => strlen($phoneNumber), - 'error_message' => $e->getMessage(), - 'error_type' => get_class($e), - ]); - - // 同时记录业务日志，便于排查 - LoggerHelper::logBusiness('phone_relation_create_failed', [ - 'user_id' => $userId, - 'phone_number' => $phoneNumber, - 'error_message' => $e->getMessage(), - 'note' => '用户已创建，但手机关联失败', - ]); - } - } elseif ($phoneNumber !== null && trim($phoneNumber) === '') { - // 手机号是空字符串，记录日志 - LoggerHelper::logBusiness('phone_relation_skipped_empty', [ - 'user_id' => $userId, - 'note' => '手机号为空字符串，跳过关联创建', - ]); - } - - return $userId; - } - - /** - * 根据手机号或身份证解析用户ID - * - * 优先级：身份证 > 手机号 - * - * @param string|null $phoneNumber 手机号 - * @param string|null $idCard 身份证号 - * @param \DateTimeInterface|null $atTime 查询时间点（用于手机号查询，默认为当前时间） - * @return string user_id - */ - public function resolvePersonId(?string $phoneNumber = null, ?string $idCard = null, ?\DateTimeInterface $atTime = null): string - { - $atTime = $atTime ?? new \DateTimeImmutable('now'); - - // 优先使用身份证 - if ($idCard !== null && !empty($idCard)) { - $userId = $this->resolvePersonIdByIdCard($idCard); - if ($userId !== null) { - // 如果身份证存在，但提供了手机号，确保手机号关联到该用户 - if ($phoneNumber !== null && !empty($phoneNumber)) { - // 在atTime时间点查询手机号关联 - $existingUserId = $this->userPhoneService->findUserByPhone($phoneNumber, $atTime); - if ($existingUserId === null) { - // 手机号未关联，建立关联（使用atTime作为生效时间） - $this->userPhoneService->addPhoneToUser($userId, $phoneNumber, [ - 'source' => 'id_card_resolved', - 'type' => 'personal', - 'effective_time' => $atTime, - ]); - } elseif ($existingUserId !== $userId) { - // 手机号已关联到其他用户，需要合并（由PersonMergeService处理） - LoggerHelper::logBusiness('phone_bound_to_different_person', [ - 'phone_number' => $phoneNumber, - 'existing_user_id' => $existingUserId, - 'id_card_user_id' => $userId, - 'at_time' => $atTime->format('Y-m-d H:i:s'), - ]); - } - } - return $userId; - } else { - // 身份证不存在，但有身份证信息，创建一个临时用户并绑定身份证（使其成为正式用户） - $userId = $this->createTemporaryPerson($phoneNumber, $atTime); - try { - $this->bindIdCardToPerson($userId, $idCard); - } catch (\Throwable $e) { - // 绑定失败不影响返回user_id - LoggerHelper::logError($e, [ - 'component' => 'IdentifierService', - 'action' => 'resolvePersonId', - 'user_id' => $userId, - ]); - } - return $userId; - } - } - - // 使用手机号（传入atTime） - if ($phoneNumber !== null && !empty($phoneNumber)) { - $userId = $this->resolvePersonIdByPhone($phoneNumber, $atTime); - - // 如果同时提供了身份证，绑定身份证 - if ($idCard !== null && !empty($idCard)) { - try { - $this->bindIdCardToPerson($userId, $idCard); - } catch (\Throwable $e) { - // 绑定失败不影响返回user_id - LoggerHelper::logError($e, [ - 'component' => 'IdentifierService', - 'action' => 'resolvePersonId', - 'user_id' => $userId, - ]); - } - } - - return $userId; - } - - // 都没有提供，创建临时人 - return $this->createTemporaryPerson(null, $atTime); - } -} - diff --git a/Moncter/Moncter/app/service/PersonMergeService.php b/Moncter/Moncter/app/service/PersonMergeService.php deleted file mode 100644 index 31ec6973..00000000 --- a/Moncter/Moncter/app/service/PersonMergeService.php +++ /dev/null @@ -1,497 +0,0 @@ -userProfileRepository->findByUserId($tempUserId); - if (!$tempUser) { - throw new \InvalidArgumentException("临时人不存在: {$tempUserId}"); - } - - if (!$tempUser->is_temporary) { - throw new \InvalidArgumentException("用户不是临时人: {$tempUserId}"); - } - - $idCardHash = \app\utils\EncryptionHelper::hash($idCard); - - // 查找该身份证是否已有正式人 - $formalUser = $this->userProfileRepository->findByIdCardHash($idCardHash); - - if ($formalUser) { - // 情况1：身份证已存在，合并临时人到正式人 - if ($formalUser->user_id === $tempUserId) { - // 已经是同一个人，只需标记为正式人（传入原始身份证号以提取信息） - $this->userProfileRepository->markAsFormal($tempUserId, $idCardHash, \app\utils\EncryptionHelper::encrypt($idCard), $idCard); - return $tempUserId; - } - - // 合并到已存在的正式人 - $this->mergeUsers($tempUserId, $formalUser->user_id); - return $formalUser->user_id; - } else { - // 情况2：身份证不存在，将临时人转为正式人（传入原始身份证号以提取信息） - $this->userProfileRepository->markAsFormal($tempUserId, $idCardHash, \app\utils\EncryptionHelper::encrypt($idCard), $idCard); - - $tempUser->id_card_type = '身份证'; - $tempUser->save(); - - LoggerHelper::logBusiness('temporary_person_converted_to_formal', [ - 'user_id' => $tempUserId, - 'id_card_hash' => $idCardHash, - ]); - - // 重新计算标签 - $this->recalculateTags($tempUserId); - - return $tempUserId; - } - } - - /** - * 合并两个用户（将sourceUserId合并到targetUserId） - * - * @param string $sourceUserId 源用户ID（将被合并的用户） - * @param string $targetUserId 目标用户ID（保留的用户） - * @return bool 是否成功 - */ - public function mergeUsers(string $sourceUserId, string $targetUserId): bool - { - if ($sourceUserId === $targetUserId) { - return true; - } - - $sourceUser = $this->userProfileRepository->findByUserId($sourceUserId); - $targetUser = $this->userProfileRepository->findByUserId($targetUserId); - - if (!$sourceUser || !$targetUser) { - throw new \InvalidArgumentException("用户不存在: source={$sourceUserId}, target={$targetUserId}"); - } - - LoggerHelper::logBusiness('person_merge_started', [ - 'source_user_id' => $sourceUserId, - 'target_user_id' => $targetUserId, - ]); - - try { - // 1. 合并统计数据 - $this->mergeStatistics($sourceUser, $targetUser); - - // 2. 合并手机号关联 - $this->mergePhoneRelations($sourceUserId, $targetUserId); - - // 3. 合并标签 - $this->mergeTags($sourceUserId, $targetUserId); - - // 4. 合并消费记录（更新user_id） - $this->mergeConsumptionRecords($sourceUserId, $targetUserId); - - // 5. 记录合并历史 - $this->recordMergeHistory($sourceUserId, $targetUserId); - - // 6. 标记源用户为已合并 - $sourceUser->status = 1; // 标记为已删除/已合并 - $sourceUser->merged_from_user_id = $targetUserId; // 记录合并到的目标用户ID - $sourceUser->update_time = new \DateTimeImmutable('now'); - $sourceUser->save(); - - // 7. 更新目标用户的标签更新时间 - $targetUser->tags_update_time = new \DateTimeImmutable('now'); - $targetUser->update_time = new \DateTimeImmutable('now'); - $targetUser->save(); - - LoggerHelper::logBusiness('person_merge_completed', [ - 'source_user_id' => $sourceUserId, - 'target_user_id' => $targetUserId, - ]); - - // 8. 重新计算目标用户的标签 - $this->recalculateTags($targetUserId); - - return true; - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'PersonMergeService', - 'action' => 'mergeUsers', - 'source_user_id' => $sourceUserId, - 'target_user_id' => $targetUserId, - ]); - throw $e; - } - } - - /** - * 合并统计数据 - * - * @param UserProfileRepository $sourceUser - * @param UserProfileRepository $targetUser - */ - private function mergeStatistics(UserProfileRepository $sourceUser, UserProfileRepository $targetUser): void - { - // 合并总金额和总次数 - $targetUser->total_amount = (float)($targetUser->total_amount ?? 0) + (float)($sourceUser->total_amount ?? 0); - $targetUser->total_count = (int)($targetUser->total_count ?? 0) + (int)($sourceUser->total_count ?? 0); - - // 取更晚的最后消费时间 - if ($sourceUser->last_consume_time && - (!$targetUser->last_consume_time || $sourceUser->last_consume_time > $targetUser->last_consume_time)) { - $targetUser->last_consume_time = $sourceUser->last_consume_time; - } - - $targetUser->save(); - } - - /** - * 合并手机号关联 - * - * @param string $sourceUserId - * @param string $targetUserId - */ - private function mergePhoneRelations(string $sourceUserId, string $targetUserId): void - { - // 获取源用户的所有手机号 - $sourcePhones = $this->userPhoneService->getUserPhoneNumbers($sourceUserId, false); - - foreach ($sourcePhones as $phoneNumber) { - try { - // 将手机号关联到目标用户 - $this->userPhoneService->addPhoneToUser($targetUserId, $phoneNumber, [ - 'source' => 'person_merge', - 'type' => 'personal', - ]); - - // 失效源用户的手机号关联 - $this->userPhoneService->removePhoneFromUser($sourceUserId, $phoneNumber); - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'PersonMergeService', - 'action' => 'mergePhoneRelations', - 'source_user_id' => $sourceUserId, - 'target_user_id' => $targetUserId, - 'phone_number' => $phoneNumber, - ]); - } - } - } - - /** - * 合并标签 - * - * 智能合并策略： - * 1. 如果目标用户没有该标签，直接复制源用户的标签 - * 2. 如果目标用户已有该标签，根据标签类型和定义决定合并策略： - * - 数值型标签（number）：根据标签定义的聚合方式（sum/max/min/avg）合并 - * - 布尔型标签（boolean）：取 OR（任一为true则为true） - * - 字符串型标签（string）：保留目标用户的值（不覆盖） - * - 枚举型标签：保留目标用户的值（不覆盖） - * 3. 置信度取两者中的较高值 - * - * @param string $sourceUserId - * @param string $targetUserId - */ - private function mergeTags(string $sourceUserId, string $targetUserId): void - { - $sourceTags = $this->userTagRepository->newQuery() - ->where('user_id', $sourceUserId) - ->get(); - - // 获取标签定义，用于判断合并策略 - $tagDefinitionRepo = new \app\repository\TagDefinitionRepository(); - - foreach ($sourceTags as $sourceTag) { - // 检查目标用户是否已有该标签 - $targetTag = $this->userTagRepository->newQuery() - ->where('user_id', $targetUserId) - ->where('tag_id', $sourceTag->tag_id) - ->first(); - - if (!$targetTag) { - // 目标用户没有该标签，复制源用户的标签 - $newTag = new UserTagRepository(); - $newTag->user_id = $targetUserId; - $newTag->tag_id = $sourceTag->tag_id; - $newTag->tag_value = $sourceTag->tag_value; - $newTag->tag_value_type = $sourceTag->tag_value_type; - $newTag->confidence = $sourceTag->confidence; - $newTag->effective_time = $sourceTag->effective_time; - $newTag->expire_time = $sourceTag->expire_time; - $newTag->create_time = new \DateTimeImmutable('now'); - $newTag->update_time = new \DateTimeImmutable('now'); - $newTag->save(); - } else { - // 目标用户已有标签，根据类型智能合并 - $mergedValue = $this->mergeTagValue( - $sourceTag, - $targetTag, - $tagDefinitionRepo->newQuery()->where('tag_id', $sourceTag->tag_id)->first() - ); - - if ($mergedValue !== null) { - $targetTag->tag_value = $mergedValue; - $targetTag->confidence = max((float)$sourceTag->confidence, (float)$targetTag->confidence); - $targetTag->update_time = new \DateTimeImmutable('now'); - $targetTag->save(); - } - } - } - - // 删除源用户的标签 - $this->userTagRepository->newQuery() - ->where('user_id', $sourceUserId) - ->delete(); - } - - /** - * 合并标签值 - * - * @param UserTagRepository $sourceTag 源标签 - * @param UserTagRepository $targetTag 目标标签 - * @param \app\repository\TagDefinitionRepository|null $tagDef 标签定义（可选） - * @return string|null 合并后的标签值，如果不需要更新返回null - */ - private function mergeTagValue( - UserTagRepository $sourceTag, - UserTagRepository $targetTag, - ?\app\repository\TagDefinitionRepository $tagDef = null - ): ?string { - $sourceValue = $sourceTag->tag_value; - $targetValue = $targetTag->tag_value; - $sourceType = $sourceTag->tag_value_type; - $targetType = $targetTag->tag_value_type; - - // 如果类型不一致，保留目标值 - if ($sourceType !== $targetType) { - return null; - } - - // 根据类型合并 - switch ($targetType) { - case 'number': - // 数值型：根据标签定义的聚合方式合并 - $aggregation = null; - if ($tagDef && isset($tagDef->rule_config)) { - $ruleConfig = is_string($tagDef->rule_config) - ? json_decode($tagDef->rule_config, true) - : $tagDef->rule_config; - $aggregation = $ruleConfig['aggregation'] ?? 'sum'; - } else { - $aggregation = 'sum'; // 默认累加 - } - - $sourceNum = (float)$sourceValue; - $targetNum = (float)$targetValue; - - return match($aggregation) { - 'sum' => (string)($sourceNum + $targetNum), - 'max' => (string)max($sourceNum, $targetNum), - 'min' => (string)min($sourceNum, $targetNum), - 'avg' => (string)(($sourceNum + $targetNum) / 2), - default => (string)($sourceNum + $targetNum), // 默认累加 - }; - - case 'boolean': - // 布尔型：取 OR（任一为true则为true） - $sourceBool = $sourceValue === 'true' || $sourceValue === '1'; - $targetBool = $targetValue === 'true' || $targetValue === '1'; - return ($sourceBool || $targetBool) ? 'true' : 'false'; - - case 'string': - case 'json': - default: - // 字符串型、JSON型等：保留目标值（不覆盖） - return null; - } - } - - /** - * 合并消费记录 - * - * @param string $sourceUserId - * @param string $targetUserId - */ - private function mergeConsumptionRecords(string $sourceUserId, string $targetUserId): void - { - // 更新消费记录的user_id - ConsumptionRecordRepository::where('user_id', $sourceUserId) - ->update(['user_id' => $targetUserId]); - } - - /** - * 重新计算用户标签 - * - * @param string $userId - */ - private function recalculateTags(string $userId): void - { - try { - // 异步触发标签计算 - QueueService::pushTagCalculation([ - 'user_id' => $userId, - 'tag_ids' => null, // 计算所有标签 - 'trigger_type' => 'person_merge', - 'timestamp' => time(), - ]); - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'PersonMergeService', - 'action' => 'recalculateTags', - 'user_id' => $userId, - ]); - } - } - - /** - * 根据手机号发现身份证后，合并相关用户 - * - * 这是场景4的实现：如果某个手机号发现了对应的身份证号， - * 查询该身份下是否有标签，如果有就会将对应的这个身份证号的所有标签重新计算同步。 - * - * @param string $phoneNumber 手机号 - * @param string $idCard 身份证号 - * @return string 正式人的user_id - */ - public function mergePhoneToIdCard(string $phoneNumber, string $idCard): string - { - // 1. 查找手机号对应的用户 - $phoneUserId = $this->userPhoneService->findUserByPhone($phoneNumber); - - // 2. 查找身份证对应的用户 - $idCardHash = \app\utils\EncryptionHelper::hash($idCard); - $idCardUser = $this->userProfileRepository->findByIdCardHash($idCardHash); - - if ($idCardUser && $phoneUserId && $idCardUser->user_id === $phoneUserId) { - // 已经是同一个人，只需确保是正式人（传入原始身份证号以提取信息） - if ($idCardUser->is_temporary) { - $this->userProfileRepository->markAsFormal($phoneUserId, $idCardHash, \app\utils\EncryptionHelper::encrypt($idCard), $idCard); - } - $this->recalculateTags($phoneUserId); - return $phoneUserId; - } - - if ($idCardUser && $phoneUserId && $idCardUser->user_id !== $phoneUserId) { - // 身份证和手机号对应不同用户，需要合并 - $this->mergeUsers($phoneUserId, $idCardUser->user_id); - $this->recalculateTags($idCardUser->user_id); - return $idCardUser->user_id; - } - - if ($idCardUser && !$phoneUserId) { - // 身份证存在，但手机号未关联，建立关联 - $this->userPhoneService->addPhoneToUser($idCardUser->user_id, $phoneNumber, [ - 'source' => 'id_card_discovered', - 'type' => 'personal', - ]); - $this->recalculateTags($idCardUser->user_id); - return $idCardUser->user_id; - } - - if (!$idCardUser && $phoneUserId) { - // 手机号存在，但身份证不存在，将临时人转为正式人 - return $this->mergeTemporaryToFormal($phoneUserId, $idCard); - } - - // 都不存在，创建正式人 - $userId = \Ramsey\Uuid\Uuid::uuid4()->toString(); - $now = new \DateTimeImmutable('now'); - - // 从身份证号中自动提取基础信息 - $idCardInfo = \app\utils\IdCardHelper::extractInfo($idCard); - - $user = new UserProfileRepository(); - $user->user_id = $userId; - $user->id_card_hash = $idCardHash; - $user->id_card_encrypted = \app\utils\EncryptionHelper::encrypt($idCard); - $user->id_card_type = '身份证'; - $user->is_temporary = false; - $user->status = 0; - $user->total_amount = 0; - $user->total_count = 0; - $user->birthday = $idCardInfo['birthday']; // 可能为 null - $user->gender = $idCardInfo['gender'] > 0 ? $idCardInfo['gender'] : null; // 解析失败则为 null - $user->create_time = $now; - $user->update_time = $now; - $user->save(); - - $this->userPhoneService->addPhoneToUser($userId, $phoneNumber, [ - 'source' => 'new_created', - 'type' => 'personal', - ]); - - return $userId; - } - - /** - * 记录合并历史 - * - * @param string $sourceUserId 源用户ID - * @param string $targetUserId 目标用户ID - */ - private function recordMergeHistory(string $sourceUserId, string $targetUserId): void - { - try { - $sourceUser = $this->userProfileRepository->findByUserId($sourceUserId); - $targetUser = $this->userProfileRepository->findByUserId($targetUserId); - - if (!$sourceUser || !$targetUser) { - return; - } - - // 记录合并信息到日志（可以扩展为独立的合并历史表） - LoggerHelper::logBusiness('person_merge_history', [ - 'source_user_id' => $sourceUserId, - 'target_user_id' => $targetUserId, - 'source_is_temporary' => $sourceUser->is_temporary ?? true, - 'target_is_temporary' => $targetUser->is_temporary ?? false, - 'source_id_card_hash' => $sourceUser->id_card_hash ?? null, - 'target_id_card_hash' => $targetUser->id_card_hash ?? null, - 'merge_time' => date('Y-m-d H:i:s'), - ]); - } catch (\Throwable $e) { - // 记录历史失败不影响合并流程 - LoggerHelper::logError($e, [ - 'component' => 'PersonMergeService', - 'action' => 'recordMergeHistory', - 'source_user_id' => $sourceUserId, - 'target_user_id' => $targetUserId, - ]); - } - } -} - diff --git a/Moncter/Moncter/app/service/StoreService.php b/Moncter/Moncter/app/service/StoreService.php deleted file mode 100644 index 3313bfa8..00000000 --- a/Moncter/Moncter/app/service/StoreService.php +++ /dev/null @@ -1,164 +0,0 @@ - $extraData 额外的门店信息 - * @return string 门店ID - */ - public function getOrCreateStoreByName( - string $storeName, - ?string $source = null, - array $extraData = [] - ): string { - // 1. 先查找是否已存在同名门店（正常状态） - $existingStore = $this->storeRepository->findByStoreName($storeName); - - if ($existingStore) { - LoggerHelper::logBusiness('store_found_by_name', [ - 'store_name' => $storeName, - 'store_id' => $existingStore->store_id, - ]); - return $existingStore->store_id; - } - - // 2. 如果不存在，创建新门店 - $storeId = $this->createStore($storeName, $source, $extraData); - - LoggerHelper::logBusiness('store_created_by_name', [ - 'store_name' => $storeName, - 'store_id' => $storeId, - 'source' => $source, - ]); - - return $storeId; - } - - /** - * 创建门店 - * - * @param string $storeName 门店名称 - * @param string|null $source 数据源标识 - * @param array $extraData 额外的门店信息 - * @return string 门店ID - */ - public function createStore(string $storeName, ?string $source = null, array $extraData = []): string - { - $now = new \DateTimeImmutable('now'); - $storeId = UuidGenerator::uuid4()->toString(); - - // 生成门店编码：如果提供了store_code则使用，否则自动生成 - $storeCode = $extraData['store_code'] ?? $this->generateStoreCode($storeName, $source); - - // 检查门店编码是否已存在 - $existingStore = $this->storeRepository->findByStoreCode($storeCode); - if ($existingStore) { - // 如果编码已存在，重新生成 - $storeCode = $this->generateStoreCode($storeName, $source, true); - } - - // 创建门店记录 - $store = new StoreRepository(); - $store->store_id = $storeId; - $store->store_code = $storeCode; - $store->store_name = $storeName; - $store->store_type = $extraData['store_type'] ?? '线上店'; // 默认线上店 - $store->store_level = $extraData['store_level'] ?? null; - $store->industry_id = $extraData['industry_id'] ?? 'default'; // 默认行业ID，后续可配置 - $store->industry_detail_id = $extraData['industry_detail_id'] ?? null; - $store->store_address = $extraData['store_address'] ?? null; - $store->store_province = $extraData['store_province'] ?? null; - $store->store_city = $extraData['store_city'] ?? null; - $store->store_district = $extraData['store_district'] ?? null; - $store->store_business_area = $extraData['store_business_area'] ?? null; - $store->store_longitude = isset($extraData['store_longitude']) ? (float)$extraData['store_longitude'] : null; - $store->store_latitude = isset($extraData['store_latitude']) ? (float)$extraData['store_latitude'] : null; - $store->store_phone = $extraData['store_phone'] ?? null; - $store->status = 0; // 0-正常 - $store->create_time = $now; - $store->update_time = $now; - - $store->save(); - - LoggerHelper::logBusiness('store_created', [ - 'store_id' => $storeId, - 'store_name' => $storeName, - 'store_code' => $storeCode, - 'store_type' => $store->store_type, - ]); - - return $storeId; - } - - /** - * 生成门店编码 - * - * @param string $storeName 门店名称 - * @param string|null $source 数据源标识 - * @param bool $addTimestamp 是否添加时间戳（用于避免重复） - * @return string 门店编码 - */ - private function generateStoreCode(string $storeName, ?string $source = null, bool $addTimestamp = false): string - { - // 清理门店名称，移除特殊字符，保留中英文和数字 - $cleanedName = preg_replace('/[^\p{L}\p{N}]/u', '', $storeName); - - // 如果名称过长，截取前20个字符 - if (mb_strlen($cleanedName) > 20) { - $cleanedName = mb_substr($cleanedName, 0, 20); - } - - // 如果名称为空，使用默认值 - if (empty($cleanedName)) { - $cleanedName = 'STORE'; - } - - // 生成编码：{source}_{cleaned_name}_{hash} - $hash = substr(md5($storeName . ($source ?? '')), 0, 8); - $code = strtoupper(($source ? $source . '_' : '') . $cleanedName . '_' . $hash); - - // 如果需要添加时间戳（用于避免重复） - if ($addTimestamp) { - $code .= '_' . time(); - } - - return $code; - } - - /** - * 根据门店ID获取门店信息 - * - * @param string $storeId 门店ID - * @return StoreRepository|null - */ - public function getStoreById(string $storeId): ?StoreRepository - { - return $this->storeRepository->newQuery() - ->where('store_id', $storeId) - ->first(); - } -} - diff --git a/Moncter/Moncter/app/service/TagInitService.php b/Moncter/Moncter/app/service/TagInitService.php deleted file mode 100644 index 28592d6b..00000000 --- a/Moncter/Moncter/app/service/TagInitService.php +++ /dev/null @@ -1,226 +0,0 @@ -= 5000） - $this->createTagIfNotExists([ - 'tag_id' => UuidGenerator::uuid4()->toString(), - 'tag_code' => 'high_consumer', - 'tag_name' => '高消费用户', - 'category' => '消费能力', - 'rule_type' => 'simple', - 'rule_config' => [ - 'rule_type' => 'simple', - 'conditions' => [ - [ - 'field' => 'total_amount', - 'operator' => '>=', - 'value' => 5000, - ], - ], - 'tag_value' => 'high', - 'confidence' => 1.0, - ], - 'update_frequency' => 'real_time', - 'priority' => 1, - 'description' => '总消费金额大于等于5000的用户', - 'status' => 0, - 'version' => 1, - 'create_time' => $now, - 'update_time' => $now, - ]); - - // 标签2：活跃用户（消费次数 >= 10） - $this->createTagIfNotExists([ - 'tag_id' => UuidGenerator::uuid4()->toString(), - 'tag_code' => 'active_user', - 'tag_name' => '活跃用户', - 'category' => '活跃度', - 'rule_type' => 'simple', - 'rule_config' => [ - 'rule_type' => 'simple', - 'conditions' => [ - [ - 'field' => 'total_count', - 'operator' => '>=', - 'value' => 10, - ], - ], - 'tag_value' => 'active', - 'confidence' => 1.0, - ], - 'update_frequency' => 'real_time', - 'priority' => 2, - 'description' => '消费次数大于等于10次的用户', - 'status' => 0, - 'version' => 1, - 'create_time' => $now, - 'update_time' => $now, - ]); - - // 标签3：中消费用户（总消费金额 >= 1000 且 < 5000） - $this->createTagIfNotExists([ - 'tag_id' => UuidGenerator::uuid4()->toString(), - 'tag_code' => 'medium_consumer', - 'tag_name' => '中消费用户', - 'category' => '消费能力', - 'rule_type' => 'simple', - 'rule_config' => [ - 'rule_type' => 'simple', - 'conditions' => [ - [ - 'field' => 'total_amount', - 'operator' => '>=', - 'value' => 1000, - ], - [ - 'field' => 'total_amount', - 'operator' => '<', - 'value' => 5000, - ], - ], - 'tag_value' => 'medium', - 'confidence' => 1.0, - ], - 'update_frequency' => 'real_time', - 'priority' => 3, - 'description' => '总消费金额在1000-5000之间的用户', - 'status' => 0, - 'version' => 1, - 'create_time' => $now, - 'update_time' => $now, - ]); - - // 标签4：低消费用户（总消费金额 < 1000） - $this->createTagIfNotExists([ - 'tag_id' => UuidGenerator::uuid4()->toString(), - 'tag_code' => 'low_consumer', - 'tag_name' => '低消费用户', - 'category' => '消费能力', - 'rule_type' => 'simple', - 'rule_config' => [ - 'rule_type' => 'simple', - 'conditions' => [ - [ - 'field' => 'total_amount', - 'operator' => '<', - 'value' => 1000, - ], - ], - 'tag_value' => 'low', - 'confidence' => 1.0, - ], - 'update_frequency' => 'real_time', - 'priority' => 4, - 'description' => '总消费金额小于1000的用户', - 'status' => 0, - 'version' => 1, - 'create_time' => $now, - 'update_time' => $now, - ]); - - // 标签5：新用户（消费次数 < 3） - $this->createTagIfNotExists([ - 'tag_id' => UuidGenerator::uuid4()->toString(), - 'tag_code' => 'new_user', - 'tag_name' => '新用户', - 'category' => '活跃度', - 'rule_type' => 'simple', - 'rule_config' => [ - 'rule_type' => 'simple', - 'conditions' => [ - [ - 'field' => 'total_count', - 'operator' => '<', - 'value' => 3, - ], - ], - 'tag_value' => 'new', - 'confidence' => 1.0, - ], - 'update_frequency' => 'real_time', - 'priority' => 5, - 'description' => '消费次数小于3次的用户', - 'status' => 0, - 'version' => 1, - 'create_time' => $now, - 'update_time' => $now, - ]); - - // 标签6：沉睡用户（最后消费时间超过90天） - $this->createTagIfNotExists([ - 'tag_id' => UuidGenerator::uuid4()->toString(), - 'tag_code' => 'dormant_user', - 'tag_name' => '沉睡用户', - 'category' => '活跃度', - 'rule_type' => 'simple', - 'rule_config' => [ - 'rule_type' => 'simple', - 'conditions' => [ - [ - 'field' => 'last_consume_time', - 'operator' => '<', - 'value' => time() - 90 * 24 * 3600, // 90天前的时间戳 - ], - ], - 'tag_value' => 'dormant', - 'confidence' => 1.0, - ], - 'update_frequency' => 'real_time', - 'priority' => 6, - 'description' => '最后消费时间超过90天的用户', - 'status' => 0, - 'version' => 1, - 'create_time' => $now, - 'update_time' => $now, - ]); - } - - /** - * 如果标签不存在则创建 - * - * @param array $tagData - */ - private function createTagIfNotExists(array $tagData): void - { - $existing = $this->tagDefinitionRepository->newQuery() - ->where('tag_code', $tagData['tag_code']) - ->first(); - - if (!$existing) { - $tag = new TagDefinitionRepository(); - foreach ($tagData as $key => $value) { - $tag->$key = $value; - } - $tag->save(); - echo "已创建标签: {$tagData['tag_name']} ({$tagData['tag_code']})\n"; - } else { - echo "标签已存在: {$tagData['tag_name']} ({$tagData['tag_code']})\n"; - } - } -} - diff --git a/Moncter/Moncter/app/service/TagRuleEngine/SimpleRuleEngine.php b/Moncter/Moncter/app/service/TagRuleEngine/SimpleRuleEngine.php deleted file mode 100644 index 98949b20..00000000 --- a/Moncter/Moncter/app/service/TagRuleEngine/SimpleRuleEngine.php +++ /dev/null @@ -1,96 +0,0 @@ - $ruleConfig 规则配置（从 tag_definitions.rule_config 解析） - * @param array $userData 用户数据（从 user_profile 获取） - * @return array{value: mixed, confidence: float} 返回标签值和置信度 - */ - public function calculate(array $ruleConfig, array $userData): array - { - if (!isset($ruleConfig['rule_type']) || $ruleConfig['rule_type'] !== 'simple') { - throw new \InvalidArgumentException('规则类型必须是 simple'); - } - - if (!isset($ruleConfig['conditions']) || !is_array($ruleConfig['conditions'])) { - throw new \InvalidArgumentException('规则配置中缺少 conditions'); - } - - // 执行所有条件判断 - $allMatch = true; - foreach ($ruleConfig['conditions'] as $condition) { - if (!$this->evaluateCondition($condition, $userData)) { - $allMatch = false; - break; - } - } - - // 如果所有条件都满足，返回标签值 - if ($allMatch) { - // 简单标签：如果满足条件，标签值为 true 或指定的值 - $tagValue = $ruleConfig['tag_value'] ?? true; - $confidence = $ruleConfig['confidence'] ?? 1.0; - - return [ - 'value' => $tagValue, - 'confidence' => (float)$confidence, - ]; - } - - // 条件不满足，返回 false - return [ - 'value' => false, - 'confidence' => 0.0, - ]; - } - - /** - * 评估单个条件 - * - * @param array $condition 条件配置：{field, operator, value} - * @param array $userData 用户数据 - * @return bool - */ - private function evaluateCondition(array $condition, array $userData): bool - { - if (!isset($condition['field']) || !isset($condition['operator']) || !isset($condition['value'])) { - throw new \InvalidArgumentException('条件配置不完整：需要 field, operator, value'); - } - - $field = $condition['field']; - $operator = $condition['operator']; - $expectedValue = $condition['value']; - - // 从用户数据中获取字段值 - if (!isset($userData[$field])) { - // 字段不存在，根据运算符判断（例如 > 0 时，不存在视为 0） - $actualValue = 0; - } else { - $actualValue = $userData[$field]; - } - - // 根据运算符进行比较 - return match ($operator) { - '>' => $actualValue > $expectedValue, - '>=' => $actualValue >= $expectedValue, - '<' => $actualValue < $expectedValue, - '<=' => $actualValue <= $expectedValue, - '=' => $actualValue == $expectedValue, - '!=' => $actualValue != $expectedValue, - 'in' => in_array($actualValue, (array)$expectedValue), - 'not_in' => !in_array($actualValue, (array)$expectedValue), - default => throw new \InvalidArgumentException("不支持的运算符: {$operator}"), - }; - } -} - diff --git a/Moncter/Moncter/app/service/TagService.php b/Moncter/Moncter/app/service/TagService.php deleted file mode 100644 index af7b0af1..00000000 --- a/Moncter/Moncter/app/service/TagService.php +++ /dev/null @@ -1,587 +0,0 @@ -|null $tagIds 要计算的标签ID列表（null 表示计算所有启用且更新频率为 real_time 的标签） - * @return array 返回更新的标签信息 - */ - public function calculateTags(string $userId, ?array $tagIds = null): array - { - // 获取用户数据 - $user = $this->userProfileRepository->findByUserId($userId); - if (!$user) { - throw new \InvalidArgumentException("用户不存在: {$userId}"); - } - - // 准备用户数据（用于规则引擎计算） - $userData = [ - 'total_amount' => (float)($user->total_amount ?? 0), - 'total_count' => (int)($user->total_count ?? 0), - 'last_consume_time' => $user->last_consume_time ? $user->last_consume_time->getTimestamp() : 0, - ]; - - // 获取要计算的标签定义 - $tagDefinitions = $this->getTagDefinitions($tagIds); - - $updatedTags = []; - $now = new \DateTimeImmutable('now'); - - foreach ($tagDefinitions as $tagDef) { - try { - // 解析规则配置 - $ruleConfig = is_string($tagDef->rule_config) - ? json_decode($tagDef->rule_config, true) - : $tagDef->rule_config; - - if (!$ruleConfig) { - continue; - } - - // 根据规则类型选择计算引擎 - if ($ruleConfig['rule_type'] === 'simple') { - $result = $this->ruleEngine->calculate($ruleConfig, $userData); - } else { - // 其他规则类型（pipeline/custom）暂不支持 - continue; - } - - // 获取旧标签值（用于历史记录） - $oldTag = $this->userTagRepository->newQuery() - ->where('user_id', $userId) - ->where('tag_id', $tagDef->tag_id) - ->first(); - - $oldValue = $oldTag ? $oldTag->tag_value : null; - - // 更新或创建标签 - if ($oldTag) { - $oldTag->tag_value = $this->formatTagValue($result['value']); - $oldTag->tag_value_type = $this->getTagValueType($result['value']); - $oldTag->confidence = $result['confidence']; - $oldTag->update_time = $now; - $oldTag->save(); - $userTag = $oldTag; - } else { - $userTag = new UserTagRepository(); - $userTag->user_id = $userId; - $userTag->tag_id = $tagDef->tag_id; - $userTag->tag_value = $this->formatTagValue($result['value']); - $userTag->tag_value_type = $this->getTagValueType($result['value']); - $userTag->confidence = $result['confidence']; - $userTag->effective_time = $now; - $userTag->create_time = $now; - $userTag->update_time = $now; - $userTag->save(); - } - - // 记录标签变更历史（仅当值发生变化时） - if ($oldValue !== $userTag->tag_value) { - $this->recordTagHistory($userId, $tagDef->tag_id, $oldValue, $userTag->tag_value, $now); - } - - $updatedTags[] = [ - 'tag_id' => $tagDef->tag_id, - 'tag_code' => $tagDef->tag_code, - 'tag_name' => $tagDef->tag_name, - 'value' => $userTag->tag_value, - 'confidence' => $userTag->confidence, - ]; - - // 记录标签计算日志 - LoggerHelper::logTagCalculation($userId, $tagDef->tag_id, [ - 'tag_code' => $tagDef->tag_code, - 'value' => $userTag->tag_value, - 'confidence' => $userTag->confidence, - ]); - } catch (\Throwable $e) { - // 记录错误但继续处理其他标签 - LoggerHelper::logError($e, [ - 'user_id' => $userId, - 'tag_id' => $tagDef->tag_id ?? null, - 'tag_code' => $tagDef->tag_code ?? null, - ]); - } - } - - // 更新用户的标签更新时间 - $user->tags_update_time = $now; - $user->save(); - - return $updatedTags; - } - - /** - * 获取标签定义列表 - * - * @param array|null $tagIds - * @return \Illuminate\Database\Eloquent\Collection - */ - private function getTagDefinitions(?array $tagIds = null) - { - $query = $this->tagDefinitionRepository->newQuery() - ->where('status', 0); // 只获取启用的标签 - - if ($tagIds !== null) { - $query->whereIn('tag_id', $tagIds); - } else { - // 默认只计算实时更新的标签 - $query->where('update_frequency', 'real_time'); - } - - return $query->get(); - } - - /** - * 格式化标签值 - * - * @param mixed $value - * @return string - */ - private function formatTagValue($value): string - { - if (is_bool($value)) { - return $value ? 'true' : 'false'; - } - if (is_array($value) || is_object($value)) { - return json_encode($value); - } - return (string)$value; - } - - /** - * 获取标签值类型 - * - * @param mixed $value - * @return string - */ - private function getTagValueType($value): string - { - if (is_bool($value)) { - return 'boolean'; - } - if (is_int($value) || is_float($value)) { - return 'number'; - } - if (is_array($value) || is_object($value)) { - return 'json'; - } - return 'string'; - } - - /** - * 记录标签变更历史 - * - * @param string $userId - * @param string $tagId - * @param mixed $oldValue - * @param string $newValue - * @param \DateTimeInterface $changeTime - */ - private function recordTagHistory(string $userId, string $tagId, $oldValue, string $newValue, \DateTimeInterface $changeTime): void - { - $history = new TagHistoryRepository(); - $history->history_id = UuidGenerator::uuid4()->toString(); - $history->user_id = $userId; - $history->tag_id = $tagId; - $history->old_value = $oldValue !== null ? (string)$oldValue : null; - $history->new_value = $newValue; - $history->change_reason = 'auto_calculate'; - $history->change_time = $changeTime; - $history->operator = 'system'; - $history->save(); - } - - /** - * 根据标签筛选用户 - * - * @param array $conditions 查询条件数组，每个条件包含： - * - tag_code: 标签编码（必填） - * - operator: 操作符（=, !=, >, >=, <, <=, in, not_in）（必填） - * - value: 标签值（必填） - * @param string $logic 多个条件之间的逻辑关系：AND 或 OR（默认 AND） - * @param int $page 页码（从1开始） - * @param int $pageSize 每页数量 - * @param bool $includeUserInfo 是否包含用户基本信息 - * @return array 返回符合条件的用户列表 - */ - public function filterUsersByTags( - array $conditions, - string $logic = 'AND', - int $page = 1, - int $pageSize = 20, - bool $includeUserInfo = false - ): array { - if (empty($conditions)) { - return [ - 'users' => [], - 'total' => 0, - 'page' => $page, - 'page_size' => $pageSize, - ]; - } - - // 1. 根据 tag_code 获取 tag_id 列表 - $tagCodes = array_column($conditions, 'tag_code'); - $tagDefinitions = $this->tagDefinitionRepository->newQuery() - ->whereIn('tag_code', $tagCodes) - ->get() - ->keyBy('tag_code'); - - $tagIdMap = []; - foreach ($tagDefinitions as $tagDef) { - $tagIdMap[$tagDef->tag_code] = $tagDef->tag_id; - } - - // 验证所有 tag_code 都存在 - $missingTags = array_diff($tagCodes, array_keys($tagIdMap)); - if (!empty($missingTags)) { - throw new \InvalidArgumentException('标签编码不存在: ' . implode(', ', $missingTags)); - } - - // 2. 根据逻辑类型处理查询 - if (strtoupper($logic) === 'OR') { - // OR 逻辑：使用 orWhere，查询满足任一条件的用户 - $query = $this->userTagRepository->newQuery(); - $query->where(function ($q) use ($conditions, $tagIdMap) { - $first = true; - foreach ($conditions as $condition) { - $tagId = $tagIdMap[$condition['tag_code']]; - $operator = $condition['operator'] ?? '='; - $value = $condition['value']; - $formattedValue = $this->formatTagValue($value); - - if ($first) { - $this->applyTagCondition($q, $tagId, $operator, $formattedValue, $value); - $first = false; - } else { - $q->orWhere(function ($subQ) use ($tagId, $operator, $formattedValue, $value) { - $this->applyTagCondition($subQ, $tagId, $operator, $formattedValue, $value); - }); - } - } - }); - - // 分页查询 - $total = $query->count(); - $userTags = $query->skip(($page - 1) * $pageSize) - ->take($pageSize) - ->get(); - - // 提取 user_id 列表 - $userIds = $userTags->pluck('user_id')->unique()->toArray(); - } else { - // AND 逻辑：所有条件都必须满足 - // 由于每个标签是独立的记录，需要分别查询每个条件，然后取交集 - $userIdsSets = []; - foreach ($conditions as $condition) { - $tagId = $tagIdMap[$condition['tag_code']]; - $tagDef = $tagDefinitions->get($condition['tag_code']); - $operator = $condition['operator'] ?? '='; - $value = $condition['value']; - $formattedValue = $this->formatTagValue($value); - - // 为每个条件单独查询满足条件的 user_id（先从标签表查询） - $subQuery = $this->userTagRepository->newQuery(); - $this->applyTagCondition($subQuery, $tagId, $operator, $formattedValue, $value); - $tagUserIds = $subQuery->pluck('user_id')->unique()->toArray(); - - // 如果标签表中没有符合条件的记录，且标签定义中有规则，则基于规则从用户档案表筛选 - // 这样可以处理用户还没有计算标签的情况 - if ($tagDef && $tagDef->rule_type === 'simple') { - $ruleConfig = is_string($tagDef->rule_config) - ? json_decode($tagDef->rule_config, true) - : $tagDef->rule_config; - - if ($ruleConfig && isset($ruleConfig['tag_value']) && $ruleConfig['tag_value'] === $value) { - // 基于规则从用户档案表筛选 - $profileQuery = $this->userProfileRepository->newQuery(); - $this->applyRuleToProfileQuery($profileQuery, $ruleConfig); - $profileUserIds = $profileQuery->pluck('user_id')->unique()->toArray(); - - // 合并标签表和用户档案表的查询结果（去重） - $tagUserIds = array_unique(array_merge($tagUserIds, $profileUserIds)); - } - } - - $userIdsSets[] = $tagUserIds; - } - - // 取交集：所有条件都满足的用户ID - if (empty($userIdsSets)) { - $userIds = []; - } else { - $userIds = $userIdsSets[0]; - for ($i = 1; $i < count($userIdsSets); $i++) { - $userIds = array_intersect($userIds, $userIdsSets[$i]); - } - } - - // 如果没有满足所有条件的用户，直接返回空结果 - if (empty($userIds)) { - return [ - 'users' => [], - 'total' => 0, - 'page' => $page, - 'page_size' => $pageSize, - 'total_pages' => 0, - ]; - } - - // 分页处理 - $total = count($userIds); - $offset = ($page - 1) * $pageSize; - $userIds = array_slice($userIds, $offset, $pageSize); - } - - // 5. 如果需要用户信息，则关联查询 - $users = []; - if ($includeUserInfo && !empty($userIds)) { - $userProfiles = $this->userProfileRepository->newQuery() - ->whereIn('user_id', $userIds) - ->get() - ->keyBy('user_id'); - - foreach ($userIds as $userId) { - $userProfile = $userProfiles->get($userId); - if ($userProfile) { - $users[] = [ - 'user_id' => $userId, - 'name' => $userProfile->name ?? null, - 'phone' => $userProfile->phone ?? null, - 'total_amount' => $userProfile->total_amount ?? 0, - 'total_count' => $userProfile->total_count ?? 0, - 'last_consume_time' => $userProfile->last_consume_time ?? null, - ]; - } else { - $users[] = [ - 'user_id' => $userId, - ]; - } - } - } else { - foreach ($userIds as $userId) { - $users[] = ['user_id' => $userId]; - } - } - - return [ - 'users' => $users, - 'total' => $total, - 'page' => $page, - 'page_size' => $pageSize, - 'total_pages' => (int)ceil($total / $pageSize), - ]; - } - - /** - * 应用标签查询条件到查询构建器 - * - * @param \Illuminate\Database\Eloquent\Builder $query 查询构建器 - * @param string $tagId 标签ID - * @param string $operator 操作符 - * @param string $formattedValue 格式化后的标签值 - * @param mixed $originalValue 原始标签值（用于 in/not_in） - */ - private function applyTagCondition($query, string $tagId, string $operator, string $formattedValue, $originalValue): void - { - $query->where('tag_id', $tagId); - - switch ($operator) { - case '=': - case '==': - $query->where('tag_value', $formattedValue); - break; - case '!=': - case '<>': - $query->where('tag_value', '!=', $formattedValue); - break; - case '>': - $query->where('tag_value', '>', $formattedValue); - break; - case '>=': - $query->where('tag_value', '>=', $formattedValue); - break; - case '<': - $query->where('tag_value', '<', $formattedValue); - break; - case '<=': - $query->where('tag_value', '<=', $formattedValue); - break; - case 'in': - if (!is_array($originalValue)) { - throw new \InvalidArgumentException('in 操作符的值必须是数组'); - } - $query->whereIn('tag_value', array_map([$this, 'formatTagValue'], $originalValue)); - break; - case 'not_in': - if (!is_array($originalValue)) { - throw new \InvalidArgumentException('not_in 操作符的值必须是数组'); - } - $query->whereNotIn('tag_value', array_map([$this, 'formatTagValue'], $originalValue)); - break; - default: - throw new \InvalidArgumentException("不支持的操作符: {$operator}"); - } - } - - /** - * 将标签规则应用到用户档案查询 - * - * @param \Illuminate\Database\Eloquent\Builder $query 查询构建器 - * @param array $ruleConfig 规则配置 - */ - private function applyRuleToProfileQuery($query, array $ruleConfig): void - { - if (!isset($ruleConfig['conditions']) || !is_array($ruleConfig['conditions'])) { - return; - } - - foreach ($ruleConfig['conditions'] as $condition) { - if (!isset($condition['field']) || !isset($condition['operator']) || !isset($condition['value'])) { - continue; - } - - $field = $condition['field']; - $operator = $condition['operator']; - $value = $condition['value']; - - // 将规则条件转换为用户档案表的查询条件 - switch ($operator) { - case '>': - $query->where($field, '>', $value); - break; - case '>=': - $query->where($field, '>=', $value); - break; - case '<': - $query->where($field, '<', $value); - break; - case '<=': - $query->where($field, '<=', $value); - break; - case '=': - case '==': - $query->where($field, $value); - break; - case '!=': - case '<>': - $query->where($field, '!=', $value); - break; - case 'in': - if (is_array($value)) { - $query->whereIn($field, $value); - } - break; - case 'not_in': - if (is_array($value)) { - $query->whereNotIn($field, $value); - } - break; - } - } - - // 只查询未删除的用户 - $query->where('status', 0); - } - - /** - * 获取指定用户的标签列表 - * - * @param string $userId - * @return array> - */ - public function getUserTags(string $userId): array - { - $userTags = $this->userTagRepository->newQuery() - ->where('user_id', $userId) - ->get(); - - $result = []; - foreach ($userTags as $userTag) { - $tagDef = $this->tagDefinitionRepository->newQuery() - ->where('tag_id', $userTag->tag_id) - ->first(); - - $result[] = [ - 'tag_id' => $userTag->tag_id, - 'tag_code' => $tagDef ? $tagDef->tag_code : null, - 'tag_name' => $tagDef ? $tagDef->tag_name : null, - 'category' => $tagDef ? $tagDef->category : null, - 'tag_value' => $userTag->tag_value, - 'tag_value_type' => $userTag->tag_value_type, - 'confidence' => $userTag->confidence, - 'effective_time' => $userTag->effective_time, - 'expire_time' => $userTag->expire_time, - 'update_time' => $userTag->update_time, - ]; - } - return $result; - } - - /** - * 删除用户的指定标签 - * - * @param string $userId 用户ID - * @param string $tagId 标签ID - * @return bool 是否删除成功 - */ - public function deleteUserTag(string $userId, string $tagId): bool - { - $userTag = $this->userTagRepository->newQuery() - ->where('user_id', $userId) - ->where('tag_id', $tagId) - ->first(); - - if (!$userTag) { - return false; - } - - $oldValue = $userTag->tag_value; - - // 删除标签 - $userTag->delete(); - - // 记录历史 - $now = new \DateTimeImmutable('now'); - $this->recordTagHistory($userId, $tagId, $oldValue, null, $now, 'tag_deleted'); - - LoggerHelper::logBusiness('tag_deleted', [ - 'user_id' => $userId, - 'tag_id' => $tagId, - ]); - - return true; - } -} - diff --git a/Moncter/Moncter/app/service/TagTaskExecutor.php b/Moncter/Moncter/app/service/TagTaskExecutor.php deleted file mode 100644 index 9ed8503b..00000000 --- a/Moncter/Moncter/app/service/TagTaskExecutor.php +++ /dev/null @@ -1,331 +0,0 @@ -taskRepository->find($taskId); - - if (!$task) { - throw new \InvalidArgumentException("任务不存在: {$taskId}"); - } - - // 创建执行记录 - $executionId = UuidGenerator::uuid4()->toString(); - $execution = $this->executionRepository->create([ - 'execution_id' => $executionId, - 'task_id' => $taskId, - 'started_at' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - 'status' => 'running', - 'processed_users' => 0, - 'success_count' => 0, - 'error_count' => 0, - ]); - - try { - // 获取用户列表 - $userIds = $this->getUserIds($task); - $totalUsers = count($userIds); - - // 更新任务进度 - $this->updateTaskProgress($taskId, [ - 'total_users' => $totalUsers, - 'processed_users' => 0, - 'success_count' => 0, - 'error_count' => 0, - 'percentage' => 0, - ]); - - // 获取目标标签ID列表 - $targetTagIds = $task->target_tag_ids ?? null; - - // 批量处理用户 - $batchSize = $task->config['batch_size'] ?? 100; - $processedCount = 0; - $successCount = 0; - $errorCount = 0; - - foreach (array_chunk($userIds, $batchSize) as $batch) { - // 检查任务状态（是否被暂停或停止） - if (!$this->checkTaskStatus($taskId)) { - LoggerHelper::logBusiness('tag_task_paused_or_stopped', [ - 'task_id' => $taskId, - 'execution_id' => $executionId, - 'processed' => $processedCount, - ]); - break; - } - - // 批量处理用户 - foreach ($batch as $userId) { - try { - // 计算用户标签 - $this->tagService->calculateTags($userId, $targetTagIds); - $successCount++; - } catch (\Exception $e) { - $errorCount++; - LoggerHelper::logError($e, [ - 'component' => 'TagTaskExecutor', - 'action' => 'calculateTags', - 'task_id' => $taskId, - 'user_id' => $userId, - ]); - - // 根据错误处理策略决定是否继续 - $errorHandling = $task->config['error_handling'] ?? 'skip'; - if ($errorHandling === 'stop') { - throw $e; - } - } - - $processedCount++; - - // 每处理一定数量更新一次进度 - if ($processedCount % 10 === 0) { - $this->updateTaskProgress($taskId, [ - 'processed_users' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'percentage' => $totalUsers > 0 ? round(($processedCount / $totalUsers) * 100, 2) : 0, - ]); - - // 更新执行记录 - $this->executionRepository->where('execution_id', $executionId)->update([ - 'processed_users' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - ]); - } - } - } - - // 更新最终进度 - $this->updateTaskProgress($taskId, [ - 'processed_users' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'percentage' => $totalUsers > 0 ? round(($processedCount / $totalUsers) * 100, 2) : 100, - ]); - - // 更新执行记录为完成 - $this->executionRepository->where('execution_id', $executionId)->update([ - 'status' => 'completed', - 'finished_at' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - 'processed_users' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - ]); - - // 更新任务统计 - $this->updateTaskStatistics($taskId, $successCount, $errorCount); - - LoggerHelper::logBusiness('tag_task_execution_completed', [ - 'task_id' => $taskId, - 'execution_id' => $executionId, - 'total_users' => $totalUsers, - 'processed' => $processedCount, - 'success' => $successCount, - 'error' => $errorCount, - ]); - - } catch (\Throwable $e) { - // 更新执行记录为失败 - $this->executionRepository->where('execution_id', $executionId)->update([ - 'status' => 'failed', - 'finished_at' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - 'error_message' => $e->getMessage(), - ]); - - // 更新任务状态为错误 - $this->taskRepository->where('task_id', $taskId)->update([ - 'status' => 'error', - 'progress.status' => 'error', - 'progress.last_error' => $e->getMessage(), - ]); - - LoggerHelper::logError($e, [ - 'component' => 'TagTaskExecutor', - 'action' => 'execute', - 'task_id' => $taskId, - 'execution_id' => $executionId, - ]); - - throw $e; - } - } - - /** - * 获取用户ID列表 - * - * @param mixed $task 任务对象 - * @return array 用户ID列表 - */ - private function getUserIds($task): array - { - $userScope = $task->user_scope ?? ['type' => 'all']; - $scopeType = $userScope['type'] ?? 'all'; - - switch ($scopeType) { - case 'all': - // 获取所有用户 - $users = $this->userProfileRepository->newQuery() - ->where('status', 0) // 只获取正常状态的用户 - ->get(); - return $users->pluck('user_id')->toArray(); - - case 'list': - // 指定用户列表 - return $userScope['user_ids'] ?? []; - - case 'filter': - // 按条件筛选 - // 这里可以扩展支持更复杂的筛选条件 - $query = $this->userProfileRepository->newQuery() - ->where('status', 0); - - // 可以添加更多筛选条件 - if (isset($userScope['conditions']) && is_array($userScope['conditions'])) { - foreach ($userScope['conditions'] as $condition) { - $field = $condition['field'] ?? ''; - $operator = $condition['operator'] ?? '='; - $value = $condition['value'] ?? null; - - if (empty($field)) { - continue; - } - - switch ($operator) { - case '>': - $query->where($field, '>', $value); - break; - case '>=': - $query->where($field, '>=', $value); - break; - case '<': - $query->where($field, '<', $value); - break; - case '<=': - $query->where($field, '<=', $value); - break; - case '=': - $query->where($field, $value); - break; - case '!=': - $query->where($field, '!=', $value); - break; - case 'in': - if (is_array($value)) { - $query->whereIn($field, $value); - } - break; - } - } - } - - $users = $query->get(); - return $users->pluck('user_id')->toArray(); - - default: - throw new \InvalidArgumentException("不支持的用户范围类型: {$scopeType}"); - } - } - - /** - * 更新任务进度 - */ - private function updateTaskProgress(string $taskId, array $progress): void - { - $task = $this->taskRepository->find($taskId); - if (!$task) { - return; - } - - $currentProgress = $task->progress ?? []; - $currentProgress = array_merge($currentProgress, $progress); - $currentProgress['status'] = 'running'; - - $this->taskRepository->where('task_id', $taskId)->update([ - 'progress' => $currentProgress, - 'updated_at' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]); - } - - /** - * 更新任务统计 - */ - private function updateTaskStatistics(string $taskId, int $successCount, int $errorCount): void - { - $task = $this->taskRepository->find($taskId); - if (!$task) { - return; - } - - $statistics = $task->statistics ?? []; - $statistics['total_executions'] = ($statistics['total_executions'] ?? 0) + 1; - $statistics['success_executions'] = ($statistics['success_executions'] ?? 0) + ($errorCount === 0 ? 1 : 0); - $statistics['failed_executions'] = ($statistics['failed_executions'] ?? 0) + ($errorCount > 0 ? 1 : 0); - $statistics['last_run_time'] = new \MongoDB\BSON\UTCDateTime(time() * 1000); - - $this->taskRepository->where('task_id', $taskId)->update([ - 'statistics' => $statistics, - 'updated_at' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]); - } - - /** - * 检查任务状态 - */ - private function checkTaskStatus(string $taskId): bool - { - // 检查Redis标志 - if (RedisHelper::exists("tag_task:{$taskId}:pause")) { - return false; - } - if (RedisHelper::exists("tag_task:{$taskId}:stop")) { - return false; - } - - // 检查数据库状态 - $task = $this->taskRepository->find($taskId); - if ($task && in_array($task->status, ['paused', 'stopped', 'error'])) { - return false; - } - - return true; - } -} - diff --git a/Moncter/Moncter/app/service/TagTaskService.php b/Moncter/Moncter/app/service/TagTaskService.php deleted file mode 100644 index d628f384..00000000 --- a/Moncter/Moncter/app/service/TagTaskService.php +++ /dev/null @@ -1,283 +0,0 @@ - $taskData 任务数据 - * @return array 创建的任务信息 - */ - public function createTask(array $taskData): array - { - $taskId = UuidGenerator::uuid4()->toString(); - - $task = [ - 'task_id' => $taskId, - 'name' => $taskData['name'] ?? '未命名标签任务', - 'description' => $taskData['description'] ?? '', - 'task_type' => $taskData['task_type'] ?? 'full', - 'target_tag_ids' => $taskData['target_tag_ids'] ?? [], - 'user_scope' => $taskData['user_scope'] ?? ['type' => 'all'], - 'schedule' => $taskData['schedule'] ?? [ - 'enabled' => false, - 'cron' => null, - ], - 'config' => $taskData['config'] ?? [ - 'concurrency' => 10, - 'batch_size' => 100, - 'error_handling' => 'skip', - ], - 'status' => 'pending', - 'progress' => [ - 'total_users' => 0, - 'processed_users' => 0, - 'success_count' => 0, - 'error_count' => 0, - 'percentage' => 0, - ], - 'statistics' => [ - 'total_executions' => 0, - 'success_executions' => 0, - 'failed_executions' => 0, - 'last_run_time' => null, - ], - 'created_by' => $taskData['created_by'] ?? 'system', - 'created_at' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - 'updated_at' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]; - - $this->taskRepository->create($task); - - LoggerHelper::logBusiness('tag_task_created', [ - 'task_id' => $taskId, - 'task_name' => $task['name'], - ]); - - return $task; - } - - /** - * 更新任务 - */ - public function updateTask(string $taskId, array $taskData): bool - { - $task = $this->taskRepository->find($taskId); - - if (!$task) { - throw new \InvalidArgumentException("任务不存在: {$taskId}"); - } - - if ($task->status === 'running') { - $allowedFields = ['name', 'description', 'schedule']; - $taskData = array_intersect_key($taskData, array_flip($allowedFields)); - } - - $taskData['updated_at'] = new \MongoDB\BSON\UTCDateTime(time() * 1000); - - return $this->taskRepository->where('task_id', $taskId)->update($taskData) > 0; - } - - /** - * 删除任务 - */ - public function deleteTask(string $taskId): bool - { - $task = $this->taskRepository->find($taskId); - - if (!$task) { - throw new \InvalidArgumentException("任务不存在: {$taskId}"); - } - - if ($task->status === 'running') { - $this->stopTask($taskId); - } - - return $this->taskRepository->where('task_id', $taskId)->delete() > 0; - } - - /** - * 启动任务 - */ - public function startTask(string $taskId): bool - { - $task = $this->taskRepository->find($taskId); - - if (!$task) { - throw new \InvalidArgumentException("任务不存在: {$taskId}"); - } - - if ($task->status === 'running') { - throw new \RuntimeException("任务已在运行中: {$taskId}"); - } - - $this->taskRepository->where('task_id', $taskId)->update([ - 'status' => 'running', - 'updated_at' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]); - - // 设置Redis标志，通知调度器启动任务 - RedisHelper::set("tag_task:{$taskId}:start", '1', 3600); - - LoggerHelper::logBusiness('tag_task_started', [ - 'task_id' => $taskId, - ]); - - return true; - } - - /** - * 暂停任务 - */ - public function pauseTask(string $taskId): bool - { - $task = $this->taskRepository->find($taskId); - - if (!$task) { - throw new \InvalidArgumentException("任务不存在: {$taskId}"); - } - - if ($task->status !== 'running') { - throw new \RuntimeException("任务未在运行中: {$taskId}"); - } - - $this->taskRepository->where('task_id', $taskId)->update([ - 'status' => 'paused', - 'updated_at' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]); - - RedisHelper::set("tag_task:{$taskId}:pause", '1', 3600); - - return true; - } - - /** - * 停止任务 - */ - public function stopTask(string $taskId): bool - { - $task = $this->taskRepository->find($taskId); - - if (!$task) { - throw new \InvalidArgumentException("任务不存在: {$taskId}"); - } - - $this->taskRepository->where('task_id', $taskId)->update([ - 'status' => 'stopped', - 'updated_at' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]); - - RedisHelper::set("tag_task:{$taskId}:stop", '1', 3600); - - return true; - } - - /** - * 获取任务列表 - */ - public function getTaskList(array $filters = [], int $page = 1, int $pageSize = 20): array - { - $query = $this->taskRepository->query(); - - if (isset($filters['status'])) { - $query->where('status', $filters['status']); - } - if (isset($filters['task_type'])) { - $query->where('task_type', $filters['task_type']); - } - if (isset($filters['name'])) { - $query->where('name', 'like', '%' . $filters['name'] . '%'); - } - - $total = $query->count(); - $tasks = $query->orderBy('created_at', 'desc') - ->skip(($page - 1) * $pageSize) - ->take($pageSize) - ->get() - ->toArray(); - - return [ - 'tasks' => $tasks, - 'total' => $total, - 'page' => $page, - 'page_size' => $pageSize, - 'total_pages' => ceil($total / $pageSize), - ]; - } - - /** - * 获取任务详情 - */ - public function getTask(string $taskId): ?array - { - $task = $this->taskRepository->find($taskId); - return $task ? $task->toArray() : null; - } - - /** - * 获取任务执行记录 - */ - public function getExecutions(string $taskId, int $page = 1, int $pageSize = 20): array - { - $query = $this->executionRepository->query()->where('task_id', $taskId); - - $total = $query->count(); - $executions = $query->orderBy('started_at', 'desc') - ->skip(($page - 1) * $pageSize) - ->take($pageSize) - ->get() - ->toArray(); - - return [ - 'executions' => $executions, - 'total' => $total, - 'page' => $page, - 'page_size' => $pageSize, - 'total_pages' => ceil($total / $pageSize), - ]; - } - - /** - * 执行任务（供调度器调用） - */ - public function executeTask(string $taskId): void - { - $executor = new \app\service\TagTaskExecutor( - $this->taskRepository, - $this->executionRepository, - $this->userProfileRepository, - new \app\repository\TagDefinitionRepository(), - $this->tagService - ); - - $executor->execute($taskId); - } -} - diff --git a/Moncter/Moncter/app/service/UserPhoneService.php b/Moncter/Moncter/app/service/UserPhoneService.php deleted file mode 100644 index 69c05997..00000000 --- a/Moncter/Moncter/app/service/UserPhoneService.php +++ /dev/null @@ -1,537 +0,0 @@ - $options 可选参数 - * - type: 手机号类型（personal/work/backup/other） - * - is_verified: 是否已验证 - * - effective_time: 生效时间（默认当前时间） - * - expire_time: 失效时间（默认null，表示当前有效） - * - source: 来源（registration/update/manual/import） - * @return string 关联ID - * @throws \InvalidArgumentException - */ - public function addPhoneToUser(string $userId, string $phoneNumber, array $options = []): string - { - \Workerman\Worker::safeEcho("\n"); - \Workerman\Worker::safeEcho("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n"); - \Workerman\Worker::safeEcho("[UserPhoneService::addPhoneToUser] 【断点1-方法入口】开始执行\n"); - \Workerman\Worker::safeEcho("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n"); - \Workerman\Worker::safeEcho("【断点1】原始传入参数:\n"); - \Workerman\Worker::safeEcho(" - userId: {$userId}\n"); - \Workerman\Worker::safeEcho(" - phoneNumber: {$phoneNumber}\n"); - \Workerman\Worker::safeEcho(" - options: " . json_encode($options, JSON_UNESCAPED_UNICODE) . "\n"); - - \Workerman\Worker::safeEcho("\n【断点2-参数处理】开始处理参数\n"); - $phoneNumber = trim($phoneNumber); - \Workerman\Worker::safeEcho(" - trim后phoneNumber: {$phoneNumber}\n"); - - // 检查手机号是否为空 - if (empty($phoneNumber)) { - \Workerman\Worker::safeEcho("【断点2】❌ 手机号为空，抛出异常\n"); - throw new \InvalidArgumentException('手机号不能为空'); - } - - // 过滤非数字字符 - $originalPhone = $phoneNumber; - \Workerman\Worker::safeEcho("\n【断点3-过滤处理】开始过滤非数字字符\n"); - $phoneNumber = $this->filterPhoneNumber($phoneNumber); - \Workerman\Worker::safeEcho(" - 原始手机号: {$originalPhone}\n"); - \Workerman\Worker::safeEcho(" - 过滤后手机号: {$phoneNumber}\n"); - \Workerman\Worker::safeEcho(" - 过滤后长度: " . strlen($phoneNumber) . "\n"); - - // 检查过滤后是否为空 - if (empty($phoneNumber)) { - \Workerman\Worker::safeEcho("【断点3】❌ 手机号过滤后为空，抛出异常\n"); - throw new \InvalidArgumentException("手机号过滤后为空: {$originalPhone}"); - } - - \Workerman\Worker::safeEcho("\n【断点4-格式验证】开始验证手机号格式\n"); - // 验证手机号格式（过滤后的手机号） - $isValid = $this->validatePhoneNumber($phoneNumber); - \Workerman\Worker::safeEcho(" - 验证结果: " . ($isValid ? '通过 ✓' : '失败 ✗') . "\n"); - if (!$isValid) { - \Workerman\Worker::safeEcho("【断点4】❌ 手机号格式验证失败，抛出异常\n"); - \Workerman\Worker::safeEcho(" - 验证规则: /^1[3-9]\\d{9}$/\n"); - \Workerman\Worker::safeEcho(" - 实际值: {$phoneNumber}\n"); - \Workerman\Worker::safeEcho(" - 长度: " . strlen($phoneNumber) . "\n"); - throw new \InvalidArgumentException("手机号格式不正确: {$originalPhone} -> {$phoneNumber} (长度: " . strlen($phoneNumber) . ")"); - } - \Workerman\Worker::safeEcho("【断点4】✓ 格式验证通过\n"); - - \Workerman\Worker::safeEcho("\n【断点5-哈希计算】开始计算手机号哈希\n"); - $phoneHash = EncryptionHelper::hash($phoneNumber); - $now = new \DateTimeImmutable('now'); - $effectiveTime = $options['effective_time'] ?? $now; - \Workerman\Worker::safeEcho(" - phoneHash: {$phoneHash}\n"); - \Workerman\Worker::safeEcho(" - effectiveTime: " . $effectiveTime->format('Y-m-d H:i:s') . "\n"); - - \Workerman\Worker::safeEcho("\n【断点6-冲突检查】检查是否存在冲突关联\n"); - // 检查该手机号在effectiveTime是否已有有效关联 - // 使用effectiveTime作为查询时间点，查找是否有冲突的关联 - $existingActive = $this->phoneRelationRepository->findActiveByPhoneHash($phoneHash, $effectiveTime); - \Workerman\Worker::safeEcho(" - 查询结果: " . ($existingActive ? "找到冲突关联 (user_id: {$existingActive->user_id})" : "无冲突") . "\n"); - - if ($existingActive && $existingActive->user_id !== $userId) { - \Workerman\Worker::safeEcho("【断点6】⚠️ 发现冲突，需要失效旧关联\n"); - // 如果手机号在effectiveTime已被其他用户使用，需要先失效旧关联 - // 过期时间设置为新关联的effectiveTime（保证时间连续，避免间隙） - $existingActive->expire_time = $effectiveTime; - $existingActive->is_active = false; - $existingActive->update_time = $now; - $existingActive->save(); - \Workerman\Worker::safeEcho(" - 旧关联已失效，expire_time: " . $effectiveTime->format('Y-m-d H:i:s') . "\n"); - - LoggerHelper::logBusiness('phone_relation_expired_due_to_conflict', [ - 'phone_number' => $phoneNumber, - 'old_user_id' => $existingActive->user_id, - 'new_user_id' => $userId, - 'expire_time' => $effectiveTime->format('Y-m-d H:i:s'), - 'effective_time' => $effectiveTime->format('Y-m-d H:i:s'), - ]); - } else { - \Workerman\Worker::safeEcho("【断点6】✓ 无冲突，继续创建新关联\n"); - } - - \Workerman\Worker::safeEcho("\n【断点7-数据准备】开始准备要保存的数据\n"); - - try { - \Workerman\Worker::safeEcho(" [7.1] 创建 UserPhoneRelationRepository 对象...\n"); - // 创建新关联 - $relation = new UserPhoneRelationRepository(); - \Workerman\Worker::safeEcho(" [7.1] ✓ 对象创建成功\n"); - - \Workerman\Worker::safeEcho(" [7.2] 设置 relation_id...\n"); - $relation->relation_id = UuidGenerator::uuid4()->toString(); - \Workerman\Worker::safeEcho(" [7.2] ✓ relation_id = {$relation->relation_id}\n"); - - \Workerman\Worker::safeEcho(" [7.3] 设置 phone_number...\n"); - $relation->phone_number = $phoneNumber; - \Workerman\Worker::safeEcho(" [7.3] ✓ phone_number = {$relation->phone_number}\n"); - - \Workerman\Worker::safeEcho(" [7.4] 设置 phone_hash...\n"); - $relation->phone_hash = $phoneHash; - \Workerman\Worker::safeEcho(" [7.4] ✓ phone_hash = {$relation->phone_hash}\n"); - - \Workerman\Worker::safeEcho(" [7.5] 设置 user_id...\n"); - $relation->user_id = $userId; - \Workerman\Worker::safeEcho(" [7.5] ✓ user_id = {$relation->user_id}\n"); - - \Workerman\Worker::safeEcho(" [7.6] 设置 effective_time...\n"); - \Workerman\Worker::safeEcho(" - effectiveTime类型: " . get_class($effectiveTime) . "\n"); - \Workerman\Worker::safeEcho(" - effectiveTime值: " . $effectiveTime->format('Y-m-d H:i:s') . "\n"); - $relation->effective_time = $effectiveTime; - \Workerman\Worker::safeEcho(" [7.6] ✓ effective_time 设置完成\n"); - - \Workerman\Worker::safeEcho(" [7.7] 设置 expire_time...\n"); - $expireTimeValue = $options['expire_time'] ?? null; - \Workerman\Worker::safeEcho(" - expireTime值: " . ($expireTimeValue ? (is_object($expireTimeValue) ? $expireTimeValue->format('Y-m-d H:i:s') : $expireTimeValue) : 'null') . "\n"); - $relation->expire_time = $expireTimeValue; - \Workerman\Worker::safeEcho(" [7.7] ✓ expire_time 设置完成\n"); - - \Workerman\Worker::safeEcho(" [7.8] 设置 is_active...\n"); - // 如果 expire_time 为 null 或不存在，则 is_active 为 true - $isActiveValue = ($options['expire_time'] ?? null) === null; - \Workerman\Worker::safeEcho(" - isActive值: " . ($isActiveValue ? 'true' : 'false') . "\n"); - $relation->is_active = $isActiveValue; - \Workerman\Worker::safeEcho(" [7.8] ✓ is_active 设置完成\n"); - - \Workerman\Worker::safeEcho(" [7.9] 设置 type...\n"); - $typeValue = $options['type'] ?? 'personal'; - \Workerman\Worker::safeEcho(" - type值: {$typeValue}\n"); - $relation->type = $typeValue; - \Workerman\Worker::safeEcho(" [7.9] ✓ type 设置完成\n"); - - \Workerman\Worker::safeEcho(" [7.10] 设置 is_verified...\n"); - $isVerifiedValue = $options['is_verified'] ?? false; - \Workerman\Worker::safeEcho(" - isVerified值: " . ($isVerifiedValue ? 'true' : 'false') . "\n"); - $relation->is_verified = $isVerifiedValue; - \Workerman\Worker::safeEcho(" [7.10] ✓ is_verified 设置完成\n"); - - \Workerman\Worker::safeEcho(" [7.11] 设置 source...\n"); - $sourceValue = $options['source'] ?? 'manual'; - \Workerman\Worker::safeEcho(" - source值: {$sourceValue}\n"); - $relation->source = $sourceValue; - \Workerman\Worker::safeEcho(" [7.11] ✓ source 设置完成\n"); - - \Workerman\Worker::safeEcho(" [7.12] 设置 create_time...\n"); - \Workerman\Worker::safeEcho(" - now类型: " . get_class($now) . "\n"); - \Workerman\Worker::safeEcho(" - now值: " . $now->format('Y-m-d H:i:s') . "\n"); - $relation->create_time = $now; - \Workerman\Worker::safeEcho(" [7.12] ✓ create_time 设置完成\n"); - - \Workerman\Worker::safeEcho(" [7.13] 设置 update_time...\n"); - $relation->update_time = $now; - \Workerman\Worker::safeEcho(" [7.13] ✓ update_time 设置完成\n"); - - \Workerman\Worker::safeEcho(" [7.14] ✓ 所有属性设置完成，准备打印数据详情\n"); - - } catch (\Throwable $e) { - \Workerman\Worker::safeEcho("\n【断点7】❌ 数据准备过程中发生异常！\n"); - \Workerman\Worker::safeEcho(" - 错误信息: " . $e->getMessage() . "\n"); - \Workerman\Worker::safeEcho(" - 错误类型: " . get_class($e) . "\n"); - \Workerman\Worker::safeEcho(" - 文件: " . $e->getFile() . ":" . $e->getLine() . "\n"); - \Workerman\Worker::safeEcho(" - 堆栈跟踪:\n"); - $trace = $e->getTraceAsString(); - $traceLines = explode("\n", $trace); - foreach (array_slice($traceLines, 0, 10) as $line) { - \Workerman\Worker::safeEcho(" " . $line . "\n"); - } - throw $e; - } - - \Workerman\Worker::safeEcho("【断点7】准备保存的数据详情:\n"); - \Workerman\Worker::safeEcho(" - relation_id: {$relation->relation_id}\n"); - \Workerman\Worker::safeEcho(" - phone_number: {$relation->phone_number}\n"); - \Workerman\Worker::safeEcho(" - phone_hash: {$relation->phone_hash}\n"); - \Workerman\Worker::safeEcho(" - user_id: {$relation->user_id}\n"); - \Workerman\Worker::safeEcho(" - effective_time: " . ($relation->effective_time ? $relation->effective_time->format('Y-m-d H:i:s') : 'null') . "\n"); - \Workerman\Worker::safeEcho(" - expire_time: " . ($relation->expire_time ? $relation->expire_time->format('Y-m-d H:i:s') : 'null') . "\n"); - \Workerman\Worker::safeEcho(" - is_active: " . ($relation->is_active ? 'true' : 'false') . "\n"); - \Workerman\Worker::safeEcho(" - type: {$relation->type}\n"); - \Workerman\Worker::safeEcho(" - is_verified: " . ($relation->is_verified ? 'true' : 'false') . "\n"); - \Workerman\Worker::safeEcho(" - source: {$relation->source}\n"); - \Workerman\Worker::safeEcho(" - create_time: " . ($relation->create_time ? $relation->create_time->format('Y-m-d H:i:s') : 'null') . "\n"); - \Workerman\Worker::safeEcho(" - update_time: " . ($relation->update_time ? $relation->update_time->format('Y-m-d H:i:s') : 'null') . "\n"); - - \Workerman\Worker::safeEcho("\n【断点8-数据库配置检查】检查数据库配置\n"); - \Workerman\Worker::safeEcho("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n"); - - try { - // 获取表名 - $tableName = $relation->getTable(); - \Workerman\Worker::safeEcho(" ✓ 目标表名: {$tableName}\n"); - - // 获取连接名 - $connectionName = $relation->getConnectionName(); - \Workerman\Worker::safeEcho(" ✓ 数据库连接名: {$connectionName}\n"); - - // 获取连接对象 - $connection = $relation->getConnection(); - \Workerman\Worker::safeEcho(" ✓ 连接对象获取成功\n"); - - // 获取数据库名 - $databaseName = $connection->getDatabaseName(); - \Workerman\Worker::safeEcho(" ✓ 数据库名: {$databaseName}\n"); - - // 获取配置信息 - $config = config('database.connections.' . $connectionName, []); - \Workerman\Worker::safeEcho("\n 数据库配置详情:\n"); - \Workerman\Worker::safeEcho(" - driver: " . ($config['driver'] ?? 'unknown') . "\n"); - \Workerman\Worker::safeEcho(" - dsn: " . ($config['dsn'] ?? 'unknown') . "\n"); - \Workerman\Worker::safeEcho(" - database: " . ($config['database'] ?? 'unknown') . "\n"); - \Workerman\Worker::safeEcho(" - username: " . (isset($config['username']) ? $config['username'] : 'null') . "\n"); - \Workerman\Worker::safeEcho(" - has_password: " . (isset($config['password']) ? 'yes' : 'no') . "\n"); - - // 尝试获取MongoDB客户端信息 - try { - $mongoClient = $connection->getMongoClient(); - if ($mongoClient) { - \Workerman\Worker::safeEcho(" - MongoDB客户端: 已获取 ✓\n"); - } - } catch (\Throwable $e) { - \Workerman\Worker::safeEcho(" - MongoDB客户端获取失败: " . $e->getMessage() . "\n"); - } - - // 测试连接 - try { - $testCollection = $connection->getCollection($tableName); - \Workerman\Worker::safeEcho(" - 集合对象获取: 成功 ✓\n"); - \Workerman\Worker::safeEcho(" - 集合名: {$tableName}\n"); - } catch (\Throwable $e) { - \Workerman\Worker::safeEcho(" - 集合对象获取失败: " . $e->getMessage() . "\n"); - } - - \Workerman\Worker::safeEcho("\n 最终写入目标:\n"); - \Workerman\Worker::safeEcho(" - 数据库: {$databaseName}\n"); - \Workerman\Worker::safeEcho(" - 集合: {$tableName}\n"); - \Workerman\Worker::safeEcho(" - 连接: {$connectionName}\n"); - \Workerman\Worker::safeEcho(" - 连接状态: 已连接 ✓\n"); - - } catch (\Throwable $e) { - \Workerman\Worker::safeEcho(" ❌ 数据库配置检查失败！\n"); - \Workerman\Worker::safeEcho(" - 错误信息: " . $e->getMessage() . "\n"); - \Workerman\Worker::safeEcho(" - 错误类型: " . get_class($e) . "\n"); - \Workerman\Worker::safeEcho(" - 文件: " . $e->getFile() . ":" . $e->getLine() . "\n"); - \Workerman\Worker::safeEcho(" - 堆栈: " . $e->getTraceAsString() . "\n"); - } - - \Workerman\Worker::safeEcho("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n"); - - \Workerman\Worker::safeEcho("\n【断点9-执行保存】开始执行 save() 操作\n"); - \Workerman\Worker::safeEcho(" - 调用: \$relation->save()\n"); - - // 执行保存 - try { - $saveResult = $relation->save(); - \Workerman\Worker::safeEcho("【断点9】save() 执行完成\n"); - \Workerman\Worker::safeEcho(" - save() 返回值: " . ($saveResult ? 'true ✓' : 'false ✗') . "\n"); - - if (!$saveResult) { - \Workerman\Worker::safeEcho(" - ❌ 警告：save() 返回 false，数据可能未保存\n"); - } - - \Workerman\Worker::safeEcho("\n【断点10-保存后验证】验证数据是否真的写入数据库\n"); - \Workerman\Worker::safeEcho(" - 查询条件: relation_id = {$relation->relation_id}\n"); - - // 验证是否真的保存成功（尝试查询） - $savedRelation = $this->phoneRelationRepository->findByRelationId($relation->relation_id); - if ($savedRelation) { - \Workerman\Worker::safeEcho(" - ✅ 验证成功：查询到保存的数据\n"); - \Workerman\Worker::safeEcho(" - 查询到的 relation_id: {$savedRelation->relation_id}\n"); - \Workerman\Worker::safeEcho(" - 查询到的 user_id: {$savedRelation->user_id}\n"); - \Workerman\Worker::safeEcho(" - 查询到的 phone_number: {$savedRelation->phone_number}\n"); - } else { - \Workerman\Worker::safeEcho(" - ❌ 验证失败：save()返回true但查询不到数据\n"); - \Workerman\Worker::safeEcho(" - 可能原因:\n"); - \Workerman\Worker::safeEcho(" 1. MongoDB写入确认问题（w=0模式）\n"); - \Workerman\Worker::safeEcho(" 2. 数据库连接问题\n"); - \Workerman\Worker::safeEcho(" 3. 事务未提交\n"); - \Workerman\Worker::safeEcho(" 4. 写入延迟\n"); - } - } catch (\Throwable $e) { - \Workerman\Worker::safeEcho("\n【断点9】❌ 保存过程中发生异常！\n"); - \Workerman\Worker::safeEcho(" - 错误信息: " . $e->getMessage() . "\n"); - \Workerman\Worker::safeEcho(" - 错误类型: " . get_class($e) . "\n"); - \Workerman\Worker::safeEcho(" - 文件: " . $e->getFile() . ":" . $e->getLine() . "\n"); - \Workerman\Worker::safeEcho(" - 堆栈跟踪:\n"); - $trace = $e->getTraceAsString(); - $traceLines = explode("\n", $trace); - foreach (array_slice($traceLines, 0, 5) as $line) { - \Workerman\Worker::safeEcho(" " . $line . "\n"); - } - throw $e; - } - - \Workerman\Worker::safeEcho("\n【断点11-日志记录】记录业务日志\n"); - - LoggerHelper::logBusiness('phone_relation_created', [ - 'relation_id' => $relation->relation_id, - 'user_id' => $userId, - 'phone_number' => $phoneNumber, - 'type' => $relation->type, - 'effective_time' => $effectiveTime->format('Y-m-d H:i:s'), - ]); - \Workerman\Worker::safeEcho("【断点11】✓ 业务日志已记录\n"); - - \Workerman\Worker::safeEcho("\n【断点12-方法返回】准备返回结果\n"); - \Workerman\Worker::safeEcho(" - 返回 relation_id: {$relation->relation_id}\n"); - \Workerman\Worker::safeEcho("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n"); - \Workerman\Worker::safeEcho("[UserPhoneService::addPhoneToUser] ✅ 方法执行完成\n"); - \Workerman\Worker::safeEcho("━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\n\n"); - - return $relation->relation_id; - } - - /** - * 移除用户的手机号（失效关联） - * - * @param string $userId 用户ID - * @param string $phoneNumber 手机号 - * @param \DateTimeInterface|null $expireTime 过期时间（默认当前时间） - * @return bool 是否成功 - */ - public function removePhoneFromUser(string $userId, string $phoneNumber, ?\DateTimeInterface $expireTime = null): bool - { - // 过滤非数字字符 - $phoneNumber = $this->filterPhoneNumber(trim($phoneNumber)); - if (empty($phoneNumber)) { - return false; - } - - $phoneHash = EncryptionHelper::hash($phoneNumber); - $expireTime = $expireTime ?? new \DateTimeImmutable('now'); - - $relations = $this->phoneRelationRepository->newQuery() - ->where('phone_hash', $phoneHash) - ->where('user_id', $userId) - ->where('is_active', true) - ->where(function($q) use ($expireTime) { - $q->whereNull('expire_time') - ->orWhere('expire_time', '>', $expireTime); - }) - ->get(); - - if ($relations->isEmpty()) { - return false; - } - - foreach ($relations as $relation) { - $relation->expire_time = $expireTime; - $relation->is_active = false; - $relation->update_time = new \DateTimeImmutable('now'); - $relation->save(); - } - - LoggerHelper::logBusiness('phone_relation_removed', [ - 'user_id' => $userId, - 'phone_number' => $phoneNumber, - 'expire_time' => $expireTime->format('Y-m-d H:i:s'), - ]); - - return true; - } - - /** - * 根据手机号查找当前用户 - * - * @param string $phoneNumber 手机号 - * @param \DateTimeInterface|null $atTime 查询时间点（默认为当前时间） - * @return string|null 用户ID - */ - public function findUserByPhone(string $phoneNumber, ?\DateTimeInterface $atTime = null): ?string - { - // 过滤非数字字符 - $phoneNumber = $this->filterPhoneNumber(trim($phoneNumber)); - if (empty($phoneNumber)) { - return null; - } - - $phoneHash = EncryptionHelper::hash($phoneNumber); - $relation = $this->phoneRelationRepository->findActiveByPhoneHash($phoneHash, $atTime); - - return $relation ? $relation->user_id : null; - } - - /** - * 获取用户的所有手机号 - * - * @param string $userId 用户ID - * @param bool $includeHistory 是否包含历史记录 - * @return array> 手机号列表 - */ - public function getUserPhones(string $userId, bool $includeHistory = false): array - { - $relations = $this->phoneRelationRepository->findByUserId($userId, $includeHistory); - - return array_map(function($relation) { - return [ - 'phone_number' => $relation->phone_number, - 'type' => $relation->type, - 'is_verified' => $relation->is_verified, - 'effective_time' => $relation->effective_time, - 'expire_time' => $relation->expire_time, - 'is_active' => $relation->is_active, - 'source' => $relation->source, - ]; - }, $relations); - } - - /** - * 获取用户的所有手机号号码（仅号码列表） - * - * @param string $userId 用户ID - * @param bool $includeHistory 是否包含历史记录 - * @return array 手机号列表 - */ - public function getUserPhoneNumbers(string $userId, bool $includeHistory = false): array - { - $relations = $this->phoneRelationRepository->findByUserId($userId, $includeHistory); - - return array_map(function($relation) { - return $relation->phone_number; - }, $relations); - } - - /** - * 获取手机号的历史关联记录 - * - * @param string $phoneNumber 手机号 - * @return array> 历史关联记录 - */ - public function getPhoneHistory(string $phoneNumber): array - { - // 过滤非数字字符 - $phoneNumber = $this->filterPhoneNumber(trim($phoneNumber)); - if (empty($phoneNumber)) { - return []; - } - - $phoneHash = EncryptionHelper::hash($phoneNumber); - $relations = $this->phoneRelationRepository->findHistoryByPhoneHash($phoneHash); - - return array_map(function($relation) { - return [ - 'relation_id' => $relation->relation_id, - 'user_id' => $relation->user_id, - 'effective_time' => $relation->effective_time, - 'expire_time' => $relation->expire_time, - 'is_active' => $relation->is_active, - 'type' => $relation->type, - 'is_verified' => $relation->is_verified, - 'source' => $relation->source, - ]; - }, $relations); - } - - /** - * 检查手机号是否已被使用（当前有效） - * - * @param string $phoneNumber 手机号 - * @return bool - */ - public function isPhoneInUse(string $phoneNumber): bool - { - // 过滤非数字字符 - $phoneNumber = $this->filterPhoneNumber(trim($phoneNumber)); - if (empty($phoneNumber)) { - return false; - } - - $phoneHash = EncryptionHelper::hash($phoneNumber); - $relation = $this->phoneRelationRepository->findActiveByPhoneHash($phoneHash); - - return $relation !== null; - } - - /** - * 过滤手机号中的非数字字符 - * - * @param string $phoneNumber 原始手机号 - * @return string 过滤后的手机号（只包含数字） - */ - protected function filterPhoneNumber(string $phoneNumber): string - { - // 移除所有非数字字符 - return preg_replace('/\D/', '', $phoneNumber); - } - - /** - * 验证手机号格式（内部使用，假设已经过滤过非数字字符） - * - * @param string $phoneNumber 已过滤的手机号（只包含数字） - * @return bool - */ - protected function validatePhoneNumber(string $phoneNumber): bool - { - // 中国大陆手机号：11位数字，以1开头 - return preg_match('/^1[3-9]\d{9}$/', $phoneNumber) === 1; - } -} - diff --git a/Moncter/Moncter/app/service/UserService.php b/Moncter/Moncter/app/service/UserService.php deleted file mode 100644 index a02c4812..00000000 --- a/Moncter/Moncter/app/service/UserService.php +++ /dev/null @@ -1,395 +0,0 @@ - $data 用户数据 - * @return array 创建的用户信息 - * @throws \InvalidArgumentException - */ - public function createUser(array $data): array - { - // 验证必填字段 - if (empty($data['id_card'])) { - throw new \InvalidArgumentException('身份证号不能为空'); - } - - $idCard = trim($data['id_card']); - $idCardType = $data['id_card_type'] ?? '身份证'; - - // 验证身份证格式（简单验证） - if ($idCardType === '身份证' && !$this->validateIdCard($idCard)) { - throw new \InvalidArgumentException('身份证号格式不正确'); - } - - // 检查是否已存在（通过身份证哈希） - $idCardHash = EncryptionHelper::hash($idCard); - $existingUser = $this->userProfileRepository->newQuery() - ->where('id_card_hash', $idCardHash) - ->first(); - - if ($existingUser) { - throw new \InvalidArgumentException('该身份证号已存在，user_id: ' . $existingUser->user_id); - } - - // 加密身份证 - $idCardEncrypted = EncryptionHelper::encrypt($idCard); - - // 生成用户ID - $userId = $data['user_id'] ?? UuidGenerator::uuid4()->toString(); - - $now = new \DateTimeImmutable('now'); - - // 从身份证号中自动提取基础信息（如果未提供） - $idCardInfo = IdCardHelper::extractInfo($idCard); - $gender = isset($data['gender']) ? (int)$data['gender'] : ($idCardInfo['gender'] > 0 ? $idCardInfo['gender'] : null); - $birthday = isset($data['birthday']) ? new \DateTimeImmutable($data['birthday']) : $idCardInfo['birthday']; - - // 创建用户记录 - $user = new UserProfileRepository(); - $user->user_id = $userId; - $user->id_card_hash = $idCardHash; - $user->id_card_encrypted = $idCardEncrypted; - $user->id_card_type = $idCardType; - $user->name = $data['name'] ?? null; - $user->phone = $data['phone'] ?? null; - $user->address = $data['address'] ?? null; - $user->email = $data['email'] ?? null; - $user->gender = $gender; - $user->birthday = $birthday; - $user->total_amount = isset($data['total_amount']) ? (float)$data['total_amount'] : 0; - $user->total_count = isset($data['total_count']) ? (int)$data['total_count'] : 0; - $user->last_consume_time = isset($data['last_consume_time']) ? new \DateTimeImmutable($data['last_consume_time']) : null; - $user->status = isset($data['status']) ? (int)$data['status'] : 0; - $user->create_time = $now; - $user->update_time = $now; - - $user->save(); - - LoggerHelper::logBusiness('user_created', [ - 'user_id' => $userId, - 'name' => $user->name, - 'id_card_type' => $idCardType, - ]); - - return [ - 'user_id' => $userId, - 'name' => $user->name, - 'phone' => $user->phone, - 'id_card_type' => $idCardType, - 'create_time' => $user->create_time, - ]; - } - - /** - * 根据 user_id 获取用户信息 - * - * @param string $userId 用户ID - * @param bool $decryptIdCard 是否解密身份证（需要权限控制） - * @return array|null 用户信息 - */ - public function getUserById(string $userId, bool $decryptIdCard = false): ?array - { - $user = $this->userProfileRepository->findByUserId($userId); - - if (!$user) { - return null; - } - - $result = [ - 'user_id' => $user->user_id, - 'name' => $user->name, - 'phone' => $user->phone, - 'address' => $user->address, - 'email' => $user->email, - 'gender' => $user->gender, - 'birthday' => $user->birthday, - 'id_card_type' => $user->id_card_type, - 'total_amount' => $user->total_amount, - 'total_count' => $user->total_count, - 'last_consume_time' => $user->last_consume_time, - 'tags_update_time' => $user->tags_update_time, - 'status' => $user->status, - 'create_time' => $user->create_time, - 'update_time' => $user->update_time, - ]; - - // 如果需要解密身份证（需要权限控制） - if ($decryptIdCard) { - try { - $result['id_card'] = EncryptionHelper::decrypt($user->id_card_encrypted); - } catch (\Throwable $e) { - LoggerHelper::logError($e, ['user_id' => $userId, 'action' => 'decrypt_id_card']); - $result['id_card'] = null; - $result['decrypt_error'] = '解密失败'; - } - } else { - // 返回脱敏的身份证 - $result['id_card_encrypted'] = $user->id_card_encrypted; - } - - return $result; - } - - /** - * 根据身份证号查找用户（通过哈希匹配） - * - * @param string $idCard 身份证号 - * @return array|null 用户信息 - */ - public function findUserByIdCard(string $idCard): ?array - { - $idCardHash = EncryptionHelper::hash($idCard); - $user = $this->userProfileRepository->newQuery() - ->where('id_card_hash', $idCardHash) - ->first(); - - if (!$user) { - return null; - } - - return $this->getUserById($user->user_id, false); - } - - /** - * 更新用户信息 - * - * @param string $userId 用户ID - * @param array $data 要更新的用户数据 - * @return array 更新后的用户信息 - * @throws \InvalidArgumentException - */ - public function updateUser(string $userId, array $data): array - { - $user = $this->userProfileRepository->findByUserId($userId); - - if (!$user) { - throw new \InvalidArgumentException("用户不存在: {$userId}"); - } - - $now = new \DateTimeImmutable('now'); - - // 更新允许修改的字段 - if (isset($data['name'])) { - $user->name = $data['name']; - } - if (isset($data['phone'])) { - $user->phone = $data['phone']; - } - if (isset($data['email'])) { - $user->email = $data['email']; - } - if (isset($data['address'])) { - $user->address = $data['address']; - } - if (isset($data['gender'])) { - $user->gender = (int)$data['gender']; - } - if (isset($data['birthday'])) { - $user->birthday = new \DateTimeImmutable($data['birthday']); - } - if (isset($data['status'])) { - $user->status = (int)$data['status']; - } - - $user->update_time = $now; - $user->save(); - - LoggerHelper::logBusiness('user_updated', [ - 'user_id' => $userId, - 'updated_fields' => array_keys($data), - ]); - - return $this->getUserById($userId, false); - } - - /** - * 删除用户（软删除，设置状态为禁用） - * - * @param string $userId 用户ID - * @return bool 是否删除成功 - * @throws \InvalidArgumentException - */ - public function deleteUser(string $userId): bool - { - $user = $this->userProfileRepository->findByUserId($userId); - - if (!$user) { - throw new \InvalidArgumentException("用户不存在: {$userId}"); - } - - // 软删除：设置状态为禁用 - $user->status = 1; // 1 表示禁用 - $user->update_time = new \DateTimeImmutable('now'); - $user->save(); - - LoggerHelper::logBusiness('user_deleted', [ - 'user_id' => $userId, - ]); - - return true; - } - - /** - * 搜索用户（支持多种条件组合） - * - * @param array $conditions 搜索条件 - * - name: 姓名（模糊搜索） - * - phone: 手机号（精确或模糊） - * - email: 邮箱（精确或模糊） - * - id_card: 身份证号（精确匹配） - * - gender: 性别（0-未知，1-男，2-女） - * - status: 状态（0-正常，1-禁用） - * - min_total_amount: 最小总消费金额 - * - max_total_amount: 最大总消费金额 - * - min_total_count: 最小消费次数 - * - max_total_count: 最大消费次数 - * @param int $page 页码（从1开始） - * @param int $pageSize 每页数量 - * @return array 返回用户列表和分页信息 - */ - public function searchUsers(array $conditions, int $page = 1, int $pageSize = 20): array - { - $query = $this->userProfileRepository->newQuery(); - - // 姓名模糊搜索（MongoDB 使用正则表达式） - if (!empty($conditions['name'])) { - $namePattern = preg_quote($conditions['name'], '/'); - $query->where('name', 'regex', "/{$namePattern}/i"); - } - - // 手机号搜索（支持精确和模糊） - if (!empty($conditions['phone'])) { - if (isset($conditions['phone_exact']) && $conditions['phone_exact']) { - // 精确匹配 - $query->where('phone', $conditions['phone']); - } else { - // 模糊匹配（MongoDB 使用正则表达式） - $phonePattern = preg_quote($conditions['phone'], '/'); - $query->where('phone', 'regex', "/{$phonePattern}/i"); - } - } - - // 邮箱搜索（支持精确和模糊） - if (!empty($conditions['email'])) { - if (isset($conditions['email_exact']) && $conditions['email_exact']) { - // 精确匹配 - $query->where('email', $conditions['email']); - } else { - // 模糊匹配（MongoDB 使用正则表达式） - $emailPattern = preg_quote($conditions['email'], '/'); - $query->where('email', 'regex', "/{$emailPattern}/i"); - } - } - - // 如果指定了 user_ids，限制搜索范围 - if (!empty($conditions['user_ids']) && is_array($conditions['user_ids'])) { - $query->whereIn('user_id', $conditions['user_ids']); - } - - // 身份证号精确匹配（通过哈希） - if (!empty($conditions['id_card'])) { - $idCardHash = EncryptionHelper::hash($conditions['id_card']); - $query->where('id_card_hash', $idCardHash); - } - - // 性别筛选 - if (isset($conditions['gender']) && $conditions['gender'] !== '') { - $query->where('gender', (int)$conditions['gender']); - } - - // 状态筛选 - if (isset($conditions['status']) && $conditions['status'] !== '') { - $query->where('status', (int)$conditions['status']); - } - - // 总消费金额范围 - if (isset($conditions['min_total_amount'])) { - $query->where('total_amount', '>=', (float)$conditions['min_total_amount']); - } - if (isset($conditions['max_total_amount'])) { - $query->where('total_amount', '<=', (float)$conditions['max_total_amount']); - } - - // 消费次数范围 - if (isset($conditions['min_total_count'])) { - $query->where('total_count', '>=', (int)$conditions['min_total_count']); - } - if (isset($conditions['max_total_count'])) { - $query->where('total_count', '<=', (int)$conditions['max_total_count']); - } - - // 分页 - $total = $query->count(); - $users = $query->skip(($page - 1) * $pageSize) - ->take($pageSize) - ->orderBy('create_time', 'desc') - ->get(); - - // 转换为数组格式 - $result = []; - foreach ($users as $user) { - $result[] = [ - 'user_id' => $user->user_id, - 'name' => $user->name, - 'phone' => $user->phone, - 'email' => $user->email, - 'address' => $user->address, - 'gender' => $user->gender, - 'birthday' => $user->birthday, - 'id_card_type' => $user->id_card_type, - 'total_amount' => $user->total_amount, - 'total_count' => $user->total_count, - 'last_consume_time' => $user->last_consume_time, - 'tags_update_time' => $user->tags_update_time, - 'status' => $user->status, - 'create_time' => $user->create_time, - 'update_time' => $user->update_time, - ]; - } - - return [ - 'users' => $result, - 'total' => $total, - 'page' => $page, - 'page_size' => $pageSize, - 'total_pages' => (int)ceil($total / $pageSize), - ]; - } - - /** - * 验证身份证号格式（简单验证） - * - * @param string $idCard 身份证号 - * @return bool - */ - protected function validateIdCard(string $idCard): bool - { - // 15位或18位数字，最后一位可能是X - return preg_match('/^(\d{15}|\d{17}[\dXx])$/', $idCard) === 1; - } -} - diff --git a/Moncter/Moncter/app/utils/ApiResponseHelper.php b/Moncter/Moncter/app/utils/ApiResponseHelper.php deleted file mode 100644 index e1433d04..00000000 --- a/Moncter/Moncter/app/utils/ApiResponseHelper.php +++ /dev/null @@ -1,137 +0,0 @@ - 0, - 'msg' => $message, - ]; - - if ($data !== null) { - $response['data'] = $data; - } - - return json($response, $httpCode); - } - - /** - * 错误响应 - * - * @param string $message 错误消息 - * @param int $code 错误码（业务错误码，非HTTP状态码） - * @param int $httpCode HTTP状态码 - * @param array $extra 额外信息 - * @return \support\Response - */ - public static function error( - string $message, - int $code = 400, - int $httpCode = 400, - array $extra = [] - ): \support\Response { - $response = [ - 'code' => $code, - 'msg' => $message, - ]; - - // 开发环境可以返回更多调试信息 - if (self::isDevelopment() && !empty($extra)) { - $response = array_merge($response, $extra); - } - - return json($response, $httpCode); - } - - /** - * 异常响应 - * - * @param \Throwable $exception 异常对象 - * @param int $httpCode HTTP状态码 - * @return \support\Response - */ - public static function exception(\Throwable $exception, int $httpCode = 500): \support\Response - { - // 记录错误日志 - LoggerHelper::logError($exception); - - $code = 500; - $message = '内部服务器错误'; - - // 根据异常类型设置错误码和消息 - if ($exception instanceof \InvalidArgumentException) { - $code = 400; - $message = $exception->getMessage(); - } elseif ($exception instanceof \RuntimeException) { - $code = 500; - $message = $exception->getMessage(); - } - - $response = [ - 'code' => $code, - 'msg' => $message, - ]; - - // 开发环境返回详细错误信息 - if (self::isDevelopment()) { - $response['debug'] = [ - 'message' => $exception->getMessage(), - 'file' => $exception->getFile(), - 'line' => $exception->getLine(), - 'trace' => $exception->getTraceAsString(), - ]; - } - - return json($response, $httpCode); - } - - /** - * 验证错误响应 - * - * @param array $errors 验证错误列表 - * @return \support\Response - */ - public static function validationError(array $errors): \support\Response - { - $message = '参数验证失败'; - if (!empty($errors)) { - $firstError = reset($errors); - $message = is_array($firstError) ? $firstError[0] : $firstError; - } - - $response = [ - 'code' => 400, - 'msg' => $message, - 'errors' => $errors, - ]; - - return json($response, 400); - } -} - diff --git a/Moncter/Moncter/app/utils/DataMaskingHelper.php b/Moncter/Moncter/app/utils/DataMaskingHelper.php deleted file mode 100644 index 7d5d6400..00000000 --- a/Moncter/Moncter/app/utils/DataMaskingHelper.php +++ /dev/null @@ -1,143 +0,0 @@ - $data 数据数组 - * @param array $sensitiveFields 敏感字段列表（如：['id_card', 'phone', 'email']） - * @return array 脱敏后的数组 - */ - public static function maskArray(array $data, array $sensitiveFields = ['id_card', 'id_card_encrypted', 'phone', 'email']): array - { - $masked = $data; - - foreach ($sensitiveFields as $field) { - if (isset($masked[$field]) && is_string($masked[$field])) { - switch ($field) { - case 'id_card': - case 'id_card_encrypted': - $masked[$field] = self::maskIdCard($masked[$field]); - break; - case 'phone': - $masked[$field] = self::maskPhone($masked[$field]); - break; - case 'email': - $masked[$field] = self::maskEmail($masked[$field]); - break; - } - } - } - - return $masked; - } -} - diff --git a/Moncter/Moncter/app/utils/EncryptionHelper.php b/Moncter/Moncter/app/utils/EncryptionHelper.php deleted file mode 100644 index 40cc7d91..00000000 --- a/Moncter/Moncter/app/utils/EncryptionHelper.php +++ /dev/null @@ -1,141 +0,0 @@ - $currentYearLastTwo) { - $year += 1900; - } else { - $year += 2000; - } - } else { - return null; - } - - // 验证日期是否有效 - if (!checkdate($month, $day, $year)) { - return null; - } - - try { - return new \DateTimeImmutable(sprintf('%04d-%02d-%02d', $year, $month, $day)); - } catch (\Throwable $e) { - return null; - } - } - - /** - * 从身份证号中提取性别 - * - * @param string $idCard 身份证号（15位或18位） - * @return int 性别：1=男，2=女，0=未知 - */ - public static function extractGender(string $idCard): int - { - $idCard = trim($idCard); - $length = strlen($idCard); - - if ($length === 18) { - // 18位身份证：第17位（索引16）是性别码 - $genderCode = (int)substr($idCard, 16, 1); - } elseif ($length === 15) { - // 15位身份证：第15位（索引14）是性别码 - $genderCode = (int)substr($idCard, 14, 1); - } else { - return 0; // 未知 - } - - // 奇数表示男性，偶数表示女性 - return ($genderCode % 2 === 1) ? 1 : 2; - } - - /** - * 从身份证号中提取所有可解析的信息 - * - * @param string $idCard 身份证号 - * @return array 包含 birthday 和 gender 的数组 - */ - public static function extractInfo(string $idCard): array - { - return [ - 'birthday' => self::extractBirthday($idCard), - 'gender' => self::extractGender($idCard), - ]; - } -} - diff --git a/Moncter/Moncter/app/utils/LogMaskingProcessor.php b/Moncter/Moncter/app/utils/LogMaskingProcessor.php deleted file mode 100644 index 355125fa..00000000 --- a/Moncter/Moncter/app/utils/LogMaskingProcessor.php +++ /dev/null @@ -1,137 +0,0 @@ - - */ - protected array $sensitiveFields = [ - 'id_card', - 'id_card_encrypted', - 'id_card_hash', - 'phone', - 'email', - 'password', - 'token', - 'secret', - ]; - - /** - * 处理日志记录，对敏感信息进行脱敏 - * - * @param array $record Monolog 2.x 格式的日志记录数组 - * @return array 处理后的日志记录数组 - */ - public function __invoke(array $record): array - { - // 处理 context 中的敏感信息 - if (isset($record['context']) && is_array($record['context'])) { - $record['context'] = $this->maskArray($record['context']); - } - - // 处理 extra 中的敏感信息 - if (isset($record['extra']) && is_array($record['extra'])) { - $record['extra'] = $this->maskArray($record['extra']); - } - - // 对消息本身也进行脱敏（如果包含敏感信息） - if (isset($record['message']) && is_string($record['message'])) { - $record['message'] = $this->maskString($record['message']); - } - - return $record; - } - - /** - * 脱敏数组中的敏感字段 - * - * @param array $data - * @return array - */ - protected function maskArray(array $data): array - { - $masked = []; - foreach ($data as $key => $value) { - $lowerKey = strtolower($key); - - // 检查字段名是否包含敏感关键词 - $isSensitive = false; - foreach ($this->sensitiveFields as $field) { - if (strpos($lowerKey, $field) !== false) { - $isSensitive = true; - break; - } - } - - if ($isSensitive && is_string($value)) { - // 根据字段类型选择脱敏方法 - if (strpos($lowerKey, 'phone') !== false) { - $masked[$key] = DataMaskingHelper::maskPhone($value); - } elseif (strpos($lowerKey, 'email') !== false) { - $masked[$key] = DataMaskingHelper::maskEmail($value); - } elseif (strpos($lowerKey, 'id_card') !== false) { - $masked[$key] = DataMaskingHelper::maskIdCard($value); - } else { - // 其他敏感字段，用*替代 - $masked[$key] = str_repeat('*', min(strlen($value), 20)); - } - } elseif (is_array($value)) { - $masked[$key] = $this->maskArray($value); - } else { - $masked[$key] = $value; - } - } - return $masked; - } - - /** - * 脱敏字符串中的敏感信息（简单模式，匹配常见格式） - * - * @param string $message - * @return string - */ - protected function maskString(string $message): string - { - // 匹配身份证号（18位或15位数字） - $message = preg_replace_callback( - '/\b\d{15}(\d{3})?[Xx]?\b/', - function ($matches) { - return DataMaskingHelper::maskIdCard($matches[0]); - }, - $message - ); - - // 匹配手机号（11位数字，1开头） - $message = preg_replace_callback( - '/\b1[3-9]\d{9}\b/', - function ($matches) { - return DataMaskingHelper::maskPhone($matches[0]); - }, - $message - ); - - // 匹配邮箱 - $message = preg_replace_callback( - '/\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b/', - function ($matches) { - return DataMaskingHelper::maskEmail($matches[0]); - }, - $message - ); - - return $message; - } -} - diff --git a/Moncter/Moncter/app/utils/LoggerHelper.php b/Moncter/Moncter/app/utils/LoggerHelper.php deleted file mode 100644 index 113c82d7..00000000 --- a/Moncter/Moncter/app/utils/LoggerHelper.php +++ /dev/null @@ -1,155 +0,0 @@ - $params 请求参数 - * @param float|null $duration 请求耗时（秒） - */ - public static function logRequest(string $method, string $path, array $params = [], ?float $duration = null): void - { - $logger = \support\Log::channel('default'); - $context = [ - 'type' => 'request', - 'method' => $method, - 'path' => $path, - 'params' => $params, - ]; - - if ($duration !== null) { - $context['duration'] = round($duration * 1000, 2) . 'ms'; - } - - $logger->info("请求: {$method} {$path}", $context); - } - - /** - * 记录业务日志 - * - * @param string $action 操作名称 - * @param array $context 上下文信息 - * @param string $level 日志级别（info/warning/error） - */ - public static function logBusiness(string $action, array $context = [], string $level = 'info'): void - { - $logger = \support\Log::channel('default'); - $context['type'] = 'business'; - $context['action'] = $action; - - $logger->$level("业务操作: {$action}", $context); - } - - /** - * 记录标签计算日志 - * - * @param string $userId 用户ID - * @param string $tagId 标签ID - * @param array $result 计算结果 - * @param float|null $duration 计算耗时（秒） - */ - public static function logTagCalculation(string $userId, string $tagId, array $result, ?float $duration = null): void - { - $logger = \support\Log::channel('default'); - $context = [ - 'type' => 'tag_calculation', - 'user_id' => $userId, - 'tag_id' => $tagId, - 'result' => $result, - ]; - - if ($duration !== null) { - $context['duration'] = round($duration * 1000, 2) . 'ms'; - } - - $logger->info("标签计算: user_id={$userId}, tag_id={$tagId}", $context); - } - - /** - * 记录错误日志 - * - * @param \Throwable $exception 异常对象 - * @param array $context 额外上下文 - */ - public static function logError(\Throwable $exception, array $context = []): void - { - $logger = \support\Log::channel('default'); - $context['type'] = 'error'; - - // 限制 trace 长度，避免内存溢出 - $trace = $exception->getTraceAsString(); - $originalTraceLength = strlen($trace); - $maxTraceLength = 5000; // 最大 trace 长度（字符数） - - // 限制 trace 行数，只保留前50行 - $traceLines = explode("\n", $trace); - $originalLineCount = count($traceLines); - - if ($originalLineCount > 50) { - $traceLines = array_slice($traceLines, 0, 50); - $trace = implode("\n", $traceLines) . "\n... (trace truncated, total lines: {$originalLineCount})"; - } - - // 限制 trace 字符长度 - if (strlen($trace) > $maxTraceLength) { - $trace = substr($trace, 0, $maxTraceLength) . "\n... (trace truncated, total length: {$originalTraceLength} bytes)"; - } - - $context['exception'] = [ - 'message' => $exception->getMessage(), - 'file' => $exception->getFile(), - 'line' => $exception->getLine(), - 'trace' => $trace, - 'class' => get_class($exception), - ]; - - // 如果上下文数据太大，也进行限制 - $contextJson = json_encode($context); - if (strlen($contextJson) > 10000) { - // 如果上下文太大，只保留关键信息 - $context = [ - 'type' => 'error', - 'exception' => [ - 'message' => $exception->getMessage(), - 'file' => $exception->getFile(), - 'line' => $exception->getLine(), - 'class' => get_class($exception), - 'trace' => substr($trace, 0, 2000) . '... (truncated)', - ], - ]; - } - - $logger->error("异常: {$exception->getMessage()}", $context); - } - - /** - * 记录性能日志 - * - * @param string $operation 操作名称 - * @param float $duration 耗时（秒） - * @param array $context 上下文信息 - */ - public static function logPerformance(string $operation, float $duration, array $context = []): void - { - $logger = \support\Log::channel('default'); - $context['type'] = 'performance'; - $context['operation'] = $operation; - $context['duration'] = round($duration * 1000, 2) . 'ms'; - - $level = $duration > 1.0 ? 'warning' : 'info'; - $logger->$level("性能: {$operation} 耗时 {$context['duration']}", $context); - } -} - diff --git a/Moncter/Moncter/app/utils/MongoDBHelper.php b/Moncter/Moncter/app/utils/MongoDBHelper.php deleted file mode 100644 index 66f63200..00000000 --- a/Moncter/Moncter/app/utils/MongoDBHelper.php +++ /dev/null @@ -1,55 +0,0 @@ - $config 数据库配置 - * @return string DSN 字符串 - */ - public static function buildDsn(array $config): string - { - $host = $config['host'] ?? '192.168.1.106'; - $port = $config['port'] ?? 27017; - $username = $config['username'] ?? ''; - $password = $config['password'] ?? ''; - $authSource = $config['auth_source'] ?? 'admin'; - - if (!empty($username) && !empty($password)) { - return "mongodb://{$username}:{$password}@{$host}:{$port}/{$authSource}"; - } - - return "mongodb://{$host}:{$port}"; - } - - /** - * 创建 MongoDB 客户端 - * - * @param array $config 数据库配置 - * @param array $options 额外选项（可选） - * @return Client MongoDB 客户端实例 - */ - public static function createClient(array $config, array $options = []): Client - { - $defaultOptions = [ - 'connectTimeoutMS' => 5000, - 'socketTimeoutMS' => 5000, - ]; - - return new Client( - self::buildDsn($config), - array_merge($defaultOptions, $options) - ); - } -} - diff --git a/Moncter/Moncter/app/utils/QueueService.php b/Moncter/Moncter/app/utils/QueueService.php deleted file mode 100644 index 59451dff..00000000 --- a/Moncter/Moncter/app/utils/QueueService.php +++ /dev/null @@ -1,247 +0,0 @@ -isConnected()) { - return; - } - - $config = config('queue.connections.rabbitmq'); - self::$config = $config; - - try { - self::$connection = new AMQPStreamConnection( - $config['host'], - $config['port'], - $config['user'], - $config['password'], - $config['vhost'], - false, // insist - 'AMQPLAIN', // login_method - null, // login_response - 'en_US', // locale - $config['timeout'] ?? 10.0, // connection_timeout - $config['timeout'] ?? 10.0, // read_write_timeout - null, // context - false, // keepalive - $config['heartbeat'] ?? 0 // heartbeat - ); - - self::$channel = self::$connection->channel(); - - // 声明数据同步交换机 - if (isset($config['exchanges']['data_sync'])) { - $exchangeConfig = $config['exchanges']['data_sync']; - self::$channel->exchange_declare( - $exchangeConfig['name'], - $exchangeConfig['type'], - false, // passive - $exchangeConfig['durable'], - $exchangeConfig['auto_delete'] - ); - } - - // 声明标签计算交换机 - if (isset($config['exchanges']['tag_calculation'])) { - $exchangeConfig = $config['exchanges']['tag_calculation']; - self::$channel->exchange_declare( - $exchangeConfig['name'], - $exchangeConfig['type'], - false, // passive - $exchangeConfig['durable'], - $exchangeConfig['auto_delete'] - ); - } - - // 声明队列 - if (isset($config['queues']['tag_calculation'])) { - $queueConfig = $config['queues']['tag_calculation']; - self::$channel->queue_declare( - $queueConfig['name'], - false, // passive - $queueConfig['durable'], - false, // exclusive - $queueConfig['auto_delete'], - false, // nowait - $queueConfig['arguments'] ?? [] - ); - - // 绑定队列到交换机 - if (isset($config['routing_keys']['tag_calculation'])) { - self::$channel->queue_bind( - $queueConfig['name'], - $config['exchanges']['tag_calculation']['name'], - $config['routing_keys']['tag_calculation'] - ); - } - } - - LoggerHelper::logBusiness('queue_connection_established', [ - 'host' => $config['host'], - 'port' => $config['port'], - ]); - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'QueueService', - 'action' => 'initConnection', - ]); - throw $e; - } - } - - /** - * 推送消息到数据同步队列 - * - * @param array $data 消息数据（包含数据源ID、数据记录等） - * @return bool 是否推送成功 - */ - public static function pushDataSync(array $data): bool - { - try { - self::initConnection(); - - $config = self::$config; - $messageConfig = config('queue.message', []); - - $messageBody = json_encode($data, JSON_UNESCAPED_UNICODE); - $message = new AMQPMessage( - $messageBody, - [ - 'delivery_mode' => $messageConfig['delivery_mode'] ?? AMQPMessage::DELIVERY_MODE_PERSISTENT, - 'content_type' => $messageConfig['content_type'] ?? 'application/json', - ] - ); - - $exchangeName = $config['exchanges']['data_sync']['name']; - $routingKey = $config['routing_keys']['data_sync']; - - self::$channel->basic_publish($message, $exchangeName, $routingKey); - - LoggerHelper::logBusiness('queue_message_pushed', [ - 'queue' => 'data_sync', - 'data' => $data, - ]); - - return true; - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'QueueService', - 'action' => 'pushDataSync', - 'data' => $data, - ]); - return false; - } - } - - /** - * 推送消息到标签计算队列 - * - * @param array $data 消息数据 - * @return bool 是否推送成功 - */ - public static function pushTagCalculation(array $data): bool - { - try { - self::initConnection(); - - $config = self::$config; - $messageConfig = config('queue.message', []); - - $messageBody = json_encode($data, JSON_UNESCAPED_UNICODE); - $message = new AMQPMessage( - $messageBody, - [ - 'delivery_mode' => $messageConfig['delivery_mode'] ?? AMQPMessage::DELIVERY_MODE_PERSISTENT, - 'content_type' => $messageConfig['content_type'] ?? 'application/json', - ] - ); - - $exchangeName = $config['exchanges']['tag_calculation']['name']; - $routingKey = $config['routing_keys']['tag_calculation']; - - self::$channel->basic_publish($message, $exchangeName, $routingKey); - - LoggerHelper::logBusiness('queue_message_pushed', [ - 'queue' => 'tag_calculation', - 'data' => $data, - ]); - - return true; - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'QueueService', - 'action' => 'pushTagCalculation', - 'data' => $data, - ]); - return false; - } - } - - /** - * 关闭连接 - */ - public static function closeConnection(): void - { - try { - if (self::$channel !== null) { - self::$channel->close(); - self::$channel = null; - } - if (self::$connection !== null && self::$connection->isConnected()) { - self::$connection->close(); - self::$connection = null; - } - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'QueueService', - 'action' => 'closeConnection', - ]); - } - } - - /** - * 获取通道（用于消费者） - * - * @return AMQPChannel - */ - public static function getChannel(): AMQPChannel - { - self::initConnection(); - return self::$channel; - } - - /** - * 获取连接（用于消费者） - * - * @return AMQPStreamConnection - */ - public static function getConnection(): AMQPStreamConnection - { - self::initConnection(); - return self::$connection; - } -} - diff --git a/Moncter/Moncter/app/utils/RedisHelper.php b/Moncter/Moncter/app/utils/RedisHelper.php deleted file mode 100644 index 78b621db..00000000 --- a/Moncter/Moncter/app/utils/RedisHelper.php +++ /dev/null @@ -1,267 +0,0 @@ - $sessionConfig['host'] ?? getenv('REDIS_HOST') ?: '127.0.0.1', - 'port' => (int)($sessionConfig['port'] ?? getenv('REDIS_PORT') ?: 6379), - 'password' => $sessionConfig['auth'] ?? getenv('REDIS_PASSWORD') ?: null, - 'database' => (int)($sessionConfig['database'] ?? getenv('REDIS_DATABASE') ?: 0), - 'timeout' => $sessionConfig['timeout'] ?? 2.0, - ]; - - $parameters = [ - 'host' => self::$config['host'], - 'port' => self::$config['port'], - ]; - - if (!empty(self::$config['password'])) { - $parameters['password'] = self::$config['password']; - } - - if (self::$config['database'] > 0) { - $parameters['database'] = self::$config['database']; - } - - $options = [ - 'timeout' => self::$config['timeout'], - ]; - - self::$client = new Client($parameters, $options); - - // 测试连接 - try { - self::$client->ping(); - LoggerHelper::logBusiness('redis_connected', [ - 'host' => self::$config['host'], - 'port' => self::$config['port'], - ]); - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'RedisHelper', - 'action' => 'getClient', - ]); - throw $e; - } - - return self::$client; - } - - /** - * 获取分布式锁 - * - * @param string $key 锁的键 - * @param int $ttl 锁的过期时间（秒） - * @param int $retryTimes 重试次数 - * @param int $retryDelay 重试延迟（毫秒） - * @return bool 是否获取成功 - */ - public static function acquireLock(string $key, int $ttl = 300, int $retryTimes = 3, int $retryDelay = 1000): bool - { - $client = self::getClient(); - $lockKey = "lock:{$key}"; - $lockValue = uniqid(gethostname() . '_', true); // 唯一值，用于安全释放锁 - - for ($i = 0; $i <= $retryTimes; $i++) { - // 尝试获取锁（SET key value NX EX ttl） - $result = $client->set($lockKey, $lockValue, 'EX', $ttl, 'NX'); - - if ($result) { - LoggerHelper::logBusiness('redis_lock_acquired', [ - 'key' => $key, - 'ttl' => $ttl, - ]); - return true; - } - - // 如果还有重试机会，等待后重试 - if ($i < $retryTimes) { - usleep($retryDelay * 1000); // 转换为微秒 - } - } - - LoggerHelper::logBusiness('redis_lock_failed', [ - 'key' => $key, - 'retry_times' => $retryTimes, - ]); - - return false; - } - - /** - * 释放分布式锁 - * - * @param string $key 锁的键 - * @return bool 是否释放成功 - */ - public static function releaseLock(string $key): bool - { - $client = self::getClient(); - $lockKey = "lock:{$key}"; - - try { - $result = $client->del([$lockKey]); - - if ($result > 0) { - LoggerHelper::logBusiness('redis_lock_released', [ - 'key' => $key, - ]); - return true; - } - - return false; - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'RedisHelper', - 'action' => 'releaseLock', - 'key' => $key, - ]); - return false; - } - } - - /** - * 设置键值对 - * - * @param string $key 键 - * @param mixed $value 值 - * @param int|null $ttl 过期时间（秒），null 表示不过期 - * @return bool 是否设置成功 - */ - public static function set(string $key, $value, ?int $ttl = null): bool - { - try { - $client = self::getClient(); - $serialized = is_string($value) ? $value : json_encode($value, JSON_UNESCAPED_UNICODE); - - if ($ttl !== null) { - $client->setex($key, $ttl, $serialized); - } else { - $client->set($key, $serialized); - } - - return true; - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'RedisHelper', - 'action' => 'set', - 'key' => $key, - ]); - return false; - } - } - - /** - * 获取键值 - * - * @param string $key 键 - * @return mixed 值，不存在返回 null - */ - public static function get(string $key) - { - try { - $client = self::getClient(); - $value = $client->get($key); - - if ($value === null) { - return null; - } - - // 尝试 JSON 解码 - $decoded = json_decode($value, true); - return $decoded !== null ? $decoded : $value; - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'RedisHelper', - 'action' => 'get', - 'key' => $key, - ]); - return null; - } - } - - /** - * 删除键 - * - * @param string $key 键 - * @return bool 是否删除成功 - */ - public static function delete(string $key): bool - { - try { - $client = self::getClient(); - $result = $client->del([$key]); - return $result > 0; - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'RedisHelper', - 'action' => 'delete', - 'key' => $key, - ]); - return false; - } - } - - /** - * 检查键是否存在 - * - * @param string $key 键 - * @return bool 是否存在 - */ - public static function exists(string $key): bool - { - try { - $client = self::getClient(); - $result = $client->exists($key); - return $result > 0; - } catch (\Throwable $e) { - LoggerHelper::logError($e, [ - 'component' => 'RedisHelper', - 'action' => 'exists', - 'key' => $key, - ]); - return false; - } - } - - /** - * 删除键（别名，兼容del方法） - * - * @param string $key 键 - * @return bool 是否删除成功 - */ - public static function del(string $key): bool - { - return self::delete($key); - } -} - diff --git a/Moncter/Moncter/app/view/index/view.html b/Moncter/Moncter/app/view/index/view.html deleted file mode 100644 index 67ebb26d..00000000 --- a/Moncter/Moncter/app/view/index/view.html +++ /dev/null @@ -1,14 +0,0 @@ - - - - - - - - webman - - - -hello - - diff --git a/Moncter/Moncter/composer.json b/Moncter/Moncter/composer.json deleted file mode 100644 index fcc1e8a5..00000000 --- a/Moncter/Moncter/composer.json +++ /dev/null @@ -1,61 +0,0 @@ -{ - "name": "workerman/webman", - "type": "project", - "keywords": [ - "high performance", - "http service" - ], - "homepage": "https://www.workerman.net", - "license": "MIT", - "description": "High performance HTTP Service Framework.", - "authors": [ - { - "name": "walkor", - "email": "walkor@workerman.net", - "homepage": "https://www.workerman.net", - "role": "Developer" - } - ], - "support": { - "email": "walkor@workerman.net", - "issues": "https://github.com/walkor/webman/issues", - "forum": "https://wenda.workerman.net/", - "wiki": "https://workerman.net/doc/webman", - "source": "https://github.com/walkor/webman" - }, - "require": { - "php": ">=8.1", - "workerman/webman-framework": "^2.1", - "monolog/monolog": "^2.0", - "mongodb/laravel-mongodb": "^4.0", - "vlucas/phpdotenv": "^5.6", - "predis/predis": "^2.0", - "dragonmantank/cron-expression": "^3.6", - "php-amqplib/php-amqplib": "^3.7", - "ramsey/uuid": "^4.7" - }, - "suggest": { - "ext-event": "For better performance. " - }, - "autoload": { - "psr-4": { - "": "./", - "app\\": "./app", - "App\\": "./app", - "app\\View\\Components\\": "./app/view/components" - } - }, - "scripts": { - "post-package-install": [ - "support\\Plugin::install" - ], - "post-package-update": [ - "support\\Plugin::install" - ], - "pre-package-uninstall": [ - "support\\Plugin::uninstall" - ] - }, - "minimum-stability": "dev", - "prefer-stable": true -} diff --git a/Moncter/Moncter/composer.lock b/Moncter/Moncter/composer.lock deleted file mode 100644 index bc535e33..00000000 --- a/Moncter/Moncter/composer.lock +++ /dev/null @@ -1,3349 +0,0 @@ -{ - "_readme": [ - "This file locks the dependencies of your project to a known state", - "Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies", - "This file is @generated automatically" - ], - "content-hash": "6cafa2c36c31a9f9ddfcf8df3e7da924", - "packages": [ - { - "name": "brick/math", - "version": "0.14.0", - "source": { - "type": "git", - "url": "https://github.com/brick/math.git", - "reference": "113a8ee2656b882d4c3164fa31aa6e12cbb7aaa2" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/brick/math/zipball/113a8ee2656b882d4c3164fa31aa6e12cbb7aaa2", - "reference": "113a8ee2656b882d4c3164fa31aa6e12cbb7aaa2", - "shasum": "" - }, - "require": { - "php": "^8.2" - }, - "require-dev": { - "php-coveralls/php-coveralls": "^2.2", - "phpstan/phpstan": "2.1.22", - "phpunit/phpunit": "^11.5" - }, - "type": "library", - "autoload": { - "psr-4": { - "Brick\\Math\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "description": "Arbitrary-precision arithmetic library", - "keywords": [ - "Arbitrary-precision", - "BigInteger", - "BigRational", - "arithmetic", - "bigdecimal", - "bignum", - "bignumber", - "brick", - "decimal", - "integer", - "math", - "mathematics", - "rational" - ], - "support": { - "issues": "https://github.com/brick/math/issues", - "source": "https://github.com/brick/math/tree/0.14.0" - }, - "funding": [ - { - "url": "https://github.com/BenMorel", - "type": "github" - } - ], - "time": "2025-08-29T12:40:03+00:00" - }, - { - "name": "carbonphp/carbon-doctrine-types", - "version": "3.2.0", - "source": { - "type": "git", - "url": "https://github.com/CarbonPHP/carbon-doctrine-types.git", - "reference": "18ba5ddfec8976260ead6e866180bd5d2f71aa1d" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/CarbonPHP/carbon-doctrine-types/zipball/18ba5ddfec8976260ead6e866180bd5d2f71aa1d", - "reference": "18ba5ddfec8976260ead6e866180bd5d2f71aa1d", - "shasum": "" - }, - "require": { - "php": "^8.1" - }, - "conflict": { - "doctrine/dbal": "<4.0.0 || >=5.0.0" - }, - "require-dev": { - "doctrine/dbal": "^4.0.0", - "nesbot/carbon": "^2.71.0 || ^3.0.0", - "phpunit/phpunit": "^10.3" - }, - "type": "library", - "autoload": { - "psr-4": { - "Carbon\\Doctrine\\": "src/Carbon/Doctrine/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "KyleKatarn", - "email": "kylekatarnls@gmail.com" - } - ], - "description": "Types to use Carbon in Doctrine", - "keywords": [ - "carbon", - "date", - "datetime", - "doctrine", - "time" - ], - "support": { - "issues": "https://github.com/CarbonPHP/carbon-doctrine-types/issues", - "source": "https://github.com/CarbonPHP/carbon-doctrine-types/tree/3.2.0" - }, - "funding": [ - { - "url": "https://github.com/kylekatarnls", - "type": "github" - }, - { - "url": "https://opencollective.com/Carbon", - "type": "open_collective" - }, - { - "url": "https://tidelift.com/funding/github/packagist/nesbot/carbon", - "type": "tidelift" - } - ], - "time": "2024-02-09T16:56:22+00:00" - }, - { - "name": "doctrine/inflector", - "version": "2.1.0", - "source": { - "type": "git", - "url": "https://github.com/doctrine/inflector.git", - "reference": "6d6c96277ea252fc1304627204c3d5e6e15faa3b" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/doctrine/inflector/zipball/6d6c96277ea252fc1304627204c3d5e6e15faa3b", - "reference": "6d6c96277ea252fc1304627204c3d5e6e15faa3b", - "shasum": "" - }, - "require": { - "php": "^7.2 || ^8.0" - }, - "require-dev": { - "doctrine/coding-standard": "^12.0 || ^13.0", - "phpstan/phpstan": "^1.12 || ^2.0", - "phpstan/phpstan-phpunit": "^1.4 || ^2.0", - "phpstan/phpstan-strict-rules": "^1.6 || ^2.0", - "phpunit/phpunit": "^8.5 || ^12.2" - }, - "type": "library", - "autoload": { - "psr-4": { - "Doctrine\\Inflector\\": "src" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Guilherme Blanco", - "email": "guilhermeblanco@gmail.com" - }, - { - "name": "Roman Borschel", - "email": "roman@code-factory.org" - }, - { - "name": "Benjamin Eberlei", - "email": "kontakt@beberlei.de" - }, - { - "name": "Jonathan Wage", - "email": "jonwage@gmail.com" - }, - { - "name": "Johannes Schmitt", - "email": "schmittjoh@gmail.com" - } - ], - "description": "PHP Doctrine Inflector is a small library that can perform string manipulations with regard to upper/lowercase and singular/plural forms of words.", - "homepage": "https://www.doctrine-project.org/projects/inflector.html", - "keywords": [ - "inflection", - "inflector", - "lowercase", - "manipulation", - "php", - "plural", - "singular", - "strings", - "uppercase", - "words" - ], - "support": { - "issues": "https://github.com/doctrine/inflector/issues", - "source": "https://github.com/doctrine/inflector/tree/2.1.0" - }, - "funding": [ - { - "url": "https://www.doctrine-project.org/sponsorship.html", - "type": "custom" - }, - { - "url": "https://www.patreon.com/phpdoctrine", - "type": "patreon" - }, - { - "url": "https://tidelift.com/funding/github/packagist/doctrine%2Finflector", - "type": "tidelift" - } - ], - "time": "2025-08-10T19:31:58+00:00" - }, - { - "name": "dragonmantank/cron-expression", - "version": "v3.6.0", - "source": { - "type": "git", - "url": "https://github.com/dragonmantank/cron-expression.git", - "reference": "d61a8a9604ec1f8c3d150d09db6ce98b32675013" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/dragonmantank/cron-expression/zipball/d61a8a9604ec1f8c3d150d09db6ce98b32675013", - "reference": "d61a8a9604ec1f8c3d150d09db6ce98b32675013", - "shasum": "" - }, - "require": { - "php": "^8.2|^8.3|^8.4|^8.5" - }, - "replace": { - "mtdowling/cron-expression": "^1.0" - }, - "require-dev": { - "phpstan/extension-installer": "^1.4.3", - "phpstan/phpstan": "^1.12.32|^2.1.31", - "phpunit/phpunit": "^8.5.48|^9.0" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "3.x-dev" - } - }, - "autoload": { - "psr-4": { - "Cron\\": "src/Cron/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Chris Tankersley", - "email": "chris@ctankersley.com", - "homepage": "https://github.com/dragonmantank" - } - ], - "description": "CRON for PHP: Calculate the next or previous run date and determine if a CRON expression is due", - "keywords": [ - "cron", - "schedule" - ], - "support": { - "issues": "https://github.com/dragonmantank/cron-expression/issues", - "source": "https://github.com/dragonmantank/cron-expression/tree/v3.6.0" - }, - "funding": [ - { - "url": "https://github.com/dragonmantank", - "type": "github" - } - ], - "time": "2025-10-31T18:51:33+00:00" - }, - { - "name": "graham-campbell/result-type", - "version": "v1.1.3", - "source": { - "type": "git", - "url": "https://github.com/GrahamCampbell/Result-Type.git", - "reference": "3ba905c11371512af9d9bdd27d99b782216b6945" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/GrahamCampbell/Result-Type/zipball/3ba905c11371512af9d9bdd27d99b782216b6945", - "reference": "3ba905c11371512af9d9bdd27d99b782216b6945", - "shasum": "" - }, - "require": { - "php": "^7.2.5 || ^8.0", - "phpoption/phpoption": "^1.9.3" - }, - "require-dev": { - "phpunit/phpunit": "^8.5.39 || ^9.6.20 || ^10.5.28" - }, - "type": "library", - "autoload": { - "psr-4": { - "GrahamCampbell\\ResultType\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Graham Campbell", - "email": "hello@gjcampbell.co.uk", - "homepage": "https://github.com/GrahamCampbell" - } - ], - "description": "An Implementation Of The Result Type", - "keywords": [ - "Graham Campbell", - "GrahamCampbell", - "Result Type", - "Result-Type", - "result" - ], - "support": { - "issues": "https://github.com/GrahamCampbell/Result-Type/issues", - "source": "https://github.com/GrahamCampbell/Result-Type/tree/v1.1.3" - }, - "funding": [ - { - "url": "https://github.com/GrahamCampbell", - "type": "github" - }, - { - "url": "https://tidelift.com/funding/github/packagist/graham-campbell/result-type", - "type": "tidelift" - } - ], - "time": "2024-07-20T21:45:45+00:00" - }, - { - "name": "illuminate/bus", - "version": "v11.46.1", - "source": { - "type": "git", - "url": "https://github.com/illuminate/bus.git", - "reference": "5f7cd1f99b2ff7dd0ef20aead81da1390c4bc8e3" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/illuminate/bus/zipball/5f7cd1f99b2ff7dd0ef20aead81da1390c4bc8e3", - "reference": "5f7cd1f99b2ff7dd0ef20aead81da1390c4bc8e3", - "shasum": "" - }, - "require": { - "illuminate/collections": "^11.0", - "illuminate/contracts": "^11.0", - "illuminate/pipeline": "^11.0", - "illuminate/support": "^11.0", - "php": "^8.2" - }, - "suggest": { - "illuminate/queue": "Required to use closures when chaining jobs (^7.0)." - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "11.x-dev" - } - }, - "autoload": { - "psr-4": { - "Illuminate\\Bus\\": "" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Taylor Otwell", - "email": "taylor@laravel.com" - } - ], - "description": "The Illuminate Bus package.", - "homepage": "https://laravel.com", - "support": { - "issues": "https://github.com/laravel/framework/issues", - "source": "https://github.com/laravel/framework" - }, - "time": "2025-03-24T11:54:20+00:00" - }, - { - "name": "illuminate/cache", - "version": "v11.46.1", - "source": { - "type": "git", - "url": "https://github.com/illuminate/cache.git", - "reference": "f9196623f6b75f7e69b9ac92f367491909753987" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/illuminate/cache/zipball/f9196623f6b75f7e69b9ac92f367491909753987", - "reference": "f9196623f6b75f7e69b9ac92f367491909753987", - "shasum": "" - }, - "require": { - "illuminate/collections": "^11.0", - "illuminate/contracts": "^11.0", - "illuminate/macroable": "^11.0", - "illuminate/support": "^11.0", - "php": "^8.2" - }, - "provide": { - "psr/simple-cache-implementation": "1.0|2.0|3.0" - }, - "suggest": { - "ext-apcu": "Required to use the APC cache driver.", - "ext-filter": "Required to use the DynamoDb cache driver.", - "ext-memcached": "Required to use the memcache cache driver.", - "illuminate/database": "Required to use the database cache driver (^11.0).", - "illuminate/filesystem": "Required to use the file cache driver (^11.0).", - "illuminate/redis": "Required to use the redis cache driver (^11.0).", - "symfony/cache": "Required to use PSR-6 cache bridge (^7.0)." - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "11.x-dev" - } - }, - "autoload": { - "psr-4": { - "Illuminate\\Cache\\": "" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Taylor Otwell", - "email": "taylor@laravel.com" - } - ], - "description": "The Illuminate Cache package.", - "homepage": "https://laravel.com", - "support": { - "issues": "https://github.com/laravel/framework/issues", - "source": "https://github.com/laravel/framework" - }, - "time": "2025-01-27T22:47:27+00:00" - }, - { - "name": "illuminate/collections", - "version": "v11.46.1", - "source": { - "type": "git", - "url": "https://github.com/illuminate/collections.git", - "reference": "856b1da953e46281ba61d7c82d337072d3ee1825" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/illuminate/collections/zipball/856b1da953e46281ba61d7c82d337072d3ee1825", - "reference": "856b1da953e46281ba61d7c82d337072d3ee1825", - "shasum": "" - }, - "require": { - "illuminate/conditionable": "^11.0", - "illuminate/contracts": "^11.0", - "illuminate/macroable": "^11.0", - "php": "^8.2" - }, - "suggest": { - "symfony/var-dumper": "Required to use the dump method (^7.0)." - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "11.x-dev" - } - }, - "autoload": { - "files": [ - "functions.php", - "helpers.php" - ], - "psr-4": { - "Illuminate\\Support\\": "" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Taylor Otwell", - "email": "taylor@laravel.com" - } - ], - "description": "The Illuminate Collections package.", - "homepage": "https://laravel.com", - "support": { - "issues": "https://github.com/laravel/framework/issues", - "source": "https://github.com/laravel/framework" - }, - "time": "2025-03-24T11:54:20+00:00" - }, - { - "name": "illuminate/conditionable", - "version": "v11.46.1", - "source": { - "type": "git", - "url": "https://github.com/illuminate/conditionable.git", - "reference": "319b717e0587bd7c8a3b44464f0e27867b4bcda9" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/illuminate/conditionable/zipball/319b717e0587bd7c8a3b44464f0e27867b4bcda9", - "reference": "319b717e0587bd7c8a3b44464f0e27867b4bcda9", - "shasum": "" - }, - "require": { - "php": "^8.0.2" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "11.x-dev" - } - }, - "autoload": { - "psr-4": { - "Illuminate\\Support\\": "" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Taylor Otwell", - "email": "taylor@laravel.com" - } - ], - "description": "The Illuminate Conditionable package.", - "homepage": "https://laravel.com", - "support": { - "issues": "https://github.com/laravel/framework/issues", - "source": "https://github.com/laravel/framework" - }, - "time": "2025-03-24T11:54:20+00:00" - }, - { - "name": "illuminate/container", - "version": "v11.46.1", - "source": { - "type": "git", - "url": "https://github.com/illuminate/container.git", - "reference": "79bf9149ad7ddd7e14326ebcdd41197d2c4ee36a" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/illuminate/container/zipball/79bf9149ad7ddd7e14326ebcdd41197d2c4ee36a", - "reference": "79bf9149ad7ddd7e14326ebcdd41197d2c4ee36a", - "shasum": "" - }, - "require": { - "illuminate/contracts": "^11.0", - "php": "^8.2", - "psr/container": "^1.1.1|^2.0.1" - }, - "provide": { - "psr/container-implementation": "1.1|2.0" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "11.x-dev" - } - }, - "autoload": { - "psr-4": { - "Illuminate\\Container\\": "" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Taylor Otwell", - "email": "taylor@laravel.com" - } - ], - "description": "The Illuminate Container package.", - "homepage": "https://laravel.com", - "support": { - "issues": "https://github.com/laravel/framework/issues", - "source": "https://github.com/laravel/framework" - }, - "time": "2025-03-24T11:54:20+00:00" - }, - { - "name": "illuminate/contracts", - "version": "v11.46.1", - "source": { - "type": "git", - "url": "https://github.com/illuminate/contracts.git", - "reference": "4b2a67d1663f50085bc91e6371492697a5d2d4e8" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/illuminate/contracts/zipball/4b2a67d1663f50085bc91e6371492697a5d2d4e8", - "reference": "4b2a67d1663f50085bc91e6371492697a5d2d4e8", - "shasum": "" - }, - "require": { - "php": "^8.2", - "psr/container": "^1.1.1|^2.0.1", - "psr/simple-cache": "^1.0|^2.0|^3.0" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "11.x-dev" - } - }, - "autoload": { - "psr-4": { - "Illuminate\\Contracts\\": "" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Taylor Otwell", - "email": "taylor@laravel.com" - } - ], - "description": "The Illuminate Contracts package.", - "homepage": "https://laravel.com", - "support": { - "issues": "https://github.com/laravel/framework/issues", - "source": "https://github.com/laravel/framework" - }, - "time": "2025-03-24T11:54:20+00:00" - }, - { - "name": "illuminate/database", - "version": "v11.46.1", - "source": { - "type": "git", - "url": "https://github.com/illuminate/database.git", - "reference": "96abcce13f405701363d916dd312835e04848d04" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/illuminate/database/zipball/96abcce13f405701363d916dd312835e04848d04", - "reference": "96abcce13f405701363d916dd312835e04848d04", - "shasum": "" - }, - "require": { - "brick/math": "^0.9.3|^0.10.2|^0.11|^0.12|^0.13|^0.14", - "ext-pdo": "*", - "illuminate/collections": "^11.0", - "illuminate/container": "^11.0", - "illuminate/contracts": "^11.0", - "illuminate/macroable": "^11.0", - "illuminate/support": "^11.0", - "laravel/serializable-closure": "^1.3|^2.0", - "php": "^8.2" - }, - "suggest": { - "ext-filter": "Required to use the Postgres database driver.", - "fakerphp/faker": "Required to use the eloquent factory builder (^1.24).", - "illuminate/console": "Required to use the database commands (^11.0).", - "illuminate/events": "Required to use the observers with Eloquent (^11.0).", - "illuminate/filesystem": "Required to use the migrations (^11.0).", - "illuminate/pagination": "Required to paginate the result set (^11.0).", - "symfony/finder": "Required to use Eloquent model factories (^7.0)." - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "11.x-dev" - } - }, - "autoload": { - "psr-4": { - "Illuminate\\Database\\": "" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Taylor Otwell", - "email": "taylor@laravel.com" - } - ], - "description": "The Illuminate Database package.", - "homepage": "https://laravel.com", - "keywords": [ - "database", - "laravel", - "orm", - "sql" - ], - "support": { - "issues": "https://github.com/laravel/framework/issues", - "source": "https://github.com/laravel/framework" - }, - "time": "2025-09-29T09:23:31+00:00" - }, - { - "name": "illuminate/events", - "version": "v11.46.1", - "source": { - "type": "git", - "url": "https://github.com/illuminate/events.git", - "reference": "b72dab66d8e05d22dc5aa949efec150bbc73e827" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/illuminate/events/zipball/b72dab66d8e05d22dc5aa949efec150bbc73e827", - "reference": "b72dab66d8e05d22dc5aa949efec150bbc73e827", - "shasum": "" - }, - "require": { - "illuminate/bus": "^11.0", - "illuminate/collections": "^11.0", - "illuminate/container": "^11.0", - "illuminate/contracts": "^11.0", - "illuminate/macroable": "^11.0", - "illuminate/support": "^11.0", - "php": "^8.2" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "11.x-dev" - } - }, - "autoload": { - "files": [ - "functions.php" - ], - "psr-4": { - "Illuminate\\Events\\": "" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Taylor Otwell", - "email": "taylor@laravel.com" - } - ], - "description": "The Illuminate Events package.", - "homepage": "https://laravel.com", - "support": { - "issues": "https://github.com/laravel/framework/issues", - "source": "https://github.com/laravel/framework" - }, - "time": "2025-03-24T11:54:20+00:00" - }, - { - "name": "illuminate/macroable", - "version": "v11.46.1", - "source": { - "type": "git", - "url": "https://github.com/illuminate/macroable.git", - "reference": "e1cb9e51b9ed5d3c9bc1ab431d0a52fe42a990ed" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/illuminate/macroable/zipball/e1cb9e51b9ed5d3c9bc1ab431d0a52fe42a990ed", - "reference": "e1cb9e51b9ed5d3c9bc1ab431d0a52fe42a990ed", - "shasum": "" - }, - "require": { - "php": "^8.2" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "11.x-dev" - } - }, - "autoload": { - "psr-4": { - "Illuminate\\Support\\": "" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Taylor Otwell", - "email": "taylor@laravel.com" - } - ], - "description": "The Illuminate Macroable package.", - "homepage": "https://laravel.com", - "support": { - "issues": "https://github.com/laravel/framework/issues", - "source": "https://github.com/laravel/framework" - }, - "time": "2024-06-28T20:10:30+00:00" - }, - { - "name": "illuminate/pipeline", - "version": "v11.46.1", - "source": { - "type": "git", - "url": "https://github.com/illuminate/pipeline.git", - "reference": "f73bb7cab13ac8ef91094dc46976f5e992eea127" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/illuminate/pipeline/zipball/f73bb7cab13ac8ef91094dc46976f5e992eea127", - "reference": "f73bb7cab13ac8ef91094dc46976f5e992eea127", - "shasum": "" - }, - "require": { - "illuminate/contracts": "^11.0", - "illuminate/support": "^11.0", - "php": "^8.2" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "11.x-dev" - } - }, - "autoload": { - "psr-4": { - "Illuminate\\Pipeline\\": "" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Taylor Otwell", - "email": "taylor@laravel.com" - } - ], - "description": "The Illuminate Pipeline package.", - "homepage": "https://laravel.com", - "support": { - "issues": "https://github.com/laravel/framework/issues", - "source": "https://github.com/laravel/framework" - }, - "time": "2025-03-24T11:54:20+00:00" - }, - { - "name": "illuminate/support", - "version": "v11.46.1", - "source": { - "type": "git", - "url": "https://github.com/illuminate/support.git", - "reference": "716b5e258ee670cf143da883495b22595db12b90" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/illuminate/support/zipball/716b5e258ee670cf143da883495b22595db12b90", - "reference": "716b5e258ee670cf143da883495b22595db12b90", - "shasum": "" - }, - "require": { - "doctrine/inflector": "^2.0", - "ext-ctype": "*", - "ext-filter": "*", - "ext-mbstring": "*", - "illuminate/collections": "^11.0", - "illuminate/conditionable": "^11.0", - "illuminate/contracts": "^11.0", - "illuminate/macroable": "^11.0", - "nesbot/carbon": "^2.72.6|^3.8.4", - "php": "^8.2", - "voku/portable-ascii": "^2.0.2" - }, - "conflict": { - "tightenco/collect": "<5.5.33" - }, - "replace": { - "spatie/once": "*" - }, - "suggest": { - "illuminate/filesystem": "Required to use the Composer class (^11.0).", - "laravel/serializable-closure": "Required to use the once function (^1.3|^2.0).", - "league/commonmark": "Required to use Str::markdown() and Stringable::markdown() (^2.7).", - "league/uri": "Required to use the Uri class (^7.5.1).", - "ramsey/uuid": "Required to use Str::uuid() (^4.7).", - "symfony/process": "Required to use the Composer class (^7.0).", - "symfony/uid": "Required to use Str::ulid() (^7.0).", - "symfony/var-dumper": "Required to use the dd function (^7.0).", - "vlucas/phpdotenv": "Required to use the Env class and env helper (^5.6.1)." - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "11.x-dev" - } - }, - "autoload": { - "files": [ - "functions.php", - "helpers.php" - ], - "psr-4": { - "Illuminate\\Support\\": "" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Taylor Otwell", - "email": "taylor@laravel.com" - } - ], - "description": "The Illuminate Support package.", - "homepage": "https://laravel.com", - "support": { - "issues": "https://github.com/laravel/framework/issues", - "source": "https://github.com/laravel/framework" - }, - "time": "2025-08-11T14:50:36+00:00" - }, - { - "name": "laravel/serializable-closure", - "version": "v2.0.6", - "source": { - "type": "git", - "url": "https://github.com/laravel/serializable-closure.git", - "reference": "038ce42edee619599a1debb7e81d7b3759492819" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/laravel/serializable-closure/zipball/038ce42edee619599a1debb7e81d7b3759492819", - "reference": "038ce42edee619599a1debb7e81d7b3759492819", - "shasum": "" - }, - "require": { - "php": "^8.1" - }, - "require-dev": { - "illuminate/support": "^10.0|^11.0|^12.0", - "nesbot/carbon": "^2.67|^3.0", - "pestphp/pest": "^2.36|^3.0", - "phpstan/phpstan": "^2.0", - "symfony/var-dumper": "^6.2.0|^7.0.0" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "2.x-dev" - } - }, - "autoload": { - "psr-4": { - "Laravel\\SerializableClosure\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Taylor Otwell", - "email": "taylor@laravel.com" - }, - { - "name": "Nuno Maduro", - "email": "nuno@laravel.com" - } - ], - "description": "Laravel Serializable Closure provides an easy and secure way to serialize closures in PHP.", - "keywords": [ - "closure", - "laravel", - "serializable" - ], - "support": { - "issues": "https://github.com/laravel/serializable-closure/issues", - "source": "https://github.com/laravel/serializable-closure" - }, - "time": "2025-10-09T13:42:30+00:00" - }, - { - "name": "mongodb/laravel-mongodb", - "version": "4.8.1", - "source": { - "type": "git", - "url": "https://github.com/mongodb/laravel-mongodb.git", - "reference": "da3a46a1b4ca25117c1d388dd6348206d04e4a9f" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/mongodb/laravel-mongodb/zipball/da3a46a1b4ca25117c1d388dd6348206d04e4a9f", - "reference": "da3a46a1b4ca25117c1d388dd6348206d04e4a9f", - "shasum": "" - }, - "require": { - "composer-runtime-api": "^2.0.0", - "ext-mongodb": "^1.15", - "illuminate/cache": "^10.36|^11", - "illuminate/container": "^10.0|^11", - "illuminate/database": "^10.30|^11", - "illuminate/events": "^10.0|^11", - "illuminate/support": "^10.0|^11", - "mongodb/mongodb": "^1.15", - "php": "^8.1" - }, - "conflict": { - "illuminate/bus": "< 10.37.2" - }, - "replace": { - "jenssegers/mongodb": "self.version" - }, - "require-dev": { - "doctrine/coding-standard": "12.0.x-dev", - "league/flysystem-gridfs": "^3.28", - "league/flysystem-read-only": "^3.0", - "mockery/mockery": "^1.4.4", - "mongodb/builder": "^0.2", - "orchestra/testbench": "^8.0|^9.0", - "phpstan/phpstan": "^1.10", - "phpunit/phpunit": "^10.3", - "spatie/laravel-query-builder": "^5.6" - }, - "suggest": { - "league/flysystem-gridfs": "Filesystem storage in MongoDB with GridFS", - "mongodb/builder": "Provides a fluent aggregation builder for MongoDB pipelines" - }, - "type": "library", - "extra": { - "laravel": { - "providers": [ - "MongoDB\\Laravel\\MongoDBServiceProvider", - "MongoDB\\Laravel\\MongoDBQueueServiceProvider", - "MongoDB\\Laravel\\MongoDBBusServiceProvider" - ] - } - }, - "autoload": { - "psr-4": { - "MongoDB\\Laravel\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Andreas Braun", - "email": "andreas.braun@mongodb.com", - "role": "Leader" - }, - { - "name": "Jérôme Tamarelle", - "email": "jerome.tamarelle@mongodb.com", - "role": "Maintainer" - }, - { - "name": "Jeremy Mikola", - "email": "jmikola@gmail.com", - "role": "Maintainer" - }, - { - "name": "Jens Segers", - "homepage": "https://jenssegers.com", - "role": "Creator" - } - ], - "description": "A MongoDB based Eloquent model and Query builder for Laravel", - "homepage": "https://github.com/mongodb/laravel-mongodb", - "keywords": [ - "database", - "eloquent", - "laravel", - "model", - "mongo", - "mongodb" - ], - "support": { - "issues": "https://www.mongodb.com/support", - "security": "https://www.mongodb.com/security", - "source": "https://github.com/mongodb/laravel-mongodb/tree/4.8.1" - }, - "time": "2024-11-20T15:01:02+00:00" - }, - { - "name": "mongodb/mongodb", - "version": "1.20.0", - "source": { - "type": "git", - "url": "https://github.com/mongodb/mongo-php-library.git", - "reference": "75da9ea3b63d97b05e0e8648d8c09a17bc54c0b6" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/mongodb/mongo-php-library/zipball/75da9ea3b63d97b05e0e8648d8c09a17bc54c0b6", - "reference": "75da9ea3b63d97b05e0e8648d8c09a17bc54c0b6", - "shasum": "" - }, - "require": { - "composer-runtime-api": "^2.0", - "ext-hash": "*", - "ext-json": "*", - "ext-mongodb": "^1.20.0", - "php": "^7.4 || ^8.0", - "psr/log": "^1.1.4|^2|^3", - "symfony/polyfill-php80": "^1.27", - "symfony/polyfill-php81": "^1.27" - }, - "require-dev": { - "doctrine/coding-standard": "^12.0", - "rector/rector": "^1.1", - "squizlabs/php_codesniffer": "^3.7", - "symfony/phpunit-bridge": "^5.2", - "vimeo/psalm": "^5.13" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "1.x-dev" - } - }, - "autoload": { - "files": [ - "src/functions.php" - ], - "psr-4": { - "MongoDB\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "Apache-2.0" - ], - "authors": [ - { - "name": "Andreas Braun", - "email": "andreas.braun@mongodb.com" - }, - { - "name": "Jeremy Mikola", - "email": "jmikola@gmail.com" - }, - { - "name": "Jérôme Tamarelle", - "email": "jerome.tamarelle@mongodb.com" - } - ], - "description": "MongoDB driver library", - "homepage": "https://jira.mongodb.org/browse/PHPLIB", - "keywords": [ - "database", - "driver", - "mongodb", - "persistence" - ], - "support": { - "issues": "https://github.com/mongodb/mongo-php-library/issues", - "source": "https://github.com/mongodb/mongo-php-library/tree/1.20.0" - }, - "time": "2024-09-25T12:54:08+00:00" - }, - { - "name": "monolog/monolog", - "version": "2.10.0", - "source": { - "type": "git", - "url": "https://github.com/Seldaek/monolog.git", - "reference": "5cf826f2991858b54d5c3809bee745560a1042a7" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/Seldaek/monolog/zipball/5cf826f2991858b54d5c3809bee745560a1042a7", - "reference": "5cf826f2991858b54d5c3809bee745560a1042a7", - "shasum": "" - }, - "require": { - "php": ">=7.2", - "psr/log": "^1.0.1 || ^2.0 || ^3.0" - }, - "provide": { - "psr/log-implementation": "1.0.0 || 2.0.0 || 3.0.0" - }, - "require-dev": { - "aws/aws-sdk-php": "^2.4.9 || ^3.0", - "doctrine/couchdb": "~1.0@dev", - "elasticsearch/elasticsearch": "^7 || ^8", - "ext-json": "*", - "graylog2/gelf-php": "^1.4.2 || ^2@dev", - "guzzlehttp/guzzle": "^7.4", - "guzzlehttp/psr7": "^2.2", - "mongodb/mongodb": "^1.8", - "php-amqplib/php-amqplib": "~2.4 || ^3", - "phpspec/prophecy": "^1.15", - "phpstan/phpstan": "^1.10", - "phpunit/phpunit": "^8.5.38 || ^9.6.19", - "predis/predis": "^1.1 || ^2.0", - "rollbar/rollbar": "^1.3 || ^2 || ^3", - "ruflin/elastica": "^7", - "swiftmailer/swiftmailer": "^5.3|^6.0", - "symfony/mailer": "^5.4 || ^6", - "symfony/mime": "^5.4 || ^6" - }, - "suggest": { - "aws/aws-sdk-php": "Allow sending log messages to AWS services like DynamoDB", - "doctrine/couchdb": "Allow sending log messages to a CouchDB server", - "elasticsearch/elasticsearch": "Allow sending log messages to an Elasticsearch server via official client", - "ext-amqp": "Allow sending log messages to an AMQP server (1.0+ required)", - "ext-curl": "Required to send log messages using the IFTTTHandler, the LogglyHandler, the SendGridHandler, the SlackWebhookHandler or the TelegramBotHandler", - "ext-mbstring": "Allow to work properly with unicode symbols", - "ext-mongodb": "Allow sending log messages to a MongoDB server (via driver)", - "ext-openssl": "Required to send log messages using SSL", - "ext-sockets": "Allow sending log messages to a Syslog server (via UDP driver)", - "graylog2/gelf-php": "Allow sending log messages to a GrayLog2 server", - "mongodb/mongodb": "Allow sending log messages to a MongoDB server (via library)", - "php-amqplib/php-amqplib": "Allow sending log messages to an AMQP server using php-amqplib", - "rollbar/rollbar": "Allow sending log messages to Rollbar", - "ruflin/elastica": "Allow sending log messages to an Elastic Search server" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-main": "2.x-dev" - } - }, - "autoload": { - "psr-4": { - "Monolog\\": "src/Monolog" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Jordi Boggiano", - "email": "j.boggiano@seld.be", - "homepage": "https://seld.be" - } - ], - "description": "Sends your logs to files, sockets, inboxes, databases and various web services", - "homepage": "https://github.com/Seldaek/monolog", - "keywords": [ - "log", - "logging", - "psr-3" - ], - "support": { - "issues": "https://github.com/Seldaek/monolog/issues", - "source": "https://github.com/Seldaek/monolog/tree/2.10.0" - }, - "funding": [ - { - "url": "https://github.com/Seldaek", - "type": "github" - }, - { - "url": "https://tidelift.com/funding/github/packagist/monolog/monolog", - "type": "tidelift" - } - ], - "time": "2024-11-12T12:43:37+00:00" - }, - { - "name": "nesbot/carbon", - "version": "3.10.3", - "source": { - "type": "git", - "url": "https://github.com/CarbonPHP/carbon.git", - "reference": "8e3643dcd149ae0fe1d2ff4f2c8e4bbfad7c165f" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/CarbonPHP/carbon/zipball/8e3643dcd149ae0fe1d2ff4f2c8e4bbfad7c165f", - "reference": "8e3643dcd149ae0fe1d2ff4f2c8e4bbfad7c165f", - "shasum": "" - }, - "require": { - "carbonphp/carbon-doctrine-types": "<100.0", - "ext-json": "*", - "php": "^8.1", - "psr/clock": "^1.0", - "symfony/clock": "^6.3.12 || ^7.0", - "symfony/polyfill-mbstring": "^1.0", - "symfony/translation": "^4.4.18 || ^5.2.1 || ^6.0 || ^7.0" - }, - "provide": { - "psr/clock-implementation": "1.0" - }, - "require-dev": { - "doctrine/dbal": "^3.6.3 || ^4.0", - "doctrine/orm": "^2.15.2 || ^3.0", - "friendsofphp/php-cs-fixer": "^v3.87.1", - "kylekatarnls/multi-tester": "^2.5.3", - "phpmd/phpmd": "^2.15.0", - "phpstan/extension-installer": "^1.4.3", - "phpstan/phpstan": "^2.1.22", - "phpunit/phpunit": "^10.5.53", - "squizlabs/php_codesniffer": "^3.13.4" - }, - "bin": [ - "bin/carbon" - ], - "type": "library", - "extra": { - "laravel": { - "providers": [ - "Carbon\\Laravel\\ServiceProvider" - ] - }, - "phpstan": { - "includes": [ - "extension.neon" - ] - }, - "branch-alias": { - "dev-2.x": "2.x-dev", - "dev-master": "3.x-dev" - } - }, - "autoload": { - "psr-4": { - "Carbon\\": "src/Carbon/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Brian Nesbitt", - "email": "brian@nesbot.com", - "homepage": "https://markido.com" - }, - { - "name": "kylekatarnls", - "homepage": "https://github.com/kylekatarnls" - } - ], - "description": "An API extension for DateTime that supports 281 different languages.", - "homepage": "https://carbon.nesbot.com", - "keywords": [ - "date", - "datetime", - "time" - ], - "support": { - "docs": "https://carbon.nesbot.com/docs", - "issues": "https://github.com/CarbonPHP/carbon/issues", - "source": "https://github.com/CarbonPHP/carbon" - }, - "funding": [ - { - "url": "https://github.com/sponsors/kylekatarnls", - "type": "github" - }, - { - "url": "https://opencollective.com/Carbon#sponsor", - "type": "opencollective" - }, - { - "url": "https://tidelift.com/subscription/pkg/packagist-nesbot-carbon?utm_source=packagist-nesbot-carbon&utm_medium=referral&utm_campaign=readme", - "type": "tidelift" - } - ], - "time": "2025-09-06T13:39:36+00:00" - }, - { - "name": "nikic/fast-route", - "version": "v1.3.0", - "source": { - "type": "git", - "url": "https://github.com/nikic/FastRoute.git", - "reference": "181d480e08d9476e61381e04a71b34dc0432e812" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/nikic/FastRoute/zipball/181d480e08d9476e61381e04a71b34dc0432e812", - "reference": "181d480e08d9476e61381e04a71b34dc0432e812", - "shasum": "" - }, - "require": { - "php": ">=5.4.0" - }, - "require-dev": { - "phpunit/phpunit": "^4.8.35|~5.7" - }, - "type": "library", - "autoload": { - "files": [ - "src/functions.php" - ], - "psr-4": { - "FastRoute\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "BSD-3-Clause" - ], - "authors": [ - { - "name": "Nikita Popov", - "email": "nikic@php.net" - } - ], - "description": "Fast request router for PHP", - "keywords": [ - "router", - "routing" - ], - "support": { - "issues": "https://github.com/nikic/FastRoute/issues", - "source": "https://github.com/nikic/FastRoute/tree/master" - }, - "time": "2018-02-13T20:26:39+00:00" - }, - { - "name": "paragonie/constant_time_encoding", - "version": "v3.1.3", - "source": { - "type": "git", - "url": "https://github.com/paragonie/constant_time_encoding.git", - "reference": "d5b01a39b3415c2cd581d3bd3a3575c1ebbd8e77" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/paragonie/constant_time_encoding/zipball/d5b01a39b3415c2cd581d3bd3a3575c1ebbd8e77", - "reference": "d5b01a39b3415c2cd581d3bd3a3575c1ebbd8e77", - "shasum": "" - }, - "require": { - "php": "^8" - }, - "require-dev": { - "infection/infection": "^0", - "nikic/php-fuzzer": "^0", - "phpunit/phpunit": "^9|^10|^11", - "vimeo/psalm": "^4|^5|^6" - }, - "type": "library", - "autoload": { - "psr-4": { - "ParagonIE\\ConstantTime\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Paragon Initiative Enterprises", - "email": "security@paragonie.com", - "homepage": "https://paragonie.com", - "role": "Maintainer" - }, - { - "name": "Steve 'Sc00bz' Thomas", - "email": "steve@tobtu.com", - "homepage": "https://www.tobtu.com", - "role": "Original Developer" - } - ], - "description": "Constant-time Implementations of RFC 4648 Encoding (Base-64, Base-32, Base-16)", - "keywords": [ - "base16", - "base32", - "base32_decode", - "base32_encode", - "base64", - "base64_decode", - "base64_encode", - "bin2hex", - "encoding", - "hex", - "hex2bin", - "rfc4648" - ], - "support": { - "email": "info@paragonie.com", - "issues": "https://github.com/paragonie/constant_time_encoding/issues", - "source": "https://github.com/paragonie/constant_time_encoding" - }, - "time": "2025-09-24T15:06:41+00:00" - }, - { - "name": "paragonie/random_compat", - "version": "v9.99.100", - "source": { - "type": "git", - "url": "https://github.com/paragonie/random_compat.git", - "reference": "996434e5492cb4c3edcb9168db6fbb1359ef965a" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/paragonie/random_compat/zipball/996434e5492cb4c3edcb9168db6fbb1359ef965a", - "reference": "996434e5492cb4c3edcb9168db6fbb1359ef965a", - "shasum": "" - }, - "require": { - "php": ">= 7" - }, - "require-dev": { - "phpunit/phpunit": "4.*|5.*", - "vimeo/psalm": "^1" - }, - "suggest": { - "ext-libsodium": "Provides a modern crypto API that can be used to generate random bytes." - }, - "type": "library", - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Paragon Initiative Enterprises", - "email": "security@paragonie.com", - "homepage": "https://paragonie.com" - } - ], - "description": "PHP 5.x polyfill for random_bytes() and random_int() from PHP 7", - "keywords": [ - "csprng", - "polyfill", - "pseudorandom", - "random" - ], - "support": { - "email": "info@paragonie.com", - "issues": "https://github.com/paragonie/random_compat/issues", - "source": "https://github.com/paragonie/random_compat" - }, - "time": "2020-10-15T08:29:30+00:00" - }, - { - "name": "php-amqplib/php-amqplib", - "version": "v3.7.4", - "source": { - "type": "git", - "url": "https://github.com/php-amqplib/php-amqplib.git", - "reference": "381b6f7c600e0e0c7463cdd7f7a1a3bc6268e5fd" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/php-amqplib/php-amqplib/zipball/381b6f7c600e0e0c7463cdd7f7a1a3bc6268e5fd", - "reference": "381b6f7c600e0e0c7463cdd7f7a1a3bc6268e5fd", - "shasum": "" - }, - "require": { - "ext-mbstring": "*", - "ext-sockets": "*", - "php": "^7.2||^8.0", - "phpseclib/phpseclib": "^2.0|^3.0" - }, - "conflict": { - "php": "7.4.0 - 7.4.1" - }, - "replace": { - "videlalvaro/php-amqplib": "self.version" - }, - "require-dev": { - "ext-curl": "*", - "nategood/httpful": "^0.2.20", - "phpunit/phpunit": "^7.5|^9.5", - "squizlabs/php_codesniffer": "^3.6" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "3.0-dev" - } - }, - "autoload": { - "psr-4": { - "PhpAmqpLib\\": "PhpAmqpLib/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "LGPL-2.1-or-later" - ], - "authors": [ - { - "name": "Alvaro Videla", - "role": "Original Maintainer" - }, - { - "name": "Raúl Araya", - "email": "nubeiro@gmail.com", - "role": "Maintainer" - }, - { - "name": "Luke Bakken", - "email": "luke@bakken.io", - "role": "Maintainer" - }, - { - "name": "Ramūnas Dronga", - "email": "github@ramuno.lt", - "role": "Maintainer" - } - ], - "description": "Formerly videlalvaro/php-amqplib. This library is a pure PHP implementation of the AMQP protocol. It's been tested against RabbitMQ.", - "homepage": "https://github.com/php-amqplib/php-amqplib/", - "keywords": [ - "message", - "queue", - "rabbitmq" - ], - "support": { - "issues": "https://github.com/php-amqplib/php-amqplib/issues", - "source": "https://github.com/php-amqplib/php-amqplib/tree/v3.7.4" - }, - "time": "2025-11-23T17:00:56+00:00" - }, - { - "name": "phpoption/phpoption", - "version": "1.9.4", - "source": { - "type": "git", - "url": "https://github.com/schmittjoh/php-option.git", - "reference": "638a154f8d4ee6a5cfa96d6a34dfbe0cffa9566d" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/schmittjoh/php-option/zipball/638a154f8d4ee6a5cfa96d6a34dfbe0cffa9566d", - "reference": "638a154f8d4ee6a5cfa96d6a34dfbe0cffa9566d", - "shasum": "" - }, - "require": { - "php": "^7.2.5 || ^8.0" - }, - "require-dev": { - "bamarni/composer-bin-plugin": "^1.8.2", - "phpunit/phpunit": "^8.5.44 || ^9.6.25 || ^10.5.53 || ^11.5.34" - }, - "type": "library", - "extra": { - "bamarni-bin": { - "bin-links": true, - "forward-command": false - }, - "branch-alias": { - "dev-master": "1.9-dev" - } - }, - "autoload": { - "psr-4": { - "PhpOption\\": "src/PhpOption/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "Apache-2.0" - ], - "authors": [ - { - "name": "Johannes M. Schmitt", - "email": "schmittjoh@gmail.com", - "homepage": "https://github.com/schmittjoh" - }, - { - "name": "Graham Campbell", - "email": "hello@gjcampbell.co.uk", - "homepage": "https://github.com/GrahamCampbell" - } - ], - "description": "Option Type for PHP", - "keywords": [ - "language", - "option", - "php", - "type" - ], - "support": { - "issues": "https://github.com/schmittjoh/php-option/issues", - "source": "https://github.com/schmittjoh/php-option/tree/1.9.4" - }, - "funding": [ - { - "url": "https://github.com/GrahamCampbell", - "type": "github" - }, - { - "url": "https://tidelift.com/funding/github/packagist/phpoption/phpoption", - "type": "tidelift" - } - ], - "time": "2025-08-21T11:53:16+00:00" - }, - { - "name": "phpseclib/phpseclib", - "version": "3.0.48", - "source": { - "type": "git", - "url": "https://github.com/phpseclib/phpseclib.git", - "reference": "64065a5679c50acb886e82c07aa139b0f757bb89" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/phpseclib/phpseclib/zipball/64065a5679c50acb886e82c07aa139b0f757bb89", - "reference": "64065a5679c50acb886e82c07aa139b0f757bb89", - "shasum": "" - }, - "require": { - "paragonie/constant_time_encoding": "^1|^2|^3", - "paragonie/random_compat": "^1.4|^2.0|^9.99.99", - "php": ">=5.6.1" - }, - "require-dev": { - "phpunit/phpunit": "*" - }, - "suggest": { - "ext-dom": "Install the DOM extension to load XML formatted public keys.", - "ext-gmp": "Install the GMP (GNU Multiple Precision) extension in order to speed up arbitrary precision integer arithmetic operations.", - "ext-libsodium": "SSH2/SFTP can make use of some algorithms provided by the libsodium-php extension.", - "ext-mcrypt": "Install the Mcrypt extension in order to speed up a few other cryptographic operations.", - "ext-openssl": "Install the OpenSSL extension in order to speed up a wide variety of cryptographic operations." - }, - "type": "library", - "autoload": { - "files": [ - "phpseclib/bootstrap.php" - ], - "psr-4": { - "phpseclib3\\": "phpseclib/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Jim Wigginton", - "email": "terrafrost@php.net", - "role": "Lead Developer" - }, - { - "name": "Patrick Monnerat", - "email": "pm@datasphere.ch", - "role": "Developer" - }, - { - "name": "Andreas Fischer", - "email": "bantu@phpbb.com", - "role": "Developer" - }, - { - "name": "Hans-Jürgen Petrich", - "email": "petrich@tronic-media.com", - "role": "Developer" - }, - { - "name": "Graham Campbell", - "email": "graham@alt-three.com", - "role": "Developer" - } - ], - "description": "PHP Secure Communications Library - Pure-PHP implementations of RSA, AES, SSH2, SFTP, X.509 etc.", - "homepage": "http://phpseclib.sourceforge.net", - "keywords": [ - "BigInteger", - "aes", - "asn.1", - "asn1", - "blowfish", - "crypto", - "cryptography", - "encryption", - "rsa", - "security", - "sftp", - "signature", - "signing", - "ssh", - "twofish", - "x.509", - "x509" - ], - "support": { - "issues": "https://github.com/phpseclib/phpseclib/issues", - "source": "https://github.com/phpseclib/phpseclib/tree/3.0.48" - }, - "funding": [ - { - "url": "https://github.com/terrafrost", - "type": "github" - }, - { - "url": "https://www.patreon.com/phpseclib", - "type": "patreon" - }, - { - "url": "https://tidelift.com/funding/github/packagist/phpseclib/phpseclib", - "type": "tidelift" - } - ], - "time": "2025-12-15T11:51:42+00:00" - }, - { - "name": "predis/predis", - "version": "v2.4.1", - "source": { - "type": "git", - "url": "https://github.com/predis/predis.git", - "reference": "07105e050622ed80bd60808367ced9e379f31530" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/predis/predis/zipball/07105e050622ed80bd60808367ced9e379f31530", - "reference": "07105e050622ed80bd60808367ced9e379f31530", - "shasum": "" - }, - "require": { - "php": "^7.2 || ^8.0" - }, - "require-dev": { - "friendsofphp/php-cs-fixer": "^3.3", - "phpstan/phpstan": "^1.9", - "phpunit/phpcov": "^6.0 || ^8.0", - "phpunit/phpunit": "^8.0 || ^9.4" - }, - "suggest": { - "ext-relay": "Faster connection with in-memory caching (>=0.6.2)" - }, - "type": "library", - "autoload": { - "psr-4": { - "Predis\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Till Krüss", - "homepage": "https://till.im", - "role": "Maintainer" - } - ], - "description": "A flexible and feature-complete Redis/Valkey client for PHP.", - "homepage": "http://github.com/predis/predis", - "keywords": [ - "nosql", - "predis", - "redis" - ], - "support": { - "issues": "https://github.com/predis/predis/issues", - "source": "https://github.com/predis/predis/tree/v2.4.1" - }, - "funding": [ - { - "url": "https://github.com/sponsors/tillkruss", - "type": "github" - } - ], - "time": "2025-11-12T18:00:11+00:00" - }, - { - "name": "psr/clock", - "version": "1.0.0", - "source": { - "type": "git", - "url": "https://github.com/php-fig/clock.git", - "reference": "e41a24703d4560fd0acb709162f73b8adfc3aa0d" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/php-fig/clock/zipball/e41a24703d4560fd0acb709162f73b8adfc3aa0d", - "reference": "e41a24703d4560fd0acb709162f73b8adfc3aa0d", - "shasum": "" - }, - "require": { - "php": "^7.0 || ^8.0" - }, - "type": "library", - "autoload": { - "psr-4": { - "Psr\\Clock\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "PHP-FIG", - "homepage": "https://www.php-fig.org/" - } - ], - "description": "Common interface for reading the clock.", - "homepage": "https://github.com/php-fig/clock", - "keywords": [ - "clock", - "now", - "psr", - "psr-20", - "time" - ], - "support": { - "issues": "https://github.com/php-fig/clock/issues", - "source": "https://github.com/php-fig/clock/tree/1.0.0" - }, - "time": "2022-11-25T14:36:26+00:00" - }, - { - "name": "psr/container", - "version": "2.0.2", - "source": { - "type": "git", - "url": "https://github.com/php-fig/container.git", - "reference": "c71ecc56dfe541dbd90c5360474fbc405f8d5963" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/php-fig/container/zipball/c71ecc56dfe541dbd90c5360474fbc405f8d5963", - "reference": "c71ecc56dfe541dbd90c5360474fbc405f8d5963", - "shasum": "" - }, - "require": { - "php": ">=7.4.0" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "2.0.x-dev" - } - }, - "autoload": { - "psr-4": { - "Psr\\Container\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "PHP-FIG", - "homepage": "https://www.php-fig.org/" - } - ], - "description": "Common Container Interface (PHP FIG PSR-11)", - "homepage": "https://github.com/php-fig/container", - "keywords": [ - "PSR-11", - "container", - "container-interface", - "container-interop", - "psr" - ], - "support": { - "issues": "https://github.com/php-fig/container/issues", - "source": "https://github.com/php-fig/container/tree/2.0.2" - }, - "time": "2021-11-05T16:47:00+00:00" - }, - { - "name": "psr/log", - "version": "3.0.2", - "source": { - "type": "git", - "url": "https://github.com/php-fig/log.git", - "reference": "f16e1d5863e37f8d8c2a01719f5b34baa2b714d3" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/php-fig/log/zipball/f16e1d5863e37f8d8c2a01719f5b34baa2b714d3", - "reference": "f16e1d5863e37f8d8c2a01719f5b34baa2b714d3", - "shasum": "" - }, - "require": { - "php": ">=8.0.0" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "3.x-dev" - } - }, - "autoload": { - "psr-4": { - "Psr\\Log\\": "src" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "PHP-FIG", - "homepage": "https://www.php-fig.org/" - } - ], - "description": "Common interface for logging libraries", - "homepage": "https://github.com/php-fig/log", - "keywords": [ - "log", - "psr", - "psr-3" - ], - "support": { - "source": "https://github.com/php-fig/log/tree/3.0.2" - }, - "time": "2024-09-11T13:17:53+00:00" - }, - { - "name": "psr/simple-cache", - "version": "3.0.0", - "source": { - "type": "git", - "url": "https://github.com/php-fig/simple-cache.git", - "reference": "764e0b3939f5ca87cb904f570ef9be2d78a07865" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/php-fig/simple-cache/zipball/764e0b3939f5ca87cb904f570ef9be2d78a07865", - "reference": "764e0b3939f5ca87cb904f570ef9be2d78a07865", - "shasum": "" - }, - "require": { - "php": ">=8.0.0" - }, - "type": "library", - "extra": { - "branch-alias": { - "dev-master": "3.0.x-dev" - } - }, - "autoload": { - "psr-4": { - "Psr\\SimpleCache\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "PHP-FIG", - "homepage": "https://www.php-fig.org/" - } - ], - "description": "Common interfaces for simple caching", - "keywords": [ - "cache", - "caching", - "psr", - "psr-16", - "simple-cache" - ], - "support": { - "source": "https://github.com/php-fig/simple-cache/tree/3.0.0" - }, - "time": "2021-10-29T13:26:27+00:00" - }, - { - "name": "ramsey/collection", - "version": "2.1.1", - "source": { - "type": "git", - "url": "https://github.com/ramsey/collection.git", - "reference": "344572933ad0181accbf4ba763e85a0306a8c5e2" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/ramsey/collection/zipball/344572933ad0181accbf4ba763e85a0306a8c5e2", - "reference": "344572933ad0181accbf4ba763e85a0306a8c5e2", - "shasum": "" - }, - "require": { - "php": "^8.1" - }, - "require-dev": { - "captainhook/plugin-composer": "^5.3", - "ergebnis/composer-normalize": "^2.45", - "fakerphp/faker": "^1.24", - "hamcrest/hamcrest-php": "^2.0", - "jangregor/phpstan-prophecy": "^2.1", - "mockery/mockery": "^1.6", - "php-parallel-lint/php-console-highlighter": "^1.0", - "php-parallel-lint/php-parallel-lint": "^1.4", - "phpspec/prophecy-phpunit": "^2.3", - "phpstan/extension-installer": "^1.4", - "phpstan/phpstan": "^2.1", - "phpstan/phpstan-mockery": "^2.0", - "phpstan/phpstan-phpunit": "^2.0", - "phpunit/phpunit": "^10.5", - "ramsey/coding-standard": "^2.3", - "ramsey/conventional-commits": "^1.6", - "roave/security-advisories": "dev-latest" - }, - "type": "library", - "extra": { - "captainhook": { - "force-install": true - }, - "ramsey/conventional-commits": { - "configFile": "conventional-commits.json" - } - }, - "autoload": { - "psr-4": { - "Ramsey\\Collection\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Ben Ramsey", - "email": "ben@benramsey.com", - "homepage": "https://benramsey.com" - } - ], - "description": "A PHP library for representing and manipulating collections.", - "keywords": [ - "array", - "collection", - "hash", - "map", - "queue", - "set" - ], - "support": { - "issues": "https://github.com/ramsey/collection/issues", - "source": "https://github.com/ramsey/collection/tree/2.1.1" - }, - "time": "2025-03-22T05:38:12+00:00" - }, - { - "name": "ramsey/uuid", - "version": "4.9.2", - "source": { - "type": "git", - "url": "https://github.com/ramsey/uuid.git", - "reference": "8429c78ca35a09f27565311b98101e2826affde0" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/ramsey/uuid/zipball/8429c78ca35a09f27565311b98101e2826affde0", - "reference": "8429c78ca35a09f27565311b98101e2826affde0", - "shasum": "" - }, - "require": { - "brick/math": "^0.8.16 || ^0.9 || ^0.10 || ^0.11 || ^0.12 || ^0.13 || ^0.14", - "php": "^8.0", - "ramsey/collection": "^1.2 || ^2.0" - }, - "replace": { - "rhumsaa/uuid": "self.version" - }, - "require-dev": { - "captainhook/captainhook": "^5.25", - "captainhook/plugin-composer": "^5.3", - "dealerdirect/phpcodesniffer-composer-installer": "^1.0", - "ergebnis/composer-normalize": "^2.47", - "mockery/mockery": "^1.6", - "paragonie/random-lib": "^2", - "php-mock/php-mock": "^2.6", - "php-mock/php-mock-mockery": "^1.5", - "php-parallel-lint/php-parallel-lint": "^1.4.0", - "phpbench/phpbench": "^1.2.14", - "phpstan/extension-installer": "^1.4", - "phpstan/phpstan": "^2.1", - "phpstan/phpstan-mockery": "^2.0", - "phpstan/phpstan-phpunit": "^2.0", - "phpunit/phpunit": "^9.6", - "slevomat/coding-standard": "^8.18", - "squizlabs/php_codesniffer": "^3.13" - }, - "suggest": { - "ext-bcmath": "Enables faster math with arbitrary-precision integers using BCMath.", - "ext-gmp": "Enables faster math with arbitrary-precision integers using GMP.", - "ext-uuid": "Enables the use of PeclUuidTimeGenerator and PeclUuidRandomGenerator.", - "paragonie/random-lib": "Provides RandomLib for use with the RandomLibAdapter", - "ramsey/uuid-doctrine": "Allows the use of Ramsey\\Uuid\\Uuid as Doctrine field type." - }, - "type": "library", - "extra": { - "captainhook": { - "force-install": true - } - }, - "autoload": { - "files": [ - "src/functions.php" - ], - "psr-4": { - "Ramsey\\Uuid\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "description": "A PHP library for generating and working with universally unique identifiers (UUIDs).", - "keywords": [ - "guid", - "identifier", - "uuid" - ], - "support": { - "issues": "https://github.com/ramsey/uuid/issues", - "source": "https://github.com/ramsey/uuid/tree/4.9.2" - }, - "time": "2025-12-14T04:43:48+00:00" - }, - { - "name": "symfony/clock", - "version": "v7.3.0", - "source": { - "type": "git", - "url": "https://github.com/symfony/clock.git", - "reference": "b81435fbd6648ea425d1ee96a2d8e68f4ceacd24" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/symfony/clock/zipball/b81435fbd6648ea425d1ee96a2d8e68f4ceacd24", - "reference": "b81435fbd6648ea425d1ee96a2d8e68f4ceacd24", - "shasum": "" - }, - "require": { - "php": ">=8.2", - "psr/clock": "^1.0", - "symfony/polyfill-php83": "^1.28" - }, - "provide": { - "psr/clock-implementation": "1.0" - }, - "type": "library", - "autoload": { - "files": [ - "Resources/now.php" - ], - "psr-4": { - "Symfony\\Component\\Clock\\": "" - }, - "exclude-from-classmap": [ - "/Tests/" - ] - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Nicolas Grekas", - "email": "p@tchwork.com" - }, - { - "name": "Symfony Community", - "homepage": "https://symfony.com/contributors" - } - ], - "description": "Decouples applications from the system clock", - "homepage": "https://symfony.com", - "keywords": [ - "clock", - "psr20", - "time" - ], - "support": { - "source": "https://github.com/symfony/clock/tree/v7.3.0" - }, - "funding": [ - { - "url": "https://symfony.com/sponsor", - "type": "custom" - }, - { - "url": "https://github.com/fabpot", - "type": "github" - }, - { - "url": "https://tidelift.com/funding/github/packagist/symfony/symfony", - "type": "tidelift" - } - ], - "time": "2024-09-25T14:21:43+00:00" - }, - { - "name": "symfony/deprecation-contracts", - "version": "v3.6.0", - "source": { - "type": "git", - "url": "https://github.com/symfony/deprecation-contracts.git", - "reference": "63afe740e99a13ba87ec199bb07bbdee937a5b62" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/symfony/deprecation-contracts/zipball/63afe740e99a13ba87ec199bb07bbdee937a5b62", - "reference": "63afe740e99a13ba87ec199bb07bbdee937a5b62", - "shasum": "" - }, - "require": { - "php": ">=8.1" - }, - "type": "library", - "extra": { - "thanks": { - "url": "https://github.com/symfony/contracts", - "name": "symfony/contracts" - }, - "branch-alias": { - "dev-main": "3.6-dev" - } - }, - "autoload": { - "files": [ - "function.php" - ] - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Nicolas Grekas", - "email": "p@tchwork.com" - }, - { - "name": "Symfony Community", - "homepage": "https://symfony.com/contributors" - } - ], - "description": "A generic function and convention to trigger deprecation notices", - "homepage": "https://symfony.com", - "support": { - "source": "https://github.com/symfony/deprecation-contracts/tree/v3.6.0" - }, - "funding": [ - { - "url": "https://symfony.com/sponsor", - "type": "custom" - }, - { - "url": "https://github.com/fabpot", - "type": "github" - }, - { - "url": "https://tidelift.com/funding/github/packagist/symfony/symfony", - "type": "tidelift" - } - ], - "time": "2024-09-25T14:21:43+00:00" - }, - { - "name": "symfony/polyfill-ctype", - "version": "v1.33.0", - "source": { - "type": "git", - "url": "https://github.com/symfony/polyfill-ctype.git", - "reference": "a3cc8b044a6ea513310cbd48ef7333b384945638" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/symfony/polyfill-ctype/zipball/a3cc8b044a6ea513310cbd48ef7333b384945638", - "reference": "a3cc8b044a6ea513310cbd48ef7333b384945638", - "shasum": "" - }, - "require": { - "php": ">=7.2" - }, - "provide": { - "ext-ctype": "*" - }, - "suggest": { - "ext-ctype": "For best performance" - }, - "type": "library", - "extra": { - "thanks": { - "url": "https://github.com/symfony/polyfill", - "name": "symfony/polyfill" - } - }, - "autoload": { - "files": [ - "bootstrap.php" - ], - "psr-4": { - "Symfony\\Polyfill\\Ctype\\": "" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Gert de Pagter", - "email": "BackEndTea@gmail.com" - }, - { - "name": "Symfony Community", - "homepage": "https://symfony.com/contributors" - } - ], - "description": "Symfony polyfill for ctype functions", - "homepage": "https://symfony.com", - "keywords": [ - "compatibility", - "ctype", - "polyfill", - "portable" - ], - "support": { - "source": "https://github.com/symfony/polyfill-ctype/tree/v1.33.0" - }, - "funding": [ - { - "url": "https://symfony.com/sponsor", - "type": "custom" - }, - { - "url": "https://github.com/fabpot", - "type": "github" - }, - { - "url": "https://github.com/nicolas-grekas", - "type": "github" - }, - { - "url": "https://tidelift.com/funding/github/packagist/symfony/symfony", - "type": "tidelift" - } - ], - "time": "2024-09-09T11:45:10+00:00" - }, - { - "name": "symfony/polyfill-mbstring", - "version": "v1.33.0", - "source": { - "type": "git", - "url": "https://github.com/symfony/polyfill-mbstring.git", - "reference": "6d857f4d76bd4b343eac26d6b539585d2bc56493" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/symfony/polyfill-mbstring/zipball/6d857f4d76bd4b343eac26d6b539585d2bc56493", - "reference": "6d857f4d76bd4b343eac26d6b539585d2bc56493", - "shasum": "" - }, - "require": { - "ext-iconv": "*", - "php": ">=7.2" - }, - "provide": { - "ext-mbstring": "*" - }, - "suggest": { - "ext-mbstring": "For best performance" - }, - "type": "library", - "extra": { - "thanks": { - "url": "https://github.com/symfony/polyfill", - "name": "symfony/polyfill" - } - }, - "autoload": { - "files": [ - "bootstrap.php" - ], - "psr-4": { - "Symfony\\Polyfill\\Mbstring\\": "" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Nicolas Grekas", - "email": "p@tchwork.com" - }, - { - "name": "Symfony Community", - "homepage": "https://symfony.com/contributors" - } - ], - "description": "Symfony polyfill for the Mbstring extension", - "homepage": "https://symfony.com", - "keywords": [ - "compatibility", - "mbstring", - "polyfill", - "portable", - "shim" - ], - "support": { - "source": "https://github.com/symfony/polyfill-mbstring/tree/v1.33.0" - }, - "funding": [ - { - "url": "https://symfony.com/sponsor", - "type": "custom" - }, - { - "url": "https://github.com/fabpot", - "type": "github" - }, - { - "url": "https://github.com/nicolas-grekas", - "type": "github" - }, - { - "url": "https://tidelift.com/funding/github/packagist/symfony/symfony", - "type": "tidelift" - } - ], - "time": "2024-12-23T08:48:59+00:00" - }, - { - "name": "symfony/polyfill-php80", - "version": "v1.33.0", - "source": { - "type": "git", - "url": "https://github.com/symfony/polyfill-php80.git", - "reference": "0cc9dd0f17f61d8131e7df6b84bd344899fe2608" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/symfony/polyfill-php80/zipball/0cc9dd0f17f61d8131e7df6b84bd344899fe2608", - "reference": "0cc9dd0f17f61d8131e7df6b84bd344899fe2608", - "shasum": "" - }, - "require": { - "php": ">=7.2" - }, - "type": "library", - "extra": { - "thanks": { - "url": "https://github.com/symfony/polyfill", - "name": "symfony/polyfill" - } - }, - "autoload": { - "files": [ - "bootstrap.php" - ], - "psr-4": { - "Symfony\\Polyfill\\Php80\\": "" - }, - "classmap": [ - "Resources/stubs" - ] - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Ion Bazan", - "email": "ion.bazan@gmail.com" - }, - { - "name": "Nicolas Grekas", - "email": "p@tchwork.com" - }, - { - "name": "Symfony Community", - "homepage": "https://symfony.com/contributors" - } - ], - "description": "Symfony polyfill backporting some PHP 8.0+ features to lower PHP versions", - "homepage": "https://symfony.com", - "keywords": [ - "compatibility", - "polyfill", - "portable", - "shim" - ], - "support": { - "source": "https://github.com/symfony/polyfill-php80/tree/v1.33.0" - }, - "funding": [ - { - "url": "https://symfony.com/sponsor", - "type": "custom" - }, - { - "url": "https://github.com/fabpot", - "type": "github" - }, - { - "url": "https://github.com/nicolas-grekas", - "type": "github" - }, - { - "url": "https://tidelift.com/funding/github/packagist/symfony/symfony", - "type": "tidelift" - } - ], - "time": "2025-01-02T08:10:11+00:00" - }, - { - "name": "symfony/polyfill-php81", - "version": "v1.33.0", - "source": { - "type": "git", - "url": "https://github.com/symfony/polyfill-php81.git", - "reference": "4a4cfc2d253c21a5ad0e53071df248ed48c6ce5c" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/symfony/polyfill-php81/zipball/4a4cfc2d253c21a5ad0e53071df248ed48c6ce5c", - "reference": "4a4cfc2d253c21a5ad0e53071df248ed48c6ce5c", - "shasum": "" - }, - "require": { - "php": ">=7.2" - }, - "type": "library", - "extra": { - "thanks": { - "url": "https://github.com/symfony/polyfill", - "name": "symfony/polyfill" - } - }, - "autoload": { - "files": [ - "bootstrap.php" - ], - "psr-4": { - "Symfony\\Polyfill\\Php81\\": "" - }, - "classmap": [ - "Resources/stubs" - ] - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Nicolas Grekas", - "email": "p@tchwork.com" - }, - { - "name": "Symfony Community", - "homepage": "https://symfony.com/contributors" - } - ], - "description": "Symfony polyfill backporting some PHP 8.1+ features to lower PHP versions", - "homepage": "https://symfony.com", - "keywords": [ - "compatibility", - "polyfill", - "portable", - "shim" - ], - "support": { - "source": "https://github.com/symfony/polyfill-php81/tree/v1.33.0" - }, - "funding": [ - { - "url": "https://symfony.com/sponsor", - "type": "custom" - }, - { - "url": "https://github.com/fabpot", - "type": "github" - }, - { - "url": "https://github.com/nicolas-grekas", - "type": "github" - }, - { - "url": "https://tidelift.com/funding/github/packagist/symfony/symfony", - "type": "tidelift" - } - ], - "time": "2024-09-09T11:45:10+00:00" - }, - { - "name": "symfony/polyfill-php83", - "version": "v1.33.0", - "source": { - "type": "git", - "url": "https://github.com/symfony/polyfill-php83.git", - "reference": "17f6f9a6b1735c0f163024d959f700cfbc5155e5" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/symfony/polyfill-php83/zipball/17f6f9a6b1735c0f163024d959f700cfbc5155e5", - "reference": "17f6f9a6b1735c0f163024d959f700cfbc5155e5", - "shasum": "" - }, - "require": { - "php": ">=7.2" - }, - "type": "library", - "extra": { - "thanks": { - "url": "https://github.com/symfony/polyfill", - "name": "symfony/polyfill" - } - }, - "autoload": { - "files": [ - "bootstrap.php" - ], - "psr-4": { - "Symfony\\Polyfill\\Php83\\": "" - }, - "classmap": [ - "Resources/stubs" - ] - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Nicolas Grekas", - "email": "p@tchwork.com" - }, - { - "name": "Symfony Community", - "homepage": "https://symfony.com/contributors" - } - ], - "description": "Symfony polyfill backporting some PHP 8.3+ features to lower PHP versions", - "homepage": "https://symfony.com", - "keywords": [ - "compatibility", - "polyfill", - "portable", - "shim" - ], - "support": { - "source": "https://github.com/symfony/polyfill-php83/tree/v1.33.0" - }, - "funding": [ - { - "url": "https://symfony.com/sponsor", - "type": "custom" - }, - { - "url": "https://github.com/fabpot", - "type": "github" - }, - { - "url": "https://github.com/nicolas-grekas", - "type": "github" - }, - { - "url": "https://tidelift.com/funding/github/packagist/symfony/symfony", - "type": "tidelift" - } - ], - "time": "2025-07-08T02:45:35+00:00" - }, - { - "name": "symfony/translation", - "version": "v7.3.4", - "source": { - "type": "git", - "url": "https://github.com/symfony/translation.git", - "reference": "ec25870502d0c7072d086e8ffba1420c85965174" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/symfony/translation/zipball/ec25870502d0c7072d086e8ffba1420c85965174", - "reference": "ec25870502d0c7072d086e8ffba1420c85965174", - "shasum": "" - }, - "require": { - "php": ">=8.2", - "symfony/deprecation-contracts": "^2.5|^3", - "symfony/polyfill-mbstring": "~1.0", - "symfony/translation-contracts": "^2.5|^3.0" - }, - "conflict": { - "nikic/php-parser": "<5.0", - "symfony/config": "<6.4", - "symfony/console": "<6.4", - "symfony/dependency-injection": "<6.4", - "symfony/http-client-contracts": "<2.5", - "symfony/http-kernel": "<6.4", - "symfony/service-contracts": "<2.5", - "symfony/twig-bundle": "<6.4", - "symfony/yaml": "<6.4" - }, - "provide": { - "symfony/translation-implementation": "2.3|3.0" - }, - "require-dev": { - "nikic/php-parser": "^5.0", - "psr/log": "^1|^2|^3", - "symfony/config": "^6.4|^7.0", - "symfony/console": "^6.4|^7.0", - "symfony/dependency-injection": "^6.4|^7.0", - "symfony/finder": "^6.4|^7.0", - "symfony/http-client-contracts": "^2.5|^3.0", - "symfony/http-kernel": "^6.4|^7.0", - "symfony/intl": "^6.4|^7.0", - "symfony/polyfill-intl-icu": "^1.21", - "symfony/routing": "^6.4|^7.0", - "symfony/service-contracts": "^2.5|^3", - "symfony/yaml": "^6.4|^7.0" - }, - "type": "library", - "autoload": { - "files": [ - "Resources/functions.php" - ], - "psr-4": { - "Symfony\\Component\\Translation\\": "" - }, - "exclude-from-classmap": [ - "/Tests/" - ] - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Fabien Potencier", - "email": "fabien@symfony.com" - }, - { - "name": "Symfony Community", - "homepage": "https://symfony.com/contributors" - } - ], - "description": "Provides tools to internationalize your application", - "homepage": "https://symfony.com", - "support": { - "source": "https://github.com/symfony/translation/tree/v7.3.4" - }, - "funding": [ - { - "url": "https://symfony.com/sponsor", - "type": "custom" - }, - { - "url": "https://github.com/fabpot", - "type": "github" - }, - { - "url": "https://github.com/nicolas-grekas", - "type": "github" - }, - { - "url": "https://tidelift.com/funding/github/packagist/symfony/symfony", - "type": "tidelift" - } - ], - "time": "2025-09-07T11:39:36+00:00" - }, - { - "name": "symfony/translation-contracts", - "version": "v3.6.0", - "source": { - "type": "git", - "url": "https://github.com/symfony/translation-contracts.git", - "reference": "df210c7a2573f1913b2d17cc95f90f53a73d8f7d" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/symfony/translation-contracts/zipball/df210c7a2573f1913b2d17cc95f90f53a73d8f7d", - "reference": "df210c7a2573f1913b2d17cc95f90f53a73d8f7d", - "shasum": "" - }, - "require": { - "php": ">=8.1" - }, - "type": "library", - "extra": { - "thanks": { - "url": "https://github.com/symfony/contracts", - "name": "symfony/contracts" - }, - "branch-alias": { - "dev-main": "3.6-dev" - } - }, - "autoload": { - "psr-4": { - "Symfony\\Contracts\\Translation\\": "" - }, - "exclude-from-classmap": [ - "/Test/" - ] - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Nicolas Grekas", - "email": "p@tchwork.com" - }, - { - "name": "Symfony Community", - "homepage": "https://symfony.com/contributors" - } - ], - "description": "Generic abstractions related to translation", - "homepage": "https://symfony.com", - "keywords": [ - "abstractions", - "contracts", - "decoupling", - "interfaces", - "interoperability", - "standards" - ], - "support": { - "source": "https://github.com/symfony/translation-contracts/tree/v3.6.0" - }, - "funding": [ - { - "url": "https://symfony.com/sponsor", - "type": "custom" - }, - { - "url": "https://github.com/fabpot", - "type": "github" - }, - { - "url": "https://tidelift.com/funding/github/packagist/symfony/symfony", - "type": "tidelift" - } - ], - "time": "2024-09-27T08:32:26+00:00" - }, - { - "name": "vlucas/phpdotenv", - "version": "v5.6.2", - "source": { - "type": "git", - "url": "https://github.com/vlucas/phpdotenv.git", - "reference": "24ac4c74f91ee2c193fa1aaa5c249cb0822809af" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/vlucas/phpdotenv/zipball/24ac4c74f91ee2c193fa1aaa5c249cb0822809af", - "reference": "24ac4c74f91ee2c193fa1aaa5c249cb0822809af", - "shasum": "" - }, - "require": { - "ext-pcre": "*", - "graham-campbell/result-type": "^1.1.3", - "php": "^7.2.5 || ^8.0", - "phpoption/phpoption": "^1.9.3", - "symfony/polyfill-ctype": "^1.24", - "symfony/polyfill-mbstring": "^1.24", - "symfony/polyfill-php80": "^1.24" - }, - "require-dev": { - "bamarni/composer-bin-plugin": "^1.8.2", - "ext-filter": "*", - "phpunit/phpunit": "^8.5.34 || ^9.6.13 || ^10.4.2" - }, - "suggest": { - "ext-filter": "Required to use the boolean validator." - }, - "type": "library", - "extra": { - "bamarni-bin": { - "bin-links": true, - "forward-command": false - }, - "branch-alias": { - "dev-master": "5.6-dev" - } - }, - "autoload": { - "psr-4": { - "Dotenv\\": "src/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "BSD-3-Clause" - ], - "authors": [ - { - "name": "Graham Campbell", - "email": "hello@gjcampbell.co.uk", - "homepage": "https://github.com/GrahamCampbell" - }, - { - "name": "Vance Lucas", - "email": "vance@vancelucas.com", - "homepage": "https://github.com/vlucas" - } - ], - "description": "Loads environment variables from `.env` to `getenv()`, `$_ENV` and `$_SERVER` automagically.", - "keywords": [ - "dotenv", - "env", - "environment" - ], - "support": { - "issues": "https://github.com/vlucas/phpdotenv/issues", - "source": "https://github.com/vlucas/phpdotenv/tree/v5.6.2" - }, - "funding": [ - { - "url": "https://github.com/GrahamCampbell", - "type": "github" - }, - { - "url": "https://tidelift.com/funding/github/packagist/vlucas/phpdotenv", - "type": "tidelift" - } - ], - "time": "2025-04-30T23:37:27+00:00" - }, - { - "name": "voku/portable-ascii", - "version": "2.0.3", - "source": { - "type": "git", - "url": "https://github.com/voku/portable-ascii.git", - "reference": "b1d923f88091c6bf09699efcd7c8a1b1bfd7351d" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/voku/portable-ascii/zipball/b1d923f88091c6bf09699efcd7c8a1b1bfd7351d", - "reference": "b1d923f88091c6bf09699efcd7c8a1b1bfd7351d", - "shasum": "" - }, - "require": { - "php": ">=7.0.0" - }, - "require-dev": { - "phpunit/phpunit": "~6.0 || ~7.0 || ~9.0" - }, - "suggest": { - "ext-intl": "Use Intl for transliterator_transliterate() support" - }, - "type": "library", - "autoload": { - "psr-4": { - "voku\\": "src/voku/" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "Lars Moelleken", - "homepage": "https://www.moelleken.org/" - } - ], - "description": "Portable ASCII library - performance optimized (ascii) string functions for php.", - "homepage": "https://github.com/voku/portable-ascii", - "keywords": [ - "ascii", - "clean", - "php" - ], - "support": { - "issues": "https://github.com/voku/portable-ascii/issues", - "source": "https://github.com/voku/portable-ascii/tree/2.0.3" - }, - "funding": [ - { - "url": "https://www.paypal.me/moelleken", - "type": "custom" - }, - { - "url": "https://github.com/voku", - "type": "github" - }, - { - "url": "https://opencollective.com/portable-ascii", - "type": "open_collective" - }, - { - "url": "https://www.patreon.com/voku", - "type": "patreon" - }, - { - "url": "https://tidelift.com/funding/github/packagist/voku/portable-ascii", - "type": "tidelift" - } - ], - "time": "2024-11-21T01:49:47+00:00" - }, - { - "name": "workerman/coroutine", - "version": "v1.1.4", - "source": { - "type": "git", - "url": "https://github.com/workerman-php/coroutine.git", - "reference": "b0bebfa9d41b992ad0a835ddf2ee8fa5d58eca44" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/workerman-php/coroutine/zipball/b0bebfa9d41b992ad0a835ddf2ee8fa5d58eca44", - "reference": "b0bebfa9d41b992ad0a835ddf2ee8fa5d58eca44", - "shasum": "" - }, - "require": { - "php": ">=8.1" - }, - "require-dev": { - "phpunit/phpunit": "^11.0", - "psr/log": "*" - }, - "type": "library", - "autoload": { - "psr-4": { - "Workerman\\": "src", - "Workerman\\Coroutine\\": "src" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "description": "Workerman coroutine", - "support": { - "issues": "https://github.com/workerman-php/coroutine/issues", - "source": "https://github.com/workerman-php/coroutine/tree/v1.1.4" - }, - "time": "2025-10-11T15:09:08+00:00" - }, - { - "name": "workerman/webman-framework", - "version": "v2.1.2", - "source": { - "type": "git", - "url": "https://github.com/walkor/webman-framework.git", - "reference": "f803bd867f07bb0929faef060b59a19a44186bfc" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/walkor/webman-framework/zipball/f803bd867f07bb0929faef060b59a19a44186bfc", - "reference": "f803bd867f07bb0929faef060b59a19a44186bfc", - "shasum": "" - }, - "require": { - "ext-json": "*", - "nikic/fast-route": "^1.3", - "php": ">=8.1", - "psr/container": ">=1.0", - "psr/log": "^3.0", - "workerman/workerman": "^5.1 || dev-master" - }, - "suggest": { - "ext-event": "For better performance. " - }, - "type": "library", - "autoload": { - "files": [ - "./src/support/helpers.php" - ], - "psr-4": { - "Webman\\": "./src", - "Support\\": "./src/support", - "support\\": "./src/support", - "Support\\View\\": "./src/support/view", - "Support\\Bootstrap\\": "./src/support/bootstrap", - "Support\\Exception\\": "./src/support/exception" - } - }, - "notification-url": "https://packagist.org/downloads/", - "license": [ - "MIT" - ], - "authors": [ - { - "name": "walkor", - "email": "walkor@workerman.net", - "homepage": "https://www.workerman.net", - "role": "Developer" - } - ], - "description": "High performance HTTP Service Framework.", - "homepage": "https://www.workerman.net", - "keywords": [ - "High Performance", - "http service" - ], - "support": { - "email": "walkor@workerman.net", - "forum": "https://wenda.workerman.net/", - "issues": "https://github.com/walkor/webman/issues", - "source": "https://github.com/walkor/webman-framework", - "wiki": "https://doc.workerman.net/" - }, - "time": "2025-03-10T11:52:22+00:00" - }, - { - "name": "workerman/workerman", - "version": "v5.1.4", - "source": { - "type": "git", - "url": "https://github.com/walkor/workerman.git", - "reference": "ff4e17babdc92b16b3252060233c88f6c2e9a61a" - }, - "dist": { - "type": "zip", - "url": "https://api.github.com/repos/walkor/workerman/zipball/ff4e17babdc92b16b3252060233c88f6c2e9a61a", - "reference": "ff4e17babdc92b16b3252060233c88f6c2e9a61a", - "shasum": "" - }, - "require": { - "ext-json": "*", - "php": ">=8.1", - "workerman/coroutine": "^1.1 || dev-main" - }, - "conflict": { - "ext-swow": "=8.1" - }, - "platform-dev": {}, - "plugin-api-version": "2.6.0" -} diff --git a/Moncter/Moncter/config/app.php b/Moncter/Moncter/config/app.php deleted file mode 100644 index f26e3584..00000000 --- a/Moncter/Moncter/config/app.php +++ /dev/null @@ -1,26 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -use support\Request; - -return [ - 'debug' => true, - 'error_reporting' => E_ALL, - 'default_timezone' => 'Asia/Shanghai', - 'request_class' => Request::class, - 'public_path' => base_path() . DIRECTORY_SEPARATOR . 'public', - 'runtime_path' => base_path(false) . DIRECTORY_SEPARATOR . 'runtime', - 'controller_suffix' => 'Controller', - 'controller_reuse' => false, -]; diff --git a/Moncter/Moncter/config/autoload.php b/Moncter/Moncter/config/autoload.php deleted file mode 100644 index 8d207b5c..00000000 --- a/Moncter/Moncter/config/autoload.php +++ /dev/null @@ -1,20 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -return [ - 'files' => [ - base_path() . '/support/Request.php', - base_path() . '/support/Response.php', - ] -]; diff --git a/Moncter/Moncter/config/bootstrap.php b/Moncter/Moncter/config/bootstrap.php deleted file mode 100644 index 63e07bd0..00000000 --- a/Moncter/Moncter/config/bootstrap.php +++ /dev/null @@ -1,18 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -return [ - support\bootstrap\Session::class, - support\bootstrap\MongoDB::class, -]; diff --git a/Moncter/Moncter/config/container.php b/Moncter/Moncter/config/container.php deleted file mode 100644 index 106b7b4a..00000000 --- a/Moncter/Moncter/config/container.php +++ /dev/null @@ -1,15 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -return new Webman\Container; \ No newline at end of file diff --git a/Moncter/Moncter/config/data_collection_tasks.php b/Moncter/Moncter/config/data_collection_tasks.php deleted file mode 100644 index c0798226..00000000 --- a/Moncter/Moncter/config/data_collection_tasks.php +++ /dev/null @@ -1,74 +0,0 @@ - [ - // 分布式锁配置 - 'distributed_lock' => [ - 'driver' => 'redis', - 'ttl' => 300, - 'retry_times' => 3, - 'retry_delay' => 1000, - ], - // 错误处理配置 - 'error_handling' => [ - 'max_retries' => 3, - 'retry_delay' => 5, - 'circuit_breaker' => [ - 'enabled' => true, - 'failure_threshold' => 10, - 'recovery_timeout' => 60, - ], - ], - ], - - // 采集任务列表（配置文件中的任务） - 'tasks' => [ - // 数据库同步任务（实时同步源数据库到目标数据库） - // 注意：这是一个系统级任务，通过配置文件管理，不从数据库加载 - 'database_sync' => [ - 'name' => '数据库实时同步', - 'enabled' => false, - - // 源数据库（从 data_sources.php 引用） - 'source_data_source' => 'kr_mongodb', - - // 目标数据库（从 data_sources.php 引用） - 'target_data_source' => 'sync_mongodb', - - // 业务处理类 - 'handler_class' => \app\service\DataCollection\Handler\DatabaseSyncHandler::class, - - // 调度配置（数据库同步是持续运行的，不需要定时调度） - 'schedule' => [ - 'cron' => null, // 不需要 Cron，启动后持续运行 - 'enabled' => false, // 禁用定时调度，启动后持续运行 - ], - - // 分片配置（每个 Worker 可以监听不同的数据库） - 'sharding' => [ - 'strategy' => 'by_database', // 按数据库分片 - 'shard_count' => 1, - ], - - // 同步状态存储配置 - 'sync_state' => [ - 'storage' => 'file', // 使用文件存储进度（DatabaseSyncService 使用文件） - 'key_prefix' => 'database_sync:', - ], - - // 注意：业务逻辑相关配置（数据库列表、排除规则、性能配置等） - // 已移到 DatabaseSyncHandler 类中，使用默认值或从独立配置读取 - ], - ], -]; - diff --git a/Moncter/Moncter/config/data_sources.php b/Moncter/Moncter/config/data_sources.php deleted file mode 100644 index 83b9ac73..00000000 --- a/Moncter/Moncter/config/data_sources.php +++ /dev/null @@ -1,63 +0,0 @@ - [ - 'type' => 'mongodb', - 'host' => getenv('KR_MONGODB_HOST'), - 'port' => (int)getenv('KR_MONGODB_PORT'), - 'database' => getenv('KR_MONGODB_DATABASE'), - 'username' => getenv('KR_MONGODB_USER'), - 'password' => getenv('KR_MONGODB_PASSWORD'), - 'auth_source' => getenv('KR_MONGODB_AUTH_SOURCE'), - 'options' => [ - 'ssl' => false, - 'connectTimeoutMS' => 3000, - 'socketTimeoutMS' => 5000, - 'authMechanism' => 'SCRAM-SHA-256', - ], - ], - - // 标签数据库（主机标签数据库） - 'tag_mongodb' => [ - 'type' => 'mongodb', - 'host' => getenv('TAG_MONGODB_HOST') ?: '192.168.1.106', - 'port' => (int)(getenv('TAG_MONGODB_PORT') ?: 27017), - 'database' => getenv('TAG_MONGODB_DATABASE') ?: 'ckb', - 'username' => getenv('TAG_MONGODB_USER') ?: 'ckb', - 'password' => getenv('TAG_MONGODB_PASSWORD') ?: '123456', - 'auth_source' => getenv('TAG_MONGODB_AUTH') ?: 'ckb', - 'options' => [ - 'ssl' => false, - 'connectTimeoutMS' => 3000, - 'socketTimeoutMS' => 5000, - 'authMechanism' => 'SCRAM-SHA-256', - ], - ], - - // 同步目标数据库（主机同步KR数据库） - 'sync_mongodb' => [ - 'type' => 'mongodb', - 'host' => getenv('SYNC_MONGODB_HOST'), - 'port' => (int)getenv('SYNC_MONGODB_PORT'), - 'database' => getenv('SYNC_MONGODB_DATABASE') ?: 'KR', - 'username' => getenv('SYNC_MONGODB_USER'), - 'password' => getenv('SYNC_MONGODB_PASS'), - 'auth_source' => getenv('SYNC_MONGODB_AUTH'), - 'options' => [ - 'ssl' => false, - 'connectTimeoutMS' => 3000, - 'socketTimeoutMS' => 5000, - 'authMechanism' => 'SCRAM-SHA-256', - ], - ], - - -]; diff --git a/Moncter/Moncter/config/database.php b/Moncter/Moncter/config/database.php deleted file mode 100644 index 630702e9..00000000 --- a/Moncter/Moncter/config/database.php +++ /dev/null @@ -1,27 +0,0 @@ - 'mysql', // 若需全局用 MongoDB，改为 'mongodb' - - 'connections' => [ - // ... 其他连接（如 mysql）保持不变 - - // MongoDB 官方连接配置 - 'mongodb' => [ - 'driver' => 'mongodb', - 'dsn' => 'mongodb://192.168.1.106:27017', // 集群可写：mongodb://node1:27017,node2:27017 - 'database' => 'ckb', // 目标数据库名 - 'username' => 'ckb', // 无认证则省略 - 'password' => '123456', // 无认证则省略 - 'options' => [ - // 'replicaSet' => '', // 副本集名称（如果使用副本集，取消注释并填写名称） - 'ssl' => false, // 是否启用 SSL - 'connectTimeoutMS' => 3000, // 连接超时 - 'socketTimeoutMS' => 5000, // 读写超时 - // 认证相关（若 MongoDB 启用认证） - 'authSource' => 'ckb', // 认证数据库（默认 admin） - 'authMechanism' => 'SCRAM-SHA-256', // 认证机制（默认推荐） - ], - ], - ], -]; \ No newline at end of file diff --git a/Moncter/Moncter/config/dependence.php b/Moncter/Moncter/config/dependence.php deleted file mode 100644 index 8e964eda..00000000 --- a/Moncter/Moncter/config/dependence.php +++ /dev/null @@ -1,15 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -return []; \ No newline at end of file diff --git a/Moncter/Moncter/config/encryption.php b/Moncter/Moncter/config/encryption.php deleted file mode 100644 index cac99334..00000000 --- a/Moncter/Moncter/config/encryption.php +++ /dev/null @@ -1,60 +0,0 @@ - [ - // 加密密钥（32字节，256位） - // 注意：生产环境应使用环境变量或密钥管理服务，不要硬编码 - // 使用 getenv() 获取环境变量，如果不存在则使用默认值 - // 默认密钥：至少32字符（实际使用时会被 SHA256 哈希处理） - 'key' => getenv('ENCRYPTION_AES_KEY') ?: 'your-32-byte-secret-key-here-12345678', - - // 加密方法 - 'cipher' => 'AES-256-CBC', - - // IV 长度（字节） - 'iv_length' => 16, - ], - - // 哈希配置 - 'hash' => [ - // 哈希算法（用于身份证哈希） - 'algorithm' => 'sha256', - - // 是否使用盐值（可选，增强安全性） - 'use_salt' => true, - - // 盐值（如果启用） - // 使用 getenv() 获取环境变量，如果不存在则使用默认值 - 'salt' => getenv('ENCRYPTION_HASH_SALT') ?: 'your-hash-salt-here', - ], - - // 脱敏配置 - 'masking' => [ - // 身份证脱敏规则：保留前6位和后4位，中间用*替代 - 'id_card' => [ - 'prefix_length' => 6, - 'suffix_length' => 4, - 'mask_char' => '*', - ], - - // 手机号脱敏规则：保留前3位和后4位，中间用*替代 - 'phone' => [ - 'prefix_length' => 3, - 'suffix_length' => 4, - 'mask_char' => '*', - ], - - // 邮箱脱敏规则：保留@前的前2位和@后的域名 - 'email' => [ - 'prefix_length' => 2, - 'mask_char' => '*', - ], - ], -]; - diff --git a/Moncter/Moncter/config/exception.php b/Moncter/Moncter/config/exception.php deleted file mode 100644 index f2aede33..00000000 --- a/Moncter/Moncter/config/exception.php +++ /dev/null @@ -1,17 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -return [ - '' => support\exception\Handler::class, -]; \ No newline at end of file diff --git a/Moncter/Moncter/config/log.php b/Moncter/Moncter/config/log.php deleted file mode 100644 index 817e13cf..00000000 --- a/Moncter/Moncter/config/log.php +++ /dev/null @@ -1,37 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -return [ - 'default' => [ - 'handlers' => [ - [ - 'class' => Monolog\Handler\RotatingFileHandler::class, - 'constructor' => [ - runtime_path() . '/logs/webman.log', - 7, //$maxFiles - Monolog\Logger::DEBUG, - ], - 'formatter' => [ - 'class' => Monolog\Formatter\LineFormatter::class, - 'constructor' => [null, 'Y-m-d H:i:s', true], - ], - ] - ], - 'processors' => [ - [ - 'class' => app\utils\LogMaskingProcessor::class, - ], - ], - ], -]; diff --git a/Moncter/Moncter/config/middleware.php b/Moncter/Moncter/config/middleware.php deleted file mode 100644 index 8e964eda..00000000 --- a/Moncter/Moncter/config/middleware.php +++ /dev/null @@ -1,15 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -return []; \ No newline at end of file diff --git a/Moncter/Moncter/config/process.php b/Moncter/Moncter/config/process.php deleted file mode 100644 index f987b577..00000000 --- a/Moncter/Moncter/config/process.php +++ /dev/null @@ -1,83 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -use support\Log; -use support\Request; -use app\process\Http; - -global $argv; - -return [ - 'webman' => [ - 'handler' => Http::class, - 'listen' => 'http://0.0.0.0:8787', - 'count' => cpu_count() * 4, - 'user' => '', - 'group' => '', - 'reusePort' => false, - 'eventLoop' => '', - 'context' => [], - 'constructor' => [ - 'requestClass' => Request::class, - 'logger' => Log::channel('default'), - 'appPath' => app_path(), - 'publicPath' => public_path() - ] - ], - // File update detection and automatic reload - 'monitor' => [ - 'handler' => app\process\Monitor::class, - 'reloadable' => false, - 'constructor' => [ - // Monitor these directories - 'monitorDir' => array_merge([ - app_path(), - config_path(), - base_path() . '/process', - base_path() . '/support', - base_path() . '/resource', - base_path() . '/.env', - ], glob(base_path() . '/plugin/*/app'), glob(base_path() . '/plugin/*/config'), glob(base_path() . '/plugin/*/api')), - // Files with these suffixes will be monitored - 'monitorExtensions' => [ - 'php', 'html', 'htm', 'env' - ], - 'options' => [ - 'enable_file_monitor' => !in_array('-d', $argv) && DIRECTORY_SEPARATOR === '/', - 'enable_memory_monitor' => DIRECTORY_SEPARATOR === '/', - ] - ] - ], - // 数据采集任务调度器（从 config/data_collection_tasks.php 读取所有采集任务配置） - 'data_sync_scheduler' => [ - 'handler' => app\process\DataSyncScheduler::class, - 'count' => 10, // Worker 进程数量（可根据任务数量调整） - 'reloadable' => false, - ], - // 数据同步 Worker（消费 RabbitMQ 消息队列） - // 处理从采集任务推送过来的数据，写入目标数据库 - 'data_sync_worker' => [ - 'handler' => app\process\DataSyncWorker::class, - 'count' => 20, // Worker 进程数量（可根据消息量调整） - 'reloadable' => false, - ], - // 标签计算 Worker（消费 RabbitMQ 消息队列） - // 根据用户数据计算标签值 - 'tag_calculation_worker' => [ - 'handler' => app\process\TagCalculationWorker::class, - 'count' => 2, // Worker 进程数量（可根据消息量调整） - 'reloadable' => false, - ], - -]; diff --git a/Moncter/Moncter/config/queue.php b/Moncter/Moncter/config/queue.php deleted file mode 100644 index 82ac9f40..00000000 --- a/Moncter/Moncter/config/queue.php +++ /dev/null @@ -1,81 +0,0 @@ - 'rabbitmq', - - 'connections' => [ - 'rabbitmq' => [ - 'driver' => 'rabbitmq', - 'host' => getenv('RABBITMQ_HOST') ?: '127.0.0.1', - 'port' => (int)(getenv('RABBITMQ_PORT') ?: 5672), - 'user' => getenv('RABBITMQ_USER') ?: 'guest', - 'password' => getenv('RABBITMQ_PASSWORD') ?: 'guest', - 'vhost' => getenv('RABBITMQ_VHOST') ?: '/', - 'timeout' => 10, // 连接超时时间（秒） - - // 队列配置 - 'queues' => [ - // 数据同步队列：外部数据源轮询后推送的数据 - 'data_sync' => [ - 'name' => 'data_sync_queue', - 'durable' => true, // 队列持久化 - 'auto_delete' => false, - 'arguments' => [], - ], - // 标签计算队列：消费记录写入后触发标签计算 - 'tag_calculation' => [ - 'name' => 'tag_calculation_queue', - 'durable' => true, // 队列持久化 - 'auto_delete' => false, - 'arguments' => [], - ], - ], - - // 交换机配置 - 'exchanges' => [ - 'data_sync' => [ - 'name' => 'data_sync_exchange', - 'type' => 'direct', - 'durable' => true, - 'auto_delete' => false, - ], - 'tag_calculation' => [ - 'name' => 'tag_calculation_exchange', - 'type' => 'direct', - 'durable' => true, - 'auto_delete' => false, - ], - ], - - // 路由键配置 - 'routing_keys' => [ - 'data_sync' => 'data.sync', - 'tag_calculation' => 'tag.calculation', - ], - ], - ], - - // 消息配置 - 'message' => [ - 'delivery_mode' => 2, // 消息持久化（2 = 持久化） - 'content_type' => 'application/json', - ], - - // 消费者配置 - 'consumer' => [ - 'data_sync' => [ - 'prefetch_count' => 10, // 每次处理10条消息（批量处理） - 'no_ack' => false, // 需要确认消息 - ], - 'tag_calculation' => [ - 'prefetch_count' => 1, // 每次只处理一条消息 - 'no_ack' => false, // 需要确认消息 - ], - ], -]; - diff --git a/Moncter/Moncter/config/route.php b/Moncter/Moncter/config/route.php deleted file mode 100644 index 3a66a3ec..00000000 --- a/Moncter/Moncter/config/route.php +++ /dev/null @@ -1,120 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -use Webman\Route; - - -// 数据库连接测试接口 -Route::get('/api/test/db', [app\controller\IndexController::class, 'testDb']); - -// ============================================ -// 用户相关接口（RESTful） -// ============================================ -Route::post('/api/users', [app\controller\UserController::class, 'store']); // 创建用户 -Route::get('/api/users/{user_id}', [app\controller\UserController::class, 'show']); // 查询用户 -Route::put('/api/users/{user_id}', [app\controller\UserController::class, 'update']); // 更新用户 -Route::delete('/api/users/{user_id}', [app\controller\UserController::class, 'destroy']); // 删除用户 -Route::get('/api/users/{user_id}/decrypt-id-card', [app\controller\UserController::class, 'decryptIdCard']); // 解密身份证 -Route::post('/api/users/search', [app\controller\UserController::class, 'search']); // 搜索用户（复杂查询） - -// ============================================ -// 用户标签相关接口（RESTful） -// ============================================ -Route::get('/api/users/{user_id}/tags', [app\controller\TagController::class, 'listByUser']); // 查询用户标签 -Route::put('/api/users/{user_id}/tags', [app\controller\TagController::class, 'calculate']); // 更新/计算用户标签 -Route::delete('/api/users/{user_id}/tags/{tag_id}', [app\controller\TagController::class, 'destroy']); // 删除用户标签 - -// ============================================ -// 消费记录相关接口 -// ============================================ -Route::post('/api/consumption/record', [app\controller\ConsumptionController::class, 'store']); // 创建消费记录 - -// ============================================ -// 标签定义相关接口（管理接口） -// ============================================ -Route::post('/api/tags/filter', [app\controller\TagController::class, 'filter']); // 根据标签筛选用户 -Route::get('/api/tags/statistics', [app\controller\TagController::class, 'statistics']); // 获取标签统计信息 -Route::get('/api/tags/history', [app\controller\TagController::class, 'history']); // 获取标签历史记录 -Route::post('/api/tag-definitions/batch', [app\controller\TagController::class, 'init']); // 批量初始化标签定义 -Route::get('/api/tag-definitions', [app\controller\TagDefinitionController::class, 'list']); // 获取标签定义列表 -Route::post('/api/tag-definitions', [app\controller\TagDefinitionController::class, 'create']); // 创建标签定义 -Route::get('/api/tag-definitions/{tag_id}', [app\controller\TagDefinitionController::class, 'detail']); // 获取标签定义详情 -Route::put('/api/tag-definitions/{tag_id}', [app\controller\TagDefinitionController::class, 'update']); // 更新标签定义 -Route::delete('/api/tag-definitions/{tag_id}', [app\controller\TagDefinitionController::class, 'delete']); // 删除标签定义 - -// ============================================ -// 标签任务管理接口 -// ============================================ -Route::post('/api/tag-tasks', [app\controller\TagTaskController::class, 'create']); // 创建标签任务 -Route::put('/api/tag-tasks/{task_id}', [app\controller\TagTaskController::class, 'update']); // 更新标签任务 -Route::delete('/api/tag-tasks/{task_id}', [app\controller\TagTaskController::class, 'delete']); // 删除标签任务 -Route::get('/api/tag-tasks', [app\controller\TagTaskController::class, 'list']); // 标签任务列表 -Route::get('/api/tag-tasks/{task_id}', [app\controller\TagTaskController::class, 'detail']); // 标签任务详情 -Route::get('/api/tag-tasks/{task_id}/executions', [app\controller\TagTaskController::class, 'executions']); // 获取任务执行记录 -Route::post('/api/tag-tasks/{task_id}/start', [app\controller\TagTaskController::class, 'start']); // 启动标签任务 -Route::post('/api/tag-tasks/{task_id}/pause', [app\controller\TagTaskController::class, 'pause']); // 暂停标签任务 -Route::post('/api/tag-tasks/{task_id}/stop', [app\controller\TagTaskController::class, 'stop']); // 停止标签任务 - -// ============================================ -// 身份合并相关接口（场景4：手机号发现身份证后合并） -// ============================================ -Route::post('/api/person-merge/phone-to-id-card', [app\controller\PersonMergeController::class, 'mergePhoneToIdCard']); // 合并手机号到身份证 -Route::post('/api/person-merge/temporary-to-formal', [app\controller\PersonMergeController::class, 'mergeTemporaryToFormal']); // 合并临时人到正式人 - -// ============================================ -// 数据库同步相关接口 -// ============================================ -Route::get('/database-sync/dashboard', [app\controller\DatabaseSyncController::class, 'dashboard']); // 同步进度看板页面 -Route::get('/api/database-sync/progress', [app\controller\DatabaseSyncController::class, 'progress']); // 查询同步进度 -Route::get('/api/database-sync/stats', [app\controller\DatabaseSyncController::class, 'stats']); // 查询同步统计 -Route::post('/api/database-sync/reset', [app\controller\DatabaseSyncController::class, 'reset']); // 重置同步进度 -Route::post('/api/database-sync/skip-error', [app\controller\DatabaseSyncController::class, 'skipError']); // 跳过错误数据库 - -// ============================================ -// 数据采集任务管理接口 -// ============================================ -Route::post('/api/data-collection-tasks', [app\controller\DataCollectionTaskController::class, 'create']); // 创建任务 -Route::put('/api/data-collection-tasks/{task_id}', [app\controller\DataCollectionTaskController::class, 'update']); // 更新任务 -Route::delete('/api/data-collection-tasks/{task_id}', [app\controller\DataCollectionTaskController::class, 'delete']); // 删除任务 -Route::get('/api/data-collection-tasks', [app\controller\DataCollectionTaskController::class, 'list']); // 任务列表 -Route::get('/api/data-collection-tasks/data-sources', [app\controller\DataCollectionTaskController::class, 'getDataSources']); // 获取数据源列表 -Route::get('/api/data-collection-tasks/{task_id}', [app\controller\DataCollectionTaskController::class, 'detail']); // 任务详情 -Route::get('/api/data-collection-tasks/{task_id}/progress', [app\controller\DataCollectionTaskController::class, 'progress']); // 任务进度 -Route::post('/api/data-collection-tasks/{task_id}/start', [app\controller\DataCollectionTaskController::class, 'start']); // 启动任务 -Route::post('/api/data-collection-tasks/{task_id}/pause', [app\controller\DataCollectionTaskController::class, 'pause']); // 暂停任务 -Route::post('/api/data-collection-tasks/{task_id}/stop', [app\controller\DataCollectionTaskController::class, 'stop']); // 停止任务 -Route::get('/api/data-collection-tasks/data-sources/{data_source_id}/databases', [app\controller\DataCollectionTaskController::class, 'getDatabases']); // 获取数据库列表 -Route::get('/api/data-collection-tasks/data-sources/{data_source_id}/databases/{database}/collections', [app\controller\DataCollectionTaskController::class, 'getCollections']); // 获取集合列表 -Route::get('/api/data-collection-tasks/data-sources/{data_source_id}/databases/{database}/collections/{collection}/fields', [app\controller\DataCollectionTaskController::class, 'getFields']); // 获取字段列表 -Route::get('/api/data-collection-tasks/handlers/{handler_type}/target-fields', [app\controller\DataCollectionTaskController::class, 'getHandlerTargetFields']); // 获取Handler的目标字段列表 -Route::post('/api/data-collection-tasks/preview-query', [app\controller\DataCollectionTaskController::class, 'previewQuery']); // 预览查询结果 - -// ============================================ -// 数据源管理接口 -// ============================================ -Route::get('/api/data-sources', [app\controller\DataSourceController::class, 'list']); // 获取数据源列表 -Route::get('/api/data-sources/{data_source_id}', [app\controller\DataSourceController::class, 'detail']); // 获取数据源详情 -Route::post('/api/data-sources', [app\controller\DataSourceController::class, 'create']); // 创建数据源 -Route::put('/api/data-sources/{data_source_id}', [app\controller\DataSourceController::class, 'update']); // 更新数据源 -Route::delete('/api/data-sources/{data_source_id}', [app\controller\DataSourceController::class, 'delete']); // 删除数据源 -Route::post('/api/data-sources/test-connection', [app\controller\DataSourceController::class, 'testConnection']); // 测试数据源连接 - -// ============================================ -// 人群快照相关接口 -// ============================================ -Route::get('/api/tag-cohorts', [app\controller\TagCohortController::class, 'list']); // 获取人群快照列表 -Route::get('/api/tag-cohorts/{cohort_id}', [app\controller\TagCohortController::class, 'detail']); // 获取人群快照详情 -Route::post('/api/tag-cohorts', [app\controller\TagCohortController::class, 'create']); // 创建人群快照 -Route::delete('/api/tag-cohorts/{cohort_id}', [app\controller\TagCohortController::class, 'delete']); // 删除人群快照 -Route::post('/api/tag-cohorts/{cohort_id}/export', [app\controller\TagCohortController::class, 'export']); // 导出人群快照 \ No newline at end of file diff --git a/Moncter/Moncter/config/server.php b/Moncter/Moncter/config/server.php deleted file mode 100644 index 054d01fb..00000000 --- a/Moncter/Moncter/config/server.php +++ /dev/null @@ -1,23 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -return [ - 'event_loop' => '', - 'stop_timeout' => 2, - 'pid_file' => runtime_path() . '/webman.pid', - 'status_file' => runtime_path() . '/webman.status', - 'stdout_file' => runtime_path() . '/logs/stdout.log', - 'log_file' => runtime_path() . '/logs/workerman.log', - 'max_package_size' => 10 * 1024 * 1024 -]; diff --git a/Moncter/Moncter/config/session.php b/Moncter/Moncter/config/session.php deleted file mode 100644 index 043f8c45..00000000 --- a/Moncter/Moncter/config/session.php +++ /dev/null @@ -1,65 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -use Webman\Session\FileSessionHandler; -use Webman\Session\RedisSessionHandler; -use Webman\Session\RedisClusterSessionHandler; - -return [ - - 'type' => 'file', // or redis or redis_cluster - - 'handler' => FileSessionHandler::class, - - 'config' => [ - 'file' => [ - 'save_path' => runtime_path() . '/sessions', - ], - 'redis' => [ - 'host' => '127.0.0.1', - 'port' => 6379, - 'auth' => '', - 'timeout' => 2, - 'database' => '', - 'prefix' => 'redis_session_', - ], - 'redis_cluster' => [ - 'host' => ['127.0.0.1:7000', '127.0.0.1:7001', '127.0.0.1:7001'], - 'timeout' => 2, - 'auth' => '', - 'prefix' => 'redis_session_', - ] - ], - - 'session_name' => 'PHPSID', - - 'auto_update_timestamp' => false, - - 'lifetime' => 7*24*60*60, - - 'cookie_lifetime' => 365*24*60*60, - - 'cookie_path' => '/', - - 'domain' => '', - - 'http_only' => true, - - 'secure' => false, - - 'same_site' => '', - - 'gc_probability' => [1, 1000], - -]; diff --git a/Moncter/Moncter/config/static.php b/Moncter/Moncter/config/static.php deleted file mode 100644 index 63136796..00000000 --- a/Moncter/Moncter/config/static.php +++ /dev/null @@ -1,23 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -/** - * Static file settings - */ -return [ - 'enable' => true, - 'middleware' => [ // Static file Middleware - //app\middleware\StaticFile::class, - ], -]; \ No newline at end of file diff --git a/Moncter/Moncter/config/translation.php b/Moncter/Moncter/config/translation.php deleted file mode 100644 index 96589b2b..00000000 --- a/Moncter/Moncter/config/translation.php +++ /dev/null @@ -1,25 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -/** - * Multilingual configuration - */ -return [ - // Default language - 'locale' => 'zh_CN', - // Fallback language - 'fallback_locale' => ['zh_CN', 'en'], - // Folder where language files are stored - 'path' => base_path() . '/resource/translations', -]; \ No newline at end of file diff --git a/Moncter/Moncter/config/view.php b/Moncter/Moncter/config/view.php deleted file mode 100644 index e3a7b856..00000000 --- a/Moncter/Moncter/config/view.php +++ /dev/null @@ -1,22 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -use support\view\Raw; -use support\view\Twig; -use support\view\Blade; -use support\view\ThinkPHP; - -return [ - 'handler' => Raw::class -]; diff --git a/Moncter/Moncter/env.txt b/Moncter/Moncter/env.txt deleted file mode 100644 index af53e480..00000000 --- a/Moncter/Moncter/env.txt +++ /dev/null @@ -1,54 +0,0 @@ -# ============================================ -# 加密配置 -# ============================================ - -# AES 加密密钥（至少32字符，建议使用随机生成的强密钥） -# 生产环境请务必修改此密钥，并妥善保管 -ENCRYPTION_AES_KEY=your-32-byte-secret-key-here-12345678 - -# 哈希盐值（用于身份证哈希，增强安全性） -# 生产环境请务必修改此盐值 -ENCRYPTION_HASH_SALT=your-hash-salt-here-change-in-production - -# ============================================ -# 应用配置 -# ============================================ - -# 应用环境（development/production） -APP_ENV=development - -# 应用调试模式（true/false） -APP_DEBUG=true - - -# ============================================ -# 以下为：超级主机资源数据库 -# ============================================ -#主机标签数据库 - -TAG_MONGODB_DRIVER = "mongodb" -TAG_MONGODB_DNS = mongodb://192.168.1.106:27017 -TAG_MONGODB_DATABASE = ckb -TAG_MONGODB_USER = ckb -TAG_MONGODB_AUTH = ckb -TAG_MONGODB_PASSWORD = 123456 - -#主机同步KR数据库 -SYNC_MONGODB_HOST = 192.168.1.106 -SYNC_MONGODB_PORT = 27017 -SYNC_MONGODB_AUTH = KR -SYNC_MONGODB_USER = KR -SYNC_MONGODB_PASS = 123456 - -# ============================================ -# 以下为：爬虫抓取的业务数据库 -# ============================================ - -#卡若的数据库 -KR_MONGODB_HOST = 192.168.2.8 -KR_MONGODB_PORT = 27017 -KR_MONGODB_DATABASE = admin -KR_MONGODB_USER = admin -KR_MONGODB_PASSWORD = key123456 -KR_MONGODB_AUTH_SOURCE=admin - diff --git a/Moncter/Moncter/go.sh b/Moncter/Moncter/go.sh deleted file mode 100644 index 6b14ac80..00000000 --- a/Moncter/Moncter/go.sh +++ /dev/null @@ -1,46 +0,0 @@ -#!/bin/bash -set -euo pipefail # 严格模式：报错立即退出、禁止未定义变量、管道错误触发退出 - -# ================= 配置项（可根据实际情况修改）================= -# PHP 脚本路径（相对路径/绝对路径均可，推荐绝对路径更稳定） -PHP_SCRIPT="start.php" -# PHP 解释器路径（默认自动查找，若提示 php 未找到，手动指定如 /usr/bin/php） -PHP_BIN=$(which php || echo "/usr/bin/php") -# ============================================================== - -# 1. 检查 PHP 解释器是否存在且可执行 -if [ ! -x "$PHP_BIN" ]; then - echo -e "\033[31m错误：未找到可执行的 PHP 解释器！\033[0m" - echo " 解决方案：" - echo " 1. 安装 PHP：sudo apt install php-cli（Ubuntu/Debian）或 sudo dnf install php-cli（CentOS/RHEL）" - echo " 2. 若已安装，手动修改脚本中的 PHP_BIN 为实际路径（通过 which php 查询）" - exit 1 -fi - -# 2. 检查 PHP 脚本是否存在 -if [ ! -f "$PHP_SCRIPT" ]; then - echo -e "\033[31m错误：未找到脚本文件 $PHP_SCRIPT！\033[0m" - echo " 请确保脚本与 $PHP_SCRIPT 在同一目录，或修改脚本中的 PHP_SCRIPT 为绝对路径" - exit 1 -fi - -# 3. 给 PHP 脚本添加执行权限（自动修复权限问题） -if [ ! -x "$PHP_SCRIPT" ]; then - echo -e "\033[33m警告：$PHP_SCRIPT 缺少执行权限，正在自动添加...\033[0m" - chmod u+x "$PHP_SCRIPT" || { - echo -e "\033[31m错误：添加执行权限失败，请用 sudo 运行脚本！\033[0m" - exit 1 - } -fi - -# 4. 执行核心命令（带日志输出优化） -echo -e "\033[32m=== 开始执行：$PHP_BIN $PHP_SCRIPT start ===\033[0m" -$PHP_BIN "$PHP_SCRIPT" start - -# 5. 执行结果判断 -if [ $? -eq 0 ]; then - echo -e "\033[32m=== 执行成功！===\033[0m" -else - echo -e "\033[31m=== 执行失败！请查看上方错误信息 ===\033[0m" - exit 1 -fi \ No newline at end of file diff --git a/Moncter/Moncter/public/database-sync-dashboard.html b/Moncter/Moncter/public/database-sync-dashboard.html deleted file mode 100644 index 40920985..00000000 --- a/Moncter/Moncter/public/database-sync-dashboard.html +++ /dev/null @@ -1,1024 +0,0 @@ - - - - - - 数据库同步进度看板 - - - -
-
-

📊 数据库同步进度看板

-
- 加载中... - 最后更新: -- -
-
- -
-
- -
正在加载数据...
-
-
- -
-

数据每 3 秒自动刷新 | 数据库同步系统

-
-
- - - - - diff --git a/Moncter/Moncter/public/favicon.ico b/Moncter/Moncter/public/favicon.ico deleted file mode 100644 index b9f722e0..00000000 Binary files a/Moncter/Moncter/public/favicon.ico and /dev/null differ diff --git a/Moncter/Moncter/src/store/module/websocket/websocket.ts b/Moncter/Moncter/src/store/module/websocket/websocket.ts deleted file mode 100644 index e69de29b..00000000 diff --git a/Moncter/Moncter/src/utils/cacheCleaner.ts b/Moncter/Moncter/src/utils/cacheCleaner.ts deleted file mode 100644 index 78f30c50..00000000 --- a/Moncter/Moncter/src/utils/cacheCleaner.ts +++ /dev/null @@ -1,73 +0,0 @@ -// 缓存清理工具，统一处理浏览器存储与 Zustand store -import { clearAllPersistedData } from "@/store"; -import { useUserStore } from "@/store/module/user"; -import { useAppStore } from "@/store/module/app"; -import { useSettingsStore } from "@/store/module/settings"; - -const isBrowser = typeof window !== "undefined"; - -const safeStorageClear = (storage?: Storage) => { - if (!storage) return; - try { - storage.clear(); - } catch (error) { - console.warn("清理存储失败:", error); - } -}; - -export const clearBrowserStorage = () => { - if (!isBrowser) return; - safeStorageClear(window.localStorage); - safeStorageClear(window.sessionStorage); - // 清理自定义持久化数据 - try { - clearAllPersistedData(); - } catch (error) { - console.warn("清理持久化 store 失败:", error); - } -}; - -export const clearAllIndexedDB = async (): Promise => { - if (!isBrowser || !window.indexedDB || !indexedDB.databases) return; - - const databases = await indexedDB.databases(); - const deleteJobs = databases - .map(db => db.name) - .filter((name): name is string => Boolean(name)) - .map( - name => - new Promise((resolve, reject) => { - const request = indexedDB.deleteDatabase(name); - request.onsuccess = () => resolve(); - request.onerror = () => - reject(new Error(`删除数据库 ${name} 失败`)); - request.onblocked = () => { - setTimeout(() => { - const retry = indexedDB.deleteDatabase(name); - retry.onsuccess = () => resolve(); - retry.onerror = () => - reject(new Error(`删除数据库 ${name} 失败`)); - }, 100); - }; - }), - ); - - await Promise.allSettled(deleteJobs); -}; - -export const resetAllStores = () => { - const userStore = useUserStore.getState(); - const appStore = useAppStore.getState(); - const settingsStore = useSettingsStore.getState(); - - userStore?.clearUser?.(); - appStore?.resetAppState?.(); - settingsStore?.resetSettings?.(); -}; - -export const clearApplicationCache = async () => { - clearBrowserStorage(); - await clearAllIndexedDB(); - resetAllStores(); -}; - diff --git a/Moncter/Moncter/start.php b/Moncter/Moncter/start.php deleted file mode 100644 index 41ad7ef2..00000000 --- a/Moncter/Moncter/start.php +++ /dev/null @@ -1,5 +0,0 @@ -#!/usr/bin/env php - /dev/null; then - echo "❌ 错误: 未找到 PHP，请先安装 PHP" - exit 1 -fi - -echo "✓ PHP 版本: $(php -v | head -n 1)" -echo "" - -# 停止已有进程 -if [ -f "runtime/webman.pid" ]; then - echo "🛑 停止已运行的进程..." - php start.php stop - sleep 2 -fi - -# 以调试模式启动（不使用 daemon 模式，输出到终端） -echo "🚀 启动 Workerman（调试模式）..." -echo " 提示: 按 Ctrl+C 停止" -echo "==================================================" -echo "" - -# 使用 start 而不是 start -d（daemon），这样输出会显示在终端 -php start.php start - diff --git a/Moncter/Moncter/support/Request.php b/Moncter/Moncter/support/Request.php deleted file mode 100644 index e3f6ac37..00000000 --- a/Moncter/Moncter/support/Request.php +++ /dev/null @@ -1,24 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -namespace support; - -/** - * Class Request - * @package support - */ -class Request extends \Webman\Http\Request -{ - -} \ No newline at end of file diff --git a/Moncter/Moncter/support/Response.php b/Moncter/Moncter/support/Response.php deleted file mode 100644 index 9bc4e1eb..00000000 --- a/Moncter/Moncter/support/Response.php +++ /dev/null @@ -1,24 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -namespace support; - -/** - * Class Response - * @package support - */ -class Response extends \Webman\Http\Response -{ - -} \ No newline at end of file diff --git a/Moncter/Moncter/support/bootstrap.php b/Moncter/Moncter/support/bootstrap.php deleted file mode 100644 index 92ce480c..00000000 --- a/Moncter/Moncter/support/bootstrap.php +++ /dev/null @@ -1,147 +0,0 @@ - - * @copyright walkor - * @link http://www.workerman.net/ - * @license http://www.opensource.org/licenses/mit-license.php MIT License - */ - -use Dotenv\Dotenv; -use support\Log; -use Webman\Bootstrap; -use Webman\Config; -use Webman\Middleware; -use Webman\Route; -use Webman\Util; -use Workerman\Events\Select; -use Workerman\Worker; - -$worker = $worker ?? null; - -if (empty(Worker::$eventLoopClass)) { - Worker::$eventLoopClass = Select::class; -} - -set_error_handler(function ($level, $message, $file = '', $line = 0) { - // 忽略 MongoDB Laravel 的废弃警告（E_USER_DEPRECATED = 16384） - // 这些警告不影响功能，只是提示使用新的API - if ($level === E_USER_DEPRECATED && strpos($message, 'Using "$collection" property is deprecated') !== false) { - return true; // 忽略此警告 - } - - if (error_reporting() & $level) { - throw new ErrorException($message, 0, $level, $file, $line); - } - - return false; -}); - -if ($worker) { - register_shutdown_function(function ($startTime) { - if (time() - $startTime <= 0.1) { - sleep(1); - } - }, time()); -} - -if (class_exists('Dotenv\Dotenv') && file_exists(base_path(false) . '/.env')) { - if (method_exists('Dotenv\Dotenv', 'createUnsafeMutable')) { - Dotenv::createUnsafeMutable(base_path(false))->load(); - } else { - Dotenv::createMutable(base_path(false))->load(); - } -} - -Config::clear(); -support\App::loadAllConfig(['route']); -if ($timezone = config('app.default_timezone')) { - date_default_timezone_set($timezone); -} - -foreach (config('autoload.files', []) as $file) { - include_once $file; -} -foreach (config('plugin', []) as $firm => $projects) { - foreach ($projects as $name => $project) { - if (!is_array($project)) { - continue; - } - foreach ($project['autoload']['files'] ?? [] as $file) { - include_once $file; - } - } - foreach ($projects['autoload']['files'] ?? [] as $file) { - include_once $file; - } -} - -Middleware::load(config('middleware', [])); -foreach (config('plugin', []) as $firm => $projects) { - foreach ($projects as $name => $project) { - if (!is_array($project) || $name === 'static') { - continue; - } - Middleware::load($project['middleware'] ?? []); - } - Middleware::load($projects['middleware'] ?? [], $firm); - if ($staticMiddlewares = config("plugin.$firm.static.middleware")) { - Middleware::load(['__static__' => $staticMiddlewares], $firm); - } -} -Middleware::load(['__static__' => config('static.middleware', [])]); - -foreach (config('bootstrap', []) as $className) { - if (!class_exists($className)) { - $log = "Warning: Class $className setting in config/bootstrap.php not found\r\n"; - echo $log; - Log::error($log); - continue; - } - /** @var Bootstrap $className */ - $className::start($worker); -} - -foreach (config('plugin', []) as $firm => $projects) { - foreach ($projects as $name => $project) { - if (!is_array($project)) { - continue; - } - foreach ($project['bootstrap'] ?? [] as $className) { - if (!class_exists($className)) { - $log = "Warning: Class $className setting in config/plugin/$firm/$name/bootstrap.php not found\r\n"; - echo $log; - Log::error($log); - continue; - } - /** @var Bootstrap $className */ - $className::start($worker); - } - } - foreach ($projects['bootstrap'] ?? [] as $className) { - /** @var string $className */ - if (!class_exists($className)) { - $log = "Warning: Class $className setting in plugin/$firm/config/bootstrap.php not found\r\n"; - echo $log; - Log::error($log); - continue; - } - /** @var Bootstrap $className */ - $className::start($worker); - } -} - -$directory = base_path() . '/plugin'; -$paths = [config_path()]; -foreach (Util::scanDir($directory) as $path) { - if (is_dir($path = "$path/config")) { - $paths[] = $path; - } -} -Route::load($paths); - diff --git a/Moncter/Moncter/support/bootstrap/MongoDB.php b/Moncter/Moncter/support/bootstrap/MongoDB.php deleted file mode 100644 index f60c1306..00000000 --- a/Moncter/Moncter/support/bootstrap/MongoDB.php +++ /dev/null @@ -1,73 +0,0 @@ -getDatabaseManager()->extend('mongodb', function ($config, $name) { - $config['name'] = $name; - return new Connection($config); - }); - - // 添加 MongoDB 连接 - $capsule->addConnection([ - 'driver' => 'mongodb', - 'dsn' => $dsn, - 'database' => $mongoConfig['database'], - 'options' => $options, - ], 'mongodb'); - - // 设置为全局连接管理器 - $capsule->setAsGlobal(); - - // 启动 Eloquent ORM - $capsule->bootEloquent(); - } - } -} - diff --git a/Moncter/Moncter/weChat.ts b/Moncter/Moncter/weChat.ts deleted file mode 100644 index b28b04f6..00000000 --- a/Moncter/Moncter/weChat.ts +++ /dev/null @@ -1,3 +0,0 @@ - - - diff --git a/Moncter/Moncter/windows.bat b/Moncter/Moncter/windows.bat deleted file mode 100644 index f07ce532..00000000 --- a/Moncter/Moncter/windows.bat +++ /dev/null @@ -1,3 +0,0 @@ -CHCP 65001 -php windows.php -pause \ No newline at end of file diff --git a/Moncter/Moncter/windows.php b/Moncter/Moncter/windows.php deleted file mode 100644 index f37a72c9..00000000 --- a/Moncter/Moncter/windows.php +++ /dev/null @@ -1,136 +0,0 @@ -load(); - } else { - Dotenv::createMutable(base_path())->load(); - } -} - -App::loadAllConfig(['route']); - -$errorReporting = config('app.error_reporting'); -if (isset($errorReporting)) { - error_reporting($errorReporting); -} - -$runtimeProcessPath = runtime_path() . DIRECTORY_SEPARATOR . '/windows'; -$paths = [ - $runtimeProcessPath, - runtime_path('logs'), - runtime_path('views') -]; -foreach ($paths as $path) { - if (!is_dir($path)) { - mkdir($path, 0777, true); - } -} - -$processFiles = []; -if (config('server.listen')) { - $processFiles[] = __DIR__ . DIRECTORY_SEPARATOR . 'start.php'; -} -foreach (config('process', []) as $processName => $config) { - $processFiles[] = write_process_file($runtimeProcessPath, $processName, ''); -} - -foreach (config('plugin', []) as $firm => $projects) { - foreach ($projects as $name => $project) { - if (!is_array($project)) { - continue; - } - foreach ($project['process'] ?? [] as $processName => $config) { - $processFiles[] = write_process_file($runtimeProcessPath, $processName, "$firm.$name"); - } - } - foreach ($projects['process'] ?? [] as $processName => $config) { - $processFiles[] = write_process_file($runtimeProcessPath, $processName, $firm); - } -} - -function write_process_file($runtimeProcessPath, $processName, $firm): string -{ - $processParam = $firm ? "plugin.$firm.$processName" : $processName; - $configParam = $firm ? "config('plugin.$firm.process')['$processName']" : "config('process')['$processName']"; - $fileContent = << true]); - if (!$resource) { - exit("Can not execute $cmd\r\n"); - } - return $resource; -} - -$resource = popen_processes($processFiles); -echo "\r\n"; -while (1) { - sleep(1); - if (!empty($monitor) && $monitor->checkAllFilesChange()) { - $status = proc_get_status($resource); - $pid = $status['pid']; - shell_exec("taskkill /F /T /PID $pid"); - proc_close($resource); - $resource = popen_processes($processFiles); - } -} diff --git a/Moncter/Moncter/yarn.lock b/Moncter/Moncter/yarn.lock deleted file mode 100644 index 157d8630..00000000 --- a/Moncter/Moncter/yarn.lock +++ /dev/null @@ -1,2492 +0,0 @@ -# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY. -# yarn lockfile v1 - - -"@ai-sdk/gateway@2.0.18": - version "2.0.18" - resolved "https://registry.npmjs.org/@ai-sdk/gateway/-/gateway-2.0.18.tgz" - integrity sha512-sDQcW+6ck2m0pTIHW6BPHD7S125WD3qNkx/B8sEzJp/hurocmJ5Cni0ybExg6sQMGo+fr/GWOwpHF1cmCdg5rQ== - dependencies: - "@ai-sdk/provider" "2.0.0" - "@ai-sdk/provider-utils" "3.0.18" - "@vercel/oidc" "3.0.5" - -"@ai-sdk/provider-utils@^3.0.0", "@ai-sdk/provider-utils@3.0.18": - version "3.0.18" - resolved "https://registry.npmjs.org/@ai-sdk/provider-utils/-/provider-utils-3.0.18.tgz" - integrity sha512-ypv1xXMsgGcNKUP+hglKqtdDuMg68nWHucPPAhIENrbFAI+xCHiqPVN8Zllxyv1TNZwGWUghPxJXU+Mqps0YRQ== - dependencies: - "@ai-sdk/provider" "2.0.0" - "@standard-schema/spec" "^1.0.0" - eventsource-parser "^3.0.6" - -"@ai-sdk/provider@^2.0.0", "@ai-sdk/provider@2.0.0": - version "2.0.0" - resolved "https://registry.npmjs.org/@ai-sdk/provider/-/provider-2.0.0.tgz" - integrity sha512-6o7Y2SeO9vFKB8lArHXehNuusnpddKPk7xqL7T2/b+OvXMRIXUO1rR4wcv1hAFUAT9avGZshty3Wlua/XA7TvA== - dependencies: - json-schema "^0.4.0" - -"@modelcontextprotocol/sdk@^1.24.2": - version "1.24.3" - resolved "https://registry.npmjs.org/@modelcontextprotocol/sdk/-/sdk-1.24.3.tgz" - integrity sha512-YgSHW29fuzKKAHTGe9zjNoo+yF8KaQPzDC2W9Pv41E7/57IfY+AMGJ/aDFlgTLcVVELoggKE4syABCE75u3NCw== - dependencies: - ajv "^8.17.1" - ajv-formats "^3.0.1" - content-type "^1.0.5" - cors "^2.8.5" - cross-spawn "^7.0.5" - eventsource "^3.0.2" - eventsource-parser "^3.0.0" - express "^5.0.1" - express-rate-limit "^7.5.0" - jose "^6.1.1" - pkce-challenge "^5.0.0" - raw-body "^3.0.0" - zod "^3.25 || ^4.0" - zod-to-json-schema "^3.25.0" - -"@mongodb-js/atlas-local-win32-x64-msvc@1.1.0": - version "1.1.0" - resolved "https://registry.npmjs.org/@mongodb-js/atlas-local-win32-x64-msvc/-/atlas-local-win32-x64-msvc-1.1.0.tgz" - integrity sha512-K+kGoGfIPqv6JstlzmnbkOOJ6/0cTH4bjRbEApmvscnTDBk1c9eO8nXORPQWFfKnMMxo+/9GQESvaxC05MhwBQ== - -"@mongodb-js/atlas-local@^1.1.0": - version "1.1.0" - resolved "https://registry.npmjs.org/@mongodb-js/atlas-local/-/atlas-local-1.1.0.tgz" - integrity sha512-ZGvBy8PdqNgcfZPaElSrmc3ItafSbodowipgQkJN65jKuAM8owETpMc8/ASxHTlI91y4sOll1V5WQR0FkqDHjA== - optionalDependencies: - "@mongodb-js/atlas-local-darwin-arm64" "1.1.0" - "@mongodb-js/atlas-local-darwin-x64" "1.1.0" - "@mongodb-js/atlas-local-linux-arm64-gnu" "1.1.0" - "@mongodb-js/atlas-local-linux-x64-gnu" "1.1.0" - "@mongodb-js/atlas-local-win32-x64-msvc" "1.1.0" - -"@mongodb-js/device-id@^0.3.1": - version "0.3.3" - resolved "https://registry.npmjs.org/@mongodb-js/device-id/-/device-id-0.3.3.tgz" - integrity sha512-y2SIriQec/HvIzFI1QCJAmjedShwbgmgUWkh3+qPgT2OyRKWE0yB7Lii/jANCtUNqqwBcUsMHY9/mPnuu5wfPQ== - -"@mongodb-js/devtools-connect@^3.9.4": - version "3.12.0" - resolved "https://registry.npmjs.org/@mongodb-js/devtools-connect/-/devtools-connect-3.12.0.tgz" - integrity sha512-/aiGAKE5k6y1noI6hFo3pkLarNCNjEn+J3iqWTAMBuX4SpKUWsDdpMAyyxkqou7qH97gvon4A7wQafWFgWTXvA== - dependencies: - "@mongodb-js/devtools-proxy-support" "^0.5.5" - "@mongodb-js/oidc-http-server-pages" "1.1.8" - lodash.merge "^4.6.2" - mongodb-connection-string-url "^3.0.0" - socks "^2.7.3" - optionalDependencies: - kerberos "^2.1.0" - mongodb-client-encryption "^6.1.0" - os-dns-native "^1.2.0" - resolve-mongodb-srv "^1.1.1" - -"@mongodb-js/devtools-proxy-support@^0.5.3", "@mongodb-js/devtools-proxy-support@^0.5.5": - version "0.5.5" - resolved "https://registry.npmjs.org/@mongodb-js/devtools-proxy-support/-/devtools-proxy-support-0.5.5.tgz" - integrity sha512-yrz6rCMgq+52SL34O16mvZsGeQs+Gq1rRmJiHxj/Ca6LNvJo+MhDHvISSwoiObl1hWdcnDz1qz5HyaxMzpkETg== - dependencies: - "@mongodb-js/socksv5" "^0.0.10" - agent-base "^7.1.1" - debug "^4.4.0" - http-proxy-agent "^7.0.2" - https-proxy-agent "^7.0.5" - lru-cache "^11.0.0" - node-fetch "^3.3.2" - pac-proxy-agent "^7.0.2" - socks-proxy-agent "^8.0.4" - ssh2 "^1.15.0" - system-ca "^2.0.1" - -"@mongodb-js/oidc-http-server-pages@1.1.8": - version "1.1.8" - resolved "https://registry.npmjs.org/@mongodb-js/oidc-http-server-pages/-/oidc-http-server-pages-1.1.8.tgz" - integrity sha512-qQjV6TTozJa3jMfsmajhCCzESnsTGa3rQwxGbC97GIplaF2v3jieLfDt9O/F6X3jLm32HzAEWHFEvKu74h2wxw== - -"@mongodb-js/oidc-plugin@^2.0.0", "@mongodb-js/oidc-plugin@^2.0.5": - version "2.0.6" - resolved "https://registry.npmjs.org/@mongodb-js/oidc-plugin/-/oidc-plugin-2.0.6.tgz" - integrity sha512-Ma38DqIKTddMQhKrHwx/T+4nFvrTAy5RVWjDQzHv4W9W/lwnZqJWxayDDKwrrd+Gg1xuj5o4GVbkr3A5hI0f1w== - dependencies: - express "^5.1.0" - node-fetch "^3.3.2" - open "^10.1.2" - openid-client "^6.6.3" - -"@mongodb-js/saslprep@^1.3.0": - version "1.3.2" - resolved "https://registry.npmjs.org/@mongodb-js/saslprep/-/saslprep-1.3.2.tgz" - integrity sha512-QgA5AySqB27cGTXBFmnpifAi7HxoGUeezwo6p9dI03MuDB6Pp33zgclqVb6oVK3j6I9Vesg0+oojW2XxB59SGg== - dependencies: - sparse-bitfield "^3.0.3" - -"@mongodb-js/socksv5@^0.0.10": - version "0.0.10" - resolved "https://registry.npmjs.org/@mongodb-js/socksv5/-/socksv5-0.0.10.tgz" - integrity sha512-JDz2fLKsjMiSNUxKrCpGptsgu7DzsXfu4gnUQ3RhUaBS1d4YbLrt6HejpckAiHIAa+niBpZAeiUsoop0IihWsw== - dependencies: - ip-address "^9.0.5" - -"@mongosh/arg-parser@^3.23.0": - version "3.23.0" - resolved "https://registry.npmjs.org/@mongosh/arg-parser/-/arg-parser-3.23.0.tgz" - integrity sha512-V9lr8LEHI9XKEgBEqPAVDi+CNdl6MNBSJ8A5LAk+vfBM+bdhhtSrANIqUhuEzyV4yNyEqNfYFtqLLhjZd7/doA== - dependencies: - "@mongosh/errors" "2.4.5" - "@mongosh/i18n" "^2.20.0" - mongodb-connection-string-url "^3.0.2" - yargs-parser "^20.2.4" - -"@mongosh/errors@^2.4.5", "@mongosh/errors@2.4.5": - version "2.4.5" - resolved "https://registry.npmjs.org/@mongosh/errors/-/errors-2.4.5.tgz" - integrity sha512-niqLgzPv6ZG9Bx0XRJP3NCA9zZM6LbRs/z05GRwJP0B3HShYDKWi7B0D4N/u6RoEt93Y6mMa9Ok72dNSDpIEwA== - -"@mongosh/i18n@^2.20.0": - version "2.20.0" - resolved "https://registry.npmjs.org/@mongosh/i18n/-/i18n-2.20.0.tgz" - integrity sha512-g0zuKuZ5JhS/ASDizZlqLDzy1yqeMbs5tg40TxMl/55rM2EOPV1MSlfBZg7X0tyxUOATy1sLHWsUquzQSFjVjQ== - dependencies: - "@mongosh/errors" "2.4.5" - -"@mongosh/service-provider-core@3.7.0": - version "3.7.0" - resolved "https://registry.npmjs.org/@mongosh/service-provider-core/-/service-provider-core-3.7.0.tgz" - integrity sha512-a9riCeAkV69UZjFsNmanufMxhqJ3oDbB+Zm5h2DS0YA/cnDarpGOGPEMhi0VmnzcbDK3nmZy6DPOxtx0w4cVZQ== - dependencies: - "@mongosh/errors" "2.4.5" - "@mongosh/shell-bson" "1.1.0" - bson "^6.10.4" - mongodb "^6.19.0" - mongodb-build-info "^1.8.1" - mongodb-connection-string-url "^3.0.2" - -"@mongosh/service-provider-node-driver@^3.17.5": - version "3.18.0" - resolved "https://registry.npmjs.org/@mongosh/service-provider-node-driver/-/service-provider-node-driver-3.18.0.tgz" - integrity sha512-tQy40wpVd/xskQTXHVLwi7v9olHxFyIqSIF06kpl/q4BPPlKxI0jCA5xdy+lk+tvDMJ80Pams0RUonRjNulVzQ== - dependencies: - "@mongodb-js/devtools-connect" "^3.9.4" - "@mongodb-js/oidc-plugin" "^2.0.5" - "@mongosh/errors" "2.4.5" - "@mongosh/service-provider-core" "3.7.0" - "@mongosh/types" "^3.14.1" - aws4 "^1.12.0" - mongodb "^6.19.0" - mongodb-build-info "^1.8.1" - mongodb-connection-string-url "^3.0.2" - socks "^2.8.3" - optionalDependencies: - kerberos "2.1.0" - mongodb-client-encryption "^6.5.0" - -"@mongosh/shell-bson@1.1.0": - version "1.1.0" - resolved "https://registry.npmjs.org/@mongosh/shell-bson/-/shell-bson-1.1.0.tgz" - integrity sha512-eSkEvikWNPiFghiOx0puWqP28nXs/4dIHxIjbqORGEmDy/DaJXxGydYQKiUL2pebQAudoSQnzsFKyb0WQvypxw== - dependencies: - "@mongosh/errors" "^2.4.5" - -"@mongosh/types@^3.14.1": - version "3.14.1" - resolved "https://registry.npmjs.org/@mongosh/types/-/types-3.14.1.tgz" - integrity sha512-jX3Z/EmKmwPiHYSSALxAjeBHXGL1XMkEBOz0CCeE1dexCyMTyGf9H78cn+emBAVuyGQHhfo5WhyQhycNtDUZAQ== - dependencies: - "@mongodb-js/devtools-connect" "^3.9.4" - -"@opentelemetry/api@1.9.0": - version "1.9.0" - resolved "https://registry.npmjs.org/@opentelemetry/api/-/api-1.9.0.tgz" - integrity sha512-3giAOQvZiH5F9bMlMiv8+GSPMeqg0dbaeo58/0SlA9sxSqZhnUtxzX9/2FzyhS9sWQf5S0GJE0AKBrFqjpeYcg== - -"@standard-schema/spec@^1.0.0": - version "1.0.0" - resolved "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.0.0.tgz" - integrity sha512-m2bOd0f2RT9k8QJx1JN85cZYyH1RqFBdlwtkSlf4tBDYLCiiZnv1fIIwacK6cqwXavOydf0NPToMQgpKq+dVlA== - -"@tootallnate/quickjs-emscripten@^0.23.0": - version "0.23.0" - resolved "https://registry.npmjs.org/@tootallnate/quickjs-emscripten/-/quickjs-emscripten-0.23.0.tgz" - integrity sha512-C5Mc6rdnsaJDjO3UpGW/CQTHtCKaYlScZTly4JIu97Jxo/odCiH0ITnDXSJPTOrEKk/ycSZ0AOgTmkDtkOsvIA== - -"@types/webidl-conversions@*": - version "7.0.3" - resolved "https://registry.npmjs.org/@types/webidl-conversions/-/webidl-conversions-7.0.3.tgz" - integrity sha512-CiJJvcRtIgzadHCYXw7dqEnMNRjhGZlYK05Mj9OyktqV8uVT8fD2BFOB7S1uwBE3Kj2Z+4UyPmFw/Ixgw/LAlA== - -"@types/whatwg-url@^11.0.2": - version "11.0.5" - resolved "https://registry.npmjs.org/@types/whatwg-url/-/whatwg-url-11.0.5.tgz" - integrity sha512-coYR071JRaHa+xoEvvYqvnIHaVqaYrLPbsufM9BF63HkwI5Lgmy2QR8Q5K/lYDYo5AK82wOvSOS0UsLTpTG7uQ== - dependencies: - "@types/webidl-conversions" "*" - -"@vercel/oidc@3.0.5": - version "3.0.5" - resolved "https://registry.npmjs.org/@vercel/oidc/-/oidc-3.0.5.tgz" - integrity sha512-fnYhv671l+eTTp48gB4zEsTW/YtRgRPnkI2nT7x6qw5rkI1Lq2hTmQIpHPgyThI0znLK+vX2n9XxKdXZ7BUbbw== - -accepts@^2.0.0: - version "2.0.0" - resolved "https://registry.npmjs.org/accepts/-/accepts-2.0.0.tgz" - integrity sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng== - dependencies: - mime-types "^3.0.0" - negotiator "^1.0.0" - -agent-base@^7.1.0, agent-base@^7.1.1, agent-base@^7.1.2: - version "7.1.4" - resolved "https://registry.npmjs.org/agent-base/-/agent-base-7.1.4.tgz" - integrity sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ== - -ai@^5.0.72: - version "5.0.107" - resolved "https://registry.npmjs.org/ai/-/ai-5.0.107.tgz" - integrity sha512-laZlS9ZC/DZfSaxPgrBqI4mM+kxRvTPBBQfa74ceBFskkunZKEsaGVFNEs4cfyGa3nCCCl1WO/fjxixp4V8Zag== - dependencies: - "@ai-sdk/gateway" "2.0.18" - "@ai-sdk/provider" "2.0.0" - "@ai-sdk/provider-utils" "3.0.18" - "@opentelemetry/api" "1.9.0" - -ajv-formats@^3.0.1: - version "3.0.1" - resolved "https://registry.npmjs.org/ajv-formats/-/ajv-formats-3.0.1.tgz" - integrity sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ== - dependencies: - ajv "^8.0.0" - -ajv@^8.0.0, ajv@^8.17.1: - version "8.17.1" - resolved "https://registry.npmjs.org/ajv/-/ajv-8.17.1.tgz" - integrity sha512-B/gBuNg5SiMTrPkC+A2+cW0RszwxYmn6VYxB/inlBStS5nx6xHIt/ehKRhIMhqusl7a8LjQoZnjCs5vhwxOQ1g== - dependencies: - fast-deep-equal "^3.1.3" - fast-uri "^3.0.1" - json-schema-traverse "^1.0.0" - require-from-string "^2.0.2" - -ansi-regex@^5.0.1: - version "5.0.1" - resolved "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz" - integrity sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ== - -ansi-styles@^4.0.0: - version "4.3.0" - resolved "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz" - integrity sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg== - dependencies: - color-convert "^2.0.1" - -argparse@^2.0.1: - version "2.0.1" - resolved "https://registry.npmjs.org/argparse/-/argparse-2.0.1.tgz" - integrity sha512-8+9WqebbFzpX9OR+Wa6O29asIogeRMzcGtAINdpMHHyAg10f05aSFVBbcEqGf/PXw1EjAZ+q2/bEBg3DvurK3Q== - -array-buffer-byte-length@^1.0.1, array-buffer-byte-length@^1.0.2: - version "1.0.2" - resolved "https://registry.npmjs.org/array-buffer-byte-length/-/array-buffer-byte-length-1.0.2.tgz" - integrity sha512-LHE+8BuR7RYGDKvnrmcuSq3tDcKv9OFEXQt/HpbZhY7V6h0zlUXutnAD82GiFx9rdieCMjkvtcsPqBwgUl1Iiw== - dependencies: - call-bound "^1.0.3" - is-array-buffer "^3.0.5" - -arraybuffer.prototype.slice@^1.0.4: - version "1.0.4" - resolved "https://registry.npmjs.org/arraybuffer.prototype.slice/-/arraybuffer.prototype.slice-1.0.4.tgz" - integrity sha512-BNoCY6SXXPQ7gF2opIP4GBE+Xw7U+pHMYKuzjgCN3GwiaIR09UUeKfheyIry77QtrCBlC0KK0q5/TER/tYh3PQ== - dependencies: - array-buffer-byte-length "^1.0.1" - call-bind "^1.0.8" - define-properties "^1.2.1" - es-abstract "^1.23.5" - es-errors "^1.3.0" - get-intrinsic "^1.2.6" - is-array-buffer "^3.0.4" - -asn1@^0.2.6: - version "0.2.6" - resolved "https://registry.npmjs.org/asn1/-/asn1-0.2.6.tgz" - integrity sha512-ix/FxPn0MDjeyJ7i/yoHGFt/EX6LyNbxSEhPPXODPL+KB0VPk86UYfL0lMdy+KCnv+fmvIzySwaK5COwqVbWTQ== - dependencies: - safer-buffer "~2.1.0" - -ast-types@^0.13.4: - version "0.13.4" - resolved "https://registry.npmjs.org/ast-types/-/ast-types-0.13.4.tgz" - integrity sha512-x1FCFnFifvYDDzTaLII71vG5uvDwgtmDTEVWAxrgeiR8VjMONcCXJx7E+USjDtHlwFmt9MysbqgF9b9Vjr6w+w== - dependencies: - tslib "^2.0.1" - -async-function@^1.0.0: - version "1.0.0" - resolved "https://registry.npmjs.org/async-function/-/async-function-1.0.0.tgz" - integrity sha512-hsU18Ae8CDTR6Kgu9DYf0EbCr/a5iGL0rytQDobUcdpYOKokk8LEjVphnXkDkgpi0wYVsqrXuP0bZxJaTqdgoA== - -available-typed-arrays@^1.0.7: - version "1.0.7" - resolved "https://registry.npmjs.org/available-typed-arrays/-/available-typed-arrays-1.0.7.tgz" - integrity sha512-wvUjBtSGN7+7SjNpq/9M2Tg350UZD3q62IFZLbRAR1bSMlCo1ZaeW+BJ+D090e4hIIZLBcTDWe4Mh4jvUDajzQ== - dependencies: - possible-typed-array-names "^1.0.0" - -aws4@^1.12.0: - version "1.13.2" - resolved "https://registry.npmjs.org/aws4/-/aws4-1.13.2.tgz" - integrity sha512-lHe62zvbTB5eEABUVi/AwVh0ZKY9rMMDhmm+eeyuuUQbQ3+J+fONVQOZyj+DdrvD4BY33uYniyRJ4UJIaSKAfw== - -base64-js@^1.3.1: - version "1.5.1" - resolved "https://registry.npmjs.org/base64-js/-/base64-js-1.5.1.tgz" - integrity sha512-AKpaYlHn8t4SVbOHCy+b5+KKgvR4vrsD8vbvrbiQJps7fKDTkjkDry6ji0rUJjC0kzbNePLwzxq8iypo41qeWA== - -basic-ftp@^5.0.2: - version "5.0.5" - resolved "https://registry.npmjs.org/basic-ftp/-/basic-ftp-5.0.5.tgz" - integrity sha512-4Bcg1P8xhUuqcii/S0Z9wiHIrQVPMermM1any+MX5GeGD7faD3/msQUDGLol9wOcz4/jbg/WJnGqoJF6LiBdtg== - -bcrypt-pbkdf@^1.0.2: - version "1.0.2" - resolved "https://registry.npmjs.org/bcrypt-pbkdf/-/bcrypt-pbkdf-1.0.2.tgz" - integrity sha512-qeFIXtP4MSoi6NLqO12WfqARWWuCKi2Rn/9hJLEmtB5yTNr9DqFWkJRCf2qShWzPeAMRnOgCrq0sg/KLv5ES9w== - dependencies: - tweetnacl "^0.14.3" - -bindings@^1.5.0: - version "1.5.0" - resolved "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz" - integrity sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ== - dependencies: - file-uri-to-path "1.0.0" - -bl@^4.0.3: - version "4.1.0" - resolved "https://registry.npmjs.org/bl/-/bl-4.1.0.tgz" - integrity sha512-1W07cM9gS6DcLperZfFSj+bWLtaPGSOHWhPiGzXmvVJbRLdG82sH/Kn8EtW1VqWVA54AKf2h5k5BbnIbwF3h6w== - dependencies: - buffer "^5.5.0" - inherits "^2.0.4" - readable-stream "^3.4.0" - -body-parser@^2.2.1: - version "2.2.1" - resolved "https://registry.npmjs.org/body-parser/-/body-parser-2.2.1.tgz" - integrity sha512-nfDwkulwiZYQIGwxdy0RUmowMhKcFVcYXUU7m4QlKYim1rUtg83xm2yjZ40QjDuc291AJjjeSc9b++AWHSgSHw== - dependencies: - bytes "^3.1.2" - content-type "^1.0.5" - debug "^4.4.3" - http-errors "^2.0.0" - iconv-lite "^0.7.0" - on-finished "^2.4.1" - qs "^6.14.0" - raw-body "^3.0.1" - type-is "^2.0.1" - -bson@^6.10.4, "bson@^6.10.4 || ^7.0.0", bson@^6.7.0, bson@6.x: - version "6.10.4" - resolved "https://registry.npmjs.org/bson/-/bson-6.10.4.tgz" - integrity sha512-WIsKqkSC0ABoBJuT1LEX+2HEvNmNKKgnTAyd0fL8qzK4SH2i9NXg+t08YtdZp/V9IZ33cxe3iV4yM0qg8lMQng== - -buffer@^5.5.0: - version "5.7.1" - resolved "https://registry.npmjs.org/buffer/-/buffer-5.7.1.tgz" - integrity sha512-EHcyIPBQ4BSGlvjB16k5KgAJ27CIsHY/2JBmCRReo48y9rQ3MaUzWX3KVlBa4U7MyX02HdVj0K7C3WaB3ju7FQ== - dependencies: - base64-js "^1.3.1" - ieee754 "^1.1.13" - -bundle-name@^4.1.0: - version "4.1.0" - resolved "https://registry.npmjs.org/bundle-name/-/bundle-name-4.1.0.tgz" - integrity sha512-tjwM5exMg6BGRI+kNmTntNsvdZS1X8BFYS6tnJ2hdH0kVxM6/eVZ2xy+FqStSWvYmtfFMDLIxurorHwDKfDz5Q== - dependencies: - run-applescript "^7.0.0" - -bytes@^3.1.2, bytes@~3.1.2: - version "3.1.2" - resolved "https://registry.npmjs.org/bytes/-/bytes-3.1.2.tgz" - integrity sha512-/Nf7TyzTx6S3yRJObOAV7956r8cr2+Oj8AC5dt8wSP3BQAoeX58NoHyCU8P8zGkNXStjTSi6fzO6F0pBdcYbEg== - -call-bind-apply-helpers@^1.0.0, call-bind-apply-helpers@^1.0.1, call-bind-apply-helpers@^1.0.2: - version "1.0.2" - resolved "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz" - integrity sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ== - dependencies: - es-errors "^1.3.0" - function-bind "^1.1.2" - -call-bind@^1.0.7, call-bind@^1.0.8: - version "1.0.8" - resolved "https://registry.npmjs.org/call-bind/-/call-bind-1.0.8.tgz" - integrity sha512-oKlSFMcMwpUg2ednkhQ454wfWiU/ul3CkJe/PEHcTKuiX6RpbehUiFMXu13HalGZxfUwCQzZG747YXBn1im9ww== - dependencies: - call-bind-apply-helpers "^1.0.0" - es-define-property "^1.0.0" - get-intrinsic "^1.2.4" - set-function-length "^1.2.2" - -call-bound@^1.0.2, call-bound@^1.0.3, call-bound@^1.0.4: - version "1.0.4" - resolved "https://registry.npmjs.org/call-bound/-/call-bound-1.0.4.tgz" - integrity sha512-+ys997U96po4Kx/ABpBCqhA9EuxJaQWDQg7295H4hBphv3IZg0boBKuwYpt4YXp6MZ5AmZQnU/tyMTlRpaSejg== - dependencies: - call-bind-apply-helpers "^1.0.2" - get-intrinsic "^1.3.0" - -chownr@^1.1.1: - version "1.1.4" - resolved "https://registry.npmjs.org/chownr/-/chownr-1.1.4.tgz" - integrity sha512-jJ0bqzaylmJtVnNgzTeSOs8DPavpbYgEr/b0YL8/2GO3xJEhInFmhKMUnEJQjZumK7KXGFhUy89PrsJWlakBVg== - -cli-table@^0.3.4: - version "0.3.11" - resolved "https://registry.npmjs.org/cli-table/-/cli-table-0.3.11.tgz" - integrity sha512-IqLQi4lO0nIB4tcdTpN4LCB9FI3uqrJZK7RC515EnhZ6qBaglkIgICb1wjeAqpdoOabm1+SuQtkXIPdYC93jhQ== - dependencies: - colors "1.0.3" - -cliui@^8.0.1: - version "8.0.1" - resolved "https://registry.npmjs.org/cliui/-/cliui-8.0.1.tgz" - integrity sha512-BSeNnyus75C4//NQ9gQt1/csTXyo/8Sb+afLAkzAptFuMsod9HFokGNudZpi/oQV73hnVK+sR+5PVRMd+Dr7YQ== - dependencies: - string-width "^4.2.0" - strip-ansi "^6.0.1" - wrap-ansi "^7.0.0" - -color-convert@^2.0.1: - version "2.0.1" - resolved "https://registry.npmjs.org/color-convert/-/color-convert-2.0.1.tgz" - integrity sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ== - dependencies: - color-name "~1.1.4" - -color-name@~1.1.4: - version "1.1.4" - resolved "https://registry.npmjs.org/color-name/-/color-name-1.1.4.tgz" - integrity sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA== - -colors@1.0.3: - version "1.0.3" - resolved "https://registry.npmjs.org/colors/-/colors-1.0.3.tgz" - integrity sha512-pFGrxThWcWQ2MsAz6RtgeWe4NK2kUE1WfsrvvlctdII745EW9I0yflqhe7++M5LEc7bV2c/9/5zc8sFcpL0Drw== - -content-disposition@^1.0.0: - version "1.0.1" - resolved "https://registry.npmjs.org/content-disposition/-/content-disposition-1.0.1.tgz" - integrity sha512-oIXISMynqSqm241k6kcQ5UwttDILMK4BiurCfGEREw6+X9jkkpEe5T9FZaApyLGGOnFuyMWZpdolTXMtvEJ08Q== - -content-type@^1.0.5: - version "1.0.5" - resolved "https://registry.npmjs.org/content-type/-/content-type-1.0.5.tgz" - integrity sha512-nTjqfcBFEipKdXCv4YDQWCfmcLZKm81ldF0pAopTvyrFGVbcR6P/VAAd5G7N+0tTr8QqiU0tFadD6FK4NtJwOA== - -cookie-signature@^1.2.1: - version "1.2.2" - resolved "https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.2.2.tgz" - integrity sha512-D76uU73ulSXrD1UXF4KE2TMxVVwhsnCgfAyTg9k8P6KGZjlXKrOLe4dJQKI3Bxi5wjesZoFXJWElNWBjPZMbhg== - -cookie@^0.7.1: - version "0.7.2" - resolved "https://registry.npmjs.org/cookie/-/cookie-0.7.2.tgz" - integrity sha512-yki5XnKuf750l50uGTllt6kKILY4nQ1eNIQatoXEByZ5dWgnKqbnqmTrBE5B4N7lrMJKQ2ytWMiTO2o0v6Ew/w== - -cors@^2.8.5: - version "2.8.5" - resolved "https://registry.npmjs.org/cors/-/cors-2.8.5.tgz" - integrity sha512-KIHbLJqu73RGr/hnbrO9uBeixNGuvSQjul/jdFvS/KFSIH1hWVd1ng7zOHx+YrEfInLG7q4n6GHQ9cDtxv/P6g== - dependencies: - object-assign "^4" - vary "^1" - -cross-spawn@^7.0.5: - version "7.0.6" - resolved "https://registry.npmjs.org/cross-spawn/-/cross-spawn-7.0.6.tgz" - integrity sha512-uV2QOWP2nWzsy2aMp8aRibhi9dlzF5Hgh5SHaB9OiTGEyDTiJJyx0uy51QXdyWbtAHNua4XJzUKca3OzKUd3vA== - dependencies: - path-key "^3.1.0" - shebang-command "^2.0.0" - which "^2.0.1" - -data-uri-to-buffer@^4.0.0: - version "4.0.1" - resolved "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-4.0.1.tgz" - integrity sha512-0R9ikRb668HB7QDxT1vkpuUBtqc53YyAwMwGeUFKRojY/NWKvdZ+9UYtRfGmhqNbRkTSVpMbmyhXipFFv2cb/A== - -data-uri-to-buffer@^6.0.2: - version "6.0.2" - resolved "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-6.0.2.tgz" - integrity sha512-7hvf7/GW8e86rW0ptuwS3OcBGDjIi6SZva7hCyWC0yYry2cOPmLIjXAUHI6DK2HsnwJd9ifmt57i8eV2n4YNpw== - -data-view-buffer@^1.0.2: - version "1.0.2" - resolved "https://registry.npmjs.org/data-view-buffer/-/data-view-buffer-1.0.2.tgz" - integrity sha512-EmKO5V3OLXh1rtK2wgXRansaK1/mtVdTUEiEI0W8RkvgT05kfxaH29PliLnpLP73yYO6142Q72QNa8Wx/A5CqQ== - dependencies: - call-bound "^1.0.3" - es-errors "^1.3.0" - is-data-view "^1.0.2" - -data-view-byte-length@^1.0.2: - version "1.0.2" - resolved "https://registry.npmjs.org/data-view-byte-length/-/data-view-byte-length-1.0.2.tgz" - integrity sha512-tuhGbE6CfTM9+5ANGf+oQb72Ky/0+s3xKUpHvShfiz2RxMFgFPjsXuRLBVMtvMs15awe45SRb83D6wH4ew6wlQ== - dependencies: - call-bound "^1.0.3" - es-errors "^1.3.0" - is-data-view "^1.0.2" - -data-view-byte-offset@^1.0.1: - version "1.0.1" - resolved "https://registry.npmjs.org/data-view-byte-offset/-/data-view-byte-offset-1.0.1.tgz" - integrity sha512-BS8PfmtDGnrgYdOonGZQdLZslWIeCGFP9tpan0hi1Co2Zr2NKADsvGYA8XxuG/4UWgJ6Cjtv+YJnB6MM69QGlQ== - dependencies: - call-bound "^1.0.2" - es-errors "^1.3.0" - is-data-view "^1.0.1" - -debug@^4.3.4, debug@^4.3.5, debug@^4.4.0, debug@^4.4.3, debug@4: - version "4.4.3" - resolved "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz" - integrity sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA== - dependencies: - ms "^2.1.3" - -decompress-response@^6.0.0: - version "6.0.0" - resolved "https://registry.npmjs.org/decompress-response/-/decompress-response-6.0.0.tgz" - integrity sha512-aW35yZM6Bb/4oJlZncMH2LCoZtJXTRxES17vE3hoRiowU2kWHaJKFkSBDnDR+cm9J+9QhXmREyIfv0pji9ejCQ== - dependencies: - mimic-response "^3.1.0" - -deep-extend@^0.6.0: - version "0.6.0" - resolved "https://registry.npmjs.org/deep-extend/-/deep-extend-0.6.0.tgz" - integrity sha512-LOHxIOaPYdHlJRtCQfDIVZtfw/ufM8+rVj649RIHzcm/vGwQRXFt6OPqIFWsm2XEMrNIEtWR64sY1LEKD2vAOA== - -default-browser-id@^5.0.0: - version "5.0.1" - resolved "https://registry.npmjs.org/default-browser-id/-/default-browser-id-5.0.1.tgz" - integrity sha512-x1VCxdX4t+8wVfd1so/9w+vQ4vx7lKd2Qp5tDRutErwmR85OgmfX7RlLRMWafRMY7hbEiXIbudNrjOAPa/hL8Q== - -default-browser@^5.2.1: - version "5.4.0" - resolved "https://registry.npmjs.org/default-browser/-/default-browser-5.4.0.tgz" - integrity sha512-XDuvSq38Hr1MdN47EDvYtx3U0MTqpCEn+F6ft8z2vYDzMrvQhVp0ui9oQdqW3MvK3vqUETglt1tVGgjLuJ5izg== - dependencies: - bundle-name "^4.1.0" - default-browser-id "^5.0.0" - -define-data-property@^1.0.1, define-data-property@^1.1.4: - version "1.1.4" - resolved "https://registry.npmjs.org/define-data-property/-/define-data-property-1.1.4.tgz" - integrity sha512-rBMvIzlpA8v6E+SJZoo++HAYqsLrkg7MSfIinMPFhmkorw7X+dOXVJQs+QT69zGkzMyfDnIMN2Wid1+NbL3T+A== - dependencies: - es-define-property "^1.0.0" - es-errors "^1.3.0" - gopd "^1.0.1" - -define-lazy-prop@^3.0.0: - version "3.0.0" - resolved "https://registry.npmjs.org/define-lazy-prop/-/define-lazy-prop-3.0.0.tgz" - integrity sha512-N+MeXYoqr3pOgn8xfyRPREN7gHakLYjhsHhWGT3fWAiL4IkAt0iDw14QiiEm2bE30c5XX5q0FtAA3CK5f9/BUg== - -define-properties@^1.2.1: - version "1.2.1" - resolved "https://registry.npmjs.org/define-properties/-/define-properties-1.2.1.tgz" - integrity sha512-8QmQKqEASLd5nx0U1B1okLElbUuuttJ/AnYmRXbbbGDWh6uS208EjD4Xqq/I9wK7u0v6O08XhTWnt5XtEbR6Dg== - dependencies: - define-data-property "^1.0.1" - has-property-descriptors "^1.0.0" - object-keys "^1.1.1" - -degenerator@^5.0.0: - version "5.0.1" - resolved "https://registry.npmjs.org/degenerator/-/degenerator-5.0.1.tgz" - integrity sha512-TllpMR/t0M5sqCXfj85i4XaAzxmS5tVA16dqvdkMwGmzI+dXLXnw3J+3Vdv7VKw+ThlTMboK6i9rnZ6Nntj5CQ== - dependencies: - ast-types "^0.13.4" - escodegen "^2.1.0" - esprima "^4.0.1" - -depd@^2.0.0, depd@~2.0.0: - version "2.0.0" - resolved "https://registry.npmjs.org/depd/-/depd-2.0.0.tgz" - integrity sha512-g7nH6P6dyDioJogAAGprGpCtVImJhpPk/roCzdb3fIh61/s/nPsfR6onyMwkCAR/OlC3yBC0lESvUoQEAssIrw== - -detect-libc@^2.0.0: - version "2.1.2" - resolved "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz" - integrity sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ== - -dunder-proto@^1.0.0, dunder-proto@^1.0.1: - version "1.0.1" - resolved "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz" - integrity sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A== - dependencies: - call-bind-apply-helpers "^1.0.1" - es-errors "^1.3.0" - gopd "^1.2.0" - -ee-first@1.1.1: - version "1.1.1" - resolved "https://registry.npmjs.org/ee-first/-/ee-first-1.1.1.tgz" - integrity sha512-WMwm9LhRUo+WUaRN+vRuETqG89IgZphVSNkdFgeb6sS/E4OrDIN7t48CAewSHXc6C8lefD8KKfr5vY61brQlow== - -emoji-regex@^8.0.0: - version "8.0.0" - resolved "https://registry.npmjs.org/emoji-regex/-/emoji-regex-8.0.0.tgz" - integrity sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A== - -encodeurl@^2.0.0: - version "2.0.0" - resolved "https://registry.npmjs.org/encodeurl/-/encodeurl-2.0.0.tgz" - integrity sha512-Q0n9HRi4m6JuGIV1eFlmvJB7ZEVxu93IrMyiMsGC0lrMJMWzRgx6WGquyfQgZVb31vhGgXnfmPNNXmxnOkRBrg== - -end-of-stream@^1.1.0, end-of-stream@^1.4.1: - version "1.4.5" - resolved "https://registry.npmjs.org/end-of-stream/-/end-of-stream-1.4.5.tgz" - integrity sha512-ooEGc6HP26xXq/N+GCGOT0JKCLDGrq2bQUZrQ7gyrJiZANJ/8YDTxTpQBXGMn+WbIQXNVpyWymm7KYVICQnyOg== - dependencies: - once "^1.4.0" - -es-abstract@^1.23.3, es-abstract@^1.23.5, es-abstract@^1.23.9: - version "1.24.0" - resolved "https://registry.npmjs.org/es-abstract/-/es-abstract-1.24.0.tgz" - integrity sha512-WSzPgsdLtTcQwm4CROfS5ju2Wa1QQcVeT37jFjYzdFz1r9ahadC8B8/a4qxJxM+09F18iumCdRmlr96ZYkQvEg== - dependencies: - array-buffer-byte-length "^1.0.2" - arraybuffer.prototype.slice "^1.0.4" - available-typed-arrays "^1.0.7" - call-bind "^1.0.8" - call-bound "^1.0.4" - data-view-buffer "^1.0.2" - data-view-byte-length "^1.0.2" - data-view-byte-offset "^1.0.1" - es-define-property "^1.0.1" - es-errors "^1.3.0" - es-object-atoms "^1.1.1" - es-set-tostringtag "^2.1.0" - es-to-primitive "^1.3.0" - function.prototype.name "^1.1.8" - get-intrinsic "^1.3.0" - get-proto "^1.0.1" - get-symbol-description "^1.1.0" - globalthis "^1.0.4" - gopd "^1.2.0" - has-property-descriptors "^1.0.2" - has-proto "^1.2.0" - has-symbols "^1.1.0" - hasown "^2.0.2" - internal-slot "^1.1.0" - is-array-buffer "^3.0.5" - is-callable "^1.2.7" - is-data-view "^1.0.2" - is-negative-zero "^2.0.3" - is-regex "^1.2.1" - is-set "^2.0.3" - is-shared-array-buffer "^1.0.4" - is-string "^1.1.1" - is-typed-array "^1.1.15" - is-weakref "^1.1.1" - math-intrinsics "^1.1.0" - object-inspect "^1.13.4" - object-keys "^1.1.1" - object.assign "^4.1.7" - own-keys "^1.0.1" - regexp.prototype.flags "^1.5.4" - safe-array-concat "^1.1.3" - safe-push-apply "^1.0.0" - safe-regex-test "^1.1.0" - set-proto "^1.0.0" - stop-iteration-iterator "^1.1.0" - string.prototype.trim "^1.2.10" - string.prototype.trimend "^1.0.9" - string.prototype.trimstart "^1.0.8" - typed-array-buffer "^1.0.3" - typed-array-byte-length "^1.0.3" - typed-array-byte-offset "^1.0.4" - typed-array-length "^1.0.7" - unbox-primitive "^1.1.0" - which-typed-array "^1.1.19" - -es-define-property@^1.0.0, es-define-property@^1.0.1: - version "1.0.1" - resolved "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz" - integrity sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g== - -es-errors@^1.3.0: - version "1.3.0" - resolved "https://registry.npmjs.org/es-errors/-/es-errors-1.3.0.tgz" - integrity sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw== - -es-object-atoms@^1.0.0, es-object-atoms@^1.1.1: - version "1.1.1" - resolved "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.1.tgz" - integrity sha512-FGgH2h8zKNim9ljj7dankFPcICIK9Cp5bm+c2gQSYePhpaG5+esrLODihIorn+Pe6FGJzWhXQotPv73jTaldXA== - dependencies: - es-errors "^1.3.0" - -es-set-tostringtag@^2.1.0: - version "2.1.0" - resolved "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz" - integrity sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA== - dependencies: - es-errors "^1.3.0" - get-intrinsic "^1.2.6" - has-tostringtag "^1.0.2" - hasown "^2.0.2" - -es-to-primitive@^1.3.0: - version "1.3.0" - resolved "https://registry.npmjs.org/es-to-primitive/-/es-to-primitive-1.3.0.tgz" - integrity sha512-w+5mJ3GuFL+NjVtJlvydShqE1eN3h3PbI7/5LAsYJP/2qtuMXjfL2LpHSRqo4b4eSF5K/DH1JXKUAHSB2UW50g== - dependencies: - is-callable "^1.2.7" - is-date-object "^1.0.5" - is-symbol "^1.0.4" - -escalade@^3.1.1: - version "3.2.0" - resolved "https://registry.npmjs.org/escalade/-/escalade-3.2.0.tgz" - integrity sha512-WUj2qlxaQtO4g6Pq5c29GTcWGDyd8itL8zTlipgECz3JesAiiOKotd8JU6otB3PACgG6xkJUyVhboMS+bje/jA== - -escape-html@^1.0.3: - version "1.0.3" - resolved "https://registry.npmjs.org/escape-html/-/escape-html-1.0.3.tgz" - integrity sha512-NiSupZ4OeuGwr68lGIeym/ksIZMJodUGOSCZ/FSnTxcrekbvqrgdUxlJOMpijaKZVjAJrWrGs/6Jy8OMuyj9ow== - -escodegen@^2.1.0: - version "2.1.0" - resolved "https://registry.npmjs.org/escodegen/-/escodegen-2.1.0.tgz" - integrity sha512-2NlIDTwUWJN0mRPQOdtQBzbUHvdGY2P1VXSyU83Q3xKxM7WHX2Ql8dKq782Q9TgQUNOLEzEYu9bzLNj1q88I5w== - dependencies: - esprima "^4.0.1" - estraverse "^5.2.0" - esutils "^2.0.2" - optionalDependencies: - source-map "~0.6.1" - -esprima@^4.0.1: - version "4.0.1" - resolved "https://registry.npmjs.org/esprima/-/esprima-4.0.1.tgz" - integrity sha512-eGuFFw7Upda+g4p+QHvnW0RyTX/SVeJBDM/gCtMARO0cLuT2HcEKnTPvhjV6aGeqrCB/sbNop0Kszm0jsaWU4A== - -estraverse@^5.2.0: - version "5.3.0" - resolved "https://registry.npmjs.org/estraverse/-/estraverse-5.3.0.tgz" - integrity sha512-MMdARuVEQziNTeJD8DgMqmhwR11BRQ/cBP+pLtYdSTnf3MIO8fFeiINEbX36ZdNlfU/7A9f3gUw49B3oQsvwBA== - -esutils@^2.0.2: - version "2.0.3" - resolved "https://registry.npmjs.org/esutils/-/esutils-2.0.3.tgz" - integrity sha512-kVscqXk4OCp68SZ0dkgEKVi6/8ij300KBWTJq32P/dYeWTSwK41WyTxalN1eRmA5Z9UU/LX9D7FWSmV9SAYx6g== - -etag@^1.8.1: - version "1.8.1" - resolved "https://registry.npmjs.org/etag/-/etag-1.8.1.tgz" - integrity sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg== - -eventsource-parser@^3.0.0, eventsource-parser@^3.0.1, eventsource-parser@^3.0.6: - version "3.0.6" - resolved "https://registry.npmjs.org/eventsource-parser/-/eventsource-parser-3.0.6.tgz" - integrity sha512-Vo1ab+QXPzZ4tCa8SwIHJFaSzy4R6SHf7BY79rFBDf0idraZWAkYrDjDj8uWaSm3S2TK+hJ7/t1CEmZ7jXw+pg== - -eventsource@^3.0.2: - version "3.0.7" - resolved "https://registry.npmjs.org/eventsource/-/eventsource-3.0.7.tgz" - integrity sha512-CRT1WTyuQoD771GW56XEZFQ/ZoSfWid1alKGDYMmkt2yl8UXrVR4pspqWNEcqKvVIzg6PAltWjxcSSPrboA4iA== - dependencies: - eventsource-parser "^3.0.1" - -expand-template@^2.0.3: - version "2.0.3" - resolved "https://registry.npmjs.org/expand-template/-/expand-template-2.0.3.tgz" - integrity sha512-XYfuKMvj4O35f/pOXLObndIRvyQ+/+6AhODh+OKWj9S9498pHHn/IMszH+gt0fBCRWMNfk1ZSp5x3AifmnI2vg== - -express-rate-limit@^7.5.0: - version "7.5.1" - resolved "https://registry.npmjs.org/express-rate-limit/-/express-rate-limit-7.5.1.tgz" - integrity sha512-7iN8iPMDzOMHPUYllBEsQdWVB6fPDMPqwjBaFrgr4Jgr/+okjvzAy+UHlYYL/Vs0OsOrMkwS6PJDkFlJwoxUnw== - -express@^5.0.1, express@^5.1.0, "express@>= 4.11": - version "5.2.1" - resolved "https://registry.npmjs.org/express/-/express-5.2.1.tgz" - integrity sha512-hIS4idWWai69NezIdRt2xFVofaF4j+6INOpJlVOLDO8zXGpUVEVzIYk12UUi2JzjEzWL3IOAxcTubgz9Po0yXw== - dependencies: - accepts "^2.0.0" - body-parser "^2.2.1" - content-disposition "^1.0.0" - content-type "^1.0.5" - cookie "^0.7.1" - cookie-signature "^1.2.1" - debug "^4.4.0" - depd "^2.0.0" - encodeurl "^2.0.0" - escape-html "^1.0.3" - etag "^1.8.1" - finalhandler "^2.1.0" - fresh "^2.0.0" - http-errors "^2.0.0" - merge-descriptors "^2.0.0" - mime-types "^3.0.0" - on-finished "^2.4.1" - once "^1.4.0" - parseurl "^1.3.3" - proxy-addr "^2.0.7" - qs "^6.14.0" - range-parser "^1.2.1" - router "^2.2.0" - send "^1.1.0" - serve-static "^2.2.0" - statuses "^2.0.1" - type-is "^2.0.1" - vary "^1.1.2" - -fast-deep-equal@^3.1.3: - version "3.1.3" - resolved "https://registry.npmjs.org/fast-deep-equal/-/fast-deep-equal-3.1.3.tgz" - integrity sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q== - -fast-uri@^3.0.1: - version "3.1.0" - resolved "https://registry.npmjs.org/fast-uri/-/fast-uri-3.1.0.tgz" - integrity sha512-iPeeDKJSWf4IEOasVVrknXpaBV0IApz/gp7S2bb7Z4Lljbl2MGJRqInZiUrQwV16cpzw/D3S5j5Julj/gT52AA== - -fetch-blob@^3.1.2, fetch-blob@^3.1.4: - version "3.2.0" - resolved "https://registry.npmjs.org/fetch-blob/-/fetch-blob-3.2.0.tgz" - integrity sha512-7yAQpD2UMJzLi1Dqv7qFYnPbaPx7ZfFK6PiIxQ4PfkGPyNyl2Ugx+a/umUonmKqjhM4DnfbMvdX6otXq83soQQ== - dependencies: - node-domexception "^1.0.0" - web-streams-polyfill "^3.0.3" - -file-uri-to-path@1.0.0: - version "1.0.0" - resolved "https://registry.npmjs.org/file-uri-to-path/-/file-uri-to-path-1.0.0.tgz" - integrity sha512-0Zt+s3L7Vf1biwWZ29aARiVYLx7iMGnEUl9x33fbB/j3jR81u/O2LbqK+Bm1CDSNDKVtJ/YjwY7TUd5SkeLQLw== - -finalhandler@^2.1.0: - version "2.1.1" - resolved "https://registry.npmjs.org/finalhandler/-/finalhandler-2.1.1.tgz" - integrity sha512-S8KoZgRZN+a5rNwqTxlZZePjT/4cnm0ROV70LedRHZ0p8u9fRID0hJUZQpkKLzro8LfmC8sx23bY6tVNxv8pQA== - dependencies: - debug "^4.4.0" - encodeurl "^2.0.0" - escape-html "^1.0.3" - on-finished "^2.4.1" - parseurl "^1.3.3" - statuses "^2.0.1" - -for-each@^0.3.3, for-each@^0.3.5: - version "0.3.5" - resolved "https://registry.npmjs.org/for-each/-/for-each-0.3.5.tgz" - integrity sha512-dKx12eRCVIzqCxFGplyFKJMPvLEWgmNtUrpTiJIR5u97zEhRG8ySrtboPHZXx7daLxQVrl643cTzbab2tkQjxg== - dependencies: - is-callable "^1.2.7" - -formdata-polyfill@^4.0.10: - version "4.0.10" - resolved "https://registry.npmjs.org/formdata-polyfill/-/formdata-polyfill-4.0.10.tgz" - integrity sha512-buewHzMvYL29jdeQTVILecSaZKnt/RJWjoZCF5OW60Z67/GmSLBkOFM7qh1PI3zFNtJbaZL5eQu1vLfazOwj4g== - dependencies: - fetch-blob "^3.1.2" - -forwarded@0.2.0: - version "0.2.0" - resolved "https://registry.npmjs.org/forwarded/-/forwarded-0.2.0.tgz" - integrity sha512-buRG0fpBtRHSTCOASe6hD258tEubFoRLb4ZNA6NxMVHNw2gOcwHo9wyablzMzOA5z9xA9L1KNjk/Nt6MT9aYow== - -fresh@^2.0.0: - version "2.0.0" - resolved "https://registry.npmjs.org/fresh/-/fresh-2.0.0.tgz" - integrity sha512-Rx/WycZ60HOaqLKAi6cHRKKI7zxWbJ31MhntmtwMoaTeF7XFH9hhBp8vITaMidfljRQ6eYWCKkaTK+ykVJHP2A== - -fs-constants@^1.0.0: - version "1.0.0" - resolved "https://registry.npmjs.org/fs-constants/-/fs-constants-1.0.0.tgz" - integrity sha512-y6OAwoSIf7FyjMIv94u+b5rdheZEjzR63GTyZJm5qh4Bi+2YgwLCcI/fPFZkL5PSixOt6ZNKm+w+Hfp/Bciwow== - -function-bind@^1.1.2: - version "1.1.2" - resolved "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz" - integrity sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA== - -function.prototype.name@^1.1.6, function.prototype.name@^1.1.8: - version "1.1.8" - resolved "https://registry.npmjs.org/function.prototype.name/-/function.prototype.name-1.1.8.tgz" - integrity sha512-e5iwyodOHhbMr/yNrc7fDYG4qlbIvI5gajyzPnb5TCwyhjApznQh1BMFou9b30SevY43gCJKXycoCBjMbsuW0Q== - dependencies: - call-bind "^1.0.8" - call-bound "^1.0.3" - define-properties "^1.2.1" - functions-have-names "^1.2.3" - hasown "^2.0.2" - is-callable "^1.2.7" - -functions-have-names@^1.2.3: - version "1.2.3" - resolved "https://registry.npmjs.org/functions-have-names/-/functions-have-names-1.2.3.tgz" - integrity sha512-xckBUXyTIqT97tq2x2AMb+g163b5JFysYk0x4qxNFwbfQkmNZoiRHb6sPzI9/QV33WeuvVYBUIiD4NzNIyqaRQ== - -generator-function@^2.0.0: - version "2.0.1" - resolved "https://registry.npmjs.org/generator-function/-/generator-function-2.0.1.tgz" - integrity sha512-SFdFmIJi+ybC0vjlHN0ZGVGHc3lgE0DxPAT0djjVg+kjOnSqclqmj0KQ7ykTOLP6YxoqOvuAODGdcHJn+43q3g== - -get-caller-file@^2.0.5: - version "2.0.5" - resolved "https://registry.npmjs.org/get-caller-file/-/get-caller-file-2.0.5.tgz" - integrity sha512-DyFP3BM/3YHTQOCUL/w0OZHR0lpKeGrxotcHWcqNEdnltqFwXVfhEBQ94eIo34AfQpo0rGki4cyIiftY06h2Fg== - -get-intrinsic@^1.2.4, get-intrinsic@^1.2.5, get-intrinsic@^1.2.6, get-intrinsic@^1.2.7, get-intrinsic@^1.3.0: - version "1.3.0" - resolved "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.3.0.tgz" - integrity sha512-9fSjSaos/fRIVIp+xSJlE6lfwhES7LNtKaCBIamHsjr2na1BiABJPo0mOjjz8GJDURarmCPGqaiVg5mfjb98CQ== - dependencies: - call-bind-apply-helpers "^1.0.2" - es-define-property "^1.0.1" - es-errors "^1.3.0" - es-object-atoms "^1.1.1" - function-bind "^1.1.2" - get-proto "^1.0.1" - gopd "^1.2.0" - has-symbols "^1.1.0" - hasown "^2.0.2" - math-intrinsics "^1.1.0" - -get-proto@^1.0.1: - version "1.0.1" - resolved "https://registry.npmjs.org/get-proto/-/get-proto-1.0.1.tgz" - integrity sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g== - dependencies: - dunder-proto "^1.0.1" - es-object-atoms "^1.0.0" - -get-symbol-description@^1.1.0: - version "1.1.0" - resolved "https://registry.npmjs.org/get-symbol-description/-/get-symbol-description-1.1.0.tgz" - integrity sha512-w9UMqWwJxHNOvoNzSJ2oPF5wvYcvP7jUvYzhp67yEhTi17ZDBBC1z9pTdGuzjD+EFIqLSYRweZjqfiPzQ06Ebg== - dependencies: - call-bound "^1.0.3" - es-errors "^1.3.0" - get-intrinsic "^1.2.6" - -get-uri@^6.0.1: - version "6.0.5" - resolved "https://registry.npmjs.org/get-uri/-/get-uri-6.0.5.tgz" - integrity sha512-b1O07XYq8eRuVzBNgJLstU6FYc1tS6wnMtF1I1D9lE8LxZSOGZ7LhxN54yPP6mGw5f2CkXY2BQUL9Fx41qvcIg== - dependencies: - basic-ftp "^5.0.2" - data-uri-to-buffer "^6.0.2" - debug "^4.3.4" - -github-from-package@0.0.0: - version "0.0.0" - resolved "https://registry.npmjs.org/github-from-package/-/github-from-package-0.0.0.tgz" - integrity sha512-SyHy3T1v2NUXn29OsWdxmK6RwHD+vkj3v8en8AOBZ1wBQ/hCAQ5bAQTD02kW4W9tUp/3Qh6J8r9EvntiyCmOOw== - -globalthis@^1.0.4: - version "1.0.4" - resolved "https://registry.npmjs.org/globalthis/-/globalthis-1.0.4.tgz" - integrity sha512-DpLKbNU4WylpxJykQujfCcwYWiV/Jhm50Goo0wrVILAv5jOr9d+H+UR3PhSCD2rCCEIg0uc+G+muBTwD54JhDQ== - dependencies: - define-properties "^1.2.1" - gopd "^1.0.1" - -gopd@^1.0.1, gopd@^1.2.0: - version "1.2.0" - resolved "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz" - integrity sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg== - -has-bigints@^1.0.2: - version "1.1.0" - resolved "https://registry.npmjs.org/has-bigints/-/has-bigints-1.1.0.tgz" - integrity sha512-R3pbpkcIqv2Pm3dUwgjclDRVmWpTJW2DcMzcIhEXEx1oh/CEMObMm3KLmRJOdvhM7o4uQBnwr8pzRK2sJWIqfg== - -has-property-descriptors@^1.0.0, has-property-descriptors@^1.0.2: - version "1.0.2" - resolved "https://registry.npmjs.org/has-property-descriptors/-/has-property-descriptors-1.0.2.tgz" - integrity sha512-55JNKuIW+vq4Ke1BjOTjM2YctQIvCT7GFzHwmfZPGo5wnrgkid0YQtnAleFSqumZm4az3n2BS+erby5ipJdgrg== - dependencies: - es-define-property "^1.0.0" - -has-proto@^1.2.0: - version "1.2.0" - resolved "https://registry.npmjs.org/has-proto/-/has-proto-1.2.0.tgz" - integrity sha512-KIL7eQPfHQRC8+XluaIw7BHUwwqL19bQn4hzNgdr+1wXoU0KKj6rufu47lhY7KbJR2C6T6+PfyN0Ea7wkSS+qQ== - dependencies: - dunder-proto "^1.0.0" - -has-symbols@^1.0.3, has-symbols@^1.1.0: - version "1.1.0" - resolved "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz" - integrity sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ== - -has-tostringtag@^1.0.2: - version "1.0.2" - resolved "https://registry.npmjs.org/has-tostringtag/-/has-tostringtag-1.0.2.tgz" - integrity sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw== - dependencies: - has-symbols "^1.0.3" - -hasown@^2.0.2: - version "2.0.2" - resolved "https://registry.npmjs.org/hasown/-/hasown-2.0.2.tgz" - integrity sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ== - dependencies: - function-bind "^1.1.2" - -heap-js@^2.3.0: - version "2.7.1" - resolved "https://registry.npmjs.org/heap-js/-/heap-js-2.7.1.tgz" - integrity sha512-EQfezRg0NCZGNlhlDR3Evrw1FVL2G3LhU7EgPoxufQKruNBSYA8MiRPHeWbU+36o+Fhel0wMwM+sLEiBAlNLJA== - -http-errors@^2.0.0, http-errors@~2.0.1: - version "2.0.1" - resolved "https://registry.npmjs.org/http-errors/-/http-errors-2.0.1.tgz" - integrity sha512-4FbRdAX+bSdmo4AUFuS0WNiPz8NgFt+r8ThgNWmlrjQjt1Q7ZR9+zTlce2859x4KSXrwIsaeTqDoKQmtP8pLmQ== - dependencies: - depd "~2.0.0" - inherits "~2.0.4" - setprototypeof "~1.2.0" - statuses "~2.0.2" - toidentifier "~1.0.1" - -http-proxy-agent@^7.0.0, http-proxy-agent@^7.0.2: - version "7.0.2" - resolved "https://registry.npmjs.org/http-proxy-agent/-/http-proxy-agent-7.0.2.tgz" - integrity sha512-T1gkAiYYDWYx3V5Bmyu7HcfcvL7mUrTWiM6yOfa3PIphViJ/gFPbvidQ+veqSOHci/PxBcDabeUNCzpOODJZig== - dependencies: - agent-base "^7.1.0" - debug "^4.3.4" - -https-proxy-agent@^7.0.5, https-proxy-agent@^7.0.6: - version "7.0.6" - resolved "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-7.0.6.tgz" - integrity sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw== - dependencies: - agent-base "^7.1.2" - debug "4" - -iconv-lite@^0.7.0, iconv-lite@~0.7.0: - version "0.7.0" - resolved "https://registry.npmjs.org/iconv-lite/-/iconv-lite-0.7.0.tgz" - integrity sha512-cf6L2Ds3h57VVmkZe+Pn+5APsT7FpqJtEhhieDCvrE2MK5Qk9MyffgQyuxQTm6BChfeZNtcOLHp9IcWRVcIcBQ== - dependencies: - safer-buffer ">= 2.1.2 < 3.0.0" - -ieee754@^1.1.13: - version "1.2.1" - resolved "https://registry.npmjs.org/ieee754/-/ieee754-1.2.1.tgz" - integrity sha512-dcyqhDvX1C46lXZcVqCpK+FtMRQVdIMN6/Df5js2zouUsqG7I6sFxitIC+7KYK29KdXOLHdu9zL4sFnoVQnqaA== - -inherits@^2.0.3, inherits@^2.0.4, inherits@~2.0.4: - version "2.0.4" - resolved "https://registry.npmjs.org/inherits/-/inherits-2.0.4.tgz" - integrity sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ== - -ini@~1.3.0: - version "1.3.8" - resolved "https://registry.npmjs.org/ini/-/ini-1.3.8.tgz" - integrity sha512-JV/yugV2uzW5iMRSiZAyDtQd+nxtUnjeLt0acNdw98kKLrvuRVyB80tsREOE7yvGVgalhZ6RNXCmEHkUKBKxew== - -internal-slot@^1.1.0: - version "1.1.0" - resolved "https://registry.npmjs.org/internal-slot/-/internal-slot-1.1.0.tgz" - integrity sha512-4gd7VpWNQNB4UKKCFFVcp1AVv+FMOgs9NKzjHKusc8jTMhd5eL1NqQqOpE0KzMds804/yHlglp3uxgluOqAPLw== - dependencies: - es-errors "^1.3.0" - hasown "^2.0.2" - side-channel "^1.1.0" - -ip-address@^10.0.1: - version "10.1.0" - resolved "https://registry.npmjs.org/ip-address/-/ip-address-10.1.0.tgz" - integrity sha512-XXADHxXmvT9+CRxhXg56LJovE+bmWnEWB78LB83VZTprKTmaC5QfruXocxzTZ2Kl0DNwKuBdlIhjL8LeY8Sf8Q== - -ip-address@^9.0.5: - version "9.0.5" - resolved "https://registry.npmjs.org/ip-address/-/ip-address-9.0.5.tgz" - integrity sha512-zHtQzGojZXTwZTHQqra+ETKd4Sn3vgi7uBmlPoXVWZqYvuKmtI0l/VZTjqGmJY9x88GGOaZ9+G9ES8hC4T4X8g== - dependencies: - jsbn "1.1.0" - sprintf-js "^1.1.3" - -ipaddr.js@1.9.1: - version "1.9.1" - resolved "https://registry.npmjs.org/ipaddr.js/-/ipaddr.js-1.9.1.tgz" - integrity sha512-0KI/607xoxSToH7GjN1FfSbLoU0+btTicjsQSWQlh/hZykN8KpmMf7uYwPW3R+akZ6R/w18ZlXSHBYXiYUPO3g== - -is-array-buffer@^3.0.4, is-array-buffer@^3.0.5: - version "3.0.5" - resolved "https://registry.npmjs.org/is-array-buffer/-/is-array-buffer-3.0.5.tgz" - integrity sha512-DDfANUiiG2wC1qawP66qlTugJeL5HyzMpfr8lLK+jMQirGzNod0B12cFB/9q838Ru27sBwfw78/rdoU7RERz6A== - dependencies: - call-bind "^1.0.8" - call-bound "^1.0.3" - get-intrinsic "^1.2.6" - -is-async-function@^2.0.0: - version "2.1.1" - resolved "https://registry.npmjs.org/is-async-function/-/is-async-function-2.1.1.tgz" - integrity sha512-9dgM/cZBnNvjzaMYHVoxxfPj2QXt22Ev7SuuPrs+xav0ukGB0S6d4ydZdEiM48kLx5kDV+QBPrpVnFyefL8kkQ== - dependencies: - async-function "^1.0.0" - call-bound "^1.0.3" - get-proto "^1.0.1" - has-tostringtag "^1.0.2" - safe-regex-test "^1.1.0" - -is-bigint@^1.1.0: - version "1.1.0" - resolved "https://registry.npmjs.org/is-bigint/-/is-bigint-1.1.0.tgz" - integrity sha512-n4ZT37wG78iz03xPRKJrHTdZbe3IicyucEtdRsV5yglwc3GyUfbAfpSeD0FJ41NbUNSt5wbhqfp1fS+BgnvDFQ== - dependencies: - has-bigints "^1.0.2" - -is-boolean-object@^1.2.1: - version "1.2.2" - resolved "https://registry.npmjs.org/is-boolean-object/-/is-boolean-object-1.2.2.tgz" - integrity sha512-wa56o2/ElJMYqjCjGkXri7it5FbebW5usLw/nPmCMs5DeZ7eziSYZhSmPRn0txqeW4LnAmQQU7FgqLpsEFKM4A== - dependencies: - call-bound "^1.0.3" - has-tostringtag "^1.0.2" - -is-callable@^1.2.7: - version "1.2.7" - resolved "https://registry.npmjs.org/is-callable/-/is-callable-1.2.7.tgz" - integrity sha512-1BC0BVFhS/p0qtw6enp8e+8OD0UrK0oFLztSjNzhcKA3WDuJxxAPXzPuPtKkjEY9UUoEWlX/8fgKeu2S8i9JTA== - -is-data-view@^1.0.1, is-data-view@^1.0.2: - version "1.0.2" - resolved "https://registry.npmjs.org/is-data-view/-/is-data-view-1.0.2.tgz" - integrity sha512-RKtWF8pGmS87i2D6gqQu/l7EYRlVdfzemCJN/P3UOs//x1QE7mfhvzHIApBTRf7axvT6DMGwSwBXYCT0nfB9xw== - dependencies: - call-bound "^1.0.2" - get-intrinsic "^1.2.6" - is-typed-array "^1.1.13" - -is-date-object@^1.0.5, is-date-object@^1.1.0: - version "1.1.0" - resolved "https://registry.npmjs.org/is-date-object/-/is-date-object-1.1.0.tgz" - integrity sha512-PwwhEakHVKTdRNVOw+/Gyh0+MzlCl4R6qKvkhuvLtPMggI1WAHt9sOwZxQLSGpUaDnrdyDsomoRgNnCfKNSXXg== - dependencies: - call-bound "^1.0.2" - has-tostringtag "^1.0.2" - -is-docker@^3.0.0: - version "3.0.0" - resolved "https://registry.npmjs.org/is-docker/-/is-docker-3.0.0.tgz" - integrity sha512-eljcgEDlEns/7AXFosB5K/2nCM4P7FQPkGc/DWLy5rmFEWvZayGrik1d9/QIY5nJ4f9YsVvBkA6kJpHn9rISdQ== - -is-finalizationregistry@^1.1.0: - version "1.1.1" - resolved "https://registry.npmjs.org/is-finalizationregistry/-/is-finalizationregistry-1.1.1.tgz" - integrity sha512-1pC6N8qWJbWoPtEjgcL2xyhQOP491EQjeUo3qTKcmV8YSDDJrOepfG8pcC7h/QgnQHYSv0mJ3Z/ZWxmatVrysg== - dependencies: - call-bound "^1.0.3" - -is-fullwidth-code-point@^3.0.0: - version "3.0.0" - resolved "https://registry.npmjs.org/is-fullwidth-code-point/-/is-fullwidth-code-point-3.0.0.tgz" - integrity sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg== - -is-generator-function@^1.0.10: - version "1.1.2" - resolved "https://registry.npmjs.org/is-generator-function/-/is-generator-function-1.1.2.tgz" - integrity sha512-upqt1SkGkODW9tsGNG5mtXTXtECizwtS2kA161M+gJPc1xdb/Ax629af6YrTwcOeQHbewrPNlE5Dx7kzvXTizA== - dependencies: - call-bound "^1.0.4" - generator-function "^2.0.0" - get-proto "^1.0.1" - has-tostringtag "^1.0.2" - safe-regex-test "^1.1.0" - -is-inside-container@^1.0.0: - version "1.0.0" - resolved "https://registry.npmjs.org/is-inside-container/-/is-inside-container-1.0.0.tgz" - integrity sha512-KIYLCCJghfHZxqjYBE7rEy0OBuTd5xCHS7tHVgvCLkx7StIoaxwNW3hCALgEUjFfeRk+MG/Qxmp/vtETEF3tRA== - dependencies: - is-docker "^3.0.0" - -is-map@^2.0.3: - version "2.0.3" - resolved "https://registry.npmjs.org/is-map/-/is-map-2.0.3.tgz" - integrity sha512-1Qed0/Hr2m+YqxnM09CjA2d/i6YZNfF6R2oRAOj36eUdS6qIV/huPJNSEpKbupewFs+ZsJlxsjjPbc0/afW6Lw== - -is-negative-zero@^2.0.3: - version "2.0.3" - resolved "https://registry.npmjs.org/is-negative-zero/-/is-negative-zero-2.0.3.tgz" - integrity sha512-5KoIu2Ngpyek75jXodFvnafB6DJgr3u8uuK0LEZJjrU19DrMD3EVERaR8sjz8CCGgpZvxPl9SuE1GMVPFHx1mw== - -is-number-object@^1.1.1: - version "1.1.1" - resolved "https://registry.npmjs.org/is-number-object/-/is-number-object-1.1.1.tgz" - integrity sha512-lZhclumE1G6VYD8VHe35wFaIif+CTy5SJIi5+3y4psDgWu4wPDoBhF8NxUOinEc7pHgiTsT6MaBb92rKhhD+Xw== - dependencies: - call-bound "^1.0.3" - has-tostringtag "^1.0.2" - -is-promise@^4.0.0: - version "4.0.0" - resolved "https://registry.npmjs.org/is-promise/-/is-promise-4.0.0.tgz" - integrity sha512-hvpoI6korhJMnej285dSg6nu1+e6uxs7zG3BYAm5byqDsgJNWwxzM6z6iZiAgQR4TJ30JmBTOwqZUw3WlyH3AQ== - -is-regex@^1.2.1: - version "1.2.1" - resolved "https://registry.npmjs.org/is-regex/-/is-regex-1.2.1.tgz" - integrity sha512-MjYsKHO5O7mCsmRGxWcLWheFqN9DJ/2TmngvjKXihe6efViPqc274+Fx/4fYj/r03+ESvBdTXK0V6tA3rgez1g== - dependencies: - call-bound "^1.0.2" - gopd "^1.2.0" - has-tostringtag "^1.0.2" - hasown "^2.0.2" - -is-set@^2.0.3: - version "2.0.3" - resolved "https://registry.npmjs.org/is-set/-/is-set-2.0.3.tgz" - integrity sha512-iPAjerrse27/ygGLxw+EBR9agv9Y6uLeYVJMu+QNCoouJ1/1ri0mGrcWpfCqFZuzzx3WjtwxG098X+n4OuRkPg== - -is-shared-array-buffer@^1.0.4: - version "1.0.4" - resolved "https://registry.npmjs.org/is-shared-array-buffer/-/is-shared-array-buffer-1.0.4.tgz" - integrity sha512-ISWac8drv4ZGfwKl5slpHG9OwPNty4jOWPRIhBpxOoD+hqITiwuipOQ2bNthAzwA3B4fIjO4Nln74N0S9byq8A== - dependencies: - call-bound "^1.0.3" - -is-string@^1.1.1: - version "1.1.1" - resolved "https://registry.npmjs.org/is-string/-/is-string-1.1.1.tgz" - integrity sha512-BtEeSsoaQjlSPBemMQIrY1MY0uM6vnS1g5fmufYOtnxLGUZM2178PKbhsk7Ffv58IX+ZtcvoGwccYsh0PglkAA== - dependencies: - call-bound "^1.0.3" - has-tostringtag "^1.0.2" - -is-symbol@^1.0.4, is-symbol@^1.1.1: - version "1.1.1" - resolved "https://registry.npmjs.org/is-symbol/-/is-symbol-1.1.1.tgz" - integrity sha512-9gGx6GTtCQM73BgmHQXfDmLtfjjTUDSyoxTCbp5WtoixAhfgsDirWIcVQ/IHpvI5Vgd5i/J5F7B9cN/WlVbC/w== - dependencies: - call-bound "^1.0.2" - has-symbols "^1.1.0" - safe-regex-test "^1.1.0" - -is-typed-array@^1.1.13, is-typed-array@^1.1.14, is-typed-array@^1.1.15: - version "1.1.15" - resolved "https://registry.npmjs.org/is-typed-array/-/is-typed-array-1.1.15.tgz" - integrity sha512-p3EcsicXjit7SaskXHs1hA91QxgTw46Fv6EFKKGS5DRFLD8yKnohjF3hxoju94b/OcMZoQukzpPpBE9uLVKzgQ== - dependencies: - which-typed-array "^1.1.16" - -is-weakmap@^2.0.2: - version "2.0.2" - resolved "https://registry.npmjs.org/is-weakmap/-/is-weakmap-2.0.2.tgz" - integrity sha512-K5pXYOm9wqY1RgjpL3YTkF39tni1XajUIkawTLUo9EZEVUFga5gSQJF8nNS7ZwJQ02y+1YCNYcMh+HIf1ZqE+w== - -is-weakref@^1.0.2, is-weakref@^1.1.1: - version "1.1.1" - resolved "https://registry.npmjs.org/is-weakref/-/is-weakref-1.1.1.tgz" - integrity sha512-6i9mGWSlqzNMEqpCp93KwRS1uUOodk2OJ6b+sq7ZPDSy2WuI5NFIxp/254TytR8ftefexkWn5xNiHUNpPOfSew== - dependencies: - call-bound "^1.0.3" - -is-weakset@^2.0.3: - version "2.0.4" - resolved "https://registry.npmjs.org/is-weakset/-/is-weakset-2.0.4.tgz" - integrity sha512-mfcwb6IzQyOKTs84CQMrOwW4gQcaTOAWJ0zzJCl2WSPDrWk/OzDaImWFH3djXhb24g4eudZfLRozAvPGw4d9hQ== - dependencies: - call-bound "^1.0.3" - get-intrinsic "^1.2.6" - -is-wsl@^3.1.0: - version "3.1.0" - resolved "https://registry.npmjs.org/is-wsl/-/is-wsl-3.1.0.tgz" - integrity sha512-UcVfVfaK4Sc4m7X3dUSoHoozQGBEFeDC+zVo06t98xe8CzHSZZBekNXH+tu0NalHolcJ/QAGqS46Hef7QXBIMw== - dependencies: - is-inside-container "^1.0.0" - -isarray@^2.0.5: - version "2.0.5" - resolved "https://registry.npmjs.org/isarray/-/isarray-2.0.5.tgz" - integrity sha512-xHjhDr3cNBK0BzdUJSPXZntQUx/mwMS5Rw4A7lPJ90XGAO6ISP/ePDNuo0vhqOZU+UD5JoodwCAAoZQd3FeAKw== - -isexe@^2.0.0: - version "2.0.0" - resolved "https://registry.npmjs.org/isexe/-/isexe-2.0.0.tgz" - integrity sha512-RHxMLp9lnKHGHRng9QFhRCMbYAcVpn69smSGcq3f36xjgVVWThj4qqLbTLlq7Ssj8B+fIQ1EuCEGI2lKsyQeIw== - -isnumber@~1.0.0: - version "1.0.0" - resolved "https://registry.npmjs.org/isnumber/-/isnumber-1.0.0.tgz" - integrity sha512-JLiSz/zsZcGFXPrB4I/AGBvtStkt+8QmksyZBZnVXnnK9XdTEyz0tX8CRYljtwYDuIuZzih6DpHQdi+3Q6zHPw== - -jose@^6.1.0, jose@^6.1.1: - version "6.1.3" - resolved "https://registry.npmjs.org/jose/-/jose-6.1.3.tgz" - integrity sha512-0TpaTfihd4QMNwrz/ob2Bp7X04yuxJkjRGi4aKmOqwhov54i6u79oCv7T+C7lo70MKH6BesI3vscD1yb/yzKXQ== - -js-yaml@^4.0.0: - version "4.1.1" - resolved "https://registry.npmjs.org/js-yaml/-/js-yaml-4.1.1.tgz" - integrity sha512-qQKT4zQxXl8lLwBtHMWwaTcGfFOZviOJet3Oy/xmGk2gZH677CJM9EvtfdSkgWcATZhj/55JZ0rmy3myCT5lsA== - dependencies: - argparse "^2.0.1" - -jsbn@1.1.0: - version "1.1.0" - resolved "https://registry.npmjs.org/jsbn/-/jsbn-1.1.0.tgz" - integrity sha512-4bYVV3aAMtDTTu4+xsDYa6sy9GyJ69/amsu9sYF2zqjiEoZA5xJi3BrfX3uY+/IekIu7MwdObdbDWpoZdBv3/A== - -json-schema-traverse@^1.0.0: - version "1.0.0" - resolved "https://registry.npmjs.org/json-schema-traverse/-/json-schema-traverse-1.0.0.tgz" - integrity sha512-NM8/P9n3XjXhIZn1lLhkFaACTOURQXjWhV4BA/RnOv8xvgqtqpAX9IO4mRQxSx1Rlo4tqzeqb0sOlruaOy3dug== - -json-schema@^0.4.0: - version "0.4.0" - resolved "https://registry.npmjs.org/json-schema/-/json-schema-0.4.0.tgz" - integrity sha512-es94M3nTIfsEPisRafak+HDLfHXnKBhV3vU5eqPcS3flIWqcxJWgXHXiey3YrpaNsanY5ei1VoYEbOzijuq9BA== - -kerberos@^2.0.1, kerberos@^2.1.0, kerberos@^2.2.2: - version "2.2.2" - resolved "https://registry.npmjs.org/kerberos/-/kerberos-2.2.2.tgz" - integrity sha512-42O7+/1Zatsc3MkxaMPpXcIl/ukIrbQaGoArZEAr6GcEi2qhfprOBYOPhj+YvSMJkEkdpTjApUx+2DuWaKwRhg== - dependencies: - node-addon-api "^6.1.0" - prebuild-install "^7.1.2" - -kerberos@2.1.0: - version "2.1.0" - resolved "https://registry.npmjs.org/kerberos/-/kerberos-2.1.0.tgz" - integrity sha512-HvOl6O6cyEN/8Z4CAocHe/sekJtvt5UrxUdCuu7bXDZ2Hnsy6OpsQbISW+lpm03vrbO2ir+1QQ5Sx/vMEhHnog== - dependencies: - bindings "^1.5.0" - node-addon-api "^6.1.0" - prebuild-install "7.1.1" - -lodash.merge@^4.6.2: - version "4.6.2" - resolved "https://registry.npmjs.org/lodash.merge/-/lodash.merge-4.6.2.tgz" - integrity sha512-0KpjqXRVvrYyCsX1swR/XTK0va6VQkQM6MNo7PqW77ByjAhoARA8EfrP1N4+KlKj8YS0ZUCtRT/YUuhyYDujIQ== - -lru-cache@^11.0.0, lru-cache@^11.1.0: - version "11.2.4" - resolved "https://registry.npmjs.org/lru-cache/-/lru-cache-11.2.4.tgz" - integrity sha512-B5Y16Jr9LB9dHVkh6ZevG+vAbOsNOYCX+sXvFWFu7B3Iz5mijW3zdbMyhsh8ANd2mSWBYdJgnqi+mL7/LrOPYg== - -math-intrinsics@^1.1.0: - version "1.1.0" - resolved "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz" - integrity sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g== - -media-typer@^1.1.0: - version "1.1.0" - resolved "https://registry.npmjs.org/media-typer/-/media-typer-1.1.0.tgz" - integrity sha512-aisnrDP4GNe06UcKFnV5bfMNPBUw4jsLGaWwWfnH3v02GnBuXX2MCVn5RbrWo0j3pczUilYblq7fQ7Nw2t5XKw== - -memory-pager@^1.0.2: - version "1.5.0" - resolved "https://registry.npmjs.org/memory-pager/-/memory-pager-1.5.0.tgz" - integrity sha512-ZS4Bp4r/Zoeq6+NLJpP+0Zzm0pR8whtGPf1XExKLJBAczGMnSi3It14OiNCStjQjM6NU1okjQGSxgEZN8eBYKg== - -merge-descriptors@^2.0.0: - version "2.0.0" - resolved "https://registry.npmjs.org/merge-descriptors/-/merge-descriptors-2.0.0.tgz" - integrity sha512-Snk314V5ayFLhp3fkUREub6WtjBfPdCPY1Ln8/8munuLuiYhsABgBVWsozAG+MWMbVEvcdcpbi9R7ww22l9Q3g== - -mime-db@^1.54.0: - version "1.54.0" - resolved "https://registry.npmjs.org/mime-db/-/mime-db-1.54.0.tgz" - integrity sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ== - -mime-types@^3.0.0, mime-types@^3.0.1: - version "3.0.2" - resolved "https://registry.npmjs.org/mime-types/-/mime-types-3.0.2.tgz" - integrity sha512-Lbgzdk0h4juoQ9fCKXW4by0UJqj+nOOrI9MJ1sSj4nI8aI2eo1qmvQEie4VD1glsS250n15LsWsYtCugiStS5A== - dependencies: - mime-db "^1.54.0" - -mimic-response@^3.1.0: - version "3.1.0" - resolved "https://registry.npmjs.org/mimic-response/-/mimic-response-3.1.0.tgz" - integrity sha512-z0yWI+4FDrrweS8Zmt4Ej5HdJmky15+L2e6Wgn3+iK5fWzb6T3fhNFq2+MeTRb064c6Wr4N/wv0DzQTjNzHNGQ== - -minimist@^1.2.0, minimist@^1.2.3: - version "1.2.8" - resolved "https://registry.npmjs.org/minimist/-/minimist-1.2.8.tgz" - integrity sha512-2yyAR8qBkN3YuheJanUpWC5U3bb5osDywNB8RzDVlDwDHbocAJveqqj1u8+SVD7jkWT4yvsHCpWqqWqAxb0zCA== - -mkdirp-classic@^0.5.2, mkdirp-classic@^0.5.3: - version "0.5.3" - resolved "https://registry.npmjs.org/mkdirp-classic/-/mkdirp-classic-0.5.3.tgz" - integrity sha512-gKLcREMhtuZRwRAfqP3RFW+TK4JqApVBtOIftVgjuABpAtpxhPGaDcfvbhNvD0B8iD1oUr/txX35NjcaY6Ns/A== - -mongodb-build-info@^1.8.1: - version "1.8.2" - resolved "https://registry.npmjs.org/mongodb-build-info/-/mongodb-build-info-1.8.2.tgz" - integrity sha512-VJ5iB7ca3bxSyBKQGNVkGszF7U3U0IP71Kym0Mw1Y7EquCIFgbBVdEgwpLa3R02les41TXIo+x4D+pUXO6ttrw== - dependencies: - debug "^4.4.0" - mongodb-connection-string-url "^3.0.0" - -mongodb-client-encryption@^6.1.0, mongodb-client-encryption@^6.5.0, "mongodb-client-encryption@>=6.0.0 <7": - version "6.5.0" - resolved "https://registry.npmjs.org/mongodb-client-encryption/-/mongodb-client-encryption-6.5.0.tgz" - integrity sha512-Gj8EeyYKsssdko0NKhWRBGDif6uVFBbv+e+Nyn7E316UmRzApc4IP+p2NLm+av+fU+dFHVT5WqfzaQVDTh8i9w== - dependencies: - node-addon-api "^4.3.0" - prebuild-install "^7.1.3" - -mongodb-connection-string-url@^3.0.0, "mongodb-connection-string-url@^3.0.1 || ^7.0.0", mongodb-connection-string-url@^3.0.2: - version "3.0.2" - resolved "https://registry.npmjs.org/mongodb-connection-string-url/-/mongodb-connection-string-url-3.0.2.tgz" - integrity sha512-rMO7CGo/9BFwyZABcKAWL8UJwH/Kc2x0g72uhDWzG48URRax5TCIcJ7Rc3RZqffZzO/Gwff/jyKwCU9TN8gehA== - dependencies: - "@types/whatwg-url" "^11.0.2" - whatwg-url "^14.1.0 || ^13.0.0" - -mongodb-log-writer@^2.4.1, mongodb-log-writer@^2.4.4: - version "2.4.4" - resolved "https://registry.npmjs.org/mongodb-log-writer/-/mongodb-log-writer-2.4.4.tgz" - integrity sha512-vGOzdF9ta+wUiHsqbEcRQ1COzTIKXy4vhOWkPCvOacWCeNsFS7LLp6Vg9LT95Z/09gSjnD1WDeYhxTqZXEgcWQ== - dependencies: - heap-js "^2.3.0" - -mongodb-mcp-server@^1.3.0: - version "1.3.0" - resolved "https://registry.npmjs.org/mongodb-mcp-server/-/mongodb-mcp-server-1.3.0.tgz" - integrity sha512-uN1fs+N8NyC/AncvZdb2LGOTSeOXkB3D6A1GlDB+U932lxsp76WNRVziVF9SNfwOo+6E0vvk9aS1mqgR5tSrXw== - dependencies: - "@modelcontextprotocol/sdk" "^1.24.2" - "@mongodb-js/device-id" "^0.3.1" - "@mongodb-js/devtools-proxy-support" "^0.5.3" - "@mongosh/arg-parser" "^3.23.0" - "@mongosh/service-provider-node-driver" "^3.17.5" - ai "^5.0.72" - bson "^6.10.4" - express "^5.1.0" - lru-cache "^11.1.0" - mongodb-connection-string-url "^3.0.2" - mongodb-log-writer "^2.4.1" - mongodb-redact "^1.3.0" - mongodb-schema "^12.6.2" - node-fetch "^3.3.2" - node-machine-id "1.1.12" - oauth4webapi "^3.8.0" - openapi-fetch "^0.15.0" - ts-levenshtein "^1.0.7" - voyage-ai-provider "^2.0.0" - zod "^3.25.76" - optionalDependencies: - "@mongodb-js/atlas-local" "^1.1.0" - kerberos "^2.2.2" - -mongodb-ns@^3.0.1: - version "3.0.3" - resolved "https://registry.npmjs.org/mongodb-ns/-/mongodb-ns-3.0.3.tgz" - integrity sha512-ctpHlSXGYlIim3JSgcsQjNkgB4CWvwWSFIxKFphSY/MoY/2i6E5rtVkOt1FBbSjEJYa2XvD4c+G2BbLfjhQqaQ== - -mongodb-redact@^1.3.0: - version "1.3.0" - resolved "https://registry.npmjs.org/mongodb-redact/-/mongodb-redact-1.3.0.tgz" - integrity sha512-6qMkQ9RnB7Z92G8c4mCkrDGaqwekuaUreaX+XnjAl/t3oKHJ+u7C+kfFLxSueJIBc1qtI4AtK0TO4yxOHyJqTw== - dependencies: - mongodb-connection-string-url "^3.0.1 || ^7.0.0" - regexp.escape "^2.0.1" - -mongodb-schema@^12.6.2: - version "12.6.3" - resolved "https://registry.npmjs.org/mongodb-schema/-/mongodb-schema-12.6.3.tgz" - integrity sha512-JiAZtM9GVMTLJYJpEnAPq0/ulH9U7qBR48Bx0mOiStVGFkY3mpIlgEGOl5tVRLEvCxDKqnvtdfSSX7pWFRLlzA== - dependencies: - reservoir "^0.1.2" - optionalDependencies: - bson "^6.7.0" - cli-table "^0.3.4" - js-yaml "^4.0.0" - mongodb "^6.6.1" - mongodb-ns "^3.0.1" - numeral "^2.0.6" - progress "^2.0.3" - stats-lite "^2.0.0" - yargs "^17.6.2" - -mongodb@^6.19.0, mongodb@^6.6.1, mongodb@^6.9.0: - version "6.21.0" - resolved "https://registry.npmjs.org/mongodb/-/mongodb-6.21.0.tgz" - integrity sha512-URyb/VXMjJ4da46OeSXg+puO39XH9DeQpWCslifrRn9JWugy0D+DvvBvkm2WxmHe61O/H19JM66p1z7RHVkZ6A== - dependencies: - "@mongodb-js/saslprep" "^1.3.0" - bson "^6.10.4" - mongodb-connection-string-url "^3.0.2" - -ms@^2.1.3: - version "2.1.3" - resolved "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz" - integrity sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA== - -napi-build-utils@^1.0.1: - version "1.0.2" - resolved "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-1.0.2.tgz" - integrity sha512-ONmRUqK7zj7DWX0D9ADe03wbwOBZxNAfF20PlGfCWQcD3+/MakShIHrMqx9YwPTfxDdF1zLeL+RGZiR9kGMLdg== - -napi-build-utils@^2.0.0: - version "2.0.0" - resolved "https://registry.npmjs.org/napi-build-utils/-/napi-build-utils-2.0.0.tgz" - integrity sha512-GEbrYkbfF7MoNaoh2iGG84Mnf/WZfB0GdGEsM8wz7Expx/LlWf5U8t9nvJKXSp3qr5IsEbK04cBGhol/KwOsWA== - -negotiator@^1.0.0: - version "1.0.0" - resolved "https://registry.npmjs.org/negotiator/-/negotiator-1.0.0.tgz" - integrity sha512-8Ofs/AUQh8MaEcrlq5xOX0CQ9ypTF5dl78mjlMNfOK08fzpgTHQRQPBxcPlEtIw0yRpws+Zo/3r+5WRby7u3Gg== - -netmask@^2.0.2: - version "2.0.2" - resolved "https://registry.npmjs.org/netmask/-/netmask-2.0.2.tgz" - integrity sha512-dBpDMdxv9Irdq66304OLfEmQ9tbNRFnFTuZiLo+bD+r332bBmMJ8GBLXklIXXgxd3+v9+KUnZaUR5PJMa75Gsg== - -node-abi@^3.3.0: - version "3.85.0" - resolved "https://registry.npmjs.org/node-abi/-/node-abi-3.85.0.tgz" - integrity sha512-zsFhmbkAzwhTft6nd3VxcG0cvJsT70rL+BIGHWVq5fi6MwGrHwzqKaxXE+Hl2GmnGItnDKPPkO5/LQqjVkIdFg== - dependencies: - semver "^7.3.5" - -node-addon-api@^4.3.0: - version "4.3.0" - resolved "https://registry.npmjs.org/node-addon-api/-/node-addon-api-4.3.0.tgz" - integrity sha512-73sE9+3UaLYYFmDsFZnqCInzPyh3MqIwZO9cw58yIqAZhONrrabrYyYe3TuIqtIiOuTXVhsGau8hcrhhwSsDIQ== - -node-addon-api@^6.1.0: - version "6.1.0" - resolved "https://registry.npmjs.org/node-addon-api/-/node-addon-api-6.1.0.tgz" - integrity sha512-+eawOlIgy680F0kBzPUNFhMZGtJ1YmqM6l4+Crf4IkImjYrO/mqPwRMh352g23uIaQKFItcQ64I7KMaJxHgAVA== - -node-domexception@^1.0.0: - version "1.0.0" - resolved "https://registry.npmjs.org/node-domexception/-/node-domexception-1.0.0.tgz" - integrity sha512-/jKZoMpw0F8GRwl4/eLROPA3cfcXtLApP0QzLmUT/HuPCZWyB7IY9ZrMeKw2O/nFIqPQB3PVM9aYm0F312AXDQ== - -node-fetch@^3.3.2: - version "3.3.2" - resolved "https://registry.npmjs.org/node-fetch/-/node-fetch-3.3.2.tgz" - integrity sha512-dRB78srN/l6gqWulah9SrxeYnxeddIG30+GOqK/9OlLVyLg3HPnr6SqOWTWOXKRwC2eGYCkZ59NNuSgvSrpgOA== - dependencies: - data-uri-to-buffer "^4.0.0" - fetch-blob "^3.1.4" - formdata-polyfill "^4.0.10" - -node-machine-id@1.1.12: - version "1.1.12" - resolved "https://registry.npmjs.org/node-machine-id/-/node-machine-id-1.1.12.tgz" - integrity sha512-QNABxbrPa3qEIfrE6GOJ7BYIuignnJw7iQ2YPbc3Nla1HzRJjXzZOiikfF8m7eAMfichLt3M4VgLOetqgDmgGQ== - -numeral@^2.0.6: - version "2.0.6" - resolved "https://registry.npmjs.org/numeral/-/numeral-2.0.6.tgz" - integrity sha512-qaKRmtYPZ5qdw4jWJD6bxEf1FJEqllJrwxCLIm0sQU/A7v2/czigzOb+C2uSiFsa9lBUzeH7M1oK+Q+OLxL3kA== - -oauth4webapi@^3.8.0, oauth4webapi@^3.8.2: - version "3.8.3" - resolved "https://registry.npmjs.org/oauth4webapi/-/oauth4webapi-3.8.3.tgz" - integrity sha512-pQ5BsX3QRTgnt5HxgHwgunIRaDXBdkT23tf8dfzmtTIL2LTpdmxgbpbBm0VgFWAIDlezQvQCTgnVIUmHupXHxw== - -object-assign@^4: - version "4.1.1" - resolved "https://registry.npmjs.org/object-assign/-/object-assign-4.1.1.tgz" - integrity sha512-rJgTQnkUnH1sFw8yT6VSU3zD3sWmu6sZhIseY8VX+GRu3P6F7Fu+JNDoXfklElbLJSnc3FUQHVe4cU5hj+BcUg== - -object-inspect@^1.13.3, object-inspect@^1.13.4: - version "1.13.4" - resolved "https://registry.npmjs.org/object-inspect/-/object-inspect-1.13.4.tgz" - integrity sha512-W67iLl4J2EXEGTbfeHCffrjDfitvLANg0UlX3wFUUSTx92KXRFegMHUVgSqE+wvhAbi4WqjGg9czysTV2Epbew== - -object-keys@^1.1.1: - version "1.1.1" - resolved "https://registry.npmjs.org/object-keys/-/object-keys-1.1.1.tgz" - integrity sha512-NuAESUOUMrlIXOfHKzD6bpPu3tYt3xvjNdRIQ+FeT0lNb4K8WR70CaDxhuNguS2XG+GjkyMwOzsN5ZktImfhLA== - -object.assign@^4.1.7: - version "4.1.7" - resolved "https://registry.npmjs.org/object.assign/-/object.assign-4.1.7.tgz" - integrity sha512-nK28WOo+QIjBkDduTINE4JkF/UJJKyf2EJxvJKfblDpyg0Q+pkOHNTL0Qwy6NP6FhE/EnzV73BxxqcJaXY9anw== - dependencies: - call-bind "^1.0.8" - call-bound "^1.0.3" - define-properties "^1.2.1" - es-object-atoms "^1.0.0" - has-symbols "^1.1.0" - object-keys "^1.1.1" - -on-finished@^2.4.1: - version "2.4.1" - resolved "https://registry.npmjs.org/on-finished/-/on-finished-2.4.1.tgz" - integrity sha512-oVlzkg3ENAhCk2zdv7IJwd/QUD4z2RxRwpkcGY8psCVcCYZNq4wYnVWALHM+brtuJjePWiYF/ClmuDr8Ch5+kg== - dependencies: - ee-first "1.1.1" - -once@^1.3.1, once@^1.4.0: - version "1.4.0" - resolved "https://registry.npmjs.org/once/-/once-1.4.0.tgz" - integrity sha512-lNaJgI+2Q5URQBkccEKHTQOPaXdUxnZZElQTZY0MFUAuaEqe1E+Nyvgdz/aIyNi6Z9MzO5dv1H8n58/GELp3+w== - dependencies: - wrappy "1" - -open@^10.1.2: - version "10.2.0" - resolved "https://registry.npmjs.org/open/-/open-10.2.0.tgz" - integrity sha512-YgBpdJHPyQ2UE5x+hlSXcnejzAvD0b22U2OuAP+8OnlJT+PjWPxtgmGqKKc+RgTM63U9gN0YzrYc71R2WT/hTA== - dependencies: - default-browser "^5.2.1" - define-lazy-prop "^3.0.0" - is-inside-container "^1.0.0" - wsl-utils "^0.1.0" - -openapi-fetch@^0.15.0: - version "0.15.0" - resolved "https://registry.npmjs.org/openapi-fetch/-/openapi-fetch-0.15.0.tgz" - integrity sha512-OjQUdi61WO4HYhr9+byCPMj0+bgste/LtSBEcV6FzDdONTs7x0fWn8/ndoYwzqCsKWIxEZwo4FN/TG1c1rI8IQ== - dependencies: - openapi-typescript-helpers "^0.0.15" - -openapi-typescript-helpers@^0.0.15: - version "0.0.15" - resolved "https://registry.npmjs.org/openapi-typescript-helpers/-/openapi-typescript-helpers-0.0.15.tgz" - integrity sha512-opyTPaunsklCBpTK8JGef6mfPhLSnyy5a0IN9vKtx3+4aExf+KxEqYwIy3hqkedXIB97u357uLMJsOnm3GVjsw== - -openid-client@^6.6.3: - version "6.8.1" - resolved "https://registry.npmjs.org/openid-client/-/openid-client-6.8.1.tgz" - integrity sha512-VoYT6enBo6Vj2j3Q5Ec0AezS+9YGzQo1f5Xc42lreMGlfP4ljiXPKVDvCADh+XHCV/bqPu/wWSiCVXbJKvrODw== - dependencies: - jose "^6.1.0" - oauth4webapi "^3.8.2" - -own-keys@^1.0.1: - version "1.0.1" - resolved "https://registry.npmjs.org/own-keys/-/own-keys-1.0.1.tgz" - integrity sha512-qFOyK5PjiWZd+QQIh+1jhdb9LpxTF0qs7Pm8o5QHYZ0M3vKqSqzsZaEB6oWlxZ+q2sJBMI/Ktgd2N5ZwQoRHfg== - dependencies: - get-intrinsic "^1.2.6" - object-keys "^1.1.1" - safe-push-apply "^1.0.0" - -pac-proxy-agent@^7.0.2: - version "7.2.0" - resolved "https://registry.npmjs.org/pac-proxy-agent/-/pac-proxy-agent-7.2.0.tgz" - integrity sha512-TEB8ESquiLMc0lV8vcd5Ql/JAKAoyzHFXaStwjkzpOpC5Yv+pIzLfHvjTSdf3vpa2bMiUQrg9i6276yn8666aA== - dependencies: - "@tootallnate/quickjs-emscripten" "^0.23.0" - agent-base "^7.1.2" - debug "^4.3.4" - get-uri "^6.0.1" - http-proxy-agent "^7.0.0" - https-proxy-agent "^7.0.6" - pac-resolver "^7.0.1" - socks-proxy-agent "^8.0.5" - -pac-resolver@^7.0.1: - version "7.0.1" - resolved "https://registry.npmjs.org/pac-resolver/-/pac-resolver-7.0.1.tgz" - integrity sha512-5NPgf87AT2STgwa2ntRMr45jTKrYBGkVU36yT0ig/n/GMAa3oPqhZfIQ2kMEimReg0+t9kZViDVZ83qfVUlckg== - dependencies: - degenerator "^5.0.0" - netmask "^2.0.2" - -parseurl@^1.3.3: - version "1.3.3" - resolved "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz" - integrity sha512-CiyeOxFT/JZyN5m0z9PfXw4SCBJ6Sygz1Dpl0wqjlhDEGGBP1GnsUVEL0p63hoG1fcj3fHynXi9NYO4nWOL+qQ== - -path-key@^3.1.0: - version "3.1.1" - resolved "https://registry.npmjs.org/path-key/-/path-key-3.1.1.tgz" - integrity sha512-ojmeN0qd+y0jszEtoY48r0Peq5dwMEkIlCOu6Q5f41lfkswXuKtYrhgoTpLnyIcHm24Uhqx+5Tqm2InSwLhE6Q== - -path-to-regexp@^8.0.0: - version "8.3.0" - resolved "https://registry.npmjs.org/path-to-regexp/-/path-to-regexp-8.3.0.tgz" - integrity sha512-7jdwVIRtsP8MYpdXSwOS0YdD0Du+qOoF/AEPIt88PcCFrZCzx41oxku1jD88hZBwbNUIEfpqvuhjFaMAqMTWnA== - -pkce-challenge@^5.0.0: - version "5.0.1" - resolved "https://registry.npmjs.org/pkce-challenge/-/pkce-challenge-5.0.1.tgz" - integrity sha512-wQ0b/W4Fr01qtpHlqSqspcj3EhBvimsdh0KlHhH8HRZnMsEa0ea2fTULOXOS9ccQr3om+GcGRk4e+isrZWV8qQ== - -possible-typed-array-names@^1.0.0: - version "1.1.0" - resolved "https://registry.npmjs.org/possible-typed-array-names/-/possible-typed-array-names-1.1.0.tgz" - integrity sha512-/+5VFTchJDoVj3bhoqi6UeymcD00DAwb1nJwamzPvHEszJ4FpF6SNNbUbOS8yI56qHzdV8eK0qEfOSiodkTdxg== - -prebuild-install@^7.1.2, prebuild-install@^7.1.3: - version "7.1.3" - resolved "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.3.tgz" - integrity sha512-8Mf2cbV7x1cXPUILADGI3wuhfqWvtiLA1iclTDbFRZkgRQS0NqsPZphna9V+HyTEadheuPmjaJMsbzKQFOzLug== - dependencies: - detect-libc "^2.0.0" - expand-template "^2.0.3" - github-from-package "0.0.0" - minimist "^1.2.3" - mkdirp-classic "^0.5.3" - napi-build-utils "^2.0.0" - node-abi "^3.3.0" - pump "^3.0.0" - rc "^1.2.7" - simple-get "^4.0.0" - tar-fs "^2.0.0" - tunnel-agent "^0.6.0" - -prebuild-install@7.1.1: - version "7.1.1" - resolved "https://registry.npmjs.org/prebuild-install/-/prebuild-install-7.1.1.tgz" - integrity sha512-jAXscXWMcCK8GgCoHOfIr0ODh5ai8mj63L2nWrjuAgXE6tDyYGnx4/8o/rCgU+B4JSyZBKbeZqzhtwtC3ovxjw== - dependencies: - detect-libc "^2.0.0" - expand-template "^2.0.3" - github-from-package "0.0.0" - minimist "^1.2.3" - mkdirp-classic "^0.5.3" - napi-build-utils "^1.0.1" - node-abi "^3.3.0" - pump "^3.0.0" - rc "^1.2.7" - simple-get "^4.0.0" - tar-fs "^2.0.0" - tunnel-agent "^0.6.0" - -progress@^2.0.3: - version "2.0.3" - resolved "https://registry.npmjs.org/progress/-/progress-2.0.3.tgz" - integrity sha512-7PiHtLll5LdnKIMw100I+8xJXR5gW2QwWYkT6iJva0bXitZKa/XMrSbdmg3r2Xnaidz9Qumd0VPaMrZlF9V9sA== - -proxy-addr@^2.0.7: - version "2.0.7" - resolved "https://registry.npmjs.org/proxy-addr/-/proxy-addr-2.0.7.tgz" - integrity sha512-llQsMLSUDUPT44jdrU/O37qlnifitDP+ZwrmmZcoSKyLKvtZxpyV0n2/bD/N4tBAAZ/gJEdZU7KMraoK1+XYAg== - dependencies: - forwarded "0.2.0" - ipaddr.js "1.9.1" - -pump@^3.0.0: - version "3.0.3" - resolved "https://registry.npmjs.org/pump/-/pump-3.0.3.tgz" - integrity sha512-todwxLMY7/heScKmntwQG8CXVkWUOdYxIvY2s0VWAAMh/nd8SoYiRaKjlr7+iCs984f2P8zvrfWcDDYVb73NfA== - dependencies: - end-of-stream "^1.1.0" - once "^1.3.1" - -punycode@^2.3.1: - version "2.3.1" - resolved "https://registry.npmjs.org/punycode/-/punycode-2.3.1.tgz" - integrity sha512-vYt7UD1U9Wg6138shLtLOvdAu+8DsC/ilFtEVHcH+wydcSpNE20AfSOduf6MkRFahL5FY7X1oU7nKVZFtfq8Fg== - -qs@^6.14.0: - version "6.14.0" - resolved "https://registry.npmjs.org/qs/-/qs-6.14.0.tgz" - integrity sha512-YWWTjgABSKcvs/nWBi9PycY/JiPJqOD4JA6o9Sej2AtvSGarXxKC3OQSk4pAarbdQlKAh5D4FCQkJNkW+GAn3w== - dependencies: - side-channel "^1.1.0" - -range-parser@^1.2.1: - version "1.2.1" - resolved "https://registry.npmjs.org/range-parser/-/range-parser-1.2.1.tgz" - integrity sha512-Hrgsx+orqoygnmhFbKaHE6c296J+HTAQXoxEF6gNupROmmGJRoyzfG3ccAveqCBrwr/2yxQ5BVd/GTl5agOwSg== - -raw-body@^3.0.0, raw-body@^3.0.1: - version "3.0.2" - resolved "https://registry.npmjs.org/raw-body/-/raw-body-3.0.2.tgz" - integrity sha512-K5zQjDllxWkf7Z5xJdV0/B0WTNqx6vxG70zJE4N0kBs4LovmEYWJzQGxC9bS9RAKu3bgM40lrd5zoLJ12MQ5BA== - dependencies: - bytes "~3.1.2" - http-errors "~2.0.1" - iconv-lite "~0.7.0" - unpipe "~1.0.0" - -rc@^1.2.7: - version "1.2.8" - resolved "https://registry.npmjs.org/rc/-/rc-1.2.8.tgz" - integrity sha512-y3bGgqKj3QBdxLbLkomlohkvsA8gdAiUQlSBJnBhfn+BPxg4bc62d8TcBW15wavDfgexCgccckhcZvywyQYPOw== - dependencies: - deep-extend "^0.6.0" - ini "~1.3.0" - minimist "^1.2.0" - strip-json-comments "~2.0.1" - -readable-stream@^3.1.1, readable-stream@^3.4.0: - version "3.6.2" - resolved "https://registry.npmjs.org/readable-stream/-/readable-stream-3.6.2.tgz" - integrity sha512-9u/sniCrY3D5WdsERHzHE4G2YCXqoG5FTHUiCC4SIbr6XcLZBY05ya9EKjYek9O5xOAwjGq+1JdGBAS7Q9ScoA== - dependencies: - inherits "^2.0.3" - string_decoder "^1.1.1" - util-deprecate "^1.0.1" - -reflect.getprototypeof@^1.0.6, reflect.getprototypeof@^1.0.9: - version "1.0.10" - resolved "https://registry.npmjs.org/reflect.getprototypeof/-/reflect.getprototypeof-1.0.10.tgz" - integrity sha512-00o4I+DVrefhv+nX0ulyi3biSHCPDe+yLv5o/p6d/UVlirijB8E16FtfwSAi4g3tcqrQ4lRAqQSoFEZJehYEcw== - dependencies: - call-bind "^1.0.8" - define-properties "^1.2.1" - es-abstract "^1.23.9" - es-errors "^1.3.0" - es-object-atoms "^1.0.0" - get-intrinsic "^1.2.7" - get-proto "^1.0.1" - which-builtin-type "^1.2.1" - -regexp.escape@^2.0.1: - version "2.0.1" - resolved "https://registry.npmjs.org/regexp.escape/-/regexp.escape-2.0.1.tgz" - integrity sha512-JItRb4rmyTzmERBkAf6J87LjDPy/RscIwmaJQ3gsFlAzrmZbZU8LwBw5IydFZXW9hqpgbPlGbMhtpqtuAhMgtg== - dependencies: - call-bind "^1.0.7" - define-properties "^1.2.1" - es-abstract "^1.23.3" - es-errors "^1.3.0" - for-each "^0.3.3" - safe-regex-test "^1.0.3" - -regexp.prototype.flags@^1.5.4: - version "1.5.4" - resolved "https://registry.npmjs.org/regexp.prototype.flags/-/regexp.prototype.flags-1.5.4.tgz" - integrity sha512-dYqgNSZbDwkaJ2ceRd9ojCGjBq+mOm9LmtXnAnEGyHhN/5R7iDW2TRw3h+o/jCFxus3P2LfWIIiwowAjANm7IA== - dependencies: - call-bind "^1.0.8" - define-properties "^1.2.1" - es-errors "^1.3.0" - get-proto "^1.0.1" - gopd "^1.2.0" - set-function-name "^2.0.2" - -require-directory@^2.1.1: - version "2.1.1" - resolved "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz" - integrity sha512-fGxEI7+wsG9xrvdjsrlmL22OMTTiHRwAMroiEeMgq8gzoLC/PQr7RsRDSTLUg/bZAZtF+TVIkHc6/4RIKrui+Q== - -require-from-string@^2.0.2: - version "2.0.2" - resolved "https://registry.npmjs.org/require-from-string/-/require-from-string-2.0.2.tgz" - integrity sha512-Xf0nWe6RseziFMu+Ap9biiUbmplq6S9/p+7w7YXP/JBHhrUDDUhwa+vANyubuqfZWTveU//DYVGsDG7RKL/vEw== - -reservoir@^0.1.2: - version "0.1.2" - resolved "https://registry.npmjs.org/reservoir/-/reservoir-0.1.2.tgz" - integrity sha512-ysyw95gLBhMAzqIVrOHJ2yMrRQHAS+h97bS9r89Z7Ou10Jhl2k5KOsyjPqrxL+WfEanov0o5bAMVzQ7AKyENHA== - -resolve-mongodb-srv@^1.1.1: - version "1.1.6" - resolved "https://registry.npmjs.org/resolve-mongodb-srv/-/resolve-mongodb-srv-1.1.6.tgz" - integrity sha512-eeHKsU9+zD7NGw+QI8SVNvcawIF0rT5/WmnVC5qHHJBsUW1ZZaJ1SBaT0eEv3ex+YBg4yaQVfwsCn8pD5q1Ggw== - dependencies: - whatwg-url "^11.0.0 || ^12.0.0 || ^13.0.0 || ^14.0.0" - -router@^2.2.0: - version "2.2.0" - resolved "https://registry.npmjs.org/router/-/router-2.2.0.tgz" - integrity sha512-nLTrUKm2UyiL7rlhapu/Zl45FwNgkZGaCpZbIHajDYgwlJCOzLSk+cIPAnsEqV955GjILJnKbdQC1nVPz+gAYQ== - dependencies: - debug "^4.4.0" - depd "^2.0.0" - is-promise "^4.0.0" - parseurl "^1.3.3" - path-to-regexp "^8.0.0" - -run-applescript@^7.0.0: - version "7.1.0" - resolved "https://registry.npmjs.org/run-applescript/-/run-applescript-7.1.0.tgz" - integrity sha512-DPe5pVFaAsinSaV6QjQ6gdiedWDcRCbUuiQfQa2wmWV7+xC9bGulGI8+TdRmoFkAPaBXk8CrAbnlY2ISniJ47Q== - -safe-array-concat@^1.1.3: - version "1.1.3" - resolved "https://registry.npmjs.org/safe-array-concat/-/safe-array-concat-1.1.3.tgz" - integrity sha512-AURm5f0jYEOydBj7VQlVvDrjeFgthDdEF5H1dP+6mNpoXOMo1quQqJ4wvJDyRZ9+pO3kGWoOdmV08cSv2aJV6Q== - dependencies: - call-bind "^1.0.8" - call-bound "^1.0.2" - get-intrinsic "^1.2.6" - has-symbols "^1.1.0" - isarray "^2.0.5" - -safe-buffer@^5.0.1, safe-buffer@~5.2.0: - version "5.2.1" - resolved "https://registry.npmjs.org/safe-buffer/-/safe-buffer-5.2.1.tgz" - integrity sha512-rp3So07KcdmmKbGvgaNxQSJr7bGVSVk5S9Eq1F+ppbRo70+YeaDxkw5Dd8NPN+GD6bjnYm2VuPuCXmpuYvmCXQ== - -safe-push-apply@^1.0.0: - version "1.0.0" - resolved "https://registry.npmjs.org/safe-push-apply/-/safe-push-apply-1.0.0.tgz" - integrity sha512-iKE9w/Z7xCzUMIZqdBsp6pEQvwuEebH4vdpjcDWnyzaI6yl6O9FHvVpmGelvEHNsoY6wGblkxR6Zty/h00WiSA== - dependencies: - es-errors "^1.3.0" - isarray "^2.0.5" - -safe-regex-test@^1.0.3, safe-regex-test@^1.1.0: - version "1.1.0" - resolved "https://registry.npmjs.org/safe-regex-test/-/safe-regex-test-1.1.0.tgz" - integrity sha512-x/+Cz4YrimQxQccJf5mKEbIa1NzeCRNI5Ecl/ekmlYaampdNLPalVyIcCZNNH3MvmqBugV5TMYZXv0ljslUlaw== - dependencies: - call-bound "^1.0.2" - es-errors "^1.3.0" - is-regex "^1.2.1" - -"safer-buffer@>= 2.1.2 < 3.0.0", safer-buffer@~2.1.0: - version "2.1.2" - resolved "https://registry.npmjs.org/safer-buffer/-/safer-buffer-2.1.2.tgz" - integrity sha512-YZo3K82SD7Riyi0E1EQPojLz7kpepnSQI9IyPbHHg1XXXevb5dJI7tpyN2ADxGcQbHG7vcyRHk0cbwqcQriUtg== - -semver@^7.3.5: - version "7.7.3" - resolved "https://registry.npmjs.org/semver/-/semver-7.7.3.tgz" - integrity sha512-SdsKMrI9TdgjdweUSR9MweHA4EJ8YxHn8DFaDisvhVlUOe4BF1tLD7GAj0lIqWVl+dPb/rExr0Btby5loQm20Q== - -send@^1.1.0, send@^1.2.0: - version "1.2.0" - resolved "https://registry.npmjs.org/send/-/send-1.2.0.tgz" - integrity sha512-uaW0WwXKpL9blXE2o0bRhoL2EGXIrZxQ2ZQ4mgcfoBxdFmQold+qWsD2jLrfZ0trjKL6vOw0j//eAwcALFjKSw== - dependencies: - debug "^4.3.5" - encodeurl "^2.0.0" - escape-html "^1.0.3" - etag "^1.8.1" - fresh "^2.0.0" - http-errors "^2.0.0" - mime-types "^3.0.1" - ms "^2.1.3" - on-finished "^2.4.1" - range-parser "^1.2.1" - statuses "^2.0.1" - -serve-static@^2.2.0: - version "2.2.0" - resolved "https://registry.npmjs.org/serve-static/-/serve-static-2.2.0.tgz" - integrity sha512-61g9pCh0Vnh7IutZjtLGGpTA355+OPn2TyDv/6ivP2h/AdAVX9azsoxmg2/M6nZeQZNYBEwIcsne1mJd9oQItQ== - dependencies: - encodeurl "^2.0.0" - escape-html "^1.0.3" - parseurl "^1.3.3" - send "^1.2.0" - -set-function-length@^1.2.2: - version "1.2.2" - resolved "https://registry.npmjs.org/set-function-length/-/set-function-length-1.2.2.tgz" - integrity sha512-pgRc4hJ4/sNjWCSS9AmnS40x3bNMDTknHgL5UaMBTMyJnU90EgWh1Rz+MC9eFu4BuN/UwZjKQuY/1v3rM7HMfg== - dependencies: - define-data-property "^1.1.4" - es-errors "^1.3.0" - function-bind "^1.1.2" - get-intrinsic "^1.2.4" - gopd "^1.0.1" - has-property-descriptors "^1.0.2" - -set-function-name@^2.0.2: - version "2.0.2" - resolved "https://registry.npmjs.org/set-function-name/-/set-function-name-2.0.2.tgz" - integrity sha512-7PGFlmtwsEADb0WYyvCMa1t+yke6daIG4Wirafur5kcf+MhUnPms1UeR0CKQdTZD81yESwMHbtn+TR+dMviakQ== - dependencies: - define-data-property "^1.1.4" - es-errors "^1.3.0" - functions-have-names "^1.2.3" - has-property-descriptors "^1.0.2" - -set-proto@^1.0.0: - version "1.0.0" - resolved "https://registry.npmjs.org/set-proto/-/set-proto-1.0.0.tgz" - integrity sha512-RJRdvCo6IAnPdsvP/7m6bsQqNnn1FCBX5ZNtFL98MmFF/4xAIJTIg1YbHW5DC2W5SKZanrC6i4HsJqlajw/dZw== - dependencies: - dunder-proto "^1.0.1" - es-errors "^1.3.0" - es-object-atoms "^1.0.0" - -setprototypeof@~1.2.0: - version "1.2.0" - resolved "https://registry.npmjs.org/setprototypeof/-/setprototypeof-1.2.0.tgz" - integrity sha512-E5LDX7Wrp85Kil5bhZv46j8jOeboKq5JMmYM3gVGdGH8xFpPWXUMsNrlODCrkoxMEeNi/XZIwuRvY4XNwYMJpw== - -shebang-command@^2.0.0: - version "2.0.0" - resolved "https://registry.npmjs.org/shebang-command/-/shebang-command-2.0.0.tgz" - integrity sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA== - dependencies: - shebang-regex "^3.0.0" - -shebang-regex@^3.0.0: - version "3.0.0" - resolved "https://registry.npmjs.org/shebang-regex/-/shebang-regex-3.0.0.tgz" - integrity sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A== - -side-channel-list@^1.0.0: - version "1.0.0" - resolved "https://registry.npmjs.org/side-channel-list/-/side-channel-list-1.0.0.tgz" - integrity sha512-FCLHtRD/gnpCiCHEiJLOwdmFP+wzCmDEkc9y7NsYxeF4u7Btsn1ZuwgwJGxImImHicJArLP4R0yX4c2KCrMrTA== - dependencies: - es-errors "^1.3.0" - object-inspect "^1.13.3" - -side-channel-map@^1.0.1: - version "1.0.1" - resolved "https://registry.npmjs.org/side-channel-map/-/side-channel-map-1.0.1.tgz" - integrity sha512-VCjCNfgMsby3tTdo02nbjtM/ewra6jPHmpThenkTYh8pG9ucZ/1P8So4u4FGBek/BjpOVsDCMoLA/iuBKIFXRA== - dependencies: - call-bound "^1.0.2" - es-errors "^1.3.0" - get-intrinsic "^1.2.5" - object-inspect "^1.13.3" - -side-channel-weakmap@^1.0.2: - version "1.0.2" - resolved "https://registry.npmjs.org/side-channel-weakmap/-/side-channel-weakmap-1.0.2.tgz" - integrity sha512-WPS/HvHQTYnHisLo9McqBHOJk2FkHO/tlpvldyrnem4aeQp4hai3gythswg6p01oSoTl58rcpiFAjF2br2Ak2A== - dependencies: - call-bound "^1.0.2" - es-errors "^1.3.0" - get-intrinsic "^1.2.5" - object-inspect "^1.13.3" - side-channel-map "^1.0.1" - -side-channel@^1.1.0: - version "1.1.0" - resolved "https://registry.npmjs.org/side-channel/-/side-channel-1.1.0.tgz" - integrity sha512-ZX99e6tRweoUXqR+VBrslhda51Nh5MTQwou5tnUDgbtyM0dBgmhEDtWGP/xbKn6hqfPRHujUNwz5fy/wbbhnpw== - dependencies: - es-errors "^1.3.0" - object-inspect "^1.13.3" - side-channel-list "^1.0.0" - side-channel-map "^1.0.1" - side-channel-weakmap "^1.0.2" - -simple-concat@^1.0.0: - version "1.0.1" - resolved "https://registry.npmjs.org/simple-concat/-/simple-concat-1.0.1.tgz" - integrity sha512-cSFtAPtRhljv69IK0hTVZQ+OfE9nePi/rtJmw5UjHeVyVroEqJXP1sFztKUy1qU+xvz3u/sfYJLa947b7nAN2Q== - -simple-get@^4.0.0: - version "4.0.1" - resolved "https://registry.npmjs.org/simple-get/-/simple-get-4.0.1.tgz" - integrity sha512-brv7p5WgH0jmQJr1ZDDfKDOSeWWg+OVypG99A/5vYGPqJ6pxiaHLy8nxtFjBA7oMa01ebA9gfh1uMCFqOuXxvA== - dependencies: - decompress-response "^6.0.0" - once "^1.3.1" - simple-concat "^1.0.0" - -smart-buffer@^4.2.0: - version "4.2.0" - resolved "https://registry.npmjs.org/smart-buffer/-/smart-buffer-4.2.0.tgz" - integrity sha512-94hK0Hh8rPqQl2xXc3HsaBoOXKV20MToPkcXvwbISWLEs+64sBq5kFgn2kJDHb1Pry9yrP0dxrCI9RRci7RXKg== - -socks-proxy-agent@^8.0.4, socks-proxy-agent@^8.0.5: - version "8.0.5" - resolved "https://registry.npmjs.org/socks-proxy-agent/-/socks-proxy-agent-8.0.5.tgz" - integrity sha512-HehCEsotFqbPW9sJ8WVYB6UbmIMv7kUUORIF2Nncq4VQvBfNBLibW9YZR5dlYCSUhwcD628pRllm7n+E+YTzJw== - dependencies: - agent-base "^7.1.2" - debug "^4.3.4" - socks "^2.8.3" - -socks@^2.7.1, socks@^2.7.3, socks@^2.8.3: - version "2.8.7" - resolved "https://registry.npmjs.org/socks/-/socks-2.8.7.tgz" - integrity sha512-HLpt+uLy/pxB+bum/9DzAgiKS8CX1EvbWxI4zlmgGCExImLdiad2iCwXT5Z4c9c3Eq8rP2318mPW2c+QbtjK8A== - dependencies: - ip-address "^10.0.1" - smart-buffer "^4.2.0" - -source-map@~0.6.1: - version "0.6.1" - resolved "https://registry.npmjs.org/source-map/-/source-map-0.6.1.tgz" - integrity sha512-UjgapumWlbMhkBgzT7Ykc5YXUT46F0iKu8SGXq0bcwP5dz/h0Plj6enJqjz1Zbq2l5WaqYnrVbwWOWMyF3F47g== - -sparse-bitfield@^3.0.3: - version "3.0.3" - resolved "https://registry.npmjs.org/sparse-bitfield/-/sparse-bitfield-3.0.3.tgz" - integrity sha512-kvzhi7vqKTfkh0PZU+2D2PIllw2ymqJKujUcyPMd9Y75Nv4nPbGJZXNhxsgdQab2BmlDct1YnfQCguEvHr7VsQ== - dependencies: - memory-pager "^1.0.2" - -sprintf-js@^1.1.3: - version "1.1.3" - resolved "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.1.3.tgz" - integrity sha512-Oo+0REFV59/rz3gfJNKQiBlwfHaSESl1pcGyABQsnnIfWOFt6JNj5gCog2U6MLZ//IGYD+nA8nI+mTShREReaA== - -ssh2@^1.15.0: - version "1.17.0" - resolved "https://registry.npmjs.org/ssh2/-/ssh2-1.17.0.tgz" - integrity sha512-wPldCk3asibAjQ/kziWQQt1Wh3PgDFpC0XpwclzKcdT1vql6KeYxf5LIt4nlFkUeR8WuphYMKqUA56X4rjbfgQ== - dependencies: - asn1 "^0.2.6" - bcrypt-pbkdf "^1.0.2" - optionalDependencies: - cpu-features "~0.0.10" - nan "^2.23.0" - -stats-lite@^2.0.0: - version "2.2.0" - resolved "https://registry.npmjs.org/stats-lite/-/stats-lite-2.2.0.tgz" - integrity sha512-/Kz55rgUIv2KP2MKphwYT/NCuSfAlbbMRv2ZWw7wyXayu230zdtzhxxuXXcvsc6EmmhS8bSJl3uS1wmMHFumbA== - dependencies: - isnumber "~1.0.0" - -statuses@^2.0.1, statuses@~2.0.2: - version "2.0.2" - resolved "https://registry.npmjs.org/statuses/-/statuses-2.0.2.tgz" - integrity sha512-DvEy55V3DB7uknRo+4iOGT5fP1slR8wQohVdknigZPMpMstaKJQWhwiYBACJE3Ul2pTnATihhBYnRhZQHGBiRw== - -stop-iteration-iterator@^1.1.0: - version "1.1.0" - resolved "https://registry.npmjs.org/stop-iteration-iterator/-/stop-iteration-iterator-1.1.0.tgz" - integrity sha512-eLoXW/DHyl62zxY4SCaIgnRhuMr6ri4juEYARS8E6sCEqzKpOiE521Ucofdx+KnDZl5xmvGYaaKCk5FEOxJCoQ== - dependencies: - es-errors "^1.3.0" - internal-slot "^1.1.0" - -string_decoder@^1.1.1: - version "1.3.0" - resolved "https://registry.npmjs.org/string_decoder/-/string_decoder-1.3.0.tgz" - integrity sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA== - dependencies: - safe-buffer "~5.2.0" - -string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.3: - version "4.2.3" - resolved "https://registry.npmjs.org/string-width/-/string-width-4.2.3.tgz" - integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g== - dependencies: - emoji-regex "^8.0.0" - is-fullwidth-code-point "^3.0.0" - strip-ansi "^6.0.1" - -string.prototype.trim@^1.2.10: - version "1.2.10" - resolved "https://registry.npmjs.org/string.prototype.trim/-/string.prototype.trim-1.2.10.tgz" - integrity sha512-Rs66F0P/1kedk5lyYyH9uBzuiI/kNRmwJAR9quK6VOtIpZ2G+hMZd+HQbbv25MgCA6gEffoMZYxlTod4WcdrKA== - dependencies: - call-bind "^1.0.8" - call-bound "^1.0.2" - define-data-property "^1.1.4" - define-properties "^1.2.1" - es-abstract "^1.23.5" - es-object-atoms "^1.0.0" - has-property-descriptors "^1.0.2" - -string.prototype.trimend@^1.0.9: - version "1.0.9" - resolved "https://registry.npmjs.org/string.prototype.trimend/-/string.prototype.trimend-1.0.9.tgz" - integrity sha512-G7Ok5C6E/j4SGfyLCloXTrngQIQU3PWtXGst3yM7Bea9FRURf1S42ZHlZZtsNque2FN2PoUhfZXYLNWwEr4dLQ== - dependencies: - call-bind "^1.0.8" - call-bound "^1.0.2" - define-properties "^1.2.1" - es-object-atoms "^1.0.0" - -string.prototype.trimstart@^1.0.8: - version "1.0.8" - resolved "https://registry.npmjs.org/string.prototype.trimstart/-/string.prototype.trimstart-1.0.8.tgz" - integrity sha512-UXSH262CSZY1tfu3G3Secr6uGLCFVPMhIqHjlgCUtCCcgihYc/xKs9djMTMUOb2j1mVSeU8EU6NWc/iQKU6Gfg== - dependencies: - call-bind "^1.0.7" - define-properties "^1.2.1" - es-object-atoms "^1.0.0" - -strip-ansi@^6.0.0, strip-ansi@^6.0.1: - version "6.0.1" - resolved "https://registry.npmjs.org/strip-ansi/-/strip-ansi-6.0.1.tgz" - integrity sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A== - dependencies: - ansi-regex "^5.0.1" - -strip-json-comments@~2.0.1: - version "2.0.1" - resolved "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-2.0.1.tgz" - integrity sha512-4gB8na07fecVVkOI6Rs4e7T6NOTki5EmL7TUduTs6bu3EdnSycntVJ4re8kgZA+wx9IueI2Y11bfbgwtzuE0KQ== - -system-ca@^2.0.1: - version "2.0.1" - resolved "https://registry.npmjs.org/system-ca/-/system-ca-2.0.1.tgz" - integrity sha512-9ZDV9yl8ph6Op67wDGPr4LykX86usE9x3le+XZSHfVMiiVJ5IRgmCWjLgxyz35ju9H3GDIJJZm4ogAeIfN5cQQ== - optionalDependencies: - macos-export-certificate-and-key "^1.2.0" - win-export-certificate-and-key "^2.1.0" - -tar-fs@^2.0.0: - version "2.1.4" - resolved "https://registry.npmjs.org/tar-fs/-/tar-fs-2.1.4.tgz" - integrity sha512-mDAjwmZdh7LTT6pNleZ05Yt65HC3E+NiQzl672vQG38jIrehtJk/J3mNwIg+vShQPcLF/LV7CMnDW6vjj6sfYQ== - dependencies: - chownr "^1.1.1" - mkdirp-classic "^0.5.2" - pump "^3.0.0" - tar-stream "^2.1.4" - -tar-stream@^2.1.4: - version "2.2.0" - resolved "https://registry.npmjs.org/tar-stream/-/tar-stream-2.2.0.tgz" - integrity sha512-ujeqbceABgwMZxEJnk2HDY2DlnUZ+9oEcb1KzTVfYHio0UE6dG71n60d8D2I4qNvleWrrXpmjpt7vZeF1LnMZQ== - dependencies: - bl "^4.0.3" - end-of-stream "^1.4.1" - fs-constants "^1.0.0" - inherits "^2.0.3" - readable-stream "^3.1.1" - -toidentifier@~1.0.1: - version "1.0.1" - resolved "https://registry.npmjs.org/toidentifier/-/toidentifier-1.0.1.tgz" - integrity sha512-o5sSPKEkg/DIQNmH43V0/uerLrpzVedkUh8tGNvaeXpfpuwjKenlSox/2O/BTlZUtEe+JG7s5YhEz608PlAHRA== - -tr46@^5.1.0: - version "5.1.1" - resolved "https://registry.npmjs.org/tr46/-/tr46-5.1.1.tgz" - integrity sha512-hdF5ZgjTqgAntKkklYw0R03MG2x/bSzTtkxmIRw/sTNV8YXsCJ1tfLAX23lhxhHJlEf3CRCOCGGWw3vI3GaSPw== - dependencies: - punycode "^2.3.1" - -ts-levenshtein@^1.0.7: - version "1.0.7" - resolved "https://registry.npmjs.org/ts-levenshtein/-/ts-levenshtein-1.0.7.tgz" - integrity sha512-wautEf7gl2ITJuRTTYxnlrLjzUUcwFSdg46bcu4RlzoE/zQM++TJjBFRf2Xhil49GiHqKCqmpjf1lBkWnAHj0A== - -tslib@^2.0.1: - version "2.8.1" - resolved "https://registry.npmjs.org/tslib/-/tslib-2.8.1.tgz" - integrity sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w== - -tunnel-agent@^0.6.0: - version "0.6.0" - resolved "https://registry.npmjs.org/tunnel-agent/-/tunnel-agent-0.6.0.tgz" - integrity sha512-McnNiV1l8RYeY8tBgEpuodCC1mLUdbSN+CYBL7kJsJNInOP8UjDDEwdk6Mw60vdLLrr5NHKZhMAOSrR2NZuQ+w== - dependencies: - safe-buffer "^5.0.1" - -tweetnacl@^0.14.3: - version "0.14.5" - resolved "https://registry.npmjs.org/tweetnacl/-/tweetnacl-0.14.5.tgz" - integrity sha512-KXXFFdAbFXY4geFIwoyNK+f5Z1b7swfXABfL7HXCmoIWMKU3dmS26672A4EeQtDzLKy7SXmfBu51JolvEKwtGA== - -type-is@^2.0.1: - version "2.0.1" - resolved "https://registry.npmjs.org/type-is/-/type-is-2.0.1.tgz" - integrity sha512-OZs6gsjF4vMp32qrCbiVSkrFmXtG/AZhY3t0iAMrMBiAZyV9oALtXO8hsrHbMXF9x6L3grlFuwW2oAz7cav+Gw== - dependencies: - content-type "^1.0.5" - media-typer "^1.1.0" - mime-types "^3.0.0" - -typed-array-buffer@^1.0.3: - version "1.0.3" - resolved "https://registry.npmjs.org/typed-array-buffer/-/typed-array-buffer-1.0.3.tgz" - integrity sha512-nAYYwfY3qnzX30IkA6AQZjVbtK6duGontcQm1WSG1MD94YLqK0515GNApXkoxKOWMusVssAHWLh9SeaoefYFGw== - dependencies: - call-bound "^1.0.3" - es-errors "^1.3.0" - is-typed-array "^1.1.14" - -typed-array-byte-length@^1.0.3: - version "1.0.3" - resolved "https://registry.npmjs.org/typed-array-byte-length/-/typed-array-byte-length-1.0.3.tgz" - integrity sha512-BaXgOuIxz8n8pIq3e7Atg/7s+DpiYrxn4vdot3w9KbnBhcRQq6o3xemQdIfynqSeXeDrF32x+WvfzmOjPiY9lg== - dependencies: - call-bind "^1.0.8" - for-each "^0.3.3" - gopd "^1.2.0" - has-proto "^1.2.0" - is-typed-array "^1.1.14" - -typed-array-byte-offset@^1.0.4: - version "1.0.4" - resolved "https://registry.npmjs.org/typed-array-byte-offset/-/typed-array-byte-offset-1.0.4.tgz" - integrity sha512-bTlAFB/FBYMcuX81gbL4OcpH5PmlFHqlCCpAl8AlEzMz5k53oNDvN8p1PNOWLEmI2x4orp3raOFB51tv9X+MFQ== - dependencies: - available-typed-arrays "^1.0.7" - call-bind "^1.0.8" - for-each "^0.3.3" - gopd "^1.2.0" - has-proto "^1.2.0" - is-typed-array "^1.1.15" - reflect.getprototypeof "^1.0.9" - -typed-array-length@^1.0.7: - version "1.0.7" - resolved "https://registry.npmjs.org/typed-array-length/-/typed-array-length-1.0.7.tgz" - integrity sha512-3KS2b+kL7fsuk/eJZ7EQdnEmQoaho/r6KUef7hxvltNA5DR8NAUM+8wJMbJyZ4G9/7i3v5zPBIMN5aybAh2/Jg== - dependencies: - call-bind "^1.0.7" - for-each "^0.3.3" - gopd "^1.0.1" - is-typed-array "^1.1.13" - possible-typed-array-names "^1.0.0" - reflect.getprototypeof "^1.0.6" - -unbox-primitive@^1.1.0: - version "1.1.0" - resolved "https://registry.npmjs.org/unbox-primitive/-/unbox-primitive-1.1.0.tgz" - integrity sha512-nWJ91DjeOkej/TA8pXQ3myruKpKEYgqvpw9lz4OPHj/NWFNluYrjbz9j01CJ8yKQd2g4jFoOkINCTW2I5LEEyw== - dependencies: - call-bound "^1.0.3" - has-bigints "^1.0.2" - has-symbols "^1.1.0" - which-boxed-primitive "^1.1.1" - -unpipe@~1.0.0: - version "1.0.0" - resolved "https://registry.npmjs.org/unpipe/-/unpipe-1.0.0.tgz" - integrity sha512-pjy2bYhSsufwWlKwPc+l3cN7+wuJlK6uz0YdJEOlQDbl6jo/YlPi4mb8agUkVC8BF7V8NuzeyPNqRksA3hztKQ== - -util-deprecate@^1.0.1: - version "1.0.2" - resolved "https://registry.npmjs.org/util-deprecate/-/util-deprecate-1.0.2.tgz" - integrity sha512-EPD5q1uXyFxJpCrLnCc1nHnq3gOa6DZBocAIiI2TaSCA7VCJ1UJDMagCzIkXNsUYfD1daK//LTEQ8xiIbrHtcw== - -vary@^1, vary@^1.1.2: - version "1.1.2" - resolved "https://registry.npmjs.org/vary/-/vary-1.1.2.tgz" - integrity sha512-BNGbWLfd0eUPabhkXUVm0j8uuvREyTh5ovRa/dyow/BqAbZJyC+5fU+IzQOzmAKzYqYRAISoRhdQr3eIZ/PXqg== - -voyage-ai-provider@^2.0.0: - version "2.0.0" - resolved "https://registry.npmjs.org/voyage-ai-provider/-/voyage-ai-provider-2.0.0.tgz" - integrity sha512-AX00egENhHOAfuHAhvmoBVQNG6+f717763CfyPefjahDTxbt6nCE0IlDXn5nkzLIu00JoM/PDFYDYQ17NYQqPw== - dependencies: - "@ai-sdk/provider" "^2.0.0" - "@ai-sdk/provider-utils" "^3.0.0" - -web-streams-polyfill@^3.0.3: - version "3.3.3" - resolved "https://registry.npmjs.org/web-streams-polyfill/-/web-streams-polyfill-3.3.3.tgz" - integrity sha512-d2JWLCivmZYTSIoge9MsgFCZrt571BikcWGYkjC1khllbTeDlGqZ2D8vD8E/lJa8WGWbb7Plm8/XJYV7IJHZZw== - -webidl-conversions@^7.0.0: - version "7.0.0" - resolved "https://registry.npmjs.org/webidl-conversions/-/webidl-conversions-7.0.0.tgz" - integrity sha512-VwddBukDzu71offAQR975unBIGqfKZpM+8ZX6ySk8nYhVoo5CYaZyzt3YBvYtRtO+aoGlqxPg/B87NGVZ/fu6g== - -"whatwg-url@^11.0.0 || ^12.0.0 || ^13.0.0 || ^14.0.0", "whatwg-url@^14.1.0 || ^13.0.0": - version "14.2.0" - resolved "https://registry.npmjs.org/whatwg-url/-/whatwg-url-14.2.0.tgz" - integrity sha512-De72GdQZzNTUBBChsXueQUnPKDkg/5A5zp7pFDuQAj5UFoENpiACU0wlCvzpAGnTkj++ihpKwKyYewn/XNUbKw== - dependencies: - tr46 "^5.1.0" - webidl-conversions "^7.0.0" - -which-boxed-primitive@^1.1.0, which-boxed-primitive@^1.1.1: - version "1.1.1" - resolved "https://registry.npmjs.org/which-boxed-primitive/-/which-boxed-primitive-1.1.1.tgz" - integrity sha512-TbX3mj8n0odCBFVlY8AxkqcHASw3L60jIuF8jFP78az3C2YhmGvqbHBpAjTRH2/xqYunrJ9g1jSyjCjpoWzIAA== - dependencies: - is-bigint "^1.1.0" - is-boolean-object "^1.2.1" - is-number-object "^1.1.1" - is-string "^1.1.1" - is-symbol "^1.1.1" - -which-builtin-type@^1.2.1: - version "1.2.1" - resolved "https://registry.npmjs.org/which-builtin-type/-/which-builtin-type-1.2.1.tgz" - integrity sha512-6iBczoX+kDQ7a3+YJBnh3T+KZRxM/iYNPXicqk66/Qfm1b93iu+yOImkg0zHbj5LNOcNv1TEADiZ0xa34B4q6Q== - dependencies: - call-bound "^1.0.2" - function.prototype.name "^1.1.6" - has-tostringtag "^1.0.2" - is-async-function "^2.0.0" - is-date-object "^1.1.0" - is-finalizationregistry "^1.1.0" - is-generator-function "^1.0.10" - is-regex "^1.2.1" - is-weakref "^1.0.2" - isarray "^2.0.5" - which-boxed-primitive "^1.1.0" - which-collection "^1.0.2" - which-typed-array "^1.1.16" - -which-collection@^1.0.2: - version "1.0.2" - resolved "https://registry.npmjs.org/which-collection/-/which-collection-1.0.2.tgz" - integrity sha512-K4jVyjnBdgvc86Y6BkaLZEN933SwYOuBFkdmBu9ZfkcAbdVbpITnDmjvZ/aQjRXQrv5EPkTnD1s39GiiqbngCw== - dependencies: - is-map "^2.0.3" - is-set "^2.0.3" - is-weakmap "^2.0.2" - is-weakset "^2.0.3" - -which-typed-array@^1.1.16, which-typed-array@^1.1.19: - version "1.1.19" - resolved "https://registry.npmjs.org/which-typed-array/-/which-typed-array-1.1.19.tgz" - integrity sha512-rEvr90Bck4WZt9HHFC4DJMsjvu7x+r6bImz0/BrbWb7A2djJ8hnZMrWnHo9F8ssv0OMErasDhftrfROTyqSDrw== - dependencies: - available-typed-arrays "^1.0.7" - call-bind "^1.0.8" - call-bound "^1.0.4" - for-each "^0.3.5" - get-proto "^1.0.1" - gopd "^1.2.0" - has-tostringtag "^1.0.2" - -which@^2.0.1: - version "2.0.2" - resolved "https://registry.npmjs.org/which/-/which-2.0.2.tgz" - integrity sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA== - dependencies: - isexe "^2.0.0" - -wrap-ansi@^7.0.0: - version "7.0.0" - resolved "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz" - integrity sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q== - dependencies: - ansi-styles "^4.0.0" - string-width "^4.1.0" - strip-ansi "^6.0.0" - -wrappy@1: - version "1.0.2" - resolved "https://registry.npmjs.org/wrappy/-/wrappy-1.0.2.tgz" - integrity sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ== - -wsl-utils@^0.1.0: - version "0.1.0" - resolved "https://registry.npmjs.org/wsl-utils/-/wsl-utils-0.1.0.tgz" - integrity sha512-h3Fbisa2nKGPxCpm89Hk33lBLsnaGBvctQopaBSOW/uIs6FTe1ATyAnKFJrzVs9vpGdsTe73WF3V4lIsk4Gacw== - dependencies: - is-wsl "^3.1.0" - -y18n@^5.0.5: - version "5.0.8" - resolved "https://registry.npmjs.org/y18n/-/y18n-5.0.8.tgz" - integrity sha512-0pfFzegeDWJHJIAmTLRP2DwHjdF5s7jo9tuztdQxAhINCdvS+3nGINqPd00AphqJR/0LhANUS6/+7SCb98YOfA== - -yargs-parser@^20.2.4: - version "20.2.9" - resolved "https://registry.npmjs.org/yargs-parser/-/yargs-parser-20.2.9.tgz" - integrity sha512-y11nGElTIV+CT3Zv9t7VKl+Q3hTQoT9a1Qzezhhl6Rp21gJ/IVTW7Z3y9EWXhuUBC2Shnf+DX0antecpAwSP8w== - -yargs-parser@^21.1.1: - version "21.1.1" - resolved "https://registry.npmjs.org/yargs-parser/-/yargs-parser-21.1.1.tgz" - integrity sha512-tVpsJW7DdjecAiFpbIB1e3qxIQsE6NoPc5/eTdrbbIC4h0LVsWhnoa3g+m2HclBIujHzsxZ4VJVA+GUuc2/LBw== - -yargs@^17.6.2: - version "17.7.2" - resolved "https://registry.npmjs.org/yargs/-/yargs-17.7.2.tgz" - integrity sha512-7dSzzRQ++CKnNI/krKnYRV7JKKPUXMEh61soaHKg9mrWEhzFWhFnxPxGl+69cD1Ou63C13NUPCnmIcrvqCuM6w== - dependencies: - cliui "^8.0.1" - escalade "^3.1.1" - get-caller-file "^2.0.5" - require-directory "^2.1.1" - string-width "^4.2.3" - y18n "^5.0.5" - yargs-parser "^21.1.1" - -zod-to-json-schema@^3.25.0: - version "3.25.0" - resolved "https://registry.npmjs.org/zod-to-json-schema/-/zod-to-json-schema-3.25.0.tgz" - integrity sha512-HvWtU2UG41LALjajJrML6uQejQhNJx+JBO9IflpSja4R03iNWfKXrj6W2h7ljuLyc1nKS+9yDyL/9tD1U/yBnQ== - -"zod@^3.25 || ^4", "zod@^3.25 || ^4.0", zod@^3.25.76, "zod@^3.25.76 || ^4", "zod@^3.25.76 || ^4.1.8": - version "3.25.76" - resolved "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz" - integrity sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ== diff --git a/Moncter/Moncter/提示词/106服务器消费记录采集表分析.md b/Moncter/Moncter/提示词/106服务器消费记录采集表分析.md deleted file mode 100644 index 7c9e4953..00000000 --- a/Moncter/Moncter/提示词/106服务器消费记录采集表分析.md +++ /dev/null @@ -1,164 +0,0 @@ -# 106服务器消费记录采集表分析报告 - -## 分析时间 -2025年12月 - -## 消费记录Handler字段要求 - -### 必填字段 -- `amount` (消费金额) - 必填 -- `actual_amount` (实际支付金额) - 必填 -- `consume_time` (消费时间) - 必填 - -### 用户标识字段（三选一，Handler会自动解析） -- `phone_number` (手机号) - 推荐 -- `id_card` (身份证) - 推荐 -- `user_id` (用户ID) - 如果源数据已有可直接使用 - -### 门店标识字段（二选一，Handler会自动转换） -- `store_name` (门店名称) - 推荐 -- `store_id` (门店ID) - 如果源数据已有可直接使用 - -### 可选字段 -- `currency` (币种) - 默认CNY -- `status` (记录状态) - 默认0 - ---- - -## 支持消费记录采集的表 - -### ✅ 1. KR_商城.21年贝蒂喜订单整合 - -**数据库**: `KR_商城` -**集合**: `21年贝蒂喜订单整合` -**记录数**: 10,439条 -**数据状态**: ✅ 可用 - -#### 字段映射关系 - -| 目标字段 | 源字段 | 字段类型 | 说明 | -|---------|--------|---------|------| -| `phone_number` | `联系手机` | String | 手机号（需去除单引号前缀） | -| `store_name` | `店铺名称` | String | 门店名称（如：贝蒂喜旗舰店） | -| `amount` | `买家应付货款` 或 `总金额` | String→Float | 消费金额（需转换为数字） | -| `actual_amount` | `买家实际支付金额` | String→Float | 实际支付金额（需转换为数字，可能为"0"） | -| `consume_time` | `订单付款时间` 或 `订单创建时间` | String→DateTime | 消费时间（格式：YYYY-MM-DD HH:mm:ss） | -| `store_id` | `店铺Id` | String | 门店ID（如："0"） | -| `status` | `订单状态` | String→Int | 订单状态（需转换：交易成功 →0，交易关闭 →1） | - -#### 字段说明 - -**源字段详情**： -- `联系手机`: 格式为 `'13759198903`（带单引号前缀），需要处理 -- `买家实际支付金额`: 可能是字符串 "0"（表示未支付），需要过滤或处理 -- `订单付款时间`: 可能为 null（未支付订单），优先使用此字段 -- `订单创建时间`: 作为备用时间字段 -- `订单状态`: 示例值："卖家已发货，等待买家确认"、"交易关闭" 等 - -#### 数据示例 -```json -{ - "_id": "68ad49c51d4abb1611aee2b9", - "联系手机": "'13759198903", - "买家应付货款": "248", - "总金额": "248", - "买家实际支付金额": "248", - "订单状态": "卖家已发货，等待买家确认", - "订单创建时间": "2021-01-31 23:44:53", - "订单付款时间": "2021-01-31 23:45:06", - "店铺名称": "贝蒂喜旗舰店", - "店铺Id": "0" -} -``` - -#### 推荐配置 - -**字段映射配置**： -```json -{ - "phone_number": "联系手机", - "amount": "买家应付货款", - "actual_amount": "买家实际支付金额", - "consume_time": "订单付款时间", - "store_name": "店铺名称" -} -``` - -**转换函数**： -- `联系手机`: 使用 `parse_phone` 去除单引号并验证 -- `买家应付货款` / `买家实际支付金额`: 使用 `parse_amount` 转换为数字 -- `订单付款时间`: 使用 `parse_datetime` 解析时间 -- `订单状态`: 自定义转换逻辑（正常→0，关闭→2） - -**过滤条件建议**： -- 过滤 `买家实际支付金额` 为 "0" 或 null 的记录（未支付订单） -- 过滤 `订单付款时间` 为 null 的记录（未支付订单） -- 或保留所有记录，在Handler中根据状态判断 - ---- - -## 其他数据库分析 - -### ❌ 2. KR_商城.凡客诚品_vancl.com -**状态**: ❌ 不支持 -**原因**: 仅包含地址、姓名、电话信息，无金额、时间等消费记录字段 - -### ❌ 3. KR_商城.嘟嘟牛 -**状态**: ❌ 不支持 -**原因**: 仅包含邮箱、用户名、密码等账户信息，无消费记录字段 - -### ❌ 4. KR_商城.购物-北京一电购公司2月整理版30万 -**状态**: ❌ 不支持 -**原因**: 仅包含号码、名字、省市等基本信息，无消费记录字段 - -### ❌ 5. KR_淘宝.卖家邮箱（去重复后300万） -**状态**: ❌ 不支持 -**原因**: 仅包含邮箱字段，无消费记录相关信息 - -### ❌ 6. KR_卡若私域.老坑爹商店 shop.lkdie.com -**状态**: ❌ 不支持 -**原因**: 包含用户账户信息（邮箱、手机、密码等），但无订单、支付等消费记录字段 - ---- - -## 总结 - -### 可用的表 -1. **KR_商城.21年贝蒂喜订单整合** ✅ - -### 使用建议 - -1. **数据预处理**： - - 手机号字段需要去除单引号前缀 - - 金额字段需要从字符串转换为数字 - - 时间字段格式为 `YYYY-MM-DD HH:mm:ss`，需要解析 - -2. **数据过滤**： - - 建议过滤未支付的订单（`买家实际支付金额` 为 "0" 或 `订单付款时间` 为 null） - - 或根据 `订单状态` 字段过滤无效订单 - -3. **字段映射**： - - 使用 `订单付款时间` 作为 `consume_time`（优先） - - 如果 `订单付款时间` 为空，可以使用 `订单创建时间` 作为备用 - - `店铺名称` 可以映射到 `store_name`，Handler会自动转换为 `store_id` - -4. **注意事项**： - - 该集合包含2021年的订单数据 - - 数据量：10,439条记录 - - 部分订单可能未支付，需要根据业务需求决定是否采集 - ---- - -## 下一步操作 - -1. 在TaskForm中创建采集任务 -2. 配置数据源：选择106服务器（ckb数据库） -3. 选择Handler：`消费记录处理（ConsumptionCollectionHandler）` -4. 配置源数据： - - 数据库：`KR_商城` - - 集合：`21年贝蒂喜订单整合` -5. 配置字段映射（按上述映射关系） -6. 配置过滤条件（可选）：过滤未支付订单 -7. 预览查询结果验证字段映射 -8. 保存并启动采集任务 - diff --git a/Moncter/Moncter/提示词/QueryBuilder折叠功能说明.md b/Moncter/Moncter/提示词/QueryBuilder折叠功能说明.md deleted file mode 100644 index 99d01a45..00000000 --- a/Moncter/Moncter/提示词/QueryBuilder折叠功能说明.md +++ /dev/null @@ -1,319 +0,0 @@ -# QueryBuilder 折叠功能说明 - -## 更新概览 - -为了改善用户体验，QueryBuilder 组件的过滤条件和联表查询区域现在支持折叠，默认为折叠状态，让界面更加简洁。 - ---- - -## 功能特性 - -### 1. 默认折叠状态 - -**过滤条件（WHERE）**： -- 默认：折叠 -- 图标：`→` (折叠) / `↓` (展开) -- 点击标题栏任意位置即可切换 - -**联表查询（JOIN/LOOKUP）**： -- 默认：折叠 -- 图标：`→` (折叠) / `↓` (展开) -- 点击标题栏任意位置即可切换 - -### 2. 智能展开 - -**自动展开时机**： -- 点击"添加条件"按钮 → 自动展开过滤条件区域 -- 点击"添加关联"按钮 → 自动展开联表查询区域 - -**好处**： -- 用户添加配置时自动展开，无需手动操作 -- 提升操作流畅度 - -### 3. 条件数量提示 - -**过滤条件**： -- 显示绿色标签：`3 个条件` -- 一目了然当前配置数量 - -**联表查询**： -- 显示橙色标签：`2 个关联` -- 直观了解关联表数量 - ---- - -## 界面效果 - -### 折叠状态（默认） - -``` -┌─ 基础配置 ─────────────────────────────────────────┐ -│ 数据源：[MongoDB数据源] │ -│ 数据库：[ckb] │ -│ 主集合：[consumption_records_202101] │ -└────────────────────────────────────────────────────┘ - -┌─ → 过滤条件（WHERE） [3个条件] [添加条件] ───┐ -│ (折叠状态，内容隐藏) │ -└────────────────────────────────────────────────────┘ - -┌─ → 联表查询（JOIN/LOOKUP） [2个关联] [添加关联] ─┐ -│ (折叠状态，内容隐藏) │ -└────────────────────────────────────────────────────┘ - -┌─ 排序和限制 ───────────────────────────────────────┐ -│ 排序字段：[create_time] 排序方式：[降序] │ -│ 限制数量：[1000] │ -└────────────────────────────────────────────────────┘ -``` - -### 展开状态 - -``` -┌─ ↓ 过滤条件（WHERE） [3个条件] [添加条件] ───┐ -│ ┌──────────────────────────────────────────────────┐│ -│ │ 逻辑 | 字段 | 运算符 | 值 | 操作 ││ -│ │ - | status | 等于 | success | [删除] ││ -│ │ AND | amount | 大于 | 1000 | [删除] ││ -│ │ AND | shop_name | 包含 | 淘宝 | [删除] ││ -│ └──────────────────────────────────────────────────┘│ -└────────────────────────────────────────────────────┘ - -┌─ ↓ 联表查询（JOIN/LOOKUP） [2个关联] [添加关联] ─┐ -│ ┌──────────────────────────────────────────────────┐│ -│ │ 关联集合 | 主字段 | 关联字段 | 结果名 | 操作 ││ -│ │ user_profile | user_id | user_id | user | [删除]││ -│ │ stores | store_id | _id | store_info | [删除] ││ -│ └──────────────────────────────────────────────────┘│ -└────────────────────────────────────────────────────┘ -``` - ---- - -## 交互设计 - -### 标题栏样式 - -**视觉提示**： -- 鼠标悬停：标题栏背景变为浅灰色 `#f5f7fa` -- 光标变化：`cursor: pointer` -- 过渡动画：0.2s 平滑过渡 - -**元素组成**： -``` -[图标 →/↓] [标题] [条件数量标签] [操作按钮] -``` - -### 点击行为 - -**标题栏点击**： -- 点击标题栏任意位置 → 切换折叠/展开状态 -- 图标跟随变化 - -**按钮点击**： -- 使用 `@click.stop` 阻止事件冒泡 -- 点击"添加条件/关联"按钮不会触发折叠切换 -- 但会自动展开对应区域 - ---- - -## 技术实现 - -### 1. 折叠状态管理 - -```typescript -// 折叠状态 -const collapseStates = reactive({ - filter: false, // 过滤条件默认折叠 - lookup: false // 联表查询默认折叠 -}) -``` - -### 2. 模板结构 - -```vue - - - - - -
- -
- - -``` - -### 3. 自动展开逻辑 - -```typescript -// 添加过滤条件 -const handleAddFilter = () => { - if (!hasCollection.value) { - ElMessage.warning('请先选择主集合') - return - } - - // 添加新条件 - queryConfig.filter.push({ - logic: queryConfig.filter.length > 0 ? 'and' : undefined, - field: '', - operator: 'eq', - value: '' - }) - - // 自动展开区域 - collapseStates.filter = true -} -``` - -### 4. 样式设计 - -```scss -.card-header { - display: flex; - justify-content: space-between; - align-items: center; - font-weight: 500; - font-size: 14px; - - // 针对可点击的标题栏 - &[style*="cursor: pointer"] { - user-select: none; - transition: background-color 0.2s; - margin: -12px -20px; - padding: 12px 20px; - border-radius: 4px; - - &:hover { - background-color: #f5f7fa; - } - } -} -``` - ---- - -## 用户操作流程 - -### 场景1：查看已有配置 - -**用户动作**： -1. 打开数据列表配置页面 -2. 看到折叠的过滤条件和联表查询 -3. 标签显示："3 个条件"、"2 个关联" - -**优势**： -- 一眼看出配置数量 -- 界面简洁，不需要滚动 - -### 场景2：添加新配置 - -**用户动作**： -1. 点击"添加条件"按钮 -2. 区域自动展开 -3. 开始配置新条件 - -**优势**： -- 无需手动展开 -- 操作流畅，减少点击次数 - -### 场景3：查看详细配置 - -**用户动作**： -1. 点击标题栏 -2. 展开查看详细配置 -3. 再次点击可折叠 - -**优势**： -- 快速切换查看状态 -- 标题栏整行可点击，点击区域大 - ---- - -## 优势总结 - -### 1. 界面简洁 -- 默认折叠，减少视觉干扰 -- 特别适合复杂配置场景 -- 聚焦当前操作 - -### 2. 信息密度优化 -- 数量标签直观显示配置数量 -- 不需要展开即可了解配置情况 -- 快速定位需要修改的区域 - -### 3. 操作便捷 -- 整行标题栏可点击 -- 自动展开机制 -- 视觉反馈清晰 - -### 4. 性能优化 -- 折叠状态下不渲染内容 -- 大量条件时渲染性能更好 -- 使用 `el-collapse-transition` 平滑动画 - ---- - -## 兼容性说明 - -### 向后兼容 -- 不影响现有数据结构 -- 只是UI层面的改进 -- 所有功能保持不变 - -### 默认行为 -- 新建时：默认折叠 -- 编辑时：默认折叠 -- 添加配置时：自动展开 - -### 状态保持 -- 折叠状态不保存 -- 每次进入页面都是默认折叠 -- 符合大多数用户习惯 - ---- - -## 相关组件 - -使用了以下 Element Plus 组件： -- `el-collapse-transition`: 折叠动画 -- `el-icon`: 图标组件 -- `el-tag`: 数量标签 - -使用了以下图标： -- `ArrowRight`: 折叠状态（→） -- `ArrowDown`: 展开状态（↓） - ---- - -**更新时间**：2025-01-XX -**版本**：QueryBuilder v2.1 -**更新内容**：添加折叠功能，优化界面布局 diff --git a/Moncter/Moncter/提示词/可视化查询构建器使用说明.md b/Moncter/Moncter/提示词/可视化查询构建器使用说明.md deleted file mode 100644 index 895bdfbb..00000000 --- a/Moncter/Moncter/提示词/可视化查询构建器使用说明.md +++ /dev/null @@ -1,311 +0,0 @@ -# 可视化查询构建器使用说明 - -## 一、概述 - -可视化查询构建器是一个用于配置MongoDB查询的UI组件，支持通过图形界面配置复杂的查询条件，包括过滤条件、联表查询、排序等功能。 - ---- - -## 二、功能特性 - -### 2.1 基础配置 -- **数据源选择**：选择已配置的数据源 -- **数据库选择**：选择数据源中的数据库 -- **主集合选择**：选择查询的主集合（表） - -### 2.2 过滤条件（WHERE） -- **逻辑关系**：支持 AND/OR 逻辑组合 -- **字段选择**：从集合字段列表中选择 -- **运算符**： - - `eq` - 等于 - - `ne` - 不等于 - - `gt` - 大于 - - `gte` - 大于等于 - - `lt` - 小于 - - `lte` - 小于等于 - - `in` - 包含（数组） - - `nin` - 不包含（数组） - - `regex` - 正则匹配 - - `exists` - 字段存在 -- **值输入**：根据字段类型自动调整输入方式 - -### 2.3 联表查询（JOIN/LOOKUP） -- **关联集合**：选择要关联的其他集合 -- **主集合字段**：主集合的关联字段 -- **关联集合字段**：关联集合的关联字段 -- **结果字段名**：关联结果存储的字段名 -- **解构**：是否将数组结果展开为对象 -- **保留空值**：LEFT JOIN效果，保留没有关联的记录 - -### 2.4 排序和限制 -- **排序字段**：选择排序的字段 -- **排序方式**：升序/降序 -- **限制数量**：限制返回的记录数 - -### 2.5 查询预览 -- **SQL预览**：实时显示生成的MongoDB聚合管道代码 -- **数据预览**：预览查询结果（最多显示预览数据） - ---- - -## 三、使用流程 - -### 3.1 创建数据列表 - -1. **进入数据列表管理页面** - - 路径：`/tag-data-lists` - - 点击"创建数据列表"按钮 - -2. **填写基本信息** - - 列表名称：如"消费记录表" - - 列表编码：如"consumption_records" - - 描述：可选 - - 状态：启用/禁用 - -3. **配置查询** - - 选择数据源 - - 选择数据库 - - 选择主集合 - - 添加过滤条件（可选） - - 添加联表查询（可选） - - 配置排序和限制（可选） - -4. **预览查询** - - 点击"预览数据"按钮 - - 查看生成的MongoDB查询代码 - - 查看预览数据 - -5. **保存配置** - - 点击"保存"按钮 - - 数据列表配置保存成功 - -### 3.2 在标签定义中使用 - -1. **创建标签定义** - - 进入标签定义管理页面 - - 点击"创建标签定义" - -2. **选择数据列表** - - 在"数据列表"下拉框中选择已创建的数据列表 - - 系统自动加载该列表的字段 - -3. **配置规则** - - 选择规则类型（运算规则/正则规则） - - 添加规则条件 - - 每个条件选择字段、运算符、值、标签值 - -4. **保存标签定义** - ---- - -## 四、配置示例 - -### 4.1 简单查询示例 - -**场景**：查询消费记录表中金额大于1000的记录 - -**配置**： -- 数据源：选择MongoDB数据源 -- 数据库：`tag_engine` -- 主集合：`consumption_records` -- 过滤条件： - - 字段：`amount` - - 运算符：`gt` - - 值：`1000` -- 排序：按 `create_time` 降序 -- 限制：1000条 - -**生成的查询**： -```javascript -db.consumption_records.aggregate([ - { $match: { amount: { $gt: 1000 } } }, - { $sort: { create_time: -1 } }, - { $limit: 1000 } -]) -``` - -### 4.2 联表查询示例 - -**场景**：查询消费记录，并关联用户信息 - -**配置**： -- 数据源：选择MongoDB数据源 -- 数据库：`tag_engine` -- 主集合：`consumption_records` -- 联表查询： - - 关联集合：`user_profile` - - 主集合字段：`user_id` - - 关联集合字段：`user_id` - - 结果字段名：`user_info` - - 解构：是 - - 保留空值：是 -- 过滤条件： - - 字段：`status` - - 运算符：`eq` - - 值：`success` -- 限制：1000条 - -**生成的查询**： -```javascript -db.consumption_records.aggregate([ - { $match: { status: { $eq: "success" } } }, - { $lookup: { - from: "user_profile", - localField: "user_id", - foreignField: "user_id", - as: "user_info" - } }, - { $unwind: { - path: "$user_info", - preserveNullAndEmptyArrays: true - } }, - { $limit: 1000 } -]) -``` - -### 4.3 复杂条件查询示例 - -**场景**：查询最近30天、金额大于1000、状态为成功的消费记录 - -**配置**： -- 数据源：选择MongoDB数据源 -- 数据库：`tag_engine` -- 主集合：`consumption_records` -- 过滤条件： - 1. 字段：`amount`，运算符：`gt`，值：`1000`，逻辑：- - 2. 字段：`status`，运算符：`eq`，值：`success`，逻辑：AND - 3. 字段：`create_time`，运算符：`gte`，值：`2024-12-01`，逻辑：AND -- 排序：按 `create_time` 降序 -- 限制：1000条 - -**生成的查询**： -```javascript -db.consumption_records.aggregate([ - { $match: { - amount: { $gt: 1000 }, - status: { $eq: "success" }, - create_time: { $gte: ISODate("2024-12-01T00:00:00Z") } - } }, - { $sort: { create_time: -1 } }, - { $limit: 1000 } -]) -``` - ---- - -## 五、技术实现 - -### 5.1 组件结构 - -``` -QueryBuilder/ - ├── QueryBuilder.vue # 主组件 - └── index.ts # 导出（可选） -``` - -### 5.2 数据格式 - -**输入格式**（v-model）： -```typescript -{ - data_source_id: string - database: string - collection: string - filter: Array<{ - logic?: 'and' | 'or' - field: string - operator: string - value: any - }> - lookups: Array<{ - from: string - local_field: string - foreign_field: string - as: string - unwrap?: boolean - preserve_null?: boolean - }> - sort_field: string - sort_order: '1' | '-1' - limit: number -} -``` - -**输出格式**（保存到数据库）： -```json -{ - "list_id": "uuid", - "list_name": "消费记录表", - "list_code": "consumption_records", - "data_source_id": "source_123", - "database": "tag_engine", - "collection": "consumption_records", - "query_config": { - "filter": [ - { - "field": "amount", - "operator": "gt", - "value": 1000 - } - ], - "lookups": [ - { - "from": "user_profile", - "local_field": "user_id", - "foreign_field": "user_id", - "as": "user_info", - "unwrap": true, - "preserve_null": true - } - ], - "sort": { - "create_time": -1 - }, - "limit": 1000 - } -} -``` - -### 5.3 API接口 - -**需要实现的接口**： - -1. `GET /api/data-sources` - 获取数据源列表 -2. `GET /api/data-sources/{id}/databases` - 获取数据库列表 -3. `GET /api/data-sources/{id}/collections` - 获取集合列表（需要database参数） -4. `GET /api/data-sources/{id}/fields` - 获取字段列表（需要database和collection参数） -5. `POST /api/data-sources/preview-query` - 预览查询结果 -6. `GET /api/tag-data-lists` - 获取数据列表列表 -7. `POST /api/tag-data-lists` - 创建数据列表 -8. `GET /api/tag-data-lists/{id}` - 获取数据列表详情 -9. `PUT /api/tag-data-lists/{id}` - 更新数据列表 -10. `DELETE /api/tag-data-lists/{id}` - 删除数据列表 -11. `GET /api/tag-data-lists/{id}/fields` - 获取数据列表字段（用于标签定义） - ---- - -## 六、使用注意事项 - -1. **数据源配置**：必须先配置数据源，才能使用查询构建器 -2. **字段类型**：系统会自动识别字段类型，调整输入方式 -3. **联表查询**：支持多个联表查询，按顺序执行 -4. **性能考虑**：建议设置合理的limit值，避免查询过多数据 -5. **预览功能**：预览数据最多显示一定数量，用于验证查询配置是否正确 - ---- - -## 七、扩展功能 - -### 7.1 未来可扩展的功能 - -1. **聚合函数**：支持 $group、$sum、$avg 等聚合操作 -2. **子查询**：支持嵌套查询 -3. **条件分支**：支持 $cond、$switch 等条件表达式 -4. **字段映射**：支持字段重命名、计算字段 -5. **查询模板**：保存常用查询为模板 -6. **查询历史**：记录查询历史，支持回滚 - ---- - -**文档更新时间**：2025-01-XX -**组件版本**：v1.0.0 diff --git a/Moncter/Moncter/提示词/多集合模式使用说明.md b/Moncter/Moncter/提示词/多集合模式使用说明.md deleted file mode 100644 index 2ba6175d..00000000 --- a/Moncter/Moncter/提示词/多集合模式使用说明.md +++ /dev/null @@ -1,425 +0,0 @@ -# 多集合模式使用说明 - -## 问题背景 - -在实际业务中，**消费记录表**并非单一集合，而是由多个集合构成： - -### 场景1：按月分片的消费记录 -在 `ckb` 数据库中，消费记录按月分片存储： -- `consumption_records_202101` -- `consumption_records_202102` -- `consumption_records_202103` -- ... (每月一个集合) - -### 场景2：按商品类型分散的消费记录 -在 `KR_淘宝` 数据库中，消费记录按商品类型分散： -- 女士内衣（132.6万条） -- 办公设备文具（64.5万条） -- 包（71万条） -- zippo1, zippo2, ... (多个集合) - -**问题**：如何在数据列表配置中选择这些分散的消费记录表？ - ---- - -## 解决方案：多集合模式 - -QueryBuilder 组件现已支持**多集合模式**，允许同时选择多个集合，查询时自动合并数据。 - ---- - -## 使用方法 - -### 1. 启用多集合模式 - -在数据列表配置页面（QueryBuilder 基础配置区域）： - -1. 选择数据源 -2. 选择数据库 -3. **开启"多集合模式"开关** - -``` -┌─ 基础配置 ─────────────────────────────────┐ -│ 数据源：[MongoDB标签引擎数据源] │ -│ 数据库：[ckb] │ -│ 多集合模式：[●启用] ○禁用 │ -│ 说明：启用后可同时选择多个集合 │ -└─────────────────────────────────────────────┘ -``` - -### 2. 选择多个集合 - -启用后，会显示集合复选框列表，并提供强大的筛选和操作功能： - -#### 筛选功能 -``` -[🔍 筛选集合名称...] -``` -- 输入关键词实时筛选集合列表 -- 例如输入 `2021` 只显示包含 2021 的集合 -- 支持模糊匹配，不区分大小写 - -#### 批量操作按钮 -``` -[全选] - 选择当前筛选结果的所有集合 -[清空] - 清空所有已选集合 -[反选] - 反选当前筛选结果的集合 -``` - -#### 快捷筛选（智能识别） -当检测到按日期分片的集合时，自动显示快捷筛选按钮： -``` -快捷筛选： -[2021年] [2022年] [2023年] [2024年] [2025年] -[最近3个月] [最近6个月] [最近12个月] -``` - -**功能说明**： -- **按年份筛选**：点击 `2021年` 自动选择所有包含 `2021` 的集合 -- **按时间范围**：点击 `最近3个月` 自动选择最近3个月的集合 - - 例如当前是 2025-01，点击"最近3个月"会选择： - - consumption_records_202501 - - consumption_records_202412 - - consumption_records_202411 - -#### 集合列表 -``` -┌─ 集合列表 ─────────────────────────────────┐ -│ ☑ consumption_records_202101 │ -│ ☑ consumption_records_202102 │ -│ ☑ consumption_records_202103 │ -│ ☑ consumption_records_202104 │ -│ ☑ consumption_records_202105 │ -│ ☑ consumption_records_202106 │ -│ ☐ user_profile │ -│ ☐ user_tags_shard_0 │ -│ ... (最多300px高度，超出可滚动) │ -└─────────────────────────────────────────────┘ - -筛选结果：28 个集合 | 已选择 6 个集合 -查询时将自动合并这些集合的数据 -``` - -### 3. 配置查询条件 - -多集合模式下，可以正常配置： -- 过滤条件（WHERE） -- 联表查询（JOIN） -- 排序和限制 - -字段列表会从**第一个选中的集合**加载。 - -### 4. 预览查询 - -点击"预览数据"，SQL预览会显示： - -```javascript -// 多集合模式：将查询以下 6 个集合并合并结果 -// consumption_records_202101, consumption_records_202102, consumption_records_202103, consumption_records_202104, consumption_records_202105, consumption_records_202106 - -db.consumption_records_202101.aggregate([ - { $match: { status: { $eq: "success" } } }, - { $sort: { create_time: -1 } }, - { $limit: 1000 } -]) -``` - ---- - -## 完整示例 - -### 示例1：按月分片的消费记录（使用快捷筛选） - -**需求**：创建一个包含2021年所有消费记录的数据列表 - -**步骤**： -1. 列表名称：`2021年全年消费记录` -2. 列表编码：`consumption_records_2021` -3. 数据源：MongoDB标签引擎 -4. 数据库：`ckb` -5. **启用多集合模式** -6. **点击快捷筛选按钮 `2021年`** 👈 自动选择所有2021年的集合！ - - ✅ 自动选中： - - consumption_records_202101 - - consumption_records_202102 - - consumption_records_202103 - - ... (所有12个月) - - 💡 提示："已选择 12 个集合" -7. 添加过滤条件（可选）： - - 字段：`status` - - 运算符：`等于` - - 值：`success` -8. 保存 - -**传统方式 vs 快捷筛选**： -- ❌ 传统方式：需要手动勾选12个复选框 -- ✅ 快捷筛选：点击1次按钮即可 - -### 示例1-2：最近半年消费记录（智能时间范围） - -**需求**：创建一个包含最近6个月消费记录的数据列表 - -**步骤**： -1. 列表名称：`最近半年消费记录` -2. 列表编码：`consumption_records_recent_6m` -3. 数据源：MongoDB标签引擎 -4. 数据库：`ckb` -5. **启用多集合模式** -6. **点击快捷筛选按钮 `最近6个月`** 👈 自动选择最近6个月的集合！ - - 系统自动计算时间范围 - - 如果当前是 2025-01，则选择： - - consumption_records_202501 - - consumption_records_202412 - - consumption_records_202411 - - consumption_records_202410 - - consumption_records_202409 - - consumption_records_202408 -7. 保存 - -### 示例2：使用筛选功能精确选择 - -**需求**：只选择2021年第一季度的消费记录 - -**步骤**： -1. 启用多集合模式 -2. **在筛选框输入 `202101`** - - 筛选结果：只显示 `consumption_records_202101` -3. **点击 `全选`** 按钮 -4. **清空筛选框，输入 `202102`** -5. **点击 `全选`** 按钮（追加选择） -6. **清空筛选框，输入 `202103`** -7. **点击 `全选`** 按钮（追加选择） -8. 最终选中3个集合 - -**高级技巧**： -- 输入 `20210` 可以同时筛选出 202101-202109 -- 点击"全选"后，再输入 `202107`，点击"反选"可以排除7月的数据 - -**保存的数据结构**： -```json -{ - "list_name": "2021年上半年消费记录", - "list_code": "consumption_records_2021_h1", - "data_source_id": "source_001", - "database": "ckb", - "collection": "consumption_records_202101", - "multi_collection": true, - "collections": [ - "consumption_records_202101", - "consumption_records_202102", - "consumption_records_202103", - "consumption_records_202104", - "consumption_records_202105", - "consumption_records_202106" - ], - "query_config": { - "filter": [ - { "field": "status", "operator": "eq", "value": "success" } - ], - "lookups": [], - "sort": { "create_time": -1 }, - "limit": 1000 - } -} -``` - -### 示例3：按商品类型的消费记录（使用文本筛选） - -**需求**：创建一个包含女性用品的消费记录数据列表 - -**步骤**： -1. 列表名称：`女性用品消费记录` -2. 列表编码：`female_products_consumption` -3. 数据源：MongoDB标签引擎 -4. 数据库：`KR_淘宝` -5. **启用多集合模式** -6. **在筛选框输入 `女`** - - 筛选结果显示：女士内衣 -7. **点击 `全选`** -8. **清空筛选框，输入 `包`** -9. **点击 `全选`**（追加选择） -10. 最终选中： - - ☑ 女士内衣（去重复后132.6万） - - ☑ 包（去重复后71万） -11. 保存 - -### 示例4：反选功能的妙用 - -**需求**：选择2021年除了1月和12月以外的所有月份 - -**步骤**： -1. **点击快捷筛选 `2021年`**（选中全年12个月） -2. **在筛选框输入 `202101`** -3. **点击 `反选`**（取消选择1月） -4. **清空筛选框，输入 `202112`** -5. **点击 `反选`**（取消选择12月） -6. 最终选中：202102-202111（10个月） - ---- - -## 技术实现 - -### 前端数据结构 - -```typescript -const queryConfig = reactive({ - data_source_id: string - database: string - collection: string // 单集合模式或多集合的第一个（兼容性） - multi_collection: boolean // 是否启用多集合模式 - collections: string[] // 多集合模式下选中的集合列表 - filter: Array<...> - lookups: Array<...> - sort_field: string - sort_order: string - limit: number -}) -``` - -### 后端查询逻辑（待实现） - -后端在执行查询时，需要处理多集合： - -```php -if ($dataList['multi_collection'] && !empty($dataList['collections'])) { - // 多集合模式：对每个集合执行查询，然后合并结果 - $allResults = []; - foreach ($dataList['collections'] as $collection) { - $results = $this->executeQuery($dataSource, $database, $collection, $queryConfig); - $allResults = array_merge($allResults, $results); - } - - // 如果有排序，需要对合并结果重新排序 - if ($queryConfig['sort']) { - $allResults = $this->sortResults($allResults, $queryConfig['sort']); - } - - // 如果有限制，需要对合并结果应用限制 - if ($queryConfig['limit']) { - $allResults = array_slice($allResults, 0, $queryConfig['limit']); - } - - return $allResults; -} else { - // 单集合模式 - return $this->executeQuery($dataSource, $database, $collection, $queryConfig); -} -``` - ---- - -## API 调整（待实现） - -### 预览查询 API - -需要支持 `collections` 参数： - -**请求**： -```json -POST /data-collection-tasks/preview-query -{ - "data_source_id": "source_001", - "database": "ckb", - "collection": "consumption_records_202101", // 兼容性保留 - "collections": [ // 多集合模式 - "consumption_records_202101", - "consumption_records_202102", - "consumption_records_202103" - ], - "filter_conditions": [...], - "lookups": [...], - "limit": 10 -} -``` - -**响应**： -```json -{ - "code": 200, - "data": { - "fields": [...], - "data": [...], // 合并后的数据 - "count": 30, // 总条数 - "collections_count": { // 各集合的数据条数（可选） - "consumption_records_202101": 10, - "consumption_records_202102": 10, - "consumption_records_202103": 10 - } - } -} -``` - ---- - -## 使用场景 - -### 1. 时间分片数据 -- 消费记录按月/季度/年分表 -- 日志数据按日期分表 -- 订单数据按时间分表 - -### 2. 业务分类数据 -- 按商品类型分表的交易数据 -- 按地区分表的用户数据 -- 按渠道分表的营销数据 - -### 3. 分库分表数据 -- 数据水平切分后的多个分片 -- 跨库查询场景 - ---- - -## 注意事项 - -### 1. 字段一致性 -多个集合应该有**相同或相似的字段结构**，否则合并查询可能出错。 - -### 2. 性能考虑 -- 选择的集合越多，查询性能越慢 -- 建议根据实际需求选择合适的集合范围 -- 可以设置合理的 `limit` 限制返回数据量 - -### 3. 排序和限制 -- 多集合模式下，排序和限制会在**合并后的结果**上应用 -- 如果每个集合返回1000条，3个集合合并后是3000条，再应用limit - -### 4. 联表查询 -- 联表查询在**每个集合上独立执行** -- 关联表应该是同一个集合（不跨集合联表） - ---- - -## 界面效果 - -### 单集合模式（默认） -``` -多集合模式：○启用 ●禁用 -主集合：[consumption_records_202101 ▼] -``` - -### 多集合模式（带筛选和快捷操作） -``` -多集合模式：●启用 ○禁用 - -┌─────────────────────────────────────────────────────┐ -│ [🔍 筛选集合名称...] [全选] [清空] [反选] │ -│ │ -│ 快捷筛选：[2021年] [2022年] [2023年] [2024年] │ -│ [最近3个月] [最近6个月] [最近12个月] │ -├─────────────────────────────────────────────────────┤ -│ 集合列表： │ -│ ☑ consumption_records_202101 │ -│ ☑ consumption_records_202102 │ -│ ☑ consumption_records_202103 │ -│ ☐ consumption_records_202104 │ -│ ☐ consumption_records_202105 │ -│ ... (滚动查看更多) │ -└─────────────────────────────────────────────────────┘ - -已选择 3 个集合，查询时将自动合并这些集合的数据 -``` - ---- - -**更新时间**：2025-01-XX -**状态**：前端已实现，后端待开发 diff --git a/Moncter/Moncter/提示词/字段定义配置管理方案对比.md b/Moncter/Moncter/提示词/字段定义配置管理方案对比.md deleted file mode 100644 index 486bc084..00000000 --- a/Moncter/Moncter/提示词/字段定义配置管理方案对比.md +++ /dev/null @@ -1,491 +0,0 @@ -# 字段定义配置管理方案对比 - -## 一、问题 - -标签定义创建时，需要选择数据源类型（user_profile、user_phone_relations、consumption_records），然后显示该数据源的字段列表供用户选择。 - -**问题**：这些字段列表应该： -1. 存储在数据库中（配置化），通过管理界面动态管理？ -2. 还是写在代码中（纯代码实现），前端直接调用API即可？ - ---- - -## 二、方案对比 - -### 方案一：纯代码实现（⭐推荐第一阶段） - -**实现方式**： -- 字段定义直接写在代码中（Service 或 Config 类） -- API 接口直接返回硬编码的字段列表 -- 需要修改字段时，修改代码并重新部署 - -**代码示例**： - -```php -// app/service/DataSourceFieldService.php -class DataSourceFieldService -{ - /** - * 获取数据源的字段列表 - */ - public function getFields(string $dataSourceType): array - { - $fieldsMap = [ - 'user_profile' => [ - [ - 'field' => 'user_id', - 'type' => 'string', - 'description' => '用户ID', - 'source' => 'user_profile', - ], - [ - 'field' => 'total_amount', - 'type' => 'number', - 'description' => '总消费金额', - 'source' => 'user_profile', - 'pre_aggregated_from' => 'consumption_records', - ], - // ... 更多字段 - ], - 'user_phone_relations' => [ - [ - 'field' => 'phone_number', - 'type' => 'string', - 'description' => '手机号', - 'source' => 'user_phone_relations', - ], - // ... - ], - 'consumption_records' => [ - // 消费记录表的预聚合字段 - [ - 'field' => 'total_amount', - 'type' => 'number', - 'description' => '总消费金额', - 'source' => 'pre_aggregated', - 'original_source' => 'consumption_records', - 'note' => '从消费记录表预聚合到 user_profile', - ], - // ... - ], - ]; - - return $fieldsMap[$dataSourceType] ?? []; - } -} -``` - -**优点**： -- ✅ **实现简单**：代码简洁，无需额外的数据库表和管理界面 -- ✅ **性能好**：直接返回，无需数据库查询 -- ✅ **类型安全**：代码中可以定义完整的类型和验证 -- ✅ **版本控制**：字段定义的变更可以通过 Git 追踪 -- ✅ **易于测试**：单元测试容易编写 - -**缺点**： -- ⚠️ **灵活性较低**：修改字段需要改代码、重新部署 -- ⚠️ **需要开发人员**：非技术人员无法直接修改字段配置 - -**适用场景**： -- 字段定义相对稳定，不频繁变化 -- 第一阶段快速实现 -- 团队较小，技术栈统一 - ---- - -### 方案二：数据库配置（🔶适合长期） - -**实现方式**： -- 创建 `data_source_fields` 集合（MongoDB） -- 存储每个数据源类型的字段定义 -- 提供管理界面，允许管理员动态添加/修改字段 - -**数据结构**： - -```javascript -// data_source_fields 集合 -{ - field_id: "uuid", - data_source_type: "consumption_records", - field: "total_amount", - type: "number", - description: "总消费金额", - source: "pre_aggregated", - original_source: "consumption_records", - note: "从消费记录表预聚合到 user_profile", - is_active: true, - sort_order: 1, - create_time: ISODate(), - update_time: ISODate(), -} -``` - -**代码示例**： - -```php -// app/repository/DataSourceFieldRepository.php -class DataSourceFieldRepository extends Model -{ - protected $table = 'data_source_fields'; - protected $primaryKey = 'field_id'; - // ... -} - -// app/service/DataSourceFieldService.php -class DataSourceFieldService -{ - public function __construct( - protected DataSourceFieldRepository $fieldRepository - ) {} - - public function getFields(string $dataSourceType): array - { - return $this->fieldRepository - ->where('data_source_type', $dataSourceType) - ->where('is_active', true) - ->orderBy('sort_order') - ->get() - ->toArray(); - } -} -``` - -**优点**： -- ✅ **灵活性高**：可以动态添加/修改字段，无需重新部署 -- ✅ **非技术人员友好**：管理员可以通过界面管理字段 -- ✅ **支持多环境**：不同环境可以有不同的字段配置 -- ✅ **支持字段元数据**：可以存储更多信息（如验证规则、默认值等） - -**缺点**： -- ⚠️ **实现复杂**：需要创建 Repository、Service、Controller、前端界面 -- ⚠️ **性能稍差**：每次请求需要查询数据库（可加缓存优化） -- ⚠️ **需要维护**：需要维护字段配置数据的正确性 -- ⚠️ **版本控制困难**：配置变更无法通过 Git 追踪 - -**适用场景**： -- 字段定义频繁变化 -- 有非技术人员需要管理字段配置 -- 需要支持多环境不同配置 -- 长期维护的大型项目 - ---- - -### 方案三：混合方案（🔶平衡方案） - -**实现方式**： -- 基础字段定义写在代码中（作为默认值） -- 支持数据库配置覆盖/扩展 -- 合并代码配置和数据库配置 - -**代码示例**： - -```php -class DataSourceFieldService -{ - /** - * 获取字段列表（代码 + 数据库） - */ - public function getFields(string $dataSourceType): array - { - // 1. 从代码获取默认字段 - $defaultFields = $this->getDefaultFields($dataSourceType); - - // 2. 从数据库获取自定义字段（如果有） - $customFields = $this->fieldRepository - ->where('data_source_type', $dataSourceType) - ->where('is_active', true) - ->get() - ->toArray(); - - // 3. 合并：数据库字段覆盖代码字段（按 field 名匹配） - $merged = []; - foreach ($defaultFields as $field) { - $merged[$field['field']] = $field; - } - foreach ($customFields as $field) { - $merged[$field['field']] = $field; - } - - // 4. 排序 - usort($merged, fn($a, $b) => ($a['sort_order'] ?? 999) <=> ($b['sort_order'] ?? 999)); - - return array_values($merged); - } - - /** - * 从代码获取默认字段 - */ - private function getDefaultFields(string $dataSourceType): array - { - // 硬编码的默认字段定义 - // ... - } -} -``` - -**优点**： -- ✅ **灵活性**：支持动态扩展 -- ✅ **稳定性**：默认字段来自代码，相对稳定 -- ✅ **向后兼容**：即使数据库没有配置，也能正常工作 - -**缺点**： -- ⚠️ **实现更复杂**：需要处理合并逻辑 -- ⚠️ **可能混淆**：代码和数据库配置可能冲突 - ---- - -## 三、项目现状分析 - -### 当前项目的配置管理方式 - -1. **数据源配置** (`data_sources` 集合)： - - ✅ 存储在数据库中 - - ✅ 有管理界面（DataSourceController） - - 说明：业务级配置，需要动态管理 - -2. **标签定义** (`tag_definitions` 集合)： - - ✅ 存储在数据库中 - - ✅ 有管理界面（TagDefinitionController） - - 说明：业务级配置，需要动态管理 - -3. **系统配置** (`config/` 目录)： - - ✅ 配置文件（如 `data_collection_tasks.php`） - - 说明：系统级配置，相对稳定 - -### 字段定义的特点 - -- **相对稳定**：表结构不会频繁变化 -- **与代码耦合**：字段名必须与实际数据库字段一致 -- **需要验证**：字段类型、可用性需要与代码逻辑保持一致 -- **可能扩展**：未来可能需要添加新的预聚合字段 - ---- - -## 四、推荐方案 - -### 推荐：方案一（纯代码实现）+ 后续扩展为方案三 - -#### 第一阶段：纯代码实现（立即实施） - -**理由**： -1. ✅ **快速实现**：代码实现简单，可以快速上线 -2. ✅ **字段相对稳定**：表字段不会频繁变化 -3. ✅ **与代码耦合**：字段名必须与代码中的字段一致，代码管理更安全 -4. ✅ **符合当前项目风格**：系统级配置用代码，业务级配置用数据库 - -**实现步骤**： - -1. 创建 `DataSourceFieldService` 类，硬编码字段定义 -2. 在 `TagDefinitionController` 添加 `getDataSourceFields()` 方法 -3. 前端调用 API 获取字段列表 - -#### 第二阶段：如需扩展，升级为混合方案 - -**如果后续需要**： -- 非技术人员管理字段配置 -- 多环境不同配置 -- 动态添加字段（如新的预聚合字段） - -**则可以升级为方案三**： -- 保留代码中的默认字段定义 -- 添加数据库配置表和管理界面 -- 支持数据库配置覆盖/扩展代码配置 - ---- - -## 五、具体实现建议 - -### 第一阶段实现（纯代码） - -```php -// app/service/DataSourceFieldService.php -> 字段列表 - */ - public function getFields(string $dataSourceType): array - { - $fieldsMap = $this->getFieldsMap(); - return $fieldsMap[$dataSourceType] ?? []; - } - - /** - * 获取所有数据源的字段映射 - * - * @return array>> - */ - private function getFieldsMap(): array - { - return [ - 'user_profile' => [ - [ - 'field' => 'user_id', - 'type' => 'string', - 'description' => '用户ID', - ], - [ - 'field' => 'total_amount', - 'type' => 'number', - 'description' => '总消费金额', - 'pre_aggregated_from' => 'consumption_records', - ], - [ - 'field' => 'total_count', - 'type' => 'number', - 'description' => '总消费次数', - 'pre_aggregated_from' => 'consumption_records', - ], - [ - 'field' => 'last_consume_time', - 'type' => 'datetime', - 'description' => '最后消费时间', - ], - [ - 'field' => 'gender', - 'type' => 'number', - 'description' => '性别（0=女，1=男，2=未知）', - ], - [ - 'field' => 'birthday', - 'type' => 'datetime', - 'description' => '生日', - ], - // ... 更多字段 - ], - - 'user_phone_relations' => [ - [ - 'field' => 'phone_number', - 'type' => 'string', - 'description' => '手机号', - ], - [ - 'field' => 'user_id', - 'type' => 'string', - 'description' => '用户ID', - ], - [ - 'field' => 'effective_time', - 'type' => 'datetime', - 'description' => '生效时间', - ], - [ - 'field' => 'expire_time', - 'type' => 'datetime', - 'description' => '失效时间', - ], - // ... 更多字段 - ], - - 'consumption_records' => [ - // 注意：这些字段实际上来自 user_profile 的预聚合字段 - [ - 'field' => 'total_amount', - 'type' => 'number', - 'description' => '总消费金额', - 'source' => 'pre_aggregated', - 'original_source' => 'consumption_records', - 'note' => '从消费记录表预聚合到 user_profile', - ], - [ - 'field' => 'total_count', - 'type' => 'number', - 'description' => '总消费次数', - 'source' => 'pre_aggregated', - 'original_source' => 'consumption_records', - 'note' => '从消费记录表预聚合到 user_profile', - ], - [ - 'field' => 'last_consume_time', - 'type' => 'datetime', - 'description' => '最后消费时间', - 'source' => 'pre_aggregated', - 'original_source' => 'consumption_records', - ], - // 未来可以添加更多预聚合字段： - // - recent_30_days_amount - // - recent_90_days_amount - // - avg_amount - // - max_amount - // ... - ], - ]; - } -} -``` - -```php -// app/controller/TagDefinitionController.php -// 添加新方法 - -/** - * 获取数据源的字段列表 - * - * GET /api/tag-definitions/data-sources/{dataSourceType}/fields - */ -public function getDataSourceFields(Request $request, string $dataSourceType): Response -{ - try { - $fieldService = new \app\service\DataSourceFieldService(); - $fields = $fieldService->getFields($dataSourceType); - - return ApiResponseHelper::success([ - 'data_source_type' => $dataSourceType, - 'fields' => $fields, - ]); - } catch (\Throwable $e) { - return ApiResponseHelper::exception($e); - } -} -``` - -```php -// config/route.php -// 添加路由 - -Route::get('/api/tag-definitions/data-sources/{dataSourceType}/fields', [TagDefinitionController::class, 'getDataSourceFields']); -``` - ---- - -## 六、总结 - -### 推荐方案 - -**第一阶段：纯代码实现** -- ✅ 简单、快速、稳定 -- ✅ 符合字段定义的特性（与代码耦合、相对稳定） -- ✅ 符合当前项目的配置管理风格 - -**后续如需要：升级为混合方案** -- 保留代码默认字段 -- 添加数据库配置支持动态扩展 -- 两全其美 - -### 关键决策点 - -- **字段定义是否频繁变化？** → 否，代码实现 -- **是否需要非技术人员管理？** → 现阶段不需要，代码实现 -- **是否需要多环境不同配置？** → 不需要，代码实现 -- **字段与代码是否强耦合？** → 是，代码实现更安全 - -**结论**：现阶段推荐纯代码实现，简单高效！ - ---- - -**文档生成时间**: 2025-01-28 diff --git a/Moncter/Moncter/提示词/当前架构设计/人物主表生成逻辑说明.md b/Moncter/Moncter/提示词/当前架构设计/人物主表生成逻辑说明.md deleted file mode 100644 index d862566a..00000000 --- a/Moncter/Moncter/提示词/当前架构设计/人物主表生成逻辑说明.md +++ /dev/null @@ -1,330 +0,0 @@ -# 人物主表生成逻辑说明 - -## 一、设计原则 - -### 1. 用户ID策略 -- **所有用户统一使用 UUID 作为 `user_id`** -- 身份证号只是 `user_profile` 表中的一个字段，不作为主键 -- 转为正式用户时，直接更新身份证相关字段（`id_card_hash`、`id_card_encrypted`），无需变更 `user_id` - -### 2. 表结构说明 -- **`user_profile`（用户主表）**： - - 主键：`user_id` (UUID) - - 身份证字段：`id_card_hash`、`id_card_encrypted`、`id_card_type` - - 标识字段：`is_temporary` (true=临时用户, false=正式用户) - -- **`user_phone_relations`（手机关联表）**： - - 管理手机号与用户的历史关联关系 - - 支持时间窗口：`effective_time`（生效时间）、`expire_time`（失效时间） - - 支持手机号回收后二次分配的场景 - -### 3. 数据来源 -- 主表数据来源于消费记录表（`consumption_records`） -- 消费记录可能来自不同的数据库，时间线可能不一致 - -## 二、核心处理流程 - -### 场景1：消费记录只有手机号，没有身份证号 - -**流程：** -``` -1. 接收消费记录：{ phone_number: "13800138000", id_card: null, consume_time: "2024-01-01 10:00:00" } -2. 使用 consume_time 作为查询时间点，在 user_phone_relations 表中查找该手机号在该时间点有效的关联 -3. 如果找到关联： - - 使用关联的 user_id - - 更新该用户的统计信息（total_amount, total_count, last_consume_time） -4. 如果找不到关联： - - 创建临时用户（is_temporary=true, user_id=UUID） - - 在 user_phone_relations 中建立关联（effective_time = consume_time） - - 更新临时用户的统计信息 -``` - -**关键点：** -- 必须使用 `consume_time` 作为查询时间点，而不是当前时间 -- 临时用户创建后，必须建立手机关联，不能跳过 - -### 场景2：消费记录只有身份证号，没有手机号 - -**流程：** -``` -1. 接收消费记录：{ phone_number: null, id_card: "110101199001011234", consume_time: "2024-01-01 10:00:00" } -2. 通过 id_card_hash 在 user_profile 中查找 -3. 如果找到： - - 使用该 user_id（可能是正式用户，也可能是临时用户） - - 更新统计信息 -4. 如果找不到： - - 创建正式用户（is_temporary=false, user_id=UUID） - - 设置 id_card_hash 和 id_card_encrypted - - 更新统计信息 -``` - -### 场景3：消费记录同时有手机号和身份证号（核心场景 - 触发合并） - -**处理逻辑：** - -#### 情况A：身份证找到用户A，手机号也关联到用户A -``` -→ 直接使用用户A，更新统计信息 -``` - -#### 情况B：身份证找到用户A（正式用户），手机号关联到用户B（可能是临时用户），且 A ≠ B -``` -→ 触发合并逻辑： - 1. 检查用户B是否为临时用户（is_temporary=true） - 2. 如果用户B是临时用户： - a. 合并用户B到用户A（PersonMergeService.mergeUsers(B, A)） - b. 合并内容包括：统计数据、标签、消费记录等 - c. 将手机号从用户B的关联标记为过期（expire_time = consume_time） - d. 建立手机号到用户A的新关联（effective_time = consume_time） - e. 标记用户B为已合并（status=1, merged_from_user_id=A） - f. 使用用户A - 3. 如果用户B也是正式用户（酒店预订等代订场景）： - a. 策略：以身份证为准，消费记录归属到身份证用户（用户A） - b. 手机号关联保持不变（不强制转移，因为可能是代订） - c. 记录异常日志，便于后续人工审核 - d. 使用用户A（身份证用户） -``` - -#### 情况C：身份证找到用户A（正式用户），手机号未关联 -``` -→ 建立手机关联到用户A（effective_time = consume_time） -→ 使用用户A，更新统计信息 -``` - -#### 情况D：身份证未找到，手机号关联到用户B -``` -→ 检查用户B是否为临时用户： - 1. 如果用户B是临时用户： - a. 更新用户B的身份证字段（id_card_hash, id_card_encrypted） - b. 将用户B标记为正式用户（is_temporary=false） - c. 使用用户B - 2. 如果用户B不是临时用户，但身份证不匹配： - a. 创建新的正式用户（user_id=UUID，设置身份证字段） - b. 将手机号从用户B的关联标记为过期（expire_time = consume_time） - c. 建立手机号到新用户的关联（effective_time = consume_time） - d. 使用新用户 -``` - -#### 情况E：身份证未找到，手机号也未关联 -``` -→ 创建正式用户（user_id=UUID, is_temporary=false） -→ 设置身份证字段（id_card_hash, id_card_encrypted） -→ 建立手机关联（effective_time = consume_time） -→ 使用新用户，更新统计信息 -``` - -## 三、时间线冲突处理方案 - -### 问题描述 - -**场景示例：** -``` -2024-01-01: 用户A（身份证I1）使用手机M，产生消费记录 -2024-06-01: 用户A更换手机，手机M不再使用 -2024-12-01: 用户B（身份证I2）开始使用手机M，产生消费记录 -``` - -### 解决方案：基于消费记录时间点的精确匹配 + 智能过期处理 - -#### 1. 查询时使用消费记录的实际时间点 -```php -// 在处理消费记录时，必须传入 consume_time -$consumeTime = new \DateTimeImmutable($payload['consume_time']); -$userId = $this->userPhoneService->findUserByPhone($phoneNumber, $consumeTime); -``` - -#### 2. 手机关联的过期时间设置策略 - -**自动过期检测：** -- 当发现同一手机号需要关联到新用户时，自动将旧关联标记为过期 -- 过期时间设置为新关联的 `effective_time`（保证时间连续，避免间隙） - -**处理逻辑：** -```php -// 当建立新关联时 -if (手机号在 effective_time 已有有效关联 && 关联的用户ID不同) { - 旧关联.expire_time = effective_time; // 标记为过期 - 旧关联.is_active = false; - 创建新关联（effective_time = consume_time）; -} -``` - -#### 3. 时间线匹配示例 - -**处理 2024-01-01 的消费记录（只有手机号M）：** -``` -1. 查询手机M在 2024-01-01 的有效关联 → 未找到 -2. 创建临时用户A（UUID-xxx） -3. 建立手机M关联（effective_time = 2024-01-01, expire_time = null） -``` - -**处理 2024-12-01 的消费记录（手机M + 身份证I2）：** -``` -1. 通过身份证I2查找用户 → 未找到 -2. 查询手机M在 2024-12-01 的有效关联 → 找到用户A的关联（expire_time=null，仍然有效） -3. 检查冲突：用户A的关联有效期包含 2024-12-01 -4. 由于提供了新身份证，判断为手机号回收场景： - a. 将用户A的手机关联标记为过期（expire_time = 2024-12-01） - b. 创建新用户B（UUID-yyy，设置身份证I2） - c. 建立手机M到用户B的新关联（effective_time = 2024-12-01） -``` - -## 四、关键实现要点 - -### 1. 时间点查询机制 -- `UserPhoneService::findUserByPhone()` 必须支持 `$atTime` 参数 -- 查询时使用 `consume_time`，而不是当前时间 -- `UserPhoneRelationRepository::findActiveByPhoneHash()` 需要基于时间窗口查询 - -### 2. 合并触发时机 -- **仅在手机号和身份证号同时出现时触发合并** -- 合并前检查是否为临时用户 -- 合并后需要处理手机关联的转移和过期 - -### 3. 用户ID一致性 -- 所有用户统一使用 UUID -- 转为正式用户时，只更新身份证字段，不改变 `user_id` -- 合并时，保持目标用户的 `user_id` 不变 - -### 4. 数据完整性 -- 临时用户必须建立手机关联（不能跳过） -- 正式用户的手机关联需要正确设置时间窗口 -- 合并时需要处理统计数据、标签、消费记录等所有关联数据 - -## 五、合并逻辑详细说明 - -### 合并内容 -1. **统计数据合并**：total_amount、total_count、last_consume_time -2. **手机号关联合并**：将所有手机号关联转移到目标用户 -3. **标签合并**：根据标签类型智能合并（数值型累加/取最值，布尔型取OR等） -4. **消费记录合并**：更新所有消费记录的 `user_id` - -### 合并后处理 -1. 标记源用户为已合并（`status=1`, `merged_from_user_id=目标用户ID`） -2. 更新目标用户的标签更新时间 -3. 触发标签重新计算（异步） -4. 记录合并历史日志 - -## 六、特殊场景处理 - -### 场景1：手机号被转手多次（历史记录链） - -**场景描述：** -一个手机号在不同时间段被分配给不同的用户，形成完整的历史记录链。 - -**时间线示例：** -``` -13800138000 的历史关联： -├─ 2024-01-01 → 2024-06-01: 用户A (effective_time: 2024-01-01, expire_time: 2024-06-01) -├─ 2024-06-01 → 2024-12-01: 用户B (effective_time: 2024-06-01, expire_time: 2024-12-01) -└─ 2024-12-01 → 永久: 用户C (effective_time: 2024-12-01, expire_time: null) -``` - -**处理逻辑：** -1. 每次手机号转手时，系统自动检测冲突 -2. 将旧关联的 `expire_time` 设置为新关联的 `effective_time`（保证时间连续） -3. 创建新关联，设置 `effective_time` 和 `expire_time` -4. 查询时按 `effective_time` 降序排序，取时间窗口内有效的关联 - -**查询示例：** -```php -// 查询 2024-03-01 时谁在使用该手机号 -$userId = $userPhoneService->findUserByPhone('13800138000', new DateTime('2024-03-01')); -// 返回：用户A（因为 2024-01-01 <= 2024-03-01 < 2024-06-01） - -// 查询 2024-08-01 时谁在使用该手机号 -$userId = $userPhoneService->findUserByPhone('13800138000', new DateTime('2024-08-01')); -// 返回：用户B（因为 2024-06-01 <= 2024-08-01 < 2024-12-01） -``` - -**性能考虑：** -- 如果转手非常频繁，历史记录可能很多 -- 查询时使用索引优化（`phone_hash` + `effective_time`） -- 考虑定期归档过期很久的历史记录 - -### 场景2：酒店预订等代订场景（手机号和身份证不匹配） - -**场景描述：** -用户使用自己的手机号，但提供了其他人的身份证（如代订酒店、代买商品等）。 - -**示例：** -``` -张三的手机号：13800138000（用户A，正式用户） -李四的身份证：110101199001011234（用户B，正式用户） - -消费记录： -{ - phone_number: "13800138000", - id_card: "110101199001011234", - consume_time: "2024-01-15 10:00:00" -} -``` - -**处理策略：** - -1. **检测冲突**： - - 通过身份证找到用户B - - 通过手机号在消费时间点查询，找到用户A - - 发现用户A ≠ 用户B，且两者都是正式用户 - -2. **决策逻辑**： - - **以身份证为准**：消费记录归属到身份证用户（用户B） - - **手机号关联保持不变**：不强制转移手机号到身份证用户 - - **原因**：这可能是代订场景，手机号仍属于原用户，不应自动转移 - -3. **日志记录**： - ```php - LoggerHelper::logBusiness('phone_id_card_mismatch_formal_users', [ - 'phone_number' => '13800138000', - 'phone_user_id' => 'user-a-uuid', - 'id_card_user_id' => 'user-b-uuid', - 'consume_time' => '2024-01-15 10:00:00', - 'decision' => 'use_id_card_user', - 'note' => '正式用户冲突，以身份证为准（可能是代订场景）', - ]); - ``` - -4. **结果**： - - 消费记录归属到用户B（身份证用户） - - 用户A的手机号关联保持不变 - - 记录异常日志，便于后续人工审核 - -**为什么这样处理？** -- 身份证在业务场景中通常更可信（需要实名验证） -- 代订场景很常见，不应该自动合并不同的正式用户 -- 保持手机号关联的准确性，避免误操作 -- 通过日志记录异常情况，可以后续人工审核是否需要调整 - -**对比：临时用户合并场景** -- 如果是临时用户（手机号）vs 正式用户（身份证），则自动合并 -- 因为临时用户通常是系统自动创建的，合并是合理的 - -## 七、与现有架构的对比 - -### 现有实现已满足的点： -✅ 手机号关联表支持时间窗口 -✅ 临时用户创建和手机关联 -✅ 用户合并服务（PersonMergeService） -✅ 身份证哈希和加密存储 - -### 需要改进的点： -⚠️ `resolvePersonId()` 需要支持 `$atTime` 参数 -⚠️ `ConsumptionService::createRecord()` 需要传入 `consume_time` -⚠️ 合并逻辑需要在手机号+身份证同时出现时自动触发 -⚠️ 手机关联建立时需要自动处理过期旧关联 - -## 八、总结 - -现有主用户表数据生成逻辑**基本满足要求**，但需要以下改进： - -1. **时间线处理**：所有手机号查询必须基于消费记录的实际时间点 -2. **自动合并**：当手机号和身份证号同时出现且关联到不同用户时，自动触发合并 -3. **冲突解决**：建立新关联时，自动将旧关联标记为过期 -4. **用户ID策略**：统一使用UUID，身份证仅作为字段存储 - -这些改进确保了： -- 用户身份信息的最大化有效性 -- 手机号回收后的正确匹配 -- 临时用户到正式用户的平滑转换 -- 跨数据源时间线不一致的正确处理 -g \ No newline at end of file diff --git a/Moncter/Moncter/提示词/当前架构设计/前端代码风格和设计思路.md b/Moncter/Moncter/提示词/当前架构设计/前端代码风格和设计思路.md deleted file mode 100644 index 48465d03..00000000 --- a/Moncter/Moncter/提示词/当前架构设计/前端代码风格和设计思路.md +++ /dev/null @@ -1,407 +0,0 @@ -# 前端代码风格和设计思路 - -## 一、技术栈 - -### 核心技术 -- **Vue 3** (v3.4.21) - 采用 Composition API 和 ` -``` - -### 3. 组件设计规范 - -- **组件命名**：使用 PascalCase，如 `StatusBadge.vue`、`TaskForm.vue` -- **单文件组件**：每个组件一个文件，文件名与组件名一致 -- **Props 定义**：使用 `defineProps` 定义，并指定类型 -- **Emits 定义**：使用 `defineEmits` 定义，并指定事件类型 -- **组件拆分**：保持组件职责单一，复杂组件拆分为多个子组件 - -```vue - -``` - -### 4. API 调用规范 - -- **统一封装**：所有 API 调用通过 `utils/request.ts` 封装的 `request` 实例 -- **按模块划分**：API 文件按业务模块划分，如 `tagQuery.ts`、`dataCollection.ts` -- **类型安全**：所有 API 方法都有完整的类型定义 -- **错误处理**：统一在请求拦截器中处理错误，组件中只需处理业务逻辑 - -```typescript -// ✅ API 定义示例 -export const getTagStatistics = (params?: { - tag_id?: string - start_date?: string - end_date?: string -}) => { - return request.get('/tags/statistics', params) -} - -// ✅ 组件中使用 -const loadStatistics = async () => { - try { - const response = await getTagStatistics() - statistics.value = response.data - } catch (error) { - console.error('加载统计失败:', error) - } -} -``` - -### 5. 状态管理规范 - -- **使用 Pinia**：所有状态管理使用 Pinia -- **按模块划分**：Store 按业务模块划分，如 `dataCollection.ts`、`tagTask.ts` -- **Actions 命名**：使用动词开头，如 `fetchTasks`、`createTask`、`updateTask` -- **Getters 使用**：使用 getters 计算派生状态 - -```typescript -// ✅ Store 定义示例 -export const useDataCollectionStore = defineStore('dataCollection', () => { - const tasks = ref([]) - - const fetchTasks = async (params: any) => { - const response = await getDataCollectionTaskList(params) - tasks.value = response.data.tasks - return response.data - } - - return { - tasks, - fetchTasks - } -}) -``` - -### 6. 样式规范 - -- **使用 Scoped CSS**：组件样式使用 ` -``` - -### 7. 文件命名规范 - -- **组件文件**：PascalCase，如 `TaskForm.vue`、`StatusBadge.vue` -- **工具文件**：camelCase，如 `request.ts`、`format.ts` -- **类型文件**：camelCase，如 `api.ts`、`index.ts` -- **目录名**：kebab-case 或 PascalCase，保持一致性 - -## 四、设计思路 - -### 1. 请求封装设计 - -**设计目标**：统一处理 HTTP 请求，提供类型安全、错误处理、Loading 状态等功能。 - -**实现特点**： -- 自动添加 Token 到请求头 -- 统一错误处理和提示 -- 自动显示/隐藏 Loading -- 完整的 TypeScript 类型支持 -- 支持自定义配置（showLoading、showError、timeout 等） - -**使用方式**： -```typescript -// GET 请求 -const response = await request.get('/tags/statistics', params) - -// POST 请求 -const response = await request.post('/data-collection-tasks', data) -``` - -### 2. 状态管理设计 - -**设计思路**： -- 按业务模块划分 Store，每个模块独立管理自己的状态 -- 使用 Composition API 风格的 Store 定义（`setup` 函数） -- Store 中封装 API 调用逻辑，组件直接调用 Store 方法 -- 支持响应式更新，组件自动响应状态变化 - -**优势**： -- 代码组织清晰，易于维护 -- 状态复用方便 -- 类型安全 - -### 3. 路由设计 - -**设计思路**： -- 使用嵌套路由，Layout 组件作为父路由 -- 路由按功能模块组织，路径清晰 -- 路由元信息（meta）存储页面标题等信息 -- 支持动态路由参数 - -**路由结构**： -``` -/ (Layout) - ├── /dashboard (首页) - ├── /data-collection (数据采集) - │ ├── /tasks (任务列表) - │ ├── /tasks/create (创建任务) - │ └── /tasks/:id (任务详情) - ├── /tag-tasks (标签任务) - ├── /tag-definitions (标签定义) - ├── /tag-filter (标签筛选) - ├── /tag-query (标签查询) - └── /data-sources (数据源) -``` - -### 4. 组件设计 - -**设计原则**： -- **单一职责**：每个组件只负责一个功能 -- **可复用性**：公共组件设计为可复用 -- **可维护性**：组件结构清晰，易于理解和修改 -- **类型安全**：Props 和 Emits 都有完整类型定义 - -**组件分类**： -- **布局组件**：Layout、Header、Sidebar -- **业务组件**：TaskForm、TaskList、StatusBadge -- **工具组件**：ProgressDisplay、各种工具函数 - -### 5. 类型系统设计 - -**设计思路**： -- 所有业务实体都有对应的 TypeScript 接口定义 -- API 请求和响应都有完整类型定义 -- 使用联合类型定义枚举值（如状态、类型等） -- 类型定义统一管理，便于维护 - -**类型文件组织**： -- `types/index.ts`：业务实体类型（Task、Tag、User 等） -- `types/api.ts`：API 相关类型（ApiResponse、RequestConfig 等） - -### 6. 工具函数设计 - -**设计原则**： -- 函数职责单一，易于测试 -- 完整的类型定义 -- 错误处理完善 -- 支持多种输入格式 - -**工具函数分类**： -- **格式化工具**：日期时间格式化（format.ts） -- **数据脱敏**：手机号、身份证等脱敏（mask.ts） -- **表单验证**：表单字段验证（validator.ts） -- **HTTP 请求**：请求封装（request.ts） - -## 五、最佳实践 - -### 1. 错误处理 - -```typescript -// ✅ 好的实践：在组件中处理错误 -const loadData = async () => { - try { - const response = await getData() - data.value = response.data - } catch (error: any) { - // 错误已在拦截器中提示，这里只需处理业务逻辑 - console.error('加载数据失败:', error) - } -} -``` - -### 2. Loading 状态 - -```typescript -// ✅ 使用组件级别的 loading 状态 -const loading = ref(false) - -const loadData = async () => { - loading.value = true - try { - await fetchData() - } finally { - loading.value = false - } -} -``` - -### 3. 响应式数据 - -```typescript -// ✅ 优先使用 ref -const count = ref(0) -const name = ref('') - -// ✅ 对象使用 reactive 或 ref -const form = reactive({ - name: '', - age: 0 -}) - -// 或 -const form = ref({ - name: '', - age: 0 -}) -``` - -### 4. 计算属性 - -```typescript -// ✅ 使用 computed 计算派生状态 -const filteredTasks = computed(() => { - return tasks.value.filter(task => task.status === 'running') -}) -``` - -### 5. 组件通信 - -```typescript -// ✅ Props 向下传递 -interface Props { - task: DataCollectionTask -} -const props = defineProps() - -// ✅ Emits 向上传递 -const emit = defineEmits<{ - update: [task: DataCollectionTask] - delete: [id: string] -}>() -``` - -## 六、开发规范总结 - -1. **代码风格**：使用 ESLint 自动格式化，保持代码风格一致 -2. **类型安全**：充分利用 TypeScript，避免使用 `any` -3. **组件化**：保持组件小而专一，提高可复用性 -4. **状态管理**：合理使用 Pinia，避免过度使用全局状态 -5. **API 调用**：统一使用封装的 request 方法，保持一致性 -6. **错误处理**：统一错误处理机制，提供良好的用户体验 -7. **代码注释**：关键逻辑添加注释，提高代码可读性 - diff --git a/Moncter/Moncter/提示词/当前架构设计/前端功能说明.md b/Moncter/Moncter/提示词/当前架构设计/前端功能说明.md deleted file mode 100644 index c1fde74b..00000000 --- a/Moncter/Moncter/提示词/当前架构设计/前端功能说明.md +++ /dev/null @@ -1,666 +0,0 @@ -# 前端功能说明 - -## 一、系统概述 - -TaskShow 是一个基于 Vue 3 + TypeScript + Element Plus 的前端管理系统，主要用于数据采集任务管理、标签任务管理、标签查询和用户管理等核心功能。 - -## 二、功能模块 - -### 1. 首页仪表盘 (Dashboard) - -**路由**: `/dashboard` -**组件**: `src/views/Dashboard/index.vue` - -**功能说明**： -- 显示系统核心统计数据 - - 数据采集任务总数 - - 标签任务总数 - - 运行中任务数量 - - 用户总数 -- 展示最近任务列表（数据采集任务和标签任务） -- 提供快速操作入口 - - 创建数据采集任务 - - 创建标签任务 - - 标签筛选 - - 标签查询 - -**数据来源**： -- 通过 Pinia Store 获取任务列表 -- 统计运行中任务数量 -- 展示最近更新的任务 - ---- - -### 2. 数据采集模块 (Data Collection) - -#### 2.1 数据采集任务列表 - -**路由**: `/data-collection/tasks` -**组件**: `src/views/DataCollection/TaskList.vue` - -**功能说明**： -- 展示所有数据采集任务列表 -- 支持按任务名称、状态筛选 -- 支持分页显示 -- 提供任务操作功能： - - 查看任务详情 - - 编辑任务 - - 删除任务 - - 启动/暂停/停止任务 - - 查看任务进度 - -**API 接口**： -- `getDataCollectionTaskList` - 获取任务列表 -- `deleteDataCollectionTask` - 删除任务 -- `startDataCollectionTask` - 启动任务 -- `pauseDataCollectionTask` - 暂停任务 -- `stopDataCollectionTask` - 停止任务 - -#### 2.2 创建/编辑数据采集任务 - -**路由**: -- 创建：`/data-collection/tasks/create` -- 编辑：`/data-collection/tasks/:id/edit` - -**组件**: `src/views/DataCollection/TaskForm.vue` - -**功能说明**： -- 创建或编辑数据采集任务 -- 配置任务基本信息： - - 任务名称、描述 - - 数据源选择（源数据源、目标数据源） - - 数据库和集合选择 - - 单集合/多集合模式 -- 配置字段映射： - - 源字段到目标字段的映射 - - 字段转换规则 - - 值映射配置 -- 配置 Lookup 关联： - - 多集合关联查询配置 -- 配置过滤条件： - - 数据筛选条件 -- 配置调度计划： - - 启用/禁用定时任务 - - Cron 表达式配置 -- 预览查询结果： - - 验证配置是否正确 - - 查看查询结果示例 - -**API 接口**： -- `getDataSources` - 获取数据源列表 -- `getDatabases` - 获取数据库列表 -- `getCollections` - 获取集合列表 -- `getFields` - 获取字段列表 -- `getHandlerTargetFields` - 获取目标字段列表 -- `previewQuery` - 预览查询结果 -- `createDataCollectionTask` - 创建任务 -- `updateDataCollectionTask` - 更新任务 -- `getDataCollectionTaskDetail` - 获取任务详情 - -#### 2.3 数据采集任务详情 - -**路由**: `/data-collection/tasks/:id` -**组件**: `src/views/DataCollection/TaskDetail.vue` - -**功能说明**： -- 展示任务详细信息 -- 显示任务配置信息 -- 显示任务执行进度： - - 处理数量、成功数量、错误数量 - - 进度百分比 - - 开始时间、结束时间 - - 最后同步时间 -- 显示任务统计信息 -- 显示任务执行历史 -- 提供任务操作按钮（启动、暂停、停止、编辑、删除） - -**API 接口**： -- `getDataCollectionTaskDetail` - 获取任务详情 -- `getDataCollectionTaskProgress` - 获取任务进度 -- 任务操作相关接口 - ---- - -### 3. 数据源管理模块 (Data Source) - -#### 3.1 数据源列表 - -**路由**: `/data-sources` -**组件**: `src/views/DataSource/List.vue` - -**功能说明**： -- 展示所有数据源配置列表 -- 支持按类型、状态、名称筛选 -- 支持分页显示 -- 显示数据源基本信息： - - 名称、类型、主机、端口、数据库 - - 状态（启用/禁用） -- 提供数据源操作： - - 查看详情 - - 编辑配置 - - 删除数据源 - - 测试连接 - -**API 接口**： -- `getDataSourceList` - 获取数据源列表 -- `deleteDataSource` - 删除数据源 -- `testDataSourceConnection` - 测试连接 - -#### 3.2 创建/编辑数据源 - -**路由**: -- 创建：`/data-sources/create` -- 编辑：`/data-sources/:id/edit` - -**组件**: `src/views/DataSource/Form.vue` - -**功能说明**： -- 创建或编辑数据源配置 -- 配置数据源基本信息： - - 名称、类型（MongoDB、MySQL、PostgreSQL） - - 主机、端口、数据库名 - - 用户名、密码（加密存储） - - 认证源（MongoDB） - - 其他选项配置 -- 测试数据源连接 -- 标记是否为标签引擎数据库 - -**API 接口**： -- `createDataSource` - 创建数据源 -- `updateDataSource` - 更新数据源 -- `getDataSourceDetail` - 获取数据源详情 -- `testDataSourceConnection` - 测试连接 - ---- - -### 4. 标签任务模块 (Tag Task) - -#### 4.1 标签任务列表 - -**路由**: `/tag-tasks` -**组件**: `src/views/TagTask/TaskList.vue` - -**功能说明**： -- 展示所有标签任务列表 -- 支持按任务名称、类型、状态筛选 -- 支持分页显示 -- 显示任务基本信息： - - 任务名称、类型、状态 - - 目标标签、用户范围 - - 创建时间、更新时间 -- 提供任务操作： - - 查看任务详情 - - 编辑任务 - - 删除任务 - - 启动/暂停/停止任务 - -**API 接口**： -- `getTagTaskList` - 获取任务列表 -- `deleteTagTask` - 删除任务 -- `startTagTask` - 启动任务 -- `pauseTagTask` - 暂停任务 -- `stopTagTask` - 停止任务 - -#### 4.2 创建/编辑标签任务 - -**路由**: -- 创建：`/tag-tasks/create` -- 编辑：`/tag-tasks/:id/edit` - -**组件**: `src/views/TagTask/TaskForm.vue` - -**功能说明**： -- 创建或编辑标签任务 -- 配置任务基本信息： - - 任务名称、描述 - - 任务类型（全量、增量、指定） -- 配置目标标签： - - 选择要计算的标签列表 -- 配置用户范围： - - 全部用户 - - 指定用户列表 - - 按条件筛选用户 -- 配置调度计划： - - 启用/禁用定时任务 - - Cron 表达式配置 -- 配置任务参数： - - 并发数、批次大小 - - 错误处理策略 - -**API 接口**： -- `getTagDefinitionList` - 获取标签定义列表（用于选择目标标签） -- `createTagTask` - 创建任务 -- `updateTagTask` - 更新任务 -- `getTagTaskDetail` - 获取任务详情 - -#### 4.3 标签任务详情 - -**路由**: `/tag-tasks/:id` -**组件**: `src/views/TagTask/TaskDetail.vue` - -**功能说明**： -- 展示任务详细信息 -- 显示任务配置信息 -- 显示任务执行进度： - - 总用户数、已处理用户数 - - 成功数量、错误数量 - - 进度百分比 -- 显示任务统计信息： - - 总执行次数 - - 成功/失败次数 - - 最后执行时间 -- 显示任务执行记录列表 -- 提供任务操作按钮（启动、暂停、停止、编辑、删除） - -**API 接口**： -- `getTagTaskDetail` - 获取任务详情 -- `getTagTaskExecutions` - 获取执行记录 - ---- - -### 5. 标签定义模块 (Tag Definition) - -#### 5.1 标签定义列表 - -**路由**: `/tag-definitions` -**组件**: `src/views/TagDefinition/List.vue` - -**功能说明**： -- 展示所有标签定义列表 -- 支持按名称、分类、状态筛选 -- 支持分页显示 -- 显示标签基本信息： - - 标签代码、名称、分类 - - 规则类型、更新频率 - - 状态（启用/禁用） - - 优先级、版本 -- 提供标签操作： - - 查看详情 - - 编辑标签 - - 删除标签 - - 批量初始化 - -**API 接口**： -- `getTagDefinitionList` - 获取标签定义列表 -- `deleteTagDefinition` - 删除标签定义 -- `batchInitTagDefinitions` - 批量初始化 - -#### 5.2 创建/编辑标签定义 - -**路由**: -- 创建：`/tag-definitions/create` -- 编辑：`/tag-definitions/:id/edit` - -**组件**: `src/views/TagDefinition/Form.vue` - -**功能说明**： -- 创建或编辑标签定义 -- 配置标签基本信息： - - 标签代码、名称、分类 - - 描述 -- 配置规则类型： - - 简单规则 - - 管道规则 - - 自定义规则 -- 配置规则条件： - - 字段、操作符、值 - - 多个条件的逻辑关系 -- 配置标签值： - - 标签值的计算方式 -- 配置更新频率： - - 实时、每日、每周、每月 -- 配置优先级和版本 - -**API 接口**： -- `createTagDefinition` - 创建标签定义 -- `updateTagDefinition` - 更新标签定义 -- `getTagDefinitionDetail` - 获取标签定义详情 - -#### 5.3 标签定义详情 - -**路由**: `/tag-definitions/:id` -**组件**: `src/views/TagDefinition/Detail.vue` - -**功能说明**： -- 展示标签定义详细信息 -- 显示标签配置信息 -- 显示规则配置详情 -- 显示标签统计信息（如果有） -- 提供编辑和删除操作 - -**API 接口**： -- `getTagDefinitionDetail` - 获取标签定义详情 - ---- - -### 6. 标签筛选模块 (Tag Filter) - -**路由**: `/tag-filter` -**组件**: `src/views/TagFilter/index.vue` - -**功能说明**： -- 根据标签条件筛选用户 -- 支持多条件组合： - - 添加多个标签条件 - - 设置条件逻辑关系（AND/OR） -- 支持多种操作符： - - 等于、不等于 - - 大于、大于等于、小于、小于等于 - - 包含、不包含 - - 在列表中、不在列表中 -- 显示筛选结果： - - 用户列表 - - 用户基本信息（姓名、手机号等） - - 用户标签信息 -- 支持分页显示 -- 支持导出筛选结果 -- 支持保存为人群快照 - -**API 接口**： -- `filterUsersByTags` - 根据标签筛选用户 -- `createTagCohort` - 创建人群快照（保存筛选结果） - ---- - -### 7. 标签查询模块 (Tag Query) - -#### 7.1 用户标签查询 - -**路由**: `/tag-query/user` -**组件**: `src/views/TagQuery/User.vue` - -**功能说明**： -- 通过用户ID或手机号查询用户标签 -- 显示用户基本信息： - - 用户ID、姓名、手机号 - - 消费总额、消费次数 - - 最后消费时间 -- 显示用户所有标签： - - 标签名称、代码、分类 - - 标签值、值类型 - - 置信度 - - 生效时间、过期时间 - - 更新时间 -- 支持重新计算用户标签 -- 支持删除用户标签 -- 支持查看标签历史记录 - -**API 接口**： -- `getUserTags` - 获取用户标签 -- `recalculateUserTags` - 重新计算用户标签 -- `deleteUserTag` - 删除用户标签 -- `getTagHistory` - 获取标签历史（按用户筛选） - -#### 7.2 标签统计 - -**路由**: `/tag-query/statistics` -**组件**: `src/views/TagQuery/Statistics.vue` - -**功能说明**： -- 展示标签统计信息 -- 标签覆盖度统计： - - 总用户数 - - 已打标签用户数 - - 覆盖率（百分比） -- 标签值分布： - - 各标签值的出现次数 - - 分布图表展示 -- 标签趋势数据： - - 按日期统计标签变更次数 - - 趋势图表展示 -- 支持按标签筛选 -- 支持按时间范围筛选 - -**API 接口**： -- `getTagStatistics` - 获取标签统计信息 - -#### 7.3 标签历史 - -**路由**: `/tag-query/history` -**组件**: `src/views/TagQuery/History.vue` - -**功能说明**： -- 展示标签变更历史记录 -- 支持多维度筛选： - - 按用户筛选 - - 按标签筛选 - - 按时间范围筛选 -- 显示历史记录详情： - - 用户ID - - 标签ID、标签名称 - - 旧值、新值 - - 变更原因 - - 变更时间 - - 操作人 -- 支持分页显示 -- 支持导出历史记录 - -**API 接口**： -- `getTagHistory` - 获取标签历史记录 - ---- - -## 三、公共组件 - -### 1. Layout 布局组件 - -**组件**: `src/components/Layout/index.vue` - -**功能说明**： -- 提供系统整体布局结构 -- 侧边栏导航菜单： - - 首页 - - 数据采集（任务列表、数据源配置） - - 标签任务（任务列表、标签定义） - - 标签筛选 - - 标签查询（用户标签、标签统计、标签历史） -- 顶部导航栏： - - 侧边栏折叠/展开按钮 - - 用户信息下拉菜单 -- 主内容区域： - - 路由视图容器 - -### 2. StatusBadge 状态徽章 - -**组件**: `src/components/StatusBadge/index.vue` - -**功能说明**： -- 显示任务状态的可视化组件 -- 支持多种状态： - - pending（待处理）- 灰色 - - running（运行中）- 蓝色 - - paused（已暂停）- 黄色 - - stopped（已停止）- 橙色 - - completed（已完成）- 绿色 - - error（错误）- 红色 - -### 3. ProgressDisplay 进度显示 - -**组件**: `src/components/ProgressDisplay/index.vue` - -**功能说明**： -- 显示任务执行进度的可视化组件 -- 显示进度条和百分比 -- 显示处理数量、成功数量、错误数量 - ---- - -## 四、工具函数 - -### 1. 格式化工具 (`utils/format.ts`) - -- `formatDateTime` - 格式化日期时间（默认格式：YYYY-MM-DD HH:mm:ss） -- `formatDate` - 格式化日期（格式：YYYY-MM-DD） -- `formatTime` - 格式化时间（格式：HH:mm:ss） -- `formatRelativeTime` - 相对时间格式化（如：1分钟前、2小时前） - -### 2. 数据脱敏工具 (`utils/mask.ts`) - -- `maskPhone` - 脱敏手机号（如：138****8000） -- `maskIdCard` - 脱敏身份证号（如：110101********1234） -- `maskBankCard` - 脱敏银行卡号 -- `maskName` - 脱敏姓名（如：张*、李**） -- `maskEmail` - 脱敏邮箱 - -### 3. 表单验证工具 (`utils/validator.ts`) - -- 提供常用的表单验证规则 -- 手机号、邮箱、身份证等格式验证 - ---- - -## 五、路由配置 - -### 路由结构 - -``` -/ (Layout) - ├── /dashboard (首页) - │ - ├── /data-collection (数据采集) - │ ├── /tasks (任务列表) - │ ├── /tasks/create (创建任务) - │ ├── /tasks/:id (任务详情) - │ └── /tasks/:id/edit (编辑任务) - │ - ├── /data-sources (数据源) - │ ├── / (数据源列表) - │ ├── /create (创建数据源) - │ └── /:id/edit (编辑数据源) - │ - ├── /tag-tasks (标签任务) - │ ├── / (任务列表) - │ ├── /create (创建任务) - │ ├── /:id (任务详情) - │ └── /:id/edit (编辑任务) - │ - ├── /tag-definitions (标签定义) - │ ├── / (标签列表) - │ ├── /create (创建标签) - │ ├── /:id (标签详情) - │ └── /:id/edit (编辑标签) - │ - ├── /tag-filter (标签筛选) - │ - └── /tag-query (标签查询) - ├── /user (用户标签查询) - ├── /statistics (标签统计) - └── /history (标签历史) -``` - ---- - -## 六、状态管理 (Pinia Store) - -### 1. User Store (`store/user.ts`) - -- 管理用户登录状态 -- 管理 Token -- 用户信息 - -### 2. DataCollection Store (`store/dataCollection.ts`) - -- 管理数据采集任务列表 -- 提供任务操作方法（获取、创建、更新、删除等） - -### 3. TagTask Store (`store/tagTask.ts`) - -- 管理标签任务列表 -- 提供任务操作方法 - -### 4. TagDefinition Store (`store/tagDefinition.ts`) - -- 管理标签定义列表 -- 提供标签定义操作方法 - -### 5. DataSource Store (`store/dataSource.ts`) - -- 管理数据源列表 -- 提供数据源操作方法 - ---- - -## 七、功能特性总结 - -### 1. 核心功能 - -- ✅ 数据采集任务管理（创建、编辑、删除、启动、暂停、停止） -- ✅ 数据源管理（配置、测试连接） -- ✅ 标签任务管理（创建、编辑、删除、执行） -- ✅ 标签定义管理（创建、编辑、删除、批量初始化） -- ✅ 标签查询（用户标签、标签统计、标签历史） -- ✅ 标签筛选（多条件组合筛选用户） -- ✅ 人群快照（保存筛选结果、导出） - -### 2. 用户体验 - -- ✅ 统一的 UI 设计（Element Plus） -- ✅ 响应式布局 -- ✅ Loading 状态提示 -- ✅ 错误提示和处理 -- ✅ 数据脱敏显示 -- ✅ 日期时间格式化显示 - -### 3. 技术特性 - -- ✅ TypeScript 类型安全 -- ✅ 组件化开发 -- ✅ 状态管理（Pinia） -- ✅ 路由管理（Vue Router） -- ✅ API 统一封装 -- ✅ 工具函数复用 - ---- - -## 八、使用说明 - -### 1. 开发环境启动 - -```bash -cd TaskShow -npm install # 或 yarn install 或 pnpm install -npm run dev # 启动开发服务器 -``` - -### 2. 构建生产版本 - -```bash -npm run build -``` - -### 3. 预览构建结果 - -```bash -npm run preview -``` - -### 4. 代码检查 - -```bash -npm run lint -``` - ---- - -## 九、注意事项 - -1. **API 地址配置**： - - 开发环境通过 Vite 代理配置 - - 生产环境通过环境变量 `VITE_API_BASE_URL` 配置 - -2. **Token 管理**： - - Token 存储在 Pinia Store 中 - - 请求时自动添加到请求头 - -3. **错误处理**： - - HTTP 错误和业务错误在请求拦截器中统一处理 - - 组件中只需处理业务逻辑 - -4. **数据脱敏**： - - 敏感信息（手机号、身份证等）使用脱敏工具函数处理 - - 确保数据安全显示 - -5. **类型安全**： - - 所有 API 接口都有完整的 TypeScript 类型定义 - - 建议启用严格模式以获得更好的类型检查 - diff --git a/Moncter/Moncter/提示词/当前架构设计/前端已对接API说明.md b/Moncter/Moncter/提示词/当前架构设计/前端已对接API说明.md deleted file mode 100644 index 5d9adb5f..00000000 --- a/Moncter/Moncter/提示词/当前架构设计/前端已对接API说明.md +++ /dev/null @@ -1,693 +0,0 @@ -# 前端已对接API说明 - -## 一、API 架构设计 - -### 1. 请求封装 - -所有 API 调用通过 `src/utils/request.ts` 封装的 `request` 实例，提供以下特性： - -- ✅ 自动添加 Token 到请求头（从 Pinia Store 获取） -- ✅ 统一错误处理和提示（HTTP 错误、业务错误） -- ✅ 自动显示/隐藏 Loading（可配置） -- ✅ 完整的 TypeScript 类型支持 -- ✅ 支持自定义配置（showLoading、showError、timeout 等） - -### 2. API 文件组织 - -API 文件按业务模块划分，位于 `src/api/` 目录： - -- `dataCollection.ts` - 数据采集任务相关 API -- `dataSource.ts` - 数据源管理相关 API -- `tagTask.ts` - 标签任务相关 API -- `tagDefinition.ts` - 标签定义相关 API -- `tagQuery.ts` - 标签查询相关 API -- `tagCohort.ts` - 人群快照相关 API -- `user.ts` - 用户相关 API - -### 3. 类型定义 - -所有 API 的请求参数和响应数据都有完整的 TypeScript 类型定义，位于： -- `src/types/index.ts` - 业务实体类型 -- `src/types/api.ts` - API 响应类型 - -## 二、已对接 API 列表 - -### 1. 数据采集任务 API (`dataCollection.ts`) - -#### 1.1 获取数据采集任务列表 -```typescript -getDataCollectionTaskList(params: { - name?: string - status?: string - page?: number - page_size?: number -}): Promise> -``` -- **路径**: `GET /data-collection-tasks` -- **功能**: 获取数据采集任务列表，支持按名称、状态筛选和分页 - -#### 1.2 获取数据采集任务详情 -```typescript -getDataCollectionTaskDetail(taskId: string): Promise> -``` -- **路径**: `GET /data-collection-tasks/:id` -- **功能**: 获取指定任务的详细信息 - -#### 1.3 创建数据采集任务 -```typescript -createDataCollectionTask(data: Partial): Promise> -``` -- **路径**: `POST /data-collection-tasks` -- **功能**: 创建新的数据采集任务 - -#### 1.4 更新数据采集任务 -```typescript -updateDataCollectionTask(taskId: string, data: Partial): Promise> -``` -- **路径**: `PUT /data-collection-tasks/:id` -- **功能**: 更新指定任务的信息 - -#### 1.5 删除数据采集任务 -```typescript -deleteDataCollectionTask(taskId: string): Promise -``` -- **路径**: `DELETE /data-collection-tasks/:id` -- **功能**: 删除指定任务 - -#### 1.6 启动数据采集任务 -```typescript -startDataCollectionTask(taskId: string): Promise -``` -- **路径**: `POST /data-collection-tasks/:id/start` -- **功能**: 启动指定任务 - -#### 1.7 暂停数据采集任务 -```typescript -pauseDataCollectionTask(taskId: string): Promise -``` -- **路径**: `POST /data-collection-tasks/:id/pause` -- **功能**: 暂停指定任务 - -#### 1.8 停止数据采集任务 -```typescript -stopDataCollectionTask(taskId: string): Promise -``` -- **路径**: `POST /data-collection-tasks/:id/stop` -- **功能**: 停止指定任务 - -#### 1.9 获取任务进度 -```typescript -getDataCollectionTaskProgress(taskId: string): Promise -``` -- **路径**: `GET /data-collection-tasks/:id/progress` -- **功能**: 获取任务执行进度 - -#### 1.10 获取数据源列表 -```typescript -getDataSources(): Promise> -``` -- **路径**: `GET /data-collection-tasks/data-sources` -- **功能**: 获取所有可用的数据源列表 - -#### 1.11 获取数据库列表 -```typescript -getDatabases(dataSourceId: string): Promise>> -``` -- **路径**: `GET /data-collection-tasks/data-sources/:id/databases` -- **功能**: 获取指定数据源的数据库列表 - -#### 1.12 获取集合列表 -```typescript -getCollections(dataSourceId: string, database: string | { name: string; id: string }): Promise>> -``` -- **路径**: `GET /data-collection-tasks/data-sources/:id/databases/:db/collections` -- **功能**: 获取指定数据库的集合列表 - -#### 1.13 获取字段列表 -```typescript -getFields(dataSourceId: string, database: string | { name: string; id: string }, collection: string | { name: string; id: string }): Promise>> -``` -- **路径**: `GET /data-collection-tasks/data-sources/:id/databases/:db/collections/:coll/fields` -- **功能**: 获取指定集合的字段列表 - -#### 1.14 获取Handler目标字段列表 -```typescript -getHandlerTargetFields(handlerType: string): Promise>> -``` -- **路径**: `GET /data-collection-tasks/handlers/:type/target-fields` -- **功能**: 获取指定 Handler 类型的目标字段列表 - -#### 1.15 预览查询结果 -```typescript -previewQuery(data: { - data_source_id: string - database: string - collection: string - lookups?: any[] - filter_conditions?: any[] - limit?: number -}): Promise - data: Array - count: number -}>> -``` -- **路径**: `POST /data-collection-tasks/preview-query` -- **功能**: 预览查询结果，用于验证配置 - ---- - -### 2. 数据源管理 API (`dataSource.ts`) - -#### 2.1 获取数据源列表 -```typescript -getDataSourceList(params?: { - type?: string - status?: number - name?: string - page?: number - page_size?: number -}): Promise> -``` -- **路径**: `GET /data-sources` -- **功能**: 获取数据源列表，支持按类型、状态、名称筛选和分页 - -#### 2.2 获取数据源详情 -```typescript -getDataSourceDetail(dataSourceId: string): Promise> -``` -- **路径**: `GET /data-sources/:id` -- **功能**: 获取指定数据源的详细信息 - -#### 2.3 创建数据源 -```typescript -createDataSource(data: Partial): Promise> -``` -- **路径**: `POST /data-sources` -- **功能**: 创建新的数据源配置 - -#### 2.4 更新数据源 -```typescript -updateDataSource(dataSourceId: string, data: Partial): Promise> -``` -- **路径**: `PUT /data-sources/:id` -- **功能**: 更新指定数据源的配置 - -#### 2.5 删除数据源 -```typescript -deleteDataSource(dataSourceId: string): Promise -``` -- **路径**: `DELETE /data-sources/:id` -- **功能**: 删除指定数据源 - -#### 2.6 测试数据源连接 -```typescript -testDataSourceConnection(data: { - type: string - host: string - port: number - database: string - username?: string - password?: string - auth_source?: string - options?: Record -}): Promise> -``` -- **路径**: `POST /data-sources/test-connection` -- **功能**: 测试数据源连接是否正常 - ---- - -### 3. 标签任务 API (`tagTask.ts`) - -#### 3.1 获取标签任务列表 -```typescript -getTagTaskList(params: { - name?: string - task_type?: string - status?: string - page?: number - page_size?: number -}): Promise> -``` -- **路径**: `GET /tag-tasks` -- **功能**: 获取标签任务列表，支持按名称、类型、状态筛选和分页 - -#### 3.2 获取标签任务详情 -```typescript -getTagTaskDetail(taskId: string): Promise> -``` -- **路径**: `GET /tag-tasks/:id` -- **功能**: 获取指定标签任务的详细信息 - -#### 3.3 创建标签任务 -```typescript -createTagTask(data: Partial): Promise> -``` -- **路径**: `POST /tag-tasks` -- **功能**: 创建新的标签任务 - -#### 3.4 更新标签任务 -```typescript -updateTagTask(taskId: string, data: Partial): Promise> -``` -- **路径**: `PUT /tag-tasks/:id` -- **功能**: 更新指定标签任务的信息 - -#### 3.5 删除标签任务 -```typescript -deleteTagTask(taskId: string): Promise -``` -- **路径**: `DELETE /tag-tasks/:id` -- **功能**: 删除指定标签任务 - -#### 3.6 启动标签任务 -```typescript -startTagTask(taskId: string): Promise -``` -- **路径**: `POST /tag-tasks/:id/start` -- **功能**: 启动指定标签任务 - -#### 3.7 暂停标签任务 -```typescript -pauseTagTask(taskId: string): Promise -``` -- **路径**: `POST /tag-tasks/:id/pause` -- **功能**: 暂停指定标签任务 - -#### 3.8 停止标签任务 -```typescript -stopTagTask(taskId: string): Promise -``` -- **路径**: `POST /tag-tasks/:id/stop` -- **功能**: 停止指定标签任务 - -#### 3.9 获取任务执行记录 -```typescript -getTagTaskExecutions(taskId: string, params?: { - page?: number - page_size?: number -}): Promise> -``` -- **路径**: `GET /tag-tasks/:id/executions` -- **功能**: 获取指定任务的执行记录列表 - ---- - -### 4. 标签定义 API (`tagDefinition.ts`) - -#### 4.1 获取标签定义列表 -```typescript -getTagDefinitionList(params?: { - name?: string - category?: string - status?: number - page?: number - page_size?: number -}): Promise> -``` -- **路径**: `GET /tag-definitions` -- **功能**: 获取标签定义列表，支持按名称、分类、状态筛选和分页 - -#### 4.2 获取标签定义详情 -```typescript -getTagDefinitionDetail(tagId: string): Promise> -``` -- **路径**: `GET /tag-definitions/:id` -- **功能**: 获取指定标签定义的详细信息 - -#### 4.3 创建标签定义 -```typescript -createTagDefinition(data: Partial): Promise> -``` -- **路径**: `POST /tag-definitions` -- **功能**: 创建新的标签定义 - -#### 4.4 更新标签定义 -```typescript -updateTagDefinition(tagId: string, data: Partial): Promise> -``` -- **路径**: `PUT /tag-definitions/:id` -- **功能**: 更新指定标签定义的信息 - -#### 4.5 删除标签定义 -```typescript -deleteTagDefinition(tagId: string): Promise -``` -- **路径**: `DELETE /tag-definitions/:id` -- **功能**: 删除指定标签定义 - -#### 4.6 批量初始化标签定义 -```typescript -batchInitTagDefinitions(data: { - definitions: Partial[] -}): Promise -``` -- **路径**: `POST /tag-definitions/batch` -- **功能**: 批量创建标签定义 - ---- - -### 5. 标签查询 API (`tagQuery.ts`) - -#### 5.1 获取用户标签 -```typescript -getUserTags(userIdOrPhone: string): Promise> -``` -- **路径**: `GET /users/:userIdOrPhone/tags` -- **功能**: 获取指定用户（通过用户ID或手机号）的所有标签 - -#### 5.2 重新计算用户标签 -```typescript -recalculateUserTags(userId: string): Promise - count: number -}>> -``` -- **路径**: `PUT /users/:userId/tags` -- **功能**: 重新计算指定用户的所有标签 - -#### 5.3 根据标签筛选用户 -```typescript -filterUsersByTags(params: { - tag_conditions: TagCondition[] - logic: 'AND' | 'OR' - page?: number - page_size?: number - include_user_info?: boolean -}): Promise> -``` -- **路径**: `POST /tags/filter` -- **功能**: 根据标签条件筛选用户列表 - -#### 5.4 获取标签统计信息 -```typescript -getTagStatistics(params?: { - tag_id?: string - start_date?: string - end_date?: string -}): Promise> -``` -- **路径**: `GET /tags/statistics` -- **功能**: 获取标签统计信息（值分布、趋势数据、覆盖度统计） - -#### 5.5 获取标签历史记录 -```typescript -getTagHistory(params?: { - user_id?: string - tag_id?: string - start_date?: string - end_date?: string - page?: number - page_size?: number -}): Promise> -``` -- **路径**: `GET /tags/history` -- **功能**: 获取标签变更历史记录，支持按用户、标签、时间范围筛选和分页 - ---- - -### 6. 人群快照 API (`tagCohort.ts`) - -#### 6.1 获取人群快照列表 -```typescript -getTagCohortList(params?: { - page?: number - page_size?: number -}): Promise> -``` -- **路径**: `GET /tag-cohorts` -- **功能**: 获取人群快照列表，支持分页 - -#### 6.2 获取人群快照详情 -```typescript -getTagCohortDetail(cohortId: string): Promise> -``` -- **路径**: `GET /tag-cohorts/:id` -- **功能**: 获取指定人群快照的详细信息 - -#### 6.3 创建人群快照 -```typescript -createTagCohort(data: { - name: string - description?: string - conditions: TagCondition[] - logic?: 'AND' | 'OR' - user_ids?: string[] - created_by?: string -}): Promise> -``` -- **路径**: `POST /tag-cohorts` -- **功能**: 创建新的人群快照（支持条件筛选或直接指定用户列表） - -#### 6.4 删除人群快照 -```typescript -deleteTagCohort(cohortId: string): Promise -``` -- **路径**: `DELETE /tag-cohorts/:id` -- **功能**: 删除指定人群快照 - -#### 6.5 导出人群快照 -```typescript -exportTagCohort(cohortId: string): Promise -``` -- **路径**: `POST /tag-cohorts/:id/export` -- **功能**: 导出人群快照为 CSV 文件（返回 Blob 对象） - ---- - -### 7. 用户 API (`user.ts`) - -#### 7.1 搜索用户 -```typescript -searchUsers(params: { - id_card?: string - phone?: string - name?: string - page?: number - page_size?: number -}): Promise> -``` -- **路径**: `POST /users/search` -- **功能**: 根据身份证、手机号、姓名搜索用户 - -#### 7.2 解密身份证 -```typescript -decryptIdCard(userId: string): Promise> -``` -- **路径**: `GET /users/:userId/decrypt-id-card` -- **功能**: 解密指定用户的身份证号（需要权限） - -#### 7.3 删除用户标签 -```typescript -deleteUserTag(userId: string, tagId: string): Promise -``` -- **路径**: `DELETE /users/:userId/tags/:tagId` -- **功能**: 删除指定用户的指定标签 - ---- - -## 三、API 使用示例 - -### 1. 基本使用 - -```typescript -import { getTagStatistics } from '@/api/tagQuery' - -// 获取所有标签的覆盖度统计 -const stats = await getTagStatistics() - -// 获取指定标签的值分布和趋势 -const tagStats = await getTagStatistics({ - tag_id: 'tag1', - start_date: '2025-01-01', - end_date: '2025-01-31' -}) -``` - -### 2. 错误处理 - -```typescript -try { - const response = await getTagStatistics() - // 处理成功响应 - console.log(response.data) -} catch (error: any) { - // 错误已在拦截器中提示，这里只需处理业务逻辑 - console.error('加载统计失败:', error) -} -``` - -### 3. 在组件中使用 - -```vue - -``` - -### 4. 在 Store 中使用 - -```typescript -import { defineStore } from 'pinia' -import { getTagStatistics } from '@/api/tagQuery' -import type { TagStatistics } from '@/types' - -export const useTagQueryStore = defineStore('tagQuery', () => { - const statistics = ref(null) - - const fetchStatistics = async (params?: any) => { - const response = await getTagStatistics(params) - statistics.value = response.data - return response.data - } - - return { - statistics, - fetchStatistics - } -}) -``` - -## 四、API 配置说明 - -### 1. 后端地址配置 - -- **开发环境**：通过 Vite 代理配置，请求 `/api` 自动转发到 `http://127.0.0.1:8787` -- **生产环境**：通过环境变量 `VITE_API_BASE_URL` 配置实际 API 地址 - -### 2. 请求拦截器 - -- 自动从 Pinia Store 获取 Token 并添加到请求头：`Authorization: Bearer ${token}` -- 自动显示 Loading（可配置 `showLoading: false` 关闭） -- 统一处理 HTTP 错误和业务错误 - -### 3. 响应拦截器 - -- 自动关闭 Loading -- 根据业务状态码（code === 200 或 code === 0）判断成功 -- 统一错误提示（可配置 `showError: false` 关闭） - -## 五、API 统计 - -| 模块 | API 数量 | 状态 | -|------|---------|------| -| 数据采集任务 | 15 | ✅ 已对接 | -| 数据源管理 | 6 | ✅ 已对接 | -| 标签任务 | 9 | ✅ 已对接 | -| 标签定义 | 6 | ✅ 已对接 | -| 标签查询 | 5 | ✅ 已对接 | -| 人群快照 | 5 | ✅ 已对接 | -| 用户 | 3 | ✅ 已对接 | -| **总计** | **49** | **✅ 全部完成** | - -## 六、注意事项 - -1. **分页参数**： - - `page` 从 1 开始 - - `page_size` 默认 20，最大 100 - -2. **日期格式**： - - `start_date` 和 `end_date` 使用 `YYYY-MM-DD` 格式 - - 例如：`'2025-01-01'` - -3. **人群快照创建**： - - 如果提供 `user_ids`，直接使用提供的用户列表 - - 如果不提供 `user_ids`，会根据 `conditions` 自动筛选用户 - - 最多支持 10000 个用户 - -4. **导出功能**： - - `exportTagCohort` 返回的是 Blob 对象 - - 需要使用 `window.URL.createObjectURL` 创建下载链接 - -5. **类型安全**： - - 所有接口都有完整的 TypeScript 类型定义 - - 建议启用严格模式以获得更好的类型检查 - -6. **错误处理**： - - HTTP 错误（401、403、404、500 等）会在拦截器中自动处理 - - 业务错误（code !== 200 && code !== 0）会在拦截器中提示 - - 组件中只需处理业务逻辑，无需重复处理错误提示 - diff --git a/Moncter/Moncter/提示词/当前架构设计/数据采集业务逻辑.md b/Moncter/Moncter/提示词/当前架构设计/数据采集业务逻辑.md deleted file mode 100644 index 1a03c759..00000000 --- a/Moncter/Moncter/提示词/当前架构设计/数据采集业务逻辑.md +++ /dev/null @@ -1,1089 +0,0 @@ -# 数据采集业务逻辑 - -## 一、系统概述 - -数据采集系统是项目的核心模块之一，负责从多个数据源采集数据并写入目标数据库。系统支持批量采集和实时监听两种模式，通过配置化的方式灵活管理采集任务。 - -### 1.1 核心组件 - -- **DataSyncScheduler** - 数据采集任务调度器（Workerman进程） -- **DataCollectionTaskService** - 数据采集任务管理服务 -- **DataSourceService** - 数据源管理服务 -- **DataSourceAdapterFactory** - 数据源适配器工厂 -- **Handler** - 业务处理类（ConsumptionCollectionHandler、GenericCollectionHandler、DatabaseSyncHandler） -- **DataSyncWorker** - 数据同步Worker（消费RabbitMQ消息队列） - -### 1.2 数据存储 - -- **data_collection_tasks** - 数据采集任务集合 -- **data_sources** - 数据源配置集合 -- **consumption_records** - 消费记录集合（按时间分表） -- **user_profile** - 用户画像集合 -- **user_phone_relations** - 手机号关联集合 - ---- - -## 二、任务配置体系 - -### 2.1 配置方式 - -系统支持两种任务配置方式： - -#### 2.1.1 配置文件方式 - -配置文件：`config/data_collection_tasks.php` - -**适用场景**： -- 系统级任务（如数据库实时同步） -- 需要版本控制的配置 -- 固定不变的采集任务 - -**配置示例**： -```php -'tasks' => [ - 'database_sync' => [ - 'name' => '数据库实时同步', - 'enabled' => false, - 'source_data_source' => 'kr_mongodb', - 'target_data_source' => 'sync_mongodb', - 'handler_class' => DatabaseSyncHandler::class, - 'schedule' => [ - 'enabled' => false, // 持续运行，不需要定时调度 - ], - 'sharding' => [ - 'strategy' => 'by_database', // 按数据库分片 - ], - ], -] -``` - -#### 2.1.2 数据库方式 - -集合：`data_collection_tasks` - -**适用场景**： -- 通过前端界面动态创建的任务 -- 需要频繁修改的配置 -- 用户自定义的采集任务 - -**任务结构**： -```javascript -{ - task_id: String, // 任务ID（UUID） - name: String, // 任务名称 - description: String, // 任务描述 - data_source_id: String, // 源数据源ID - database: String, // 源数据库名 - collection: String, // 源集合名（单集合模式） - collections: Array, // 源集合列表（多集合模式） - target_type: String, // 目标类型（consumption_record/generic） - target_data_source_id: String, // 目标数据源ID - target_database: String, // 目标数据库名 - target_collection: String, // 目标集合名 - mode: String, // 采集模式（batch/realtime） - field_mappings: Array, // 字段映射配置 - collection_field_mappings: Object, // 多集合字段映射（key为集合名） - lookups: Array, // 连表查询配置 - collection_lookups: Object, // 多集合连表查询配置 - filter_conditions: Array, // 过滤条件 - schedule: { // 调度计划 - enabled: Boolean, // 是否启用定时任务 - cron: String // Cron 表达式 - }, - status: String, // 任务状态（pending/running/paused/stopped/error） - progress: { // 任务进度 - status: String, // 进度状态（idle/running/paused/completed/error） - processed_count: Number, // 已处理数量 - success_count: Number, // 成功数量 - error_count: Number, // 错误数量 - total_count: Number, // 总数量 - percentage: Number, // 完成百分比 - start_time: Date, // 开始时间 - end_time: Date, // 结束时间 - last_sync_time: Date // 最后同步时间 - }, - statistics: { // 任务统计 - total_processed: Number, // 总处理数量 - total_success: Number, // 总成功数量 - total_error: Number, // 总错误数量 - last_run_time: Date // 最后执行时间 - }, - created_by: String, - created_at: Date, - updated_at: Date -} -``` - -### 2.2 数据源配置 - -数据源配置存储在 `data_sources` 集合中，支持从数据库动态加载。 - -**数据源结构**： -```javascript -{ - data_source_id: String, // 数据源ID - name: String, // 数据源名称 - type: String, // 数据源类型（mongodb/mysql/postgresql） - host: String, // 主机地址 - port: Number, // 端口号 - database: String, // 数据库名 - username: String, // 用户名 - password: String, // 密码（加密存储） - auth_source: String, // 认证源（MongoDB） - is_tag_engine: Boolean, // 是否为标签引擎数据库 - status: Number, // 状态（0:禁用, 1:启用） - options: Object, // 连接选项 - created_at: Date, - updated_at: Date -} -``` - ---- - -## 三、任务执行流程 - -### 3.1 任务创建流程 - -``` -用户在前端创建任务 - ↓ -填写任务配置： - - 数据源选择 - - 数据库/集合选择 - - 字段映射配置 - - 过滤条件 - - 调度计划 - ↓ -API: POST /api/data-collection-tasks - ↓ -DataCollectionTaskService->createTask() - ├─→ 根据 target_type 自动处理目标数据源配置 - │ ├─→ consumption_record: 自动使用标签数据库 - │ └─→ generic: 使用用户指定的目标数据源 - ├─→ 清理字段映射（移除无效项） - ├─→ 构建任务文档 - └─→ 保存到 data_collection_tasks 集合 - ↓ -如果任务状态为 running，设置 Redis 启动标志 -``` - -### 3.2 任务启动流程 - -``` -用户点击"启动"按钮 - ↓ -API: POST /api/data-collection-tasks/{taskId}/start - ↓ -DataCollectionTaskService->startTask() - ├─→ 检查任务状态（不能是 running） - ├─→ 更新任务状态为 running - ├─→ 更新进度状态为 running - └─→ 设置 Redis 标志：data_collection_task:{taskId}:start - ↓ -DataSyncScheduler 进程检测到 Redis 标志 - ├─→ 从数据库加载任务配置 - ├─→ 转换为内部配置格式 - └─→ 根据任务模式执行： - ├─→ batch 模式：根据 schedule 配置定时执行或立即执行 - └─→ realtime 模式：立即启动，持续运行（Change Stream监听） -``` - -### 3.3 任务执行流程 - -``` -DataSyncScheduler->executeTask() - ├─→ 获取分布式锁（防止重复执行） - ├─→ 查询数据源配置（从缓存或数据库） - ├─→ 创建数据源适配器（DataSourceAdapterFactory） - ├─→ 实例化 Handler（根据 target_type 选择） - └─→ 调用 Handler->collect(adapter, taskConfig) - ↓ - Handler 执行采集逻辑 - ├─→ batch 模式：分页查询数据，批量处理 - └─→ realtime 模式：启动 Change Stream 监听 -``` - -### 3.4 任务状态管理 - -#### 3.4.1 暂停任务 - -``` -用户点击"暂停"按钮 - ↓ -API: POST /api/data-collection-tasks/{taskId}/pause - ↓ -DataCollectionTaskService->pauseTask() - ├─→ 检查任务状态（必须是 running） - ├─→ 更新任务状态为 paused - └─→ 设置 Redis 标志：data_collection_task:{taskId}:pause - ↓ -Handler 检测到暂停标志 - ↓ -停止处理新数据，等待当前批次完成 -``` - -#### 3.4.2 停止任务 - -``` -用户点击"停止"按钮 - ↓ -API: POST /api/data-collection-tasks/{taskId}/stop - ↓ -DataCollectionTaskService->stopTask() - ├─→ 更新任务状态为 stopped - ├─→ 更新进度状态为 stopped - └─→ 设置 Redis 标志：data_collection_task:{taskId}:stop - ↓ -Handler 检测到停止标志 - ↓ -立即停止处理 -``` - ---- - -## 四、数据采集模式 - -### 4.1 批量采集模式（Batch） - -**适用场景**： -- 历史数据迁移 -- 定时增量同步 -- 一次性数据采集 - -**执行流程**： -``` -1. 构建查询条件 - - 如果有 last_sync_time，只查询新数据 - - 应用 filter_conditions 过滤条件 - -2. 分页查询数据 - - 使用 batch_size 控制每批数量（默认1000） - - 按时间排序，确保顺序处理 - -3. 批量处理数据 - For each batch: - a. 应用字段映射（field_mappings） - b. 执行字段转换（类型转换、值映射） - c. 执行连表查询（lookups，如果有） - d. 写入目标数据库 - e. 更新任务进度 - -4. 更新同步状态 - - 记录 last_sync_time - - 更新任务进度和统计 -``` - -**调度方式**： -- **定时执行**：根据 Cron 表达式定时执行 -- **立即执行**：调度被禁用时，立即执行一次 - -### 4.2 实时监听模式（Realtime） - -**适用场景**： -- 实时数据同步 -- 数据库变更监听 -- 持续运行的数据采集 - -**执行流程**： -``` -1. 启动 Change Stream 监听 - - 监听指定集合的变更事件 - - 过滤操作类型（insert/update/delete） - -2. 持续监听变更 - For each change event: - a. 解析变更文档 - b. 应用字段映射和转换 - c. 写入目标数据库 - d. 更新同步状态 - -3. 错误处理和重连 - - 连接断开时自动重连 - - 记录错误日志 -``` - -**特点**： -- 持续运行，不自动停止 -- 实时响应数据变更 -- 支持断点续传（记录同步位置） - ---- - -## 五、Handler 处理机制 - -### 5.1 Handler 类型 - -系统通过 Handler 模式实现不同类型的数据采集： - -#### 5.1.1 ConsumptionCollectionHandler（消费记录采集） - -**职责**： -- 专门处理消费记录/订单数据采集 -- 支持 KR_商城 和 KR_金融 两种数据源 -- 自动提取手机号、解析用户ID -- 写入消费记录表（按时间分表） - -**处理流程**： -``` -1. 从源数据库查询订单数据 - - 支持单集合和多集合模式 - - 应用过滤条件 - -2. 字段映射和转换 - - 提取手机号（去除单引号等特殊字符） - - 转换金额字段（字符串→数字） - - 解析时间字段（字符串→DateTime） - - 转换订单状态 - -3. 身份解析 - - 通过手机号解析 user_id - - 如果不存在，创建临时人 - -4. 门店处理 - - 通过门店名称查找或创建门店 - - 获取或生成 store_id - -5. 写入消费记录 - - 调用 ConsumptionService->createRecord() - - 自动按时间分表（consumption_records_YYYYMM） - - 更新用户统计信息（total_amount, total_count） - - 触发标签计算（异步） - -6. 更新任务进度 -``` - -**字段映射示例**（KR_商城.21年贝蒂喜订单整合）： -```json -{ - "phone_number": "联系手机", // 需要去除单引号 - "amount": "买家应付货款", // 字符串转数字 - "actual_amount": "买家实际支付金额", // 字符串转数字 - "consume_time": "订单付款时间", // 字符串转DateTime - "store_name": "店铺名称" // 自动转换为store_id -} -``` - -#### 5.1.2 GenericCollectionHandler（通用数据采集） - -**职责**： -- 支持动态字段映射 -- 支持批量采集和实时监听两种模式 -- 支持连表查询（lookup） -- 支持多集合采集 - -**处理流程**： -``` -1. 确定要处理的集合 - - 单集合模式：使用 collection - - 多集合模式：使用 collections 数组 - -2. 批量采集模式 - For each collection: - a. 构建查询条件（filter_conditions） - b. 分页查询数据 - c. 应用字段映射（field_mappings 或 collection_field_mappings） - d. 执行连表查询（lookups 或 collection_lookups） - e. 写入目标数据库 - f. 更新任务进度 - -3. 实时监听模式 - For each collection: - a. 启动 Change Stream 监听 - b. 处理变更事件 - c. 应用字段映射和转换 - d. 写入目标数据库 -``` - -**字段映射配置**： -```json -{ - "field_mappings": [ - { - "source_field": "源字段名", - "target_field": "目标字段名", - "transform": "转换函数名", // 可选 - "value_mapping": [ // 可选：值映射 - { - "source_value": "源值", - "target_value": "目标值" - } - ] - } - ] -} -``` - -**连表查询配置**： -```json -{ - "lookups": [ - { - "from": "关联集合名", - "local_field": "本地字段", - "foreign_field": "关联字段", - "as": "结果字段名" - } - ] -} -``` - -#### 5.1.3 DatabaseSyncHandler（数据库同步） - -**职责**： -- 全量同步：首次启动时同步所有数据 -- 增量同步：使用 MongoDB Change Streams 实时监听 -- 支持按数据库分片（多进程并行） - -**处理流程**： -``` -1. 全量同步（首次启动） - - 遍历源数据库的所有集合 - - 批量读取数据 - - 写入目标数据库 - - 记录同步进度 - -2. 增量同步（持续运行） - - 启动 Change Stream 监听 - - 监听所有集合的变更 - - 实时同步到目标数据库 - - 记录同步位置 -``` - -### 5.2 Handler 选择规则 - -系统根据任务的 `target_type` 自动选择 Handler： - -- `target_type = 'consumption_record'` → `ConsumptionCollectionHandler` -- `target_type = 'generic'` → `GenericCollectionHandler` -- 配置文件中的 `database_sync` 任务 → `DatabaseSyncHandler` - ---- - -## 六、数据源适配器 - -### 6.1 适配器工厂 - -`DataSourceAdapterFactory` 负责创建数据源适配器： - -```php -$adapter = DataSourceAdapterFactory::create( - $dataSourceConfig['type'], // mongodb/mysql/postgresql - $dataSourceConfig // 数据源配置 -); -``` - -### 6.2 支持的适配器类型 - -#### 6.2.1 MongoDBAdapter - -**功能**： -- 连接 MongoDB 数据库 -- 执行查询操作 -- 支持 Change Stream 监听 - -**使用场景**： -- MongoDB 数据源采集 -- 数据库实时同步 - -#### 6.2.2 MySQLAdapter - -**功能**： -- 连接 MySQL 数据库 -- 执行 SQL 查询 -- 支持事务处理 - -**使用场景**： -- MySQL 数据源采集 -- 关系型数据库同步 - -### 6.3 适配器接口 - -所有适配器实现 `DataSourceAdapterInterface` 接口： - -```php -interface DataSourceAdapterInterface -{ - public function connect(array $config): bool; - public function isConnected(): bool; - public function query(string $sql, array $params = []): array; - public function close(): void; -} -``` - ---- - -## 七、消息队列处理 - -### 7.1 数据同步队列 - -**队列名**：`data_sync` - -**消息格式**： -```json -{ - "source_id": "数据源ID", - "data": [ - { - "action": "insert|update|delete", - "collection": "集合名", - "document": {...} - } - ] -} -``` - -### 7.2 DataSyncWorker 处理流程 - -``` -Worker 启动 - ↓ -初始化 RabbitMQ 连接 - ├─→ 建立连接 - ├─→ 声明队列 - ├─→ 设置 QoS（prefetch_count = 10） - └─→ 开始消费消息 - ↓ -监听消息 - ↓ -收到消息 - ↓ -processMessage() - ├─→ 解析消息（JSON） - ├─→ 调用 DataSyncService->syncData() - │ ├─→ 写入目标数据库 - │ ├─→ 更新用户统计（如果是消费记录） - │ └─→ 返回同步结果 - ├─→ 记录日志 - └─→ 确认消息（ACK） - ├─→ 成功：message->ack() - └─→ 失败：记录错误，ack（不重试，避免重复数据） -``` - -### 7.3 错误处理 - -- **业务错误**：直接确认消息，不重试（避免重复数据） -- **系统错误**：记录错误日志，确认消息（不重试） - ---- - -## 八、任务调度机制 - -### 8.1 DataSyncScheduler 调度流程 - -``` -Worker 启动 - ↓ -onWorkerStart() - ├─→ 加载配置文件中的任务 - ├─→ 从数据库加载数据源配置 - ├─→ 加载数据库中的动态任务 - └─→ 每30秒刷新一次数据库任务列表 - ↓ -加载配置文件任务 - ├─→ 检查任务是否启用 - ├─→ 检查是否应该由当前 Worker 处理（分片分配） - └─→ 根据调度配置执行： - ├─→ schedule.enabled = false：立即启动（持续运行） - └─→ schedule.enabled = true：使用 Cron 定时执行 - ↓ -加载数据库任务 - ├─→ 查询所有 status = 'running' 的任务 - ├─→ 检查 Redis 启动标志 - └─→ 根据任务模式执行： - ├─→ realtime 模式：立即启动，持续运行 - └─→ batch 模式：根据 schedule 配置执行 - ↓ -定时刷新（每30秒） - ├─→ 检查新任务（status = 'running'） - ├─→ 检查任务状态变更（启动/暂停/停止） - └─→ 更新运行中的任务列表 -``` - -### 8.2 分片策略 - -系统支持多种分片策略，实现多进程并行处理： - -#### 8.2.1 None（无分片） - -- 所有 Worker 都处理 -- 通过分布式锁保证只有一个执行 - -#### 8.2.2 By Database（按数据库分片） - -- 所有 Worker 都处理 -- 每个 Worker 处理不同的数据库 -- 在 Handler 中进行数据库分配 - -#### 8.2.3 其他分片策略 - -- 按 Worker ID 取模分配 -- 根据 `shard_count` 配置分片数量 - -### 8.3 分布式锁 - -**锁键**：`lock:data_collection:{taskId}` - -**配置**： -- TTL：300秒（默认） -- 重试次数：3次 -- 重试延迟：1000毫秒 - -**使用场景**： -- 防止同一任务被多个 Worker 重复执行 -- 保证任务执行的唯一性 - ---- - -## 九、字段映射和转换 - -### 9.1 字段映射配置 - -**单集合模式**： -```json -{ - "field_mappings": [ - { - "source_field": "源字段名", - "target_field": "目标字段名", - "transform": "转换函数名", - "value_mapping": [ - { - "source_value": "源值", - "target_value": "目标值" - } - ] - } - ] -} -``` - -**多集合模式**： -```json -{ - "collection_field_mappings": { - "集合1": [ - { - "source_field": "源字段名", - "target_field": "目标字段名" - } - ], - "集合2": [ - { - "source_field": "源字段名", - "target_field": "目标字段名" - } - ] - } -} -``` - -### 9.2 支持的转换函数 - -- `parse_phone` - 解析手机号（去除特殊字符） -- `parse_amount` - 解析金额（字符串转数字） -- `parse_datetime` - 解析时间（字符串转DateTime） -- `parse_int` - 解析整数 -- `parse_float` - 解析浮点数 -- `parse_bool` - 解析布尔值 - -### 9.3 值映射 - -支持将源字段的值映射为目标值： - -```json -{ - "value_mapping": [ - { - "source_value": "交易成功", - "target_value": "0" - }, - { - "source_value": "交易关闭", - "target_value": "1" - } - ] -} -``` - ---- - -## 十、连表查询（Lookup） - -### 10.1 Lookup 配置 - -**单集合模式**： -```json -{ - "lookups": [ - { - "from": "关联集合名", - "local_field": "本地字段", - "foreign_field": "关联字段", - "as": "结果字段名" - } - ] -} -``` - -**多集合模式**： -```json -{ - "collection_lookups": { - "集合1": [ - { - "from": "关联集合名", - "local_field": "本地字段", - "foreign_field": "关联字段", - "as": "结果字段名" - } - ] - } -} -``` - -### 10.2 Lookup 执行 - -系统使用 MongoDB 的 `$lookup` 聚合管道实现连表查询： - -```javascript -{ - $lookup: { - from: "关联集合名", - localField: "本地字段", - foreignField: "关联字段", - as: "结果字段名" - } -} -``` - ---- - -## 十一、过滤条件 - -### 11.1 过滤条件配置 - -```json -{ - "filter_conditions": [ - { - "field": "字段名", - "operator": "操作符", - "value": "值" - } - ] -} -``` - -### 11.2 支持的操作符 - -- `>` - 大于 -- `>=` - 大于等于 -- `<` - 小于 -- `<=` - 小于等于 -- `=` / `==` - 等于 -- `!=` - 不等于 -- `in` - 在列表中 -- `not_in` - 不在列表中 -- `like` - 模糊匹配（MongoDB 使用正则表达式） - -### 11.3 过滤条件应用 - -系统将过滤条件转换为 MongoDB 查询条件： - -```php -$filter = []; -foreach ($filterConditions as $condition) { - $field = $condition['field']; - $operator = $condition['operator']; - $value = $condition['value']; - - switch ($operator) { - case '>': - $filter[$field] = ['$gt' => $value]; - break; - case '>=': - $filter[$field] = ['$gte' => $value]; - break; - // ... 其他操作符 - } -} -``` - ---- - -## 十二、任务进度跟踪 - -### 12.1 进度更新 - -Handler 在处理数据时，定期更新任务进度： - -```php -$this->taskService->updateProgress($taskId, [ - 'status' => 'running', - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'total_count' => $totalCount, - 'percentage' => ($processedCount / $totalCount) * 100, - 'last_sync_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), -]); -``` - -### 12.2 进度查询 - -**API**：`GET /api/data-collection-tasks/{taskId}/progress` - -**返回格式**： -```json -{ - "status": "running", - "processed_count": 1000, - "success_count": 995, - "error_count": 5, - "total_count": 10000, - "percentage": 10.0, - "start_time": "2025-01-01T10:00:00.000Z", - "last_sync_time": "2025-01-01T10:05:00.000Z" -} -``` - ---- - -## 十三、错误处理和重试 - -### 13.1 错误处理策略 - -**单个记录错误**： -- 记录错误日志 -- 错误计数 +1 -- 继续处理下一条记录 - -**任务级错误**： -- 更新任务状态为 `error` -- 记录错误信息到 `progress.last_error` -- 记录错误日志 - -### 13.2 重试机制 - -**批量采集**： -- 支持断点续传（记录 `last_sync_time`） -- 下次执行时从上次位置继续 - -**实时监听**： -- 连接断开时自动重连 -- 记录同步位置，避免数据丢失 - ---- - -## 十四、数据流图 - -### 14.1 批量采集数据流 - -``` -数据源（MongoDB/MySQL） - ↓ -DataSyncScheduler（调度器） - ├─→ 创建数据源适配器 - └─→ 调用 Handler->collect() - ↓ - Handler 执行采集 - ├─→ 分页查询数据 - ├─→ 应用字段映射 - ├─→ 执行字段转换 - ├─→ 执行连表查询 - └─→ 写入目标数据库 - ├─→ 直接写入（同步） - └─→ 推送到队列（异步） - ↓ - DataSyncWorker 消费消息 - ↓ - 写入目标数据库 -``` - -### 14.2 实时监听数据流 - -``` -数据源（MongoDB） - ↓ -DataSyncScheduler（调度器） - ├─→ 创建数据源适配器 - └─→ 调用 Handler->collect() - ↓ - Handler 启动 Change Stream - ↓ - 持续监听变更事件 - ↓ - 处理变更事件 - ├─→ 解析变更文档 - ├─→ 应用字段映射 - └─→ 写入目标数据库 -``` - -### 14.3 消费记录采集数据流 - -``` -订单数据（KR_商城） - ↓ -ConsumptionCollectionHandler - ├─→ 提取手机号 - ├─→ 字段映射和转换 - └─→ 调用 ConsumptionService->createRecord() - ├─→ 身份解析（IdentifierService） - │ ├─→ 通过手机号解析 user_id - │ └─→ 如果不存在，创建临时人 - ├─→ 门店处理（StoreService） - │ └─→ 查找或创建门店，获取 store_id - ├─→ 写入 consumption_records（按时间分表） - ├─→ 更新 user_profile 统计信息 - └─→ 触发标签计算（推送到 RabbitMQ） - ↓ - TagCalculationWorker 消费消息 - ↓ - 计算用户标签 -``` - ---- - -## 十五、API 接口汇总 - -### 15.1 数据采集任务接口 - -- `GET /api/data-collection-tasks` - 获取任务列表 -- `POST /api/data-collection-tasks` - 创建任务 -- `GET /api/data-collection-tasks/{taskId}` - 获取任务详情 -- `PUT /api/data-collection-tasks/{taskId}` - 更新任务 -- `DELETE /api/data-collection-tasks/{taskId}` - 删除任务 -- `POST /api/data-collection-tasks/{taskId}/start` - 启动任务 -- `POST /api/data-collection-tasks/{taskId}/pause` - 暂停任务 -- `POST /api/data-collection-tasks/{taskId}/stop` - 停止任务 -- `GET /api/data-collection-tasks/{taskId}/progress` - 获取任务进度 - -### 15.2 数据源接口 - -- `GET /api/data-sources` - 获取数据源列表 -- `POST /api/data-sources` - 创建数据源 -- `GET /api/data-sources/{dataSourceId}` - 获取数据源详情 -- `PUT /api/data-sources/{dataSourceId}` - 更新数据源 -- `DELETE /api/data-sources/{dataSourceId}` - 删除数据源 -- `POST /api/data-sources/test-connection` - 测试数据源连接 - -### 15.3 辅助接口 - -- `GET /api/data-collection-tasks/data-sources` - 获取数据源列表（用于选择） -- `GET /api/data-collection-tasks/data-sources/{id}/databases` - 获取数据库列表 -- `GET /api/data-collection-tasks/data-sources/{id}/databases/{db}/collections` - 获取集合列表 -- `GET /api/data-collection-tasks/data-sources/{id}/databases/{db}/collections/{coll}/fields` - 获取字段列表 -- `GET /api/data-collection-tasks/handlers/{type}/target-fields` - 获取Handler目标字段列表 -- `POST /api/data-collection-tasks/preview-query` - 预览查询结果 - ---- - -## 十六、关键设计要点 - -### 16.1 配置化设计 - -- **任务配置**：支持配置文件和数据库两种方式 -- **字段映射**：通过配置实现灵活的字段映射 -- **数据源配置**：统一管理，支持动态加载 - -### 16.2 高可用性 - -- **分布式锁**：防止任务重复执行 -- **断点续传**：支持从上次位置继续 -- **错误隔离**：单个记录错误不影响整体任务 -- **自动重连**：实时监听模式支持自动重连 - -### 16.3 可扩展性 - -- **Handler 模式**：易于添加新的采集类型 -- **适配器模式**：易于支持新的数据源类型 -- **分片策略**：支持多进程并行处理 - -### 16.4 可观测性 - -- **进度跟踪**：实时跟踪任务进度 -- **日志记录**：完整的业务日志和错误日志 -- **统计信息**：记录任务执行统计 - ---- - -## 十七、使用示例 - -### 17.1 创建消费记录采集任务 - -**场景**：从 KR_商城.21年贝蒂喜订单整合 采集消费记录 - -**配置步骤**： - -1. **选择数据源**：选择 KR_商城 数据源 -2. **选择数据库和集合**：`KR_商城` / `21年贝蒂喜订单整合` -3. **选择Handler**：`消费记录处理（ConsumptionCollectionHandler）` -4. **配置字段映射**： - ```json - { - "phone_number": "联系手机", - "amount": "买家应付货款", - "actual_amount": "买家实际支付金额", - "consume_time": "订单付款时间", - "store_name": "店铺名称" - } - ``` -5. **配置过滤条件**（可选）： - ```json - [ - { - "field": "买家实际支付金额", - "operator": "!=", - "value": "0" - } - ] - ``` -6. **配置调度计划**： - - 模式：`batch`（批量采集） - - 调度：启用定时任务，Cron 表达式 `0 2 * * *`（每天凌晨2点） -7. **保存并启动任务** - -### 17.2 创建通用数据采集任务 - -**场景**：从多个集合采集数据并写入目标数据库 - -**配置步骤**： - -1. **选择数据源**：选择源数据源 -2. **选择数据库和集合**：支持多集合模式 -3. **选择Handler**：`通用数据采集（GenericCollectionHandler）` -4. **配置目标数据源**：选择目标数据源、数据库、集合 -5. **配置字段映射**：为每个集合配置字段映射 -6. **配置连表查询**（可选）：配置 lookup 查询 -7. **配置过滤条件**（可选） -8. **配置调度计划** -9. **保存并启动任务** - ---- - -## 十八、总结 - -数据采集系统提供了完整的数据采集、同步和管理功能，支持批量采集和实时监听两种模式，通过配置化的方式灵活管理采集任务。 - -### 18.1 核心特性 - -- ✅ **多数据源支持**：MongoDB、MySQL、PostgreSQL -- ✅ **多种采集模式**：批量采集、实时监听 -- ✅ **灵活的字段映射**：支持字段转换、值映射 -- ✅ **连表查询支持**：支持 MongoDB lookup 查询 -- ✅ **任务管理**：完整的任务创建、执行、监控功能 -- ✅ **进度跟踪**：实时跟踪任务进度和统计 -- ✅ **错误处理**：完善的错误处理和重试机制 - -### 18.2 使用场景 - -1. **历史数据迁移**：批量采集历史数据 -2. **实时数据同步**：实时监听数据变更 -3. **消费记录采集**：采集订单数据并触发标签计算 -4. **数据库同步**：数据库间的实时同步 - ---- - -**文档生成时间**：2025-01-XX -**项目版本**：基于当前代码库分析 - diff --git a/Moncter/Moncter/提示词/当前架构设计/标签引擎相关功能流程逻辑.md b/Moncter/Moncter/提示词/当前架构设计/标签引擎相关功能流程逻辑.md deleted file mode 100644 index c24210d7..00000000 --- a/Moncter/Moncter/提示词/当前架构设计/标签引擎相关功能流程逻辑.md +++ /dev/null @@ -1,884 +0,0 @@ -# 标签引擎相关功能流程逻辑 - -## 一、系统概述 - -标签引擎是系统的核心功能模块，负责根据用户数据自动计算和更新用户标签。系统支持实时计算、批量计算和定时计算三种模式，通过规则引擎实现灵活的标签计算逻辑。 - -### 1.1 核心组件 - -- **TagService** - 标签计算服务，核心业务逻辑 -- **TagTaskService** - 标签任务管理服务 -- **TagTaskExecutor** - 标签任务执行器 -- **SimpleRuleEngine** - 简单规则引擎 -- **TagCalculationWorker** - 标签计算Worker（异步处理） -- **TagDefinitionRepository** - 标签定义数据访问 -- **UserTagRepository** - 用户标签数据访问 -- **TagHistoryRepository** - 标签历史数据访问 - -### 1.2 数据存储 - -- **tag_definitions** - 标签定义集合（规则配置） -- **user_tags** - 用户标签集合（标签值） -- **tag_history** - 标签变更历史集合 -- **tag_tasks** - 标签任务集合 -- **tag_task_executions** - 标签任务执行记录集合 - ---- - -## 二、标签定义管理 - -### 2.1 标签定义结构 - -标签定义存储在 `tag_definitions` 集合中，包含以下关键字段： - -```javascript -{ - tag_id: String, // 标签ID（UUID） - tag_code: String, // 标签代码（唯一标识） - tag_name: String, // 标签名称 - category: String, // 标签分类 - rule_type: String, // 规则类型（simple/pipeline/custom） - rule_config: Object, // 规则配置（JSON） - update_frequency: String, // 更新频率（real_time/daily/weekly/monthly） - status: Number, // 状态（0:启用, 1:禁用） - priority: Number, // 优先级 - version: Number, // 版本号 - create_time: Date, - update_time: Date -} -``` - -### 2.2 规则配置格式 - -#### Simple 规则配置示例 - -```json -{ - "rule_type": "simple", - "conditions": [ - { - "field": "total_amount", - "operator": ">=", - "value": 1000 - }, - { - "field": "total_count", - "operator": ">=", - "value": 10 - } - ], - "tag_value": "VIP", - "confidence": 0.9 -} -``` - -#### 支持的运算符 - -- `>` - 大于 -- `>=` - 大于等于 -- `<` - 小于 -- `<=` - 小于等于 -- `=` / `==` - 等于 -- `!=` - 不等于 -- `in` - 在列表中 -- `not_in` - 不在列表中 - -### 2.3 标签定义管理流程 - -``` -创建/编辑标签定义 - ↓ -验证规则配置格式 - ↓ -保存到 tag_definitions 集合 - ↓ -如果状态为启用，立即生效 -``` - ---- - -## 三、标签计算流程 - -### 3.1 标签计算触发方式 - -系统支持三种标签计算触发方式： - -#### 3.1.1 实时计算（Real-time） - -**触发场景**： -- 消费记录写入时自动触发 -- 用户数据更新时触发 - -**流程**： -``` -消费记录写入 - ↓ -ConsumptionService->createRecord() - ├─→ 写入 consumption_records - ├─→ 更新 user_profile 统计信息 - └─→ 推送标签计算消息到 RabbitMQ - ↓ - TagCalculationWorker 消费消息 - ↓ - TagService->calculateTags() - ├─→ 获取用户数据 - ├─→ 获取所有 real_time 标签定义 - ├─→ 规则引擎计算标签值 - ├─→ 更新 user_tags - └─→ 记录 tag_history（如果值变化） -``` - -**消息格式**： -```json -{ - "user_id": "用户ID", - "tag_ids": null, // null 表示计算所有 real_time 标签 - "trigger_type": "consumption_record", - "record_id": "记录ID", - "timestamp": 1234567890 -} -``` - -#### 3.1.2 批量计算（Batch） - -**触发场景**： -- 通过标签任务手动触发 -- 定时任务触发（Cron） - -**流程**： -``` -创建标签任务 - ├─→ 任务类型：full（全量）/ incremental（增量）/ specified（指定） - ├─→ 目标标签：指定标签ID列表 - ├─→ 用户范围：all（全部）/ list（指定用户）/ filter（条件筛选） - └─→ 调度计划：Cron 表达式 - ↓ -用户点击"启动"按钮 - ↓ -TagTaskService->startTask() - ├─→ 更新任务状态为 running - └─→ 设置 Redis 标志：tag_task:{taskId}:start - ↓ -TagTaskExecutor->execute() - ├─→ 获取用户ID列表（根据 user_scope） - ├─→ 批量处理用户（批次大小可配置） - │ ├─→ 遍历每个用户 - │ ├─→ TagService->calculateTags(userId, targetTagIds) - │ └─→ 更新任务进度 - └─→ 记录执行结果 -``` - -#### 3.1.3 手动计算（Manual） - -**触发场景**： -- 通过 API 手动触发单个用户的标签计算 -- 用户标签查询页面点击"重新计算" - -**流程**： -``` -API: PUT /api/users/{user_id}/tags - ↓ -TagController->calculate() - ↓ -TagService->calculateTags(userId) - ├─→ 获取用户数据 - ├─→ 获取所有 real_time 标签定义 - ├─→ 规则引擎计算 - └─→ 更新标签 -``` - -### 3.2 标签计算核心流程 - -#### 3.2.1 TagService->calculateTags() 详细流程 - -```php -1. 获取用户数据 - - 从 user_profile 获取用户统计信息 - - 准备用户数据数组： - * total_amount: 总消费金额 - * total_count: 总消费次数 - * last_consume_time: 最后消费时间（时间戳） - -2. 获取标签定义列表 - - 如果指定 tagIds，只获取指定的标签定义 - - 如果 tagIds 为 null，获取所有启用且 update_frequency = 'real_time' 的标签 - - 只获取 status = 0（启用）的标签 - -3. 遍历每个标签定义 - For each tagDef: - a. 解析规则配置（rule_config） - b. 根据规则类型选择计算引擎 - - simple: 使用 SimpleRuleEngine - - pipeline: 暂不支持 - - custom: 暂不支持 - c. 规则引擎计算标签值 - - 评估所有条件（conditions） - - 如果所有条件满足，返回 tag_value 和 confidence - - 如果条件不满足，返回 false 和 confidence = 0.0 - d. 获取旧标签值（用于历史记录） - e. 更新或创建 user_tags 记录 - - 如果标签已存在，更新 tag_value、confidence、update_time - - 如果标签不存在，创建新记录 - f. 记录标签变更历史（仅当值发生变化时） - - 写入 tag_history 集合 - - 记录 old_value、new_value、change_reason、change_time - g. 记录计算日志 - -4. 更新用户的标签更新时间 - - user_profile.tags_update_time = now() - -5. 返回更新的标签列表 -``` - -#### 3.2.2 SimpleRuleEngine 计算逻辑 - -```php -1. 验证规则配置 - - rule_type 必须是 'simple' - - conditions 必须存在且为数组 - -2. 评估所有条件 - For each condition: - a. 从 userData 获取字段值 - b. 根据 operator 进行比较 - - >, >=, <, <=: 数值比较 - - =, !=: 相等比较 - - in, not_in: 数组包含判断 - c. 如果字段不存在，默认值为 0 - -3. 判断结果 - - 如果所有条件都满足（allMatch = true）： - * value = ruleConfig['tag_value'] ?? true - * confidence = ruleConfig['confidence'] ?? 1.0 - - 如果任一条件不满足： - * value = false - * confidence = 0.0 - -4. 返回计算结果 - { - value: mixed, - confidence: float - } -``` - -### 3.3 标签值存储格式 - -标签值统一转换为字符串格式存储： - -- **布尔值**：`true` → `"true"`, `false` → `"false"` -- **数值**：直接转换为字符串 -- **数组/对象**：JSON 序列化 -- **字符串**：直接存储 - -标签值类型（tag_value_type）： -- `boolean` - 布尔类型 -- `number` - 数值类型 -- `string` - 字符串类型 -- `json` - JSON 类型 - ---- - -## 四、标签任务管理 - -### 4.1 标签任务结构 - -```javascript -{ - task_id: String, // 任务ID（UUID） - name: String, // 任务名称 - description: String, // 任务描述 - task_type: String, // 任务类型（full/incremental/specified） - target_tag_ids: Array, // 目标标签ID列表 - user_scope: { // 用户范围 - type: String, // all/list/filter - user_ids: Array, // 指定用户列表（type=list时） - conditions: Array // 筛选条件（type=filter时） - }, - schedule: { // 调度计划 - enabled: Boolean, // 是否启用定时任务 - cron: String // Cron 表达式 - }, - config: { // 任务配置 - concurrency: Number, // 并发数 - batch_size: Number, // 批次大小 - error_handling: String // 错误处理策略（skip/stop） - }, - status: String, // 任务状态（pending/running/paused/stopped/error） - progress: { // 任务进度 - total_users: Number, // 总用户数 - processed_users: Number, // 已处理用户数 - success_count: Number, // 成功数量 - error_count: Number, // 错误数量 - percentage: Number // 完成百分比 - }, - statistics: { // 任务统计 - total_executions: Number, // 总执行次数 - success_executions: Number, // 成功执行次数 - failed_executions: Number, // 失败执行次数 - last_run_time: Date // 最后执行时间 - }, - created_by: String, - created_at: Date, - updated_at: Date -} -``` - -### 4.2 任务类型说明 - -#### 4.2.1 Full（全量计算） - -- 计算所有用户的所有指定标签 -- 适用于首次初始化或全量更新 - -#### 4.2.2 Incremental（增量计算） - -- 只计算有数据变更的用户 -- 适用于定期更新场景 - -#### 4.2.3 Specified（指定计算） - -- 只计算指定用户列表的标签 -- 适用于特定用户群体 - -### 4.3 用户范围配置 - -#### 4.3.1 All（全部用户） - -```json -{ - "type": "all" -} -``` - -获取所有 `status = 0` 的用户。 - -#### 4.3.2 List（指定用户列表） - -```json -{ - "type": "list", - "user_ids": ["user1", "user2", "user3"] -} -``` - -只计算指定用户ID列表的标签。 - -#### 4.3.3 Filter（条件筛选） - -```json -{ - "type": "filter", - "conditions": [ - { - "field": "total_amount", - "operator": ">=", - "value": 1000 - }, - { - "field": "total_count", - "operator": ">=", - "value": 10 - } - ] -} -``` - -根据条件筛选用户，支持多个条件（AND 逻辑）。 - -### 4.4 任务执行流程 - -``` -1. 创建执行记录 - - execution_id: UUID - - task_id: 任务ID - - started_at: 开始时间 - - status: running - -2. 获取用户ID列表 - - 根据 user_scope 配置获取用户列表 - - 计算总用户数 - -3. 更新任务进度 - - total_users: 总用户数 - - processed_users: 0 - - success_count: 0 - - error_count: 0 - - percentage: 0 - -4. 批量处理用户 - For each batch (batch_size = 100): - a. 检查任务状态（是否被暂停/停止） - b. 遍历批次中的每个用户 - - TagService->calculateTags(userId, targetTagIds) - - 成功：success_count++ - - 失败：error_count++ - - 根据 error_handling 策略决定是否继续 - c. 每处理 10 个用户更新一次进度 - - processed_users - - success_count - - error_count - - percentage = (processed_users / total_users) * 100 - -5. 更新最终进度 - - percentage = 100（或实际完成百分比） - -6. 更新执行记录 - - status: completed/failed - - finished_at: 结束时间 - - processed_users, success_count, error_count - -7. 更新任务统计 - - total_executions++ - - success_executions++ / failed_executions++ - - last_run_time = now() -``` - -### 4.5 任务状态管理 - -#### 4.5.1 启动任务 - -``` -用户点击"启动"按钮 - ↓ -API: POST /api/tag-tasks/{task_id}/start - ↓ -TagTaskService->startTask() - ├─→ 检查任务状态（不能是 running） - ├─→ 更新任务状态为 running - └─→ 设置 Redis 标志：tag_task:{taskId}:start - ↓ -TagTaskExecutor 检测到标志 - ↓ -执行任务（见 4.4 任务执行流程） -``` - -#### 4.5.2 暂停任务 - -``` -用户点击"暂停"按钮 - ↓ -API: POST /api/tag-tasks/{task_id}/pause - ↓ -TagTaskService->pauseTask() - ├─→ 检查任务状态（必须是 running） - ├─→ 更新任务状态为 paused - └─→ 设置 Redis 标志：tag_task:{taskId}:pause - ↓ -TagTaskExecutor 检测到标志 - ↓ -停止处理新批次，等待当前批次完成 -``` - -#### 4.5.3 停止任务 - -``` -用户点击"停止"按钮 - ↓ -API: POST /api/tag-tasks/{task_id}/stop - ↓ -TagTaskService->stopTask() - ├─→ 更新任务状态为 stopped - └─→ 设置 Redis 标志：tag_task:{taskId}:stop - ↓ -TagTaskExecutor 检测到标志 - ↓ -立即停止处理 -``` - ---- - -## 五、标签查询和筛选 - -### 5.1 用户标签查询 - -#### 5.1.1 查询单个用户的所有标签 - -``` -API: GET /api/users/{user_id}/tags - ↓ -TagController->getUserTags() - ↓ -TagService->getUserTags(userId) - ├─→ 从 user_tags 查询该用户的所有标签 - ├─→ 关联 tag_definitions 获取标签定义信息 - └─→ 返回标签列表（包含标签值、置信度、生效时间等） -``` - -**返回格式**： -```json -{ - "user_id": "用户ID", - "tags": [ - { - "tag_id": "标签ID", - "tag_code": "标签代码", - "tag_name": "标签名称", - "category": "标签分类", - "tag_value": "标签值", - "tag_value_type": "值类型", - "confidence": 0.9, - "effective_time": "生效时间", - "expire_time": "过期时间", - "update_time": "更新时间" - } - ], - "count": 10 -} -``` - -#### 5.1.2 重新计算用户标签 - -``` -API: PUT /api/users/{user_id}/tags - ↓ -TagController->calculate() - ↓ -TagService->calculateTags(userId) - └─→ 执行标签计算流程（见 3.2.1） -``` - -### 5.2 标签筛选用户 - -#### 5.2.1 根据标签条件筛选用户 - -``` -API: POST /api/tags/filter - ↓ -TagController->filterUsers() - ↓ -TagService->filterUsersByTags() - ├─→ 根据 tag_code 获取 tag_id 列表 - ├─→ 根据逻辑类型处理查询 - │ ├─→ AND 逻辑：分别查询每个条件，取交集 - │ └─→ OR 逻辑：使用 orWhere 查询 - ├─→ 如果标签未计算，基于规则从 user_profile 筛选 - ├─→ 分页处理 - └─→ 返回用户列表（可选包含用户信息） -``` - -**请求格式**： -```json -{ - "tag_conditions": [ - { - "tag_code": "VIP", - "operator": "=", - "value": "true" - }, - { - "tag_code": "消费金额等级", - "operator": ">=", - "value": "1000" - } - ], - "logic": "AND", - "page": 1, - "page_size": 20, - "include_user_info": true -} -``` - -**筛选逻辑**： - -1. **AND 逻辑**： - - 分别查询每个条件满足的用户ID - - 取所有条件的交集 - - 如果标签未计算，基于规则从 `user_profile` 筛选 - -2. **OR 逻辑**： - - 使用 `orWhere` 查询满足任一条件的用户 - - 去重后返回 - -3. **支持的操作符**： - - `=`, `!=` - 等于/不等于 - - `>`, `>=`, `<`, `<=` - 数值比较 - - `in`, `not_in` - 列表包含判断 - -### 5.3 标签统计 - -#### 5.3.1 获取标签统计信息 - -``` -API: GET /api/tags/statistics - ↓ -TagController->getStatistics() - ↓ -TagService->getTagStatistics() - ├─→ 标签覆盖度统计 - │ ├─→ 总用户数 - │ ├─→ 已打标签用户数 - │ └─→ 覆盖率 = (已打标签用户数 / 总用户数) * 100 - ├─→ 标签值分布 - │ └─→ 各标签值的出现次数 - └─→ 标签趋势数据 - └─→ 按日期统计标签变更次数 -``` - -### 5.4 标签历史查询 - -#### 5.4.1 获取标签变更历史 - -``` -API: GET /api/tags/history - ↓ -TagController->getHistory() - ↓ -TagHistoryRepository->query() - ├─→ 支持筛选条件： - │ ├─→ user_id: 按用户筛选 - │ ├─→ tag_id: 按标签筛选 - │ └─→ start_date, end_date: 按时间范围筛选 - ├─→ 分页查询 - └─→ 返回历史记录列表 -``` - -**历史记录结构**： -```javascript -{ - history_id: String, // 历史记录ID - user_id: String, // 用户ID - tag_id: String, // 标签ID - old_value: String, // 旧值 - new_value: String, // 新值 - change_reason: String, // 变更原因（auto_calculate/manual/tag_deleted） - change_time: Date, // 变更时间 - operator: String // 操作人（system/user_id） -} -``` - ---- - -## 六、异步处理机制 - -### 6.1 RabbitMQ 队列配置 - -#### 6.1.1 标签计算队列 - -- **队列名**：`tag_calculation` -- **交换机**：`tag_calculation_exchange` -- **路由键**：`tag.calculation` -- **持久化**：是 -- **消息格式**：JSON - -#### 6.1.2 消息推送 - -```php -QueueService::pushTagCalculation([ - 'user_id' => $userId, - 'tag_ids' => null, // null 表示计算所有 real_time 标签 - 'trigger_type' => 'consumption_record', - 'record_id' => $recordId, - 'timestamp' => time(), -]); -``` - -### 6.2 TagCalculationWorker 处理流程 - -``` -Worker 启动 - ↓ -初始化 RabbitMQ 连接 - ├─→ 建立连接 - ├─→ 声明队列 - ├─→ 设置 QoS（prefetch_count = 1） - └─→ 开始消费消息 - ↓ -监听消息（每 0.1 秒检查一次） - ↓ -收到消息 - ↓ -processMessage() - ├─→ 解析消息（JSON） - ├─→ 验证必要字段（user_id） - ├─→ 创建 TagService 实例 - ├─→ 执行标签计算 - │ └─→ TagService->calculateTags(userId, tagIds) - ├─→ 记录日志和性能指标 - └─→ 确认消息（ACK） - ├─→ 成功：message->ack() - └─→ 失败： - ├─→ 业务错误（InvalidArgumentException）：ack（不重试） - └─→ 系统错误：nack（重新入队） -``` - -### 6.3 错误处理和重试 - -#### 6.3.1 业务错误 - -- **类型**：`InvalidArgumentException`（如用户不存在） -- **处理**：直接确认消息（ACK），不重试 -- **原因**：业务逻辑错误，重试不会改变结果 - -#### 6.3.2 系统错误 - -- **类型**：数据库连接失败、网络错误等 -- **处理**：拒绝消息并重新入队（NACK with requeue = true） -- **原因**：临时性错误，重试可能成功 - -### 6.4 降级策略 - -如果 RabbitMQ 不可用或推送失败： - -``` -推送消息到队列失败 - ↓ -记录错误日志 - ↓ -降级到同步调用 - ↓ -TagService->calculateTags()（同步执行） - ↓ -返回结果（包含标签信息） -``` - ---- - -## 七、数据流图 - -### 7.1 实时标签计算数据流 - -``` -数据采集任务 - ↓ -消费记录写入 - ↓ -ConsumptionService->createRecord() - ├─→ 写入 consumption_records - ├─→ 更新 user_profile（total_amount, total_count） - └─→ 推送消息到 RabbitMQ - ↓ - TagCalculationWorker（异步） - ↓ - TagService->calculateTags() - ├─→ 获取用户数据（user_profile） - ├─→ 获取标签定义（tag_definitions） - ├─→ 规则引擎计算（SimpleRuleEngine） - ├─→ 更新用户标签（user_tags） - └─→ 记录变更历史（tag_history） -``` - -### 7.2 批量标签计算数据流 - -``` -创建标签任务 - ↓ -用户启动任务 - ↓ -TagTaskService->startTask() - ├─→ 更新任务状态 - └─→ 设置 Redis 标志 - ↓ -TagTaskExecutor->execute() - ├─→ 获取用户列表（根据 user_scope） - ├─→ 批量处理用户 - │ └─→ TagService->calculateTags() - └─→ 更新任务进度和统计 -``` - -### 7.3 标签查询数据流 - -``` -API 请求 - ↓ -TagController - ↓ -TagService - ├─→ 查询 user_tags（用户标签） - ├─→ 关联 tag_definitions（标签定义） - ├─→ 筛选条件处理（AND/OR） - └─→ 返回结果 -``` - ---- - -## 八、关键设计要点 - -### 8.1 性能优化 - -1. **异步处理**：标签计算通过 RabbitMQ 异步处理，不阻塞主流程 -2. **批量处理**：标签任务支持批量处理，提高效率 -3. **增量更新**：只计算有数据变更的用户 -4. **缓存机制**：标签定义可以缓存，减少数据库查询 - -### 8.2 数据一致性 - -1. **事务处理**：标签更新和历史记录在同一事务中 -2. **幂等性**：标签计算支持重复执行，结果一致 -3. **错误隔离**：单个标签计算失败不影响其他标签 - -### 8.3 可扩展性 - -1. **规则引擎扩展**：支持添加新的规则类型（pipeline/custom） -2. **多数据源**：可以从多个数据源获取用户数据 -3. **自定义计算**：支持自定义计算逻辑 - -### 8.4 可观测性 - -1. **日志记录**：完整的业务日志和性能日志 -2. **进度跟踪**：实时跟踪任务进度 -3. **历史记录**：记录所有标签变更历史 -4. **统计信息**：提供标签统计和趋势分析 - ---- - -## 九、API 接口汇总 - -### 9.1 标签定义接口 - -- `GET /api/tag-definitions` - 获取标签定义列表 -- `POST /api/tag-definitions` - 创建标签定义 -- `GET /api/tag-definitions/{tag_id}` - 获取标签定义详情 -- `PUT /api/tag-definitions/{tag_id}` - 更新标签定义 -- `DELETE /api/tag-definitions/{tag_id}` - 删除标签定义 -- `POST /api/tag-definitions/batch` - 批量创建标签定义 - -### 9.2 标签任务接口 - -- `GET /api/tag-tasks` - 获取标签任务列表 -- `POST /api/tag-tasks` - 创建标签任务 -- `GET /api/tag-tasks/{task_id}` - 获取任务详情 -- `PUT /api/tag-tasks/{task_id}` - 更新任务 -- `DELETE /api/tag-tasks/{task_id}` - 删除任务 -- `POST /api/tag-tasks/{task_id}/start` - 启动任务 -- `POST /api/tag-tasks/{task_id}/pause` - 暂停任务 -- `POST /api/tag-tasks/{task_id}/stop` - 停止任务 -- `GET /api/tag-tasks/{task_id}/executions` - 获取执行记录 - -### 9.3 标签查询接口 - -- `GET /api/users/{user_id}/tags` - 获取用户标签 -- `PUT /api/users/{user_id}/tags` - 重新计算用户标签 -- `DELETE /api/users/{user_id}/tags/{tag_id}` - 删除用户标签 -- `POST /api/tags/filter` - 根据标签筛选用户 -- `GET /api/tags/statistics` - 获取标签统计信息 -- `GET /api/tags/history` - 获取标签历史记录 - ---- - -## 十、总结 - -标签引擎系统提供了完整的标签计算、管理和查询功能，支持实时计算、批量计算和定时计算三种模式。通过规则引擎实现灵活的标签计算逻辑，通过异步处理保证系统性能，通过历史记录提供完整的审计追踪。 - -### 10.1 核心特性 - -- ✅ **实时计算**：消费记录写入时自动触发标签计算 -- ✅ **批量计算**：支持全量、增量、指定用户的批量计算 -- ✅ **规则引擎**：支持简单规则，可扩展支持复杂规则 -- ✅ **异步处理**：通过 RabbitMQ 异步处理，不阻塞主流程 -- ✅ **任务管理**：完整的任务创建、执行、监控功能 -- ✅ **标签筛选**：支持多条件组合筛选用户 -- ✅ **历史追踪**：完整的标签变更历史记录 - -### 10.2 使用场景 - -1. **实时标签更新**：用户消费后自动更新标签 -2. **批量标签初始化**：首次上线时批量计算所有用户标签 -3. **定时标签更新**：按日/周/月定期更新标签 -4. **用户分群**：根据标签筛选特定用户群体 -5. **标签分析**：统计标签覆盖度、值分布、趋势数据 - ---- - -**文档生成时间**：2025-01-XX -**项目版本**：基于当前代码库分析 - diff --git a/Moncter/Moncter/提示词/当前架构设计/采集进度100%停止机制分析报告.md b/Moncter/Moncter/提示词/当前架构设计/采集进度100%停止机制分析报告.md deleted file mode 100644 index 288f7d28..00000000 --- a/Moncter/Moncter/提示词/当前架构设计/采集进度100%停止机制分析报告.md +++ /dev/null @@ -1,278 +0,0 @@ -# 采集进度100%停止机制分析报告 - -## 分析目标 -检查当采集进度达到100%后，代码是否会正确停止采集。 - -## 代码流程分析 - -### 1. ConsumptionCollectionHandler::collectFromKrMall() 方法 - -#### 流程概览 -``` -初始化 → do-while循环 → 查询批次数据 → 处理批次 → 更新进度 → 检查退出条件 -``` - -#### 详细流程 - -**步骤1：初始化（第166-172行）** -```php -$offset = 0; -$processedCount = 0; -$successCount = 0; -$errorCount = 0; -$lastUpdateCount = 0; -$isCompleted = false; // 标记是否已完成 -``` - -**步骤2：主循环开始（第174行）** -```php -do { - // 查询批次数据 - // 处理批次 - // 更新进度 -} while (count($batch) === $batchSize && !$isCompleted); -``` -**退出条件**：当批次为空（`count($batch) < $batchSize`）**或** `$isCompleted = true` 时退出循环。 - -**步骤3：批次处理循环（第212-248行）** -```php -foreach ($batch as $index => $orderData) { - // 检查是否已达到总数（第222-225行） - if ($totalCount > 0 && $processedCount >= $totalCount) { - $isCompleted = true; - break; // 跳出当前批次处理循环 - } - - $processedCount++; // 第227行：递增处理计数 - - // 处理订单数据 - $this->processKrMallOrder($orderData, $taskConfig); - $successCount++; -} -``` - -**⚠️ 问题1：检查时机问题** -- 第222行检查 `processedCount >= totalCount` 时，`processedCount` 还没有递增 -- 第227行才会递增 `processedCount` -- **这意味着**：如果 `processedCount == totalCount - 1`，检查通过，然后递增后变成 `totalCount`，但此时已经处理了 `totalCount` 条数据 -- **实际上**：这个检查应该能正常工作，因为检查的是"是否已经达到或超过总数" - -**步骤4：批次处理完成后更新进度（第250-279行）** -```php -if (!empty($taskId) && $totalCount > 0) { - if (($processedCount - $lastUpdateCount) >= $updateInterval || $processedCount >= $totalCount) { - $percentage = round(($processedCount / $totalCount) * 100, 2); - - // 检查是否达到100%（第257-268行） - if ($processedCount >= $totalCount) { - // 进度达到100%，停止采集并更新状态为已完成 - $this->updateProgress($taskId, [ - 'status' => 'completed', - 'processed_count' => $processedCount, - 'success_count' => $successCount, - 'error_count' => $errorCount, - 'percentage' => 100, - 'end_time' => new \MongoDB\BSON\UTCDateTime(time() * 1000), - ]); - \Workerman\Worker::safeEcho("[ConsumptionCollectionHandler] ✅ 采集完成，进度已达到100%，已停止采集\n"); - $isCompleted = true; // 标记为已完成 - } - } -} -``` - -**步骤5：循环退出检查（第296行）** -```php -} while (count($batch) === $batchSize && !$isCompleted); -``` -- 如果 `$isCompleted = true`，循环会退出 -- 如果批次为空（`count($batch) < $batchSize`），循环也会退出 - -#### 停止机制分析 - -**✅ 停止机制1：在批次处理循环中检查（第222-225行）** -- **触发条件**：`processedCount >= totalCount` -- **动作**：设置 `$isCompleted = true`，`break` 跳出内层循环 -- **效果**：停止处理当前批次的剩余数据，但**不会立即退出主循环** - -**✅ 停止机制2：在批次处理完成后检查（第257-268行）** -- **触发条件**：`processedCount >= totalCount` -- **动作**：设置 `$isCompleted = true`，更新状态为 `completed` -- **效果**：标记任务完成，下次循环检查时会退出 - -**✅ 停止机制3：主循环退出条件（第296行）** -- **退出条件**：`!$isCompleted` 为 false（即 `$isCompleted = true`） -- **效果**：退出主循环，停止采集 - -#### 潜在问题 - -**问题1：第222行检查后，第227行仍会执行** -- 如果 `processedCount == totalCount - 1`，第222行检查通过，不会 break -- 第227行递增后，`processedCount` 变成 `totalCount` -- 然后处理数据，`processedCount` 实际上会超过 `totalCount` -- **影响**：可能会多处理一条数据 - -**问题2：第222行 break 后，第257行的检查不会执行** -- 如果第222行 break，会跳出内层循环 -- 第257行的检查不会执行（因为已经跳出循环） -- 但是 `$isCompleted` 已经设置为 `true`，主循环会在下次迭代时退出 -- **影响**：状态可能不会立即更新为 `completed`，但会在循环结束后更新（第298-316行） - -**问题3：循环退出后再次检查状态（第298-316行）** -- 循环退出后，会再次检查任务状态 -- 如果任务状态不是 `completed`、`paused`、`stopped`，会更新为 `completed` -- **这是好的**：确保状态最终被更新 - -### 2. ConsumptionCollectionHandler::collectFromKrFinance() 方法 - -#### 流程概览 -``` -初始化 → foreach循环遍历集合 → 查询数据 → 处理数据 → 更新进度 → 检查退出条件 -``` - -#### 详细流程 - -**步骤1：初始化（第570-590行）** -```php -$processedCount = 0; -$successCount = 0; -$errorCount = 0; -$lastUpdateCount = 0; -$isCompleted = false; -``` - -**步骤2：外层循环遍历集合（第592行）** -```php -foreach ($collections as $collectionName) { - if ($isCompleted) { - break; // 如果已完成，退出外层循环 - } - - // 查询和处理数据 -} -``` - -**步骤3：内层循环处理数据（第590-655行）** -```php -foreach ($cursor as $doc) { - // 检查任务状态（第608-612行） - if (!empty($taskId) && !$this->checkTaskStatus($taskId)) { - $isCompleted = true; - break 2; // 跳出两层循环 - } - - // 检查是否已达到总数（第615-618行） - if ($totalCount > 0 && $processedCount >= $totalCount) { - $isCompleted = true; - break 2; // 跳出两层循环 - } - - $processedCount++; - - // 处理数据 - // 更新进度（第626-659行） - if ($totalCount > 0 && $processedCount >= $totalCount) { - // 更新状态为 completed - $isCompleted = true; - break 2; // 跳出两层循环 - } -} -``` - -#### 停止机制分析 - -**✅ 停止机制1：在每条处理前检查（第615-618行）** -- **触发条件**：`processedCount >= totalCount` -- **动作**：设置 `$isCompleted = true`，`break 2` 跳出两层循环 -- **效果**：立即停止采集 - -**✅ 停止机制2：在进度更新时检查（第637-650行）** -- **触发条件**：`processedCount >= totalCount` -- **动作**：设置 `$isCompleted = true`，`break 2` 跳出两层循环，更新状态为 `completed` -- **效果**：立即停止采集并更新状态 - -**✅ 停止机制3：外层循环退出条件（第594-596行）** -- **退出条件**：`$isCompleted = true` -- **效果**：退出外层循环，停止采集 - -### 3. GenericCollectionHandler::collect() 方法 - -#### 流程概览 -``` -初始化 → do-while循环 → 查询批次数据 → 处理批次 → 更新进度 → 检查退出条件 -``` - -#### 详细流程 - -**步骤1：主循环（第274行）** -```php -do { - // 查询和处理数据 -} while (count($batch) === $batchSize && $processedCount < $totalCount); -``` -**退出条件**：当批次为空**或** `processedCount >= totalCount` 时退出循环。 - -**步骤2：进度更新检查（第247-259行）** -```php -if ($totalCount > 0 && $processedCount >= $totalCount) { - // 更新状态为 completed - break 2; // 跳出两层循环 -} -``` - -#### 停止机制分析 - -**✅ 停止机制1：循环退出条件（第274行）** -- **退出条件**：`$processedCount >= $totalCount` -- **效果**：退出主循环，停止采集 - -**✅ 停止机制2：进度更新时检查（第247-259行）** -- **触发条件**：`processedCount >= totalCount` -- **动作**：更新状态为 `completed`，`break 2` 跳出两层循环 -- **效果**：立即停止采集并更新状态 - -## 总结 - -### ✅ 停止机制正常工作的情况 - -1. **ConsumptionCollectionHandler::collectFromKrMall()** - - ✅ 在批次处理循环中检查 `processedCount >= totalCount`，设置 `$isCompleted = true` - - ✅ 在批次处理完成后检查，设置 `$isCompleted = true` 并更新状态 - - ✅ 主循环退出条件检查 `!$isCompleted`，当 `$isCompleted = true` 时退出 - - ✅ 循环退出后再次检查状态，确保更新为 `completed` - -2. **ConsumptionCollectionHandler::collectFromKrFinance()** - - ✅ 在每条处理前检查，`break 2` 跳出两层循环 - - ✅ 在进度更新时检查，`break 2` 跳出两层循环并更新状态 - - ✅ 外层循环检查 `$isCompleted`，当为 `true` 时退出 - -3. **GenericCollectionHandler::collect()** - - ✅ 循环退出条件直接检查 `$processedCount < $totalCount` - - ✅ 在进度更新时检查，`break 2` 跳出两层循环 - -### ⚠️ 潜在问题 - -1. **ConsumptionCollectionHandler::collectFromKrMall() 第222行检查时机** - - 检查在 `$processedCount++` 之前 - - 可能导致多处理一条数据（影响较小） - -2. **状态更新时机** - - 如果第222行 break，状态可能不会立即更新 - - 但会在循环结束后更新（第298-316行），所以最终状态是正确的 - -### 建议优化 - -1. **优化检查时机**：将第222行的检查移到 `$processedCount++` 之后 -2. **确保状态立即更新**：如果第222行 break，也应该立即更新状态 - -## 结论 - -**✅ 采集进度达到100%后，代码会停止采集** - -所有三个Handler都有多个停止机制，确保当 `processedCount >= totalCount` 时： -1. 设置完成标志 -2. 退出循环 -3. 更新状态为 `completed` - -虽然存在一些小的时机问题，但整体逻辑是正确的，能够确保采集在达到100%后停止。 - diff --git a/Moncter/Moncter/提示词/数据列表动态打标签实现方案.md b/Moncter/Moncter/提示词/数据列表动态打标签实现方案.md deleted file mode 100644 index 7b041b6d..00000000 --- a/Moncter/Moncter/提示词/数据列表动态打标签实现方案.md +++ /dev/null @@ -1,682 +0,0 @@ -# 数据列表动态打标签实现方案 - -## 一、核心思路 - -将SQL/MongoDB查询配置保存到数据表中，标签任务执行时从配置表读取查询配置，动态执行查询并打标签。 - ---- - -## 二、数据表设计 - -### 2.1 数据列表配置表（tag_data_lists） - -```javascript -{ - list_id: String, // 列表ID（UUID） - list_code: String, // 列表编码（唯一，如: consumption_records） - list_name: String, // 列表名称（如: 消费记录表） - data_source_id: String, // 数据源ID - database: String, // 数据库名 - collection: String, // 主集合名 - query_config: { // 查询配置 - filter: [ // 过滤条件（WHERE） - { - logic: 'and', // 逻辑关系（and/or） - field: String, // 字段名 - operator: String, // 运算符 - value: Any // 值 - } - ], - lookups: [ // 联表查询（JOIN） - { - from: String, // 关联集合 - local_field: String, // 主集合字段 - foreign_field: String, // 关联集合字段 - as: String, // 结果字段名 - unwrap: Boolean, // 是否解构 - preserve_null: Boolean // 是否保留空值 - } - ], - sort: { // 排序 - field_name: 1/-1 // 1=升序，-1=降序 - }, - limit: Number // 限制数量 - }, - description: String, // 描述 - status: Number, // 状态（0=禁用，1=启用） - create_time: Date, - update_time: Date -} -``` - -### 2.2 标签定义表（tag_definitions） - -```javascript -{ - tag_id: String, - tag_code: String, - tag_name: String, - category: String, - description: String, - rule_type: String, // 规则类型（simple/regex） - rule_config: { - rule_type: String, - data_list_id: String, // 关联的数据列表ID ⭐ - data_list_name: String, // 数据列表名称 - conditions: [ // 规则条件 - { - field: String, // 字段中文名 - field_name: String, // 字段英文名 - operator: String, // 运算符 - value: Any, // 值 - tag_value: String // 标签值 ⭐ - } - ] - }, - update_frequency: String, - status: Number, - create_time: Date, - update_time: Date -} -``` - ---- - -## 三、执行流程 - -### 3.1 标签任务启动流程 - -``` -1. 用户启动标签任务 - ↓ -2. TagTaskExecutor->execute() - ├─→ 获取目标标签ID列表（target_tag_ids） - ├─→ 遍历每个标签定义 - │ ↓ - 3. 读取标签定义 - ├─→ 从 tag_definitions 表读取 - ├─→ 获取 rule_config.data_list_id - │ ↓ - 4. 读取数据列表配置 - ├─→ 根据 data_list_id 从 tag_data_lists 表读取 - ├─→ 获取 query_config（查询配置） - │ ↓ - 5. 执行动态查询 - ├─→ 连接数据源 - ├─→ 根据 query_config 构建查询 - │ ├─→ 构建过滤条件（filter） - │ ├─→ 构建联表查询（lookups） - │ ├─→ 构建排序（sort） - │ └─→ 构建限制（limit） - ├─→ 执行查询，获取数据 - │ ↓ - 6. 遍历查询结果，动态打标签 - For each record in results: - ├─→ 提取 user_id（从记录中获取） - ├─→ 遍历 rule_config.conditions - │ ├─→ 根据 field_name 获取字段值 - │ ├─→ 根据 operator 进行比较 - │ │ ├─→ 运算规则：>, >=, <, <=, =, !=, in, not_in - │ │ └─→ 正则规则：正则表达式匹配 - │ └─→ 如果条件满足，使用该条件的 tag_value - ├─→ 更新或创建 user_tags 记录 - ├─→ 记录标签变更历史（tag_history） - └─→ 更新进度 -``` - -### 3.2 查询构建示例 - -**数据列表配置**： -```json -{ - "list_id": "list_001", - "list_name": "消费记录表", - "data_source_id": "source_001", - "database": "tag_engine", - "collection": "consumption_records", - "query_config": { - "filter": [ - { - "logic": "and", - "field": "status", - "operator": "eq", - "value": "success" - } - ], - "lookups": [ - { - "from": "user_profile", - "local_field": "user_id", - "foreign_field": "user_id", - "as": "user_info", - "unwrap": true, - "preserve_null": true - } - ], - "sort": { - "create_time": -1 - }, - "limit": 10000 - } -} -``` - -**执行时构建的MongoDB查询**： -```javascript -db.consumption_records.aggregate([ - { $match: { status: { $eq: "success" } } }, - { $lookup: { - from: "user_profile", - localField: "user_id", - foreignField: "user_id", - as: "user_info" - } }, - { $unwind: { - path: "$user_info", - preserveNullAndEmptyArrays: true - } }, - { $sort: { create_time: -1 } }, - { $limit: 10000 } -]) -``` - -### 3.3 标签计算示例 - -**标签定义**： -```json -{ - "tag_id": "tag_001", - "tag_code": "consumer_level", - "tag_name": "消费等级", - "rule_type": "simple", - "rule_config": { - "rule_type": "simple", - "data_list_id": "list_001", - "data_list_name": "消费记录表", - "conditions": [ - { - "field": "交易金额", - "field_name": "amount", - "operator": "<", - "value": 3000, - "tag_value": "低价值用户" - }, - { - "field": "交易金额", - "field_name": "amount", - "operator": ">=", - "value": 3000, - "tag_value": "中等价值用户" - }, - { - "field": "交易金额", - "field_name": "amount", - "operator": ">", - "value": 9000, - "tag_value": "高价值用户" - } - ] - } -} -``` - -**执行流程**： -```php -// 1. 读取数据列表配置 -$dataList = TagDataListRepository::find('list_001'); -$queryConfig = $dataList->query_config; - -// 2. 执行查询 -$results = $this->executeQuery($dataList->data_source_id, $dataList->database, $dataList->collection, $queryConfig); - -// 3. 遍历结果，打标签 -foreach ($results as $record) { - $userId = $record['user_id']; - $amount = $record['amount']; - - // 4. 根据规则条件匹配标签值 - $tagValue = null; - foreach ($tagDef->rule_config['conditions'] as $condition) { - $fieldValue = $record[$condition['field_name']]; // 获取字段值 - - // 5. 判断条件是否满足 - if ($this->evaluateCondition($fieldValue, $condition['operator'], $condition['value'])) { - $tagValue = $condition['tag_value']; - break; // 满足第一个条件即可 - } - } - - // 6. 保存标签 - if ($tagValue !== null) { - $this->saveUserTag($userId, $tagDef->tag_id, $tagValue); - } -} -``` - ---- - -## 四、后端实现要点 - -### 4.1 需要创建的文件 - -**控制器**： -- `app/controller/TagDataListController.php` - 数据列表管理控制器 - -**服务**： -- `app/service/TagDataListService.php` - 数据列表服务 - -**仓储**： -- `app/repository/TagDataListRepository.php` - 数据列表数据访问 - -**修改的文件**： -- `app/service/TagTaskExecutor.php` - 标签任务执行器（核心逻辑） -- `app/service/TagService.php` - 标签计算服务 - -### 4.2 TagDataListController 接口 - -```php -taskRepository->find($taskId); - $targetTagIds = $task->target_tag_ids; - - // 遍历每个标签定义 - foreach ($targetTagIds as $tagId) { - $tagDef = $this->tagDefinitionRepository->find($tagId); - $ruleConfig = $tagDef->rule_config; - - // ⭐ 获取数据列表配置 - $dataListId = $ruleConfig['data_list_id']; - $dataList = $this->tagDataListRepository->find($dataListId); - - if (!$dataList) { - // 记录错误并跳过 - continue; - } - - // ⭐ 执行动态查询 - $results = $this->executeDynamicQuery($dataList); - - // ⭐ 遍历查询结果，动态打标签 - foreach ($results as $record) { - $userId = $this->extractUserId($record); - - if (!$userId) { - continue; - } - - // 根据规则条件计算标签值 - $tagValue = $this->calculateTagValue($record, $ruleConfig); - - if ($tagValue !== null) { - // 保存标签 - $this->saveUserTag($userId, $tagId, $tagValue); - } - - // 更新进度 - $this->updateProgress($taskId); - } - } - } - - /** - * 执行动态查询 - */ - private function executeDynamicQuery(TagDataList $dataList): array - { - $queryConfig = $dataList->query_config; - - // 1. 连接数据源 - $adapter = $this->getDataSourceAdapter($dataList->data_source_id); - - // 2. 构建MongoDB聚合管道 - $pipeline = []; - - // 2.1 过滤条件（$match） - if (!empty($queryConfig['filter'])) { - $filter = $this->buildFilter($queryConfig['filter']); - $pipeline[] = ['$match' => $filter]; - } - - // 2.2 联表查询（$lookup） - if (!empty($queryConfig['lookups'])) { - foreach ($queryConfig['lookups'] as $lookup) { - $pipeline[] = [ - '$lookup' => [ - 'from' => $lookup['from'], - 'localField' => $lookup['local_field'], - 'foreignField' => $lookup['foreign_field'], - 'as' => $lookup['as'] - ] - ]; - - // 解构 - if ($lookup['unwrap'] ?? false) { - $pipeline[] = [ - '$unwind' => [ - 'path' => '$' . $lookup['as'], - 'preserveNullAndEmptyArrays' => $lookup['preserve_null'] ?? true - ] - ]; - } - } - } - - // 2.3 排序（$sort） - if (!empty($queryConfig['sort'])) { - $pipeline[] = ['$sort' => $queryConfig['sort']]; - } - - // 2.4 限制（$limit） - if (!empty($queryConfig['limit'])) { - $pipeline[] = ['$limit' => $queryConfig['limit']]; - } - - // 3. 执行查询 - $collection = $adapter->getCollection($dataList->database, $dataList->collection); - $cursor = $collection->aggregate($pipeline); - - return iterator_to_array($cursor); - } - - /** - * 构建过滤条件 - */ - private function buildFilter(array $filterConditions): array - { - $filter = []; - - foreach ($filterConditions as $condition) { - $field = $condition['field']; - $operator = $condition['operator']; - $value = $condition['value']; - - // 转换为MongoDB运算符 - $mongoFilter = match($operator) { - 'eq' => ['$eq' => $value], - 'ne' => ['$ne' => $value], - 'gt' => ['$gt' => $value], - 'gte' => ['$gte' => $value], - 'lt' => ['$lt' => $value], - 'lte' => ['$lte' => $value], - 'in' => ['$in' => (array)$value], - 'nin' => ['$nin' => (array)$value], - 'regex' => ['$regex' => $value, '$options' => 'i'], - 'exists' => ['$exists' => $value], - default => $value - }; - - $filter[$field] = $mongoFilter; - } - - return $filter; - } - - /** - * 计算标签值 - */ - private function calculateTagValue(array $record, array $ruleConfig): ?string - { - $conditions = $ruleConfig['conditions']; - - foreach ($conditions as $condition) { - $fieldName = $condition['field_name']; - $operator = $condition['operator']; - $expectedValue = $condition['value']; - $tagValue = $condition['tag_value']; - - // 获取字段值（支持嵌套字段，如：user_info.mobile） - $actualValue = $this->getFieldValue($record, $fieldName); - - // 判断条件是否满足 - $match = $this->evaluateCondition($actualValue, $operator, $expectedValue); - - // 满足第一个条件即返回 - if ($match) { - return $tagValue; - } - } - - // 所有条件都不满足 - return null; - } - - /** - * 评估条件 - */ - private function evaluateCondition($actualValue, string $operator, $expectedValue): bool - { - // 运算规则 - if (in_array($operator, ['>', '>=', '<', '<=', '=', '!=', 'in', 'not_in'])) { - return match($operator) { - '>' => $actualValue > $expectedValue, - '>=' => $actualValue >= $expectedValue, - '<' => $actualValue < $expectedValue, - '<=' => $actualValue <= $expectedValue, - '=' => $actualValue == $expectedValue, - '!=' => $actualValue != $expectedValue, - 'in' => in_array($actualValue, (array)$expectedValue), - 'not_in' => !in_array($actualValue, (array)$expectedValue), - default => false - }; - } - - // 正则规则（operator是正则表达式字符串） - if (str_starts_with($operator, '/')) { - $pattern = $operator; - return preg_match($pattern, (string)$actualValue) ? true : false; - } - - return false; - } - - /** - * 提取用户ID - */ - private function extractUserId(array $record): ?string - { - // 优先从record中获取user_id - if (isset($record['user_id'])) { - return (string)$record['user_id']; - } - - // 如果有联表的user_info，从user_info中获取 - if (isset($record['user_info']['user_id'])) { - return (string)$record['user_info']['user_id']; - } - - return null; - } - - /** - * 获取字段值（支持嵌套字段） - */ - private function getFieldValue(array $record, string $fieldName) - { - // 支持嵌套字段，如：user_info.mobile - if (str_contains($fieldName, '.')) { - $parts = explode('.', $fieldName); - $value = $record; - foreach ($parts as $part) { - if (!isset($value[$part])) { - return null; - } - $value = $value[$part]; - } - return $value; - } - - return $record[$fieldName] ?? null; - } -} -``` - ---- - -## 五、API接口实现 - -### 5.1 数据列表管理接口 - -``` -GET /api/tag-data-lists - 获取数据列表列表 -POST /api/tag-data-lists - 创建数据列表 -GET /api/tag-data-lists/{list_id} - 获取数据列表详情 -PUT /api/tag-data-lists/{list_id} - 更新数据列表 -DELETE /api/tag-data-lists/{list_id} - 删除数据列表 -GET /api/tag-data-lists/{list_id}/fields - 获取数据列表字段 -``` - -### 5.2 辅助接口 - -``` -GET /api/data-sources/{id}/databases - 获取数据库列表 -GET /api/data-sources/{id}/collections - 获取集合列表 -GET /api/data-sources/{id}/fields - 获取字段列表 -POST /api/data-sources/preview-query - 预览查询结果 -``` - ---- - -## 六、使用场景示例 - -### 场景1：根据消费金额分级 - -**步骤**： -1. 创建数据列表"消费记录表" - - 主集合：`consumption_records` - - 过滤：`status = success` - - 联表：关联 `user_profile` - -2. 创建标签定义"消费等级" - - 选择数据列表：消费记录表 - - 添加条件： - - `amount < 3000` → `"低价值用户"` - - `amount >= 3000` → `"中等价值用户"` - - `amount > 9000` → `"高价值用户"` - -3. 创建标签任务 - - 选择标签：消费等级 - - 启动任务 - -4. 任务执行 - - 查询消费记录（状态=成功，关联用户信息） - - 遍历记录，根据金额判断等级 - - 保存标签到 user_tags - -### 场景2：识别淘宝用户 - -**步骤**： -1. 创建数据列表"消费记录表" - - 主集合：`consumption_records` - - 过滤：无 - - 联表：无 - -2. 创建标签定义"平台识别" - - 选择数据列表：消费记录表 - - 规则类型：正则规则 - - 添加条件： - - 字段：`shop_name` - - 运算符：`/淘宝/` - - 值：`匹配` - - 标签值：`"淘宝平台"` - -3. 启动任务，自动识别淘宝用户 - ---- - -## 七、优势 - -1. **配置化**：SQL/查询配置保存在数据表，无需修改代码 -2. **可复用**：同一个数据列表可被多个标签定义使用 -3. **灵活性**：支持复杂查询，包括联表、过滤、排序 -4. **可视化**：通过UI界面配置，无需手写SQL -5. **动态执行**：任务执行时动态读取配置并执行 - ---- - -## 八、注意事项 - -1. **性能优化**： - - 设置合理的 limit 值 - - 建立合适的索引 - - 避免查询过多数据 - -2. **错误处理**： - - 数据列表配置不存在时的处理 - - 查询执行失败时的处理 - - 字段不存在时的默认值处理 - -3. **数据一致性**： - - user_id 必须存在 - - 字段名必须与实际字段匹配 - - 联表字段必须正确 - ---- - -**文档更新时间**：2025-01-XX -**实现方案**：基于数据列表动态打标签 diff --git a/Moncter/Moncter/提示词/数据列表管理_真实API接入说明.md b/Moncter/Moncter/提示词/数据列表管理_真实API接入说明.md deleted file mode 100644 index 5f52eb39..00000000 --- a/Moncter/Moncter/提示词/数据列表管理_真实API接入说明.md +++ /dev/null @@ -1,417 +0,0 @@ -# 数据列表管理 - 真实API接入说明 - -## 修改概览 - -已将 `QueryBuilder` 组件中的基础配置部分从 Mock 数据切换到真实 API，复用了数据采集页面的 API。 - -## 修改的文件 - -### TaskShow/src/components/QueryBuilder/QueryBuilder.vue - -**主要修改**： -1. 导入数据采集API模块 -2. 替换所有基础配置相关的 Mock 数据为真实 API 调用 -3. 添加数据库和集合对象的缓存机制 - -**修改详情**： - -#### 1. 导入 API 模块 - -```typescript -import * as dataCollectionApi from '@/api/dataCollection' -``` - -#### 2. 添加对象缓存变量 - -```typescript -// 保存完整的对象用于API调用 -const databaseObjects = ref([]) -const collectionObjects = ref([]) -``` - -**用途**：因为 API 可能返回对象格式（如 `{ id, name }`），需要缓存这些对象用于后续 API 调用。 - -#### 3. 真实 API 调用 - -##### 加载数据源 - -**API路径**：`GET /data-collection-tasks/data-sources` - -```typescript -const loadDataSources = async () => { - try { - dataSourceLoading.value = true - const response = await dataCollectionApi.getDataSources() - - // 转换数据格式，兼容旧的data_source_id字段 - dataSources.value = (response.data || []).map((ds: any) => ({ - data_source_id: ds.id || ds.data_source_id, - name: ds.name || ds.id, - type: ds.type, - status: 1 - })) - } catch (error: any) { - ElMessage.error(error.message || '加载数据源失败') - } finally { - dataSourceLoading.value = false - } -} -``` - -##### 加载数据库列表 - -**API路径**：`GET /data-collection-tasks/data-sources/{dataSourceId}/databases` - -```typescript -const loadDatabases = async (dataSourceId: string) => { - try { - databaseLoading.value = true - const response = await dataCollectionApi.getDatabases(dataSourceId) - - // 转换为字符串数组（只返回数据库名称） - databases.value = (response.data || []).map((db: any) => - typeof db === 'string' ? db : db.name - ) - - // 保存完整的数据库对象供后续使用 - databaseObjects.value = response.data || [] - } catch (error: any) { - ElMessage.error(error.message || '加载数据库列表失败') - } finally { - databaseLoading.value = false - } -} -``` - -##### 加载集合列表 - -**API路径**：`GET /data-collection-tasks/data-sources/{dataSourceId}/databases/{databaseId}/collections` - -```typescript -const loadCollections = async (dataSourceId: string, database: string) => { - try { - collectionLoading.value = true - - // 查找数据库对象 - const dbObj = databaseObjects.value.find((db: any) => - (typeof db === 'object' && db.name === database) || db === database - ) - - const response = await dataCollectionApi.getCollections( - dataSourceId, - dbObj || database - ) - - // 转换为字符串数组（只返回集合名称） - collections.value = (response.data || []).map((coll: any) => - typeof coll === 'string' ? coll : coll.name - ) - - // 保存完整的集合对象供后续使用 - collectionObjects.value = response.data || [] - } catch (error: any) { - ElMessage.error(error.message || '加载集合列表失败') - } finally { - collectionLoading.value = false - } -} -``` - -##### 加载字段列表 - -**API路径**：`GET /data-collection-tasks/data-sources/{dataSourceId}/databases/{databaseId}/collections/{collectionId}/fields` - -```typescript -const loadFields = async (dataSourceId: string, database: string, collection: string) => { - try { - // 查找数据库和集合对象 - const dbObj = databaseObjects.value.find((db: any) => - (typeof db === 'object' && db.name === database) || db === database - ) - const collObj = collectionObjects.value.find((coll: any) => - (typeof coll === 'object' && coll.name === collection) || coll === collection - ) - - const response = await dataCollectionApi.getFields( - dataSourceId, - dbObj || database, - collObj || collection - ) - - // 转换字段格式：API返回的是 { name, type }，需要转换为 { field, field_name, type } - availableFields.value = (response.data || []).map((field: any) => ({ - field: field.name, // 显示名称使用字段名 - field_name: field.name, // 实际字段名 - type: field.type || 'string' - })) - } catch (error: any) { - ElMessage.error(error.message || '加载字段列表失败') - availableFields.value = [] - } -} -``` - -##### 预览查询数据 - -**API路径**：`POST /data-collection-tasks/preview-query` - -```typescript -const handlePreview = async () => { - if (!canPreview.value) { - ElMessage.warning('请先完成基础配置') - return - } - - try { - // 构建查询预览请求 - const lookups = queryConfig.lookups.map((lookup: any) => ({ - from: lookup.foreign_collection, - localField: lookup.local_field, - foreignField: lookup.foreign_field, - as: lookup.as, - unwind: lookup.unwind, - preserveNullAndEmptyArrays: lookup.preserve_null - })) - - const filterConditions = queryConfig.filter.map((filter: any) => ({ - field: filter.field_name, - operator: filter.operator, - value: filter.value, - logic: filter.logic || 'and' - })) - - const response = await dataCollectionApi.previewQuery({ - data_source_id: queryConfig.data_source_id, - database: queryConfig.database, - collection: queryConfig.collection, - lookups: lookups.length > 0 ? lookups : undefined, - filter_conditions: filterConditions.length > 0 ? filterConditions : undefined, - limit: queryConfig.limit || 10 - }) - - previewData.value = response.data?.data || [] - ElMessage.success(`预览成功，共 ${previewData.value.length} 条数据`) - } catch (error: any) { - ElMessage.error(error.message || '预览失败') - } -} -``` - -#### 4. 变化处理函数优化 - -添加了对缓存对象的清理： - -```typescript -const handleDataSourceChange = async (dataSourceId: string) => { - if (!dataSourceId) { - databases.value = [] - collections.value = [] - availableFields.value = [] - databaseObjects.value = [] // 新增 - collectionObjects.value = [] // 新增 - queryConfig.database = '' - queryConfig.collection = '' - return - } - await loadDatabases(dataSourceId) -} - -const handleDatabaseChange = async (database: string) => { - if (!database || !queryConfig.data_source_id) { - collections.value = [] - availableFields.value = [] - collectionObjects.value = [] // 新增 - queryConfig.collection = '' - return - } - await loadCollections(queryConfig.data_source_id, database) -} -``` - -## 复用的 API 模块 - -### 文件位置 -`TaskShow/src/api/dataCollection.ts` - -### API 列表 - -| API 函数 | HTTP 方法 | 路径 | 用途 | -|---------|----------|------|------| -| `getDataSources()` | GET | `/data-collection-tasks/data-sources` | 获取数据源列表 | -| `getDatabases(dataSourceId)` | GET | `/data-collection-tasks/data-sources/{id}/databases` | 获取数据库列表 | -| `getCollections(dataSourceId, database)` | GET | `/data-collection-tasks/data-sources/{id}/databases/{dbId}/collections` | 获取集合列表 | -| `getFields(dataSourceId, database, collection)` | GET | `/data-collection-tasks/data-sources/{id}/databases/{dbId}/collections/{collId}/fields` | 获取字段列表 | -| `previewQuery(data)` | POST | `/data-collection-tasks/preview-query` | 预览查询结果 | - -## 数据格式说明 - -### 数据源（Data Source） -```typescript -{ - id: string - name: string - type: 'mongodb' | 'mysql' | ... -} -``` - -### 数据库（Database） -```typescript -{ - id: string - name: string -} -// 或简单的字符串数组 -string[] -``` - -### 集合（Collection） -```typescript -{ - id: string - name: string -} -// 或简单的字符串数组 -string[] -``` - -### 字段（Field） -```typescript -{ - name: string - type: 'string' | 'number' | 'datetime' | ... -} -``` - -### 预览查询请求（Preview Query Request） -```typescript -{ - data_source_id: string - database: string - collection: string - lookups?: Array<{ - from: string - localField: string - foreignField: string - as: string - unwind?: boolean - preserveNullAndEmptyArrays?: boolean - }> - filter_conditions?: Array<{ - field: string - operator: string - value: any - logic: 'and' | 'or' - }> - limit?: number -} -``` - -### 预览查询响应（Preview Query Response） -```typescript -{ - fields: Array<{ name: string; type: string }> - data: Array - count: number -} -``` - -## 兼容性处理 - -### 1. 数据库/集合对象 vs 字符串 -API 可能返回对象（`{ id, name }`）或字符串数组，代码中做了兼容处理： - -```typescript -// 显示用字符串数组 -databases.value = (response.data || []).map((db: any) => - typeof db === 'string' ? db : db.name -) - -// 缓存完整对象 -databaseObjects.value = response.data || [] - -// 查找时兼容两种格式 -const dbObj = databaseObjects.value.find((db: any) => - (typeof db === 'object' && db.name === database) || db === database -) -``` - -### 2. 字段名转换 -API 返回的字段格式与 QueryBuilder 使用的格式不同： - -```typescript -// API 返回: { name, type } -// QueryBuilder 需要: { field, field_name, type } - -availableFields.value = (response.data || []).map((field: any) => ({ - field: field.name, // 显示名称 - field_name: field.name, // 实际字段名 - type: field.type || 'string' -})) -``` - -## 测试建议 - -1. **数据源加载测试** - - 打开数据列表配置页面 - - 检查数据源下拉框是否正确加载 - -2. **级联加载测试** - - 选择数据源 → 检查数据库列表 - - 选择数据库 → 检查集合列表 - - 选择集合 → 检查字段列表 - -3. **预览功能测试** - - 配置完整查询（包含过滤、联表） - - 点击"预览数据" - - 检查返回结果是否正确 - -4. **错误处理测试** - - 模拟网络错误 - - 检查错误提示是否友好 - -## 注意事项 - -1. **后端 API 必须已实现** - - 所有使用的 API 端点都必须在后端正确实现 - - 数据格式必须与前端期望一致 - -2. **性能优化** - - 字段列表加载可能较慢（特别是大集合） - - 建议后端添加缓存机制 - -3. **错误处理** - - 所有 API 调用都有 try-catch 包裹 - - 错误信息会通过 ElMessage 显示给用户 - -4. **数据转换** - - 注意 API 返回的数据格式 - - 确保前端正确转换为组件所需格式 - -## 剩余 Mock 数据 - -以下部分仍使用 Mock 数据（可后续接入真实 API）： - -1. **TagDataList/List.vue** - - 数据列表的 CRUD 操作 - - API 端点：`/api/tag-data-lists` - -2. **TagDataList/Form.vue** - - 数据列表配置的保存 - - API 端点：`/api/tag-data-lists` - -3. **TagDefinition/Form.vue** - - 数据列表下拉选择 - - 字段列表加载 - - API 端点：`/api/tag-data-lists` 和 `/api/tag-data-lists/{id}/fields` - -## 下一步工作 - -1. 开发数据列表管理的后端 API -2. 将 TagDataList 的 CRUD 操作接入真实 API -3. 将 TagDefinition 中的数据列表选择接入真实 API -4. 完整的集成测试 - ---- - -**更新时间**：2025-01-XX -**状态**：QueryBuilder 基础配置已接入真实 API diff --git a/Moncter/Moncter/提示词/数据列表管理界面使用说明.md b/Moncter/Moncter/提示词/数据列表管理界面使用说明.md deleted file mode 100644 index 45c8e067..00000000 --- a/Moncter/Moncter/提示词/数据列表管理界面使用说明.md +++ /dev/null @@ -1,327 +0,0 @@ -# 数据列表管理界面使用说明 - -## 一、界面概览 - -数据列表管理界面用于可视化配置MongoDB查询，支持过滤条件、联表查询、排序等功能。配置保存后，可在标签定义中引用。 - -### 访问路径 -- 列表页面：http://localhost:5173/tag-data-lists -- 创建页面：http://localhost:5173/tag-data-lists/create -- 编辑页面：http://localhost:5173/tag-data-lists/:id/edit - -### 菜单位置 -左侧菜单 → 标签任务 → 数据列表管理 - ---- - -## 二、界面功能 - -### 2.1 数据列表管理（List.vue） - -**功能列表**： -- ✅ 展示所有数据列表配置 -- ✅ 按名称搜索 -- ✅ 按状态筛选 -- ✅ 创建新数据列表 -- ✅ 编辑数据列表 -- ✅ 删除数据列表 -- ✅ 分页展示 - -**表格列**： -- 列表名称 -- 列表编码 -- 数据源ID -- 数据库 -- 主集合 -- 状态（启用/禁用） -- 描述 -- 创建时间 -- 操作（编辑、删除） - -### 2.2 数据列表配置（Form.vue） - -**基本信息配置**： -- 列表名称（必填）- 如："消费记录表" -- 列表编码（必填）- 如："consumption_records" -- 描述（可选） -- 状态（启用/禁用） - -**查询配置（QueryBuilder组件）**： - -#### 1. 基础配置 -- 数据源选择 -- 数据库选择 -- 主集合选择 - -#### 2. 过滤条件（WHERE） -- 支持多个条件 -- 逻辑关系：AND/OR -- 字段选择（自动加载） -- 运算符选择： - - 等于、不等于 - - 大于、大于等于、小于、小于等于 - - 包含、不包含 - - 模糊匹配、存在 -- 值输入（根据字段类型自动调整） - -#### 3. 联表查询（JOIN/LOOKUP） -- 支持多个联表 -- 关联集合选择 -- 主集合字段选择 -- 关联集合字段输入 -- 结果字段名配置 -- 解构开关（是否展开数组） -- 保留空值开关（LEFT JOIN效果） - -#### 4. 排序和限制 -- 排序字段选择 -- 排序方式（升序/降序） -- 限制数量（默认1000） - -#### 5. 查询预览 -- 实时显示生成的MongoDB聚合管道代码 -- 预览查询结果数据（最多显示配置的limit条） - ---- - -## 三、使用示例 - -### 示例1：创建简单的消费记录查询 - -**步骤**： -1. 访问 `/tag-data-lists`，点击"创建数据列表" -2. 填写基本信息： - - 列表名称：`消费记录表` - - 列表编码：`consumption_records` - - 描述：`用于标签定义的消费记录数据` - - 状态：`启用` -3. 配置查询： - - 数据源：选择"MongoDB标签引擎数据源" - - 数据库：选择"tag_engine" - - 主集合：选择"consumption_records" -4. 添加过滤条件： - - 字段：`交易状态` - - 运算符：`等于` - - 值：`success` -5. 配置排序： - - 排序字段：`创建时间` - - 排序方式：`降序` - - 限制数量：`1000` -6. 点击"预览数据"查看效果 -7. 点击"保存" - -**生成的查询**： -```javascript -db.consumption_records.aggregate([ - { $match: { status: { $eq: "success" } } }, - { $sort: { create_time: -1 } }, - { $limit: 1000 } -]) -``` - -### 示例2：创建带联表的查询 - -**步骤**： -1. 基本配置同上 -2. 添加过滤条件： - - 字段：`交易金额` - - 运算符：`大于` - - 值：`1000` -3. 添加联表查询： - - 关联集合：`user_profile` - - 主集合字段：`user_id` - - 关联集合字段：`user_id` - - 结果字段名：`user_info` - - 解构：`是` - - 保留空值：`是` -4. 保存 - -**生成的查询**： -```javascript -db.consumption_records.aggregate([ - { $match: { amount: { $gt: 1000 } } }, - { $lookup: { - from: "user_profile", - localField: "user_id", - foreignField: "user_id", - as: "user_info" - } }, - { $unwind: { - path: "$user_info", - preserveNullAndEmptyArrays: true - } }, - { $limit: 1000 } -]) -``` - -### 示例3：在标签定义中使用 - -**步骤**： -1. 访问 `/tag-definitions/create` -2. 填写标签编码和名称 -3. 选择数据列表：`消费记录表` -4. 系统自动加载字段：`交易金额`、`店铺名称`、`交易状态`等 -5. 添加规则条件： - - 字段：`交易金额` - - 运算符：`<` - - 值：`3000` - - 标签值：`低价值用户` -6. 保存 - ---- - -## 四、Mock数据位置 - -所有Mock数据都标记为 `TODO: 替换为真实API`，方便后续替换。 - -### 4.1 List.vue - -**位置**：`loadData()` 函数 - -```typescript -// 第106-142行 -// Mock数据 -const mockData = [ - { - list_id: 'list_001', - list_code: 'consumption_records', - list_name: '消费记录表', - // ... - }, - // ... -] -``` - -### 4.2 QueryBuilder.vue - -**位置1**：`loadDataSources()` - 第368-383行 -```typescript -dataSources.value = [ - { data_source_id: 'source_001', name: 'MongoDB标签引擎数据源' } -] -``` - -**位置2**：`loadDatabases()` - 第398-410行 -```typescript -databases.value = ['tag_engine', 'business_db', 'analytics_db'] -``` - -**位置3**：`loadCollections()` - 第425-443行 -```typescript -if (database === 'tag_engine') { - collections.value = ['consumption_records', 'user_profile', ...] -} -``` - -**位置4**：`loadFields()` - 第458-492行 -```typescript -if (collection === 'consumption_records') { - availableFields.value = [ - { field: '交易金额', field_name: 'amount', type: 'number' }, - // ... - ] -} -``` - -**位置5**：`handlePreview()` - 第530-569行 -```typescript -previewData.value = [ - { user_id: 'user_001', amount: 5000, ... }, - // ... -] -``` - -### 4.3 TagDefinition/Form.vue - -**位置1**：`loadDataLists()` - 第285-303行 -```typescript -dataLists.value = [ - { list_id: 'list_001', list_name: '消费记录表' }, - { list_id: 'list_002', list_name: '用户档案表' } -] -``` - -**位置2**：`loadFields()` - 第318-347行 -```typescript -if (listId === 'list_001') { - fields.value = [ - { field: '交易金额', field_name: 'amount', ... }, - // ... - ] -} -``` - ---- - -## 五、后续开发计划 - -### 阶段1：完善界面（✅ 已完成） -- ✅ QueryBuilder可视化查询构建器 -- ✅ 数据列表管理（列表、创建、编辑） -- ✅ 标签定义表单（集成数据列表选择） -- ✅ 所有Mock数据，界面可独立运行 - -### 阶段2：后端API开发（待开发） -- ⏳ TagDataListController - 数据列表管理API -- ⏳ TagDataListRepository - 数据访问层 -- ⏳ TagDataListService - 业务逻辑层 -- ⏳ 数据源相关API（databases、collections、fields） -- ⏳ 查询预览API - -### 阶段3：集成联调（待开发） -- ⏳ 删除Mock代码 -- ⏳ 集成真实API -- ⏳ 测试完整流程 -- ⏳ 错误处理优化 - -### 阶段4：标签任务执行（待开发） -- ⏳ 修改TagTaskExecutor -- ⏳ 从tag_data_lists读取配置 -- ⏳ 执行动态查询 -- ⏳ 遍历结果打标签 - ---- - -## 六、界面预览 - -### 数据列表管理列表 -![列表页面](略) -- 显示所有配置的数据列表 -- 支持搜索和筛选 -- 操作按钮：编辑、删除 - -### 数据列表配置表单 -![配置表单](略) -- 基本信息区域 -- 可视化查询构建器 -- 实时查询预览 -- 数据预览 - -### 标签定义表单 -![标签定义](略) -- 选择数据列表 -- 自动加载字段 -- 配置规则条件 - ---- - -## 七、技术要点 - -### 7.1 组件通信 -- QueryBuilder使用 `v-model` 双向绑定 -- 父组件监听配置变化 - -### 7.2 字段映射 -- 前端显示：中文字段名（field） -- 后端存储：英文字段名（field_name） -- 保存时同时存储两者 - -### 7.3 查询构建 -- 实时生成MongoDB聚合管道 -- 支持复杂的嵌套查询 -- 可视化预览查询结果 - ---- - -**文档更新时间**：2025-01-XX -**状态**：界面已完成，使用Mock数据 diff --git a/Moncter/Moncter/提示词/最新架构逻辑.md b/Moncter/Moncter/提示词/最新架构逻辑.md deleted file mode 100644 index 51ac630a..00000000 --- a/Moncter/Moncter/提示词/最新架构逻辑.md +++ /dev/null @@ -1,454 +0,0 @@ -# Moncter 系统最新架构逻辑提示词 - -> **用途说明**：本文档作为系统架构的核心记忆点，用于快速理解和开发维护 Moncter 系统。 - ---- - -## 一、系统定位 - -**Moncter** 是一个基于 **Webman (Workerman)** 框架的**用户标签引擎和数据采集中心**，核心功能包括： - -1. **多数据源数据采集**：支持 MongoDB、MySQL 等多种数据源，支持批量采集和实时监听两种模式 -2. **用户标签计算引擎**：基于用户消费数据实时计算和更新标签，支持规则引擎配置 -3. **用户身份管理**：支持身份证、手机号等标识的统一管理，处理临时用户、正式用户转换和身份合并 -4. **数据库实时同步**：使用 MongoDB Change Streams 实现数据库间实时同步 - ---- - -## 二、技术栈 - -### 后端 -- **框架**：Webman (Workerman) - 高性能 PHP 框架，多进程架构 -- **PHP版本**：>= 8.1 -- **数据库**：MongoDB (主数据库) -- **消息队列**：RabbitMQ - 异步任务处理 -- **缓存/锁**：Redis - 分布式锁、任务状态存储 - -### 前端 -- **框架**：Vue 3 + TypeScript -- **UI组件**：Element Plus -- **状态管理**：Pinia -- **构建工具**：Vite - ---- - -## 三、系统架构 - -### 3.1 分层架构 - -``` -┌─────────────────────────────────────────┐ -│ 应用层 (HTTP API) │ -│ Controller 层：处理HTTP请求 │ -└──────────────────┬──────────────────────┘ - │ -┌──────────────────▼──────────────────────┐ -│ 业务服务层 (Service) │ -│ - ConsumptionService: 消费记录处理 │ -│ - IdentifierService: 身份解析 │ -│ - TagService: 标签计算 │ -│ - DataCollectionTaskService: 任务管理 │ -└──────────────────┬──────────────────────┘ - │ -┌──────────────────▼──────────────────────┐ -│ 数据访问层 (Repository) │ -│ - UserProfileRepository │ -│ - UserPhoneRelationRepository │ -│ - ConsumptionRecordRepository │ -└──────────────────┬──────────────────────┘ - │ -┌──────────────────▼──────────────────────┐ -│ 数据存储层 │ -│ MongoDB / Redis / RabbitMQ │ -└─────────────────────────────────────────┘ -``` - -### 3.2 进程架构（Workerman） - -系统包含以下进程（配置在 `config/process.php`）： - -1. **webman (HTTP Server)** - - 处理 HTTP API 请求 - - 进程数：`CPU核心数 × 4` - - 监听端口：`8787` - -2. **monitor (文件监控)** - - 监控文件变化，自动重载 - - 单进程 - -3. **data_sync_scheduler (数据采集任务调度器)** - - 读取任务配置（配置文件 + 数据库） - - 启动和管理所有数据采集任务 - - 进程数：`10` - -4. **data_sync_worker (数据同步Worker)** - - 消费 RabbitMQ `data_sync` 队列 - - 写入目标数据库，更新用户统计 - - 进程数：`20` - -5. **tag_calculation_worker (标签计算Worker)** - - 消费 RabbitMQ `tag_calculation` 队列 - - 根据用户数据计算标签值 - - 进程数：`2` - ---- - -## 四、核心业务逻辑 - -### 4.1 消费记录到用户创建流程 - -**核心流程：消费记录 → 身份解析 → 用户创建/关联 → 手机关联建立 → 写入记录** - -#### 关键步骤 - -1. **身份解析** (`IdentifierService::resolvePersonId`) - - 优先级：身份证 > 手机号 - - 如果只有手机号，通过 `UserPhoneService::findUserByPhone` 查询 - - **关键**：使用 `consume_time` 作为查询时间点（不是当前时间） - -2. **临时用户创建** (`IdentifierService::createTemporaryPerson`) - - 生成 UUID 作为 `user_id` - - 设置 `is_temporary = true` - - **必须**建立手机关联（使用消费时间作为 `effective_time`） - -3. **手机关联建立** (`UserPhoneService::addPhoneToUser`) - - 冲突检测：检查手机号在 `effective_time` 是否已有有效关联 - - 自动过期旧关联：如果冲突，将旧关联的 `expire_time` 设置为新关联的 `effective_time` - - 创建新关联：`effective_time = consume_time`，`expire_time = null` - -4. **用户合并** (`ConsumptionService::handleMergeIfNeeded`) - - 当手机号和身份证同时出现，且关联到不同用户时触发 - - 临时用户 → 自动合并到正式用户 - - 正式用户冲突 → 以身份证为准（代订场景），记录日志 - -5. **写入消费记录** (`ConsumptionService::createRecord`) - - 写入 `consumption_records` 表（按时间分表：`consumption_records_YYYYMM`） - - 更新 `user_profile` 统计信息（`total_amount`、`total_count`、`last_consume_time`） - - 触发标签计算（推送到 RabbitMQ 队列） - -#### 关键设计要点 - -- **时间点精确匹配**：所有手机号查询必须基于 `consume_time`，支持历史数据导入 -- **时间窗口管理**：手机关联支持 `effective_time` 和 `expire_time`，支持手机号回收 -- **临时用户机制**：处理只有手机号的情况，后续可转换为正式用户 -- **UUID策略**：所有用户使用 UUID 作为 `user_id`，身份证只作为字段存储 - -### 4.2 数据采集系统 - -#### 任务配置方式 - -1. **配置文件方式** (`config/data_collection_tasks.php`) - - 适用于系统级任务（如数据库实时同步） - - 需要版本控制 - -2. **数据库方式** (`data_collection_tasks` 集合) - - 通过前端界面动态创建和管理 - - 支持批量采集（batch）和实时监听（realtime）两种模式 - -#### Handler 类型 - -- **ConsumptionCollectionHandler**：消费记录采集，自动处理用户身份解析 -- **GenericCollectionHandler**：通用数据采集，支持字段映射、过滤条件、连表查询 -- **DatabaseSyncHandler**：数据库同步，支持全量同步和增量同步（Change Streams） - -#### 采集模式 - -- **批量采集（Batch）**：分页查询数据，支持定时调度（Cron） -- **实时监听（Realtime）**：使用 MongoDB Change Streams 持续监听变更 - -### 4.3 标签计算系统 - -#### 标签定义 - -存储在 `tag_definitions` 集合，包含： -- `tag_id`、`tag_code`、`tag_name` -- `rule_config`（规则配置，JSON格式） -- `update_frequency`（real_time/daily/weekly） - -#### 标签计算流程 - -``` -消费记录写入 - ↓ -更新用户统计信息 - ↓ -推送标签计算消息到 RabbitMQ - ↓ -TagCalculationWorker 消费消息 - ↓ -TagService::calculateTags() - ├─→ 获取用户数据 - ├─→ 获取标签定义列表 - ├─→ SimpleRuleEngine 计算标签值 - ├─→ 更新或创建 user_tags 记录 - └─→ 记录 tag_history 变更历史 -``` - ---- - -## 五、数据存储设计 - -### 5.1 MongoDB 核心集合 - -#### 用户相关 - -**user_profile**（用户画像） -- `user_id`：UUID（主键） -- `id_card_hash`：身份证哈希值 -- `id_card_encrypted`：加密的身份证号 -- `is_temporary`：是否为临时用户（true/false） -- `total_amount`：总消费金额 -- `total_count`：总消费次数 -- `last_consume_time`：最后消费时间 - -**user_phone_relations**（手机关联表） -- `phone_number`：手机号 -- `phone_hash`：手机号哈希值 -- `user_id`：关联的用户ID -- `effective_time`：生效时间 -- `expire_time`：过期时间（null 表示当前有效） -- `is_active`：是否有效 - -**consumption_records**（消费记录） -- 按时间分表：`consumption_records_YYYYMM` -- `record_id`：记录ID（UUID） -- `user_id`：用户ID -- `consume_time`：消费时间 -- `amount`：消费金额 -- `actual_amount`：实际金额 - -#### 标签相关 - -**tag_definitions**（标签定义） -- `tag_id`、`tag_code`、`tag_name` -- `rule_config`：规则配置（JSON） -- `update_frequency`：更新频率 - -**user_tags**（用户标签） -- `user_id`、`tag_id` -- `tag_value`：标签值 -- `confidence`：置信度 - -**tag_history**（标签变更历史） -- `user_id`、`tag_id` -- `old_value`、`new_value` -- `change_time`：变更时间 - -#### 任务相关 - -**data_collection_tasks**（数据采集任务） -- `task_id`：任务ID（UUID） -- `mode`：采集模式（batch/realtime） -- `field_mappings`：字段映射配置 -- `filter_conditions`：过滤条件 -- `progress`：任务进度 -- `status`：任务状态（pending/running/paused/stopped） - -**data_sources**（数据源） -- `data_source_id`：数据源ID -- `type`：数据源类型（mongodb/mysql） -- `host`、`port`、`database` - -### 5.2 Redis 存储 - -- **分布式锁**：`lock:data_collection:{task_id}` -- **任务状态标志**：`data_collection_task:{task_id}:start/pause/stop` -- **同步状态**：`data_collection:{task_id}:last_sync_time` - -### 5.3 RabbitMQ 队列 - -- **data_sync**：数据同步队列 -- **tag_calculation**：标签计算队列 - ---- - -## 六、设计模式 - -### 6.1 Handler 模式 -- `ConsumptionCollectionHandler`：消费记录采集 -- `GenericCollectionHandler`：通用数据采集 -- `DatabaseSyncHandler`：数据库同步 - -### 6.2 适配器模式 -- `DataSourceAdapterInterface`：数据源适配器接口 -- `MongoDBAdapter`：MongoDB适配器 -- `MySQLAdapter`：MySQL适配器 - -### 6.3 仓库模式（Repository） -- 所有 Repository 类封装数据访问逻辑 -- 提供统一的查询接口 - -### 6.4 工厂模式 -- `DataSourceAdapterFactory`：创建数据源适配器 -- `PollingStrategyFactory`：创建轮询策略 - ---- - -## 七、关键代码位置 - -### 核心服务类 - -**ConsumptionService** (`app/service/ConsumptionService.php`) -- `createRecord()`：创建消费记录的主入口 -- `handleMergeIfNeeded()`：处理用户合并逻辑 - -**IdentifierService** (`app/service/IdentifierService.php`) -- `resolvePersonId()`：解析用户ID（统一入口） -- `resolvePersonIdByPhone()`：通过手机号解析 -- `resolvePersonIdByIdCard()`：通过身份证解析 -- `createTemporaryPerson()`：创建临时用户 -- `bindIdCardToPerson()`：绑定身份证 - -**UserPhoneService** (`app/service/UserPhoneService.php`) -- `addPhoneToUser()`：添加手机号关联（核心方法） -- `findUserByPhone()`：根据手机号查找用户（支持时间点查询） -- `removePhoneFromUser()`：移除手机号关联 - -**TagService** (`app/service/TagService.php`) -- `calculateTags()`：计算用户标签 - -**DataCollectionTaskService** (`app/service/DataCollectionTaskService.php`) -- 数据采集任务的创建、启动、暂停、停止等操作 - -### 进程类 - -- **DataSyncScheduler** (`app/process/DataSyncScheduler.php`)：数据采集任务调度器 -- **DataSyncWorker** (`app/process/DataSyncWorker.php`)：数据同步Worker -- **TagCalculationWorker** (`app/process/TagCalculationWorker.php`)：标签计算Worker - ---- - -## 八、API 接口体系 - -### 用户相关 -- `POST /api/users`：创建用户 -- `GET /api/users/{user_id}`：查询用户 -- `POST /api/users/search`：搜索用户 - -### 标签相关 -- `GET /api/users/{user_id}/tags`：查询用户标签 -- `PUT /api/users/{user_id}/tags`：计算/更新用户标签 -- `POST /api/tags/filter`：根据标签筛选用户 - -### 数据采集任务 -- `POST /api/data-collection-tasks`：创建任务 -- `POST /api/data-collection-tasks/{task_id}/start`：启动任务 -- `POST /api/data-collection-tasks/{task_id}/pause`：暂停任务 -- `POST /api/data-collection-tasks/{task_id}/stop`：停止任务 -- `GET /api/data-collection-tasks/{task_id}/progress`：获取任务进度 - -### 数据源 -- `GET /api/data-sources`：获取数据源列表 -- `POST /api/data-sources`：创建数据源 -- `POST /api/data-sources/test-connection`：测试连接 - ---- - -## 九、关键设计原则 - -### 9.1 时间点精确匹配 -- 所有手机号查询必须基于消费记录的实际时间点（`consume_time`） -- 支持历史数据导入和批量处理 -- 正确处理手机号回收后的重新分配 - -### 9.2 临时用户机制 -- 只有手机号时创建临时用户（`is_temporary=true`） -- 临时用户必须建立手机关联 -- 绑定身份证后自动转为正式用户 - -### 9.3 时间窗口管理 -- 手机关联支持 `effective_time` 和 `expire_time` -- 时间窗口必须连续，不能有间隙 -- 支持手机号在不同时间段属于不同用户 - -### 9.4 UUID 策略 -- 所有用户统一使用 UUID 作为 `user_id` -- 身份证只作为字段存储，不作为主键 -- 转为正式用户时，只更新身份证字段，不改变 `user_id` - -### 9.5 配置化设计 -- 任务配置支持配置文件和数据库两种方式 -- 数据源配置统一管理 -- 业务逻辑与配置分离 - ---- - -## 十、特殊场景处理 - -### 场景1：手机号回收 -- 一个手机号在不同时间段被不同用户使用 -- 通过时间窗口精确管理历史关联 -- 新关联建立时，自动将旧关联标记为过期 - -### 场景2：临时用户转正式用户 -- 第一次消费：只有手机号 → 创建临时用户 -- 第二次消费：手机号 + 身份证 → 临时用户转为正式用户 - -### 场景3：代订场景 -- 手机号和身份证关联到不同的正式用户 -- 策略：以身份证为准，消费记录归属到身份证用户 -- 手机号关联保持不变（可能是代订），记录日志 - -### 场景4：用户合并 -- 临时用户自动合并到正式用户 -- 合并内容：统计数据、标签、消费记录、手机关联 - ---- - -## 十一、数据流图 - -``` -数据源 (MongoDB/MySQL) - ↓ -数据采集任务 (DataSyncScheduler) - ├─→ ConsumptionCollectionHandler - │ ├─→ 批量采集：分页查询 - │ └─→ 实时监听：Change Streams - │ - └─→ DatabaseSyncHandler - ├─→ 全量同步：批量读取写入 - └─→ 增量同步：Change Streams - ↓ -消费记录写入 (ConsumptionService) - ├─→ 身份解析 (IdentifierService) - │ ├─→ 手机号 → user_id - │ └─→ 如果不存在，创建临时用户 - │ - ├─→ 写入 consumption_records - ├─→ 更新 user_profile 统计 - └─→ 触发标签计算（RabbitMQ） - ↓ -标签计算 (TagCalculationWorker) - ├─→ TagService::calculateTags() - ├─→ SimpleRuleEngine 计算 - ├─→ 更新 user_tags - └─→ 记录 tag_history -``` - ---- - -## 十二、配置文件位置 - -- **应用配置**：`config/app.php` -- **进程配置**：`config/process.php` -- **路由配置**：`config/route.php` -- **数据采集任务配置**：`config/data_collection_tasks.php` -- **数据源配置**：`config/data_sources.php` -- **数据库配置**：`config/database.php` - ---- - -## 十三、关键注意事项 - -1. **时间点查询**：所有手机号查询必须传入 `consume_time`，不能使用当前时间 -2. **临时用户关联**：临时用户创建后必须建立手机关联，不能跳过 -3. **时间窗口连续性**：手机关联的时间窗口必须连续，不能有间隙 -4. **UUID策略**：用户ID统一使用UUID，身份证只作为字段存储 -5. **合并时机**：仅在手机号和身份证号同时出现且关联到不同用户时触发合并 -6. **代订场景**：正式用户冲突时，以身份证为准，手机号关联保持不变 - ---- - -**文档版本**：v1.0 -**最后更新**：2025-01-28 -**维护说明**：本文档作为系统架构的核心记忆点，如有架构变更，请及时更新本文档 diff --git a/Moncter/Moncter/提示词/标签定义逻辑对比分析.md b/Moncter/Moncter/提示词/标签定义逻辑对比分析.md deleted file mode 100644 index e993317a..00000000 --- a/Moncter/Moncter/提示词/标签定义逻辑对比分析.md +++ /dev/null @@ -1,294 +0,0 @@ -# 标签定义逻辑文档对比分析 - -## 一、文档概述 - -- **`标签定逻辑.md`**：原始需求文档（66行）- 简洁版本 -- **`标签定逻辑真实.md`**：完善后的技术文档（579行）- 详细版本 - ---- - -## 二、核心含义一致性分析 - -### ✅ 2.1 一致的部分 - -#### 1. **后台预先工作** -- **原始版本**：提到需要预先处理数据列表API，通过SQL查询配置化数据集合 -- **详细版本**：详细说明了数据源类型和字段API的实现 -- **结论**：✅ **含义一致**，详细版本是对原始需求的扩展说明 - -#### 2. **规则配置概念** -- **原始版本**：提到运算规则和正则规则两种类型 -- **详细版本**：详细说明了simple规则和regex规则（扩展方案） -- **结论**：✅ **含义一致**，详细版本补充了实现细节 - -#### 3. **标签任务处理流程** -- **原始版本**：4个简单步骤 - ``` - 1、选择已定义的标签 - 2、选择后点击保存，状态为待启动 - 3、在列表启动的时候，后台先从后台读取定义标签的集合，并查询出来 - 4、根据定义标签的规则条件，直接根据用户表遍历过去进行打标签，然后将标签存储。 - ``` -- **详细版本**：详细的流程图和代码逻辑 -- **结论**：✅ **含义一致**，详细版本是对原始流程的细化 - -#### 4. **标签基本信息** -- **原始版本**：提到"标签编码、标签名称" -- **详细版本**：详细列出了所有字段（tag_code、tag_name、category、description等） -- **结论**：✅ **含义一致**，详细版本补充了完整字段列表 - ---- - -## 三、存在差异的部分 - -### ⚠️ 3.1 JSON配置格式差异 - -#### 原始版本的格式： -```json -[ - { - field:"交易金额", - operator:"<", - value:3000, - tag_value："低价值用户" // ❌ tag_value在条件中 - }, - { - field:"交易金额", - operator:">=", - value:3000, - tag_value："中等价值用户" // ❌ tag_value在条件中 - } -] -``` - -#### 详细版本的格式（符合实际代码）： -```json -{ - "rule_type": "simple", - "conditions": [ - { - "field": "total_amount", - "operator": "<", - "value": 3000 - // ✅ tag_value不在条件中 - } - ], - "tag_value": "低价值用户", // ✅ tag_value在rule_config中 - "confidence": 1.0 -} -``` - -**差异说明**： -- ❌ **原始版本**：每个条件都有独立的 `tag_value`，这不符合实际的数据结构 -- ✅ **详细版本**：`tag_value` 在 `rule_config` 中，所有条件满足时使用同一个 `tag_value` - -**实际代码验证**： -根据 `SimpleRuleEngine.php` 和 `TagService.php` 的代码，正确的格式应该是： -- `tag_value` 在 `rule_config` 中，不在 `conditions` 中 -- 多个条件之间是 AND 关系，所有条件满足时使用同一个 `tag_value` - -**建议**： -- 原始版本可能是业务需求的简化描述 -- 实际实现应该按照详细版本的格式 - ---- - -### ⚠️ 3.2 字段命名差异 - -#### 原始版本： -- 使用中文字段名：`"交易金额"`、`"店铺名称"`、`"交易状态"`、`"手机号"` - -#### 详细版本： -- 使用英文字段名：`"total_amount"`、`"shop_name"`、`"status"`、`"phone"` - -**差异说明**： -- 原始版本可能是面向业务人员的描述 -- 详细版本是面向开发人员的技术实现 -- 实际数据库中应该使用英文字段名 - -**建议**： -- 前端界面可以显示中文名称 -- 后端存储和计算使用英文字段名 -- 需要建立字段映射关系 - ---- - -### ⚠️ 3.3 "数据列表"概念差异 - -#### 原始版本： -``` --先用sql语句查询出符合条件的结果，这段数据库的查询操作可以配置化，来增减数据集合，这个结果可以命名，例如:消费记录表。 -``` - -#### 详细版本： -``` -系统支持以下数据源类型： -1. user_profile - 用户档案表 -2. consumption_records - 消费记录表 -3. user_phone_relations - 用户手机号关系表 -``` - -**差异说明**： -- 原始版本提到"SQL语句"，但MongoDB使用查询语句，不是SQL -- 原始版本提到"配置化查询"，可能是指数据源配置 -- 详细版本明确了数据源类型和字段定义 - -**建议**： -- 原始版本中的"数据列表"应该理解为"数据源" -- "SQL语句"应该理解为"MongoDB查询"或"数据源配置" -- 需要明确数据源配置的管理方式 - ---- - -### ⚠️ 3.4 正则规则实现差异 - -#### 原始版本： -```json -{ - field:"店铺名称", - operator:"/淘宝/", // ❌ 直接使用正则表达式作为operator - value:true, - tag_value："淘宝平台" -} -``` - -#### 详细版本： -```json -{ - "rule_type": "regex", // ✅ 使用rule_type区分 - "conditions": [ - { - "field": "shop_name", - "operator": "match", - "pattern": "/淘宝/", // ✅ pattern字段 - "value": true - } - ], - "tag_value": "淘宝平台" -} -``` - -**差异说明**： -- 原始版本：正则表达式直接作为 `operator` -- 详细版本：需要 `rule_type="regex"` 和 `pattern` 字段 -- **当前代码**：实际只支持 `simple` 规则，正则规则需要扩展实现 - -**建议**： -- 如果使用 `simple` 规则，可以用 `in` 运算符替代： - ```json - { - "field": "shop_name", - "operator": "in", - "value": ["淘宝", "天猫"] - } - ``` - ---- - -## 四、关键差异总结 - -| 对比项 | 原始版本 | 详细版本 | 实际代码 | 建议 | -|--------|---------|---------|---------|------| -| **JSON格式** | tag_value在条件中 | tag_value在rule_config中 | ✅ 详细版本正确 | 使用详细版本格式 | -| **字段命名** | 中文名称 | 英文名称 | ✅ 详细版本正确 | 前端显示中文，后端使用英文 | -| **数据列表** | SQL查询配置 | 数据源类型定义 | ✅ 详细版本更准确 | 明确为数据源配置 | -| **正则规则** | operator直接使用正则 | 需要rule_type和pattern | ⚠️ 当前未实现 | 使用simple规则的in运算符替代 | - ---- - -## 五、一致性结论 - -### ✅ 核心含义：**基本一致** - -1. **业务逻辑一致**： - - 都描述了标签定义、规则配置、任务处理的流程 - - 都支持运算规则和正则规则的概念 - -2. **实现细节差异**： - - 原始版本是业务需求的简化描述 - - 详细版本是技术实现的完整说明 - - 差异主要体现在数据格式和实现细节上 - -3. **建议**： - - ✅ **使用详细版本的JSON格式**（符合实际代码） - - ✅ **使用英文字段名**（符合数据库设计） - - ✅ **明确数据源配置方式**（不是SQL，是MongoDB查询或配置） - - ⚠️ **正则规则当前未实现**，建议使用simple规则的in运算符替代 - ---- - -## 六、修正建议 - -### 6.1 修正原始版本的JSON格式 - -**原始版本（需要修正）**： -```json -[ - { - field:"交易金额", - operator:"<", - value:3000, - tag_value："低价值用户" // ❌ 错误位置 - } -] -``` - -**修正后的格式**： -```json -{ - "rule_type": "simple", - "conditions": [ - { - "field": "total_amount", // ✅ 使用英文字段名 - "operator": "<", - "value": 3000 - } - ], - "tag_value": "低价值用户", // ✅ 在rule_config中 - "confidence": 1.0 -} -``` - -### 6.2 明确数据源配置 - -**原始版本描述**： -> "先用sql语句查询出符合条件的结果，这段数据库的查询操作可以配置化" - -**修正为**： -> "配置数据源类型（如：consumption_records），系统提供该数据源的字段列表API，前端选择字段进行规则配置" - -### 6.3 正则规则说明 - -**原始版本**： -> operator:"/淘宝/" - -**修正为**： -> 当前系统支持 `simple` 规则，正则匹配可以通过 `in` 运算符实现： -> ```json -> { -> "field": "shop_name", -> "operator": "in", -> "value": ["淘宝", "天猫"] -> } -> ``` - ---- - -## 七、最终结论 - -**两个文档的核心含义基本一致**，但存在以下差异： - -1. ✅ **业务逻辑**：完全一致 -2. ⚠️ **数据格式**：原始版本需要修正（tag_value位置、字段命名） -3. ⚠️ **技术细节**：详细版本更准确（符合实际代码实现） -4. ✅ **流程描述**：详细版本是对原始版本的细化 - -**建议**： -- 保留详细版本作为技术实现文档 -- 修正原始版本中的格式问题 -- 明确数据源配置方式（不是SQL，是MongoDB数据源配置） - ---- - -**分析时间**：2025-01-XX -**基于代码版本**：当前代码库分析 diff --git a/Moncter/Moncter/提示词/标签定逻辑.md b/Moncter/Moncter/提示词/标签定逻辑.md deleted file mode 100644 index 8cbfaddd..00000000 --- a/Moncter/Moncter/提示词/标签定逻辑.md +++ /dev/null @@ -1,68 +0,0 @@ -1、标签定义 - -## 后台预先工作： -// 标签定义数据列表API,由于mongodb表具有特殊性，因此需要预先处理 --用代码先写好数据查询，如果可以，可以做个手动配置sql的方式，例如 -标签表：[ - 消费数据=> 消费记录表的sql -] - -// 数据列表字段API --通过上方的列表id，返回展示下字段 - -## 前台界面 --标签规则配置，这块在UI界面是可以选择的 - -1、运算规则 - 选择"数据列表字段API"展示出来的API，例如消费记录展示了字段：交易金额、店铺名称、交易状态、手机号 - 配置示例JSON： - - [ - { - field:"交易金额", - operator:"<" , - value:3000, - tag_value："低价值用户" - }, - { - field:"交易金额", - operator:">=" , - value:3000, - tag_value："中等价值用户" - }, - { - field:"交易金额", - operator:">" , - value:9000, - tag_value："高价值用户" - }, - - ] - 注意：一个item代表一个条件， - - - -2、正则规则 - #判断字符串是否含有淘宝二字，如果有就是淘宝平台的 - [ - { - field:"店铺名称", - operator:"/淘宝/" , - value:true, - tag_value："淘宝平台" - }, - - ] - 注意：一个item代表一个条件。 - - -标签保存的时候，要输入标签编码、标签名称 - - -##标签任务处理逻辑顺序 - -1、选择已定义的标签 -2、选择后点击保存，状态为待启动 -3、在列表启动的时候，后台先从后台读取定义标签的集合，并查询出来 -4、根据定义标签的规则条件，直接根据用户表遍历过去进行打标签，然后将标签存储。 - \ No newline at end of file diff --git a/Moncter/Moncter/提示词/标签定逻辑真实.md b/Moncter/Moncter/提示词/标签定逻辑真实.md deleted file mode 100644 index 2587d318..00000000 --- a/Moncter/Moncter/提示词/标签定逻辑真实.md +++ /dev/null @@ -1,737 +0,0 @@ -# 标签定义逻辑说明 - -## 一、概述 - -标签定义是标签引擎的核心功能，用于定义如何根据用户数据自动计算和打标签。系统支持两种规则类型：运算规则和正则规则。 - ---- - -## 二、后台预先工作 - -### 2.1 标签定义数据列表API - -由于 MongoDB 表的特殊性，需要预先配置数据查询操作。 - -#### 2.1.1 数据列表配置 - -**功能说明**： -- 通过配置化的查询操作，从 MongoDB 集合中查询出符合条件的结果 -- 这个查询操作可以配置化，支持动态增减数据集合 -- 每个查询结果可以命名，例如：`消费记录表`、`用户档案表`、`订单明细表` 等 - -**配置示例**： -```json -{ - "list_id": "consumption_records_list", - "list_name": "消费记录表", - "data_source_id": "source_123", - "database": "tag_engine", - "collection": "consumption_records", - "query_config": { - "filter": {}, - "sort": { "create_time": -1 }, - "limit": 1000 - }, - "description": "消费记录数据列表", - "status": 1, - "create_time": "2025-01-01T00:00:00Z" -} -``` - -**数据结构**： -```javascript -{ - list_id: String, // 列表ID（唯一标识） - list_name: String, // 列表名称（如：消费记录表） - data_source_id: String, // 数据源ID - database: String, // 数据库名 - collection: String, // 集合名 - query_config: Object, // 查询配置（MongoDB查询条件） - description: String, // 描述 - status: Number, // 状态（0:禁用, 1:启用） - create_time: Date, - update_time: Date -} -``` - -**API接口**： -- `GET /api/tag-data-lists` - 获取数据列表列表 -- `POST /api/tag-data-lists` - 创建数据列表配置 -- `GET /api/tag-data-lists/{list_id}` - 获取数据列表详情 -- `PUT /api/tag-data-lists/{list_id}` - 更新数据列表配置 -- `DELETE /api/tag-data-lists/{list_id}` - 删除数据列表配置 - -#### 2.1.2 数据列表字段API - -**功能说明**： -- 通过数据列表ID，返回该列表的字段信息 -- 用于前端界面选择字段进行规则配置 - -**API接口**：`GET /api/tag-data-lists/{list_id}/fields` - -**返回格式**： -```json -{ - "code": 200, - "message": "查询成功", - "data": { - "list_id": "consumption_records_list", - "list_name": "消费记录表", - "fields": [ - { - "field": "交易金额", - "field_name": "amount", - "type": "number", - "description": "交易金额" - }, - { - "field": "店铺名称", - "field_name": "shop_name", - "type": "string", - "description": "店铺名称" - }, - { - "field": "交易状态", - "field_name": "status", - "type": "string", - "description": "交易状态" - }, - { - "field": "手机号", - "field_name": "phone", - "type": "string", - "description": "手机号" - } - ] - } -} -``` - -**字段说明**： -- `field`：前端显示的字段名称（中文） -- `field_name`：数据库中的实际字段名（英文） -- `type`：字段类型（number、string、datetime、boolean等） -- `description`：字段描述 - -**实现方式**： -1. 从数据列表配置中获取 `collection` 信息 -2. 查询该集合的样本数据（如：前10条） -3. 分析样本数据的字段结构 -4. 返回字段列表（包含中英文映射） - ---- - -## 三、前台界面配置 - -### 3.1 标签基本信息 - -创建标签定义时需要填写以下基本信息： - -- **标签编码** (`tag_code`)：唯一标识，如 `high_consumer`、`vip_user` -- **标签名称** (`tag_name`)：显示名称，如 `高消费用户`、`VIP用户` - -**可选字段**： -- **分类** (`category`)：标签分类，如 `消费能力`、`活跃度`、`风险等级`、`生命周期` -- **描述** (`description`)：标签的详细说明 -- **更新频率** (`update_frequency`)： - - `real_time` - 实时更新 - - `daily` - 每日更新 - - `weekly` - 每周更新 - - `monthly` - 每月更新 -- **状态** (`status`)： - - `0` - 启用 - - `1` - 禁用 - -### 3.2 规则配置 - -标签规则配置在UI界面可以选择两种类型：**运算规则** 和 **正则规则**。 - -#### 3.2.1 运算规则 - -**配置流程**： -1. 选择数据列表（如：消费记录表） -2. 调用数据列表字段API，获取字段列表 -3. 选择字段（如：交易金额、店铺名称、交易状态、手机号） -4. 配置条件：字段、运算符、值、标签值 - -**配置示例JSON**： -```json -[ - { - "field": "交易金额", - "field_name": "amount", - "operator": "<", - "value": 3000, - "tag_value": "低价值用户" - }, - { - "field": "交易金额", - "field_name": "amount", - "operator": ">=", - "value": 3000, - "tag_value": "中等价值用户" - }, - { - "field": "交易金额", - "field_name": "amount", - "operator": ">", - "value": 9000, - "tag_value": "高价值用户" - } -] -``` - -**重要说明**： -- **一个item代表一个条件** -- 每个条件都有独立的 `tag_value` -- 当条件满足时，用户将被标记为该条件的 `tag_value` -- 多个条件之间是**互斥**的（OR关系），满足任一条件即可 - -**支持的运算符**： -- `>` - 大于 -- `>=` - 大于等于 -- `<` - 小于 -- `<=` - 小于等于 -- `=` / `==` - 等于 -- `!=` - 不等于 -- `in` - 在列表中（值为数组） -- `not_in` - 不在列表中（值为数组） - -**存储格式**（转换为标准格式）： -```json -{ - "rule_type": "simple", - "data_list_id": "consumption_records_list", - "conditions": [ - { - "field": "交易金额", - "field_name": "amount", - "operator": "<", - "value": 3000, - "tag_value": "低价值用户" - }, - { - "field": "交易金额", - "field_name": "amount", - "operator": ">=", - "value": 3000, - "tag_value": "中等价值用户" - }, - { - "field": "交易金额", - "field_name": "amount", - "operator": ">", - "value": 9000, - "tag_value": "高价值用户" - } - ] -} -``` - -#### 3.2.2 正则规则 - -**配置说明**： -- 使用正则表达式匹配字符串字段 -- 判断字符串是否含有指定的模式 - -**配置示例**： -```json -[ - { - "field": "店铺名称", - "field_name": "shop_name", - "operator": "/淘宝/", - "value": true, - "tag_value": "淘宝平台" - } -] -``` - -**重要说明**： -- **一个item代表一个条件** -- `operator` 字段直接使用正则表达式（如：`/淘宝/`） -- `value` 字段表示是否匹配（`true` 表示匹配，`false` 表示不匹配） -- 当正则匹配成功时，用户将被标记为 `tag_value` - -**存储格式**（转换为标准格式）： -```json -{ - "rule_type": "regex", - "data_list_id": "consumption_records_list", - "conditions": [ - { - "field": "店铺名称", - "field_name": "shop_name", - "operator": "/淘宝/", - "pattern": "淘宝", - "value": true, - "tag_value": "淘宝平台" - } - ] -} -``` - -**正则表达式说明**： -- 支持标准的正则表达式语法 -- 常用模式： - - `/淘宝/` - 包含"淘宝" - - `/^淘宝/` - 以"淘宝"开头 - - `/淘宝$/` - 以"淘宝"结尾 - - `/淘宝|天猫/` - 包含"淘宝"或"天猫" - -### 3.3 完整的标签定义数据结构 - -```json -{ - "tag_id": "uuid", - "tag_code": "consumer_level", - "tag_name": "消费等级", - "category": "消费能力", - "description": "根据消费金额划分用户等级", - "rule_type": "simple", - "rule_config": { - "rule_type": "simple", - "data_list_id": "consumption_records_list", - "data_list_name": "消费记录表", - "conditions": [ - { - "field": "交易金额", - "field_name": "amount", - "operator": "<", - "value": 3000, - "tag_value": "低价值用户" - }, - { - "field": "交易金额", - "field_name": "amount", - "operator": ">=", - "value": 3000, - "tag_value": "中等价值用户" - }, - { - "field": "交易金额", - "field_name": "amount", - "operator": ">", - "value": 9000, - "tag_value": "高价值用户" - } - ] - }, - "update_frequency": "real_time", - "status": 0, - "priority": 1, - "version": 1, - "create_time": "2025-01-01T00:00:00Z", - "update_time": "2025-01-01T00:00:00Z" -} -``` - ---- - -## 四、标签任务处理逻辑 - -### 4.1 标签任务创建流程 - -``` -1. 用户在前台创建标签任务 - ├─→ 选择已定义的标签（可多选） - ├─→ 配置任务类型（full/incremental/specified） - ├─→ 配置用户范围（all/list/filter） - ├─→ 配置调度计划（可选） - └─→ 点击保存 - -2. 任务保存后 - ├─→ 状态设置为 "pending"（待启动） - ├─→ 任务信息保存到 tag_tasks 集合 - └─→ 返回任务ID -``` - -### 4.2 标签任务启动流程 - -``` -1. 用户在任务列表点击"启动"按钮 - ↓ -2. 调用API: POST /api/tag-tasks/{task_id}/start - ↓ -3. TagTaskService->startTask() - ├─→ 检查任务状态（不能是 running） - ├─→ 更新任务状态为 "running" - └─→ 设置 Redis 标志：tag_task:{taskId}:start - ↓ -4. TagTaskExecutor->execute() - ├─→ 创建执行记录（tag_task_executions） - ├─→ 获取目标标签ID列表（target_tag_ids） - ├─→ 遍历每个标签定义 - │ ├─→ 从 tag_definitions 读取标签定义 - │ ├─→ 获取 rule_config 中的 data_list_id - │ ├─→ 根据 data_list_id 查询数据列表配置 - │ ├─→ 根据数据列表配置查询数据集合 - │ └─→ 获取用户ID列表（从数据集合中提取） - │ - └─→ 批量处理用户（批次大小可配置，默认100） - ↓ - 5. 遍历每个用户批次 - For each user in batch: - ├─→ 检查任务状态（是否被暂停/停止） - ├─→ 根据标签定义的规则条件计算标签值 - │ ├─→ 获取用户在该数据列表中的数据 - │ ├─→ 遍历 rule_config.conditions - │ │ ├─→ 根据 field_name 获取字段值 - │ │ ├─→ 根据 operator 进行比较 - │ │ │ ├─→ 运算规则：数值/字符串比较 - │ │ │ └─→ 正则规则：正则表达式匹配 - │ │ └─→ 如果条件满足，使用该条件的 tag_value - │ ├─→ 如果多个条件满足，使用第一个满足条件的 tag_value - │ └─→ 如果所有条件都不满足，标签值为 null（不打标签） - ├─→ 更新或创建 user_tags 记录 - ├─→ 记录标签变更历史（tag_history） - ├─→ 成功：success_count++ - ├─→ 失败：error_count++ - └─→ 每处理10个用户更新一次进度 - ├─→ processed_users - ├─→ success_count - ├─→ error_count - └─→ percentage = (processed_users / total_users) * 100 - ↓ -6. 更新最终进度和统计 - ├─→ percentage = 100 - ├─→ 更新执行记录状态为 "completed" - └─→ 更新任务统计信息 -``` - -### 4.3 标签计算核心逻辑 - -#### 4.3.1 数据获取 - -**步骤1：读取标签定义** -```php -// 从 tag_definitions 集合读取标签定义 -$tagDefinition = TagDefinitionRepository::find($tagId); -$ruleConfig = $tagDefinition->rule_config; -$dataListId = $ruleConfig['data_list_id']; -``` - -**步骤2：获取数据列表配置** -```php -// 根据 data_list_id 查询数据列表配置 -$dataList = TagDataListRepository::find($dataListId); -$collection = $dataList->collection; -$database = $dataList->database; -$queryConfig = $dataList->query_config; -``` - -**步骤3：查询数据集合** -```php -// 根据数据列表配置查询数据 -$collectionObj = MongoDB::connection($database)->collection($collection); -$data = $collectionObj->find($queryConfig['filter']) - ->sort($queryConfig['sort'] ?? []) - ->limit($queryConfig['limit'] ?? 1000) - ->toArray(); -``` - -#### 4.3.2 规则计算 - -**运算规则计算**： -```php -foreach ($ruleConfig['conditions'] as $condition) { - $fieldName = $condition['field_name']; - $operator = $condition['operator']; - $expectedValue = $condition['value']; - $tagValue = $condition['tag_value']; - - // 从数据中获取字段值 - $actualValue = $data[$fieldName] ?? null; - - // 根据运算符进行比较 - $match = false; - switch ($operator) { - case '>': - $match = $actualValue > $expectedValue; - break; - case '>=': - $match = $actualValue >= $expectedValue; - break; - case '<': - $match = $actualValue < $expectedValue; - break; - case '<=': - $match = $actualValue <= $expectedValue; - break; - case '=': - case '==': - $match = $actualValue == $expectedValue; - break; - case '!=': - $match = $actualValue != $expectedValue; - break; - case 'in': - $match = in_array($actualValue, (array)$expectedValue); - break; - case 'not_in': - $match = !in_array($actualValue, (array)$expectedValue); - break; - } - - // 如果条件满足，返回该条件的 tag_value - if ($match) { - return $tagValue; - } -} - -// 所有条件都不满足，返回 null -return null; -``` - -**正则规则计算**： -```php -foreach ($ruleConfig['conditions'] as $condition) { - $fieldName = $condition['field_name']; - $pattern = $condition['pattern'] ?? $condition['operator']; // 从operator或pattern获取正则 - $expectedMatch = $condition['value']; // true表示匹配，false表示不匹配 - $tagValue = $condition['tag_value']; - - // 从数据中获取字段值 - $actualValue = $data[$fieldName] ?? ''; - - // 执行正则匹配 - $isMatch = preg_match($pattern, $actualValue); - - // 判断是否满足条件 - if (($expectedMatch && $isMatch) || (!$expectedMatch && !$isMatch)) { - return $tagValue; - } -} - -// 所有条件都不满足，返回 null -return null; -``` - -#### 4.3.3 标签值存储 - -**存储到 user_tags 集合**： -```javascript -{ - user_id: "用户ID", - tag_id: "标签ID", - tag_value: "标签值（字符串）", // 从条件的tag_value获取 - tag_value_type: "string", - confidence: 1.0, - effective_time: "生效时间", - create_time: "创建时间", - update_time: "更新时间" -} -``` - -**重要说明**： -- 如果计算出的标签值为 `null`，则不创建或删除该用户的该标签记录 -- 如果标签值发生变化，记录到 `tag_history` 集合 - ---- - -## 五、数据存储结构 - -### 5.1 标签定义集合（tag_definitions） - -```javascript -{ - tag_id: String, // 标签ID（UUID） - tag_code: String, // 标签代码（唯一标识） - tag_name: String, // 标签名称 - category: String, // 标签分类（可选） - description: String, // 描述（可选） - rule_type: String, // 规则类型（simple/regex） - rule_config: Object, // 规则配置（JSON） - update_frequency: String, // 更新频率（可选） - status: Number, // 状态（0:启用, 1:禁用） - priority: Number, // 优先级（可选） - version: Number, // 版本号（可选） - create_time: Date, - update_time: Date -} -``` - -### 5.2 数据列表配置集合（tag_data_lists） - -```javascript -{ - list_id: String, // 列表ID（UUID） - list_name: String, // 列表名称（如：消费记录表） - data_source_id: String, // 数据源ID - database: String, // 数据库名 - collection: String, // 集合名 - query_config: Object, // 查询配置（MongoDB查询条件） - description: String, // 描述 - status: Number, // 状态（0:禁用, 1:启用） - create_time: Date, - update_time: Date -} -``` - -### 5.3 用户标签集合（user_tags） - -```javascript -{ - user_id: String, // 用户ID - tag_id: String, // 标签ID - tag_value: String, // 标签值（字符串格式） - tag_value_type: String, // 值类型（string） - confidence: Number, // 置信度（0.0-1.0） - effective_time: Date, // 生效时间 - expire_time: Date, // 过期时间（可选） - create_time: Date, - update_time: Date -} -``` - -### 5.4 标签历史集合（tag_history） - -```javascript -{ - history_id: String, // 历史记录ID - user_id: String, // 用户ID - tag_id: String, // 标签ID - old_value: String, // 旧值 - new_value: String, // 新值 - change_reason: String, // 变更原因 - change_time: Date, // 变更时间 - operator: String // 操作人 -} -``` - -### 5.5 标签任务集合（tag_tasks） - -```javascript -{ - task_id: String, // 任务ID - name: String, // 任务名称 - description: String, // 任务描述 - task_type: String, // 任务类型（full/incremental/specified） - target_tag_ids: Array, // 目标标签ID列表 - user_scope: Object, // 用户范围配置 - schedule: Object, // 调度计划 - config: Object, // 任务配置 - status: String, // 任务状态（pending/running/paused/stopped/error） - progress: Object, // 任务进度 - statistics: Object, // 任务统计 - created_by: String, - created_at: Date, - updated_at: Date -} -``` - ---- - -## 六、API接口说明 - -### 6.1 数据列表接口 - -- `GET /api/tag-data-lists` - 获取数据列表列表 -- `POST /api/tag-data-lists` - 创建数据列表配置 -- `GET /api/tag-data-lists/{list_id}` - 获取数据列表详情 -- `PUT /api/tag-data-lists/{list_id}` - 更新数据列表配置 -- `DELETE /api/tag-data-lists/{list_id}` - 删除数据列表配置 -- `GET /api/tag-data-lists/{list_id}/fields` - 获取数据列表字段 - -### 6.2 标签定义接口 - -- `GET /api/tag-definitions` - 获取标签定义列表 -- `POST /api/tag-definitions` - 创建标签定义 -- `GET /api/tag-definitions/{tag_id}` - 获取标签定义详情 -- `PUT /api/tag-definitions/{tag_id}` - 更新标签定义 -- `DELETE /api/tag-definitions/{tag_id}` - 删除标签定义 - -### 6.3 标签任务接口 - -- `GET /api/tag-tasks` - 获取标签任务列表 -- `POST /api/tag-tasks` - 创建标签任务 -- `GET /api/tag-tasks/{task_id}` - 获取任务详情 -- `POST /api/tag-tasks/{task_id}/start` - 启动任务 -- `POST /api/tag-tasks/{task_id}/pause` - 暂停任务 -- `POST /api/tag-tasks/{task_id}/stop` - 停止任务 - ---- - -## 七、关键代码文件 - -### 7.1 后端代码（需要实现） - -- `app/controller/TagDataListController.php` - 数据列表控制器 -- `app/repository/TagDataListRepository.php` - 数据列表数据访问 -- `app/service/TagDataListService.php` - 数据列表服务 -- `app/controller/TagDefinitionController.php` - 标签定义控制器 -- `app/service/TagService.php` - 标签计算服务（需要修改） -- `app/service/TagRuleEngine/SimpleRuleEngine.php` - 简单规则引擎（需要修改） -- `app/service/TagRuleEngine/RegexRuleEngine.php` - 正则规则引擎（需要新增） -- `app/service/TagTaskService.php` - 标签任务服务 -- `app/service/TagTaskExecutor.php` - 标签任务执行器（需要修改） - -### 7.2 前端代码（需要实现） - -- `TaskShow/src/views/TagDataList/List.vue` - 数据列表管理 -- `TaskShow/src/views/TagDataList/Form.vue` - 数据列表配置表单 -- `TaskShow/src/views/TagDefinition/Form.vue` - 标签定义表单（需要修改） -- `TaskShow/src/views/TagDefinition/List.vue` - 标签定义列表 -- `TaskShow/src/views/TagTask/TaskForm.vue` - 标签任务表单 -- `TaskShow/src/store/tagDataList.ts` - 数据列表状态管理（需要新增） - ---- - -## 八、注意事项 - -1. **数据列表配置**：必须先配置数据列表，才能创建标签定义 -2. **字段映射**：前端使用中文字段名（field），后端使用英文字段名（field_name） -3. **条件逻辑**：多个条件之间是互斥的（OR关系），满足第一个条件即使用该条件的tag_value -4. **标签值**：如果所有条件都不满足，标签值为null，不创建标签记录 -5. **数据查询**：根据数据列表配置的query_config查询数据，支持MongoDB查询语法 -6. **正则规则**：operator字段直接存储正则表达式字符串，需要解析后使用preg_match匹配 - ---- - -## 九、实现要点 - -### 9.1 数据列表配置 - -1. **查询配置格式**： - ```json - { - "filter": { "status": "active" }, - "sort": { "create_time": -1 }, - "limit": 1000 - } - ``` - -2. **字段自动识别**： - - 查询样本数据（前10条） - - 分析字段结构 - - 返回字段列表（包含中英文映射） - -### 9.2 规则计算逻辑 - -1. **运算规则**： - - 从数据中获取字段值 - - 根据运算符进行比较 - - 满足条件即返回该条件的tag_value - -2. **正则规则**： - - 从数据中获取字段值 - - 使用preg_match进行正则匹配 - - 匹配成功即返回该条件的tag_value - -### 9.3 标签任务执行 - -1. **数据获取**： - - 根据标签定义的data_list_id获取数据列表配置 - - 根据数据列表配置查询数据集合 - - 提取用户ID列表 - -2. **标签计算**： - - 遍历每个用户 - - 获取该用户在数据列表中的数据 - - 根据规则条件计算标签值 - - 存储标签结果 - ---- - -**文档更新时间**：2025-01-XX -**基于需求**：按照原始思路完善 diff --git a/Moncter/Moncter/提示词/集合筛选功能使用技巧.md b/Moncter/Moncter/提示词/集合筛选功能使用技巧.md deleted file mode 100644 index 93c64aab..00000000 --- a/Moncter/Moncter/提示词/集合筛选功能使用技巧.md +++ /dev/null @@ -1,342 +0,0 @@ -# 集合筛选功能使用技巧 - -## 功能概览 - -在多集合模式下，提供了强大的筛选和批量操作功能，让选择大量集合变得简单快捷。 - ---- - -## 核心功能 - -### 1. 文本筛选 - -**输入框筛选**： -``` -[🔍 筛选集合名称...] -``` - -**功能**： -- 实时筛选集合列表 -- 支持模糊匹配 -- 不区分大小写 -- 支持中文和英文 - -**示例**： -``` -输入 "2021" → 显示所有包含"2021"的集合 -输入 "女" → 显示所有包含"女"的集合 -输入 "cons" → 显示所有包含"cons"的集合 -``` - -### 2. 批量操作按钮 - -#### 全选 -- **功能**：选择当前筛选结果的**所有**集合 -- **行为**：追加到已选列表（不清除其他已选项） -- **场景**：快速选择某一类集合 - -**示例**： -``` -1. 输入 "2021" -2. 点击 [全选] -3. 清空筛选 -4. 输入 "2022" -5. 点击 [全选] -→ 结果：同时选中2021和2022的所有集合 -``` - -#### 清空 -- **功能**：清空**所有**已选集合 -- **行为**：清除整个选择列表 -- **场景**：重新开始选择 - -#### 反选 -- **功能**：反选当前筛选结果的集合 -- **行为**： - - 已选中的 → 取消选择 - - 未选中的 → 选中 - - 不在筛选结果中的已选项 → 保持选中 -- **场景**：排除某些集合 - -**示例**： -``` -1. 点击快捷筛选 [2021年]（选中全年） -2. 输入 "202101"（只显示1月） -3. 点击 [反选]（取消选择1月） -→ 结果：选中2021年的2-12月 -``` - -### 3. 快捷筛选（智能按钮） - -当检测到按日期格式的集合（如 `collection_202101`）时，自动显示快捷筛选按钮。 - -#### 按年份筛选 -``` -[2021年] [2022年] [2023年] [2024年] [2025年] -``` - -**功能**：一键选择某一年的所有集合 - -**匹配规则**：包含年份字符串（如 `2021`）的集合 - -**示例**： -``` -点击 [2021年] -→ 自动选中： - consumption_records_202101 - consumption_records_202102 - ... - consumption_records_202112 -``` - -#### 按时间范围筛选 -``` -[最近3个月] [最近6个月] [最近12个月] -``` - -**功能**：智能计算并选择最近N个月的集合 - -**匹配规则**： -1. 获取当前年月（如 2025-01） -2. 向前推算N个月 -3. 查找包含这些年月（YYYYMM格式）的集合 - -**示例**（假设当前是 2025-01）： -``` -点击 [最近3个月] -→ 自动选中： - consumption_records_202501 (2025-01) - consumption_records_202412 (2024-12) - consumption_records_202411 (2024-11) -``` - -``` -点击 [最近6个月] -→ 自动选中： - consumption_records_202501 - consumption_records_202412 - consumption_records_202411 - consumption_records_202410 - consumption_records_202409 - consumption_records_202408 -``` - ---- - -## 使用技巧 - -### 技巧1：组合使用快捷筛选和文本筛选 - -**需求**：选择2021年第一季度 - -**方法**： -``` -1. 输入筛选 "20210" - → 显示 202101, 202102, 202103, ..., 202109 -2. 点击 [全选] -3. 输入 "202104" - → 只显示 202104 -4. 点击 [反选]（取消4月） -5. 重复步骤3-4，排除5-9月 -``` - -**更简单的方法**： -``` -1. 输入 "202101"，点击 [全选] -2. 输入 "202102"，点击 [全选] -3. 输入 "202103"，点击 [全选] -``` - -### 技巧2：使用反选排除特定月份 - -**需求**：选择2021年除了春节月份（1、2月）的所有数据 - -**方法**： -``` -1. 点击快捷筛选 [2021年] - → 选中全年12个月 -2. 输入 "202101" -3. 点击 [反选] - → 取消1月 -4. 输入 "202102" -5. 点击 [反选] - → 取消2月 -6. 清空筛选框查看结果 - → 已选中：202103-202112（10个月） -``` - -### 技巧3：跨年选择 - -**需求**：选择2020年下半年和2021年上半年 - -**方法**： -``` -1. 输入 "202007"，点击 [全选] -2. 输入 "202008"，点击 [全选] -3. 输入 "202009"，点击 [全选] -4. 输入 "202010"，点击 [全选] -5. 输入 "202011"，点击 [全选] -6. 输入 "202012"，点击 [全选] -7. 输入 "202101"，点击 [全选] -8. 输入 "202102"，点击 [全选] -9. 输入 "202103"，点击 [全选] -10. 输入 "202104"，点击 [全选] -11. 输入 "202105"，点击 [全选] -12. 输入 "202106"，点击 [全选] -``` - -**更快的方法**（如果命名规则一致）： -``` -1. 输入 "2020" -2. 点击 [全选]（选中2020全年） -3. 输入 "20200"（202001-202009，前9个月） -4. 点击 [反选]（排除前6个月，只留7-12月） -5. 输入 "2021" -6. 点击 [全选] -7. 输入 "202107"（7月及以后） -8. 向后筛选并反选，只留1-6月 -``` - -### 技巧4：按商品类型批量选择 - -**需求**：在 `KR_淘宝` 数据库中选择所有zippo相关的集合 - -**方法**： -``` -1. 启用多集合模式 -2. 输入 "zippo" - → 显示：zippo1, zippo2, zippo3, zippo4, zippo5 -3. 点击 [全选] - → 一次性选中所有5个zippo集合 -``` - -### 技巧5：逐步累加选择 - -**需求**：精确选择特定几个月份（不连续） - -**方法**： -``` -需要：1月、3月、6月、9月、12月 - -1. 输入 "202101"，点击 [全选] -2. 输入 "202103"，点击 [全选] -3. 输入 "202106"，点击 [全选] -4. 输入 "202109"，点击 [全选] -5. 输入 "202112"，点击 [全选] -6. 清空筛选框查看 - → 已选中5个月份 -``` - ---- - -## 最佳实践 - -### 1. 大量集合的选择策略 - -**场景**：有100+个集合，需要选择大部分 - -**推荐流程**： -``` -1. 使用快捷筛选或全选按钮一次性选中 -2. 使用文本筛选+反选排除不需要的 -3. 清空筛选查看最终结果 -``` - -**示例**： -``` -选择2021-2023的所有数据，但排除测试月份 - -1. 点击 [2021年] -2. 点击 [2022年] -3. 点击 [2023年] - → 已选36个月 -4. 输入 "test" -5. 点击 [反选] - → 排除测试集合 -``` - -### 2. 少量集合的选择策略 - -**场景**：只需要选择几个集合 - -**推荐流程**： -``` -直接使用文本筛选+全选 -``` - -**示例**： -``` -只选择3个月 - -1. 输入 "202101"，点击 [全选] -2. 输入 "202102"，点击 [全选] -3. 输入 "202103"，点击 [全选] -``` - -### 3. 动态时间范围 - -**场景**：需要最近的数据（随时间变化） - -**推荐**： -``` -使用 [最近N个月] 快捷按钮 -优点： -- 自动计算当前时间 -- 配置一次，永久有效 -- 不需要手动更新月份 -``` - -### 4. 固定时间范围 - -**场景**：需要特定历史时期的数据 - -**推荐**： -``` -使用年份按钮或文本筛选 -例如：统计2021年的历史数据 -→ 点击 [2021年] -``` - ---- - -## 常见场景快速指南 - -| 需求 | 最佳方法 | 步骤 | -|------|----------|------| -| 选择某一年全年数据 | 年份按钮 | 点击 `[2021年]` | -| 选择最近半年数据 | 时间范围按钮 | 点击 `[最近6个月]` | -| 选择第一季度 | 文本筛选+全选 | 输入 `Q1` 或逐月选择 | -| 选择所有zippo集合 | 文本筛选+全选 | 输入 `zippo`，点击 `[全选]` | -| 排除某几个月 | 全选+反选 | 先全选年份，再筛选排除项并反选 | -| 选择不连续月份 | 逐个筛选+全选 | 每个月份单独筛选后全选 | -| 跨年选择 | 多次年份按钮 | 点击多个年份按钮 | - ---- - -## 注意事项 - -1. **筛选不影响已选项** - - 筛选只影响显示，不会取消已选中的集合 - - 即使筛选后看不到某个已选集合，它仍然被选中 - -2. **全选是追加操作** - - 点击"全选"会追加到已选列表 - - 不会清除之前的选择 - - 如需重新开始，先点击"清空" - -3. **反选的作用域** - - 反选只对当前筛选结果生效 - - 不在筛选结果中的已选项不受影响 - -4. **快捷筛选的智能识别** - - 快捷按钮仅在检测到日期格式集合时显示 - - 如果集合命名不包含日期，不会显示这些按钮 - -5. **性能考虑** - - 选择大量集合（50+）可能影响查询性能 - - 建议根据实际需求选择合适的时间范围 - ---- - -**更新时间**：2025-01-XX -**适用版本**：QueryBuilder v2.0+ diff --git a/Moncter/Moncter/提示词/项目完整代码逻辑分析报告.md b/Moncter/Moncter/提示词/项目完整代码逻辑分析报告.md deleted file mode 100644 index f1275446..00000000 --- a/Moncter/Moncter/提示词/项目完整代码逻辑分析报告.md +++ /dev/null @@ -1,858 +0,0 @@ -# 项目完整代码逻辑分析报告 - -## 一、项目概述 - -### 1.1 项目定位 -本项目是一个**基于Webman框架的用户标签引擎和数据采集中心**，主要功能包括： -- **多数据源数据采集**：支持MongoDB、MySQL等多种数据源 -- **数据库实时同步**：使用MongoDB Change Streams实现数据库间实时同步 -- **用户标签计算引擎**：基于用户消费数据实时计算和更新标签 -- **任务配置化管理**：通过配置文件统一管理所有数据采集任务 -- **用户身份管理**：支持身份证、手机号等标识的统一管理 - -### 1.2 技术栈 -- **框架**：Webman (Workerman) - 高性能PHP框架 -- **数据库**：MongoDB (主数据库) -- **消息队列**：RabbitMQ - 异步任务处理 -- **缓存/锁**：Redis - 分布式锁、状态存储 -- **前端**：Vue 3 + TypeScript + Element Plus -- **PHP版本**：>= 8.1 - ---- - -## 二、系统架构 - -### 2.1 分层架构 - -``` -┌─────────────────────────────────────────┐ -│ 应用层 (HTTP API) │ -│ User/Tag/Task/DataSource API │ -└──────────────────┬──────────────────────┘ - │ -┌──────────────────▼──────────────────────┐ -│ 业务服务层 (Service) │ -│ UserService / TagService / │ -│ DataCollectionTaskService / │ -│ ConsumptionService │ -└──────────────────┬──────────────────────┘ - │ -┌──────────────────▼──────────────────────┐ -│ 数据访问层 (Repository) │ -│ UserProfile / UserTag / │ -│ ConsumptionRecord / DataCollectionTask │ -└──────────────────┬──────────────────────┘ - │ -┌──────────────────▼──────────────────────┐ -│ 数据存储层 │ -│ MongoDB / Redis / RabbitMQ │ -└─────────────────────────────────────────┘ -``` - -### 2.2 进程架构 - -系统使用Workerman多进程架构，包含以下进程： - -1. **webman (HTTP Server)** - - 处理HTTP请求 - - 进程数：CPU核心数 × 4 - -2. **monitor (文件监控)** - - 监控文件变化，自动重载 - -3. **data_sync_scheduler (数据采集任务调度器)** - - 读取任务配置（配置文件 + 数据库） - - 启动和管理所有数据采集任务 - - 进程数：10 - -4. **data_sync_worker (数据同步Worker)** - - 消费RabbitMQ消息队列 - - 写入目标数据库，更新用户统计 - - 进程数：20 - -5. **tag_calculation_worker (标签计算Worker)** - - 消费RabbitMQ标签计算队列 - - 根据用户数据计算标签值 - - 进程数：2 - ---- - -## 三、核心业务逻辑 - -### 3.1 数据采集系统 - -#### 3.1.1 任务配置体系 - -系统支持两种任务配置方式： - -**1. 配置文件方式** (`config/data_collection_tasks.php`) -```php -'tasks' => [ - 'database_sync' => [ - 'name' => '数据库实时同步', - 'source_data_source' => 'kr_mongodb', - 'target_data_source' => 'sync_mongodb', - 'handler_class' => DatabaseSyncHandler::class, - 'schedule' => ['enabled' => false], // 持续运行 - ], -] -``` - -**2. 数据库方式** (`data_collection_tasks`集合) -- 通过前端界面动态创建和管理任务 -- 支持批量采集（batch）和实时监听（realtime）两种模式 -- 支持字段映射、过滤条件、连表查询等配置 - -#### 3.1.2 任务执行流程 - -``` -用户创建任务 - ↓ -保存到MongoDB (data_collection_tasks集合) - ↓ -用户点击"启动"按钮 - ↓ -API: POST /api/data-collection-tasks/{taskId}/start - ↓ -DataCollectionTaskService->startTask() - - 更新任务状态为 'running' - - Redis设置标志: data_collection_task:{taskId}:start - ↓ -DataSyncScheduler进程检测到Redis标志 - ↓ -从数据库加载任务配置 - ↓ -根据任务模式执行： - - batch模式：根据schedule配置定时执行或立即执行 - - realtime模式：立即启动，持续运行（Change Stream监听） -``` - -#### 3.1.3 Handler处理机制 - -系统通过Handler模式实现不同类型的数据采集： - -**ConsumptionCollectionHandler** - 消费记录采集 -- 专门处理消费记录采集 -- 支持KR_商城和KR_金融两种数据源 -- 自动提取手机号、解析用户ID、写入消费记录 - -**GenericCollectionHandler** - 通用数据采集 -- 支持动态字段映射 -- 支持批量采集和实时监听两种模式 -- 支持连表查询（lookup） - -**DatabaseSyncHandler** - 数据库同步 -- 全量同步：首次启动时同步所有数据 -- 增量同步：使用MongoDB Change Streams实时监听 - -### 3.2 用户身份管理系统 - -#### 3.2.1 身份标识体系 - -系统采用**以身份证为主键，手机号为弱标识**的设计： - -1. **user_profile (用户画像表)** - - `user_id`: 用户唯一标识（UUID） - - `id_card_hash`: 身份证哈希值（用于匹配） - - `id_card_encrypted`: 加密的身份证号 - - `is_temporary`: 是否为临时人（true/false） - - `total_amount`: 总消费金额 - - `total_count`: 总消费次数 - -2. **user_phone_relations (手机号关联表)** - - `phone_number`: 手机号 - - `user_id`: 关联的用户ID - - `is_primary`: 是否为主手机号 - -#### 3.2.2 身份解析流程 - -``` -收到数据（包含手机号） - ↓ -IdentifierService->resolvePersonIdByPhone() - ↓ -查询user_phone_relations表 - ├─→ 找到 → 返回user_id - └─→ 未找到 → 创建临时人 - ├─→ 创建user_profile记录（is_temporary=true） - ├─→ 创建user_phone_relations关联 - └─→ 返回user_id -``` - -#### 3.2.3 身份合并机制 - -当发现手机号对应的身份证后： - -``` -发现身份证 - ↓ -IdentifierService->bindIdCardToPerson() - ↓ -检查身份证是否已被其他用户使用 - ├─→ 是 → 抛出异常 - └─→ 否 → 更新用户信息 - ├─→ 设置id_card_hash和id_card_encrypted - ├─→ is_temporary = false - └─→ 标记为正式人 - ↓ -PersonMergeService->mergePhoneToIdCard() - ↓ -重新计算所有标签 -``` - -### 3.3 标签计算系统 - -#### 3.3.1 标签定义 - -标签定义存储在`tag_definitions`集合中： - -```javascript -{ - tag_id: "标签ID", - tag_code: "标签代码", - tag_name: "标签名称", - rule_config: { - rule_type: "simple", - conditions: [...], - // 规则配置 - }, - update_frequency: "real_time" | "daily" | "weekly", - status: 0 // 0:启用, 1:禁用 -} -``` - -#### 3.3.2 标签计算流程 - -``` -消费记录写入 - ↓ -ConsumptionService->createRecord() - ├─→ 写入consumption_records表 - ├─→ 更新user_profile统计信息（total_amount, total_count） - └─→ 触发标签计算（异步） - ↓ - 推送到RabbitMQ队列 - ↓ - TagCalculationWorker消费消息 - ↓ - TagService->calculateTags() - ├─→ 获取用户数据 - ├─→ 获取标签定义列表 - ├─→ 遍历每个标签 - │ ├─→ 解析规则配置 - │ ├─→ SimpleRuleEngine计算标签值 - │ ├─→ 更新或创建user_tags记录 - │ └─→ 记录标签变更历史（如果值发生变化） - └─→ 返回更新的标签列表 -``` - -#### 3.3.3 规则引擎 - -系统使用`SimpleRuleEngine`计算标签值： - -```php -// 示例规则配置 -{ - "rule_type": "simple", - "conditions": [ - { - "field": "total_amount", - "operator": ">=", - "value": 1000 - } - ], - "result": { - "value": "VIP", - "confidence": 0.9 - } -} -``` - -### 3.4 消费记录处理 - -#### 3.4.1 消费记录数据结构 - -```javascript -{ - record_id: "记录ID", - user_id: "用户ID", - consume_time: "消费时间", - amount: 100.00, // 消费金额 - actual_amount: 95.00, // 实际金额 - currency: "CNY", - store_id: "店铺ID", - status: 0, - create_time: "创建时间" -} -``` - -#### 3.4.2 处理流程 - -``` -数据采集任务采集到订单数据 - ↓ -ConsumptionCollectionHandler处理 - ├─→ 提取手机号 - ├─→ 字段映射和转换 - └─→ 调用ConsumptionService->createRecord() - ↓ - IdentifierService->resolvePersonId() - ├─→ 通过手机号解析user_id - └─→ 如果不存在，创建临时人 - ↓ - 写入consumption_records表 - ↓ - UserProfileRepository->increaseStats() - ├─→ total_amount += actual_amount - ├─→ total_count += 1 - └─→ last_consume_time = consume_time - ↓ - 触发标签计算（异步） -``` - ---- - -## 四、数据存储设计 - -### 4.1 MongoDB集合 - -#### 4.1.1 用户相关集合 - -**user_profile (用户画像)** -```javascript -{ - user_id: String, // 用户唯一标识 - id_card_hash: String, // 身份证哈希 - id_card_encrypted: String, // 加密身份证 - id_card_type: String, // 身份证类型 - is_temporary: Boolean, // 是否临时人 - name: String, - phone: String, - total_amount: Number, // 总消费金额 - total_count: Number, // 总消费次数 - last_consume_time: Date, // 最后消费时间 - tags_update_time: Date, // 标签更新时间 - status: Number, - create_time: Date, - update_time: Date -} -``` - -**user_phone_relations (手机号关联)** -```javascript -{ - phone_number: String, // 手机号 - user_id: String, // 用户ID - is_primary: Boolean, // 是否为主手机号 - effective_time: Date, - expire_time: Date, - create_time: Date -} -``` - -**consumption_records (消费记录)** -```javascript -{ - record_id: String, - user_id: String, - consume_time: Date, - amount: Number, - actual_amount: Number, - currency: String, - store_id: String, - status: Number, - create_time: Date -} -``` - -#### 4.1.2 标签相关集合 - -**tag_definitions (标签定义)** -```javascript -{ - tag_id: String, - tag_code: String, - tag_name: String, - rule_config: Object, // 规则配置（JSON） - update_frequency: String, // real_time/daily/weekly - status: Number, - create_time: Date, - update_time: Date -} -``` - -**user_tags (用户标签)** -```javascript -{ - user_id: String, - tag_id: String, - tag_value: String, // 标签值 - tag_value_type: String, // 值类型 - confidence: Number, // 置信度 - effective_time: Date, - create_time: Date, - update_time: Date -} -``` - -**tag_history (标签变更历史)** -```javascript -{ - user_id: String, - tag_id: String, - old_value: String, - new_value: String, - change_time: Date -} -``` - -#### 4.1.3 任务相关集合 - -**data_collection_tasks (数据采集任务)** -```javascript -{ - task_id: String, - name: String, - data_source_id: String, - database: String, - collection: String, - target_type: String, // consumption_record/generic - handler_type: String, - mode: String, // batch/realtime - schedule: { - enabled: Boolean, - cron: String - }, - field_mappings: Array, - filter_conditions: Object, - progress: { - status: String, - processed_count: Number, - success_count: Number, - error_count: Number, - total_count: Number, - percentage: Number - }, - status: String, // pending/running/paused/stopped - create_time: Date, - update_time: Date -} -``` - -**data_sources (数据源)** -```javascript -{ - data_source_id: String, - name: String, - type: String, // mongodb/mysql - host: String, - port: Number, - database: String, - username: String, - password: String, - is_tag_engine: Boolean, // 是否为标签引擎数据库 - status: Number, - create_time: Date, - update_time: Date -} -``` - -### 4.2 Redis存储 - -#### 4.2.1 分布式锁 -- `lock:data_collection:{task_id}` - 数据采集任务锁 -- `lock:database_sync` - 数据库同步锁 - -#### 4.2.2 任务状态标志 -- `data_collection_task:{task_id}:start` - 启动标志 -- `data_collection_task:{task_id}:pause` - 暂停标志 -- `data_collection_task:{task_id}:stop` - 停止标志 - -#### 4.2.3 同步状态 -- `data_collection:{task_id}:last_sync_time` - 上次同步时间 -- `data_collection:{task_id}:last_sync_id` - 上次同步ID - -### 4.3 RabbitMQ队列 - -#### 4.3.1 数据同步队列 -- 队列名：`data_sync` -- 消息格式： -```json -{ - "user_id": "用户ID", - "data": {...}, - "action": "insert|update|delete" -} -``` - -#### 4.3.2 标签计算队列 -- 队列名：`tag_calculation` -- 消息格式： -```json -{ - "user_id": "用户ID", - "tag_ids": null, // null表示计算所有real_time标签 - "trigger_type": "consumption_record", - "record_id": "记录ID", - "timestamp": 1234567890 -} -``` - ---- - -## 五、API接口体系 - -### 5.1 用户相关接口 - -- `POST /api/users` - 创建用户 -- `GET /api/users/{user_id}` - 查询用户 -- `PUT /api/users/{user_id}` - 更新用户 -- `DELETE /api/users/{user_id}` - 删除用户 -- `POST /api/users/search` - 搜索用户（复杂查询） -- `GET /api/users/{user_id}/decrypt-id-card` - 解密身份证 - -### 5.2 标签相关接口 - -- `GET /api/users/{user_id}/tags` - 查询用户标签 -- `PUT /api/users/{user_id}/tags` - 计算/更新用户标签 -- `DELETE /api/users/{user_id}/tags/{tag_id}` - 删除用户标签 -- `POST /api/tags/filter` - 根据标签筛选用户 -- `GET /api/tags/statistics` - 获取标签统计信息 -- `GET /api/tags/history` - 获取标签历史记录 - -### 5.3 标签定义接口 - -- `GET /api/tag-definitions` - 获取标签定义列表 -- `POST /api/tag-definitions` - 创建标签定义 -- `GET /api/tag-definitions/{tag_id}` - 获取标签定义详情 -- `PUT /api/tag-definitions/{tag_id}` - 更新标签定义 -- `DELETE /api/tag-definitions/{tag_id}` - 删除标签定义 - -### 5.4 数据采集任务接口 - -- `POST /api/data-collection-tasks` - 创建任务 -- `PUT /api/data-collection-tasks/{task_id}` - 更新任务 -- `DELETE /api/data-collection-tasks/{task_id}` - 删除任务 -- `GET /api/data-collection-tasks` - 任务列表 -- `GET /api/data-collection-tasks/{task_id}` - 任务详情 -- `GET /api/data-collection-tasks/{task_id}/progress` - 任务进度 -- `POST /api/data-collection-tasks/{task_id}/start` - 启动任务 -- `POST /api/data-collection-tasks/{task_id}/pause` - 暂停任务 -- `POST /api/data-collection-tasks/{task_id}/stop` - 停止任务 - -### 5.5 数据源接口 - -- `GET /api/data-sources` - 获取数据源列表 -- `GET /api/data-sources/{data_source_id}` - 获取数据源详情 -- `POST /api/data-sources` - 创建数据源 -- `PUT /api/data-sources/{data_source_id}` - 更新数据源 -- `DELETE /api/data-sources/{data_source_id}` - 删除数据源 -- `POST /api/data-sources/test-connection` - 测试数据源连接 - -### 5.6 身份合并接口 - -- `POST /api/person-merge/phone-to-id-card` - 合并手机号到身份证 -- `POST /api/person-merge/temporary-to-formal` - 合并临时人到正式人 - -### 5.7 数据库同步接口 - -- `GET /api/database-sync/progress` - 查询同步进度 -- `GET /api/database-sync/stats` - 查询同步统计 -- `POST /api/database-sync/reset` - 重置同步进度 -- `POST /api/database-sync/skip-error` - 跳过错误数据库 - ---- - -## 六、关键设计模式 - -### 6.1 工厂模式 -- `DataSourceAdapterFactory` - 创建数据源适配器 -- `PollingStrategyFactory` - 创建轮询策略 - -### 6.2 策略模式 -- `PollingStrategyInterface` - 轮询策略接口 - - `MongoDBConsumptionStrategy` - MongoDB消费策略 - - `DefaultConsumptionStrategy` - 默认消费策略 - -### 6.3 适配器模式 -- `DataSourceAdapterInterface` - 数据源适配器接口 - - `MongoDBAdapter` - MongoDB适配器 - - `MySQLAdapter` - MySQL适配器 - -### 6.4 仓库模式 -- 所有Repository类继承MongoDB Model -- 封装数据访问逻辑 -- 提供统一的查询接口 - -### 6.5 Handler模式 -- `ConsumptionCollectionHandler` - 消费记录采集 -- `GenericCollectionHandler` - 通用数据采集 -- `DatabaseSyncHandler` - 数据库同步 - ---- - -## 七、数据流图 - -### 7.1 完整数据流 - -``` -数据源 (KR MongoDB) - ↓ -数据采集任务 (DataSyncScheduler) - ├─→ ConsumptionCollectionHandler - │ ├─→ 批量采集模式：分页查询数据 - │ └─→ 实时监听模式：Change Stream监听 - │ - └─→ DatabaseSyncHandler - ├─→ 全量同步：批量读取写入 - └─→ 增量同步：Change Stream监听 - ↓ -消费记录写入 (ConsumptionService) - ├─→ 身份解析 (IdentifierService) - │ ├─→ 手机号 → user_id - │ └─→ 如果不存在，创建临时人 - │ - ├─→ 写入consumption_records表 - ├─→ 更新user_profile统计信息 - └─→ 触发标签计算（推送到RabbitMQ） - ↓ -标签计算 (TagCalculationWorker) - ├─→ TagService->calculateTags() - ├─→ SimpleRuleEngine计算标签值 - ├─→ 更新user_tags表 - └─→ 记录tag_history变更历史 -``` - -### 7.2 任务执行时序图 - -``` -用户操作 - │ - ├─→ 创建任务 → 保存到MongoDB - │ - └─→ 启动任务 → API → DataCollectionTaskService - │ - ├─→ 更新任务状态为running - └─→ Redis设置启动标志 - │ -DataSyncScheduler进程 - │ - ├─→ 检测Redis标志 - ├─→ 从数据库加载任务配置 - ├─→ 创建数据源适配器 - ├─→ 实例化Handler - └─→ 调用Handler->collect() - │ - ├─→ 批量模式：定时执行 - └─→ 实时模式：持续运行 -``` - ---- - -## 八、关键技术实现 - -### 8.1 MongoDB Change Streams实时监听 - -```php -// 实时监听集合变更 -$changeStream = $collection->watch( - [ - ['$match' => ['operationType' => ['$in' => ['insert', 'update']]]], - ], - ['fullDocument' => 'updateLookup'] -); - -foreach ($changeStream as $change) { - // 处理变更事件 - $document = $change['fullDocument']; - // 处理数据... -} -``` - -### 8.2 分布式锁实现 - -```php -// 使用Redis实现分布式锁 -$lockKey = "lock:data_collection:{$taskId}"; -$locked = RedisHelper::setnx($lockKey, time(), 300); // TTL 300秒 - -if ($locked) { - try { - // 执行任务 - } finally { - RedisHelper::del($lockKey); - } -} -``` - -### 8.3 异步消息队列 - -```php -// 推送标签计算任务到队列 -QueueService::pushTagCalculation([ - 'user_id' => $userId, - 'tag_ids' => null, - 'trigger_type' => 'consumption_record', - 'record_id' => $recordId, -]); - -// Worker消费队列 -$message = $channel->basic_get('tag_calculation'); -if ($message) { - $data = json_decode($message->body, true); - $tagService->calculateTags($data['user_id']); - $channel->basic_ack($message->delivery_info['delivery_tag']); -} -``` - -### 8.4 身份证加密存储 - -```php -// 加密存储 -$idCardEncrypted = EncryptionHelper::encrypt($idCard); -$idCardHash = EncryptionHelper::hash($idCard); // 用于匹配 - -// 解密读取（需要权限） -$idCard = EncryptionHelper::decrypt($idCardEncrypted); -``` - ---- - -## 九、系统特性 - -### 9.1 配置化设计 -- 数据源配置统一管理 -- 任务配置支持配置文件和数据库两种方式 -- 业务逻辑与配置分离 - -### 9.2 高可用性 -- 多进程架构，提高并发能力 -- 分布式锁防止任务重复执行 -- 错误重试机制 -- 断点续传支持 - -### 9.3 可扩展性 -- Handler模式易于添加新的采集任务 -- 适配器模式易于支持新的数据源类型 -- 策略模式易于扩展业务逻辑 - -### 9.4 可观测性 -- 完善的日志系统（业务日志、错误日志、性能日志） -- 实时进度跟踪 -- API接口提供进度和统计查询 - -### 9.5 数据安全性 -- 身份证加密存储 -- 支持数据脱敏 -- 日志脱敏处理 - ---- - -## 十、前端架构 - -### 10.1 技术栈 -- Vue 3 + TypeScript -- Element Plus (UI组件库) -- Pinia (状态管理) -- Vue Router (路由) -- Axios (HTTP请求) -- Vite (构建工具) - -### 10.2 主要功能模块 - -1. **数据采集任务管理** - - 任务列表、创建、编辑、删除 - - 任务启动、暂停、停止 - - 任务进度查看 - - 数据源选择、数据库/集合选择 - - 字段映射配置 - -2. **标签管理** - - 标签定义管理 - - 用户标签查看 - - 标签筛选功能 - -3. **数据源管理** - - 数据源列表、创建、编辑、删除 - - 连接测试 - ---- - -## 十一、总结 - -### 11.1 系统核心价值 - -1. **统一的数据采集中心** - - 支持多种数据源 - - 支持批量采集和实时监听 - - 配置化管理，易于扩展 - -2. **智能的用户标签引擎** - - 基于规则引擎自动计算标签 - - 支持实时更新和定时更新 - - 标签变更历史追踪 - -3. **灵活的身份管理体系** - - 支持身份证、手机号等多种标识 - - 临时人机制 - - 身份合并功能 - -### 11.2 系统优势 - -- ✅ **高性能**：基于Workerman多进程架构 -- ✅ **高可用**：分布式锁、错误重试、断点续传 -- ✅ **易扩展**：配置化、组件化设计 -- ✅ **可观测**：完善的日志和监控 -- ✅ **安全**：数据加密、权限控制 - -### 11.3 系统流程总结 - -**核心流程：数据源 → 采集任务 → 消息队列 → 数据同步 → 标签计算 → 存储** - -1. 数据采集：从多个数据源采集数据 -2. 身份解析：根据手机号/身份证解析用户ID -3. 数据存储：写入消费记录，更新用户统计 -4. 标签计算：基于用户数据计算标签值 -5. 标签应用：支持标签筛选、人群分析等应用场景 - ---- - -## 十二、扩展建议 - -### 12.1 功能扩展 - -1. **定时批量标签更新** - - 支持daily/weekly频率的标签批量更新 - - 添加定时任务配置 - -2. **标签血缘关系** - - 追踪标签来源和数据血缘 - - 标签影响分析 - -3. **人群分析功能** - - 基于标签的人群分群 - - 人群画像分析 - -4. **数据质量监控** - - 数据采集质量监控 - - 异常数据告警 - -### 12.2 性能优化 - -1. **批量处理优化** - - 增加批量大小 - - 优化数据库查询 - -2. **缓存优化** - - 标签定义缓存 - - 用户数据缓存 - -3. **分片策略优化** - - 按时间分片 - - 按数据库分片 - ---- - -**报告生成时间**: 2025-12-26 -**项目版本**: 基于当前代码库分析 - diff --git a/Moncter/runtime/logs/workerman.log b/Moncter/runtime/logs/workerman.log index 36fd4faa..e5739243 100644 --- a/Moncter/runtime/logs/workerman.log +++ b/Moncter/runtime/logs/workerman.log @@ -7,3 +7,4 @@ 2026-01-05 10:46:34 pid:112703 Workerman[start.php] received signal SIGINT 2026-01-05 10:46:34 pid:112703 Workerman[start.php] stopping 2026-01-05 10:46:34 pid:112703 Workerman[start.php] has been stopped +2026-01-05 10:52:58 pid:115905 Workerman[start.php] start in DEBUG mode