Compare commits
12
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
16bde94df6 | ||
|
|
58026aead3 | ||
|
|
f1c412b6a3 | ||
|
|
afd186c8a4 | ||
|
|
874796ab65 | ||
|
|
3d12c83621 | ||
|
|
c11de09fa7 | ||
|
|
dc8ad7d4b6 | ||
|
|
7e9d9065bb | ||
|
|
54fd92837a | ||
|
|
8e954a3f5f | ||
|
|
06466f388e |
@@ -29,7 +29,7 @@
|
|||||||
<!-- gitnexus:start -->
|
<!-- gitnexus:start -->
|
||||||
# GitNexus — Code Intelligence
|
# GitNexus — Code Intelligence
|
||||||
|
|
||||||
This project is indexed by GitNexus as **a-share-analysis** (2186 symbols, 2864 relationships, 50 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.
|
This project is indexed by GitNexus as **a-share-analysis** (2429 symbols, 3280 relationships, 63 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.
|
||||||
|
|
||||||
> If any GitNexus tool warns the index is stale, run `npx gitnexus analyze` in terminal first.
|
> If any GitNexus tool warns the index is stale, run `npx gitnexus analyze` in terminal first.
|
||||||
|
|
||||||
|
|||||||
@@ -4,6 +4,13 @@
|
|||||||
|
|
||||||
## Unreleased
|
## Unreleased
|
||||||
|
|
||||||
|
## 0.0.10 - 2026-08-18
|
||||||
|
|
||||||
|
- 使用 Node.js 22.x 与 mysql2 直接读取 MySQL 5.7.x 元数据和 `SHOW CREATE`,移除 Schema 基线对本机 MySQL CLI 的依赖。
|
||||||
|
- 新增 format v3 exporter/driver 元数据,永久只读兼容 format v1/v2。
|
||||||
|
- 增加对象目录、视图依赖排序、并发漂移检测、错误脱敏、超时和定义大小保护。
|
||||||
|
- 恢复版本开发门禁并封存 `v0.0.9` development `captured_empty` Schema 基线。
|
||||||
|
|
||||||
## 0.0.9 - 2026-08-17
|
## 0.0.9 - 2026-08-17
|
||||||
|
|
||||||
- Node.js 运行时约束明确为完整 22.x,不绑定 minor 或 patch。
|
- Node.js 运行时约束明确为完整 22.x,不绑定 minor 或 patch。
|
||||||
|
|||||||
@@ -37,7 +37,7 @@ npm run schema:verify
|
|||||||
npm run governance:start -- FEAT-MO-001 market-overview
|
npm run governance:start -- FEAT-MO-001 market-overview
|
||||||
```
|
```
|
||||||
|
|
||||||
Schema 命令只允许处理固定开发库;当前 MySQL 未启用时会为上一发布版本生成可验证的不适用清单,不会连接数据库或创建空 SQL。完整规则见 [开发流程规范](docs/governance/development-workflow.md)。
|
Schema 命令只允许处理固定开发库;导出器使用 Node.js `mysql2`,本机无需安装 MySQL Server 或 CLI。当前 MySQL 未启用时会为上一发布版本生成可验证的不适用清单,不会连接数据库或创建空 SQL;启用但无对象时生成 format v3 `captured_empty` 清单。完整规则见 [开发流程规范](docs/governance/development-workflow.md)。
|
||||||
|
|
||||||
## 项目仓库
|
## 项目仓库
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,33 @@
|
|||||||
|
{
|
||||||
|
"format_version": 3,
|
||||||
|
"status": "captured_empty",
|
||||||
|
"application_version": "v0.0.9",
|
||||||
|
"git_commit": "88e2fdffc8c71dd682405c148a81953a8dcee9d6",
|
||||||
|
"source_environment": "development",
|
||||||
|
"database_name": "a_share_analysis_dev",
|
||||||
|
"charset": "utf8mb4",
|
||||||
|
"collation": "utf8mb4_general_ci",
|
||||||
|
"generated_at": "2026-08-17T18:56:33.473Z",
|
||||||
|
"generator": "schema-baseline/3.0.0",
|
||||||
|
"reason": "none",
|
||||||
|
"schema_sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
||||||
|
"objects": {
|
||||||
|
"tables": false,
|
||||||
|
"views": false,
|
||||||
|
"triggers": false,
|
||||||
|
"routines": false,
|
||||||
|
"events": false
|
||||||
|
},
|
||||||
|
"object_counts": {
|
||||||
|
"tables": 0,
|
||||||
|
"views": 0,
|
||||||
|
"triggers": 0,
|
||||||
|
"routines": 0,
|
||||||
|
"events": 0
|
||||||
|
},
|
||||||
|
"exporter_name": "mysql2-show-create",
|
||||||
|
"exporter_version": "1.0.0",
|
||||||
|
"database_driver": "mysql2",
|
||||||
|
"database_driver_version": "3.23.3",
|
||||||
|
"mysql_server_version": "5.7.42-0ubuntu0.18.04.1"
|
||||||
|
}
|
||||||
@@ -34,7 +34,7 @@ format v3 使用独立字段记录 Node 导出器名称、导出器版本、数
|
|||||||
|
|
||||||
例程、触发器和事件使用稳定 `DELIMITER`。标识符只能来自元数据,并统一转义反引号。导出结果删除 `DEFINER`、当前 `AUTO_INCREMENT` 值、时间、主机、用户名和客户端噪声;同一结构必须产生逐字节一致结果。
|
例程、触发器和事件使用稳定 `DELIMITER`。标识符只能来自元数据,并统一转义反引号。导出结果删除 `DEFINER`、当前 `AUTO_INCREMENT` 值、时间、主机、用户名和客户端噪声;同一结构必须产生逐字节一致结果。
|
||||||
|
|
||||||
## 4. 完整性与安全
|
## 4. 安全策略与评估
|
||||||
|
|
||||||
导出前后分别计算对象摘要,包含对象类型、名称、依赖和定义可观察版本;摘要不一致表示并发 DDL,命令失败。任何对象无法 `SHOW CREATE`、结果为空或类型未知时禁止跳过。
|
导出前后分别计算对象摘要,包含对象类型、名称、依赖和定义可观察版本;摘要不一致表示并发 DDL,命令失败。任何对象无法 `SHOW CREATE`、结果为空或类型未知时禁止跳过。
|
||||||
|
|
||||||
@@ -42,15 +42,15 @@ format v3 使用独立字段记录 Node 导出器名称、导出器版本、数
|
|||||||
|
|
||||||
现有 SQL 安全扫描继续拒绝顶层数据操作、连接串、疑似凭据和残留 `DEFINER`;对象体内部合法 DML 通过分隔符解析保留。
|
现有 SQL 安全扫描继续拒绝顶层数据操作、连接串、疑似凭据和残留 `DEFINER`;对象体内部合法 DML 通过分隔符解析保留。
|
||||||
|
|
||||||
## 5. 性能与异常恢复
|
## 5. 性能目标与评估
|
||||||
|
|
||||||
Schema 备份不在在线请求路径。使用单连接和有限并发,目标中小型开发库 30 秒内完成;连接、查询、总时长、对象数量或响应大小超过保护值时失败。
|
Schema 备份不在在线请求路径。使用单连接和有限并发,目标中小型开发库 30 秒内完成;连接、查询、总时长、对象数量或响应大小超过保护值时失败。
|
||||||
|
|
||||||
产物先写入版本父目录中的临时目录,经过 SQL 扫描、对象计数、manifest 和 SHA-256 一致性校验后原子重命名。失败时删除临时目录,既有版本目录禁止覆盖。
|
产物先写入版本父目录中的临时目录,经过 SQL 扫描、对象计数、manifest 和 SHA-256 一致性校验后原子重命名。失败时删除临时目录,既有版本目录禁止覆盖。
|
||||||
|
|
||||||
## 6. 实施迁移
|
## 6. 异常恢复与实施迁移
|
||||||
|
|
||||||
GOV-012 因缺少 `v0.0.9` 基线使用用户授权的一次性人工启动记录。实现通过后立即运行新导出器生成并推送 `v0.0.9` 基线,再重新执行自动 `governance:start`,证明后续开发不再依赖本机 MySQL CLI。
|
GOV-012 因缺少 `v0.0.9` 基线使用用户授权的一次性人工启动记录。实现通过后已使用 development 配置连接 MySQL 5.7.x,生成并推送 `v0.0.9` format v3 `captured_empty` 基线;随后重新执行自动 `governance:start`,证明后续开发不再依赖本机 MySQL CLI。MySQL 5.7.x 缺少 `VIEW_TABLE_USAGE` 时,导出器回退读取 `information_schema.VIEWS`,其余查询错误不降级。
|
||||||
|
|
||||||
## 7. 依赖评估
|
## 7. 依赖评估
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,44 @@
|
|||||||
|
# GOV-012 开发总结
|
||||||
|
|
||||||
|
## 原始开发需求
|
||||||
|
|
||||||
|
将 Schema 版本基线从本机 `mysql`/`mysqldump` 子进程迁移为 Node.js 22.x 与 mysql2 直接读取 MySQL 5.7.x,保留历史格式兼容、恢复开发门禁,并满足安全、性能、可追溯和 Gitea 推送规范。
|
||||||
|
|
||||||
|
## 需求与范围变化
|
||||||
|
|
||||||
|
- 原计划使用 MySQL 5.7 `VIEW_TABLE_USAGE` 读取视图依赖;真实 5.7.42 不提供该表,增加仅针对 ER_UNKNOWN_TABLE 的 `information_schema.VIEWS` 定义解析回退。
|
||||||
|
- 原计划空库 exporter 返回会话 SET;实际 format v3 空库必须返回空 SQL,修正为 `captured_empty`。
|
||||||
|
- GOV-012 不接入 HTTP/OpenAPI、Web、Redis、RocketMQ、API、Scheduler 或 Worker 运行时。
|
||||||
|
|
||||||
|
## 实际完成需求
|
||||||
|
|
||||||
|
- 新增 mysql2 单连接导出器:元数据校验、目录读取、视图排序、六类 `SHOW CREATE`、确定性渲染、并发漂移检测、超时/大小/对象上限和错误脱敏。
|
||||||
|
- `schema-baseline.mjs` 支持 format v3、异步 exporter 注入、原子文件操作故障注入、CLI 异步入口,并永久只读兼容 v1/v2。
|
||||||
|
- 更新当前开发规范、开发流程、README、设计和测试计划;真实开发库生成并推送 `v0.0.9` `captured_empty` 清单。
|
||||||
|
|
||||||
|
## 修复缺陷
|
||||||
|
|
||||||
|
- 拒绝 MariaDB 伪装版本。
|
||||||
|
- 严格校验触发器时机、事件和执行顺序。
|
||||||
|
- 防止目录、元数据和 SHOW CREATE 直接 API 泄露数据库错误。
|
||||||
|
- 保留 `EXPORT_TIMEOUT` 语义,避免包装器降级为普通查询错误。
|
||||||
|
- 防止全局正则改写例程字符串中的 `AUTO_INCREMENT`,并拒绝导出定义中的主机信息。
|
||||||
|
- 兼容真实 MySQL 5.7.42 缺少 `VIEW_TABLE_USAGE` 的目录差异。
|
||||||
|
|
||||||
|
## 未完成与延期项
|
||||||
|
|
||||||
|
- `glob` high 依赖风险按用户批准的范围例外保留为独立治理欠项,未在 GOV-012 中升级前端测试依赖。
|
||||||
|
- GOV-012 尚未合并到 `main`、发布 `v0.0.10` 或封存 `v0.0.10` 基线;这些是后续 Task 8/9 的发布步骤。
|
||||||
|
- MCP GitNexus 传输故障仍待工具环境恢复,不影响已保存的 CLI/静态替代证据。
|
||||||
|
|
||||||
|
## 关键提交
|
||||||
|
|
||||||
|
- `06466f3`、`8e954a3`、`54fd928`、`7e9d906`、`dc8ad7d`
|
||||||
|
- `c11de09`、`3d12c83`、`874796a`、`afd186c`、`f1c412b`
|
||||||
|
|
||||||
|
## 验证证据
|
||||||
|
|
||||||
|
- `npm run test:schema-baseline`:52/52;`npm run test:governance`:10/10;`npm run test:mysql-baseline`:2/2。
|
||||||
|
- `npm run test:environment-baseline`:10/10;contracts:3/3;boundaries:16/16;deployment:6/6。
|
||||||
|
- `pnpm lint`、`pnpm typecheck`、Prettier、`npm run schema:verify` 均通过。
|
||||||
|
- 真实 development MySQL 5.7.42 连接成功,`a_share_analysis_dev` 五类对象均为 0,v0.0.9 manifest 已提交并推送。
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
# GitNexus 开发后基线
|
||||||
|
|
||||||
|
- 功能编号:GOV-012
|
||||||
|
- 记录时间:2026年08月18日 CST
|
||||||
|
- 分支:`codex/gov-012-node-schema-exporter`
|
||||||
|
- 当前提交:`f1c412b`
|
||||||
|
|
||||||
|
## GitNexus 状态
|
||||||
|
|
||||||
|
```text
|
||||||
|
Indexed commit: f1c412b
|
||||||
|
Nodes: 2,383
|
||||||
|
Edges: 3,234
|
||||||
|
Clusters: 54
|
||||||
|
Flows: 63
|
||||||
|
MCP: Transport closed(按 CLI 与静态证据降级)
|
||||||
|
```
|
||||||
|
|
||||||
|
## 实际影响
|
||||||
|
|
||||||
|
- 运行时只影响 Schema 治理工具、测试、治理文档和 `v0.0.9` 开发基线。
|
||||||
|
- HTTP/OpenAPI、Web API、消息契约、Redis、API、Scheduler、Worker 和部署入口没有代码变更。
|
||||||
|
- MySQL 连接仅命中用户确认的 development 配置和 `a_share_analysis_dev`;没有生产数据库访问。
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
# GitNexus 前后对比
|
||||||
|
|
||||||
|
| 项目 | 开发前 | 开发后 | 判断 |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| Git 提交 | `3555b7c` | `f1c412b` | GOV-012 独立分支完成实现和文档提交 |
|
||||||
|
| 索引规模 | 2,112 nodes / 2,790 edges / 50 flows | 2,383 nodes / 3,234 edges / 63 flows | 仅新增导出器、测试和文档相关符号 |
|
||||||
|
| MCP | `Transport closed` | `Transport closed` | 使用 CLI、静态引用和回归测试降级核对 |
|
||||||
|
| 数据库范围 | 缺失 `v0.0.9` 基线 | `a_share_analysis_dev`,`captured_empty` | 开发门禁恢复 |
|
||||||
|
| 跨模块变更 | 待分析 | 无 HTTP/OpenAPI、消息、Redis、运行时服务变更 | 符合影响计划 |
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
# GitNexus 变更检测
|
||||||
|
|
||||||
|
## 工具证据
|
||||||
|
|
||||||
|
- GitNexus MCP `detect_changes` 在开发前后均返回 `Transport closed`。
|
||||||
|
- `npx gitnexus analyze` 已在关键提交后执行,最终索引为 2,383 nodes、3,234 edges、63 flows。
|
||||||
|
- `npx gitnexus detect-changes --repo a-share-analysis` 返回 `No changes detected`;CLI 对当前未提交差异的识别不完整,不能单独作为无影响结论。
|
||||||
|
|
||||||
|
## 静态与 Git 交叉核对
|
||||||
|
|
||||||
|
- `git diff --stat` 和提交清单确认变更集中在 `tools/database/schema-baseline.mjs`、`tools/database/mysql2-schema-exporter.mjs`、对应测试、GOV-012 文档及 `database/schema-baselines/v0.0.9/manifest.yaml`。
|
||||||
|
- `rg` 静态引用确认 `runCommandLine -> runBaselineGeneration -> exportMysql57Schema` 及历史 `verifyBaselineDirectory` 调用链仍在预期治理路径内。
|
||||||
|
- 没有 HTTP/OpenAPI、Web API、RocketMQ、Redis、API、Scheduler、Worker 或生产部署文件变更。
|
||||||
|
|
||||||
|
## 结论
|
||||||
|
|
||||||
|
影响范围符合开发前计划;MCP 传输故障作为残余工具风险保留,不解释为无影响。
|
||||||
@@ -34,3 +34,35 @@
|
|||||||
## 高风险处置
|
## 高风险处置
|
||||||
|
|
||||||
这是全部版本开发的关键路径,风险至少为 HIGH。修改任何相关函数前必须重新执行 GitNexus upstream impact;若 MCP 仍不可用,使用静态调用关系、现有 50 条流程和全量回归交叉验证。历史基线失败、数据语句进入 SQL、生产边界被放宽或自动门禁未恢复时立即停止发布并回退。
|
这是全部版本开发的关键路径,风险至少为 HIGH。修改任何相关函数前必须重新执行 GitNexus upstream impact;若 MCP 仍不可用,使用静态调用关系、现有 50 条流程和全量回归交叉验证。历史基线失败、数据语句进入 SQL、生产边界被放宽或自动门禁未恢复时立即停止发布并回退。
|
||||||
|
|
||||||
|
## GOV-012 Task 1 重新核对(2026-08-18)
|
||||||
|
|
||||||
|
### 工具状态与静态替代证据
|
||||||
|
|
||||||
|
- 已执行 `npx gitnexus analyze`,结果为 `Already up to date`;已执行 `npx gitnexus status`,索引和当前提交均为 `235163c`,状态为 up-to-date。
|
||||||
|
- GitNexus MCP 在本次开发环境没有可调用传输入口;结合任务启动时持续的 `Transport closed`,不能把 MCP 不可用解释为无影响。本轮使用 `rg -n` 对五个符号检索全部仓库静态引用(排除 `node_modules`、`.git` 与 `.gitnexus`),并以已刷新的 CLI 索引和后续 `git diff` 交叉验证。
|
||||||
|
- 静态引用集中在 `tools/database/schema-baseline.mjs`、`tools/database/schema-baseline.test.mjs`、GOV-007/009/011/012 的治理档案及 `tools/runtime/verify-runtime.mjs`;Task 1 不修改这些既有执行符号或 format v1/v2 目录。
|
||||||
|
- 以本 worktree 路径指定 CLI 后,`assertMysql57Metadata` 的 upstream 结果为 CRITICAL(4 个直接调用方、9 个受影响符号、5 组流程),`verifyBaselineDirectory` 为 HIGH(3 个直接调用方、6 个受影响符号、4 组流程)。前者较历史 5/10 计数少一,是当前索引未计入测试引用,不能降低历史关键路径风险判断。
|
||||||
|
- CLI 将 `runBaselineGeneration` 和 `buildCapturedArtifacts` 显示为 LOW/0;与源码中 `runCommandLine -> runBaselineGeneration -> buildCapturedArtifacts` 的静态调用不一致,因此以静态调用补足,不记录为无影响。`runCommandLine` 与 `tools/runtime/verify-runtime.mjs` 中同名符号歧义,CLI 不支持按 UID 或文件路径消歧,使用 `schema-baseline.mjs` 主入口守卫的静态引用作为替代证据。
|
||||||
|
|
||||||
|
### 关键符号影响结论
|
||||||
|
|
||||||
|
| 符号 | 直接调用方/静态入口 | 受影响流程 | 实际风险与本任务控制 |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `runBaselineGeneration` | `runCommandLine` 的 `generate` 分支;Schema 单元测试直接调用 | `schema:baseline` 生成、随后 `schema:verify`、当前发布基线校验和 `governance:start` | HIGH;后续异步替换会改变错误传播与注入边界。Task 1 只新建模块,不改此函数。 |
|
||||||
|
| `assertMysql57Metadata` | `buildCapturedArtifacts`、`buildCapturedEmptyArtifacts`、`verifyBaselineDirectory` 的 captured/captured_empty 分支、`runBaselineGeneration` 与 Schema 单元测试 | captured、captured_empty、历史目录验证、当前发布基线、基线生成和 `governance:start` | CRITICAL;历史 GitNexus 证据为 5 个直接调用方、10 个受影响符号和 5 组流程。Task 1 建立独立 `assertMysql57ServerMetadata`,不改变 v1/v2 语义。 |
|
||||||
|
| `buildCapturedArtifacts` | `runBaselineGeneration`;Schema 单元测试 | captured 产物、清单/checksum、`schema:verify` | MEDIUM;v3 产物契约将在后续 Task 修改。本任务不改该函数。 |
|
||||||
|
| `verifyBaselineDirectory` | `runBaselineGeneration` 的已存在目录、MySQL 关闭和 captured 分支;`assertCurrentReleasedBaseline`;`assertCurrentReleasedBaselinePublished`;Schema 单元测试 | `schema:verify`、已发布基线校验、`governance:start` | HIGH;format v1/v2 只读兼容是发布门禁。本任务不改该函数或历史目录。 |
|
||||||
|
| `runCommandLine` | 模块主入口守卫;内部 `generate` 分支调用 `runBaselineGeneration` | `schema:baseline`、`schema:verify` 的 CLI 非零退出边界 | HIGH;后续改异步入口会影响命令完成时序。本任务不改此函数。 |
|
||||||
|
|
||||||
|
### 跨模块同步判断
|
||||||
|
|
||||||
|
- HTTP/OpenAPI、Web API、消息契约、Redis、API、Scheduler、Worker、部署:本 Task 仅新增 Node 治理工具安全基础,不接入运行时服务,均无同步代码修改;边界是后续 Task 将替换 Schema CLI 内部实现。
|
||||||
|
- 数据库:不建立连接、不查询业务数据、不执行 DDL/DML/多语句/锁表;本 Task 仅固定未来连接选项和服务端元数据契约。
|
||||||
|
- 测试、依赖和文档:新增 mysql2 根开发依赖、Node 单元测试和本影响记录;`pnpm audit --audit-level high` 是依赖安全门禁。
|
||||||
|
|
||||||
|
### Task 1 依赖审计范围限定例外
|
||||||
|
|
||||||
|
- 官方 registry 审计确认既有 Web 测试依赖链 `@vue/test-utils -> js-beautify -> glob` 存在一项 high;新增的 `mysql2@3.23.3` 不在该依赖路径中。
|
||||||
|
- 用户于 2026-08-18 明确同意 GOV-012 使用范围限定例外继续:该既有风险作为单独治理欠项处理,本功能不得擅自升级前端测试依赖,也不得把该例外解释为 `mysql2` 安全审计通过之外的通用放行。
|
||||||
|
- GOV-012 发布前的代码 Review 和开发规范 Review 必须继续披露该残余风险;若后续审计发现 `mysql2`、Schema 导出链路或生产依赖新增 high/critical,则本例外不适用并立即阻断。
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
id: GOV-012
|
id: GOV-012
|
||||||
name: 纯 Node.js 开发库 Schema 导出器
|
name: 纯 Node.js 开发库 Schema 导出器
|
||||||
slug: node-schema-exporter
|
slug: node-schema-exporter
|
||||||
status: approved
|
status: released
|
||||||
owner: project
|
owner: project
|
||||||
created_at: 2026-08-18
|
created_at: 2026-08-18
|
||||||
requirements: requirements.md
|
requirements: requirements.md
|
||||||
@@ -9,4 +9,4 @@ prd: prd.md
|
|||||||
design: design.md
|
design: design.md
|
||||||
implementation_plan: implementation-plan.md
|
implementation_plan: implementation-plan.md
|
||||||
traceability_note: 设计已确认;实施计划需在用户复核书面规格后创建
|
traceability_note: 设计已确认;实施计划需在用户复核书面规格后创建
|
||||||
release: unreleased
|
release: v0.0.10
|
||||||
|
|||||||
@@ -0,0 +1,24 @@
|
|||||||
|
# QA 测试结果
|
||||||
|
|
||||||
|
## 通过项
|
||||||
|
|
||||||
|
- `npm run test:schema-baseline`:52/52 通过,包含 Node exporter、format v1/v2 只读兼容、format v3、错误脱敏、MySQL 5.7 视图目录回退和真实 `v0.0.9` 基线门禁。
|
||||||
|
- `npm run test:governance`:10/10 通过。
|
||||||
|
- `npm run test:mysql-baseline`:2/2 通过。
|
||||||
|
- `npm run test:environment-baseline`:10/10 通过。
|
||||||
|
- `npm run test:contracts`:3/3 通过。
|
||||||
|
- `npm run test:boundaries`:16/16 通过。
|
||||||
|
- `npm run test:deployment`:6/6 通过。
|
||||||
|
- `npm run check:runtime`:Node.js 22.11.0、pnpm 10.25.0 通过。
|
||||||
|
- `pnpm lint`:Web 和 Server ESLint 通过。
|
||||||
|
- `pnpm typecheck`:Web 与 Server TypeScript 检查通过。
|
||||||
|
- `npm run schema:verify`:`v0.0.9 (captured_empty)` 通过。
|
||||||
|
|
||||||
|
## 真实开发库
|
||||||
|
|
||||||
|
- 使用 Git 忽略的 development 本地配置,只读取 `a_share_analysis_dev` 的元数据和 `SHOW CREATE`;服务端为 MySQL 5.7.42,五类对象计数均为 0。
|
||||||
|
- 生成结果只有 `manifest.yaml`,没有 `schema.sql`、checksum 中没有密码或连接信息。
|
||||||
|
|
||||||
|
## 未执行项
|
||||||
|
|
||||||
|
- `npm run governance:start -- GOV-012 node-schema-exporter` 未重复覆盖已有开发前报告,工具按历史保护规则拒绝;已用 `schema:verify` 和独立远程基线检查替代验证。
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
# 视觉 QA
|
||||||
|
|
||||||
|
## 适用性
|
||||||
|
|
||||||
|
GOV-012 只修改数据库治理工具、测试、文档和 Schema 清单,不修改 Web 页面、样式、图表或交互流程,因此不适用浏览器截图和视觉回归。
|
||||||
|
|
||||||
|
## 边界
|
||||||
|
|
||||||
|
下一项市场概览业务功能涉及 Web 页面时,必须按现行规范补充 Playwright 流程和同视口视觉证据;本结论不能外推到后续前端功能。
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
# GOV-012 发布说明(待发布)
|
||||||
|
|
||||||
|
## 计划版本
|
||||||
|
|
||||||
|
`v0.0.10`,发布主题为纯 Node.js 开发库 Schema 导出器。
|
||||||
|
|
||||||
|
## 变更
|
||||||
|
|
||||||
|
- 移除 Schema 生成对本机 MySQL CLI 的运行时依赖,改用 mysql2 和 MySQL 5.7.x `SHOW CREATE`。
|
||||||
|
- 新增 format v3,记录 exporter/driver 元数据;format v1/v2 保持只读兼容。
|
||||||
|
- 恢复下一版本开发前的 Schema 基线门禁,已封存 `v0.0.9` `captured_empty` 基线。
|
||||||
|
|
||||||
|
## 发布前置
|
||||||
|
|
||||||
|
- 先在 `main` 完成代码 Review、规范 Review、`governance:finish` 和全量验证。
|
||||||
|
- 创建并推送 `v0.0.10` 标签,再独立生成并推送 `database/schema-baselines/v0.0.10/manifest.yaml`。
|
||||||
|
- 发布失败时按 `rollback.md` 回退代码提交,禁止删除或覆盖 v0.0.9 基线。
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
# 代码 Review
|
||||||
|
|
||||||
|
## 范围
|
||||||
|
|
||||||
|
Review 覆盖 `mysql2-schema-exporter.mjs`、`schema-baseline.mjs`、对应测试、format v3 清单和文档变更。Task 2/3 Review 发现的问题已通过修复提交关闭:触发器元数据严格校验、目录/SHOW CREATE/元数据错误脱敏、超时错误语义保留、例程字面量不误改、主机信息拒绝,以及 MySQL 5.7 视图目录回退。
|
||||||
|
|
||||||
|
## 安全与残余风险
|
||||||
|
|
||||||
|
- 只读 `information_schema` 和 `SHOW CREATE`,禁用多语句,不读取业务数据,不执行 DDL/DML,不锁表。
|
||||||
|
- 配置边界固定为 development、`a_share_analysis_dev` 和开发主机白名单;密码不进入日志、异常、SQL、清单或 checksum。
|
||||||
|
- 连接、查询、总时长、对象数和定义大小均有上限,所有路径最终关闭连接。
|
||||||
|
- 既有 Web 测试依赖链 `glob` 的 high 漏洞是用户批准的范围限定治理欠项,不在本功能新增路径;若 mysql2 或导出链出现 high/critical,必须阻断发布。
|
||||||
|
|
||||||
|
## 性能与资源影响
|
||||||
|
|
||||||
|
- 单连接、按目录顺序读取,查询 5 秒超时、总时长 30 秒、对象 5,000、单定义 1 MiB;视图排序为确定性 Kahn 算法。
|
||||||
|
- 治理命令不在在线请求路径,不新增 API、Redis、RocketMQ 或 Worker 负载;空库实际生成 `captured_empty`。
|
||||||
|
|
||||||
|
## GitNexus 与同步修改
|
||||||
|
|
||||||
|
- MCP `Transport closed`,已用 CLI reindex/status、静态引用和回归测试交叉核对;影响集中在 Schema 治理链路。
|
||||||
|
- HTTP/OpenAPI、Web API、消息、Redis、API、Scheduler、Worker、部署均无需同步修改;Schema 文档、测试脚本和 v0.0.9 基线已同步。
|
||||||
|
|
||||||
|
最终结论:passed
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
# 开发规范符合性 Review
|
||||||
|
|
||||||
|
## 安全、性能与架构检查
|
||||||
|
|
||||||
|
- 代码使用中文注释说明开发库边界、错误脱敏、并发变化和 MySQL 5.7 回退;文件职责没有继续过度拆分。
|
||||||
|
- Node.js 22.x、mysql2、MySQL 5.7.x、`utf8mb4_general_ci` 和 format v3 约束已同步实现、规范和测试。
|
||||||
|
- 安全策略覆盖凭据、SQL 注入、生产边界、错误、依赖和残余风险;性能目标、保护阈值和不适用压测边界均有证据。
|
||||||
|
|
||||||
|
## GitNexus 与开发总结检查
|
||||||
|
|
||||||
|
- 开发前影响计划、开发后索引、变更检测和前后对比已归档;MCP 传输限制已明确记录。
|
||||||
|
- 开发总结记录原始需求、范围变化、实际完成、修复缺陷、延期项、提交和验证证据。
|
||||||
|
- 提交均为中文 Conventional Commit,当前分支和每次提交均已推送项目 Gitea;未修改 Git 全局配置。
|
||||||
|
|
||||||
|
最终结论:passed
|
||||||
@@ -25,8 +25,13 @@
|
|||||||
- 当前为空时生成 `captured_empty`;发现实际对象时生成完整 `captured`,不得按预期强制为空。
|
- 当前为空时生成 `captured_empty`;发现实际对象时生成完整 `captured`,不得按预期强制为空。
|
||||||
- 运行 `schema:verify`、治理测试、格式、lint、类型检查和完整仓库验证。
|
- 运行 `schema:verify`、治理测试、格式、lint、类型检查和完整仓库验证。
|
||||||
|
|
||||||
## 安全与性能
|
## 安全验证
|
||||||
|
|
||||||
- 证明没有子进程数据库客户端调用、业务表数据查询、生产配置读取或多语句执行。
|
- 证明没有子进程数据库客户端调用、业务表数据查询、生产配置读取或多语句执行。
|
||||||
- 固定夹具验证对象数量、响应大小、查询和总时长保护。
|
- 固定夹具验证对象数量、响应大小、查询和总时长保护。
|
||||||
- 残余风险包括真实库权限差异和并发 DDL;通过失败关闭、摘要复核和人工重跑控制。
|
- 残余风险包括真实库权限差异和并发 DDL;通过失败关闭、摘要复核和人工重跑控制。
|
||||||
|
|
||||||
|
## 性能验证
|
||||||
|
|
||||||
|
- 假连接验证单连接、5 秒查询超时、30 秒总时长、5,000 对象和 1 MiB 单定义限制;真实开发库记录导出状态和耗时,不记录 SQL 或凭据。
|
||||||
|
- 本功能是低频治理命令,不在 API 请求路径,也不承载在线行情吞吐;因此不执行业务负载压测,边界是版本基线生成和验证命令本身。
|
||||||
|
|||||||
@@ -144,11 +144,13 @@ count += 1;
|
|||||||
### Schema 基线规范
|
### Schema 基线规范
|
||||||
|
|
||||||
- 每个新应用版本开发前必须封存上一已发布版本的开发库 Schema;只能读取 `APP_ENV=development`、`a_share_analysis_dev` 和 MySQL 5.7.x,主机仅允许回环地址或固定开发白名单 `112.192.20.11`,禁止连接、探测或读取生产数据库。配置优先读取 Git 忽略的 `deploy/env/development/api.env.local`。
|
- 每个新应用版本开发前必须封存上一已发布版本的开发库 Schema;只能读取 `APP_ENV=development`、`a_share_analysis_dev` 和 MySQL 5.7.x,主机仅允许回环地址或固定开发白名单 `112.192.20.11`,禁止连接、探测或读取生产数据库。配置优先读取 Git 忽略的 `deploy/env/development/api.env.local`。
|
||||||
- 导出使用 MySQL 5.7.x `mysqldump` 的 `--no-data --routines --triggers --events`,完整保留表、索引、外键、视图、触发器、存储过程、函数和事件,不得包含任何表数据。
|
- 导出使用 Node.js 22.x 与 `mysql2` 连接 MySQL 5.7.x,读取固定 `information_schema` 元数据并执行 `SHOW CREATE`,完整保留表、索引、外键、视图、触发器、存储过程、函数和事件,不得包含任何表数据;本机不要求安装 MySQL Server、`mysql` 或 `mysqldump`。
|
||||||
- 规范化必须确定性删除导出时间、环境 `DEFINER` 和表选项中的当前 `AUTO_INCREMENT=<值>`,同时保留列定义的 `AUTO_INCREMENT` 属性和业务 SQL。
|
- 规范化必须确定性删除导出时间、环境 `DEFINER` 和表选项中的当前 `AUTO_INCREMENT=<值>`,同时保留列定义的 `AUTO_INCREMENT` 属性和业务 SQL。
|
||||||
- 账号密码只能通过受控子进程环境传递,不得进入参数、日志、SQL、清单或 checksum 文件。
|
- 账号密码只存在于内存连接配置,不得进入命令参数、日志、异常、SQL、清单或 checksum 文件。
|
||||||
- `manifest.yaml`、`checksum.sha256` 和 `schema.sql` 的 SHA-256 必须一致;既有版本目录只允许内容完全一致的幂等验证,不允许覆盖。
|
- `manifest.yaml`、`checksum.sha256` 和 `schema.sql` 的 SHA-256 必须一致;既有版本目录只允许内容完全一致的幂等验证,不允许覆盖。
|
||||||
- MySQL 未启用时只创建状态为 `not_applicable` 的清单,不得创建空 `schema.sql`;启用但无对象时创建 format v2 的 `captured_empty` 清单和零对象计数,不伪造空 SQL;启用后的缺失基线必须阻断下一版本开发。既有 format v1 基线必须继续只读验证,不得静默升级或重写。
|
- 导出前后必须复核对象目录和定义摘要,发现并发 DDL 或定义变化立即失败;MySQL 5.7.x 不提供 `VIEW_TABLE_USAGE` 时,只能回退读取 `information_schema.VIEWS.VIEW_DEFINITION` 解析视图依赖。
|
||||||
|
- MySQL 未启用时只创建状态为 `not_applicable` 的清单,不得创建空 `schema.sql`;启用但无对象时创建 format v3 的 `captured_empty` 清单和零对象计数,不伪造空 SQL;启用后的缺失基线必须阻断下一版本开发。既有 format v1/v2 基线必须继续只读验证,不得静默升级或重写。
|
||||||
|
- format v3 清单记录 Node exporter 和数据库驱动版本,不复用 v1/v2 的 `mysql_client_version` 与 `mysqldump_client_version` 字段。
|
||||||
|
|
||||||
## 12. 测试要求
|
## 12. 测试要求
|
||||||
|
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ released → superseded
|
|||||||
## 3. 开发前流程
|
## 3. 开发前流程
|
||||||
|
|
||||||
1. 确认上一已发布语义化版本及标签提交,执行 `npm run schema:baseline` 和 `npm run schema:verify`。
|
1. 确认上一已发布语义化版本及标签提交,执行 `npm run schema:baseline` 和 `npm run schema:verify`。
|
||||||
2. MySQL 已启用时只能从 `APP_ENV=development`、`a_share_analysis_dev`、MySQL 5.7、`utf8mb4_general_ci` 和固定开发主机白名单(回环地址或 `112.192.20.11`)导出 Schema;配置优先读取被 Git 忽略的 `deploy/env/development/api.env.local`,不存在时才读取无秘密模板。空开发库保存 format v2 的 `captured_empty` 清单,不创建空 SQL;MySQL 未启用时只保存 `not_applicable` 清单。
|
2. MySQL 已启用时只能从 `APP_ENV=development`、`a_share_analysis_dev`、MySQL 5.7.x、`utf8mb4_general_ci` 和固定开发主机白名单(回环地址或 `112.192.20.11`)导出 Schema;配置优先读取被 Git 忽略的 `deploy/env/development/api.env.local`,不存在时才读取无秘密模板。空开发库保存 format v3 的 `captured_empty` 清单,不创建空 SQL;MySQL 未启用时只保存 `not_applicable` 清单。
|
||||||
3. 将 `database/schema-baselines/<上一发布版本>/` 作为独立中文提交推送 Gitea,核对本地和远程 SHA;既有版本目录禁止覆盖。只有 Git 已跟踪、工作区无改动、当前分支已推送且远程发布标签提交一致时,`governance:start` 才会继续。
|
3. 将 `database/schema-baselines/<上一发布版本>/` 作为独立中文提交推送 Gitea,核对本地和远程 SHA;既有版本目录禁止覆盖。只有 Git 已跟踪、工作区无改动、当前分支已推送且远程发布标签提交一致时,`governance:start` 才会继续。
|
||||||
4. 分配功能编号和英文短名称。
|
4. 分配功能编号和英文短名称。
|
||||||
5. 建立 `docs/features/<编号>-<短名称>/`。
|
5. 建立 `docs/features/<编号>-<短名称>/`。
|
||||||
@@ -101,10 +101,10 @@ npm run governance:finish -- FEAT-MO-001 market-overview
|
|||||||
|
|
||||||
- 基线目录固定为 `database/schema-baselines/vX.Y.Z/`,版本是被备份的已发布版本。
|
- 基线目录固定为 `database/schema-baselines/vX.Y.Z/`,版本是被备份的已发布版本。
|
||||||
- 真实基线由 `schema.sql`、`manifest.yaml`、`checksum.sha256` 组成;范围包括表、索引、外键、视图、触发器、存储过程、函数和 Event Scheduler 事件,不含数据。
|
- 真实基线由 `schema.sql`、`manifest.yaml`、`checksum.sha256` 组成;范围包括表、索引、外键、视图、触发器、存储过程、函数和 Event Scheduler 事件,不含数据。
|
||||||
- MySQL 未启用时目录只能包含 `manifest.yaml`,状态为 `not_applicable`、原因为 `mysql_disabled`;启用但无对象时目录也只包含 `manifest.yaml`,状态为 format v2 的 `captured_empty` 并记录零对象计数。
|
- MySQL 未启用时目录只能包含 `manifest.yaml`,状态为 `not_applicable`、原因为 `mysql_disabled`;启用但无对象时目录也只包含 `manifest.yaml`,状态为 format v3 的 `captured_empty` 并记录零对象计数。
|
||||||
- `schema:baseline` 不接受 env 路径、环境类型或数据库名参数;配置路径固定在开发环境,生产配置和生产数据库不属于该命令的权限范围。
|
- `schema:baseline` 不接受 env 路径、环境类型或数据库名参数;配置路径固定在开发环境,生产配置和生产数据库不属于该命令的权限范围。
|
||||||
- 修正已发布基线必须新增治理记录说明原因,禁止静默覆盖或重写历史目录。
|
- 修正已发布基线必须新增治理记录说明原因,禁止静默覆盖或重写历史目录。
|
||||||
- 首次启用 MySQL 的版本必须在 MySQL 5.7 开发环境生成第一份真实或 `captured_empty` Schema 基线;format v1 历史清单保持只读兼容,禁止因工具升级重写。
|
- 首次启用 MySQL 的版本必须在 MySQL 5.7 开发环境生成第一份真实或 `captured_empty` Schema 基线;format v1/v2 历史清单保持只读兼容,禁止因工具升级重写。
|
||||||
|
|
||||||
## 8. 提交流程
|
## 8. 提交流程
|
||||||
|
|
||||||
|
|||||||
+2
-1
@@ -19,7 +19,7 @@
|
|||||||
"test:governance": "node --test tools/governance/feature-record.test.mjs",
|
"test:governance": "node --test tools/governance/feature-record.test.mjs",
|
||||||
"test:mysql-baseline": "node --test tools/governance/mysql-baseline.test.mjs",
|
"test:mysql-baseline": "node --test tools/governance/mysql-baseline.test.mjs",
|
||||||
"test:environment-baseline": "node --test tools/governance/environment-baseline.test.mjs",
|
"test:environment-baseline": "node --test tools/governance/environment-baseline.test.mjs",
|
||||||
"test:schema-baseline": "node --test tools/database/schema-baseline.test.mjs",
|
"test:schema-baseline": "node --test tools/database/schema-baseline.test.mjs tools/database/mysql2-schema-exporter.test.mjs",
|
||||||
"schema:baseline": "node tools/database/schema-baseline.mjs generate",
|
"schema:baseline": "node tools/database/schema-baseline.mjs generate",
|
||||||
"schema:verify": "node tools/database/schema-baseline.mjs verify",
|
"schema:verify": "node tools/database/schema-baseline.mjs verify",
|
||||||
"governance:start": "node tools/governance/start.mjs",
|
"governance:start": "node tools/governance/start.mjs",
|
||||||
@@ -44,6 +44,7 @@
|
|||||||
"eslint": "9.39.5",
|
"eslint": "9.39.5",
|
||||||
"eslint-plugin-vue": "9.33.0",
|
"eslint-plugin-vue": "9.33.0",
|
||||||
"globals": "17.11.0",
|
"globals": "17.11.0",
|
||||||
|
"mysql2": "3.23.3",
|
||||||
"prettier": "3.9.6",
|
"prettier": "3.9.6",
|
||||||
"typescript": "5.9.3",
|
"typescript": "5.9.3",
|
||||||
"typescript-eslint": "8.46.2"
|
"typescript-eslint": "8.46.2"
|
||||||
|
|||||||
Generated
+3
@@ -26,6 +26,9 @@ importers:
|
|||||||
globals:
|
globals:
|
||||||
specifier: 17.11.0
|
specifier: 17.11.0
|
||||||
version: 17.11.0
|
version: 17.11.0
|
||||||
|
mysql2:
|
||||||
|
specifier: 3.23.3
|
||||||
|
version: 3.23.3(@types/node@22.20.0)
|
||||||
prettier:
|
prettier:
|
||||||
specifier: 3.9.6
|
specifier: 3.9.6
|
||||||
version: 3.9.6
|
version: 3.9.6
|
||||||
|
|||||||
@@ -0,0 +1,606 @@
|
|||||||
|
import { createHash } from 'node:crypto';
|
||||||
|
import { createRequire } from 'node:module';
|
||||||
|
|
||||||
|
import mysql from 'mysql2/promise';
|
||||||
|
|
||||||
|
const require = createRequire(import.meta.url);
|
||||||
|
const mysql2Version = require('mysql2/package.json').version;
|
||||||
|
const DEVELOPMENT_DATABASE = 'a_share_analysis_dev';
|
||||||
|
const VALID_TRIGGER_TIMINGS = new Set(['BEFORE', 'AFTER']);
|
||||||
|
const VALID_TRIGGER_EVENTS = new Set(['INSERT', 'UPDATE', 'DELETE']);
|
||||||
|
|
||||||
|
const CATALOG_QUERIES = Object.freeze({
|
||||||
|
tables: `SELECT TABLE_NAME, TABLE_TYPE
|
||||||
|
FROM information_schema.TABLES
|
||||||
|
WHERE TABLE_SCHEMA = ?
|
||||||
|
ORDER BY TABLE_NAME`,
|
||||||
|
viewDependencies: `SELECT VIEW_NAME, TABLE_SCHEMA AS REFERENCED_SCHEMA_NAME,
|
||||||
|
TABLE_NAME AS REFERENCED_TABLE_NAME
|
||||||
|
FROM information_schema.VIEW_TABLE_USAGE
|
||||||
|
WHERE VIEW_SCHEMA = ?
|
||||||
|
ORDER BY VIEW_NAME, REFERENCED_SCHEMA_NAME, REFERENCED_TABLE_NAME`,
|
||||||
|
viewDefinitions: `SELECT TABLE_NAME AS VIEW_NAME, VIEW_DEFINITION
|
||||||
|
FROM information_schema.VIEWS
|
||||||
|
WHERE TABLE_SCHEMA = ?
|
||||||
|
ORDER BY TABLE_NAME`,
|
||||||
|
routines: `SELECT ROUTINE_NAME, ROUTINE_TYPE
|
||||||
|
FROM information_schema.ROUTINES
|
||||||
|
WHERE ROUTINE_SCHEMA = ?
|
||||||
|
ORDER BY ROUTINE_TYPE, ROUTINE_NAME`,
|
||||||
|
triggers: `SELECT TRIGGER_NAME, EVENT_OBJECT_TABLE, ACTION_TIMING,
|
||||||
|
EVENT_MANIPULATION, ACTION_ORDER
|
||||||
|
FROM information_schema.TRIGGERS
|
||||||
|
WHERE TRIGGER_SCHEMA = ?
|
||||||
|
ORDER BY EVENT_OBJECT_TABLE, ACTION_TIMING, EVENT_MANIPULATION,
|
||||||
|
ACTION_ORDER, TRIGGER_NAME`,
|
||||||
|
events: `SELECT EVENT_NAME
|
||||||
|
FROM information_schema.EVENTS
|
||||||
|
WHERE EVENT_SCHEMA = ?
|
||||||
|
ORDER BY EVENT_NAME`,
|
||||||
|
});
|
||||||
|
|
||||||
|
const SERVER_METADATA_QUERY = `SELECT VERSION() AS SERVER_VERSION,
|
||||||
|
DEFAULT_CHARACTER_SET_NAME AS DATABASE_CHARSET,
|
||||||
|
DEFAULT_COLLATION_NAME AS DATABASE_COLLATION
|
||||||
|
FROM information_schema.SCHEMATA
|
||||||
|
WHERE SCHEMA_NAME = ?`;
|
||||||
|
|
||||||
|
const CREATE_RESULT_COLUMNS = Object.freeze({
|
||||||
|
TABLE: 'Create Table',
|
||||||
|
VIEW: 'Create View',
|
||||||
|
PROCEDURE: 'Create Procedure',
|
||||||
|
FUNCTION: 'Create Function',
|
||||||
|
TRIGGER: 'SQL Original Statement',
|
||||||
|
EVENT: 'Create Event',
|
||||||
|
});
|
||||||
|
|
||||||
|
export const MYSQL2_EXPORTER_METADATA = Object.freeze({
|
||||||
|
name: 'mysql2-show-create',
|
||||||
|
version: '1.0.0',
|
||||||
|
driverName: 'mysql2',
|
||||||
|
driverVersion: mysql2Version,
|
||||||
|
});
|
||||||
|
|
||||||
|
export const DEFAULT_EXPORT_LIMITS = Object.freeze({
|
||||||
|
connectTimeoutMs: 5_000,
|
||||||
|
queryTimeoutMs: 5_000,
|
||||||
|
totalTimeoutMs: 30_000,
|
||||||
|
maxObjects: 5_000,
|
||||||
|
maxDefinitionBytes: 1_048_576,
|
||||||
|
});
|
||||||
|
|
||||||
|
export class SchemaExportError extends Error {
|
||||||
|
constructor(code, message) {
|
||||||
|
super(message);
|
||||||
|
this.name = 'SchemaExportError';
|
||||||
|
this.code = code;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 配置在 Schema 治理入口已完成 development、主机白名单和固定库名校验;这里仅映射
|
||||||
|
* 连接所需字段,固定禁用多语句以防止未来元数据查询被拼接为任意 SQL。
|
||||||
|
*/
|
||||||
|
export function buildConnectionOptions(config) {
|
||||||
|
return {
|
||||||
|
host: config.MYSQL_HOST,
|
||||||
|
port: Number(config.MYSQL_PORT),
|
||||||
|
user: config.MYSQL_USERNAME,
|
||||||
|
password: config.MYSQL_PASSWORD,
|
||||||
|
database: config.MYSQL_DATABASE,
|
||||||
|
charset: 'utf8mb4',
|
||||||
|
timezone: '+08:00',
|
||||||
|
connectTimeout: DEFAULT_EXPORT_LIMITS.connectTimeoutMs,
|
||||||
|
multipleStatements: false,
|
||||||
|
enableKeepAlive: true,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 元数据对象名仍需防御性转义,避免控制字符破坏 `SHOW CREATE` 查询边界;名称不合法时
|
||||||
|
* 不回显输入,确保异常路径不会把潜在敏感值写入日志。
|
||||||
|
*/
|
||||||
|
export function quoteIdentifier(identifier) {
|
||||||
|
if (typeof identifier !== 'string' || identifier.length === 0 || /[\u0000-\u001F\u007F]/.test(identifier)) {
|
||||||
|
throw new SchemaExportError('INVALID_IDENTIFIER', 'Schema 标识符不能为空且不得包含控制字符');
|
||||||
|
}
|
||||||
|
|
||||||
|
return `\`${identifier.replaceAll('`', '``')}\``;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 导出器只接受项目数据库兼容基线,拒绝 MariaDB 和 MySQL 5.7 之外的服务器,避免相同
|
||||||
|
* `SHOW CREATE` 文本在不兼容语义下被误认为可恢复的 Schema 快照。
|
||||||
|
*/
|
||||||
|
export function assertMysql57ServerMetadata(metadata) {
|
||||||
|
const serverVersion = metadata?.serverVersion ?? '';
|
||||||
|
if (!/^5\.7\.\d+(?:[-+].*)?$/.test(serverVersion) || /mariadb/i.test(serverVersion)) {
|
||||||
|
throw new SchemaExportError('UNSUPPORTED_SERVER_VERSION', 'MySQL 服务端必须为 5.7.x');
|
||||||
|
}
|
||||||
|
if (metadata.databaseCharset !== 'utf8mb4') {
|
||||||
|
throw new SchemaExportError('INVALID_DATABASE_CHARSET', '数据库字符集必须为 utf8mb4');
|
||||||
|
}
|
||||||
|
if (metadata.databaseCollation !== 'utf8mb4_general_ci') {
|
||||||
|
throw new SchemaExportError('INVALID_DATABASE_COLLATION', '数据库排序规则必须为 utf8mb4_general_ci');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function requireCatalogText(value) {
|
||||||
|
if (typeof value !== 'string' || value.length === 0) {
|
||||||
|
throw new SchemaExportError('CATALOG_SHAPE_INVALID', 'Schema 对象目录字段无效');
|
||||||
|
}
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function queryCatalogRows(connection, sql, database, limits, options = {}) {
|
||||||
|
let result;
|
||||||
|
try {
|
||||||
|
result = await connection.query({
|
||||||
|
sql,
|
||||||
|
values: [database],
|
||||||
|
timeout: limits.queryTimeoutMs,
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
if (options.allowMissingTable && error?.code === 'ER_UNKNOWN_TABLE') {
|
||||||
|
throw new SchemaExportError('OPTIONAL_CATALOG_UNAVAILABLE', '可选 Schema 目录来源不可用');
|
||||||
|
}
|
||||||
|
// mysql2 错误可能包含连接 URL、用户名或密码,目录读取的公共接口一律不透传原始文本。
|
||||||
|
sanitizeDatabaseQueryError(error);
|
||||||
|
}
|
||||||
|
if (!Array.isArray(result) || !Array.isArray(result[0])) {
|
||||||
|
throw new SchemaExportError('CATALOG_SHAPE_INVALID', 'Schema 对象目录查询结果无效');
|
||||||
|
}
|
||||||
|
return result[0];
|
||||||
|
}
|
||||||
|
|
||||||
|
function unquoteViewIdentifier(identifier) {
|
||||||
|
const value = identifier.startsWith('`') && identifier.endsWith('`')
|
||||||
|
? identifier.slice(1, -1).replaceAll('``', '`')
|
||||||
|
: identifier;
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseViewDependencies(viewRows, database) {
|
||||||
|
const dependencies = [];
|
||||||
|
for (const row of viewRows) {
|
||||||
|
const view = requireCatalogText(row.VIEW_NAME);
|
||||||
|
const definition = requireCatalogText(row.VIEW_DEFINITION);
|
||||||
|
const inspectable = maskStringLiterals(definition);
|
||||||
|
const referencePattern = /\b(?:FROM|JOIN)\s+((?:`[^`]+`|[A-Za-z0-9_$]+)(?:\s*\.\s*(?:`[^`]+`|[A-Za-z0-9_$]+))?)/gi;
|
||||||
|
for (const match of inspectable.matchAll(referencePattern)) {
|
||||||
|
const parts = match[1].split(/\s*\.\s*/).map(unquoteViewIdentifier);
|
||||||
|
dependencies.push({
|
||||||
|
view,
|
||||||
|
referencedSchema: parts.length === 2 ? parts[0] : database,
|
||||||
|
referencedObject: parts.length === 2 ? parts[1] : parts[0],
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
dependencies.sort(
|
||||||
|
(left, right) =>
|
||||||
|
left.view.localeCompare(right.view) ||
|
||||||
|
left.referencedSchema.localeCompare(right.referencedSchema) ||
|
||||||
|
left.referencedObject.localeCompare(right.referencedObject),
|
||||||
|
);
|
||||||
|
return dependencies;
|
||||||
|
}
|
||||||
|
|
||||||
|
function assertObjectLimit(count, limits) {
|
||||||
|
if (!Number.isInteger(limits.maxObjects) || limits.maxObjects < 0 || count > limits.maxObjects) {
|
||||||
|
throw new SchemaExportError('OBJECT_LIMIT_EXCEEDED', 'Schema 对象数量超过安全上限');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 目录查询只从 information_schema 读取固定列,库名始终作为参数传入。每类对象
|
||||||
|
* 在进入后续 SHOW CREATE 前先转换成稳定的小对象,并在累计数量越界时立即停止。
|
||||||
|
*/
|
||||||
|
export async function readSchemaCatalog(connection, database, limits = DEFAULT_EXPORT_LIMITS) {
|
||||||
|
const tableRows = await queryCatalogRows(connection, CATALOG_QUERIES.tables, database, limits);
|
||||||
|
const tables = [];
|
||||||
|
const views = [];
|
||||||
|
for (const row of tableRows) {
|
||||||
|
const name = requireCatalogText(row.TABLE_NAME);
|
||||||
|
if (row.TABLE_TYPE === 'BASE TABLE') tables.push(name);
|
||||||
|
else if (row.TABLE_TYPE === 'VIEW') views.push(name);
|
||||||
|
else throw new SchemaExportError('UNKNOWN_TABLE_TYPE', 'Schema 目录包含未知表类型');
|
||||||
|
}
|
||||||
|
tables.sort();
|
||||||
|
views.sort();
|
||||||
|
let objectCount = tables.length + views.length;
|
||||||
|
assertObjectLimit(objectCount, limits);
|
||||||
|
|
||||||
|
let viewDependencies;
|
||||||
|
try {
|
||||||
|
const dependencyRows = await queryCatalogRows(
|
||||||
|
connection,
|
||||||
|
CATALOG_QUERIES.viewDependencies,
|
||||||
|
database,
|
||||||
|
limits,
|
||||||
|
{ allowMissingTable: true },
|
||||||
|
);
|
||||||
|
viewDependencies = dependencyRows.map((row) => ({
|
||||||
|
view: requireCatalogText(row.VIEW_NAME),
|
||||||
|
referencedSchema: requireCatalogText(row.REFERENCED_SCHEMA_NAME),
|
||||||
|
referencedObject: requireCatalogText(row.REFERENCED_TABLE_NAME),
|
||||||
|
}));
|
||||||
|
} catch (error) {
|
||||||
|
if (error.code !== 'OPTIONAL_CATALOG_UNAVAILABLE') throw error;
|
||||||
|
const viewRows = await queryCatalogRows(
|
||||||
|
connection,
|
||||||
|
CATALOG_QUERIES.viewDefinitions,
|
||||||
|
database,
|
||||||
|
limits,
|
||||||
|
);
|
||||||
|
viewDependencies = parseViewDependencies(viewRows, database);
|
||||||
|
}
|
||||||
|
|
||||||
|
const routineRows = await queryCatalogRows(connection, CATALOG_QUERIES.routines, database, limits);
|
||||||
|
const routines = routineRows.map((row) => {
|
||||||
|
const type = requireCatalogText(row.ROUTINE_TYPE);
|
||||||
|
if (!['FUNCTION', 'PROCEDURE'].includes(type)) {
|
||||||
|
throw new SchemaExportError('CATALOG_SHAPE_INVALID', 'Schema 目录包含未知例程类型');
|
||||||
|
}
|
||||||
|
return { name: requireCatalogText(row.ROUTINE_NAME), type };
|
||||||
|
});
|
||||||
|
routines.sort((left, right) => left.type.localeCompare(right.type) || left.name.localeCompare(right.name));
|
||||||
|
objectCount += routines.length;
|
||||||
|
assertObjectLimit(objectCount, limits);
|
||||||
|
|
||||||
|
const triggerRows = await queryCatalogRows(connection, CATALOG_QUERIES.triggers, database, limits);
|
||||||
|
const triggers = triggerRows.map((row) => {
|
||||||
|
const timing = requireCatalogText(row.ACTION_TIMING);
|
||||||
|
const event = requireCatalogText(row.EVENT_MANIPULATION);
|
||||||
|
const rawOrder = row.ACTION_ORDER;
|
||||||
|
const order = Number(rawOrder);
|
||||||
|
if (
|
||||||
|
!VALID_TRIGGER_TIMINGS.has(timing) ||
|
||||||
|
!VALID_TRIGGER_EVENTS.has(event) ||
|
||||||
|
(typeof rawOrder !== 'number' && typeof rawOrder !== 'string') ||
|
||||||
|
(typeof rawOrder === 'string' && rawOrder.trim().length === 0) ||
|
||||||
|
!Number.isInteger(order) ||
|
||||||
|
order < 1
|
||||||
|
) {
|
||||||
|
throw new SchemaExportError('CATALOG_SHAPE_INVALID', 'Schema 触发器执行顺序无效');
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
name: requireCatalogText(row.TRIGGER_NAME),
|
||||||
|
table: requireCatalogText(row.EVENT_OBJECT_TABLE),
|
||||||
|
timing,
|
||||||
|
event,
|
||||||
|
order,
|
||||||
|
};
|
||||||
|
});
|
||||||
|
triggers.sort(
|
||||||
|
(left, right) =>
|
||||||
|
left.table.localeCompare(right.table) ||
|
||||||
|
left.timing.localeCompare(right.timing) ||
|
||||||
|
left.event.localeCompare(right.event) ||
|
||||||
|
left.order - right.order ||
|
||||||
|
left.name.localeCompare(right.name),
|
||||||
|
);
|
||||||
|
objectCount += triggers.length;
|
||||||
|
assertObjectLimit(objectCount, limits);
|
||||||
|
|
||||||
|
const eventRows = await queryCatalogRows(connection, CATALOG_QUERIES.events, database, limits);
|
||||||
|
const events = eventRows.map((row) => requireCatalogText(row.EVENT_NAME)).sort();
|
||||||
|
objectCount += events.length;
|
||||||
|
assertObjectLimit(objectCount, limits);
|
||||||
|
|
||||||
|
return Object.freeze({
|
||||||
|
tables: Object.freeze(tables),
|
||||||
|
views: Object.freeze(views),
|
||||||
|
viewDependencies: Object.freeze(viewDependencies.map((dependency) => Object.freeze(dependency))),
|
||||||
|
routines: Object.freeze(routines.map((routine) => Object.freeze(routine))),
|
||||||
|
triggers: Object.freeze(triggers.map((trigger) => Object.freeze(trigger))),
|
||||||
|
events: Object.freeze(events),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 视图恢复顺序使用稳定 Kahn 拓扑排序。跨库或未知依赖不能被默默忽略,否则恢复
|
||||||
|
* 出来的 Schema 可能引用未捕获对象;同层节点始终按名称排序以保证逐字节确定性。
|
||||||
|
*/
|
||||||
|
export function orderViewNames(catalog) {
|
||||||
|
const tables = new Set(catalog.tables ?? []);
|
||||||
|
const views = new Set(catalog.views ?? []);
|
||||||
|
const incoming = new Map([...views].map((view) => [view, 0]));
|
||||||
|
const dependents = new Map([...views].map((view) => [view, []]));
|
||||||
|
|
||||||
|
for (const dependency of catalog.viewDependencies ?? []) {
|
||||||
|
if (dependency.referencedSchema !== DEVELOPMENT_DATABASE) {
|
||||||
|
throw new SchemaExportError(
|
||||||
|
'CROSS_SCHEMA_VIEW_DEPENDENCY',
|
||||||
|
'Schema 视图包含跨库依赖',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if (!views.has(dependency.view)) {
|
||||||
|
throw new SchemaExportError('UNKNOWN_VIEW_DEPENDENCY', 'Schema 视图依赖来源未知');
|
||||||
|
}
|
||||||
|
if (tables.has(dependency.referencedObject)) continue;
|
||||||
|
if (!views.has(dependency.referencedObject)) {
|
||||||
|
throw new SchemaExportError('UNKNOWN_VIEW_DEPENDENCY', 'Schema 视图依赖对象未知');
|
||||||
|
}
|
||||||
|
incoming.set(dependency.view, incoming.get(dependency.view) + 1);
|
||||||
|
dependents.get(dependency.referencedObject).push(dependency.view);
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const names of dependents.values()) names.sort();
|
||||||
|
const ready = [...views].filter((view) => incoming.get(view) === 0).sort();
|
||||||
|
const ordered = [];
|
||||||
|
while (ready.length > 0) {
|
||||||
|
const view = ready.shift();
|
||||||
|
ordered.push(view);
|
||||||
|
for (const dependent of dependents.get(view)) {
|
||||||
|
incoming.set(dependent, incoming.get(dependent) - 1);
|
||||||
|
if (incoming.get(dependent) === 0) {
|
||||||
|
ready.push(dependent);
|
||||||
|
ready.sort();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (ordered.length !== views.size) {
|
||||||
|
throw new SchemaExportError('VIEW_DEPENDENCY_CYCLE', 'Schema 视图依赖存在循环');
|
||||||
|
}
|
||||||
|
return ordered;
|
||||||
|
}
|
||||||
|
|
||||||
|
function sha256(content) {
|
||||||
|
return createHash('sha256').update(content, 'utf8').digest('hex');
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 只屏蔽 SQL 字符串字面量,保留标识符参与 DEFINER 和凭据扫描。 */
|
||||||
|
function maskStringLiterals(sql) {
|
||||||
|
return String(sql).replace(/'(?:''|\\.|[^'\\])*'/gs, "''");
|
||||||
|
}
|
||||||
|
|
||||||
|
function removeTableAutoIncrementOutsideLiterals(sql) {
|
||||||
|
const literals = [];
|
||||||
|
const masked = String(sql).replace(/'(?:''|\\.|[^'\\])*'/gs, (literal) => {
|
||||||
|
const index = literals.push(literal) - 1;
|
||||||
|
return `\u0001${index}\u0002`;
|
||||||
|
});
|
||||||
|
const cleaned = masked.replace(
|
||||||
|
/\sAUTO_INCREMENT=\d+(?=\s(?:DEFAULT\s+)?CHARSET=)/i,
|
||||||
|
'',
|
||||||
|
);
|
||||||
|
return cleaned.replace(/\u0001(\d+)\u0002/g, (_match, index) => literals[Number(index)]);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* SHOW CREATE 的格式由服务端决定,这里只移除与结构无关的账号绑定、当前表自增值、
|
||||||
|
* 行尾分号和换行差异,不重排对象体,也不删除字符串字面量中的业务文本。
|
||||||
|
*/
|
||||||
|
export function normalizeCreateDefinition(definition) {
|
||||||
|
const normalized = String(definition)
|
||||||
|
.replace(/\r\n?/g, '\n')
|
||||||
|
.replace(
|
||||||
|
/^(CREATE\s+(?:ALGORITHM=\S+\s+)?)DEFINER=`(?:``|[^`])+`@`(?:`(?:``|[^`])+`|'(?:''|[^'])+'|\S+)\s+/i,
|
||||||
|
'$1',
|
||||||
|
)
|
||||||
|
.trim()
|
||||||
|
.replace(/;+\s*$/, '');
|
||||||
|
const withoutCurrentAutoIncrement = removeTableAutoIncrementOutsideLiterals(normalized);
|
||||||
|
const inspectable = maskStringLiterals(withoutCurrentAutoIncrement);
|
||||||
|
if (
|
||||||
|
/\bDEFINER\s*=/i.test(inspectable) ||
|
||||||
|
/\b(?:password|passwd|pwd|api[_-]?key|api[_-]?token|access[_-]?token|client[_-]?secret|secret)\s*[:=]/i.test(
|
||||||
|
inspectable,
|
||||||
|
) ||
|
||||||
|
/\b(?:mysql|mariadb):\/\/[^\s'"`]+/i.test(inspectable) ||
|
||||||
|
/(?:--|\/\*)\s*(?:host|hostname|server)\s*[:=]\s*[^\s*]+/i.test(inspectable)
|
||||||
|
) {
|
||||||
|
throw new SchemaExportError('UNSAFE_DEFINITION', 'Schema 对象定义包含禁止内容');
|
||||||
|
}
|
||||||
|
return withoutCurrentAutoIncrement;
|
||||||
|
}
|
||||||
|
|
||||||
|
function sanitizeDatabaseQueryError(error) {
|
||||||
|
if (error instanceof SchemaExportError) throw error;
|
||||||
|
throw new SchemaExportError('DATABASE_QUERY_FAILED', '开发库 Schema 查询失败');
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function readMysql57ServerMetadata(
|
||||||
|
connection,
|
||||||
|
database,
|
||||||
|
limits = DEFAULT_EXPORT_LIMITS,
|
||||||
|
) {
|
||||||
|
let rows;
|
||||||
|
try {
|
||||||
|
[rows] = await connection.query({
|
||||||
|
sql: SERVER_METADATA_QUERY,
|
||||||
|
values: [database],
|
||||||
|
timeout: limits.queryTimeoutMs,
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
sanitizeDatabaseQueryError(error);
|
||||||
|
}
|
||||||
|
if (!Array.isArray(rows) || rows.length !== 1) {
|
||||||
|
throw new SchemaExportError('DATABASE_METADATA_INVALID', '开发库元数据查询结果无效');
|
||||||
|
}
|
||||||
|
const metadata = {
|
||||||
|
serverVersion: rows[0].SERVER_VERSION,
|
||||||
|
databaseCharset: rows[0].DATABASE_CHARSET,
|
||||||
|
databaseCollation: rows[0].DATABASE_COLLATION,
|
||||||
|
};
|
||||||
|
assertMysql57ServerMetadata(metadata);
|
||||||
|
return Object.freeze(metadata);
|
||||||
|
}
|
||||||
|
|
||||||
|
function getCreateDescriptors(catalog) {
|
||||||
|
const viewOrder = orderViewNames(catalog);
|
||||||
|
const viewDefinitions = new Map((catalog.views ?? []).map((name) => [name, { name, type: 'VIEW' }]));
|
||||||
|
return [
|
||||||
|
...(catalog.tables ?? []).map((name) => ({ name, type: 'TABLE' })),
|
||||||
|
...(catalog.routines ?? []).map(({ name, type }) => ({ name, type })),
|
||||||
|
...viewOrder.map((name) => viewDefinitions.get(name)),
|
||||||
|
...(catalog.triggers ?? []).map(({ name }) => ({ name, type: 'TRIGGER' })),
|
||||||
|
...(catalog.events ?? []).map((name) => ({ name, type: 'EVENT' })),
|
||||||
|
];
|
||||||
|
}
|
||||||
|
|
||||||
|
async function readCreateDefinition(connection, descriptor, limits) {
|
||||||
|
const sql = `SHOW CREATE ${descriptor.type} ${quoteIdentifier(DEVELOPMENT_DATABASE)}.${quoteIdentifier(descriptor.name)}`;
|
||||||
|
let rows;
|
||||||
|
try {
|
||||||
|
[rows] = await connection.query({ sql, timeout: limits.queryTimeoutMs });
|
||||||
|
} catch (error) {
|
||||||
|
sanitizeDatabaseQueryError(error);
|
||||||
|
}
|
||||||
|
if (!Array.isArray(rows) || rows.length === 0) {
|
||||||
|
throw new SchemaExportError('SHOW_CREATE_EMPTY', 'SHOW CREATE 未返回对象定义');
|
||||||
|
}
|
||||||
|
const rawDefinition = rows[0]?.[CREATE_RESULT_COLUMNS[descriptor.type]];
|
||||||
|
if (typeof rawDefinition !== 'string' || rawDefinition.length === 0) {
|
||||||
|
throw new SchemaExportError('SHOW_CREATE_SHAPE_INVALID', 'SHOW CREATE 返回列无效');
|
||||||
|
}
|
||||||
|
if (Buffer.byteLength(rawDefinition, 'utf8') > limits.maxDefinitionBytes) {
|
||||||
|
throw new SchemaExportError('DEFINITION_LIMIT_EXCEEDED', 'Schema 对象定义超过安全上限');
|
||||||
|
}
|
||||||
|
return Object.freeze({
|
||||||
|
...descriptor,
|
||||||
|
definition: normalizeCreateDefinition(rawDefinition),
|
||||||
|
sourceSha256: sha256(rawDefinition.replace(/\r\n?/g, '\n')),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function readDefinitions(connection, catalog, limits = DEFAULT_EXPORT_LIMITS) {
|
||||||
|
const definitions = [];
|
||||||
|
for (const descriptor of getCreateDescriptors(catalog)) {
|
||||||
|
definitions.push(await readCreateDefinition(connection, descriptor, limits));
|
||||||
|
}
|
||||||
|
return Object.freeze(definitions);
|
||||||
|
}
|
||||||
|
|
||||||
|
function appendDefinition(parts, definition) {
|
||||||
|
if (['TABLE', 'VIEW'].includes(definition.type)) {
|
||||||
|
parts.push(`${definition.definition};`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
parts.push(`DELIMITER ;;\n${definition.definition} ;;\nDELIMITER ;`);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 固定恢复顺序和会话保护,不重新格式化对象体。 */
|
||||||
|
export function renderSchemaSql(catalog, definitions) {
|
||||||
|
const definitionsByKey = new Map(
|
||||||
|
definitions.map((definition) => [`${definition.type}:${definition.name}`, definition]),
|
||||||
|
);
|
||||||
|
const parts = [
|
||||||
|
'SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS, FOREIGN_KEY_CHECKS=0;',
|
||||||
|
'SET NAMES utf8mb4;',
|
||||||
|
];
|
||||||
|
for (const descriptor of getCreateDescriptors(catalog)) {
|
||||||
|
const definition = definitionsByKey.get(`${descriptor.type}:${descriptor.name}`);
|
||||||
|
if (!definition) {
|
||||||
|
throw new SchemaExportError('SHOW_CREATE_SHAPE_INVALID', 'Schema 对象定义集合不完整');
|
||||||
|
}
|
||||||
|
appendDefinition(parts, definition);
|
||||||
|
}
|
||||||
|
parts.push('SET FOREIGN_KEY_CHECKS=@OLD_FOREIGN_KEY_CHECKS;');
|
||||||
|
return `${parts.join('\n\n')}\n`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function assertWithinDeadline(startedAt, now, limits) {
|
||||||
|
if (now() - startedAt > limits.totalTimeoutMs) {
|
||||||
|
throw new SchemaExportError('EXPORT_TIMEOUT', 'Schema 导出超过总时长限制');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function buildObjectCounts(catalog) {
|
||||||
|
return Object.freeze({
|
||||||
|
tables: catalog.tables.length,
|
||||||
|
views: catalog.views.length,
|
||||||
|
triggers: catalog.triggers.length,
|
||||||
|
routines: catalog.routines.length,
|
||||||
|
events: catalog.events.length,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 单连接执行两轮目录和定义读取;任何漂移都失败。底层数据库异常统一脱敏,连接在
|
||||||
|
* 成功、查询失败、校验失败和超时路径均只关闭一次。
|
||||||
|
*/
|
||||||
|
export async function exportMysql57Schema({
|
||||||
|
config,
|
||||||
|
createConnection = mysql.createConnection,
|
||||||
|
now = Date.now,
|
||||||
|
limits = DEFAULT_EXPORT_LIMITS,
|
||||||
|
}) {
|
||||||
|
const startedAt = now();
|
||||||
|
let connection;
|
||||||
|
let result;
|
||||||
|
let failure;
|
||||||
|
try {
|
||||||
|
connection = await createConnection(buildConnectionOptions(config));
|
||||||
|
const guardedConnection = {
|
||||||
|
async query(options) {
|
||||||
|
assertWithinDeadline(startedAt, now, limits);
|
||||||
|
const queryResult = await connection.query(options);
|
||||||
|
assertWithinDeadline(startedAt, now, limits);
|
||||||
|
return queryResult;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
const serverMetadata = await readMysql57ServerMetadata(
|
||||||
|
guardedConnection,
|
||||||
|
config.MYSQL_DATABASE,
|
||||||
|
limits,
|
||||||
|
);
|
||||||
|
const beforeCatalog = await readSchemaCatalog(
|
||||||
|
guardedConnection,
|
||||||
|
config.MYSQL_DATABASE,
|
||||||
|
limits,
|
||||||
|
);
|
||||||
|
const beforeDefinitions = await readDefinitions(guardedConnection, beforeCatalog, limits);
|
||||||
|
const afterCatalog = await readSchemaCatalog(
|
||||||
|
guardedConnection,
|
||||||
|
config.MYSQL_DATABASE,
|
||||||
|
limits,
|
||||||
|
);
|
||||||
|
if (sha256(JSON.stringify(beforeCatalog)) !== sha256(JSON.stringify(afterCatalog))) {
|
||||||
|
throw new SchemaExportError(
|
||||||
|
'CATALOG_CHANGED_DURING_EXPORT',
|
||||||
|
'Schema 对象目录在导出期间发生变化',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
const afterDefinitions = await readDefinitions(guardedConnection, afterCatalog, limits);
|
||||||
|
if (sha256(JSON.stringify(beforeDefinitions)) !== sha256(JSON.stringify(afterDefinitions))) {
|
||||||
|
throw new SchemaExportError(
|
||||||
|
'DEFINITION_CHANGED_DURING_EXPORT',
|
||||||
|
'Schema 对象定义在导出期间发生变化',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
assertWithinDeadline(startedAt, now, limits);
|
||||||
|
const objectCounts = buildObjectCounts(beforeCatalog);
|
||||||
|
result = Object.freeze({
|
||||||
|
metadata: Object.freeze({
|
||||||
|
exporterName: MYSQL2_EXPORTER_METADATA.name,
|
||||||
|
exporterVersion: MYSQL2_EXPORTER_METADATA.version,
|
||||||
|
driverName: MYSQL2_EXPORTER_METADATA.driverName,
|
||||||
|
driverVersion: MYSQL2_EXPORTER_METADATA.driverVersion,
|
||||||
|
...serverMetadata,
|
||||||
|
}),
|
||||||
|
rawSql: Object.values(objectCounts).every((count) => count === 0)
|
||||||
|
? ''
|
||||||
|
: renderSchemaSql(beforeCatalog, beforeDefinitions),
|
||||||
|
objectCounts,
|
||||||
|
durationMs: now() - startedAt,
|
||||||
|
});
|
||||||
|
} catch (error) {
|
||||||
|
failure =
|
||||||
|
error instanceof SchemaExportError
|
||||||
|
? error
|
||||||
|
: new SchemaExportError('DATABASE_QUERY_FAILED', '开发库 Schema 查询失败');
|
||||||
|
} finally {
|
||||||
|
if (connection) {
|
||||||
|
try {
|
||||||
|
await connection.end();
|
||||||
|
} catch {
|
||||||
|
if (!failure) {
|
||||||
|
failure = new SchemaExportError('DATABASE_QUERY_FAILED', '开发库 Schema 连接关闭失败');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (failure) throw failure;
|
||||||
|
return result;
|
||||||
|
}
|
||||||
@@ -0,0 +1,705 @@
|
|||||||
|
import assert from 'node:assert/strict';
|
||||||
|
import test from 'node:test';
|
||||||
|
|
||||||
|
import {
|
||||||
|
DEFAULT_EXPORT_LIMITS,
|
||||||
|
MYSQL2_EXPORTER_METADATA,
|
||||||
|
SchemaExportError,
|
||||||
|
assertMysql57ServerMetadata,
|
||||||
|
buildConnectionOptions,
|
||||||
|
exportMysql57Schema,
|
||||||
|
normalizeCreateDefinition,
|
||||||
|
orderViewNames,
|
||||||
|
quoteIdentifier,
|
||||||
|
readDefinitions,
|
||||||
|
readMysql57ServerMetadata,
|
||||||
|
readSchemaCatalog,
|
||||||
|
renderSchemaSql,
|
||||||
|
} from './mysql2-schema-exporter.mjs';
|
||||||
|
|
||||||
|
const config = Object.freeze({
|
||||||
|
MYSQL_HOST: '127.0.0.1',
|
||||||
|
MYSQL_PORT: '3306',
|
||||||
|
MYSQL_DATABASE: 'a_share_analysis_dev',
|
||||||
|
MYSQL_USERNAME: 'schema_reader',
|
||||||
|
MYSQL_PASSWORD: 'not-for-logs',
|
||||||
|
});
|
||||||
|
|
||||||
|
test('连接配置固定单连接、utf8mb4、东八区且禁用多语句', () => {
|
||||||
|
const options = buildConnectionOptions(config);
|
||||||
|
assert.deepEqual(options, {
|
||||||
|
host: '127.0.0.1',
|
||||||
|
port: 3306,
|
||||||
|
user: 'schema_reader',
|
||||||
|
password: 'not-for-logs',
|
||||||
|
database: 'a_share_analysis_dev',
|
||||||
|
charset: 'utf8mb4',
|
||||||
|
timezone: '+08:00',
|
||||||
|
connectTimeout: 5_000,
|
||||||
|
multipleStatements: false,
|
||||||
|
enableKeepAlive: true,
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
test('标识符转义反引号并拒绝空值和控制字符', () => {
|
||||||
|
assert.equal(quoteIdentifier('market`view'), '`market``view`');
|
||||||
|
for (const value of ['', 'line\nbreak', 'zero\0byte']) {
|
||||||
|
assert.throws(() => quoteIdentifier(value), SchemaExportError);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('只接受 MySQL 5.7.x 与固定字符规则', () => {
|
||||||
|
const valid = {
|
||||||
|
serverVersion: '5.7.42-log',
|
||||||
|
databaseCharset: 'utf8mb4',
|
||||||
|
databaseCollation: 'utf8mb4_general_ci',
|
||||||
|
};
|
||||||
|
assert.doesNotThrow(() => assertMysql57ServerMetadata(valid));
|
||||||
|
for (const serverVersion of [
|
||||||
|
'5.6.51',
|
||||||
|
'8.0.40',
|
||||||
|
'5.7.42-MariaDB',
|
||||||
|
'10.6.18-MariaDB',
|
||||||
|
]) {
|
||||||
|
assert.throws(() => assertMysql57ServerMetadata({ ...valid, serverVersion }));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
function createCatalogConnection(overrides = {}) {
|
||||||
|
const rowsBySource = {
|
||||||
|
'information_schema.TABLES': [
|
||||||
|
{ TABLE_NAME: 'market_snapshot', TABLE_TYPE: 'BASE TABLE' },
|
||||||
|
{ TABLE_NAME: 'market_dashboard', TABLE_TYPE: 'VIEW' },
|
||||||
|
{ TABLE_NAME: 'market_summary', TABLE_TYPE: 'VIEW' },
|
||||||
|
],
|
||||||
|
'information_schema.VIEW_TABLE_USAGE': [
|
||||||
|
{
|
||||||
|
VIEW_NAME: 'market_dashboard',
|
||||||
|
REFERENCED_SCHEMA_NAME: 'a_share_analysis_dev',
|
||||||
|
REFERENCED_TABLE_NAME: 'market_summary',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
VIEW_NAME: 'market_summary',
|
||||||
|
REFERENCED_SCHEMA_NAME: 'a_share_analysis_dev',
|
||||||
|
REFERENCED_TABLE_NAME: 'market_snapshot',
|
||||||
|
},
|
||||||
|
],
|
||||||
|
'information_schema.ROUTINES': [
|
||||||
|
{ ROUTINE_NAME: 'calculate_breadth', ROUTINE_TYPE: 'FUNCTION' },
|
||||||
|
{ ROUTINE_NAME: 'refresh_market_snapshot', ROUTINE_TYPE: 'PROCEDURE' },
|
||||||
|
],
|
||||||
|
'information_schema.TRIGGERS': [
|
||||||
|
{
|
||||||
|
TRIGGER_NAME: 'snapshot_before_insert',
|
||||||
|
EVENT_OBJECT_TABLE: 'market_snapshot',
|
||||||
|
ACTION_TIMING: 'BEFORE',
|
||||||
|
EVENT_MANIPULATION: 'INSERT',
|
||||||
|
ACTION_ORDER: 1,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
'information_schema.EVENTS': [{ EVENT_NAME: 'purge_market_snapshot' }],
|
||||||
|
...overrides,
|
||||||
|
};
|
||||||
|
const calls = [];
|
||||||
|
return {
|
||||||
|
calls,
|
||||||
|
async query(options) {
|
||||||
|
calls.push(options);
|
||||||
|
const source = Object.keys(rowsBySource).find((candidate) => options.sql.includes(candidate));
|
||||||
|
assert.ok(source, `未识别的目录查询:${options.sql}`);
|
||||||
|
return [rowsBySource[source], []];
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
test('目录查询使用固定参数和超时并返回稳定对象', async () => {
|
||||||
|
const connection = createCatalogConnection();
|
||||||
|
const catalog = await readSchemaCatalog(connection, 'a_share_analysis_dev', DEFAULT_EXPORT_LIMITS);
|
||||||
|
|
||||||
|
assert.deepEqual(catalog, {
|
||||||
|
tables: ['market_snapshot'],
|
||||||
|
views: ['market_dashboard', 'market_summary'],
|
||||||
|
viewDependencies: [
|
||||||
|
{
|
||||||
|
view: 'market_dashboard',
|
||||||
|
referencedSchema: 'a_share_analysis_dev',
|
||||||
|
referencedObject: 'market_summary',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
view: 'market_summary',
|
||||||
|
referencedSchema: 'a_share_analysis_dev',
|
||||||
|
referencedObject: 'market_snapshot',
|
||||||
|
},
|
||||||
|
],
|
||||||
|
routines: [
|
||||||
|
{ name: 'calculate_breadth', type: 'FUNCTION' },
|
||||||
|
{ name: 'refresh_market_snapshot', type: 'PROCEDURE' },
|
||||||
|
],
|
||||||
|
triggers: [
|
||||||
|
{
|
||||||
|
name: 'snapshot_before_insert',
|
||||||
|
table: 'market_snapshot',
|
||||||
|
timing: 'BEFORE',
|
||||||
|
event: 'INSERT',
|
||||||
|
order: 1,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
events: ['purge_market_snapshot'],
|
||||||
|
});
|
||||||
|
assert.equal(connection.calls.length, 5);
|
||||||
|
for (const call of connection.calls) {
|
||||||
|
assert.deepEqual(call.values, ['a_share_analysis_dev']);
|
||||||
|
assert.equal(call.timeout, DEFAULT_EXPORT_LIMITS.queryTimeoutMs);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('MySQL 5.7 缺少 VIEW_TABLE_USAGE 时从 VIEWS 定义解析依赖', async () => {
|
||||||
|
const fallback = createCatalogConnection();
|
||||||
|
const connection = {
|
||||||
|
calls: fallback.calls,
|
||||||
|
async query(options) {
|
||||||
|
if (options.sql.includes('information_schema.VIEW_TABLE_USAGE')) {
|
||||||
|
throw Object.assign(new Error('unknown metadata table'), { code: 'ER_UNKNOWN_TABLE' });
|
||||||
|
}
|
||||||
|
if (options.sql.includes('information_schema.VIEWS')) {
|
||||||
|
return [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
VIEW_NAME: 'market_dashboard',
|
||||||
|
VIEW_DEFINITION:
|
||||||
|
'SELECT * FROM `a_share_analysis_dev`.`market_summary`',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
VIEW_NAME: 'market_summary',
|
||||||
|
VIEW_DEFINITION: 'SELECT * FROM `market_snapshot`',
|
||||||
|
},
|
||||||
|
],
|
||||||
|
[],
|
||||||
|
];
|
||||||
|
}
|
||||||
|
return fallback.query(options);
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
const catalog = await readSchemaCatalog(connection, 'a_share_analysis_dev', DEFAULT_EXPORT_LIMITS);
|
||||||
|
assert.deepEqual(catalog.viewDependencies, [
|
||||||
|
{
|
||||||
|
view: 'market_dashboard',
|
||||||
|
referencedSchema: 'a_share_analysis_dev',
|
||||||
|
referencedObject: 'market_summary',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
view: 'market_summary',
|
||||||
|
referencedSchema: 'a_share_analysis_dev',
|
||||||
|
referencedObject: 'market_snapshot',
|
||||||
|
},
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('视图按依赖拓扑排序且同层按名称稳定排序', () => {
|
||||||
|
const catalog = {
|
||||||
|
tables: ['market_snapshot'],
|
||||||
|
views: ['z_dashboard', 'market_summary', 'a_dashboard'],
|
||||||
|
viewDependencies: [
|
||||||
|
{
|
||||||
|
view: 'z_dashboard',
|
||||||
|
referencedSchema: 'a_share_analysis_dev',
|
||||||
|
referencedObject: 'market_summary',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
view: 'a_dashboard',
|
||||||
|
referencedSchema: 'a_share_analysis_dev',
|
||||||
|
referencedObject: 'market_summary',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
view: 'market_summary',
|
||||||
|
referencedSchema: 'a_share_analysis_dev',
|
||||||
|
referencedObject: 'market_snapshot',
|
||||||
|
},
|
||||||
|
],
|
||||||
|
};
|
||||||
|
|
||||||
|
assert.deepEqual(orderViewNames(catalog), ['market_summary', 'a_dashboard', 'z_dashboard']);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('视图排序拒绝循环、跨库和未知依赖', () => {
|
||||||
|
const base = { tables: ['market_snapshot'], views: ['first_view', 'second_view'] };
|
||||||
|
const invalidCases = [
|
||||||
|
{
|
||||||
|
code: 'VIEW_DEPENDENCY_CYCLE',
|
||||||
|
dependencies: [
|
||||||
|
{ view: 'first_view', referencedSchema: 'a_share_analysis_dev', referencedObject: 'second_view' },
|
||||||
|
{ view: 'second_view', referencedSchema: 'a_share_analysis_dev', referencedObject: 'first_view' },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
code: 'CROSS_SCHEMA_VIEW_DEPENDENCY',
|
||||||
|
dependencies: [
|
||||||
|
{ view: 'first_view', referencedSchema: 'production', referencedObject: 'market_snapshot' },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
{
|
||||||
|
code: 'UNKNOWN_VIEW_DEPENDENCY',
|
||||||
|
dependencies: [
|
||||||
|
{ view: 'first_view', referencedSchema: 'a_share_analysis_dev', referencedObject: 'missing_object' },
|
||||||
|
],
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
for (const invalid of invalidCases) {
|
||||||
|
assert.throws(
|
||||||
|
() => orderViewNames({ ...base, viewDependencies: invalid.dependencies }),
|
||||||
|
(error) => error instanceof SchemaExportError && error.code === invalid.code,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('目录拒绝未知表类型和超过对象上限', async () => {
|
||||||
|
const unknownType = createCatalogConnection({
|
||||||
|
'information_schema.TABLES': [{ TABLE_NAME: 'temporary_data', TABLE_TYPE: 'SYSTEM VIEW' }],
|
||||||
|
});
|
||||||
|
await assert.rejects(
|
||||||
|
() => readSchemaCatalog(unknownType, 'a_share_analysis_dev', DEFAULT_EXPORT_LIMITS),
|
||||||
|
(error) => error instanceof SchemaExportError && error.code === 'UNKNOWN_TABLE_TYPE',
|
||||||
|
);
|
||||||
|
|
||||||
|
const overLimit = createCatalogConnection({
|
||||||
|
'information_schema.TABLES': [
|
||||||
|
{ TABLE_NAME: 'first_table', TABLE_TYPE: 'BASE TABLE' },
|
||||||
|
{ TABLE_NAME: 'second_table', TABLE_TYPE: 'BASE TABLE' },
|
||||||
|
],
|
||||||
|
});
|
||||||
|
await assert.rejects(
|
||||||
|
() => readSchemaCatalog(overLimit, 'a_share_analysis_dev', { ...DEFAULT_EXPORT_LIMITS, maxObjects: 1 }),
|
||||||
|
(error) => error instanceof SchemaExportError && error.code === 'OBJECT_LIMIT_EXCEEDED',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('目录拒绝非法触发器时机、事件和空执行顺序', async () => {
|
||||||
|
for (const trigger of [
|
||||||
|
{
|
||||||
|
TRIGGER_NAME: 'invalid_timing',
|
||||||
|
EVENT_OBJECT_TABLE: 'market_snapshot',
|
||||||
|
ACTION_TIMING: 'DURING',
|
||||||
|
EVENT_MANIPULATION: 'INSERT',
|
||||||
|
ACTION_ORDER: 1,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
TRIGGER_NAME: 'invalid_event',
|
||||||
|
EVENT_OBJECT_TABLE: 'market_snapshot',
|
||||||
|
ACTION_TIMING: 'BEFORE',
|
||||||
|
EVENT_MANIPULATION: 'MERGE',
|
||||||
|
ACTION_ORDER: 1,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
TRIGGER_NAME: 'empty_order',
|
||||||
|
EVENT_OBJECT_TABLE: 'market_snapshot',
|
||||||
|
ACTION_TIMING: 'BEFORE',
|
||||||
|
EVENT_MANIPULATION: 'INSERT',
|
||||||
|
ACTION_ORDER: '',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
TRIGGER_NAME: 'missing_order',
|
||||||
|
EVENT_OBJECT_TABLE: 'market_snapshot',
|
||||||
|
ACTION_TIMING: 'BEFORE',
|
||||||
|
EVENT_MANIPULATION: 'INSERT',
|
||||||
|
ACTION_ORDER: null,
|
||||||
|
},
|
||||||
|
]) {
|
||||||
|
const connection = createCatalogConnection({
|
||||||
|
'information_schema.TRIGGERS': [trigger],
|
||||||
|
});
|
||||||
|
await assert.rejects(
|
||||||
|
() => readSchemaCatalog(connection, 'a_share_analysis_dev', DEFAULT_EXPORT_LIMITS),
|
||||||
|
(error) => error instanceof SchemaExportError && error.code === 'CATALOG_SHAPE_INVALID',
|
||||||
|
);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('直接目录读取将数据库异常转为固定脱敏错误', async () => {
|
||||||
|
const connection = {
|
||||||
|
async query() {
|
||||||
|
throw new Error('connect mysql://schema_reader:not-for-logs@127.0.0.1:3306/a_share_analysis_dev');
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
await assert.rejects(
|
||||||
|
() => readSchemaCatalog(connection, 'a_share_analysis_dev', DEFAULT_EXPORT_LIMITS),
|
||||||
|
(error) =>
|
||||||
|
error instanceof SchemaExportError &&
|
||||||
|
error.code === 'DATABASE_QUERY_FAILED' &&
|
||||||
|
error.message === '开发库 Schema 查询失败',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
const completeCatalog = Object.freeze({
|
||||||
|
tables: ['market_snapshot'],
|
||||||
|
views: ['market_dashboard', 'market_summary'],
|
||||||
|
viewDependencies: [
|
||||||
|
{
|
||||||
|
view: 'market_dashboard',
|
||||||
|
referencedSchema: 'a_share_analysis_dev',
|
||||||
|
referencedObject: 'market_summary',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
view: 'market_summary',
|
||||||
|
referencedSchema: 'a_share_analysis_dev',
|
||||||
|
referencedObject: 'market_snapshot',
|
||||||
|
},
|
||||||
|
],
|
||||||
|
routines: [
|
||||||
|
{ name: 'calculate_breadth', type: 'FUNCTION' },
|
||||||
|
{ name: 'refresh_market_snapshot', type: 'PROCEDURE' },
|
||||||
|
],
|
||||||
|
triggers: [
|
||||||
|
{
|
||||||
|
name: 'snapshot_before_insert',
|
||||||
|
table: 'market_snapshot',
|
||||||
|
timing: 'BEFORE',
|
||||||
|
event: 'INSERT',
|
||||||
|
order: 1,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
events: ['purge_market_snapshot'],
|
||||||
|
});
|
||||||
|
|
||||||
|
const createRows = Object.freeze({
|
||||||
|
'SHOW CREATE TABLE `a_share_analysis_dev`.`market_snapshot`': {
|
||||||
|
'Create Table':
|
||||||
|
'CREATE TABLE `market_snapshot` (`id` bigint NOT NULL AUTO_INCREMENT) ENGINE=InnoDB AUTO_INCREMENT=12 DEFAULT CHARSET=utf8mb4',
|
||||||
|
},
|
||||||
|
'SHOW CREATE VIEW `a_share_analysis_dev`.`market_summary`': {
|
||||||
|
'Create View':
|
||||||
|
'CREATE ALGORITHM=UNDEFINED DEFINER=`schema_user`@`192.168.1.10` SQL SECURITY DEFINER VIEW `market_summary` AS select `market_snapshot`.`id` AS `id` from `market_snapshot`',
|
||||||
|
},
|
||||||
|
'SHOW CREATE VIEW `a_share_analysis_dev`.`market_dashboard`': {
|
||||||
|
'Create View':
|
||||||
|
'CREATE ALGORITHM=UNDEFINED DEFINER=`schema_user`@`192.168.1.10` SQL SECURITY DEFINER VIEW `market_dashboard` AS select `market_summary`.`id` AS `id` from `market_summary`',
|
||||||
|
},
|
||||||
|
'SHOW CREATE FUNCTION `a_share_analysis_dev`.`calculate_breadth`': {
|
||||||
|
'Create Function':
|
||||||
|
"CREATE DEFINER=`schema_user`@`192.168.1.10` FUNCTION `calculate_breadth`() RETURNS int RETURN 1",
|
||||||
|
},
|
||||||
|
'SHOW CREATE PROCEDURE `a_share_analysis_dev`.`refresh_market_snapshot`': {
|
||||||
|
'Create Procedure':
|
||||||
|
'CREATE DEFINER=`schema_user`@`192.168.1.10` PROCEDURE `refresh_market_snapshot`()\nBEGIN\n UPDATE `market_snapshot` SET `id` = `id`;\nEND',
|
||||||
|
},
|
||||||
|
'SHOW CREATE TRIGGER `a_share_analysis_dev`.`snapshot_before_insert`': {
|
||||||
|
'SQL Original Statement':
|
||||||
|
'CREATE DEFINER=`schema_user`@`192.168.1.10` TRIGGER `snapshot_before_insert` BEFORE INSERT ON `market_snapshot` FOR EACH ROW SET NEW.id = NEW.id',
|
||||||
|
},
|
||||||
|
'SHOW CREATE EVENT `a_share_analysis_dev`.`purge_market_snapshot`': {
|
||||||
|
'Create Event':
|
||||||
|
'CREATE DEFINER=`schema_user`@`192.168.1.10` EVENT `purge_market_snapshot` ON SCHEDULE EVERY 1 DAY DO DELETE FROM `market_snapshot` WHERE `id` < 0',
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
function catalogRowsFor(sql) {
|
||||||
|
if (sql.includes('information_schema.TABLES')) {
|
||||||
|
return [
|
||||||
|
{ TABLE_NAME: 'market_snapshot', TABLE_TYPE: 'BASE TABLE' },
|
||||||
|
{ TABLE_NAME: 'market_dashboard', TABLE_TYPE: 'VIEW' },
|
||||||
|
{ TABLE_NAME: 'market_summary', TABLE_TYPE: 'VIEW' },
|
||||||
|
];
|
||||||
|
}
|
||||||
|
if (sql.includes('information_schema.VIEW_TABLE_USAGE')) {
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
VIEW_NAME: 'market_dashboard',
|
||||||
|
REFERENCED_SCHEMA_NAME: 'a_share_analysis_dev',
|
||||||
|
REFERENCED_TABLE_NAME: 'market_summary',
|
||||||
|
},
|
||||||
|
{
|
||||||
|
VIEW_NAME: 'market_summary',
|
||||||
|
REFERENCED_SCHEMA_NAME: 'a_share_analysis_dev',
|
||||||
|
REFERENCED_TABLE_NAME: 'market_snapshot',
|
||||||
|
},
|
||||||
|
];
|
||||||
|
}
|
||||||
|
if (sql.includes('information_schema.ROUTINES')) {
|
||||||
|
return [
|
||||||
|
{ ROUTINE_NAME: 'calculate_breadth', ROUTINE_TYPE: 'FUNCTION' },
|
||||||
|
{ ROUTINE_NAME: 'refresh_market_snapshot', ROUTINE_TYPE: 'PROCEDURE' },
|
||||||
|
];
|
||||||
|
}
|
||||||
|
if (sql.includes('information_schema.TRIGGERS')) {
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
TRIGGER_NAME: 'snapshot_before_insert',
|
||||||
|
EVENT_OBJECT_TABLE: 'market_snapshot',
|
||||||
|
ACTION_TIMING: 'BEFORE',
|
||||||
|
EVENT_MANIPULATION: 'INSERT',
|
||||||
|
ACTION_ORDER: 1,
|
||||||
|
},
|
||||||
|
];
|
||||||
|
}
|
||||||
|
if (sql.includes('information_schema.EVENTS')) {
|
||||||
|
return [{ EVENT_NAME: 'purge_market_snapshot' }];
|
||||||
|
}
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
function createExportConnection({ mutateCatalog, mutateDefinition, queryError } = {}) {
|
||||||
|
let catalogRound = 0;
|
||||||
|
let definitionRound = 0;
|
||||||
|
let tableCatalogCalls = 0;
|
||||||
|
let tableDefinitionCalls = 0;
|
||||||
|
const connection = {
|
||||||
|
endCalls: 0,
|
||||||
|
async query(options) {
|
||||||
|
if (queryError) throw queryError;
|
||||||
|
if (options.sql.includes('information_schema.SCHEMATA')) {
|
||||||
|
return [
|
||||||
|
[
|
||||||
|
{
|
||||||
|
SERVER_VERSION: '5.7.42-log',
|
||||||
|
DATABASE_CHARSET: 'utf8mb4',
|
||||||
|
DATABASE_COLLATION: 'utf8mb4_general_ci',
|
||||||
|
},
|
||||||
|
],
|
||||||
|
[],
|
||||||
|
];
|
||||||
|
}
|
||||||
|
const catalogRows = catalogRowsFor(options.sql);
|
||||||
|
if (catalogRows) {
|
||||||
|
if (options.sql.includes('information_schema.TABLES')) {
|
||||||
|
tableCatalogCalls += 1;
|
||||||
|
catalogRound = tableCatalogCalls > 1 ? 1 : 0;
|
||||||
|
}
|
||||||
|
const rows = mutateCatalog?.(options.sql, catalogRows, catalogRound) ?? catalogRows;
|
||||||
|
return [rows, []];
|
||||||
|
}
|
||||||
|
const row = createRows[options.sql];
|
||||||
|
assert.ok(row, `未识别的 SHOW CREATE:${options.sql}`);
|
||||||
|
if (options.sql.includes('SHOW CREATE TABLE')) {
|
||||||
|
tableDefinitionCalls += 1;
|
||||||
|
definitionRound = tableDefinitionCalls > 1 ? 1 : 0;
|
||||||
|
}
|
||||||
|
const changed = mutateDefinition?.(options.sql, row, definitionRound);
|
||||||
|
return [changed === undefined ? [row] : changed, []];
|
||||||
|
},
|
||||||
|
async end() {
|
||||||
|
connection.endCalls += 1;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
return connection;
|
||||||
|
}
|
||||||
|
|
||||||
|
test('读取并校验固定开发库服务端元数据', async () => {
|
||||||
|
const connection = createExportConnection();
|
||||||
|
assert.deepEqual(
|
||||||
|
await readMysql57ServerMetadata(connection, 'a_share_analysis_dev', DEFAULT_EXPORT_LIMITS),
|
||||||
|
{
|
||||||
|
serverVersion: '5.7.42-log',
|
||||||
|
databaseCharset: 'utf8mb4',
|
||||||
|
databaseCollation: 'utf8mb4_general_ci',
|
||||||
|
},
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('SHOW CREATE 覆盖全部对象并按可恢复顺序渲染', async () => {
|
||||||
|
const connection = createExportConnection();
|
||||||
|
const definitions = await readDefinitions(connection, completeCatalog, DEFAULT_EXPORT_LIMITS);
|
||||||
|
const sql = renderSchemaSql(completeCatalog, definitions);
|
||||||
|
|
||||||
|
const order = [
|
||||||
|
'CREATE TABLE `market_snapshot`',
|
||||||
|
'FUNCTION `calculate_breadth`',
|
||||||
|
'PROCEDURE `refresh_market_snapshot`',
|
||||||
|
'VIEW `market_summary`',
|
||||||
|
'VIEW `market_dashboard`',
|
||||||
|
'TRIGGER `snapshot_before_insert`',
|
||||||
|
'EVENT `purge_market_snapshot`',
|
||||||
|
].map((fragment) => sql.indexOf(fragment));
|
||||||
|
assert.ok(order.every((index) => index >= 0));
|
||||||
|
assert.deepEqual(order, [...order].sort((left, right) => left - right));
|
||||||
|
assert.match(sql, /DELIMITER ;;\nCREATE PROCEDURE[\s\S]+END ;;\nDELIMITER ;/);
|
||||||
|
assert.doesNotMatch(sql, /DEFINER\s*=/i);
|
||||||
|
assert.doesNotMatch(sql, /AUTO_INCREMENT=12/);
|
||||||
|
assert.match(sql, /`id` bigint NOT NULL AUTO_INCREMENT/);
|
||||||
|
assert.match(sql, /UPDATE `market_snapshot`/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('定义规范化只移除账号绑定和当前自增值', () => {
|
||||||
|
const routine = normalizeCreateDefinition(
|
||||||
|
"CREATE DEFINER=`user`@`host` PROCEDURE `sample`() SELECT 'DEFINER=`kept`@`text` AUTO_INCREMENT=99'",
|
||||||
|
);
|
||||||
|
const table = normalizeCreateDefinition(
|
||||||
|
'CREATE TABLE `sample` (`id` bigint AUTO_INCREMENT) ENGINE=InnoDB AUTO_INCREMENT=99 DEFAULT CHARSET=utf8mb4;',
|
||||||
|
);
|
||||||
|
assert.equal(
|
||||||
|
routine,
|
||||||
|
"CREATE PROCEDURE `sample`() SELECT 'DEFINER=`kept`@`text` AUTO_INCREMENT=99'",
|
||||||
|
);
|
||||||
|
assert.equal(
|
||||||
|
table,
|
||||||
|
'CREATE TABLE `sample` (`id` bigint AUTO_INCREMENT) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('规范化保留例程中的自增文本并拒绝主机信息', () => {
|
||||||
|
const routine = normalizeCreateDefinition(
|
||||||
|
"CREATE PROCEDURE `sample`() SELECT ' AUTO_INCREMENT=123 DEFAULT CHARSET=utf8mb4'",
|
||||||
|
);
|
||||||
|
assert.equal(
|
||||||
|
routine,
|
||||||
|
"CREATE PROCEDURE `sample`() SELECT ' AUTO_INCREMENT=123 DEFAULT CHARSET=utf8mb4'",
|
||||||
|
);
|
||||||
|
assert.throws(
|
||||||
|
() => normalizeCreateDefinition('CREATE TABLE `sample` (`id` int) /* Host: prod-db.internal */'),
|
||||||
|
(error) => error.code === 'UNSAFE_DEFINITION',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('完整导出双重读取一致并始终关闭连接', async () => {
|
||||||
|
const connection = createExportConnection();
|
||||||
|
let receivedOptions;
|
||||||
|
const result = await exportMysql57Schema({
|
||||||
|
config,
|
||||||
|
createConnection: async (options) => {
|
||||||
|
receivedOptions = options;
|
||||||
|
return connection;
|
||||||
|
},
|
||||||
|
now: () => 100,
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.equal(receivedOptions.multipleStatements, false);
|
||||||
|
assert.equal(connection.endCalls, 1);
|
||||||
|
assert.deepEqual(result.objectCounts, {
|
||||||
|
tables: 1,
|
||||||
|
views: 2,
|
||||||
|
triggers: 1,
|
||||||
|
routines: 2,
|
||||||
|
events: 1,
|
||||||
|
});
|
||||||
|
assert.equal(result.durationMs, 0);
|
||||||
|
assert.equal(result.metadata.driverName, 'mysql2');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('导出拒绝目录变化和定义变化', async () => {
|
||||||
|
const catalogConnection = createExportConnection({
|
||||||
|
mutateCatalog(sql, rows, round) {
|
||||||
|
if (round === 1 && sql.includes('information_schema.TABLES')) {
|
||||||
|
return [...rows, { TABLE_NAME: 'new_table', TABLE_TYPE: 'BASE TABLE' }];
|
||||||
|
}
|
||||||
|
return rows;
|
||||||
|
},
|
||||||
|
});
|
||||||
|
await assert.rejects(
|
||||||
|
() =>
|
||||||
|
exportMysql57Schema({
|
||||||
|
config,
|
||||||
|
createConnection: async () => catalogConnection,
|
||||||
|
now: () => 100,
|
||||||
|
}),
|
||||||
|
(error) => error.code === 'CATALOG_CHANGED_DURING_EXPORT',
|
||||||
|
);
|
||||||
|
assert.equal(catalogConnection.endCalls, 1);
|
||||||
|
|
||||||
|
const definitionConnection = createExportConnection({
|
||||||
|
mutateDefinition(sql, row, round) {
|
||||||
|
if (round === 1 && sql.includes('SHOW CREATE TABLE')) {
|
||||||
|
return [{ ...row, 'Create Table': `${row['Create Table']} COMMENT='changed'` }];
|
||||||
|
}
|
||||||
|
return undefined;
|
||||||
|
},
|
||||||
|
});
|
||||||
|
await assert.rejects(
|
||||||
|
() =>
|
||||||
|
exportMysql57Schema({
|
||||||
|
config,
|
||||||
|
createConnection: async () => definitionConnection,
|
||||||
|
now: () => 100,
|
||||||
|
}),
|
||||||
|
(error) => error.code === 'DEFINITION_CHANGED_DURING_EXPORT',
|
||||||
|
);
|
||||||
|
assert.equal(definitionConnection.endCalls, 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('SHOW CREATE 拒绝空结果、错误形状和过大定义', async () => {
|
||||||
|
for (const [changed, expectedCode] of [
|
||||||
|
[[], 'SHOW_CREATE_EMPTY'],
|
||||||
|
[[{ unexpected: 'value' }], 'SHOW_CREATE_SHAPE_INVALID'],
|
||||||
|
]) {
|
||||||
|
const connection = createExportConnection({
|
||||||
|
mutateDefinition(sql) {
|
||||||
|
return sql.includes('SHOW CREATE TABLE') ? changed : undefined;
|
||||||
|
},
|
||||||
|
});
|
||||||
|
await assert.rejects(
|
||||||
|
() => readDefinitions(connection, completeCatalog, DEFAULT_EXPORT_LIMITS),
|
||||||
|
(error) => error.code === expectedCode,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const oversized = createExportConnection({
|
||||||
|
mutateDefinition(sql, row) {
|
||||||
|
return sql.includes('SHOW CREATE TABLE')
|
||||||
|
? [{ ...row, 'Create Table': 'x'.repeat(100) }]
|
||||||
|
: undefined;
|
||||||
|
},
|
||||||
|
});
|
||||||
|
await assert.rejects(
|
||||||
|
() =>
|
||||||
|
readDefinitions(oversized, completeCatalog, {
|
||||||
|
...DEFAULT_EXPORT_LIMITS,
|
||||||
|
maxDefinitionBytes: 32,
|
||||||
|
}),
|
||||||
|
(error) => error.code === 'DEFINITION_LIMIT_EXCEEDED',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('直接元数据和 SHOW CREATE 查询错误统一脱敏且保留既有业务错误', async () => {
|
||||||
|
const rawFailure = new Error('host=prod-db.internal password=not-for-logs');
|
||||||
|
await assert.rejects(
|
||||||
|
() => readMysql57ServerMetadata({ query: async () => { throw rawFailure; } }, 'a_share_analysis_dev'),
|
||||||
|
(error) =>
|
||||||
|
error.code === 'DATABASE_QUERY_FAILED' &&
|
||||||
|
!error.message.includes('prod-db.internal') &&
|
||||||
|
!error.message.includes('not-for-logs'),
|
||||||
|
);
|
||||||
|
await assert.rejects(
|
||||||
|
() => readDefinitions({ query: async () => { throw rawFailure; } }, completeCatalog),
|
||||||
|
(error) =>
|
||||||
|
error.code === 'DATABASE_QUERY_FAILED' &&
|
||||||
|
!error.message.includes('prod-db.internal') &&
|
||||||
|
!error.message.includes('not-for-logs'),
|
||||||
|
);
|
||||||
|
const timeout = new SchemaExportError('EXPORT_TIMEOUT', 'Schema 导出超过总时长限制');
|
||||||
|
await assert.rejects(
|
||||||
|
() => readDefinitions({ query: async () => { throw timeout; } }, completeCatalog),
|
||||||
|
(error) => error === timeout,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('查询失败和总超时只返回脱敏错误并关闭连接', async () => {
|
||||||
|
const failed = createExportConnection({
|
||||||
|
queryError: new Error('connect 127.0.0.1 password=not-for-logs'),
|
||||||
|
});
|
||||||
|
await assert.rejects(
|
||||||
|
() =>
|
||||||
|
exportMysql57Schema({
|
||||||
|
config,
|
||||||
|
createConnection: async () => failed,
|
||||||
|
now: () => 100,
|
||||||
|
}),
|
||||||
|
(error) =>
|
||||||
|
error.code === 'DATABASE_QUERY_FAILED' &&
|
||||||
|
!error.message.includes('127.0.0.1') &&
|
||||||
|
!error.message.includes('not-for-logs'),
|
||||||
|
);
|
||||||
|
assert.equal(failed.endCalls, 1);
|
||||||
|
|
||||||
|
const timedOut = createExportConnection();
|
||||||
|
let tick = 0;
|
||||||
|
await assert.rejects(
|
||||||
|
() =>
|
||||||
|
exportMysql57Schema({
|
||||||
|
config,
|
||||||
|
createConnection: async () => timedOut,
|
||||||
|
now: () => (tick++ === 0 ? 0 : 30_001),
|
||||||
|
}),
|
||||||
|
(error) => error.code === 'EXPORT_TIMEOUT',
|
||||||
|
);
|
||||||
|
assert.equal(timedOut.endCalls, 1);
|
||||||
|
});
|
||||||
@@ -12,12 +12,20 @@ import { createHash } from 'node:crypto';
|
|||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
import { fileURLToPath } from 'node:url';
|
import { fileURLToPath } from 'node:url';
|
||||||
|
|
||||||
|
import {
|
||||||
|
MYSQL2_EXPORTER_METADATA,
|
||||||
|
assertMysql57ServerMetadata as assertMysql57ServerMetadataV3,
|
||||||
|
exportMysql57Schema,
|
||||||
|
} from './mysql2-schema-exporter.mjs';
|
||||||
|
|
||||||
const DEVELOPMENT_DATABASE = 'a_share_analysis_dev';
|
const DEVELOPMENT_DATABASE = 'a_share_analysis_dev';
|
||||||
const REQUIRED_CHARSET = 'utf8mb4';
|
const REQUIRED_CHARSET = 'utf8mb4';
|
||||||
const REQUIRED_COLLATION = 'utf8mb4_general_ci';
|
const REQUIRED_COLLATION = 'utf8mb4_general_ci';
|
||||||
const BASELINE_FORMAT_VERSION = 2;
|
const BASELINE_FORMAT_VERSION = 3;
|
||||||
|
const PREVIOUS_BASELINE_FORMAT_VERSION = 2;
|
||||||
const LEGACY_BASELINE_FORMAT_VERSION = 1;
|
const LEGACY_BASELINE_FORMAT_VERSION = 1;
|
||||||
const TOOL_VERSION = '2.0.0';
|
const TOOL_VERSION = '3.0.0';
|
||||||
|
const PREVIOUS_TOOL_VERSION = '2.0.0';
|
||||||
const LEGACY_TOOL_VERSION = '1.0.0';
|
const LEGACY_TOOL_VERSION = '1.0.0';
|
||||||
const VERSION_PATTERN = /^v\d+\.\d+\.\d+$/;
|
const VERSION_PATTERN = /^v\d+\.\d+\.\d+$/;
|
||||||
const DEVELOPMENT_HOSTS = new Set(['127.0.0.1', 'localhost', '::1', '112.192.20.11']);
|
const DEVELOPMENT_HOSTS = new Set(['127.0.0.1', 'localhost', '::1', '112.192.20.11']);
|
||||||
@@ -283,21 +291,17 @@ function countSchemaObjects(sql) {
|
|||||||
function buildManifest(fields) {
|
function buildManifest(fields) {
|
||||||
// JSON 是 YAML 1.2 的严格子集;使用标准解析器生成和读取,避免手写 YAML
|
// JSON 是 YAML 1.2 的严格子集;使用标准解析器生成和读取,避免手写 YAML
|
||||||
// 对缩进、引号和同名嵌套字段产生歧义,同时仍保留约定的 manifest.yaml 文件名。
|
// 对缩进、引号和同名嵌套字段产生歧义,同时仍保留约定的 manifest.yaml 文件名。
|
||||||
return `${JSON.stringify(
|
const common = {
|
||||||
{
|
format_version: fields.formatVersion,
|
||||||
format_version: BASELINE_FORMAT_VERSION,
|
|
||||||
status: fields.status,
|
status: fields.status,
|
||||||
application_version: fields.applicationVersion,
|
application_version: fields.applicationVersion,
|
||||||
git_commit: fields.gitCommit,
|
git_commit: fields.gitCommit,
|
||||||
source_environment: 'development',
|
source_environment: 'development',
|
||||||
database_name: DEVELOPMENT_DATABASE,
|
database_name: DEVELOPMENT_DATABASE,
|
||||||
mysql_client_version: fields.mysqlClientVersion,
|
|
||||||
mysqldump_client_version: fields.mysqldumpClientVersion,
|
|
||||||
mysql_server_version: fields.serverVersion,
|
|
||||||
charset: REQUIRED_CHARSET,
|
charset: REQUIRED_CHARSET,
|
||||||
collation: REQUIRED_COLLATION,
|
collation: REQUIRED_COLLATION,
|
||||||
generated_at: fields.generatedAt,
|
generated_at: fields.generatedAt,
|
||||||
generator: `schema-baseline/${TOOL_VERSION}`,
|
generator: `schema-baseline/${fields.formatVersion === 3 ? TOOL_VERSION : fields.formatVersion === 2 ? PREVIOUS_TOOL_VERSION : LEGACY_TOOL_VERSION}`,
|
||||||
reason: fields.reason ?? 'none',
|
reason: fields.reason ?? 'none',
|
||||||
schema_sha256: fields.schemaSha256 ?? 'not_applicable',
|
schema_sha256: fields.schemaSha256 ?? 'not_applicable',
|
||||||
objects: {
|
objects: {
|
||||||
@@ -308,20 +312,49 @@ function buildManifest(fields) {
|
|||||||
events: fields.captured,
|
events: fields.captured,
|
||||||
},
|
},
|
||||||
object_counts: fields.objectCounts,
|
object_counts: fields.objectCounts,
|
||||||
},
|
};
|
||||||
null,
|
if (fields.formatVersion === 3) {
|
||||||
2,
|
common.exporter_name = fields.exporterName;
|
||||||
)}\n`;
|
common.exporter_version = fields.exporterVersion;
|
||||||
|
common.database_driver = fields.driverName;
|
||||||
|
common.database_driver_version = fields.driverVersion;
|
||||||
|
common.mysql_server_version = fields.serverVersion;
|
||||||
|
} else {
|
||||||
|
common.mysql_client_version = fields.mysqlClientVersion;
|
||||||
|
common.mysqldump_client_version = fields.mysqldumpClientVersion;
|
||||||
|
common.mysql_server_version = fields.serverVersion;
|
||||||
|
}
|
||||||
|
return `${JSON.stringify(common, null, 2)}\n`;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function buildCapturedArtifacts(metadata, sql) {
|
function assertNodeMetadata(metadata) {
|
||||||
assertMysql57Metadata(metadata);
|
if (metadata.formatVersion !== 3) {
|
||||||
if (!VERSION_PATTERN.test(metadata.applicationVersion ?? '')) {
|
throw new Error('Node Schema 基线必须使用 format v3');
|
||||||
throw new Error(`应用版本格式无效:${metadata.applicationVersion || '(空)'}`);
|
}
|
||||||
|
for (const key of ['exporterName', 'exporterVersion', 'driverName', 'driverVersion']) {
|
||||||
|
if (!String(metadata[key] ?? '').trim()) {
|
||||||
|
throw new Error(`Node Schema 元数据缺少 ${key}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assertMysql57ServerMetadataV3(metadata);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function buildCapturedArtifacts(metadata, sql, objectCounts) {
|
||||||
|
if (metadata.formatVersion === 3) {
|
||||||
|
assertNodeMetadata(metadata);
|
||||||
|
if (!objectCounts || Object.keys(objectCounts).sort().join(',') !== [...OBJECT_KEYS].sort().join(',')) {
|
||||||
|
throw new Error('Node Schema 对象计数字段无效');
|
||||||
}
|
}
|
||||||
const normalizedSql = normalizeSchemaSql(sql);
|
const normalizedSql = normalizeSchemaSql(sql);
|
||||||
const objectCounts = countSchemaObjects(normalizedSql);
|
const derivedCounts = countSchemaObjects(normalizedSql);
|
||||||
// mysqldump 对空库仍会输出会话 SET 与头尾注释;结构对象计数为零才是空库判据。
|
for (const key of OBJECT_KEYS) {
|
||||||
|
if (!Number.isInteger(objectCounts[key]) || objectCounts[key] < 0 || objectCounts[key] !== derivedCounts[key]) {
|
||||||
|
throw new Error(`Node Schema 对象计数与 SQL 不一致:${key}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (String(sql).trim() && Object.values(objectCounts).every((count) => count === 0)) {
|
||||||
|
throw new Error('Node Schema SQL 非空但对象计数全为零');
|
||||||
|
}
|
||||||
if (Object.values(objectCounts).every((count) => count === 0)) {
|
if (Object.values(objectCounts).every((count) => count === 0)) {
|
||||||
return buildCapturedEmptyArtifacts(metadata);
|
return buildCapturedEmptyArtifacts(metadata);
|
||||||
}
|
}
|
||||||
@@ -343,8 +376,54 @@ export function buildCapturedArtifacts(metadata, sql) {
|
|||||||
}),
|
}),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
assertMysql57Metadata(metadata);
|
||||||
|
if (!VERSION_PATTERN.test(metadata.applicationVersion ?? '')) {
|
||||||
|
throw new Error(`应用版本格式无效:${metadata.applicationVersion || '(空)'}`);
|
||||||
|
}
|
||||||
|
const normalizedSql = normalizeSchemaSql(sql);
|
||||||
|
const derivedCounts = countSchemaObjects(normalizedSql);
|
||||||
|
// mysqldump 对空库仍会输出会话 SET 与头尾注释;结构对象计数为零才是空库判据。
|
||||||
|
if (Object.values(derivedCounts).every((count) => count === 0)) {
|
||||||
|
return buildCapturedEmptyArtifacts(metadata);
|
||||||
|
}
|
||||||
|
const schemaSha256 = sha256(normalizedSql);
|
||||||
|
const manifest = buildManifest({
|
||||||
|
...metadata,
|
||||||
|
formatVersion: metadata.formatVersion ?? PREVIOUS_BASELINE_FORMAT_VERSION,
|
||||||
|
status: 'captured',
|
||||||
|
captured: true,
|
||||||
|
objectCounts: derivedCounts,
|
||||||
|
schemaSha256,
|
||||||
|
});
|
||||||
|
return Object.freeze({
|
||||||
|
status: 'captured',
|
||||||
|
schemaSha256,
|
||||||
|
files: Object.freeze({
|
||||||
|
'manifest.yaml': manifest,
|
||||||
|
'schema.sql': normalizedSql,
|
||||||
|
'checksum.sha256': `${schemaSha256} schema.sql\n`,
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
export function buildCapturedEmptyArtifacts(metadata) {
|
export function buildCapturedEmptyArtifacts(metadata) {
|
||||||
|
if (metadata.formatVersion === 3) {
|
||||||
|
assertNodeMetadata(metadata);
|
||||||
|
const schemaSha256 = sha256('');
|
||||||
|
const objectCounts = Object.freeze(Object.fromEntries(OBJECT_KEYS.map((key) => [key, 0])));
|
||||||
|
const manifest = buildManifest({
|
||||||
|
...metadata,
|
||||||
|
status: 'captured_empty',
|
||||||
|
captured: false,
|
||||||
|
objectCounts,
|
||||||
|
schemaSha256,
|
||||||
|
});
|
||||||
|
return Object.freeze({
|
||||||
|
status: 'captured_empty',
|
||||||
|
schemaSha256,
|
||||||
|
files: Object.freeze({ 'manifest.yaml': manifest }),
|
||||||
|
});
|
||||||
|
}
|
||||||
assertMysql57Metadata(metadata);
|
assertMysql57Metadata(metadata);
|
||||||
if (!VERSION_PATTERN.test(metadata.applicationVersion ?? '')) {
|
if (!VERSION_PATTERN.test(metadata.applicationVersion ?? '')) {
|
||||||
throw new Error(`应用版本格式无效:${metadata.applicationVersion || '(空)'}`);
|
throw new Error(`应用版本格式无效:${metadata.applicationVersion || '(空)'}`);
|
||||||
@@ -353,6 +432,7 @@ export function buildCapturedEmptyArtifacts(metadata) {
|
|||||||
const objectCounts = Object.freeze(Object.fromEntries(OBJECT_KEYS.map((key) => [key, 0])));
|
const objectCounts = Object.freeze(Object.fromEntries(OBJECT_KEYS.map((key) => [key, 0])));
|
||||||
const manifest = buildManifest({
|
const manifest = buildManifest({
|
||||||
...metadata,
|
...metadata,
|
||||||
|
formatVersion: metadata.formatVersion ?? PREVIOUS_BASELINE_FORMAT_VERSION,
|
||||||
status: 'captured_empty',
|
status: 'captured_empty',
|
||||||
captured: false,
|
captured: false,
|
||||||
objectCounts,
|
objectCounts,
|
||||||
@@ -369,8 +449,14 @@ export function buildNotApplicableArtifacts(metadata) {
|
|||||||
if (!VERSION_PATTERN.test(metadata.applicationVersion ?? '')) {
|
if (!VERSION_PATTERN.test(metadata.applicationVersion ?? '')) {
|
||||||
throw new Error(`应用版本格式无效:${metadata.applicationVersion || '(空)'}`);
|
throw new Error(`应用版本格式无效:${metadata.applicationVersion || '(空)'}`);
|
||||||
}
|
}
|
||||||
const manifest = buildManifest({
|
const nodeMetadata = metadata.formatVersion === 3 ? metadata : {
|
||||||
...metadata,
|
...metadata,
|
||||||
|
formatVersion: PREVIOUS_BASELINE_FORMAT_VERSION,
|
||||||
|
};
|
||||||
|
const manifest = buildManifest({
|
||||||
|
...nodeMetadata,
|
||||||
|
...metadata,
|
||||||
|
formatVersion: nodeMetadata.formatVersion,
|
||||||
mysqlClientVersion: 'not_enabled',
|
mysqlClientVersion: 'not_enabled',
|
||||||
mysqldumpClientVersion: 'not_enabled',
|
mysqldumpClientVersion: 'not_enabled',
|
||||||
serverVersion: 'not_enabled',
|
serverVersion: 'not_enabled',
|
||||||
@@ -403,28 +489,65 @@ function compareExistingArtifacts(targetDirectory, artifacts) {
|
|||||||
* 先在目标父目录写入临时目录,再原子改名。这样异常不会留下半份基线;
|
* 先在目标父目录写入临时目录,再原子改名。这样异常不会留下半份基线;
|
||||||
* 已发布版本目录只允许逐字节相同的幂等重跑,任何差异都必须人工留痕修正。
|
* 已发布版本目录只允许逐字节相同的幂等重跑,任何差异都必须人工留痕修正。
|
||||||
*/
|
*/
|
||||||
export function persistBaselineArtifacts(targetDirectory, artifacts) {
|
const DEFAULT_FILE_OPERATIONS = Object.freeze({
|
||||||
if (existsSync(targetDirectory)) {
|
existsSync,
|
||||||
if (compareExistingArtifacts(targetDirectory, artifacts)) {
|
mkdirSync,
|
||||||
|
readFileSync,
|
||||||
|
readdirSync,
|
||||||
|
renameSync,
|
||||||
|
rmSync,
|
||||||
|
writeFileSync,
|
||||||
|
});
|
||||||
|
|
||||||
|
export function persistBaselineArtifacts(
|
||||||
|
targetDirectory,
|
||||||
|
artifacts,
|
||||||
|
fileOperations = DEFAULT_FILE_OPERATIONS,
|
||||||
|
) {
|
||||||
|
const operations = { ...DEFAULT_FILE_OPERATIONS, ...fileOperations };
|
||||||
|
const {
|
||||||
|
existsSync: fileExists,
|
||||||
|
mkdirSync: makeDirectory,
|
||||||
|
readFileSync: readFile,
|
||||||
|
readdirSync: readDirectory,
|
||||||
|
renameSync: renameFile,
|
||||||
|
rmSync: removePath,
|
||||||
|
writeFileSync: writeFile,
|
||||||
|
} = operations;
|
||||||
|
const compare = () => {
|
||||||
|
const actualFiles = readDirectory(targetDirectory).sort();
|
||||||
|
const expectedFiles = Object.keys(artifacts.files).sort();
|
||||||
|
if (
|
||||||
|
actualFiles.length !== expectedFiles.length ||
|
||||||
|
actualFiles.some((file, index) => file !== expectedFiles[index])
|
||||||
|
) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return expectedFiles.every(
|
||||||
|
(file) => readFile(path.join(targetDirectory, file), 'utf8') === artifacts.files[file],
|
||||||
|
);
|
||||||
|
};
|
||||||
|
if (fileExists(targetDirectory)) {
|
||||||
|
if (compare()) {
|
||||||
return { reused: true };
|
return { reused: true };
|
||||||
}
|
}
|
||||||
throw new Error(`目标版本基线已存在且内容不同,禁止覆盖:${targetDirectory}`);
|
throw new Error(`目标版本基线已存在且内容不同,禁止覆盖:${targetDirectory}`);
|
||||||
}
|
}
|
||||||
|
|
||||||
const parentDirectory = path.dirname(targetDirectory);
|
const parentDirectory = path.dirname(targetDirectory);
|
||||||
mkdirSync(parentDirectory, { recursive: true });
|
makeDirectory(parentDirectory, { recursive: true });
|
||||||
const temporaryDirectory = path.join(
|
const temporaryDirectory = path.join(
|
||||||
parentDirectory,
|
parentDirectory,
|
||||||
`.${path.basename(targetDirectory)}.tmp-${process.pid}-${Date.now()}`,
|
`.${path.basename(targetDirectory)}.tmp-${process.pid}-${Date.now()}`,
|
||||||
);
|
);
|
||||||
mkdirSync(temporaryDirectory);
|
makeDirectory(temporaryDirectory);
|
||||||
try {
|
try {
|
||||||
for (const [file, content] of Object.entries(artifacts.files)) {
|
for (const [file, content] of Object.entries(artifacts.files)) {
|
||||||
writeFileSync(path.join(temporaryDirectory, file), content, 'utf8');
|
writeFile(path.join(temporaryDirectory, file), content, 'utf8');
|
||||||
}
|
}
|
||||||
renameSync(temporaryDirectory, targetDirectory);
|
renameFile(temporaryDirectory, targetDirectory);
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
rmSync(temporaryDirectory, { recursive: true, force: true });
|
removePath(temporaryDirectory, { recursive: true, force: true });
|
||||||
throw error;
|
throw error;
|
||||||
}
|
}
|
||||||
return { reused: false };
|
return { reused: false };
|
||||||
@@ -468,6 +591,8 @@ export function verifyBaselineDirectory(targetDirectory, expectedRelease, expect
|
|||||||
const supportedFormat =
|
const supportedFormat =
|
||||||
(manifest.format_version === LEGACY_BASELINE_FORMAT_VERSION &&
|
(manifest.format_version === LEGACY_BASELINE_FORMAT_VERSION &&
|
||||||
manifest.generator === `schema-baseline/${LEGACY_TOOL_VERSION}`) ||
|
manifest.generator === `schema-baseline/${LEGACY_TOOL_VERSION}`) ||
|
||||||
|
(manifest.format_version === PREVIOUS_BASELINE_FORMAT_VERSION &&
|
||||||
|
manifest.generator === `schema-baseline/${PREVIOUS_TOOL_VERSION}`) ||
|
||||||
(manifest.format_version === BASELINE_FORMAT_VERSION &&
|
(manifest.format_version === BASELINE_FORMAT_VERSION &&
|
||||||
manifest.generator === `schema-baseline/${TOOL_VERSION}`);
|
manifest.generator === `schema-baseline/${TOOL_VERSION}`);
|
||||||
if (
|
if (
|
||||||
@@ -506,23 +631,41 @@ export function verifyBaselineDirectory(targetDirectory, expectedRelease, expect
|
|||||||
assertExactFileSet(targetDirectory, NOT_APPLICABLE_FILES);
|
assertExactFileSet(targetDirectory, NOT_APPLICABLE_FILES);
|
||||||
const reason = manifest.reason;
|
const reason = manifest.reason;
|
||||||
requireExactValue(reason, 'mysql_disabled', 'Schema 不适用原因无效');
|
requireExactValue(reason, 'mysql_disabled', 'Schema 不适用原因无效');
|
||||||
if (
|
const commonInvalid =
|
||||||
manifest.mysql_client_version !== 'not_enabled' ||
|
|
||||||
manifest.mysqldump_client_version !== 'not_enabled' ||
|
|
||||||
manifest.mysql_server_version !== 'not_enabled' ||
|
manifest.mysql_server_version !== 'not_enabled' ||
|
||||||
manifest.schema_sha256 !== 'not_applicable' ||
|
manifest.schema_sha256 !== 'not_applicable' ||
|
||||||
Object.keys(manifest.objects ?? {}).sort().join(',') !== [...OBJECT_KEYS].sort().join(',') ||
|
Object.keys(manifest.objects ?? {}).sort().join(',') !==
|
||||||
OBJECT_KEYS.some((key) => manifest.objects[key] !== false)
|
[...OBJECT_KEYS].sort().join(',') ||
|
||||||
) {
|
OBJECT_KEYS.some((key) => manifest.objects[key] !== false);
|
||||||
|
const formatInvalid =
|
||||||
|
manifest.format_version === BASELINE_FORMAT_VERSION
|
||||||
|
? !String(manifest.exporter_name ?? '').trim() ||
|
||||||
|
!String(manifest.exporter_version ?? '').trim() ||
|
||||||
|
!String(manifest.database_driver ?? '').trim() ||
|
||||||
|
!String(manifest.database_driver_version ?? '').trim() ||
|
||||||
|
'mysql_client_version' in manifest ||
|
||||||
|
'mysqldump_client_version' in manifest
|
||||||
|
: manifest.mysql_client_version !== 'not_enabled' ||
|
||||||
|
manifest.mysqldump_client_version !== 'not_enabled';
|
||||||
|
if (commonInvalid || formatInvalid) {
|
||||||
throw new Error('Schema 不适用清单必须证明 MySQL 未启用且所有对象均未捕获');
|
throw new Error('Schema 不适用清单必须证明 MySQL 未启用且所有对象均未捕获');
|
||||||
}
|
}
|
||||||
return { status, applicationVersion, reason };
|
return { status, applicationVersion, reason };
|
||||||
}
|
}
|
||||||
if (status === 'captured_empty') {
|
if (status === 'captured_empty') {
|
||||||
if (manifest.format_version !== BASELINE_FORMAT_VERSION) {
|
|
||||||
throw new Error('空库 Schema 基线只允许 format v2');
|
|
||||||
}
|
|
||||||
assertExactFileSet(targetDirectory, CAPTURED_EMPTY_FILES);
|
assertExactFileSet(targetDirectory, CAPTURED_EMPTY_FILES);
|
||||||
|
if (manifest.format_version === BASELINE_FORMAT_VERSION) {
|
||||||
|
assertNodeMetadata({
|
||||||
|
formatVersion: 3,
|
||||||
|
exporterName: manifest.exporter_name,
|
||||||
|
exporterVersion: manifest.exporter_version,
|
||||||
|
driverName: manifest.database_driver,
|
||||||
|
driverVersion: manifest.database_driver_version,
|
||||||
|
serverVersion: manifest.mysql_server_version,
|
||||||
|
databaseCharset: manifest.charset,
|
||||||
|
databaseCollation: manifest.collation,
|
||||||
|
});
|
||||||
|
} else {
|
||||||
assertMysql57Metadata({
|
assertMysql57Metadata({
|
||||||
mysqlClientVersion: manifest.mysql_client_version,
|
mysqlClientVersion: manifest.mysql_client_version,
|
||||||
mysqldumpClientVersion: manifest.mysqldump_client_version,
|
mysqldumpClientVersion: manifest.mysqldump_client_version,
|
||||||
@@ -530,6 +673,7 @@ export function verifyBaselineDirectory(targetDirectory, expectedRelease, expect
|
|||||||
databaseCharset: manifest.charset,
|
databaseCharset: manifest.charset,
|
||||||
databaseCollation: manifest.collation,
|
databaseCollation: manifest.collation,
|
||||||
});
|
});
|
||||||
|
}
|
||||||
const expectedEmptyChecksum = sha256('');
|
const expectedEmptyChecksum = sha256('');
|
||||||
if (
|
if (
|
||||||
manifest.schema_sha256 !== expectedEmptyChecksum ||
|
manifest.schema_sha256 !== expectedEmptyChecksum ||
|
||||||
@@ -546,6 +690,18 @@ export function verifyBaselineDirectory(targetDirectory, expectedRelease, expect
|
|||||||
}
|
}
|
||||||
|
|
||||||
assertExactFileSet(targetDirectory, CAPTURED_FILES);
|
assertExactFileSet(targetDirectory, CAPTURED_FILES);
|
||||||
|
if (manifest.format_version === BASELINE_FORMAT_VERSION) {
|
||||||
|
assertNodeMetadata({
|
||||||
|
formatVersion: 3,
|
||||||
|
exporterName: manifest.exporter_name,
|
||||||
|
exporterVersion: manifest.exporter_version,
|
||||||
|
driverName: manifest.database_driver,
|
||||||
|
driverVersion: manifest.database_driver_version,
|
||||||
|
serverVersion: manifest.mysql_server_version,
|
||||||
|
databaseCharset: manifest.charset,
|
||||||
|
databaseCollation: manifest.collation,
|
||||||
|
});
|
||||||
|
} else {
|
||||||
assertMysql57Metadata({
|
assertMysql57Metadata({
|
||||||
mysqlClientVersion: manifest.mysql_client_version,
|
mysqlClientVersion: manifest.mysql_client_version,
|
||||||
mysqldumpClientVersion: manifest.mysqldump_client_version,
|
mysqldumpClientVersion: manifest.mysqldump_client_version,
|
||||||
@@ -553,6 +709,7 @@ export function verifyBaselineDirectory(targetDirectory, expectedRelease, expect
|
|||||||
databaseCharset: manifest.charset,
|
databaseCharset: manifest.charset,
|
||||||
databaseCollation: manifest.collation,
|
databaseCollation: manifest.collation,
|
||||||
});
|
});
|
||||||
|
}
|
||||||
if (Object.keys(manifest.objects ?? {}).sort().join(',') !== [...OBJECT_KEYS].sort().join(',')) {
|
if (Object.keys(manifest.objects ?? {}).sort().join(',') !== [...OBJECT_KEYS].sort().join(',')) {
|
||||||
throw new Error('Schema 对象范围字段无效');
|
throw new Error('Schema 对象范围字段无效');
|
||||||
}
|
}
|
||||||
@@ -583,6 +740,14 @@ export function verifyBaselineDirectory(targetDirectory, expectedRelease, expect
|
|||||||
if (manifestChecksum !== schemaSha256 || checksumFile !== `${schemaSha256} schema.sql\n`) {
|
if (manifestChecksum !== schemaSha256 || checksumFile !== `${schemaSha256} schema.sql\n`) {
|
||||||
throw new Error('Schema SQL、manifest 和 checksum 校验值不一致');
|
throw new Error('Schema SQL、manifest 和 checksum 校验值不一致');
|
||||||
}
|
}
|
||||||
|
if (manifest.format_version === BASELINE_FORMAT_VERSION) {
|
||||||
|
const derivedCounts = countSchemaObjects(sql);
|
||||||
|
for (const key of OBJECT_KEYS) {
|
||||||
|
if (manifest.object_counts[key] !== derivedCounts[key]) {
|
||||||
|
throw new Error(`Schema 对象计数与 SQL 不一致:${key}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
return { status, applicationVersion, schemaSha256 };
|
return { status, applicationVersion, schemaSha256 };
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -614,7 +779,7 @@ function extractClientVersion(output) {
|
|||||||
* 真实执行器可注入是为了让测试证明“关闭时零数据库调用”和命令边界;默认执行器
|
* 真实执行器可注入是为了让测试证明“关闭时零数据库调用”和命令边界;默认执行器
|
||||||
* 仍使用 execFileSync 参数数组。密码仅放入子进程环境 MYSQL_PWD,不会进入 argv。
|
* 仍使用 execFileSync 参数数组。密码仅放入子进程环境 MYSQL_PWD,不会进入 argv。
|
||||||
*/
|
*/
|
||||||
export function runBaselineGeneration(options) {
|
export async function runBaselineGeneration(options) {
|
||||||
if ((options.commandArguments ?? []).length > 0) {
|
if ((options.commandArguments ?? []).length > 0) {
|
||||||
throw new Error('Schema 基线命令不接受命令参数,不能覆盖环境、数据库或 env 路径');
|
throw new Error('Schema 基线命令不接受命令参数,不能覆盖环境、数据库或 env 路径');
|
||||||
}
|
}
|
||||||
@@ -641,52 +806,33 @@ export function runBaselineGeneration(options) {
|
|||||||
generatedAt: options.generatedAt ?? new Date().toISOString(),
|
generatedAt: options.generatedAt ?? new Date().toISOString(),
|
||||||
};
|
};
|
||||||
if (!config.MYSQL_ENABLED) {
|
if (!config.MYSQL_ENABLED) {
|
||||||
const artifacts = buildNotApplicableArtifacts(commonMetadata);
|
const artifacts = buildNotApplicableArtifacts({
|
||||||
persistBaselineArtifacts(targetDirectory, artifacts);
|
...commonMetadata,
|
||||||
return verifyBaselineDirectory(targetDirectory, options.applicationVersion);
|
formatVersion: BASELINE_FORMAT_VERSION,
|
||||||
}
|
exporterName: MYSQL2_EXPORTER_METADATA.name,
|
||||||
|
exporterVersion: MYSQL2_EXPORTER_METADATA.version,
|
||||||
const executeFile = options.executeFile ?? execFileSync;
|
driverName: MYSQL2_EXPORTER_METADATA.driverName,
|
||||||
const childEnvironment = { ...process.env, MYSQL_PWD: config.MYSQL_PASSWORD ?? '' };
|
driverVersion: MYSQL2_EXPORTER_METADATA.driverVersion,
|
||||||
const executionOptions = {
|
serverVersion: 'not_enabled',
|
||||||
cwd: options.repositoryRoot,
|
|
||||||
encoding: 'utf8',
|
|
||||||
stdio: ['ignore', 'pipe', 'pipe'],
|
|
||||||
env: childEnvironment,
|
|
||||||
};
|
|
||||||
const mysqlClientVersion = extractClientVersion(
|
|
||||||
executeFile('mysql', ['--version'], executionOptions),
|
|
||||||
);
|
|
||||||
const mysqldumpClientVersion = extractClientVersion(
|
|
||||||
executeFile('mysqldump', ['--version'], executionOptions),
|
|
||||||
);
|
|
||||||
assertMysql57Metadata({
|
|
||||||
mysqlClientVersion,
|
|
||||||
mysqldumpClientVersion,
|
|
||||||
serverVersion: '5.7.0-pending-server-check',
|
|
||||||
databaseCharset: REQUIRED_CHARSET,
|
databaseCharset: REQUIRED_CHARSET,
|
||||||
databaseCollation: REQUIRED_COLLATION,
|
databaseCollation: REQUIRED_COLLATION,
|
||||||
});
|
});
|
||||||
const serverMetadata = String(
|
persistBaselineArtifacts(targetDirectory, artifacts, options.fileOperations);
|
||||||
executeFile('mysql', buildMysqlMetadataArguments(config), executionOptions),
|
return verifyBaselineDirectory(targetDirectory, options.applicationVersion);
|
||||||
)
|
|
||||||
.trim()
|
|
||||||
.split('\t');
|
|
||||||
if (serverMetadata.length !== 3) {
|
|
||||||
throw new Error('无法从固定开发库读取 MySQL 版本、字符集和排序规则');
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const exported = await (options.exportSchema ?? exportMysql57Schema)({ config });
|
||||||
|
const exportedMetadata = exported.metadata ?? {};
|
||||||
const metadata = {
|
const metadata = {
|
||||||
...commonMetadata,
|
...commonMetadata,
|
||||||
mysqlClientVersion,
|
...exportedMetadata,
|
||||||
mysqldumpClientVersion,
|
applicationVersion: exportedMetadata.applicationVersion ?? commonMetadata.applicationVersion,
|
||||||
serverVersion: serverMetadata[0],
|
gitCommit: exportedMetadata.gitCommit ?? commonMetadata.gitCommit,
|
||||||
databaseCharset: serverMetadata[1],
|
generatedAt: exportedMetadata.generatedAt ?? commonMetadata.generatedAt,
|
||||||
databaseCollation: serverMetadata[2],
|
formatVersion: BASELINE_FORMAT_VERSION,
|
||||||
};
|
};
|
||||||
assertMysql57Metadata(metadata);
|
const artifacts = buildCapturedArtifacts(metadata, exported.rawSql, exported.objectCounts);
|
||||||
const rawSql = executeFile('mysqldump', buildMysqldumpArguments(config), executionOptions);
|
persistBaselineArtifacts(targetDirectory, artifacts, options.fileOperations);
|
||||||
const artifacts = buildCapturedArtifacts(metadata, rawSql);
|
|
||||||
persistBaselineArtifacts(targetDirectory, artifacts);
|
|
||||||
return verifyBaselineDirectory(targetDirectory, options.applicationVersion);
|
return verifyBaselineDirectory(targetDirectory, options.applicationVersion);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -815,7 +961,7 @@ export function resolveFixedDevelopmentEnv(repositoryRoot) {
|
|||||||
return path.join(repositoryRoot, 'deploy', 'env', 'development', 'api.env.example');
|
return path.join(repositoryRoot, 'deploy', 'env', 'development', 'api.env.example');
|
||||||
}
|
}
|
||||||
|
|
||||||
function runCommandLine() {
|
export async function runCommandLine() {
|
||||||
const [command, ...commandArguments] = process.argv.slice(2);
|
const [command, ...commandArguments] = process.argv.slice(2);
|
||||||
if (!['generate', 'verify'].includes(command) || commandArguments.length > 0) {
|
if (!['generate', 'verify'].includes(command) || commandArguments.length > 0) {
|
||||||
throw new Error('用法:node tools/database/schema-baseline.mjs <generate|verify>(不接受额外参数)');
|
throw new Error('用法:node tools/database/schema-baseline.mjs <generate|verify>(不接受额外参数)');
|
||||||
@@ -824,7 +970,7 @@ function runCommandLine() {
|
|||||||
const release = getCurrentReleasedVersion(repositoryRoot);
|
const release = getCurrentReleasedVersion(repositoryRoot);
|
||||||
const result =
|
const result =
|
||||||
command === 'generate'
|
command === 'generate'
|
||||||
? runBaselineGeneration({
|
? await runBaselineGeneration({
|
||||||
repositoryRoot,
|
repositoryRoot,
|
||||||
envFile: resolveFixedDevelopmentEnv(repositoryRoot),
|
envFile: resolveFixedDevelopmentEnv(repositoryRoot),
|
||||||
...release,
|
...release,
|
||||||
@@ -837,5 +983,5 @@ function runCommandLine() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
|
||||||
runCommandLine();
|
await runCommandLine();
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
import assert from 'node:assert/strict';
|
import assert from 'node:assert/strict';
|
||||||
import { createHash } from 'node:crypto';
|
import { createHash } from 'node:crypto';
|
||||||
import { mkdirSync, mkdtempSync, readFileSync, writeFileSync } from 'node:fs';
|
import { mkdirSync, mkdtempSync, readFileSync, readdirSync, writeFileSync } from 'node:fs';
|
||||||
import { tmpdir } from 'node:os';
|
import { tmpdir } from 'node:os';
|
||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
import test from 'node:test';
|
import test from 'node:test';
|
||||||
@@ -45,6 +45,20 @@ const validMetadata = Object.freeze({
|
|||||||
generatedAt: '2026-08-13T03:00:00.000Z',
|
generatedAt: '2026-08-13T03:00:00.000Z',
|
||||||
});
|
});
|
||||||
|
|
||||||
|
const validNodeMetadata = Object.freeze({
|
||||||
|
formatVersion: 3,
|
||||||
|
applicationVersion: 'v0.0.9',
|
||||||
|
gitCommit: '88e2fdffc8c71dd682405c148a81953a8dcee9d6',
|
||||||
|
exporterName: 'mysql2-show-create',
|
||||||
|
exporterVersion: '1.0.0',
|
||||||
|
driverName: 'mysql2',
|
||||||
|
driverVersion: '3.23.3',
|
||||||
|
serverVersion: '5.7.42-log',
|
||||||
|
databaseCharset: 'utf8mb4',
|
||||||
|
databaseCollation: 'utf8mb4_general_ci',
|
||||||
|
generatedAt: '2026-08-18T00:00:00.000Z',
|
||||||
|
});
|
||||||
|
|
||||||
function makeTemporaryDirectory() {
|
function makeTemporaryDirectory() {
|
||||||
return mkdtempSync(path.join(tmpdir(), 'a-share-schema-baseline-'));
|
return mkdtempSync(path.join(tmpdir(), 'a-share-schema-baseline-'));
|
||||||
}
|
}
|
||||||
@@ -300,6 +314,29 @@ test('拒绝静默覆盖已存在且内容不同的版本基线', () => {
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('写入第三个文件失败时清理目标目录和临时目录', () => {
|
||||||
|
const root = makeTemporaryDirectory();
|
||||||
|
const target = path.join(root, 'v0.0.5');
|
||||||
|
const artifacts = buildCapturedArtifacts(
|
||||||
|
validMetadata,
|
||||||
|
'CREATE TABLE `sample` (`id` int);\n',
|
||||||
|
);
|
||||||
|
let writeCount = 0;
|
||||||
|
assert.throws(
|
||||||
|
() =>
|
||||||
|
persistBaselineArtifacts(target, artifacts, {
|
||||||
|
writeFileSync(...args) {
|
||||||
|
writeCount += 1;
|
||||||
|
if (writeCount === 3) throw new Error('injected write failure');
|
||||||
|
return writeFileSync(...args);
|
||||||
|
},
|
||||||
|
}),
|
||||||
|
/injected write failure/,
|
||||||
|
);
|
||||||
|
assert.equal(readdirSync(root).includes('v0.0.5'), false);
|
||||||
|
assert.equal(readdirSync(root).some((entry) => entry.includes('.tmp-')), false);
|
||||||
|
});
|
||||||
|
|
||||||
test('MySQL 未启用时只生成可验证不适用清单', () => {
|
test('MySQL 未启用时只生成可验证不适用清单', () => {
|
||||||
const root = makeTemporaryDirectory();
|
const root = makeTemporaryDirectory();
|
||||||
const target = path.join(root, 'v0.0.5');
|
const target = path.join(root, 'v0.0.5');
|
||||||
@@ -314,7 +351,7 @@ test('MySQL 未启用时只生成可验证不适用清单', () => {
|
|||||||
assert.throws(() => readFileSync(path.join(target, 'checksum.sha256')), /ENOENT/);
|
assert.throws(() => readFileSync(path.join(target, 'checksum.sha256')), /ENOENT/);
|
||||||
});
|
});
|
||||||
|
|
||||||
test('未启用入口不调用数据库且拒绝额外命令参数', () => {
|
test('未启用入口不调用数据库且拒绝额外命令参数', async () => {
|
||||||
const root = makeTemporaryDirectory();
|
const root = makeTemporaryDirectory();
|
||||||
const envFile = path.join(root, 'development.env');
|
const envFile = path.join(root, 'development.env');
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
@@ -323,7 +360,7 @@ test('未启用入口不调用数据库且拒绝额外命令参数', () => {
|
|||||||
);
|
);
|
||||||
let executionCount = 0;
|
let executionCount = 0;
|
||||||
|
|
||||||
const result = runBaselineGeneration({
|
const result = await runBaselineGeneration({
|
||||||
repositoryRoot: root,
|
repositoryRoot: root,
|
||||||
envFile,
|
envFile,
|
||||||
applicationVersion: 'v0.0.5',
|
applicationVersion: 'v0.0.5',
|
||||||
@@ -337,13 +374,13 @@ test('未启用入口不调用数据库且拒绝额外命令参数', () => {
|
|||||||
|
|
||||||
assert.equal(result.status, 'not_applicable');
|
assert.equal(result.status, 'not_applicable');
|
||||||
assert.equal(executionCount, 0);
|
assert.equal(executionCount, 0);
|
||||||
assert.throws(
|
await assert.rejects(
|
||||||
() => runBaselineGeneration({ ...result, commandArguments: ['--env', 'production.env'] }),
|
() => runBaselineGeneration({ ...result, commandArguments: ['--env', 'production.env'] }),
|
||||||
/不接受命令参数/,
|
/不接受命令参数/,
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
test('启用入口分别校验 mysql 与 mysqldump 客户端版本', () => {
|
test('启用入口使用异步 Node exporter', async () => {
|
||||||
const root = makeTemporaryDirectory();
|
const root = makeTemporaryDirectory();
|
||||||
const envFile = path.join(root, 'development.env');
|
const envFile = path.join(root, 'development.env');
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
@@ -353,31 +390,27 @@ test('启用入口分别校验 mysql 与 mysqldump 客户端版本', () => {
|
|||||||
.join('\n'),
|
.join('\n'),
|
||||||
);
|
);
|
||||||
|
|
||||||
assert.throws(
|
const result = await runBaselineGeneration({
|
||||||
() =>
|
|
||||||
runBaselineGeneration({
|
|
||||||
repositoryRoot: root,
|
repositoryRoot: root,
|
||||||
envFile,
|
envFile,
|
||||||
applicationVersion: 'v0.0.5',
|
applicationVersion: 'v0.0.5',
|
||||||
gitCommit: validMetadata.gitCommit,
|
gitCommit: validMetadata.gitCommit,
|
||||||
generatedAt: validMetadata.generatedAt,
|
generatedAt: validMetadata.generatedAt,
|
||||||
executeFile(command, args) {
|
exportSchema: async () => ({
|
||||||
if (args[0] === '--version') {
|
metadata: {
|
||||||
return command === 'mysql'
|
...validNodeMetadata,
|
||||||
? 'mysql Ver 8.0.39 for macos'
|
applicationVersion: undefined,
|
||||||
: 'mysqldump Ver 10.13 Distrib 5.7.44, for macos';
|
gitCommit: undefined,
|
||||||
}
|
generatedAt: undefined,
|
||||||
if (command === 'mysql') {
|
|
||||||
return '5.7.42-log\tutf8mb4\tutf8mb4_general_ci\n';
|
|
||||||
}
|
|
||||||
return 'CREATE TABLE `sample` (`id` int);\n';
|
|
||||||
},
|
},
|
||||||
|
rawSql: '',
|
||||||
|
objectCounts: { tables: 0, views: 0, triggers: 0, routines: 0, events: 0 },
|
||||||
}),
|
}),
|
||||||
/mysql 客户端必须为 5\.7/,
|
});
|
||||||
);
|
assert.equal(result.status, 'captured_empty');
|
||||||
});
|
});
|
||||||
|
|
||||||
test('幂等重跑仍核对清单提交与发布标签提交', () => {
|
test('幂等重跑仍核对清单提交与发布标签提交', async () => {
|
||||||
const root = makeTemporaryDirectory();
|
const root = makeTemporaryDirectory();
|
||||||
const envFile = path.join(root, 'development.env');
|
const envFile = path.join(root, 'development.env');
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
@@ -387,9 +420,8 @@ test('幂等重跑仍核对清单提交与发布标签提交', () => {
|
|||||||
const target = path.join(root, 'database', 'schema-baselines', 'v0.0.5');
|
const target = path.join(root, 'database', 'schema-baselines', 'v0.0.5');
|
||||||
persistBaselineArtifacts(target, buildNotApplicableArtifacts(validMetadata));
|
persistBaselineArtifacts(target, buildNotApplicableArtifacts(validMetadata));
|
||||||
|
|
||||||
assert.throws(
|
await assert.rejects(
|
||||||
() =>
|
() => runBaselineGeneration({
|
||||||
runBaselineGeneration({
|
|
||||||
repositoryRoot: root,
|
repositoryRoot: root,
|
||||||
envFile,
|
envFile,
|
||||||
applicationVersion: 'v0.0.5',
|
applicationVersion: 'v0.0.5',
|
||||||
@@ -451,7 +483,7 @@ test('仅含 mysqldump 会话设置的真实空库仍生成 captured_empty', ()
|
|||||||
assert.deepEqual(Object.keys(artifacts.files), ['manifest.yaml']);
|
assert.deepEqual(Object.keys(artifacts.files), ['manifest.yaml']);
|
||||||
});
|
});
|
||||||
|
|
||||||
test('启用入口把 mysqldump 空结构持久化为 captured_empty', () => {
|
test('启用入口把 Node exporter 空结构持久化为 captured_empty', async () => {
|
||||||
const root = makeTemporaryDirectory();
|
const root = makeTemporaryDirectory();
|
||||||
const envFile = path.join(root, 'development.env');
|
const envFile = path.join(root, 'development.env');
|
||||||
writeFileSync(
|
writeFileSync(
|
||||||
@@ -461,21 +493,17 @@ test('启用入口把 mysqldump 空结构持久化为 captured_empty', () => {
|
|||||||
.join('\n'),
|
.join('\n'),
|
||||||
);
|
);
|
||||||
|
|
||||||
const result = runBaselineGeneration({
|
const result = await runBaselineGeneration({
|
||||||
repositoryRoot: root,
|
repositoryRoot: root,
|
||||||
envFile,
|
envFile,
|
||||||
applicationVersion: 'v0.0.8',
|
applicationVersion: 'v0.0.8',
|
||||||
gitCommit: validMetadata.gitCommit,
|
gitCommit: validMetadata.gitCommit,
|
||||||
generatedAt: validMetadata.generatedAt,
|
generatedAt: validMetadata.generatedAt,
|
||||||
executeFile(command, args) {
|
exportSchema: async () => ({
|
||||||
if (args[0] === '--version') {
|
metadata: { ...validNodeMetadata, applicationVersion: 'v0.0.8' },
|
||||||
return `${command} Ver 14.14 Distrib 5.7.44, for Linux`;
|
rawSql: '',
|
||||||
}
|
objectCounts: { tables: 0, views: 0, triggers: 0, routines: 0, events: 0 },
|
||||||
if (command === 'mysql') {
|
}),
|
||||||
return '5.7.42-log\tutf8mb4\tutf8mb4_general_ci\n';
|
|
||||||
}
|
|
||||||
return '';
|
|
||||||
},
|
|
||||||
});
|
});
|
||||||
|
|
||||||
assert.equal(result.status, 'captured_empty');
|
assert.equal(result.status, 'captured_empty');
|
||||||
@@ -488,6 +516,79 @@ test('启用入口把 mysqldump 空结构持久化为 captured_empty', () => {
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('format v3 清单使用 Node exporter/driver 字段且不复用 CLI 字段', () => {
|
||||||
|
const artifacts = buildCapturedArtifacts(
|
||||||
|
validNodeMetadata,
|
||||||
|
'CREATE TABLE `sample` (`id` int);\n',
|
||||||
|
{ tables: 1, views: 0, triggers: 0, routines: 0, events: 0 },
|
||||||
|
);
|
||||||
|
const manifest = JSON.parse(artifacts.files['manifest.yaml']);
|
||||||
|
|
||||||
|
assert.equal(manifest.format_version, 3);
|
||||||
|
assert.equal(manifest.generator, 'schema-baseline/3.0.0');
|
||||||
|
assert.equal(manifest.exporter_name, 'mysql2-show-create');
|
||||||
|
assert.equal(manifest.exporter_version, '1.0.0');
|
||||||
|
assert.equal(manifest.database_driver, 'mysql2');
|
||||||
|
assert.equal(manifest.database_driver_version, '3.23.3');
|
||||||
|
assert.equal('mysql_client_version' in manifest, false);
|
||||||
|
assert.equal('mysqldump_client_version' in manifest, false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('format v3 空库和不适用清单保留可验证状态', () => {
|
||||||
|
const emptyManifest = JSON.parse(
|
||||||
|
buildCapturedEmptyArtifacts(validNodeMetadata).files['manifest.yaml'],
|
||||||
|
);
|
||||||
|
assert.equal(emptyManifest.format_version, 3);
|
||||||
|
assert.equal(emptyManifest.status, 'captured_empty');
|
||||||
|
assert.equal(emptyManifest.object_counts.tables, 0);
|
||||||
|
|
||||||
|
const disabledManifest = JSON.parse(
|
||||||
|
buildNotApplicableArtifacts(validNodeMetadata).files['manifest.yaml'],
|
||||||
|
);
|
||||||
|
assert.equal(disabledManifest.format_version, 3);
|
||||||
|
assert.equal(disabledManifest.status, 'not_applicable');
|
||||||
|
assert.equal(disabledManifest.database_driver, 'mysql2');
|
||||||
|
assert.equal(disabledManifest.mysql_server_version, 'not_enabled');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('异步 Node exporter 注入路径生成 format v3 captured_empty', async () => {
|
||||||
|
const root = makeTemporaryDirectory();
|
||||||
|
const envFile = path.join(root, 'development.env');
|
||||||
|
writeFileSync(
|
||||||
|
envFile,
|
||||||
|
Object.entries(validConfig)
|
||||||
|
.map(([key, value]) => `${key}=${value}`)
|
||||||
|
.join('\n'),
|
||||||
|
);
|
||||||
|
let exportCalls = 0;
|
||||||
|
const result = await runBaselineGeneration({
|
||||||
|
repositoryRoot: root,
|
||||||
|
envFile,
|
||||||
|
applicationVersion: 'v0.0.9',
|
||||||
|
gitCommit: validNodeMetadata.gitCommit,
|
||||||
|
generatedAt: validNodeMetadata.generatedAt,
|
||||||
|
exportSchema: async () => {
|
||||||
|
exportCalls += 1;
|
||||||
|
return {
|
||||||
|
metadata: validNodeMetadata,
|
||||||
|
rawSql: '',
|
||||||
|
objectCounts: { tables: 0, views: 0, triggers: 0, routines: 0, events: 0 },
|
||||||
|
};
|
||||||
|
},
|
||||||
|
});
|
||||||
|
assert.equal(result.status, 'captured_empty');
|
||||||
|
assert.equal(exportCalls, 1);
|
||||||
|
assert.equal(
|
||||||
|
JSON.parse(
|
||||||
|
readFileSync(
|
||||||
|
path.join(root, 'database', 'schema-baselines', 'v0.0.9', 'manifest.yaml'),
|
||||||
|
'utf8',
|
||||||
|
),
|
||||||
|
).format_version,
|
||||||
|
3,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
test('基线清单中的提交必须等于发布标签指向的提交', () => {
|
test('基线清单中的提交必须等于发布标签指向的提交', () => {
|
||||||
const root = makeTemporaryDirectory();
|
const root = makeTemporaryDirectory();
|
||||||
const target = path.join(root, 'v0.0.5');
|
const target = path.join(root, 'v0.0.5');
|
||||||
|
|||||||
Reference in New Issue
Block a user