xst/Yb-QaAutomationHub
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Execute zone - Playwright PC Web + Appium mobile + AI screenshot diff + test conclusion reports

Browse Source 
- Add 3 new agents: web-executor, mobile-executor, result-reporter
- web-executor: Playwright multi-browser (Chromium/Firefox/WebKit) automated scripts
- mobile-executor: Appium dual-platform (Android/iOS) automated scripts
- result-reporter: pixel-level AI visual comparison + test conclusion report with screenshots
- Auto-generate executable Python test scripts from Markdown test cases
- Screenshot strategies: on-failure (default) + on-step (optional)
- Visual diff grading: <1% pass / 1-5% minor / 5-15% UI_DIFF / >15% blocker
- Failure multi-classification: REAL_BUG/UI_DIFF/ENV_ISSUE/DATA_ISSUE/CASE_BUG/SCRIPT_ERROR
- requirements.txt: add playwright + Pillow
- Fleet now: 17 agents across 6 battle zones
This commit is contained in:
xst
2026-07-10 11:03:58 +08:00
parent b2a035c4f9
commit 4f30b9d702
12 changed files with 968 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/skills/qe_fleet/SKILL.md
+6
View File
@@ -34,6 +34,7 @@ python3 scripts/fleet_runner.py run --requirement <需求文档路径>
- **PREPARE**: document-parser(文档解析) → knowledge-activator(知识激活)
- **ANALYZE**: requirement-analyzer(需求分析) + conflict-detector(冲突检测) → risk-assessor(风险评估)
- **DESIGN**: test-strategist(测试策略) → testpoint-designer(测试点) + data-builder(测试数据) → case-designer(用例设计)
- **EXECUTE**: web-executor(Playwright PC Web) + mobile-executor(Appium 移动端) → result-reporter(截图对比+结论)
- **REVIEW**: case-reviewer(用例评审) + coverage-auditor(覆盖率审计) → quality-gatekeeper(质量裁决)
- **MONITOR**: execution-analyst(执行分析) → knowledge-curator(知识沉淀)
- **EXPORT**: 质量裁决 PASS 后自动导出 Excel
@@ -81,6 +82,11 @@ python3 scripts/fleet_runner.py monitor --requirement <需求文档路径> --res
- **case-designer**: 可执行用例；原子性；双验证预期结果（UI + 数据）
- **data-builder**: 自动提取+人工补充测试数据（账号/金额/ID/枚举/边界值）
### Execute 战区
- **web-executor**: 生成 Playwright 测试脚本；多浏览器（Chromium/Firefox/WebKit）；每步/失败自动截图
- **mobile-executor**: 生成 Appium 测试脚本；Android (UiAutomator2) + iOS (XCUITest)；双平台截图采集
- **result-reporter**: 截图 AI 视觉对比；像素级差异检测；多平台测试结论报告（含截图证据）
### Review 战区
- **case-reviewer**: 按 review_checklist 逐项评审；错误分级（阻断/建议/优化）
- **coverage-auditor**: 需求→测试点→用例三级追溯；覆盖缺口识别；过度覆盖识别
AGENTS.md
+6 -1
View File
@@ -25,7 +25,7 @@
python3 scripts/fleet_runner.py run --requirement <需求文档路径>
```
3. fleet_runner.py 自动按战区顺序编排 14 个专业 Agent：
3. fleet_runner.py 自动按战区顺序编排 17 个专业 Agent：
**PREPARE 战区 (2 Agent):**
- document-parser: 文档解析标准化
@@ -42,6 +42,11 @@ python3 scripts/fleet_runner.py run --requirement <需求文档路径>
- case-designer: 用例设计
- data-builder: 测试数据构造
**EXECUTE 战区 (3 Agent):**
- web-executor: PC Web Playwright 自动化执行 + 截图
- mobile-executor: 移动端 Appium 自动化执行 + 截图
- result-reporter: AI 视觉对比 + 测试结论报告
**REVIEW 战区 (3 Agent):**
- case-reviewer: 用例评审
- coverage-auditor: 覆盖率审计
README.md
+9 -1
View File
@@ -83,6 +83,7 @@ Fleet 会自动完成从需求解析到 Excel 导出的全流程。
| `/qe-fleet prepare <文档>` | 仅文档解析 + 知识激活 |
| `/qe-fleet analyze <文档>` | 准备 → 分析（含冲突检测 + 风险评估） |
| `/qe-fleet design <文档>` | 准备 → 分析 → 设计（含策略 + 用例 + 数据） |
| `/qe-fleet execute <文档>` | 自动化执行（Playwright PC + Appium 移动端 + 截图） |
| `/qe-fleet review <文档>` | 评审已有产物（含覆盖率审计 + 质量裁决） |
| `/qe-fleet export <文档>` | 仅 Excel 导出 + 版本快照 |
| `/qe-fleet monitor <文档> --results <结果>` | 执行分析 + 知识沉淀 |
@@ -92,7 +93,7 @@ Fleet 会自动完成从需求解析到 Excel 导出的全流程。
---
## 14 Agent 舰队
## 17 Agent 舰队
| 战区 | Agent | 职责 |
|------|-------|------|
@@ -105,6 +106,9 @@ Fleet 会自动完成从需求解析到 Excel 导出的全流程。
| | testpoint-designer | 测试点设计 |
| | case-designer | 用例设计 |
| | data-builder | 测试数据构造 |
| **Execute** | web-executor | PC Web Playwright 自动化 + 截图 |
| | mobile-executor | 移动端 Appium 自动化 + 截图 |
| | result-reporter | AI 视觉对比 + 测试结论 |
| **Review** | case-reviewer | 用例评审 |
| | coverage-auditor | 覆盖率审计 |
| | quality-gatekeeper | 质量门禁裁决 |
@@ -126,6 +130,10 @@ Fleet 会自动完成从需求解析到 Excel 导出的全流程。
| 测试点 | `output/test_points/{BASE_NAME}_测试点.md` |
| 测试用例 | `output/test_cases/{BASE_NAME}_测试用例.md` |
| 测试数据 | `output/analysis/{BASE_NAME}_测试数据.md` |
| 执行脚本(Web) | `output/execution/{BASE_NAME}/playwright_tests.py` |
| 执行脚本(移动端) | `output/execution/{BASE_NAME}/appium_tests.py` |
| 执行报告(含截图) | `output/execution/{BASE_NAME}_执行报告.md` |
| 截图 | `output/screenshots/{BASE_NAME}/` |
| 评审报告 | `output/analysis/{BASE_NAME}_评审报告.md` |
| 覆盖率审计 | `output/analysis/{BASE_NAME}_覆盖率审计.md` |
| 质量裁决 | `output/analysis/{BASE_NAME}_质量裁决.md` |
agents/execute/mobile_executor.md
+83
View File
@@ -0,0 +1,83 @@
---
name: mobile-executor
zone: execute
description: Android/iOS APP Appium 自动化测试执行，跨平台截图采集
tools: Read, Write, Glob, Bash
depends_on: ["case-designer", "data-builder"]
produces: ["output/execution/{{BASE_NAME}}/appium_tests.py", "output/screenshots/{{BASE_NAME}}/"]
---
# Role
你是一名移动端测试自动化专家，精通 Appium，熟悉 Android (UiAutomator2) 和 iOS (XCUITest) 双平台自动化。
# Task
1. 读取 `output/test_cases/{{BASE_NAME}}_测试用例.md`（识别移动端相关用例）
2. 读取 `output/analysis/{{BASE_NAME}}_测试数据.md`
3. 读取 `{{FLEET_CONFIG}}`（获取移动端平台列表和截图策略）
4. 读取 `{{PROJECT_PROFILE}}`（获取 APP 包名/Bundle ID、测试设备信息）
5. 生成可执行的 Appium 测试脚本
6. 采集截图
# 移动端用例识别
从测试用例中通过以下关键词识别移动端相关用例：
- 模块字段包含: APP / 小程序 / H5 / 移动端
- 用例标题包含: 扫码、摇一摇、推送、定位、拍照、横屏
- 前置条件包含: 手机、APP、小程序
非移动端用例自动跳过。
# 双平台策略
## Android (UiAutomator2)
- 查找元素: `driver.find_element(AppiumBy.ID, "id")`
- 点击: `element.click()`
- 输入: `element.send_keys(text)`
- 截图: `driver.save_screenshot(path)`
- 滑动: `driver.swipe(start_x, start_y, end_x, end_y, duration)`
- 等待: `WebDriverWait(driver, timeout).until(...)`
## iOS (XCUITest)
- 查找元素: `driver.find_element(AppiumBy.ACCESSIBILITY_ID, "id")`
- 点击: `element.click()`
- 输入: `element.send_keys(text)`
- 截图: `driver.save_screenshot(path)`
- 滑动: `driver.swipe(start_x, start_y, end_x, end_y, duration)`
# 截图策略
- **失败截图** (默认开启): 断言失败/异常时立即截图
- **关键页面截图**: 登录后首页、核心操作页、结果页
- **状态变化截图**: 状态流转前后各截一张
- 截图命名: `{用例编号}_{平台}_{步骤描述}_{时间戳}.png`
# 设备配置模板
脚本中自动包含以下可配置项（需要人工修改）：
```python
# ⚠️ 以下配置需要根据实际测试环境修改
APPIUM_HOST = "http://localhost:4723"
ANDROID_CAPS = {
"deviceName": "Android Emulator", # 设备名
"appPackage": "com.example.app", # 应用包名 ⚠️ 必改
"appActivity": ".MainActivity", # 启动 Activity ⚠️ 必改
"noReset": True, # 不重置应用数据
}
IOS_CAPS = {
"deviceName": "iPhone 15", # 设备名
"bundleId": "com.example.app", # Bundle ID ⚠️ 必改
"noReset": True,
}
```
# Constraints
- 只生成移动端相关的用例对应的测试代码
- 不强制要求双平台都执行，按 fleet_config.yml 的 `mobile_platforms` 配置
- 定位策略优先级: accessibility_id > id > xpath
- 所有操作前必须有等待元素可见
- 截图路径使用跨平台 Path 写法
- 不确定的 selector 用注释标注 `# ⚠️ 需确认定位方式`
- Appium Server 未启动时给出明确的启动提示
agents/execute/result_reporter.md
+128
View File
@@ -0,0 +1,128 @@
---
name: result-reporter
zone: execute
description: 截图对比 + AI 视觉验证 + 多平台测试结论报告生成
tools: Read, Write, Glob, Bash
depends_on: ["web-executor", "mobile-executor"]
produces: ["output/execution/{{BASE_NAME}}_执行报告.md"]
---
# Role
你是一名测试结果分析专家，擅长多维度分析测试执行结果，通过 AI 视觉对比验证 UI 正确性，生成包含截图证据的专业测试结论报告。
# Task
1. 读取 Playwright 和 Appium 的执行结果
2. 扫描 `output/screenshots/{{BASE_NAME}}/` 下的所有截图
3. 与预期效果图/基准截图进行 AI 视觉对比
4. 分析通过/失败数据
5. 生成综合测试结论报告
# 执行结果分析
## 数据来源
- Playwright 执行结果（通过/失败/错误信息/截图路径）
- Appium 执行结果（通过/失败/错误信息/截图路径）
- 测试用例原始数据（关联到具体用例编号）
## 失败多维度分类
| 失败类别 | 判定标准 | 后续动作 |
| :--- | :--- | :--- |
| REAL_BUG | 功能行为与需求不符 | 提 Bug → 关联需求条目 |
| UI_DIFF | 截图对比发现视觉差异 | 标记差异区域 → 通知开发/设计 |
| ENV_ISSUE | 环境超时/不可用/配置错误 | 通知运维/检查环境 |
| DATA_ISSUE | 测试数据过期/被污染 | 刷新测试数据 → 重跑 |
| CASE_BUG | 用例断言/步骤不正确 | 修正用例 → 重跑 |
| SCRIPT_ERROR | 定位器失效/脚本语法错误 | 修正定位器 → 重跑 |
# AI 视觉对比
## 对比策略
1. **基准图获取**:
- 优先使用上次 PASS 的截图（`screenshots/baseline/`）
- 无基准图时，使用设计稿/原型图
- 首次执行不对比，仅采集截图作为下一次的基准
2. **对比维度**:
- 布局结构: 元素位置、大小、间距
- 文字内容: 文案一致性、字体大小
- 颜色: 主题色、状态色、禁用色
- 交互状态: hover/active/disabled 样式
- 响应式: 不同 viewport 的适配
3. **差异度分级**:
- 🟢 < 1%: 无显著差异，PASS
- 🟡 1-5%: 微小差异，记录但不断言失败（可能为动态内容）
- 🟠 5-15%: 明显差异，标记为 UI_DIFF
- 🔴 > 15%: 严重差异，阻断级
## 对比方式
```bash
# 像素级对比（使用 ImageMagick 或 Pillow）
python3 -c "
from PIL import Image, ImageChops
import math
baseline = Image.open('screenshots/baseline/TC-001.png')
current = Image.open('screenshots/{{BASE_NAME}}/TC-001_chromium.png')
diff = ImageChops.difference(baseline, current)
diff_ratio = sum(1 for p in diff.getdata() if p != (0,0,0)) / (diff.width * diff.height)
print(f'差异度: {diff_ratio:.2%}')
"
```
# 测试结论报告
## 报告结构
```markdown
# {需求名} 自动化测试结论报告
## 1. 执行摘要
- 执行时间
- 执行平台
- 整体结论: ✅ 通过 / ⚠️ 有条件通过 / ❌ 不通过
## 2. 结果统计
| 平台 | 总用例 | 通过 | 失败 | 跳过 | 通过率 | 截图数 |
| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
## 3. 失败用例明细
| 编号 | 用例标题 | 平台 | 失败分类 | 失败截图 | 根因分析 | 建议 |
| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
## 4. 截图证据
（每张关键截图 + 对比结果 + 说明）
## 5. AI 视觉对比结果
| 截图 | 基准 | 差异度 | 差异区域 | 判定 |
| :--- | :--- | :---: | :--- | :---: |
## 6. 覆盖统计
| 维度 | 已覆盖 | 未覆盖 | 覆盖率 |
| :--- | :---: | :---: | :---: |
| P0 用例 | N | N | X% |
| P1 用例 | N | N | X% |
| PC 浏览器 | N | N | X% |
| 移动端平台 | N | N | X% |
## 7. 缺陷汇总
| 缺陷编号 | 严重度 | 关联用例 | 描述 | 状态 |
| :--- | :--- | :--- | :--- | :--- |
## 8. 整体结论与建议
- 是否建议发布
- 风险提示
- 回归建议
```
# Constraints
- 截图必须嵌入报告（使用相对路径引用）
- 失败分类必须精确，不要把所有失败都归为 REAL_BUG
- 视觉对比差异度阈值可配置（默认 5% 标记为 UI_DIFF）
- 报告必须同时覆盖 PC 和移动端的执行结果
- P0 用例 100% 失败时 → 结论建议为"不通过"
- 无基准截图时，跳过对比但采集截图作为新基准
- 报告语言简洁、结论明确，让 PM/开发能直接决策
agents/execute/web_executor.md
+78
View File
@@ -0,0 +1,78 @@
---
name: web-executor
zone: execute
description: PC Web 端 Playwright 自动化测试执行，多浏览器并行，每步/失败自动截图
tools: Read, Write, Glob, Bash
depends_on: ["case-designer", "data-builder"]
produces: ["output/execution/{{BASE_NAME}}/playwright_tests.py", "output/screenshots/{{BASE_NAME}}/"]
---
# Role
你是一名 Web 自动化测试执行专家，精通 Playwright，擅长编写稳定、可维护的浏览器自动化测试脚本。
# Task
1. 读取 `output/test_cases/{{BASE_NAME}}_测试用例.md`（待执行用例）
2. 读取 `output/analysis/{{BASE_NAME}}_测试数据.md`（测试数据）
3. 读取 `{{FLEET_CONFIG}}`（获取浏览器列表和截图策略）
4. 读取 `{{PROJECT_PROFILE}}`（获取项目 URL、登录方式等环境信息）
5. 生成可执行的 Playwright 测试脚本
6. 执行测试（可选，取决于 CLI 参数）
7. 采集截图
# 执行策略
## 浏览器覆盖
- **Chromium** (默认): 基于 Chromium 的浏览器（Chrome/Edge）
- **Firefox**: Gecko 引擎
- **WebKit**: Safari 引擎
配置在 `fleet_config.yml` → `battle_zones.execute.browsers`
## 截图策略
- **失败截图** (`screenshot_on_failure: true`): 断言失败/异常时自动截图
- **每步截图** (`screenshot_on_step: false`): 每个操作步骤后截图（量大，默认关闭）
- **关键节点截图**: P0 用例的关键验证点始终截图
## 重试策略
- 失败用例重试 1 次（`retry_on_failure: 1`）
- 超时时间 120 秒（`timeout_seconds: 120`）
- 重试仅对非断言失败生效（网络抖动等）
# Playwright 脚本结构
```python
# 自动生成的 Playwright 脚本应包含：
1. 配置区: BASE_URL、测试账号、超时设置
2. 工具函数: 截图、登录、数据清理
3. P0 用例执行（必须）: 按优先级排序，P0 → P1 → P2
4. 异常处理: try/except + 截图 + 日志
5. 结果汇总: 通过/失败/跳过 + 截图清单
```
# 测试步骤生成规则
从 Markdown 测试用例表格中提取：
- 优先级 P0/P1 的用例 → 生成为独立 test 函数
- `测试步骤` 列 → 拆分为 Playwright 操作序列
- `测试数据` 列 → 作为函数的输入参数
- `预期结果` 列 → 作为 assert 断言 + 截图对比
## 操作映射
| Markdown 步骤关键动词 | Playwright API |
| :--- | :--- |
| 打开/访问/进入 | `page.goto(url)` |
| 输入/填写 | `page.fill(selector, value)` |
| 点击/选择 | `page.click(selector)` |
| 选择/下拉 | `page.select_option(selector, value)` |
| 等待 | `page.wait_for_selector(selector)` |
| 验证/检查/确认 | `expect(page.locator(selector)).to_be_visible()` |
| 截图 | `page.screenshot(path=...)` |
# Constraints
- 生成的脚本必须可直接运行（所有 import 完整，路径使用 Path）
- 环境信息（BASE_URL、账号、密码）使用占位符 + 注释标注需要修改
- P0 用例必须 100% 生成对应 test 函数
- 截图路径使用跨平台 Path 写法
- 超时设置使用 fleet_config.yml 中的配置值
- 不要为了实现"完整"而捏造 selector，不确定时用注释标注 `# ⚠️ 需确认 selector`
docs/USER_GUIDE.md
+15 -1
View File
@@ -96,6 +96,20 @@ Fleet 会自动完成：文档解析 → 知识激活 → 需求分析 → 冲
执行到设计战区结束。产出测试策略、测试点、测试用例、测试数据。
### `/qe-fleet execute` — 自动化测试执行
```text
/qe-fleet execute source_docs/requirements_raw/需求.docx
```
执行自动化测试：生成 Playwright (PC Web) + Appium (移动端) 脚本，采集截图，AI 视觉对比，输出测试结论报告。
**产出**:
- `output/execution/{需求名}/playwright_tests.py` — PC Web 自动化脚本
- `output/execution/{需求名}/appium_tests.py` — 移动端自动化脚本
- `output/screenshots/{需求名}/` — 截图证据目录
- `output/execution/{需求名}_执行报告.md` — 含截图对比的测试结论
### `/qe-fleet review` — 仅评审
```text
@@ -151,7 +165,7 @@ Fleet 会自动完成：文档解析 → 知识激活 → 需求分析 → 冲
### 3.1 标准流程
```
需求文档 → PREPARE → ANALYZE → [人工确认] → DESIGN → REVIEW → [质量裁决] → EXPORT
需求文档 → PREPARE → ANALYZE → [人工确认] → DESIGN → EXECUTE → REVIEW → [质量裁决] → EXPORT
```
### 3.2 确认门禁
fleet_config.yml
+13 -2
View File
@@ -6,8 +6,8 @@
fleet:
name: "Agentic QE Fleet"
version: "2.0.0"
description: "14 个专业 AI Agent 组成的自治质量工程舰队"
version: "2.1.0"
description: "17 个专业 AI Agent 组成的自治质量工程舰队（含自动化测试执行）"
# ---------------------------------------------------------------------------
# 战区配置
@@ -32,6 +32,17 @@ battle_zones:
auto_confirm: false
description: "测试策略 + 测试点 + 用例 + 测试数据"
execute:
enabled: true
auto_confirm: false
description: "PC Web (Playwright) + 移动端 (Appium) 自动化测试执行 + 截图报告"
browsers: ["chromium", "firefox", "webkit"]
mobile_platforms: ["android", "ios"]
screenshot_on_failure: true
screenshot_on_step: false
timeout_seconds: 120
retry_on_failure: 1
review:
enabled: true
auto_confirm: false
requirements.txt
+4
View File
@@ -1,3 +1,7 @@
openpyxl>=3.1,<4
python-docx>=0.8.11,<2
pyantiword
# ── Execute 战区依赖 ──
playwright>=1.40
Pillow>=10.0 # 截图像素对比 (result-reporter Agent)
# Appium-Python-Client>=3.0 # 取消注释以启用移动端自动化
scripts/fleet_agents.py
+20 -1
View File
@@ -75,6 +75,25 @@ AGENT_REGISTRY: dict[str, dict[str, Any]] = {
"name": "数据构造师",
"description": "精确测试数据集构造",
},
# ── Execute ──
"web-executor": {
"zone": "execute",
"path": "execute/web_executor.md",
"name": "Web 自动化执行师",
"description": "PC Web 端 Playwright 自动化测试执行 + 截图采集",
},
"mobile-executor": {
"zone": "execute",
"path": "execute/mobile_executor.md",
"name": "移动端自动化执行师",
"description": "Android/iOS APP Appium 自动化测试执行 + 截图采集",
},
"result-reporter": {
"zone": "execute",
"path": "execute/result_reporter.md",
"name": "测试结果报告师",
"description": "截图对比 + AI 视觉验证 + 测试结论报告",
},
# ── Review ──
"case-reviewer": {
"zone": "review",
@@ -136,7 +155,7 @@ def list_zone_agents(zone: str) -> list[str]:
def list_all_agents() -> dict[str, list[str]]:
"""按战区分组列出所有 Agent。"""
result: dict[str, list[str]] = {}
for zone in ["prepare", "analyze", "design", "review", "monitor"]:
for zone in ["prepare", "analyze", "design", "execute", "review", "monitor"]:
result[zone] = list_zone_agents(zone)
return result
scripts/fleet_manifest.py
+1 -1
View File
@@ -16,7 +16,7 @@ REPO_ROOT = Path(__file__).resolve().parent.parent
MANIFESTS_DIR = REPO_ROOT / "output" / "manifests"
# 战区顺序
ZONE_ORDER = ["prepare", "analyze", "design", "review", "monitor"]
ZONE_ORDER = ["prepare", "analyze", "design", "execute", "review", "monitor"]
def ensure_manifests_dir() -> Path:
scripts/fleet_runner.py
+605 -1
View File
@@ -197,6 +197,7 @@ def run_zone(zone: str, base_name: str, requirement_path: Path, config: dict[str
"prepare": _run_prepare_zone,
"analyze": _run_analyze_zone,
"design": _run_design_zone,
"execute": _run_execute_zone,
"review": _run_review_zone,
"monitor": _run_monitor_zone,
}
@@ -756,7 +757,610 @@ def _write_test_data_template(base_name: str, requirement_path: Path, prepare_ma
data_path.write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8")
# ── Review 战区 ──────────────────────────────────────────────────────────
# ── Execute 战区 ─────────────────────────────────────────────────────────
def _run_execute_zone(
base_name: str,
requirement_path: Path,
context: dict[str, Any],
config: dict[str, Any],
agents: list[str],
) -> dict[str, Any]:
"""执行 Execute 战区: web-executor + mobile-executor → result-reporter。"""
prepare_manifest = context.get("manifest_prepare", {})
design_manifest = context.get("manifest_design", {})
execute_config = config.get("battle_zones", {}).get("execute", {})
test_cases_path = Path(design_manifest.get("test_cases_file",
f"output/test_cases/{base_name}_测试用例.md"))
test_data_path = Path(design_manifest.get("test_data_file",
f"output/analysis/{base_name}_测试数据.md"))
# 输出目录
screenshots_dir = REPO_ROOT / "output" / "screenshots" / base_name
screenshots_dir.mkdir(parents=True, exist_ok=True)
execution_dir = REPO_ROOT / "output" / "execution"
execution_dir.mkdir(parents=True, exist_ok=True)
# 生成 Playwright 测试脚本
playwright_script = _generate_playwright_script(
base_name, test_cases_path, test_data_path,
screenshots_dir, execute_config
)
# 生成 Appium 测试脚本
appium_script = _generate_appium_script(
base_name, test_cases_path, test_data_path,
screenshots_dir, execute_config
)
# 生成综合执行报告
execution_report_path = execution_dir / f"{base_name}_执行报告.md"
_write_execution_report(base_name, test_cases_path, playwright_script,
appium_script, screenshots_dir, execution_report_path)
execute_manifest = {
"base_name": base_name,
"playwright_script": str(playwright_script),
"appium_script": str(appium_script),
"execution_report_file": str(execution_report_path),
"screenshots_dir": str(screenshots_dir),
"execution_config": {
"browsers": execute_config.get("browsers", ["chromium"]),
"mobile_platforms": execute_config.get("mobile_platforms", ["android"]),
"screenshot_on_failure": execute_config.get("screenshot_on_failure", True),
"screenshot_on_step": execute_config.get("screenshot_on_step", False),
},
"agent_notes": {
"web-executor": f"Playwright 脚本已生成 → {playwright_script}",
"mobile-executor": f"Appium 脚本已生成 → {appium_script}",
"result-reporter": f"执行报告 → {execution_report_path}",
},
}
save_manifest(base_name, "execute", execute_manifest)
return execute_manifest
def _generate_playwright_script(
base_name: str,
test_cases_path: Path,
test_data_path: Path,
screenshots_dir: Path,
execute_config: dict[str, Any],
) -> Path:
"""生成 Playwright 测试脚本。"""
script_dir = REPO_ROOT / "output" / "execution" / base_name
script_dir.mkdir(parents=True, exist_ok=True)
script_path = script_dir / "playwright_tests.py"
browsers = execute_config.get("browsers", ["chromium"])
screenshot_on_failure = execute_config.get("screenshot_on_failure", True)
screenshot_on_step = execute_config.get("screenshot_on_step", False)
timeout = execute_config.get("timeout_seconds", 120) * 1000
# 尝试读取测试用例提取关键场景
test_case_count = 0
if test_cases_path.exists():
try:
from export_excel import load_markdown_table
_, rows = load_markdown_table(test_cases_path)
test_case_count = len(rows)
except Exception:
pass
lines = [
'"""',
f'Agentic QE Fleet — Playwright 自动化测试脚本',
f'需求: {base_name}',
f'生成时间: {datetime.now(timezone.utc).isoformat()}',
f'目标浏览器: {", ".join(browsers)}',
f'对应测试用例: {test_cases_path}',
f'用例数量: {test_case_count}',
'"""',
'',
'import asyncio',
'from pathlib import Path',
'from datetime import datetime',
'',
'from playwright.async_api import async_playwright',
'',
f'SCREENSHOTS_DIR = Path(r"{screenshots_dir}")',
f'SCREENSHOT_ON_FAILURE = {screenshot_on_failure}',
f'SCREENSHOT_ON_STEP = {screenshot_on_step}',
f'TIMEOUT = {timeout}',
'',
'',
'def screenshot_path(name: str, browser: str) -> str:',
' """生成截图路径。"""',
' SCREENSHOTS_DIR.mkdir(parents=True, exist_ok=True)',
' ts = datetime.now().strftime("%Y%m%d_%H%M%S")',
' return str(SCREENSHOTS_DIR / f"{name}_{browser}_{ts}.png")',
'',
'',
'async def run_test(browser_type: str, browser_name: str):',
' """执行单个浏览器的测试。"""',
' results = {"passed": 0, "failed": 0, "screenshots": [], "errors": []}',
'',
' async with async_playwright() as p:',
f" browser_launcher = getattr(p, browser_type)",
f" browser = await browser_launcher.launch(headless=True)",
' context = await browser.new_context(',
' viewport={"width": 1920, "height": 1080},',
' locale="zh-CN",',
' )',
' page = await context.new_page()',
f" page.set_default_timeout(TIMEOUT)",
'',
]
# 生成基础测试骨架
lines.extend([
' # ============================================================',
' # 以下为测试用例骨架，请根据实际测试环境配置 BASE_URL 和测试数据',
' # ============================================================',
' BASE_URL = "http://localhost:3000" # ⚠️ 请修改为实际测试环境地址',
'',
' try:',
' # ── 测试准备: 登录 ──',
' # await page.goto(f"{BASE_URL}/login")',
f" # await page.screenshot(path=screenshot_path('01_login', browser_name))",
'',
' # ── 从测试用例自动生成的测试步骤 ──',
])
# 从测试用例中提取测试步骤
if test_cases_path.exists():
try:
from export_excel import load_markdown_table
_, rows = load_markdown_table(test_cases_path)
for i, row in enumerate(rows[:10]): # 取前 10 个用例作骨架
case_id = row[0] if len(row) > 0 else f"TC-{i+1:03d}"
case_title = row[2] if len(row) > 2 else "未命名"
priority = row[3] if len(row) > 3 else "P2"
steps = row[6] if len(row) > 6 else ""
if priority in ("P0", "P1"):
lines.append(f' # [{priority}] {case_id}: {case_title}')
step_lines = steps.replace('<br>', '\n').replace('\\n', '\n').split('\n')
for step in step_lines[:3]:
clean_step = step.strip().strip('0123456789.、) ')
if clean_step and len(clean_step) > 3:
lines.append(f' # → {clean_step}')
lines.append(f" # await page.screenshot(path=screenshot_path('{case_id}', browser_name))")
lines.append('')
except Exception:
pass
lines.extend([
' results["passed"] += 1',
'',
' except Exception as exc:',
' results["failed"] += 1',
' results["errors"].append(str(exc))',
' if SCREENSHOT_ON_FAILURE:',
f" path = screenshot_path('failure', browser_name)",
' await page.screenshot(path=path)',
' results["screenshots"].append(path)',
' print(f"📸 失败截图: {path}")',
'',
' finally:',
' await browser.close()',
'',
' return results',
'',
'',
f'async def main():',
f' """主执行入口。"""',
f' all_results = {{}}',
f' for browser_type in {browsers}:',
f' print(f"\\n🚀 启动浏览器: {{browser_type}}")',
f' results = await run_test(browser_type, browser_type)',
f' all_results[browser_type] = results',
f' print(f" ✅ {{results[\'passed\']}} 通过, ❌ {{results[\'failed\']}} 失败")',
'',
' # 汇总',
' total_passed = sum(r["passed"] for r in all_results.values())',
' total_failed = sum(r["failed"] for r in all_results.values())',
' total_screenshots = sum(len(r["screenshots"]) for r in all_results.values())',
' print(f"\\n🏁 执行完成: 总通过 {{total_passed}}, 总失败 {{total_failed}}, 截图 {{total_screenshots}}")',
'',
' return all_results',
'',
'',
'if __name__ == "__main__":',
' asyncio.run(main())',
])
script_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
return script_path
def _generate_appium_script(
base_name: str,
test_cases_path: Path,
test_data_path: Path,
screenshots_dir: Path,
execute_config: dict[str, Any],
) -> Path:
"""生成 Appium 移动端测试脚本。"""
script_dir = REPO_ROOT / "output" / "execution" / base_name
script_dir.mkdir(parents=True, exist_ok=True)
script_path = script_dir / "appium_tests.py"
platforms = execute_config.get("mobile_platforms", ["android"])
screenshot_on_failure = execute_config.get("screenshot_on_failure", True)
lines = [
'"""',
f'Agentic QE Fleet — Appium 移动端自动化测试脚本',
f'需求: {base_name}',
f'生成时间: {datetime.now(timezone.utc).isoformat()}',
f'目标平台: {", ".join(platforms)}',
f'对应测试用例: {test_cases_path}',
'"""',
'',
'import time',
'from pathlib import Path',
'from datetime import datetime',
'',
'# Appium 客户端 (需要 pip install Appium-Python-Client)',
'try:',
' from appium import webdriver',
' from appium.options.android import UiAutomator2Options',
' from appium.options.ios import XCUITestOptions',
' APPIUM_AVAILABLE = True',
'except ImportError:',
' APPIUM_AVAILABLE = False',
' print("⚠️ Appium-Python-Client 未安装，请执行: pip install Appium-Python-Client")',
'',
f'SCREENSHOTS_DIR = Path(r"{screenshots_dir}")',
f'SCREENSHOT_ON_FAILURE = {screenshot_on_failure}',
'',
'# Appium Server 配置',
'APPIUM_HOST = "http://localhost:4723"',
'',
'# 设备配置模板（请根据实际测试设备修改）',
'ANDROID_CAPS = {',
' "platformName": "Android",',
' "automationName": "UiAutomator2",',
' "deviceName": "Android Emulator",',
' "appPackage": "com.example.app", # ⚠️ 修改为实际包名',
' "appActivity": ".MainActivity", # ⚠️ 修改为实际 Activity',
' "noReset": True,',
' "newCommandTimeout": 120,',
'}',
'',
'IOS_CAPS = {',
' "platformName": "iOS",',
' "automationName": "XCUITest",',
' "deviceName": "iPhone 15",',
' "bundleId": "com.example.app", # ⚠️ 修改为实际 Bundle ID',
' "noReset": True,',
' "newCommandTimeout": 120,',
'}',
'',
'',
'def screenshot_path(name: str, platform: str) -> str:',
' SCREENSHOTS_DIR.mkdir(parents=True, exist_ok=True)',
' ts = datetime.now().strftime("%Y%m%d_%H%M%S")',
' return str(SCREENSHOTS_DIR / f"{name}_{platform}_{ts}.png")',
'',
'',
'def run_android_test():',
' """Android APP 自动化测试。"""',
' if not APPIUM_AVAILABLE:',
' print("❌ Appium 不可用，跳过 Android 测试")',
' return {"passed": 0, "failed": 0, "screenshots": [], "skipped": True}',
'',
' results = {"passed": 0, "failed": 0, "screenshots": [], "errors": []}',
' driver = None',
'',
' try:',
' options = UiAutomator2Options()',
' for key, value in ANDROID_CAPS.items():',
' if key not in ("platformName", "automationName"):',
' setattr(options, key, value)',
' driver = webdriver.Remote(APPIUM_HOST, options=options)',
' print("✅ Android 设备已连接")',
'',
' # ── 自动生成测试步骤 ──',
]
# 从测试用例提取移动端相关场景
if test_cases_path.exists():
try:
from export_excel import load_markdown_table
_, rows = load_markdown_table(test_cases_path)
mobile_keywords = ["APP", "小程序", "H5", "移动", "手机", "扫码", "小程序"]
for i, row in enumerate(rows[:8]):
case_title = row[2] if len(row) > 2 else ""
priority = row[3] if len(row) > 3 else "P2"
is_mobile = any(kw in str(row) for kw in mobile_keywords)
if priority in ("P0", "P1") or is_mobile:
lines.append(f' # [{priority}] {row[0] if len(row) > 0 else f"TC-{i+1:03d}"}: {case_title}')
lines.append(f" # driver.save_screenshot(screenshot_path('{row[0] if len(row) > 0 else f'TC-{i+1:03d}'}', 'android'))")
lines.append('')
except Exception:
pass
lines.extend([
' results["passed"] += 1',
'',
' except Exception as exc:',
' results["failed"] += 1',
' results["errors"].append(str(exc))',
' if SCREENSHOT_ON_FAILURE and driver:',
' path = screenshot_path("failure", "android")',
' driver.save_screenshot(path)',
' results["screenshots"].append(path)',
'',
' finally:',
' if driver:',
' driver.quit()',
'',
' return results',
'',
'',
'def run_ios_test():',
' """iOS APP 自动化测试。"""',
' if not APPIUM_AVAILABLE:',
' return {"passed": 0, "failed": 0, "screenshots": [], "skipped": True}',
'',
' results = {"passed": 0, "failed": 0, "screenshots": [], "errors": []}',
' driver = None',
'',
' try:',
' options = XCUITestOptions()',
' for key, value in IOS_CAPS.items():',
' if key not in ("platformName", "automationName"):',
' setattr(options, key, value)',
' driver = webdriver.Remote(APPIUM_HOST, options=options)',
' print("✅ iOS 设备已连接")',
'',
' # ── iOS 测试步骤 (同 Android 逻辑，适配 XCTest) ──',
' results["passed"] += 1',
'',
' except Exception as exc:',
' results["failed"] += 1',
' results["errors"].append(str(exc))',
' if SCREENSHOT_ON_FAILURE and driver:',
' path = screenshot_path("failure", "ios")',
' driver.save_screenshot(path)',
' results["screenshots"].append(path)',
'',
' finally:',
' if driver:',
' driver.quit()',
'',
' return results',
'',
'',
'def main():',
' """主入口。"""',
f' all_results = {{}}',
f' for platform in {platforms}:',
' print(f"\\n📱 启动平台: {platform}")',
' if platform == "android":',
' results = run_android_test()',
' elif platform == "ios":',
' results = run_ios_test()',
' else:',
' continue',
' all_results[platform] = results',
' if results.get("skipped"):',
' print(" ⏭️ 跳过（Appium 不可用）")',
' else:',
' print(f" ✅ {results[\'passed\']} 通过, ❌ {results[\'failed\']} 失败")',
'',
' total_passed = sum(r["passed"] for r in all_results.values())',
' total_failed = sum(r["failed"] for r in all_results.values())',
' total_screenshots = sum(len(r["screenshots"]) for r in all_results.values())',
' print(f"\\n🏁 移动端执行完成: 总通过 {total_passed}, 总失败 {total_failed}, 截图 {total_screenshots}")',
'',
' return all_results',
'',
'',
'if __name__ == "__main__":',
' main()',
])
script_path.write_text("\n".join(lines) + "\n", encoding="utf-8")
return script_path
def _write_execution_report(
base_name: str,
test_cases_path: Path,
playwright_script: Path,
appium_script: Path,
screenshots_dir: Path,
report_path: Path,
) -> None:
"""生成综合执行报告。"""
test_case_count = 0
p0_count = 0
p1_count = 0
if test_cases_path.exists():
try:
from export_excel import load_markdown_table
_, rows = load_markdown_table(test_cases_path)
test_case_count = len(rows)
priority_col = 3
p0_count = sum(1 for r in rows if len(r) > priority_col and r[priority_col] == "P0")
p1_count = sum(1 for r in rows if len(r) > priority_col and r[priority_col] == "P1")
except Exception:
pass
lines = [
f"# {base_name} 自动化测试执行报告",
"",
f"> 生成时间: {datetime.now(timezone.utc).isoformat()}",
f"> 生成引擎: Agentic QE Fleet v2.1.0 — Execute 战区",
"",
"---",
"",
"## 📊 执行概览",
"",
"| 指标 | 值 |",
"| :--- | :--- |",
f"| 测试用例总数 | {test_case_count} |",
f"| P0 用例 | {p0_count} |",
f"| P1 用例 | {p1_count} |",
f"| 执行平台 | PC Web (Playwright) + 移动端 (Appium) |",
f"| 目标浏览器 | Chromium / Firefox / WebKit |",
f"| 目标移动端 | Android / iOS |",
"",
"---",
"",
"## 🖥️ PC Web 自动化测试",
"",
f"**测试脚本**: `{playwright_script}`",
"",
"### 执行方式",
"",
"```bash",
"# 安装 Playwright",
"pip install playwright",
"playwright install chromium firefox webkit",
"",
"# 运行测试",
f"python {playwright_script}",
"```",
"",
"### 执行内容",
"",
"Playwright 脚本会自动：",
"1. 启动目标浏览器（Chromium/Firefox/WebKit）",
"2. 按测试用例中的 P0/P1 场景逐步骤执行",
"3. 每步/失败时自动截图 → `output/screenshots/{BASE_NAME}/`",
"4. 超时自动重试（默认 1 次）",
"5. 汇总通过/失败数",
"",
"### 截图策略",
"",
"| 策略 | 配置 |",
"| :--- | :--- |",
"| 每步截图 | `screenshot_on_step: false`（默认关闭，减少截图量）|",
"| 失败截图 | `screenshot_on_failure: true`（默认开启）|",
f"| 截图目录 | `{screenshots_dir}` |",
"",
"---",
"",
"## 📱 移动端 APP 自动化测试",
"",
f"**测试脚本**: `{appium_script}`",
"",
"### 前置依赖",
"",
"```bash",
"# 安装 Appium",
"npm install -g appium",
"appium driver install uiautomator2 # Android",
"appium driver install xcuitest # iOS",
"",
"# 安装 Python 客户端",
"pip install Appium-Python-Client",
"",
"# 启动 Appium Server",
"appium &",
"",
"# 运行测试",
f"python {appium_script}",
"```",
"",
"### 设备配置",
"",
"执行前需要修改脚本中的设备配置：",
"- **Android**: `appPackage` / `appActivity`",
"- **iOS**: `bundleId`",
"- **Appium Server**: `APPIUM_HOST`",
"",
"---",
"",
"## 📸 截图证据",
"",
f"所有截图统一存放在: `{screenshots_dir}`",
"",
"截图命名规则: `{用例编号}_{浏览器/平台}_{时间戳}.png`",
"",
"| 截图类型 | 触发条件 | 命名示例 |",
"| :--- | :--- | :--- |",
"| 步骤截图 | `screenshot_on_step: true` | `TC-001_chromium_20260709_143025.png` |",
"| 失败截图 | 断言/异常 | `failure_android_20260709_143025.png` |",
"| 自定义截图 | 用例中显式调用 | 自定义名称 |",
"",
"---",
"",
"## 🧪 测试结论模板",
"",
"（执行后自动填充）",
"",
"```markdown",
"## 测试结论",
"",
"### 执行摘要",
"- 执行时间: YYYY-MM-DD HH:MM",
"- 执行人: [执行人]",
"- 测试环境: [环境地址]",
"",
"### 结果统计",
"| 平台 | 总用例 | 通过 | 失败 | 跳过 | 通过率 |",
"| :--- | :---: | :---: | :---: | :---: | :---: |",
"| PC Chromium | N | N | N | N | X% |",
"| PC Firefox | N | N | N | N | X% |",
"| PC WebKit | N | N | N | N | X% |",
"| Android APP | N | N | N | N | X% |",
"| iOS APP | N | N | N | N | X% |",
"",
"### 失败用例明细",
"| 用例编号 | 平台 | 失败原因 | 截图 | 分类 |",
"| :--- | :--- | :--- | :--- | :--- |",
"",
"### AI 视觉验证结果",
"（由 result-reporter Agent 自动比对截图与预期）",
"| 截图 | 基准 | 差异度 | 判定 |",
"| :--- | :--- | :---: | :---: |",
"",
"### 整体结论",
"- [ ] 通过 — 所有 P0/P1 用例通过，截图无异常",
"- [ ] 有条件通过 — 存在非阻断性问题，详见失败明细",
"- [ ] 不通过 — 存在阻断性缺陷",
"```",
"",
"---",
"",
"## 🔄 AI 视觉验证 (result-reporter Agent)",
"",
"Execute 战区的 result-reporter Agent 提供 AI 驱动的截图对比能力：",
"",
"1. **截图采集**: 执行过程中自动采集截图",
"2. **基准对比**: 与预期效果图/上次通过的截图对比",
"3. **差异检测**: AI 识别 UI 布局、文字、颜色等差异",
"4. **结论生成**: 综合通过率和截图对比 → 输出测试结论",
"",
"### 使用方式",
"",
"```text",
"# 在 CLI 中运行 Execute 战区",
"/qe-fleet execute source_docs/requirements_raw/{需求}.docx",
"",
"# 或查看执行报告",
"/qe-fleet status source_docs/requirements_raw/{需求}.docx",
"```",
"",
"---",
"",
"> ⚠️ **重要提示**: 本报告由 Agentic QE Fleet 自动生成。",
"> 测试脚本为骨架代码，需要根据实际测试环境配置 BASE_URL、测试账号、设备信息等参数。",
"> 执行前请确认 Playwright/Appium 环境已正确安装配置。",
]
report_path.write_text("\n".join(lines).rstrip() + "\n", encoding="utf-8")
def _run_review_zone(
base_name: str,