init the project
This commit is contained in:
evyzacq
+127
@@ -0,0 +1,127 @@
|
||||
---
|
||||
name: 文档分析代理
|
||||
description: 一个智能代理，用于分析文档（.docx, .pdf），提取和结构化内容，检测文本与图表之间的冲突，并生成结构化的JSON中间表示。
|
||||
---
|
||||
|
||||
# 文档分析代理
|
||||
|
||||
## 环境变量配置
|
||||
|
||||
在执行任何分析之前，必须先检查用户是否配置了DASHSCOPE_API_KEY，如果没有提示用户设置为环境变量DASHSCOPE_API_KEY。
|
||||
所有脚本通过该环境变量读取 API Key。严禁在对话或命令行中明文写入或显示 API Key。
|
||||
|
||||
### 配置方式
|
||||
|
||||
```
|
||||
openclaw config set env.DASHSCOPE_API_KEY "llm-api-key"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 功能
|
||||
|
||||
代理能够：
|
||||
- 解析各种文档格式（.docx, .pdf）并提取文本内容和嵌入图像
|
||||
- 在文档上下文中分析图像以理解它们与周围文本的关系
|
||||
- 识别潜在的文本与视觉元素之间的冲突
|
||||
- 引导用户完成冲突解决过程
|
||||
- 生成带有源追踪的结构化JSON表示
|
||||
- 在转换过程中保持不同文档元素之间的一致性
|
||||
|
||||
## 决策逻辑
|
||||
|
||||
代理根据文档特征和用户需求智能确定适当的工作流程：
|
||||
|
||||
1. **文档评估阶段**：当用户提供文档时，代理首先根据文档格式和内容复杂性确定适当的解析方法。
|
||||
|
||||
2. **内容分析阶段**：代理分析提取的内容以识别需要特殊处理的图表、流程图、架构图、状态图和序列图。
|
||||
|
||||
3. **冲突检测阶段**：代理识别文本内容与视觉元素之间的潜在差异，特别关注条件不匹配和矛盾信息。
|
||||
|
||||
4. **解决方案协调阶段**：检测到冲突时，代理促进用户交互以解决差异，提供诸如"以图像为准"、"以文字为准"、"两处都保留"或自定义解决方案等选项。
|
||||
|
||||
5. **表示生成阶段**：代理综合所有输入并生成带源追踪的结构化JSON中间表示。
|
||||
|
||||
## 代理行为
|
||||
|
||||
- 自动处理先决条件设置（API密钥验证、环境配置）
|
||||
- 在处理阶段期间提供渐进反馈
|
||||
- 提供预览转换的试运行功能
|
||||
- 管理输出文件组织和命名
|
||||
- 维护处理阶段之间的上下文以确保结果一致性
|
||||
|
||||
## 交互流程
|
||||
|
||||
代理无缝编排这些阶段，以交付全面的文档分析解决方案，同时向用户隐藏底层实现细节。
|
||||
自动执行所有阶段，无需询问用户是否执行下一步，除非需要用户介入协助。
|
||||
|
||||
### 1. 初始化
|
||||
验证先决条件并准备处理环境。
|
||||
|
||||
### 2. 解析
|
||||
从输入文档中提取内容和结构（运行 doc_parser_skill）。产出：`<basename>_parsed.json`。
|
||||
|
||||
### 3. 分析
|
||||
识别关键元素和可能需要关注的区域。
|
||||
|
||||
### 4. 冲突解决
|
||||
运行 conflict_detection_skill，检测文本与图表之间的差异。如发现冲突，请求用户裁决后执行 resolution_application_skill。产出：`<basename>_updated.json`。
|
||||
|
||||
### 5. 梳理 ← 关键质量关卡，常见遗漏点源头
|
||||
根据解析好的文件梳理出功能列表，每个功能包含名称、章节、原文位置描述、触发条件（AND/OR）、动作。
|
||||
|
||||
**必须同时从以下三种来源穷举功能点，缺一不可，且不能跳过b/c直接执行：**
|
||||
|
||||
**a. 文本与表格** — 提取所有章节中段落和表格明确描述的功能逻辑。
|
||||
|
||||
**b. 流程图/状态图/架构图描述**（image_analysis[] 中 type 为 flowchart / state / architecture / sequence / activity 的条目）：
|
||||
- 以决策树的视角逐条追踪每个菱形判断节点，沿"是/否"两条分支分别行走
|
||||
- **每条可达决策路径必须对应一个独立的 IR 条目**，不得将多条路径合并为一条概括性描述
|
||||
- 从起始节点出发，沿着箭头逐层展开，确保没有遗漏任何分支末端
|
||||
- 每条路径提取：触发条件链（AND/OR组合）+ 执行动作 + 分支上的具体条件值
|
||||
- 特别注意图中那些"文字没有写"的隐性条件、默认行为、边界值
|
||||
|
||||
**c. 交互图/UI场景图描述**（image_analysis[] 中 type 为 other 但 description 描述UI交互场景的条目）：
|
||||
- 分析交互图中的每个场景编号（如01、02、A、B、C等）
|
||||
- 每个场景拆分出一个独立的功能点，包含：触发条件（用户操作或系统状态）、系统响应（Toast文案、页面变化等）
|
||||
- 如果多个场景在逻辑上形成流程链，保留上下游关系
|
||||
|
||||
**d. 交叉验证** — 对每个已识别的功能点标注来源（text / image_rIdXX），确保 text 来源覆盖了文档所有文字，image 来源覆盖了所有 image_analysis 中有行为逻辑的条目。
|
||||
|
||||
### 6. 合成
|
||||
根据梳理的功能列表，生成最终结构化表示（IR），输出到输出目录的 `<文档名>_ir.json` 文件中。
|
||||
使用 ir_generation_skill 脚本。
|
||||
|
||||
### 7. 检查
|
||||
对比解析好的文件（parsed.json/updated.json）和合成的 IR 文件，**对照以下清单逐项验证**，找到遗漏点：
|
||||
|
||||
**7a. 文本覆盖检查** — 遍历 parsed.json 中所有 sections[].blocks（包括 para 和 table），对每个段/格判断：
|
||||
- 是否描述了某个功能行为？
|
||||
- 若是，该行为是否已在 IR 中？
|
||||
- 遗漏的标记出来。
|
||||
|
||||
**7b. 流程图路径完整性检查** — 逐张流程图 description，对照决策树结构：
|
||||
- □ 每条"是"分支路径 → IR 中有对应条目
|
||||
- □ 每条"否"分支路径 → IR 中有对应条目
|
||||
- □ 起始节点的每个出口都被追踪到
|
||||
- □ 边角情况（分支合并、默认走法）也已覆盖
|
||||
|
||||
**7c. 交互图场景检查** — 逐张交互图 description：
|
||||
- □ 每个场景编号（01/02/A/B/C等）→ IR 中有对应条目
|
||||
- □ 每个 UX 文案 → IR 中有对应条目或作为已有条目的动作细节
|
||||
- □ 特殊行为 → IR 中有对应条件
|
||||
|
||||
**7d. 功能列表交叉检查** — 如果文档有功能列表表格：
|
||||
- □ 表格中的每一行 → IR 中有对应条目
|
||||
- □ 章节中的详细描述 → IR 中有对应条目，且粒度不粗于功能列表
|
||||
|
||||
### 8. 补充
|
||||
根据检查结果，将遗漏功能加入已有 IR 文件。
|
||||
使用 ir_generation_skill 添加每个遗漏点。
|
||||
**每补充完一批，回到步骤7重新检查**，直到所有检查项均为 ✅（即功能点一致）。
|
||||
|
||||
### 9. 输出
|
||||
提供带追踪信息的完整分析结果，包括：
|
||||
- 功能点清单总览
|
||||
- 冲突解决记录
|
||||
- 文档质量备注（如图片错误、占位符、不一致等）
|
||||
+2
@@ -0,0 +1,2 @@
|
||||
openai
|
||||
python-docx
|
||||
@@ -0,0 +1,37 @@
|
||||
---
|
||||
name: 冲突检测技能
|
||||
description: 分析解析后的文档，检测图表类图像与其相应文本描述之间的矛盾和条件不匹配。
|
||||
---
|
||||
|
||||
# 冲突检测技能
|
||||
|
||||
## 概述
|
||||
|
||||
此技能识别解析文档中文本内容与视觉内容之间的潜在冲突。它特别针对图表类图像（流程图、架构图、状态图、序列图和活动图）并交叉检查其描述与同文档部分的文本内容。
|
||||
|
||||
## 功能
|
||||
|
||||
该技能：
|
||||
- 从解析的文档结构中识别图表类图像
|
||||
- 将图像描述与同一文档部分中的相应文本内容进行交叉引用
|
||||
- 检测视觉表示和文本表示之间的矛盾和条件不匹配
|
||||
- 生成包含其位置的已识别冲突的结构化列表
|
||||
- 专门针对流程图、架构图、状态图、序列图和活动图
|
||||
|
||||
## 输入要求
|
||||
|
||||
- 解析文档JSON文件的路径（由文档解析技能生成）
|
||||
- 可选输出目录规范
|
||||
- 可选试运行标志，在不调用API的情况下预览大语言模型提示
|
||||
|
||||
## 输出
|
||||
|
||||
该技能生成一个结构化JSON文件，文件名为输入文档的基本名称后跟'_conflicts.json'，包含：
|
||||
- 带有关于差异详情的冲突对象列表
|
||||
- 标识每个冲突发生位置的节标识符
|
||||
- 冲突图像和文本内容的片段
|
||||
- 每个冲突的类型分类（例如，矛盾、条件不匹配）
|
||||
|
||||
## 集成点
|
||||
|
||||
此技能消耗文档解析技能的输出并为解决方案应用技能提供输入。冲突解决过程通常需要人工输入才能进入下一阶段。
|
||||
+105
@@ -0,0 +1,105 @@
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
from typing import Optional
|
||||
|
||||
from openai import OpenAI
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class LLMClient:
|
||||
"""Low-level OpenAI-compatible LLM client with retry and token tracking.
|
||||
|
||||
Usage::
|
||||
|
||||
llm = LLMClient()
|
||||
content = llm.chat("qwen3.5-flash", [{"role": "user", "content": "Hello"}])
|
||||
print(llm.usage)
|
||||
"""
|
||||
|
||||
IMAGE_MODEL = "qwen3-vl-plus"
|
||||
TEXT_MODEL = "qwen3.5-flash-2026-02-23"
|
||||
TIMEOUT = 120
|
||||
MAX_RETRIES = 3
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
base_url: str = "https://dashscope.aliyuncs.com/compatible-mode/v1",
|
||||
timeout: int | None = None,
|
||||
):
|
||||
key = os.environ.get("DASHSCOPE_API_KEY", "")
|
||||
if not key:
|
||||
raise ValueError("DASHSCOPE_API_KEY environment variable is not set.")
|
||||
self._client = OpenAI(api_key=key, base_url=base_url)
|
||||
self._timeout = timeout or self.TIMEOUT
|
||||
self._prompt_tokens = 0
|
||||
self._completion_tokens = 0
|
||||
|
||||
@property
|
||||
def usage(self) -> dict:
|
||||
"""Return accumulated token counts as ``{prompt, completion, total}``."""
|
||||
return {
|
||||
"prompt_tokens": self._prompt_tokens,
|
||||
"completion_tokens": self._completion_tokens,
|
||||
"total_tokens": self._prompt_tokens + self._completion_tokens,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def estimate_tokens(text: str) -> int:
|
||||
"""Quick token estimate. CJK ≈1.7/token, others ≈3.0/token."""
|
||||
cjk = sum(1 for c in text if '一' <= c <= '鿿' or ' ' <= c <= '〿')
|
||||
other = len(text) - cjk
|
||||
return max(1, int(cjk / 1.7 + other / 3.0))
|
||||
|
||||
@staticmethod
|
||||
def estimate_image_tokens() -> int:
|
||||
"""Fixed estimate for one vision-model image (~500 tokens)."""
|
||||
return 500
|
||||
|
||||
def chat(
|
||||
self, model: str, messages: list[dict], *, timeout: int | None = None,
|
||||
response_format: dict | None = None,
|
||||
) -> str:
|
||||
"""Send a chat completion request and return the response content.
|
||||
|
||||
Automatically retries on failure and accumulates token usage.
|
||||
"""
|
||||
label = f"chat({model})"
|
||||
|
||||
def _call():
|
||||
t0 = time.time()
|
||||
kwargs = dict(model=model, messages=messages, timeout=timeout or self._timeout)
|
||||
if response_format is not None:
|
||||
kwargs["response_format"] = response_format
|
||||
kwargs["temperature"] = 0
|
||||
resp = self._client.chat.completions.create(**kwargs)
|
||||
content = resp.choices[0].message.content
|
||||
usg = resp.usage
|
||||
if usg:
|
||||
self._prompt_tokens += usg.prompt_tokens
|
||||
self._completion_tokens += usg.completion_tokens
|
||||
elapsed = time.time() - t0
|
||||
logger.info("%s: %d chars in %.1fs", label, len(content) if content else 0, elapsed)
|
||||
if not content:
|
||||
raise RuntimeError("Empty response from LLM")
|
||||
return content
|
||||
|
||||
return self._retry(_call, label)
|
||||
|
||||
def _retry(self, fn, label: str) -> str:
|
||||
"""Call *fn()* with exponential-backoff retry."""
|
||||
last_error: Optional[Exception] = None
|
||||
for attempt in range(self.MAX_RETRIES):
|
||||
try:
|
||||
return fn()
|
||||
except Exception as e:
|
||||
last_error = e
|
||||
logger.warning(
|
||||
"%s error (attempt %d/%d): %s",
|
||||
label, attempt + 1, self.MAX_RETRIES, e,
|
||||
)
|
||||
if attempt < self.MAX_RETRIES - 1:
|
||||
time.sleep(2 ** attempt)
|
||||
raise RuntimeError(f"{label}: all retries exhausted") from last_error
|
||||
Executable
+280
@@ -0,0 +1,280 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Detect logical conflicts between image analysis and text in ``_parsed.json``.
|
||||
|
||||
Usage::
|
||||
|
||||
python scripts/detect_conflicts.py D:/projects/jike/output/车机娱乐系统禁止功能文档_精简_parsed.json [--output-dir DIR]
|
||||
|
||||
For each diagram-type image (flowchart, architecture, state, sequence, activity),
|
||||
the script locates its section via *image_sources*, grabs the corresponding text
|
||||
blocks, and calls an LLM to find contradictions/condition-mismatches between the
|
||||
image description and the text.
|
||||
|
||||
Output: ``<basename>_conflicts.json``
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import time
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from LLM import LLMClient
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s [%(levelname)s] %(message)s",
|
||||
datefmt="%Y-%m-%d %H:%M:%S",
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
RATE_LIMIT_DELAY = 0.5
|
||||
DIAGRAM_TYPES = {"flowchart", "architecture", "state", "sequence", "activity"}
|
||||
MIN_TEXT_CHARS = 20
|
||||
|
||||
PROMPT_DETECT_CONFLICT = """你是一个文档一致性检查专家。以下内容来自同一份需求文档的同一个章节，包含两部分：
|
||||
|
||||
## 部分1：图片（流程图/架构图/状态图）的描述
|
||||
```
|
||||
{image_description}
|
||||
```
|
||||
|
||||
## 部分2：同章节的文字描述
|
||||
```
|
||||
{text_description}
|
||||
```
|
||||
|
||||
## 你的任务
|
||||
检查这两部分之间是否存在**逻辑矛盾或条件不一致**。
|
||||
|
||||
你需要关注的冲突类型：
|
||||
|
||||
1. **condition_mismatch**（条件不一致）：两者描述了同一规则，但触发条件、阈值、时序不同。
|
||||
例如：图片说"车速≥15km/h且持续5秒"，文字说"车速≥10km/h且持续3秒"。
|
||||
例如：图片说"非P档限制"，文字说"车速>0限制"。
|
||||
|
||||
2. **contradiction**（直接矛盾）：两者对同一事物的描述完全相反。
|
||||
例如：图片说"功能X被禁止"，文字说"功能X可用"。
|
||||
例如：图片说"开关默认关闭"，文字说"开关默认开启"。
|
||||
|
||||
3. **scope_mismatch**（范围不一致）：两者描述的场景/地域/设备范围不同。
|
||||
例如：图片说"国内方案"，文字说"海外方案"。
|
||||
例如：图片说"CSD中控屏"，文字描述包含"PSD副驾屏"。
|
||||
|
||||
## 输出格式
|
||||
|
||||
如果**没有冲突**，只输出：
|
||||
```
|
||||
[[NO_CONFLICT]]
|
||||
```
|
||||
|
||||
如果**有冲突**，输出以下JSON数组（不要任何其他文字）：
|
||||
|
||||
```json
|
||||
[
|
||||
{{
|
||||
"conflict_type": "condition_mismatch",
|
||||
"severity": "high",
|
||||
"section": "{section_name}",
|
||||
"image_snippet": "图片中描述的关键内容（摘录）",
|
||||
"text_snippet": "文字中描述的关键内容（摘录）",
|
||||
"description": "用中文说明冲突的具体差异"
|
||||
}}
|
||||
]
|
||||
```
|
||||
|
||||
注意：
|
||||
- 每个冲突一个条目，不要合并
|
||||
- severity: high（功能正确性受影响）| medium（边界条件模糊）| low（表达方式差异）
|
||||
- 输出必须是**严格合法的JSON数组**，不要有尾随逗号
|
||||
- 如果没有严格冲突，输出 [[NO_CONFLICT]]
|
||||
"""
|
||||
|
||||
|
||||
def _build_text_for_section(sections: list[dict], section_name: str) -> str:
|
||||
"""Build a single text block for the given section name."""
|
||||
texts: list[str] = []
|
||||
for sec in sections:
|
||||
if sec.get("source", "") == section_name:
|
||||
for blk in sec.get("blocks", []):
|
||||
if blk["type"] == "para":
|
||||
texts.append(blk["text"])
|
||||
elif blk["type"] == "table":
|
||||
table_lines = [f"表格 {blk['table']}:"]
|
||||
for ri, row in enumerate(blk.get("rows", [])):
|
||||
cols = row.get("columns", [])
|
||||
parts = [f"{c['name']}: {c['text']}" for c in cols]
|
||||
table_lines.append(f" 行{ri + 1}: {' | '.join(parts)}")
|
||||
texts.append("\n".join(table_lines))
|
||||
return "\n\n".join(texts)
|
||||
|
||||
|
||||
def _parse_conflict_json(content: str) -> list[dict]:
|
||||
"""Extract JSON array from LLM response, handling markdown fences."""
|
||||
stripped = content.strip()
|
||||
|
||||
if "[[NO_CONFLICT]]" in stripped:
|
||||
return []
|
||||
|
||||
# Remove markdown code fences
|
||||
if "```json" in stripped:
|
||||
stripped = stripped.split("```json", 1)[1]
|
||||
if "```" in stripped:
|
||||
stripped = stripped.split("```", 1)[0]
|
||||
elif "```" in stripped:
|
||||
stripped = stripped.split("```", 1)[1]
|
||||
if "```" in stripped:
|
||||
stripped = stripped.split("```", 1)[0]
|
||||
|
||||
stripped = stripped.strip()
|
||||
if not stripped:
|
||||
return []
|
||||
|
||||
# Try to find a JSON array
|
||||
match = re.search(r"\[\s*\{.*\}\s*\]", stripped, re.DOTALL)
|
||||
if match:
|
||||
stripped = match.group()
|
||||
|
||||
try:
|
||||
conflicts = json.loads(stripped)
|
||||
if isinstance(conflicts, list):
|
||||
return conflicts
|
||||
return []
|
||||
except json.JSONDecodeError as e:
|
||||
logger.warning("Failed to parse conflict JSON: %s", e)
|
||||
logger.debug("Raw content: %s", stripped)
|
||||
return []
|
||||
|
||||
|
||||
def detect_conflicts(
|
||||
parsed_path: str,
|
||||
output_dir: str | None = None,
|
||||
*,
|
||||
dry_run: bool = False,
|
||||
) -> list[dict]:
|
||||
"""Load ``_parsed.json`` and detect image-vs-text conflicts.
|
||||
|
||||
Returns a flat list of conflict dicts and writes to ``<basename>_conflicts.json``.
|
||||
"""
|
||||
with open(parsed_path, "r", encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
|
||||
basename = os.path.splitext(os.path.basename(parsed_path))[0]
|
||||
if basename.endswith("_parsed"):
|
||||
basename = basename[:-7]
|
||||
|
||||
if output_dir is None:
|
||||
output_dir = os.path.dirname(os.path.abspath(parsed_path))
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
sections = data.get("sections", [])
|
||||
image_sources = data.get("image_sources", {})
|
||||
image_analysis = data.get("image_analysis", [])
|
||||
|
||||
llm = LLMClient()
|
||||
all_conflicts: list[dict] = []
|
||||
|
||||
# ---- For each diagram image, compare with its section text -------------
|
||||
for img in image_analysis:
|
||||
img_type = img.get("type", "other")
|
||||
rid = img.get("rid", "")
|
||||
description = img.get("description", "").strip()
|
||||
|
||||
if img_type not in DIAGRAM_TYPES or not description:
|
||||
logger.info("Skip conflict check: rid=%s type=%s", rid, img_type)
|
||||
continue
|
||||
|
||||
# Find source section
|
||||
src = image_sources.get(rid, {})
|
||||
section_name = src.get("section", "")
|
||||
|
||||
if not section_name:
|
||||
logger.warning("No section found for rid=%s, skipping", rid)
|
||||
continue
|
||||
|
||||
# Build text from the same section
|
||||
text_content = _build_text_for_section(sections, section_name)
|
||||
text_len = len(text_content.strip())
|
||||
if text_len < MIN_TEXT_CHARS:
|
||||
logger.info("Section text too short (%d chars) for rid=%s, skip", text_len, rid)
|
||||
continue
|
||||
|
||||
logger.info("Checking conflicts: rid=%s section=%s (desc=%d chars, text=%d chars)",
|
||||
rid, section_name, len(description), text_len)
|
||||
|
||||
if dry_run:
|
||||
logger.info(" [DRY RUN] would call LLM to detect conflicts")
|
||||
continue
|
||||
|
||||
prompt = PROMPT_DETECT_CONFLICT.format(
|
||||
image_description=description,
|
||||
text_description=text_content,
|
||||
section_name=section_name,
|
||||
)
|
||||
|
||||
try:
|
||||
raw = llm.chat(
|
||||
model=LLMClient.TEXT_MODEL,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
)
|
||||
logger.info("Conflict check response: %d chars", len(raw))
|
||||
except RuntimeError as e:
|
||||
logger.error("Conflict check failed: %s", e)
|
||||
continue
|
||||
|
||||
conflicts = _parse_conflict_json(raw)
|
||||
|
||||
# Enrich with location info
|
||||
for c in conflicts:
|
||||
c["rid"] = rid
|
||||
c["image_path"] = img.get("path", "")
|
||||
if "section" not in c:
|
||||
c["section"] = section_name
|
||||
if src.get("table"):
|
||||
c.setdefault("source_location", {})["table"] = src["table"]
|
||||
if src.get("row"):
|
||||
c.setdefault("source_location", {})["image_row"] = src["row"]
|
||||
|
||||
all_conflicts.extend(conflicts)
|
||||
logger.info(" Found %d conflicts for rid=%s", len(conflicts), rid)
|
||||
|
||||
if any(x.get("type") in DIAGRAM_TYPES
|
||||
for x in image_analysis
|
||||
if x.get("rid", "") != rid):
|
||||
time.sleep(RATE_LIMIT_DELAY)
|
||||
|
||||
# ---- Save ---------------------------------------------------------------
|
||||
conflicts_path = os.path.join(output_dir, f"{basename}_conflicts.json")
|
||||
with open(conflicts_path, "w", encoding="utf-8") as f:
|
||||
json.dump(all_conflicts, f, ensure_ascii=False, indent=2)
|
||||
logger.info("Saved: %s (%d conflicts)", conflicts_path, len(all_conflicts))
|
||||
|
||||
# ---- Summary ------------------------------------------------------------
|
||||
usg = llm.usage
|
||||
logger.info("Tokens: %d prompt + %d completion = %d total",
|
||||
usg["prompt_tokens"], usg["completion_tokens"], usg["total_tokens"])
|
||||
|
||||
return all_conflicts
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Detect image-vs-text conflicts in parsed document.",
|
||||
)
|
||||
parser.add_argument("input", metavar="parsed.json", help="Path to _parsed.json from doc_parser")
|
||||
parser.add_argument("--output-dir", metavar="DIR", default=None,
|
||||
help="Output directory (default: same as input)")
|
||||
parser.add_argument("--dry-run", action="store_true",
|
||||
help="Print LLM prompts without calling the API.")
|
||||
|
||||
args = parser.parse_args()
|
||||
detect_conflicts(args.input, output_dir=args.output_dir, dry_run=args.dry_run)
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
name: 文档解析技能
|
||||
description: 解析文档（.docx, .pdf）以提取图像和文本结构，并使用视觉大语言模型分析每个图像的类型和描述。
|
||||
---
|
||||
|
||||
# 文档解析技能
|
||||
|
||||
## 概述
|
||||
|
||||
此技能从文档（.docx, .pdf）中提取内容并准备进行进一步分析。它提取文本内容和嵌入图像，并对图像执行初始分析以了解其类型和内容。
|
||||
|
||||
## 功能
|
||||
|
||||
该技能：
|
||||
- 从文档中提取文本结构（段落、表格、标题）
|
||||
- 识别并提取嵌入的图像
|
||||
- 使用视觉大语言模型分析每个图像并确定其类型和内容描述
|
||||
- 生成结构化输出，将图像映射到其在文档中的位置
|
||||
- 创建文档的初始解析表示，供后续处理阶段使用
|
||||
|
||||
## 输入要求
|
||||
|
||||
- 文档文件路径（必需，支持.docx和.pdf格式）
|
||||
- 可选输出目录（默认为'output/'）
|
||||
- 可选试运行标志，在不调用API的情况下预览大语言模型提示
|
||||
|
||||
## 输出
|
||||
|
||||
该技能生成一个结构化JSON文件，文件名为输入文档的基本名称后跟'_parsed.json'，包含：
|
||||
- `sections`：按标题分组的文档文本结构
|
||||
- `image_sources`：从图像标识符到其在文档中位置的映射
|
||||
- `image_analysis`：由视觉大语言模型确定的每个图像的类型和内容描述
|
||||
|
||||
## 集成点
|
||||
|
||||
此技能作为文档分析管道中的初始处理步骤。其输出被冲突检测技能消费以识别文本和视觉内容之间的差异。
|
||||
@@ -0,0 +1,105 @@
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
from typing import Optional
|
||||
|
||||
from openai import OpenAI
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class LLMClient:
|
||||
"""Low-level OpenAI-compatible LLM client with retry and token tracking.
|
||||
|
||||
Usage::
|
||||
|
||||
llm = LLMClient()
|
||||
content = llm.chat("qwen3.5-flash", [{"role": "user", "content": "Hello"}])
|
||||
print(llm.usage)
|
||||
"""
|
||||
|
||||
IMAGE_MODEL = "qwen3-vl-plus"
|
||||
TEXT_MODEL = "qwen3.5-flash-2026-02-23"
|
||||
TIMEOUT = 120
|
||||
MAX_RETRIES = 3
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
base_url: str = "https://dashscope.aliyuncs.com/compatible-mode/v1",
|
||||
timeout: int | None = None,
|
||||
):
|
||||
key = os.environ.get("DASHSCOPE_API_KEY", "")
|
||||
if not key:
|
||||
raise ValueError("DASHSCOPE_API_KEY environment variable is not set.")
|
||||
self._client = OpenAI(api_key=key, base_url=base_url)
|
||||
self._timeout = timeout or self.TIMEOUT
|
||||
self._prompt_tokens = 0
|
||||
self._completion_tokens = 0
|
||||
|
||||
@property
|
||||
def usage(self) -> dict:
|
||||
"""Return accumulated token counts as ``{prompt, completion, total}``."""
|
||||
return {
|
||||
"prompt_tokens": self._prompt_tokens,
|
||||
"completion_tokens": self._completion_tokens,
|
||||
"total_tokens": self._prompt_tokens + self._completion_tokens,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def estimate_tokens(text: str) -> int:
|
||||
"""Quick token estimate. CJK ≈1.7/token, others ≈3.0/token."""
|
||||
cjk = sum(1 for c in text if '一' <= c <= '鿿' or ' ' <= c <= '〿')
|
||||
other = len(text) - cjk
|
||||
return max(1, int(cjk / 1.7 + other / 3.0))
|
||||
|
||||
@staticmethod
|
||||
def estimate_image_tokens() -> int:
|
||||
"""Fixed estimate for one vision-model image (~500 tokens)."""
|
||||
return 500
|
||||
|
||||
def chat(
|
||||
self, model: str, messages: list[dict], *, timeout: int | None = None,
|
||||
response_format: dict | None = None,
|
||||
) -> str:
|
||||
"""Send a chat completion request and return the response content.
|
||||
|
||||
Automatically retries on failure and accumulates token usage.
|
||||
"""
|
||||
label = f"chat({model})"
|
||||
|
||||
def _call():
|
||||
t0 = time.time()
|
||||
kwargs = dict(model=model, messages=messages, timeout=timeout or self._timeout)
|
||||
if response_format is not None:
|
||||
kwargs["response_format"] = response_format
|
||||
kwargs["temperature"] = 0
|
||||
resp = self._client.chat.completions.create(**kwargs)
|
||||
content = resp.choices[0].message.content
|
||||
usg = resp.usage
|
||||
if usg:
|
||||
self._prompt_tokens += usg.prompt_tokens
|
||||
self._completion_tokens += usg.completion_tokens
|
||||
elapsed = time.time() - t0
|
||||
logger.info("%s: %d chars in %.1fs", label, len(content) if content else 0, elapsed)
|
||||
if not content:
|
||||
raise RuntimeError("Empty response from LLM")
|
||||
return content
|
||||
|
||||
return self._retry(_call, label)
|
||||
|
||||
def _retry(self, fn, label: str) -> str:
|
||||
"""Call *fn()* with exponential-backoff retry."""
|
||||
last_error: Optional[Exception] = None
|
||||
for attempt in range(self.MAX_RETRIES):
|
||||
try:
|
||||
return fn()
|
||||
except Exception as e:
|
||||
last_error = e
|
||||
logger.warning(
|
||||
"%s error (attempt %d/%d): %s",
|
||||
label, attempt + 1, self.MAX_RETRIES, e,
|
||||
)
|
||||
if attempt < self.MAX_RETRIES - 1:
|
||||
time.sleep(2 ** attempt)
|
||||
raise RuntimeError(f"{label}: all retries exhausted") from last_error
|
||||
+106
@@ -0,0 +1,106 @@
|
||||
import argparse
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from image_parser import ImageParser
|
||||
from LLM import LLMClient
|
||||
from word_parser import WordParser
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s [%(levelname)s] %(message)s",
|
||||
datefmt="%Y-%m-%d %H:%M:%S",
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
RATE_LIMIT_DELAY = 0.5
|
||||
|
||||
|
||||
def parse_document(
|
||||
docx_path: str,
|
||||
output_dir: str = "output",
|
||||
*,
|
||||
dry_run: bool = False,
|
||||
) -> dict:
|
||||
"""Parse a .docx file: extract text structure and parse embedded images.
|
||||
|
||||
Produces ``<basename>_parsed.json`` in *output_dir*.
|
||||
"""
|
||||
word = WordParser(docx_path)
|
||||
|
||||
basename = os.path.splitext(os.path.basename(docx_path))[0]
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
images_dir = os.path.join(output_dir, "images")
|
||||
|
||||
# ---- extract sections and images -----------------------------------------
|
||||
sections, image_sources = word.extract_sections()
|
||||
logger.info("Document has %d sections, %d image sources", len(sections), len(image_sources))
|
||||
|
||||
# ---- parse images ----------------------------------------------------------
|
||||
images = word.extract_images(images_dir)
|
||||
logger.info("Found %d images in document", len(images))
|
||||
|
||||
image_analysis: list[dict] = []
|
||||
|
||||
if images:
|
||||
llm = ImageParser()
|
||||
for i, img in enumerate(images):
|
||||
logger.info("[image %d/%d] rid=%s", i + 1, len(images), img["rid"])
|
||||
if dry_run:
|
||||
est = LLMClient.estimate_image_tokens()
|
||||
logger.info(" [DRY RUN] would call vision LLM (~%d tokens)", est)
|
||||
result = {"type": "other", "description": "[DRY RUN]"}
|
||||
else:
|
||||
result = llm.parse_image(img["path"])
|
||||
if result is None:
|
||||
result = {"type": "other", "description": ""}
|
||||
result["rid"] = img["rid"]
|
||||
result["path"] = img["path"]
|
||||
image_analysis.append(result)
|
||||
if i < len(images) - 1:
|
||||
time.sleep(RATE_LIMIT_DELAY)
|
||||
|
||||
usg = llm.usage
|
||||
logger.info("Tokens: %d prompt + %d completion = %d total",
|
||||
usg["prompt_tokens"], usg["completion_tokens"], usg["total_tokens"])
|
||||
else:
|
||||
logger.info("No images found in document")
|
||||
|
||||
# ---- build output --------------------------------------------------------
|
||||
output = {
|
||||
"source": os.path.abspath(docx_path),
|
||||
"sections": sections,
|
||||
"image_sources": image_sources,
|
||||
"image_analysis": image_analysis,
|
||||
}
|
||||
|
||||
parsed_path = os.path.join(output_dir, f"{basename}_parsed.json")
|
||||
with open(parsed_path, "w", encoding="utf-8") as f:
|
||||
json.dump(output, f, ensure_ascii=False, indent=2)
|
||||
logger.info("Saved: %s", parsed_path)
|
||||
|
||||
return output
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Parse a .docx file: extract text structure and parse images.",
|
||||
)
|
||||
parser.add_argument("input", metavar="input.docx", help="Path to the Word document")
|
||||
parser.add_argument("output_dir", nargs="?", default="output", metavar="output_dir",
|
||||
help="Directory for output files (default: output/)")
|
||||
parser.add_argument("--dry-run", action="store_true",
|
||||
help="Print LLM prompts without calling the API.")
|
||||
|
||||
args = parser.parse_args()
|
||||
parse_document(args.input, args.output_dir, dry_run=args.dry_run)
|
||||
+123
@@ -0,0 +1,123 @@
|
||||
import base64
|
||||
import logging
|
||||
import os
|
||||
from typing import Optional
|
||||
|
||||
from LLM import LLMClient
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Prompts
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
PROMPT_IMAGE = """请分析这张图片，判断类型并输出文字描述。
|
||||
|
||||
## 判断图片类型
|
||||
|
||||
如果是 **流程图 / 架构图 / 状态图 / 时序图 / 活动图**，详细描述：
|
||||
- 图中所有节点/步骤/状态/组件的名称
|
||||
- 所有连线/箭头/转换关系及其方向
|
||||
- 所有分支条件、判断逻辑和判断结果
|
||||
- 所有文字标注、注释、标签
|
||||
- 图的整体结构和逻辑流程
|
||||
- 如果图片包含多个子图，拆解描述
|
||||
|
||||
如果是 **其他类型**（UI原型图 / 界面截图 / 设计稿 / 手机屏幕截图 / 网页截图等），简要描述图片内容。
|
||||
|
||||
## 输出格式
|
||||
|
||||
**1. 类型标签（单独一行）：**
|
||||
type: <flowchart|architecture|state|sequence|activity|other>
|
||||
|
||||
**2. 文字描述：**
|
||||
该图片的详细文字描述。
|
||||
|
||||
不要输出 ---YAML--- 分隔符或 YAML 内容，不要添加任何额外的解释或问候语。"""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# ImageParser
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class ImageParser:
|
||||
"""Vision LLM wrapper for parsing images (type + description).
|
||||
|
||||
Usage::
|
||||
|
||||
parser = ImageParser()
|
||||
result = parser.parse_image("images/img1.png")
|
||||
"""
|
||||
|
||||
_VALID_TYPES = {"flowchart", "architecture", "state", "sequence", "activity", "text"}
|
||||
|
||||
def __init__(self, llm: LLMClient | None = None):
|
||||
self._llm = llm or LLMClient()
|
||||
|
||||
@property
|
||||
def usage(self) -> dict:
|
||||
return self._llm.usage
|
||||
|
||||
def parse_image(self, image_path: str) -> Optional[dict]:
|
||||
"""Parse an image and return its type and description (no YAML IR).
|
||||
|
||||
Returns ``{type, description}``, or *None* for UI mockups.
|
||||
"""
|
||||
logger.info("Parsing image: %s", image_path)
|
||||
|
||||
with open(image_path, "rb") as f:
|
||||
img_b64 = base64.b64encode(f.read()).decode()
|
||||
mime = self._mime_type(image_path)
|
||||
|
||||
try:
|
||||
content = self._llm.chat(
|
||||
model=LLMClient.IMAGE_MODEL,
|
||||
messages=[{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "image_url", "image_url": {"url": f"data:{mime};base64,{img_b64}"}},
|
||||
{"type": "text", "text": PROMPT_IMAGE},
|
||||
],
|
||||
}],
|
||||
)
|
||||
except RuntimeError as e:
|
||||
logger.error(str(e))
|
||||
return {"type": "other", "description": "", "error": str(e)}
|
||||
|
||||
parsed = self._parse_type_and_description(content)
|
||||
if parsed is None:
|
||||
return None
|
||||
return {"type": parsed[0], "description": parsed[1]}
|
||||
|
||||
# ---- internals ----------------------------------------------------------
|
||||
|
||||
def _parse_type_and_description(self, content: str) -> Optional[tuple[str, str]]:
|
||||
"""Extract ``(type, description)`` from LLM response.
|
||||
|
||||
Returns *None* for ``[[UI]]`` (skip).
|
||||
"""
|
||||
content = content.strip()
|
||||
if content == "[[UI]]" or content.startswith("[[UI]]"):
|
||||
return None
|
||||
|
||||
parsed_type = "other"
|
||||
desc_lines: list[str] = []
|
||||
for line in content.splitlines():
|
||||
stripped = line.strip()
|
||||
if (stripped.startswith("type:") or stripped.startswith("类型:")) and parsed_type == "other":
|
||||
type_val = stripped.split(":", 1)[1].strip().lower()
|
||||
if type_val in self._VALID_TYPES:
|
||||
parsed_type = type_val
|
||||
else:
|
||||
desc_lines.append(line)
|
||||
|
||||
return parsed_type, "\n".join(desc_lines).strip()
|
||||
|
||||
@staticmethod
|
||||
def _mime_type(image_path: str) -> str:
|
||||
ext = os.path.splitext(image_path)[1].lstrip(".").lower()
|
||||
return {
|
||||
"png": "image/png", "jpg": "image/jpeg", "jpeg": "image/jpeg",
|
||||
"gif": "image/gif", "bmp": "image/bmp",
|
||||
"webp": "image/webp", "svg": "image/svg+xml", "tiff": "image/tiff",
|
||||
}.get(ext, "image/png")
|
||||
+239
@@ -0,0 +1,239 @@
|
||||
import logging
|
||||
import os
|
||||
|
||||
from docx import Document
|
||||
from docx.table import Table
|
||||
from docx.text.paragraph import Paragraph
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
IMAGE_EXT = {
|
||||
"image/png": ".png",
|
||||
"image/jpeg": ".jpg",
|
||||
"image/gif": ".gif",
|
||||
"image/bmp": ".bmp",
|
||||
"image/tiff": ".tiff",
|
||||
"image/webp": ".webp",
|
||||
"image/x-emf": ".emf",
|
||||
"image/x-wmf": ".wmf",
|
||||
"image/svg+xml": ".svg",
|
||||
}
|
||||
|
||||
|
||||
class WordParser:
|
||||
"""Parse a .docx file — extract images, split body into sections.
|
||||
|
||||
Usage::
|
||||
|
||||
parser = WordParser("doc.docx")
|
||||
parser.extract_images("images/")
|
||||
sections, image_sources = parser.extract_sections()
|
||||
"""
|
||||
|
||||
HEADER_CELL_MAX_LEN = 20 # max chars per cell to treat first row as header
|
||||
|
||||
WML_NS = "http://schemas.openxmlformats.org/wordprocessingml/2006/main"
|
||||
DRAW_NS = "http://schemas.openxmlformats.org/drawingml/2006/main"
|
||||
REL_NS = "http://schemas.openxmlformats.org/officeDocument/2006/relationships"
|
||||
|
||||
def __init__(self, docx_path: str):
|
||||
if not os.path.isfile(docx_path):
|
||||
raise FileNotFoundError(f"Document not found: {docx_path}")
|
||||
self._doc = Document(docx_path)
|
||||
|
||||
# ---- public API ---------------------------------------------------------
|
||||
|
||||
def extract_images(self, images_dir: str) -> list[dict]:
|
||||
"""Save all images to *images_dir*. Returns ``[{rid, path}, ...]``."""
|
||||
os.makedirs(images_dir, exist_ok=True)
|
||||
images: list[dict] = []
|
||||
for rel in self._doc.part.rels.values():
|
||||
if "image" not in rel.reltype:
|
||||
continue
|
||||
ext = IMAGE_EXT.get(rel.target_part.content_type, ".png")
|
||||
name = f"image_{rel.rId}{ext}"
|
||||
path = os.path.join(images_dir, name)
|
||||
with open(path, "wb") as f:
|
||||
f.write(rel.target_part.blob)
|
||||
images.append({"rid": rel.rId, "path": path})
|
||||
return images
|
||||
|
||||
def extract_sections(self) -> tuple[list[dict], dict[str, dict]]:
|
||||
"""Walk document body and split into sections by heading.
|
||||
|
||||
Returns:
|
||||
*sections* — ``[{source, blocks, images}, ...]``
|
||||
Each block is ``{type, index, text}`` (paragraph) or
|
||||
``{type, table, headers, rows}`` (table).
|
||||
*image_sources* — ``rid → {section, table?, row?, column?, name?}``
|
||||
"""
|
||||
sections: list[dict] = []
|
||||
current_source = ""
|
||||
blocks: list[dict] = []
|
||||
section_images: list[str] = []
|
||||
image_sources: dict[str, dict] = {}
|
||||
para_idx = 0
|
||||
tbl_idx = 0
|
||||
|
||||
for child in self._doc.element.body:
|
||||
tag = child.tag.split("}")[-1] if "}" in child.tag else child.tag
|
||||
|
||||
if tag == "p":
|
||||
para = Paragraph(child, self._doc)
|
||||
|
||||
if self._heading_level(para) is not None:
|
||||
heading_text = para.text.strip()
|
||||
if heading_text: # ignore empty heading-like paragraphs
|
||||
if blocks or section_images:
|
||||
sections.append({
|
||||
"source": current_source,
|
||||
"blocks": blocks,
|
||||
"images": list(section_images),
|
||||
})
|
||||
blocks = []
|
||||
section_images = []
|
||||
para_idx = 0
|
||||
tbl_idx = 0
|
||||
current_source = heading_text
|
||||
continue
|
||||
|
||||
text = para.text.strip()
|
||||
|
||||
# Scan for images — append [[IMAGE:rid]] markers
|
||||
for run in para.runs:
|
||||
for rid in self._images_in(run._element):
|
||||
text += f" [[IMAGE:{rid}]]"
|
||||
section_images.append(rid)
|
||||
image_sources[rid] = {"section": current_source}
|
||||
|
||||
if text.strip():
|
||||
blocks.append({"type": "para", "index": para_idx + 1, "text": text.strip()})
|
||||
para_idx += 1
|
||||
|
||||
elif tag == "tbl":
|
||||
tbl_idx += 1
|
||||
table = Table(child, self._doc)
|
||||
|
||||
# Collect all rows as [[cell_text, ...], ...]
|
||||
all_rows: list[list[str]] = []
|
||||
all_images: list[list[list[str]]] = [] # row → col → [rids]
|
||||
for row in table.rows:
|
||||
row_texts: list[str] = []
|
||||
row_cell_images: list[list[str]] = []
|
||||
for cell in row.cells:
|
||||
cell_text = cell.text.strip()
|
||||
cell_imgs: list[str] = []
|
||||
for cp in cell.paragraphs:
|
||||
for run in cp.runs:
|
||||
for rid in self._images_in(run._element):
|
||||
cell_imgs.append(rid)
|
||||
# Replace images with markers in text
|
||||
for rid in cell_imgs:
|
||||
cell_text += f" [[IMAGE:{rid}]]"
|
||||
section_images.append(rid)
|
||||
row_texts.append(cell_text.strip())
|
||||
row_cell_images.append(cell_imgs)
|
||||
if any(row_texts) or any(row_cell_images):
|
||||
all_rows.append(row_texts)
|
||||
all_images.append(row_cell_images)
|
||||
|
||||
if len(all_rows) >= 2:
|
||||
# Heuristic: first row is a header if every cell is short
|
||||
first_row = all_rows[0]
|
||||
has_header = all(len(c) < self.HEADER_CELL_MAX_LEN for c in first_row)
|
||||
if has_header:
|
||||
headers = first_row
|
||||
data_rows_slice = zip(all_rows[1:], all_images[1:])
|
||||
else:
|
||||
headers = [f"列{ci + 1}" for ci in range(len(first_row))]
|
||||
data_rows_slice = zip(all_rows, all_images)
|
||||
|
||||
data_rows: list[dict] = []
|
||||
|
||||
for ri, (row_data, row_imgs) in enumerate(data_rows_slice):
|
||||
columns: list[dict] = []
|
||||
max_cols = max(len(headers), len(row_data))
|
||||
for ci in range(max_cols):
|
||||
hdr = headers[ci] if ci < len(headers) else ""
|
||||
txt = row_data[ci] if ci < len(row_data) else ""
|
||||
columns.append({
|
||||
"name": hdr,
|
||||
"row": ri + 1,
|
||||
"col": ci + 1,
|
||||
"text": txt,
|
||||
})
|
||||
|
||||
# Register image sources with structured location
|
||||
imgs = row_imgs[ci] if ci < len(row_imgs) else []
|
||||
for rid in imgs:
|
||||
image_sources[rid] = {
|
||||
"section": current_source,
|
||||
"table": tbl_idx,
|
||||
"row": ri + 1,
|
||||
"column": ci + 1,
|
||||
"name": hdr,
|
||||
}
|
||||
|
||||
data_rows.append({"columns": columns})
|
||||
|
||||
blocks.append({
|
||||
"type": "table",
|
||||
"table": tbl_idx,
|
||||
"headers": headers,
|
||||
"rows": data_rows,
|
||||
})
|
||||
elif all_rows:
|
||||
# Degenerate table (only header or single row) — treat as plain rows
|
||||
for ri, row_data in enumerate(all_rows):
|
||||
row_text = " | ".join(row_data)
|
||||
if row_text.strip():
|
||||
blocks.append({
|
||||
"type": "para",
|
||||
"index": para_idx + 1,
|
||||
"text": row_text,
|
||||
})
|
||||
para_idx += 1
|
||||
|
||||
if blocks or section_images:
|
||||
sections.append({
|
||||
"source": current_source,
|
||||
"blocks": blocks,
|
||||
"images": list(section_images),
|
||||
})
|
||||
|
||||
return sections, image_sources
|
||||
|
||||
# ---- internals ----------------------------------------------------------
|
||||
|
||||
def _heading_level(self, para: Paragraph) -> int | None:
|
||||
"""Heading level 1-9, or *None* if not a heading."""
|
||||
if para.style and para.style.name:
|
||||
name = para.style.name
|
||||
for prefix in ("Heading", "标题"):
|
||||
if name.startswith(prefix):
|
||||
try:
|
||||
return int(name.split()[-1])
|
||||
except (ValueError, IndexError):
|
||||
pass
|
||||
pPr = para._element.find(f"{{{self.WML_NS}}}pPr")
|
||||
if pPr is not None:
|
||||
ol = pPr.find(f"{{{self.WML_NS}}}outlineLvl")
|
||||
if ol is not None:
|
||||
val = ol.get(f"{{{self.WML_NS}}}val")
|
||||
if val is not None:
|
||||
try:
|
||||
return int(val) + 1
|
||||
except ValueError:
|
||||
pass
|
||||
return None
|
||||
|
||||
def _images_in(self, element) -> list[str]:
|
||||
"""Return rId values for drawings embedded in *element*."""
|
||||
rids: list[str] = []
|
||||
for drawing in element.findall(f".//{{{self.WML_NS}}}drawing"):
|
||||
blip = drawing.find(f".//{{{self.DRAW_NS}}}blip")
|
||||
if blip is not None:
|
||||
rid = blip.get(f"{{{self.REL_NS}}}embed")
|
||||
if rid:
|
||||
rids.append(rid)
|
||||
return rids
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
name: ir_generation_skill
|
||||
description: 通过自然语言描述，向JSON文件中添加以IR表示的功能点。
|
||||
---
|
||||
|
||||
# add_ir
|
||||
|
||||
## 概述
|
||||
|
||||
通过自然语言描述由大语言模型自动转换为标准IR格式并写入到JSON文件或追加到JSON文件中。
|
||||
|
||||
## 用法
|
||||
|
||||
- **命令行传入**：直接传入功能的文字描述
|
||||
- **文件传入**：通过 `-f` 参数从文本文件读取描述
|
||||
- **追加模式**：新生成的功能点追加到已有 `_ir.json` 末尾，不影响已有条目
|
||||
|
||||
## 输入要求
|
||||
|
||||
- 功能的自然语言描述（命令行参数或 `-f` 文件）
|
||||
- 目标IR JSON文件路径（`--ir`，默认 `output/ir.json`）
|
||||
|
||||
## 输出
|
||||
|
||||
在目标IR JSON文件中追加新的功能点条目，每个条目包含：
|
||||
- `function`：功能名称
|
||||
- `source.section`：所属章节
|
||||
- `source.location`：位置描述
|
||||
- `trigger`：触发条件（可选）
|
||||
- `actions`：执行动作（可选）
|
||||
|
||||
## 使用提示
|
||||
|
||||
描述中应尽量包含功能的触发条件、执行动作和所属章节信息，以提高转换质量。
|
||||
@@ -0,0 +1,105 @@
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
from typing import Optional
|
||||
|
||||
from openai import OpenAI
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class LLMClient:
|
||||
"""Low-level OpenAI-compatible LLM client with retry and token tracking.
|
||||
|
||||
Usage::
|
||||
|
||||
llm = LLMClient()
|
||||
content = llm.chat("qwen3.5-flash", [{"role": "user", "content": "Hello"}])
|
||||
print(llm.usage)
|
||||
"""
|
||||
|
||||
IMAGE_MODEL = "qwen3-vl-plus"
|
||||
TEXT_MODEL = "qwen3.5-flash-2026-02-23"
|
||||
TIMEOUT = 120
|
||||
MAX_RETRIES = 3
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
base_url: str = "https://dashscope.aliyuncs.com/compatible-mode/v1",
|
||||
timeout: int | None = None,
|
||||
):
|
||||
key = os.environ.get("DASHSCOPE_API_KEY", "")
|
||||
if not key:
|
||||
raise ValueError("DASHSCOPE_API_KEY environment variable is not set.")
|
||||
self._client = OpenAI(api_key=key, base_url=base_url)
|
||||
self._timeout = timeout or self.TIMEOUT
|
||||
self._prompt_tokens = 0
|
||||
self._completion_tokens = 0
|
||||
|
||||
@property
|
||||
def usage(self) -> dict:
|
||||
"""Return accumulated token counts as ``{prompt, completion, total}``."""
|
||||
return {
|
||||
"prompt_tokens": self._prompt_tokens,
|
||||
"completion_tokens": self._completion_tokens,
|
||||
"total_tokens": self._prompt_tokens + self._completion_tokens,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def estimate_tokens(text: str) -> int:
|
||||
"""Quick token estimate. CJK ≈1.7/token, others ≈3.0/token."""
|
||||
cjk = sum(1 for c in text if '一' <= c <= '鿿' or ' ' <= c <= '〿')
|
||||
other = len(text) - cjk
|
||||
return max(1, int(cjk / 1.7 + other / 3.0))
|
||||
|
||||
@staticmethod
|
||||
def estimate_image_tokens() -> int:
|
||||
"""Fixed estimate for one vision-model image (~500 tokens)."""
|
||||
return 500
|
||||
|
||||
def chat(
|
||||
self, model: str, messages: list[dict], *, timeout: int | None = None,
|
||||
response_format: dict | None = None,
|
||||
) -> str:
|
||||
"""Send a chat completion request and return the response content.
|
||||
|
||||
Automatically retries on failure and accumulates token usage.
|
||||
"""
|
||||
label = f"chat({model})"
|
||||
|
||||
def _call():
|
||||
t0 = time.time()
|
||||
kwargs = dict(model=model, messages=messages, timeout=timeout or self._timeout)
|
||||
if response_format is not None:
|
||||
kwargs["response_format"] = response_format
|
||||
kwargs["temperature"] = 0
|
||||
resp = self._client.chat.completions.create(**kwargs)
|
||||
content = resp.choices[0].message.content
|
||||
usg = resp.usage
|
||||
if usg:
|
||||
self._prompt_tokens += usg.prompt_tokens
|
||||
self._completion_tokens += usg.completion_tokens
|
||||
elapsed = time.time() - t0
|
||||
logger.info("%s: %d chars in %.1fs", label, len(content) if content else 0, elapsed)
|
||||
if not content:
|
||||
raise RuntimeError("Empty response from LLM")
|
||||
return content
|
||||
|
||||
return self._retry(_call, label)
|
||||
|
||||
def _retry(self, fn, label: str) -> str:
|
||||
"""Call *fn()* with exponential-backoff retry."""
|
||||
last_error: Optional[Exception] = None
|
||||
for attempt in range(self.MAX_RETRIES):
|
||||
try:
|
||||
return fn()
|
||||
except Exception as e:
|
||||
last_error = e
|
||||
logger.warning(
|
||||
"%s error (attempt %d/%d): %s",
|
||||
label, attempt + 1, self.MAX_RETRIES, e,
|
||||
)
|
||||
if attempt < self.MAX_RETRIES - 1:
|
||||
time.sleep(2 ** attempt)
|
||||
raise RuntimeError(f"{label}: all retries exhausted") from last_error
|
||||
+191
@@ -0,0 +1,191 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Add missing features to an existing ``_ir.json``.
|
||||
|
||||
Converts a natural-language description of missing features into structured
|
||||
IR entries via LLM, then appends them to the IR file.
|
||||
|
||||
Usage::
|
||||
|
||||
python scripts/add_missing.py "遗漏功能描述" --ir output/<basename>_ir.json
|
||||
python scripts/add_missing.py -f missing.txt --ir output/<basename>_ir.json
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
|
||||
from LLM import LLMClient
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s [%(levelname)s] %(message)s",
|
||||
datefmt="%Y-%m-%d %H:%M:%S",
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Prompt
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
PROMPT = """你是一个需求文档分析助手。用户发现已生成的功能点列表中有遗漏，请将以下描述中提到的遗漏功能点转换为标准JSON格式。
|
||||
|
||||
## 遗漏功能描述
|
||||
|
||||
{description}
|
||||
|
||||
## JSON格式要求
|
||||
|
||||
每个功能点输出为：
|
||||
|
||||
{
|
||||
"function": "功能名称",
|
||||
"source": {
|
||||
"section": "章节名（如果描述中提到了章节信息则填写，否则填"未知"）",
|
||||
"location": "原文位置描述（如果描述中提供了则填写，否则填"用户补充"）"
|
||||
},
|
||||
"trigger": {
|
||||
"type": "AND或者OR",
|
||||
"conditions": [
|
||||
"触发条件1",
|
||||
"触发条件2"
|
||||
]
|
||||
},
|
||||
"actions": {
|
||||
"场景/角色": [
|
||||
"动作1",
|
||||
"动作2"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
## 输出原则
|
||||
|
||||
1. 将描述中的每个独立功能点拆分为一个独立的JSON对象
|
||||
2. 如果描述中包含多个功能点，输出一个JSON数组
|
||||
3. 没有 trigger 或 actions 的字段直接**省略**，不要写 null 或空列表/空对象
|
||||
4. 尽量从描述中推断 source.section 和 source.location
|
||||
5. 功能名称应简洁明确，概括该功能的核心行为
|
||||
6. 直接输出纯JSON，不要用 ```json 代码块包裹
|
||||
"""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _parse_llm_response(raw: str) -> list | dict | str | None:
|
||||
"""Parse JSON from LLM response, handling markdown code fences."""
|
||||
if raw is None:
|
||||
return None
|
||||
stripped = raw.strip()
|
||||
if stripped.startswith("```"):
|
||||
nl = stripped.find("\n")
|
||||
stripped = stripped[nl + 1:] if nl != -1 else stripped[3:]
|
||||
if stripped.endswith("```"):
|
||||
stripped = stripped[:-3]
|
||||
try:
|
||||
return json.loads(stripped)
|
||||
except json.JSONDecodeError:
|
||||
logger.warning("Failed to parse JSON, returning raw text")
|
||||
return raw
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Core
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def add_missing(description: str, ir_path: str) -> list[dict]:
|
||||
"""Convert missing feature description to IR entries and append to IR file.
|
||||
|
||||
Returns the newly added entries.
|
||||
"""
|
||||
llm = LLMClient()
|
||||
|
||||
prompt = PROMPT.replace("{description}", description)
|
||||
|
||||
logger.info("Sending missing features to LLM...")
|
||||
try:
|
||||
raw = llm.chat(
|
||||
model=LLMClient.TEXT_MODEL,
|
||||
messages=[{"role": "user", "content": prompt}],
|
||||
response_format={"type": "json_object"},
|
||||
)
|
||||
logger.info("Response: %d chars", len(raw))
|
||||
except RuntimeError as e:
|
||||
logger.error("LLM call failed: %s", e)
|
||||
sys.exit(1)
|
||||
|
||||
parsed = _parse_llm_response(raw)
|
||||
if isinstance(parsed, list):
|
||||
new_entries = parsed
|
||||
elif isinstance(parsed, dict):
|
||||
new_entries = [parsed]
|
||||
else:
|
||||
logger.error("Unparseable response")
|
||||
sys.exit(1)
|
||||
|
||||
# Load existing IR or start fresh
|
||||
if os.path.exists(ir_path):
|
||||
with open(ir_path, "r", encoding="utf-8") as f:
|
||||
existing = json.load(f)
|
||||
logger.info("Loaded existing IR: %d entries", len(existing))
|
||||
else:
|
||||
existing = []
|
||||
logger.info("No existing IR file, creating new")
|
||||
|
||||
existing.extend(new_entries)
|
||||
|
||||
# Write back
|
||||
os.makedirs(os.path.dirname(ir_path) or ".", exist_ok=True)
|
||||
with open(ir_path, "w", encoding="utf-8") as f:
|
||||
json.dump(existing, f, ensure_ascii=False, indent=2)
|
||||
|
||||
logger.info("Saved: %s (%d entries, +%d new)", ir_path, len(existing), len(new_entries))
|
||||
|
||||
usg = llm.usage
|
||||
logger.info("Tokens: %d prompt + %d completion = %d total",
|
||||
usg["prompt_tokens"], usg["completion_tokens"], usg["total_tokens"])
|
||||
|
||||
return new_entries
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Add missing features to an IR JSON file via LLM.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"description", nargs="?", metavar="description",
|
||||
help="Natural-language description of missing features",
|
||||
)
|
||||
parser.add_argument(
|
||||
"-f", "--file", metavar="file.txt",
|
||||
help="Read description from a file instead of command line",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--ir", default="output/ir.json", metavar="ir.json",
|
||||
help="Path to the IR JSON file (default: output/ir.json)",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.file:
|
||||
with open(args.file, "r", encoding="utf-8") as f:
|
||||
description = f.read().strip()
|
||||
elif args.description:
|
||||
description = args.description
|
||||
else:
|
||||
parser.error("Either provide a description string or use -f to read from a file")
|
||||
|
||||
if not description:
|
||||
parser.error("Description is empty")
|
||||
|
||||
add_missing(description, args.ir)
|
||||
+42
@@ -0,0 +1,42 @@
|
||||
---
|
||||
name: 解决方案应用技能
|
||||
description: 应用用户提供的冲突解决方案以创建合并了更正的更新文档表示。
|
||||
---
|
||||
|
||||
# 解决方案应用技能
|
||||
|
||||
## 概述
|
||||
|
||||
此技能采用用户提供的冲突解决方案并将它们应用到解析的文档中，创建一个协调文本和视觉内容差异的校正版本。它生成包含应用更正的更新文档表示。
|
||||
|
||||
## 功能
|
||||
|
||||
该技能：
|
||||
- 接受用户提供的冲突解决方案决策
|
||||
- 支持多种解决方案类型："以图像为准"、"以文字为准"、"两处都保留"或自定义文本
|
||||
- 更新解析的文档结构以合并解决方案决策
|
||||
- 创建文档表示的校正版本，包含应用的更改
|
||||
- 维护所有应用更正的源可追溯性
|
||||
- 向输出添加包含更正指令的resolved_conflicts数组
|
||||
|
||||
## 输入要求
|
||||
|
||||
- 解析文档JSON文件的路径（带有已识别的冲突）
|
||||
- 包含用户决策的解决方案JSON文件的路径
|
||||
- 可选输出目录规范
|
||||
- 解决方案JSON应包含具有以下内容的对象：
|
||||
- `conflict_id`：冲突数组中的冲突索引
|
||||
- `resolution`：决策类型（"以图片为准"、"以文字为准"、"两处都保留"）或自定义文本
|
||||
- `custom_text`：解决方案的可选自定义文本
|
||||
|
||||
## 输出
|
||||
|
||||
该技能生成一个结构化JSON文件，文件名为输入文档的基本名称后跟'_updated.json'，包含：
|
||||
- 包含应用更正的原始文档结构
|
||||
- 详细说明应用更改的`resolved_conflicts`数组
|
||||
- 关于每个冲突类型和应用更正的信息
|
||||
- 用于可追溯性的源跟踪信息
|
||||
|
||||
## 集成点
|
||||
|
||||
此技能消耗冲突检测技能的输出（带冲突的文档）和用户提供的解决方案。其输出被IR生成技能使用以创建最终结构化表示。
|
||||
Executable
+144
@@ -0,0 +1,144 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Apply user resolutions to ``_parsed.json`` using ``_conflicts.json``.
|
||||
|
||||
Usage::
|
||||
|
||||
python scripts/apply_resolutions.py <parsed.json> --resolutions <resolutions.json> [--output-dir DIR]
|
||||
|
||||
The *resolutions.json* file is created by the agent after user arbitration.
|
||||
Each resolution maps a conflict_id to a decision.
|
||||
|
||||
Resolution format (``resolutions.json``)::
|
||||
|
||||
[
|
||||
{
|
||||
"conflict_id": 0, // 0-based index into conflicts array
|
||||
"resolution": "以文字为准",
|
||||
"custom_text": null
|
||||
}
|
||||
]
|
||||
|
||||
Outputs ``<basename>_updated.json`` — identical to *parsed.json* plus a
|
||||
``resolved_conflicts`` top-level array with correction instructions for the IR generator.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s [%(levelname)s] %(message)s",
|
||||
datefmt="%Y-%m-%d %H:%M:%S",
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def apply_resolutions(
|
||||
parsed_path: str,
|
||||
resolutions_path: str,
|
||||
output_dir: str | None = None,
|
||||
) -> dict:
|
||||
"""Load *parsed.json*, apply resolutions, write *updated.json*."""
|
||||
with open(parsed_path, "r", encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
|
||||
with open(resolutions_path, "r", encoding="utf-8") as f:
|
||||
resolutions = json.load(f)
|
||||
|
||||
base_dir = os.path.dirname(os.path.abspath(parsed_path))
|
||||
|
||||
# Try to find _conflicts.json alongside parsed.json
|
||||
basename = os.path.splitext(os.path.basename(parsed_path))[0]
|
||||
stem = basename[:-7] if basename.endswith("_parsed") else basename
|
||||
candidate = os.path.join(base_dir, f"{stem}_conflicts.json")
|
||||
|
||||
conflicts = data.get("_conflicts", [])
|
||||
if not conflicts and os.path.isfile(candidate):
|
||||
with open(candidate, "r", encoding="utf-8") as f:
|
||||
conflicts = json.load(f)
|
||||
|
||||
if output_dir is None:
|
||||
output_dir = base_dir
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
# Build resolved_conflicts with correction instructions for ir_generator
|
||||
resolved = []
|
||||
for res in resolutions:
|
||||
cid = res.get("conflict_id")
|
||||
if cid is None or cid < 0 or cid >= len(conflicts):
|
||||
logger.warning("Invalid conflict_id: %s", cid)
|
||||
continue
|
||||
|
||||
conflict = conflicts[cid]
|
||||
choice = res.get("resolution", "")
|
||||
custom = res.get("custom_text")
|
||||
|
||||
entry = {
|
||||
"conflict_id": cid,
|
||||
"conflict_type": conflict.get("conflict_type"),
|
||||
"section": conflict.get("section", ""),
|
||||
"resolution": choice,
|
||||
}
|
||||
|
||||
# Build a correction instruction string
|
||||
image_val = conflict.get("image_snippet", "")
|
||||
text_val = conflict.get("text_snippet", "")
|
||||
if choice == "以图片为准":
|
||||
entry["correction"] = image_val
|
||||
entry["source"] = "图片"
|
||||
elif choice == "以文字为准":
|
||||
entry["correction"] = text_val
|
||||
entry["source"] = "文字"
|
||||
elif choice == "两处都保留":
|
||||
entry["correction"] = f"{text_val}（另外的观点：{image_val}）"
|
||||
entry["source"] = "两者兼容"
|
||||
elif custom:
|
||||
entry["correction"] = custom
|
||||
entry["source"] = "自定义"
|
||||
logger.info("Conflict %d: custom: %s", cid, custom[:60])
|
||||
else:
|
||||
entry["correction"] = text_val
|
||||
entry["source"] = "文字（默认）"
|
||||
logger.warning("Conflict %d: unknown resolution '%s', defaulting to text", cid, choice)
|
||||
|
||||
logger.info("Conflict %d: %s → %s", cid, choice, entry["source"])
|
||||
|
||||
resolved.append(entry)
|
||||
|
||||
data["resolved_conflicts"] = resolved
|
||||
logger.info("Applied %d resolutions", len(resolved))
|
||||
|
||||
# Write output
|
||||
if basename.endswith("_parsed"):
|
||||
out_name = f"{stem}_updated.json"
|
||||
else:
|
||||
out_name = f"{basename}_updated.json"
|
||||
|
||||
output_path = os.path.join(output_dir, out_name)
|
||||
with open(output_path, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, ensure_ascii=False, indent=2)
|
||||
|
||||
logger.info("Saved: %s", output_path)
|
||||
return data
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Apply user resolutions to parsed.json.",
|
||||
)
|
||||
parser.add_argument("input", metavar="parsed.json",
|
||||
help="Path to _parsed.json")
|
||||
parser.add_argument("--resolutions", "-r", required=True,
|
||||
help="Path to resolutions JSON file")
|
||||
parser.add_argument("--output-dir", default=None,
|
||||
help="Output directory (default: same as input)")
|
||||
|
||||
args = parser.parse_args()
|
||||
apply_resolutions(args.input, args.resolutions, args.output_dir)
|
||||
Reference in New Issue
Block a user