diff --git a/.cursor/rules/progress-and-planning.mdc b/.cursor/rules/progress-and-planning.mdc index b561050..aded4c1 100644 --- a/.cursor/rules/progress-and-planning.mdc +++ b/.cursor/rules/progress-and-planning.mdc @@ -1,14 +1,14 @@ --- -description: 每轮工作同步维护开发进度与规划方案 +description: 实际开发与工程验证维护进度，已确定的方案调整维护规划，普通解释问答不记录 alwaysApply: true --- # 进度与规划维护 -- 适用本项目每轮开发、修复、验证、讨论、澄清和方案调整。 -- 开始时读取 `docs/开发进度.md` 当前状态及最新记录，核对 `docs/整体方案-v0.2.md` 的近期实施规划。 -- 结束前，在 `docs/开发进度.md` 追加本轮记录，写明日期、目标、实际完成、涉及文件、验证结果、未完成项及提交/推送状态；无代码变更时明确说明。 -- 优化建议、设计决定、优先级和后续实施步骤更新到 `docs/整体方案-v0.2.md`，保持唯一当前规划；调整现有条目，不另建互相冲突的计划。 +- 适用本项目实际开发、修复、工程验证及已确定的方案调整。单纯概念解释、使用问答和一般澄清不写进进度或规划，也不因解释顺带新增规划。 +- 开展上述实际工作时，读取 `docs/开发进度.md` 当前状态及最新记录，核对 `docs/整体方案-v0.2.md` 的近期实施规划。 +- 实际开发、修复或工程验证结束前，在 `docs/开发进度.md` 追加记录，写明日期、目标、实际完成、涉及文件、验证结果、未完成项及提交/推送状态；无代码变更时明确说明。仅调整记录规则或清理解释记录时，不为此另写进度条目。 +- 已确定的优化、设计决定、优先级和后续实施步骤更新到 `docs/整体方案-v0.2.md`，保持唯一当前规划；调整现有条目，不另建互相冲突的计划。一般讨论中的举例或建议不自动成为实施计划。 - 进度记录引用方案中的下一步，避免复制维护两份完整待办；新事实改变旧计划时，注明调整原因和影响。 - 历史记录保留，新记录可更正旧结论，不能用新结果抹掉失败或未完成状态。 - 区分已实现、静态检查、模拟测试、真实浏览器验证、真实平台验证、双端验证和正式发布；写明样本及环境边界，不把阶段测试次数累加或当作当前重跑结果。 diff --git a/.env.example b/.env.example index 3f7ce52..73e6347 100644 --- a/.env.example +++ b/.env.example @@ -13,3 +13,14 @@ ASSISTANT_LEARNING_HOME=.runtime/learning # 默认使用 Playwright Chromium；可在本机开发时改为已安装的 msedge 或 chrome。 ASSISTANT_BROWSER_CHANNEL=chromium ASSISTANT_BROWSER_HEADLESS=0 + +# 需求分析仅由服务端调用；真实 key 填到忽略的 .env，不使用 VITE_ 前缀。 +# 型号和 Key 独立配置：备用 Key 只在受限错误时切换，不自动升级模型。 +ASSISTANT_MODEL_PROVIDER=deepseek +ASSISTANT_MODEL_BASE_URL=https://api.deepseek.com/v1 +ASSISTANT_MODEL_NAME=deepseek-flash +ASSISTANT_MODEL_API_KEY= +ASSISTANT_MODEL_FALLBACK_API_KEY= +ASSISTANT_MODEL_THINKING=disabled +ASSISTANT_MODEL_TIMEOUT_SECONDS=30 +ASSISTANT_MODEL_MAX_OUTPUT_TOKENS=4096 diff --git a/AGENTS.md b/AGENTS.md index 160941f..e20e878 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,7 +1,7 @@ # assistant 项目协作约定 -开始本项目工作时，读取 [开发进度](docs/开发进度.md) 的当前状态与最新记录，以及 [整体方案](docs/整体方案-v0.2.md) 的当前实施规划。 +开始本项目实际开发、修复或工程验证时，读取 [开发进度](docs/开发进度.md) 的当前状态与最新记录，以及 [整体方案](docs/整体方案-v0.2.md) 的当前实施规划。 -每轮工作遵守 [进度与规划维护规则](.cursor/rules/progress-and-planning.mdc)：结束前将本轮实际进展写入进度文档，将优化、设计决定与实施顺序更新到规划方案。纯讨论、澄清和验证轮次也要保留简明记录。 +实际开发、修复和工程验证遵守 [进度与规划维护规则](.cursor/rules/progress-and-planning.mdc)：结束前将实际进展写入进度文档，将已确定的优化、设计决定与实施顺序更新到规划方案。单纯概念解释、使用问答和一般澄清不写入这两类文档，也不因解释顺带新增规划。 保护共享工作区已有改动。学习端首期由维护者与本地 Codex 协作学习；实际平台能力、测试、发布与交付状态必须按证据分别报告。 diff --git a/README.md b/README.md index 10c38b4..b5efb91 100644 --- a/README.md +++ b/README.md @@ -1,32 +1,45 @@ # assistant -面向国内商家的 AI 获客助手，已开始建设本机学习与验证后台。 +面向国内商家的 AI 获客助手，当前优先完成平台链路学习与复用，并按功能模块逐层扩展当前账号网页版可见入口。后台新增“学习 → 链路复用”：选择已学搜索/详情版本，召回锁定摘要，换新关键词执行并核对正文、有限评论和来源，全程不调用业务分析模型。 + +真实小红书以新词“装修顺序”完成独立搜索到详情/首层评论的组合探索：搜索3条、正文1条、评论6条，5批、去重1条、范围截断。旧八步定位失败与首个七步部分结果保留；补录长正文评论范围后冻结新的七步候选，完整复验通过。这是当前可见结果范围内的单样本探索，正式基线、任务后反馈、双端和发布仍待验收。 + +装修初筛、商家条件、需求/咨询意向和人工核对已有实现与历史保留，当前暂停扩展，后置到操作复用闭环之后；后台业务验证不代表桌面端已经完成。当前状态与证据见开发进度，唯一后续顺序见整体方案第14节。 首版桌面端同时适配 Windows 和 macOS，两端按同一业务范围开发和验收。 -产品设想是让商家配置商品与目标客户条件，由软件通过搜索或浏览视频发现相关账号，再结合主页和评论识别潜在需求，整理线索并执行后续跟进任务。 +产品设想是让商家配置商品与目标客户条件，由软件搜索或浏览图文/视频，结合主页和评论识别需求，整理线索，并按配置关注、私信、评论或回复指定评论；也可按用户问题检索信息并返回来源。 + +后台“学习 → 平台路径”（/#/platform-paths）维护功能层级与证据状态；初始33个模板不等于全站总量，当前18项有可核对观察。发布者主页导航通过，评论者主页最终复验在官方安全验证处暂停；菜单/标签/回复与消息窗口仍待补学。入口可见不等于动作或固化通过，动态内容取样，业务分类继续后置。 ## 当前文档 -- [开发进度](docs/开发进度.md)：逐轮记录实际完成、验证、未完成项与提交状态；每轮结束前更新。 -- [AI 获客助手方案图解](docs/方案图解.html)：独立离线 HTML，包含架构、获客、学习发布、AI 额度、任务恢复、登录会话与实施阶段；支持节点查看、路径演示、明暗主题及 SVG、PNG 导出。 +- [开发进度](docs/开发进度.md)：记录实际开发、修复、工程验证及提交状态；普通解释与使用问答不记录。 +- [AI 获客助手方案图解](docs/方案图解.html)：独立离线 HTML，包含架构、获客、学习发布、任务与检索、AI 额度、任务恢复、登录会话与实施阶段；支持节点查看、路径演示、明暗主题及 SVG、PNG 导出。 - [AI 获客助手整体方案](docs/整体方案-v0.2.md)：设计决定、优化方向和实施顺序的当前依据，第 14 节维护近期规划；调整时同步现有方案与图解。 +- [学习端实施方案](docs/整体方案-v0.2.md#511-学习端实施方案与首条任务)：首条小红书范围、分工、六步学习、任务后官方消息观察和开始学习前的待补项。 +- [本机协作学习入口](tools/learning/README.md)：Codex 受限控制操作、浏览器工具、登录检测与人工质量记录。 - [AI 获客助手初步讨论方案](docs/初步讨论方案-v0.1.md)：业务场景、学习固化链路、AI 分工、端形态、收费思路及首版验证范围。 +当前研发主线已收回“链路学习 → 固化 → 轻量召回 → 变参执行”。装修初筛、咨询判断和人工核对保留为后置业务验证，不再继续扩展。维护后台的业务验证不代表桌面端已经完成；本阶段不依赖业务模型识别每篇内容。 + ## 当前路线 -- 使用 Electron 开发 Windows、macOS 双平台桌面端。 -- 以 Playwright 固定执行为主线，Midscene 作为可选辅助；学习后台以 A2A learning 的适用模块为基础裁剪适配，桌面壳另行提取验证。 -- 学习端围绕少量平台操作完成观察取证、结构化整理、候选生成、固定校验、变参试跑和版本发布，运行异常回流维护端修订。 +- 当前按功能模块逐层盘点小红书网页版入口，把可复用操作分别固化，再按平台/任务类型路由；桌面阶段采用 Electron，同时面向 Windows 和 macOS。 +- 已确定学习与桌面共用 Playwright 固定执行、同一版本合同及结果核对，不另建 browser-use `Agent.run` 路径；Midscene 仍是后续可选辅助。学习后台以 A2A learning 的适用模块为基础裁剪适配，桌面壳另行提取验证。 +- 学习端按页面类型和入口来源记录层级、下级控件及返回路径，分别完成观察取证、候选生成、固定校验和变参试跑；运行异常回流维护端修订。 - 首期由开发者与本地 Codex 协作学习：维护者在平台页面登录，Codex 观察、编写参数化实现、提交候选并依据校验和试跑结果修订；后台管理证据与版本。后台内置自主学习 Agent 后续按需求建设。 - 学习后台使用独立 PostgreSQL，检索库准备 `pg_search` 与 `pgvector`；商家桌面端仍按 SQLite 规划。平台账号在官方页面自行登录，每个账号复用专用浏览器资料目录。 - `browser_use` 测试工程的 Electron 桌面壳、产品认证和管理后台作为候选复用基础；云端可沿用其 FastAPI 骨架，本地 SQLite、获客执行与商业模块按当前方案适配或新增，范围见整体方案第 4.4 节。 - 模型由我们在运营后台按文本判断、复杂复核、视觉操作等用途配置，不绑定具体型号。 - 第一条学习链路选定小红书「搜索笔记 → 详情 → 评论读取」，先用本机演示页跑通学习与验证，再在维护者自行登录的平台页面上适配。 +- 新小红书任务默认“有界网页操作（原生通信）”，也可明确选旧模式作对照。新模式只允许已观察唯一搜索输入/回车、固定官网标准笔记链接和普通区域有界读取；15 秒动作窗口内，当前主框架发起的第一方 HTTPS `xhr/fetch POST` 自然通信（包括安全通信）可正常进行。它不是网络沙箱，也不承诺服务端绝无副作用；不直接请求接口、运行任意 JavaScript 或添加指纹伪装。旧模式（legacy）的四类精确请求策略原样保留。 +- 当前浏览器执行为 Playwright，尚未接入 browser-use。平台提醒的识别、暂停与人工恢复需按真实页面补齐；任何浏览器框架都不作为不会被识别或限制账号的保证。 +- 学习探索记录与正式运行模式验证分开：已实现运行前基线、观察窗口、人工追加反馈和候选质量状态；晚到异常保留原记录并将同版本置为待复核。当前只评价维护环境，平台自动提醒识别、真实平台表现与商家发布仍待验证。 ## 项目状态 -当前包含方案文档、学习后台前后端和独立的 Playwright 浏览器工作进程。可以建立维护者账号、创建学习任务、打开浏览器人工登录、观察取证、录制参数化步骤、生成不可变候选以及保存校验和实际试跑记录。第一版操作为搜索、详情和读取，尚未接入 AI 自动探索或完成真实小红书链路验证。 +当前包含方案文档、学习后台前后端和独立的 Playwright 浏览器工作进程。支持可见页面登录检测、人工确认、七类有界动作、动态目标匹配、有界容器目标定位、集合等待、有界分批读取与业务 ID 去重、显式文字/媒体内容、步骤阶段耗时、明确空态合同及不可变候选；逐批截图绑定摘要，失败保留实际证据。小红书图文及有限首层评论已通过两组正式模式执行；真实平台空态、登录失效、回复层级及更多范围仍待补样本。已有任务后人工观察机制及明确复查待办，尚未接入后台 AI 自主探索。 当前服务是本机维护工具；Electron 商家端、macOS 实机验收、能力正式发布、AI 网关和商业计量尚未完成。文档中的价格、额度和成本示例不是最终产品承诺。 @@ -51,18 +64,46 @@ macOS 创建虚拟环境使用 `python3 -m venv .venv`，安装依赖使用 `.ve 打开 [学习工作台](http://127.0.0.1:5179)，空数据库首次启动时自动初始化维护者账号 `admin`，初始密码 `123456`。已有账号时跳过初始化，后续重启不会覆盖账号或密码。密码以 scrypt 摘要保存。API 默认端口为 `8810`。可将 `.env.example` 复制为 `.env` 调整本机配置；使用已安装的 Edge 时设置 `ASSISTANT_BROWSER_CHANNEL=msedge`。 -平台登录在工作台打开的浏览器中直接完成。首次需要人工确认当前身份，固定账号资料目录用于后续复用；目前没有自动核验平台身份的适配器。学习账本保存到 PostgreSQL，浏览器资料与证据文件保存在 `.runtime/learning/`，均不提交 Git。 +[内容初筛](http://127.0.0.1:5179/#/discovery) 使用已学习的四步搜索读取候选和已确认会话，按装修规则初筛，最多保留 3 篇。当前只保存地区条件，未作地域过滤；保留相关同行案例作为评论来源，不据标题确认业主身份或采购意图。 + +打开一条有保留笔记的初筛记录，选择同站的七步详情候选或有界定位八步修订候选，可逐篇点击“读取正文与评论”。使用详情学习任务已确认的会话，未就绪时前往该学习会话处理；当前不自动打开或切换浏览器。八步候选还会显示目标进入视口时的滚动次数、逐次位移与停止原因，目标已可操作时可为零次。每篇独立保存正文、首层评论、来源核对、验证与截图；正文已读但评论失败时保留部分结果，不把未读到评论等同于确认无评论。旧文字候选仍要求文字；新内容候选明确区分文字、媒体和混合内容，媒体保留实际批次截图并标记未解释，不做 OCR。新回放另保存步骤与阶段耗时，旧记录缺失不补零。总预算到期会关闭本次浏览器并保留已完成证据，重新运行前须重开并确认会话。 + +整页滚动页面可另学显式 viewport 模式，目标仍限定在已观察的普通容器内，预算、对象和来源核对保持不变。维护入口可按本任务真实步骤编号冻结修订版本，不修改旧步骤或候选；新组合须独立检查和回放，不能继承旧成功。字段和边界见 [本机学习入口](tools/learning/README.md)。真实平台适配进展见进度文档。 + +最新文字/图片评论八步候选在两篇真实装修笔记上探索通过，各取得正文和 6 条首层评论，其中一篇包含纯图片评论。当前检查复用原核验语义并限定到已观察目标，每步后的完整取证保留；本机工作进程耗时约 46/49 秒。该版本仍为维护实验候选，未正式发布，模型和需求质量尚未验收；历史失败、样本范围及分阶段测试见 [开发进度](docs/开发进度.md)。 + +平台登录在工作台打开的浏览器中直接完成。打开/刷新时先检测可见页面的登录状态：已登录时无需重复扫码，未登录或需验证时由维护者在官方窗口处理；状态不确定时允许人工核对。当前账号身份仍需确认，页面检测不代替身份认证。固定资料目录用于后续复用。学习账本保存到 PostgreSQL，浏览器资料与证据文件保存在 `.runtime/learning/`，均不提交 Git。 + +## 装修需求分析 + +在[内容初筛](http://127.0.0.1:5179/#/discovery)顶部打开“商家需求”，输入任务文字，核对逐字段原文依据和缺项后确认条件，再点击“用于本次初筛”。本轮只支持小红书装修服务寻找业主，使用第一个搜索词；服务地区可保留未知，不默认全国。确认版本单独追加保存，初筛和阅读继承条件引用，仍需已有候选及已确认会话。 + +在已取得可用正文并通过来源核对的阅读详情中，点击“分析本次已读内容”。这会分析该条已保存的文字，不重新打开或操作平台。正文与每条首层评论分别给出明确需求、疑似需求、普通讨论或信息不足，并保留逐字依据、来源、缺失信息、模型配置和请求记录。纯图片/视频标为信息不足、内容未解释；需求信号不等于已确认客户。失败阅读不能分析，有正文的有效部分阅读会保留范围说明。 + +新版另列当前角色信号、多选表达、业务相关性及根据商家条件派生的初步匹配和建议。“自身要装修”可同时“提问、求建议”；服务方也可有自身需求，短句“哪里”不确认角色。阅读默认沿用关联条件，旧阅读可选择已确认条件追加新分析；条件摘要参与结果复用，旧分析仍可查看。地区或上下文不足会提示补充信息。建议未执行。 + +新分析合同为 `renovation-demand-v4 / demand-result-v3`。评论通过服务端生成的 `context_refs:["note"]` 引用同次阅读的笔记，笔记没有背景引用，正文只向模型提供一次；总范围仍为21个来源、16,000字。新增独立 `consultation`（有／无／信息不足），本条逐字依据与笔记背景依据分开保存，不能引用其他评论或继承作者的身份、需求、项目地区和预算。程序派生“保留待核实”等展示结果，不据咨询意向提升业务匹配或触发联系；这也不是独立线索库。新旧提示和结果版本隔离复用，不重写旧分析。 + +人工核对：打开一条完成的分析，在“人工核对”区域按“核对此条”，对照原文选择判断正确、需要修正、暂不能确定或当前文字不可核对。`demand-result-v3` 使用六轴 `analysis-review-v2`，增加独立咨询意向和笔记背景依据；`demand-result-v2` 仍按原五轴 `analysis-review-v1` 保存，更早结构只读。修正时填写标签、说明和本条原文依据；评论确认有咨询意向时另附所关联笔记的逐字背景依据，再显式保存。无法确定时直接写原因，不必猜身份。每次修订另存，旧模型与人工版本保留，未保存的预填不算核对；纯媒体不作文字判断。一致率仅统计已判定样本的最新版本，不是模型真实准确率或召回率；核对不调用模型、重新读取平台或触发联系动作。 + +查看已有结果时，在分析历史中点击“查看分析”，会直接打开“分析详情”侧栏，首先显示分类、原文和依据；关闭后可再次打开同一条记录。模型配置、用量与请求记录在侧栏下方折叠区中。查看历史只打开已有结果，不调用模型；人工核对与纠错另行显式追加保存，原模型结果保留。 + +服务端模型使用 [.env.example](.env.example) 的 `ASSISTANT_MODEL_*` 配置：供应商、接口、型号、主/备用 Key、思考模式、超时和输出上限。真实 Key 只填被忽略的 `.env`，不使用 `VITE_` 前缀或提交仓库，修改后重启本机服务。当前以 `deepseek-flash`、关闭思考为基线；型号可换，主备只切 Key，不自动升级模型。每请求最多 30 秒、合计最多 60 秒；最多 21 个来源和 16,000 字，超限拒绝，不自动截断。 + +前端只显示安全配置状态，“已配置”不等于已验证连通。相同阅读、配置和分析版本复用已完成结果，失败不会自动重试；新失败记录单独保留。Token 仅汇总已返回用量，失败/超时的未知消耗不计入汇总，费用显示未知，尚未接商家扣费。调用、工程回归与真实内容试用的具体状态见 [开发进度](docs/开发进度.md)。 ## 学习数据库 `pnpm db:up` 创建本项目的独立容器、数据卷和本机连接配置，端口默认为 `127.0.0.1:55432`。使用固定版本的 `paradedb/paradedb:0.26.1-pg17`，包含 PostgreSQL 17、`pg_search 0.26.1` 与 `pgvector 0.8.6`。随机数据库密码和连接串仅写入被 Git 忽略的 `.env`，与后台登录密码分别管理。 -- `assistant_learning`：维护者、任务、观察、候选和验证记录；默认 schema 为 `assistant_learning`。 +- `assistant_learning`：维护者、任务、观察、候选、验证、阅读和独立需求分析记录；默认 schema 为 `assistant_learning`。 - `assistant_retrieval`：独立能力知识检索底座，业务索引与模型 embedding 尚未接入。 - `assistant_learning_test`：隔离工程测试，每轮使用新建的随机 schema。 已有 SQLite 原型数据时，先停止本项目后台，执行 `pnpm db:migrate-sqlite`，再启动后台。迁移只向空 PostgreSQL 命名空间复制，同一事务核对行数；原 `.runtime/learning/learning.sqlite` 及一致备份均保留，不覆盖已存在的 PostgreSQL 数据。未配置 `ASSISTANT_DATABASE_URL` 时仍可用 SQLite 做离线开发；它不是当前 Docker 学习环境的存储方式。 +schema 10 为学习任务增加可空 `execution_profile`；旧任务保留空值且 API 省略字段，不追溯补成新模式。候选清单保存声明合同，实际会话、验证、初筛和阅读各自保存已核验回显；缺字段显示未记录，不用任务选项替代实际执行证据。 + `pnpm db:status` 查看容器，`pnpm db:stop` 停止本项目容器并保留数据卷。普通重启不重建数据库。 ## 检查 @@ -72,14 +113,33 @@ pnpm check pnpm build pnpm test:server pnpm test:learning +pnpm test:discovery +pnpm test:reading +pnpm test:reading-seek +pnpm test:reading-media +pnpm test:demand-analysis +pnpm test:business-workbench +pnpm test:analysis-review +pnpm test:control +pnpm test:session +pnpm test:platform-policy pnpm db:check node tools/learning/test-browser-worker.mjs +node scripts/test-migrate-learning.mjs ``` -`test:learning` 使用隔离的测试库/随机 schema、真实浏览器与本机演示页，覆盖参数化步骤、结果断言、证据回看及重启持久化，不访问真实平台账号。未配置测试 DSN 时回退到临时 SQLite，绝不借用开发库。Windows 默认使用本机 Edge，其他系统默认使用已安装的 Playwright Chromium；可用 `ASSISTANT_TEST_BROWSER_CHANNEL` 覆盖。单端通过不会被记作 Windows/macOS 均通过。 +`test:learning` 使用隔离的测试库/随机 schema、真实浏览器与本机演示页，覆盖参数化对象、结构化读取、试跑/质量界面、晚到异常、证据回看及重启持久化，不访问真实平台账号。未配置测试 DSN 时回退到临时 SQLite，绝不借用开发库。Windows 默认使用本机 Edge，其他系统默认使用已安装的 Playwright Chromium；可用 `ASSISTANT_TEST_BROWSER_CHANNEL` 覆盖。单端通过不会被记作 Windows/macOS 均通过。 `db:check` 实际验证中文 BM25、平台过滤、搜索索引计划与固定样本向量排序，测试数据在专用测试库中创建后清理。样本向量不能代表模型 embedding、召回率或完整混合检索已完成。 +`test:discovery` 使用独立服务、随机测试 schema 与合成页面，验证装修规则、来源/标题不足时不选、界面理由、历史、失败记录及重启保存。`test:reading` 在独立页面验证初筛到单篇阅读、对象绑定、部分结果、历史及重启；`test:reading-seek` 独立验证八步候选录制、结果重排后的滚动定位及有界失败；`test:reading-media` 验证显式文字/媒体/混合内容、媒体截断、原文字合同、步骤耗时、截图及历史。以上脚本均不访问真实账号。`test-migrate-learning.mjs` 仅使用测试 DSN 和随机 schema 验证迁移，包括旧 SQLite 没有质量观察/内容初筛/阅读表的兼容、新记录复制及失败整体回滚；不会向开发库导入测试数据。 + +`test:demand-analysis` 使用随机测试 schema、合成浏览器页面和本机 HTTP 假模型，覆盖需求依据、主备切换、双失败、错误依据拒绝、已完成结果复用、历史/重启及密钥不暴露；显式覆盖所有模型环境变量，不使用真实模型 Key 或真实平台账号。迁移脚本也兼容旧库缺分析表，按阅读后的依赖顺序复制分析事实。 + +`test:business-workbench` 在独立 PostgreSQL、Edge 和本机假模型验证需求解析、显式确认、已学初筛/阅读、多轴分析、历史与复用。schema 8 增加不可变商家需求版本；迁移兼容旧库缺该表，并在同一事务内核对父版本关系和失败回滚。各脚本的实际执行结果与未覆盖项见开发进度，不能以脚本存在代替通过。 + +`test:analysis-review` 使用独立 PostgreSQL、Edge 和本机假模型准备合成阅读/分析，验证四种核对、修正依据、并发旧版本拒绝、最新版本统计及重启；真实业务结论仍需维护者核对。schema 9 增加不可变核对链，迁移兼容旧库缺表、保留修订顺序并拒绝坏父引用。 + ## 远程仓库 [zwt-cmd/assistant](https://github.com/zwt-cmd/assistant) diff --git a/apps/admin/src/A2AWorkbench.tsx b/apps/admin/src/A2AWorkbench.tsx index 0be9378..a5bb5bc 100644 --- a/apps/admin/src/A2AWorkbench.tsx +++ b/apps/admin/src/A2AWorkbench.tsx @@ -8,6 +8,9 @@ import { ProgressPage } from './pages/ProgressPage'; import { TaskDetailPage } from './pages/TaskDetailPage'; import { CandidatesPage } from './pages/CandidatesPage'; import { TestsPage } from './pages/TestsPage'; +import { DiscoveryPage } from './pages/DiscoveryPage'; +import { RoutesPage } from './pages/RoutesPage'; +import { PlatformPathsPage } from './pages/PlatformPathsPage'; import type { AuthStatus } from './types'; function routePath() { return location.hash.slice(1) || '/tasks'; } @@ -15,18 +18,24 @@ export function A2AWorkbench({ auth, onLogout, dark, onTheme }: { auth: AuthStat const [path, setPath] = useState(routePath); useEffect(() => { const update = () => setPath(routePath()); window.addEventListener('hashchange', update); return () => window.removeEventListener('hashchange', update); }, []); const navigate = useCallback((next: string) => { location.hash = next; }, []); - const category = path.split('/')[1] || 'tasks'; - const detailId = path.split('/')[2]; + const [pathname, query = ''] = path.split('?', 2); + const category = pathname.split('/')[1] || 'tasks'; + const detailId = pathname.split('/')[2]; + const selectedReadingId = new URLSearchParams(query).get('reading') || undefined; + const selectedValidationId = new URLSearchParams(query).get('validation') || undefined; const accountName = auth.user?.display_name || auth.user?.username || '维护者'; let content = ; if (category === 'tasks' && detailId) content = ; if (category === 'overview') content = ; if (category === 'candidates') content = ; if (category === 'validations') content = ; + if (category === 'discovery') content = ; + if (category === 'routes') content = ; + if (category === 'platform-paths') content = ; return
-
学习管理 / {({ tasks: '学习批次', overview: '账本总览', candidates: '能力实体', validations: '脚本测试' } as Record)[category] || '学习批次'}
{content}
+
学习管理 / {({ tasks: '学习批次', routes: '链路复用', 'platform-paths': '平台路径', overview: '账本总览', candidates: '能力实体', validations: '脚本测试', discovery: '内容初筛' } as Record)[category] || '学习批次'}
{content}
; } diff --git a/apps/admin/src/components/ActionFailureNotice.tsx b/apps/admin/src/components/ActionFailureNotice.tsx new file mode 100644 index 0000000..e19ac50 --- /dev/null +++ b/apps/admin/src/components/ActionFailureNotice.tsx @@ -0,0 +1,18 @@ +import { Alert, Typography } from 'antd'; +import type { Observation } from '../types'; +import { failurePhaseName, RequestFailureDetails } from './VerificationFailureNotice'; + +/** Render only the bounded diagnostic fields supplied by the learning API. */ +export function ActionFailureNotice({ observation }: { observation: Observation }) { + if (observation.source !== 'action_failure' && !observation.action_failure) return null; + const failure = observation.action_failure; + return +

页面可能已有局部变化，请先核对本次前后证据，再决定下一步。

+ {failure && <> + 错误代码：{failure.code} · 阶段：{failurePhaseName(failure.phase)} · 阻止请求总数：{failure.diagnostics?.blocked_request_count ?? '未记录'} · {failure.moment === 'before' ? '操作前证据' : '失败后证据'} + + } + } />; +} diff --git a/apps/admin/src/components/AnalysisReviewPanel.tsx b/apps/admin/src/components/AnalysisReviewPanel.tsx new file mode 100644 index 0000000..50007a1 --- /dev/null +++ b/apps/admin/src/components/AnalysisReviewPanel.tsx @@ -0,0 +1,162 @@ +import { Alert, Button, Card, Collapse, Descriptions, Drawer, Input, Select, Space, Tag, Typography } from 'antd'; +import { createContext, useContext, useEffect, useRef, useState, type ReactNode } from 'react'; +import { api, ApiError } from '../api'; +import type { AnalysisReview, AnalysisReviewSummary, AnalysisReviewDecision, AnalysisReviewLabels, DemandAnalysis, PageResult } from '../types'; +import { formatTime } from './common'; +import { ConsultationBackground, consultationLabels } from './ConsultationDetails'; + +type Source = DemandAnalysis['sources'][number]; +type Result = DemandAnalysis['results'][number]; +const decisions: Record = { confirmed: '判断正确', corrected: '需要修正', uncertain: '暂不能确定', unreviewable: '当前文字不可核对' }; +const roleLabels = { suspected_demand_side: '疑似需求方', suspected_service_provider: '疑似服务方', participant: '普通参与者', unknown: '角色未知' }; +const demandLabels = { explicit: '明确自身需求', possible: '疑似自身需求', discussion: '普通讨论', insufficient: '信息不足' }; +const expressionLabels = { self_need: '自身需求', question: '询问', seek_advice: '寻求建议', service_offer: '提供服务', advice: '给出建议', case_share: '案例分享', other: '其他表达', unknown: '表达不明' }; +const relevanceLabels = { relevant: '相关', irrelevant: '不相关', unclear: '不明确' }; +const matchLabels = { matched: '条件初步匹配', possible: '可能匹配', insufficient: '条件不足', not_matched: '不匹配' }; +const options = (labels: Record) => Object.entries(labels).map(([value, label]) => ({ value, label })); +function modelLabels(result: Result, withConsultation = false): AnalysisReviewLabels | undefined { + if (!result.role || !result.expressions || !result.business_relevance || !result.match) return undefined; + if (withConsultation && !result.consultation) return undefined; + return { role: result.role, demand: result.demand, expressions: result.expressions.map(item => item.type), business_relevance: result.business_relevance.value, match: result.match, ...(withConsultation ? { consultation: result.consultation!.value } : {}) }; +} +function LabelDetails({ labels }: { labels: AnalysisReviewLabels }) { return expressionLabels[value]).join('、') }, + { key: 'related', label: '服务相关性', children: relevanceLabels[labels.business_relevance] }, { key: 'match', label: '业务匹配', children: matchLabels[labels.match] }, + { key: 'consultation', label: '咨询意向', children: labels.consultation ? consultationLabels[labels.consultation] : '历史未记录' }, +]} />; } +interface ReviewContextValue { enabled: boolean; loading: boolean; latest: (ref: string) => AnalysisReview | undefined; edit: (source: Source, result: Result) => void } +const ReviewContext = createContext(undefined); +export function SourceReviewControl({ source, result }: { source: Source; result: Result }) { + const context = useContext(ReviewContext); + if (!context?.enabled || !modelLabels(result)) return null; + const latest = context.latest(source.source_ref); + return
+ {latest ? ({ confirmed: '人工已确认', corrected: '人工已纠错', uncertain: '有争议 / 待判断', unreviewable: '当前不可核对' }[latest.decision]) : '待人工核对'} + + {latest && 最近核对：{formatTime(latest.created_at)} · {decisions[latest.decision]}{latest.labels ? ` · ${demandLabels[latest.labels.demand]} / ${matchLabels[latest.labels.match]} / 咨询：${latest.labels.consultation ? consultationLabels[latest.labels.consultation] : '历史未记录'}` : ''}。原模型结果保留。} +
; +} + +export function AnalysisReviewPanel({ analysis, children }: { analysis: DemandAnalysis; children: ReactNode }) { + const withConsultation = analysis.result_version === 'demand-result-v3'; + const enabled = analysis.status === 'completed' && ['demand-result-v2', 'demand-result-v3'].includes(analysis.result_version); + const scalarAxes = withConsultation ? ['role', 'demand', 'business_relevance', 'match', 'consultation'] as const : ['role', 'demand', 'business_relevance', 'match'] as const; + const fields: Array = ['role', 'demand', 'expressions', 'business_relevance', 'match', ...(withConsultation ? ['consultation' as const] : [])]; + const [summary, setSummary] = useState(); + const [history, setHistory] = useState([]); + const [loading, setLoading] = useState(enabled); + const [loadError, setLoadError] = useState(false); + const [editing, setEditing] = useState<{ source: Source; result: Result; previousId: string | null }>(); + const [decision, setDecision] = useState(); + const [labels, setLabels] = useState(); + const [evidenceText, setEvidenceText] = useState(''); + const [contextEvidenceText, setContextEvidenceText] = useState(''); + const [note, setNote] = useState(''); + const [busy, setBusy] = useState(false); + const [error, setError] = useState(''); + const [saved, setSaved] = useState(false); + const [requiresReopen, setRequiresReopen] = useState(false); + const feedback = useRef(null); + const mounted = useRef(true), active = useRef(false), generation = useRef(0); + useEffect(() => { if (error || saved) feedback.current?.scrollIntoView({ block: 'nearest', inline: 'nearest' }); }, [error, saved]); + const route = `/discovery/analyses/${encodeURIComponent(analysis.id)}/reviews`; + useEffect(() => { mounted.current = true; return () => { mounted.current = false; generation.current += 1; }; }, []); + async function refresh() { + const current = ++generation.current; setLoading(true); setLoadError(false); + try { + const [records, metrics] = await Promise.all([api.get>(route), api.get(`/discovery/analyses/${encodeURIComponent(analysis.id)}/review-summary`)]); + if (!mounted.current || current !== generation.current) return; + if (metrics.analysis_id !== analysis.id) throw new Error('Review summary source mismatch'); + setSummary(metrics); + setHistory(records.items.filter(item => item.analysis_id === analysis.id && analysis.sources.some(source => source.source_ref === item.source_ref))); + } catch { if (mounted.current && current === generation.current) setLoadError(true); } + finally { if (mounted.current && current === generation.current) setLoading(false); } + } + useEffect(() => { if (enabled) void refresh(); }, [analysis.id, enabled]); + const latest = (ref: string) => summary?.latest_reviews.find(item => item.analysis_id === analysis.id && item.source_ref === ref); + function edit(source: Source, result: Result) { + if (active.current || loading || loadError) return; + const current = latest(source.source_ref); + setEditing({ source, result, previousId: current?.id || null }); setDecision(undefined); + // Only the current result version supplies editable axes. Old saved labels + // are never supplemented with a consultation label they did not contain. + setLabels(current?.labels || modelLabels(result, withConsultation)); setEvidenceText(''); setContextEvidenceText(''); setNote(''); setError(''); setSaved(false); setRequiresReopen(false); + } + const textAvailable = Boolean(editing?.source.text.trim()); + const selectedLabels = decision === 'confirmed' && editing ? modelLabels(editing.result, withConsultation) : labels; + const background = editing?.source.kind === 'comment' ? analysis.sources.find(source => source.kind === 'note' && editing.source.context_refs?.includes(source.source_ref)) : undefined; + function chooseDecision(value: AnalysisReviewDecision) { + setDecision(value); setError(''); + if (withConsultation && editing && value === 'confirmed') { + setEvidenceText(editing.result.consultation?.evidence.join('\n') || ''); + setContextEvidenceText(editing.result.consultation?.context_evidence.filter(item => item.source_ref === background?.source_ref).flatMap(item => item.evidence).join('\n') || ''); + } + } + async function save() { + if (active.current || !editing || !decision || requiresReopen || loadError) return; + const captured = editing; + const judgedDecision = ['confirmed', 'corrected'].includes(decision); + const evidence = judgedDecision ? evidenceText.split('\n').map(value => value.trim()).filter(Boolean) : []; + const contextQuotes = judgedDecision && withConsultation ? contextEvidenceText.split('\n').map(value => value.trim()).filter(Boolean) : []; + if (evidence.length > 3 || evidence.some(quote => quote.length > 1000 || !captured.source.text.includes(quote))) { setError('依据须为本条原文中的逐字片段，每条最多 1000 字、总共最多 3 条。'); return; } + if ((!textAvailable && ['confirmed', 'corrected'].includes(decision)) || (decision === 'corrected' && (!note.trim() || evidence.length < 1 || evidence.length > 3 || evidence.some(quote => !captured.source.text.includes(quote)))) || (['uncertain', 'unreviewable'].includes(decision) && !note.trim())) { + setError('请补充说明；修正需 1–3 条来自本条原文的逐字依据。纯媒体不能确认或修正文字判断。'); return; + } + if (judgedDecision && (!selectedLabels || selectedLabels.expressions.length < 1 || (withConsultation && !selectedLabels.consultation))) { setError(`请补齐${withConsultation ? '六' : '五'}个判断维度。`); return; } + if (contextQuotes.length > 3 || contextQuotes.some(quote => quote.length > 1000 || !background?.text.includes(quote))) { setError('笔记背景依据须为已关联笔记正文的逐字片段，每条最多 1000 字、总共最多 3 条。'); return; } + if (judgedDecision && withConsultation && selectedLabels?.consultation === 'yes') { + if (!evidence.length || selectedLabels.business_relevance !== 'relevant' || !selectedLabels.expressions.some(value => ['question', 'seek_advice', 'self_need'].includes(value))) { setError('有咨询意向须提供本条逐字依据、服务相关性为相关，且包含询问、寻求建议或自身需求表达。'); return; } + if (captured.source.kind === 'comment' && (!background || !contextQuotes.length)) { setError('评论的咨询意向还须引用本条已关联的笔记背景依据，不能引用其他评论。'); return; } + } + const originalLabels = modelLabels(captured.result, withConsultation); + if (decision === 'corrected' && selectedLabels && originalLabels && scalarAxes.every(axis => selectedLabels[axis] === originalLabels[axis]) && [...selectedLabels.expressions].sort().join('|') === [...originalLabels.expressions].sort().join('|')) { setError('需要修正时至少改变一个维度；完全同意原模型请选择判断正确。'); return; } + active.current = true; setBusy(true); setError(''); + try { + const result = await api.post(route, { source_ref: captured.source.source_ref, previous_review_id: captured.previousId, decision, labels: judgedDecision ? selectedLabels : null, evidence, note: note.trim(), ...(withConsultation ? { context_evidence: contextQuotes.length ? [{ source_ref: background!.source_ref, evidence: contextQuotes }] : [] } : {}) }); + if (!mounted.current) return; + if (result.analysis_id !== analysis.id || result.source_ref !== captured.source.source_ref) { setRequiresReopen(true); setError('返回记录与当前来源不符，请关闭并刷新核对记录。'); return; } + setSaved(true); setRequiresReopen(true); await refresh(); + } catch (failure) { + if (!mounted.current) return; + if (failure instanceof ApiError && failure.status === 422) { setError(`核对未保存：${failure.message}`); return; } + setRequiresReopen(true); setError(failure instanceof ApiError && failure.status === 409 ? '这条来源已有更新的核对。已刷新历史；请关闭后重新核对此条，再决定是否追加修订。' : '保存请求未完成。请先查看最新历史，再关闭并重新核对此条；不会自动重试。'); await refresh(); + } finally { active.current = false; if (mounted.current) setBusy(false); } + } + const counts = summary?.decision_counts || { confirmed: 0, corrected: 0, uncertain: 0, unreviewable: 0 }; + const judged = summary?.judged || 0; + return + void refresh()}>刷新核对记录}> + {!enabled ? {analysis.status === 'completed' ? '这条历史使用旧版结果结构，只可查看。请在新版多维分析中逐条核对，旧记录不补造维度。' : '分析尚未完成，没有可核对的完整结果。'} : <> + {loadError && } + {!summary && !loadError && 正在读取人工核对记录…} + {summary && <>待核对 {summary?.pending ?? analysis.sources.length}已确认 {counts.confirmed}已纠错 {counts.corrected}有争议 {counts.uncertain}不可核对 {counts.unreviewable} + 已判定 {judged} / {summary?.total ?? analysis.sources.length}；已判定样本一致率：{summary?.agreement_rate !== undefined && summary.agreement_rate !== null ? `${(summary.agreement_rate * 100).toFixed(1)}%` : '尚无已判定样本'}。 + {fields.map(field => {({ role: '角色', demand: '自身需求', expressions: '表达', business_relevance: '相关性', match: '匹配', consultation: '咨询意向' })[field]}差异：{summary.axis_diff_counts[field] ?? '历史未记录'})}} + 统计仅取每条来源最新核对，确认和纠错计入已判定；一致率不是模型真实准确率。保存只追加人工记录，不修改模型结果、不重新调用模型或浏览器、不执行互动。 + } + + {children} + { if (!busy) setEditing(undefined); }} closable={{ 'aria-label': '关闭人工核对' }} size="min(760px, 100vw)" destroyOnHidden styles={{ body: { padding: 20, overflowWrap: 'anywhere' } }}> + {editing &&
+ {editing.source.kind === 'note' ? '笔记正文' : `首层评论 ${editing.source.comment_index}`}当前模型与人工记录分开保存 +

本条已保存原文

{editing.source.text || '无可读文字；媒体未解释，不能据此认定没有需求。'}
+ {withConsultation && editing.source.kind === 'comment' &&

已关联的笔记背景

来源：本次已保存的笔记正文。只帮助理解咨询对象，不补成评论者自身需求、地区或身份。{background?.text || '未记录关联背景，不补推断。'}
} + {withConsultation && <>本条咨询依据：{editing.result.consultation?.evidence.map(value => `「${value}」`).join('；') || '无'}} }]} /> + {(saved || error) &&
+ {saved && } + {error && } +
} +

选择本次核对结论

setLabels(previous => previous ? { ...previous, [field]: value } : previous)} />
)}} + {decision && <> + {textAvailable && ['confirmed', 'corrected'].includes(decision) &&

逐字原文依据

每行一条，最多 3 条；修正或确认有咨询意向必须提供本来源原文，不引用其他评论。 setEvidenceText(event.target.value)} rows={3} maxLength={3002} />
} + {withConsultation && background && ['confirmed', 'corrected'].includes(decision) &&

笔记背景依据

来源：上方已关联的笔记正文。每行一条，最多3条、每条最多1000字；评论判有咨询意向时必填，不能替代本条原文依据。 setContextEvidenceText(event.target.value)} rows={3} maxLength={3002} />
} +

核对说明{decision !== 'confirmed' ? '（必填）' : '（可选）'}

setNote(event.target.value)} rows={2} maxLength={1000} />
+ + } +

此条人工核对历史

{history.filter(item => item.source_ref === editing.source.source_ref).length ? history.filter(item => item.source_ref === editing.source.source_ref).map(record =>
{decisions[record.decision]}{formatTime(record.created_at)}{record.labels && }说明：{record.note || '未另填说明'}本条原文依据：{record.evidence.map(value => `「${value}」`).join('；') || '无'}{record.contract_version === 'analysis-review-v2' && }
) : 尚无人工核对；当前预填标签未保存。}
+ } + + ; +} diff --git a/apps/admin/src/components/BusinessBriefPanel.tsx b/apps/admin/src/components/BusinessBriefPanel.tsx new file mode 100644 index 0000000..c7da4a6 --- /dev/null +++ b/apps/admin/src/components/BusinessBriefPanel.tsx @@ -0,0 +1,92 @@ +import { Alert, Button, Card, Descriptions, Drawer, Form, Input, Select, Space, Table, Tag, Typography } from 'antd'; +import { useEffect, useRef, useState } from 'react'; +import { api, errorText } from '../api'; +import type { BusinessBrief, BusinessConditions, PageResult } from '../types'; +import { formatTime, LoadError, useResource } from './common'; + +export const briefFieldLabels: Record = { platform: '目标平台', service: '商品或服务', target_customer: '目标客户', search_terms: '搜索词', region: '服务地区', region_scope: '地区范围', exclude_service_promotions: '排除服务推广' }; +const briefStatusLabels: Record = { draft: '待人工确认', needs_clarification: '需要补充条件', unsupported: '超出当前支持范围', failed: '解析未完成', confirmed: '条件已确认' }; +export function RegionScope({ conditions }: { conditions: Pick }) { return <>{conditions.region_scope === 'specified' ? conditions.region : conditions.region_scope === 'unrestricted' ? '不限地区（已明确选择）' : '未设置，不推断全国'}; } +export function BriefConditions({ brief }: { brief: BusinessBrief }) { return }, + { key: 'exclude', label: '服务推广', children: brief.conditions.exclude_service_promotions === null ? '待确认' : brief.conditions.exclude_service_promotions ? '匹配时排除服务推广信号' : '不额外排除，仍独立判断自身需求' }, +]} />; } + +export function BusinessBriefPanel({ onUse }: { onUse: (brief: BusinessBrief) => void }) { + const records = useResource>('/business/briefs'); + const [open, setOpen] = useState(false); + const [selected, setSelected] = useState(); + const [requestText, setRequestText] = useState(''); + const [error, setError] = useState(''); + const [busy, setBusy] = useState(false); + const [confirmForm] = Form.useForm(); + const active = useRef(false), mounted = useRef(true); + useEffect(() => { mounted.current = true; return () => { mounted.current = false; }; }, []); + const scope = Form.useWatch('region_scope', confirmForm); + function show(brief?: BusinessBrief) { + setError(''); setSelected(brief); + confirmForm.resetFields(); + if (brief) { setRequestText(brief.request_text); confirmForm.setFieldsValue(brief.conditions); } + setOpen(true); + } + async function parse() { + if (active.current || !requestText.trim()) return; + active.current = true; setBusy(true); setError(''); + try { const result = await api.post('/business/briefs', { request_text: requestText.trim() }); if (!mounted.current) return; setSelected(result); confirmForm.resetFields(); confirmForm.setFieldsValue(result.conditions); await records.reload(); } + catch (failure) { if (mounted.current) { setError(errorText(failure)); await records.reload(); } } + finally { active.current = false; if (mounted.current) setBusy(false); } + } + async function confirm(values: BusinessConditions) { + if (active.current || !selected || !['draft', 'needs_clarification'].includes(selected.status)) return; + active.current = true; setBusy(true); setError(''); + try { + const result = await api.post(`/business/briefs/${selected.id}/confirm`, { ...values, search_terms: values.search_terms.map(value => value.trim()), region: values.region_scope === 'specified' ? values.region?.trim() || null : null }); + if (!mounted.current) return; setSelected(result); await records.reload(); + } catch (failure) { if (mounted.current) setError(errorText(failure)); } + finally { active.current = false; if (mounted.current) setBusy(false); } + } + const canConfirm = selected && ['draft', 'needs_clarification'].includes(selected.status); + return { setRequestText(''); show(); }}>输入商家需求}> + 先说明你提供什么服务、要找谁和服务地区，解析后由你确认条件，再用于搜索和已读内容分析。 + 当前支持小红书装修服务样例；解析不会打开平台或执行关注、私信和评论。确认业务条件不等于已核实客户身份。 + + data-testid="business-brief-history" size="small" rowKey="id" loading={records.loading} dataSource={records.data?.items || []} pagination={{ pageSize: 3, showSizeChanger: false }} scroll={{ x: 620 }} locale={{ emptyText: '尚无商家需求，点击右上角输入。' }} columns={[ + { title: '时间', key: 'created_at', render: (_, row) => formatTime(row.created_at) }, + { title: '需求', key: 'request', render: (_, row) => {row.request_text} }, + { title: '状态', key: 'status', render: (_, row) => {briefStatusLabels[row.status]} }, + { title: '操作', key: 'view', render: (_, row) => {row.status === 'confirmed' && } }, + ]} /> + { if (!busy) setOpen(false); }} closable={{ 'aria-label': '关闭商家需求' }} size="min(800px, 100vw)" destroyOnHidden styles={{ body: { padding: 20, overflowWrap: 'anywhere' } }}> +
+ 例如：我是做装修服务的，想在小红书找准备装修的业主，服务杭州，排除装修公司推广。 + setRequestText(event.target.value)} placeholder="说明商品或服务、目标客户、平台和地区；没有提供的条件会留待确认。" /> + + {error && } + {selected &&
+ {briefStatusLabels[selected.status]}{formatTime(selected.created_at)} + 本次原始需求：{selected.request_text} + {selected.missing_fields.length > 0 && briefFieldLabels[field as keyof BusinessConditions] || field).join('、')} />} + {selected.status === 'failed' && } + {selected.status === 'unsupported' && } + {selected.uncertainties.length > 0 &&
    {selected.uncertainties.map((value, index) =>
  • {value}
    • )}
} + {selected.kind === 'parse' &&

解析字段与原文依据

{Object.entries(selected.field_evidence).map(([field, quotes]) => {briefFieldLabels[field as keyof BusinessConditions] || field}：{quotes.length ? quotes.map(quote => `「${quote}」`).join('；') : '原文未提供，不能自动补全'})}
} + {canConfirm && form={confirmForm} layout="vertical" disabled={busy} onFinish={values => void confirm(values)} data-testid="business-brief-confirm-form"> +

人工核对并确认条件

确认会追加一条条件记录，保留原解析；所有确认字段记为维护者确认，不冒充模型原文依据。 +
+ + } + + + } + {values?.[name]?.source === 'content' &&
+ + + + +
} + 每项必须包含该字段 +
)} + + + }; +} + +export function LearningActionForm({ selected, latestElements, parameters, busy, ready, onExecute }: { + selected?: BrowserElement; + latestElements: BrowserElement[]; + parameters: Record; + busy: boolean; + ready: boolean; + onExecute: (action: RecordedStep) => Promise; +}) { + const [form] = Form.useForm(); + const kind = Form.useWatch('kind', form) || 'read'; + const parameter = Form.useWatch('parameter', form); + const waitState = Form.useWatch('state', form); + const minCount = Form.useWatch('min_count', form); + const waitScope = Form.useWatch('wait_scope', form); + const useTarget = Form.useWatch('use_target', form); + const targetField = Form.useWatch('target_field', form); + const targetParameter = Form.useWatch('target_parameter', form); + const rowSelector = Form.useWatch('row_selector', form); + const useBatch = Form.useWatch('use_batch', form); + const allowEmpty = Form.useWatch('allow_empty', form); + const useUntilTarget = Form.useWatch('use_until_target', form); + const untilTargetField = Form.useWatch('until_target_field', form); + const untilTargetParameter = Form.useWatch('until_target_parameter', form); + const fields = Form.useWatch('fields', form) as ExtractionField[] | undefined; + const containerRows = Boolean(rowSelector?.trim()); + const dynamic = ['click', 'read', 'extract'].includes(kind) && !(kind === 'extract' && (containerRows || useBatch || allowEmpty)); + const dedupeOptions = (fields || []).filter(field => field.required && field.source === 'attribute' && /^(?:id|data-[A-Za-z0-9_.:-]+)$/.test(field.attribute?.trim() || '') && identifier.test(field.name?.trim() || '')).map(field => ({ value: field.name.trim(), label: `${field.name.trim()} (${field.attribute?.trim()})` })); + const parameterOptions = Object.entries(parameters).map(([value, sample]) => ({ value, label: `${value} = ${String(sample)}` })); + const scopedWait = kind === 'wait' && waitState === 'visible' && typeof minCount === 'number'; + const waitScopeOptions = latestElements.filter(element => element.kind === 'read' && element.unique === true && !element.sensitive && element.tag && !['body', 'html', 'form'].includes(element.tag.toLowerCase())).map(element => ({ value: element.selector, label: `${element.selector}${element.label ? ` · ${element.label}` : ''}` })); + useEffect(() => { + if (selected) form.setFieldsValue({ selector: selected.selector, kind: guessKind(selected), parameter: undefined, value: '', use_target: false, use_viewport_scroll: false, wait_scope: undefined }); + }, [selected?.selector, selected?.kind, selected?.tag, form]); + useEffect(() => { + if (waitScope && (!scopedWait || !waitScopeOptions.some(option => option.value === waitScope))) form.setFieldValue('wait_scope', undefined); + }, [scopedWait, waitScope, latestElements, form]); + + async function submit(values: ActionValues) { + const action: RecordedStep = { kind: values.kind, selector: values.selector.trim() }; + if (values.kind === 'press') action.key = 'Enter'; + if (values.kind === 'fill') { + if (values.parameter) action.parameter = values.parameter; + else action.value = values.value || ''; + } + if (values.kind === 'read' || values.kind === 'extract') action.output_name = values.output_name?.trim(); + if (values.kind === 'scroll') Object.assign(action, { direction: values.direction, pixels: values.pixels, max_scrolls: values.max_scrolls, + ...(values.use_until_target ? { until_target: { + selector: values.until_target_selector?.trim(), field: values.until_target_field || 'text', + ...(values.until_target_parameter ? { parameter: values.until_target_parameter } : { value: values.until_target_value?.trim() }), + ...(values.until_target_field === 'attribute' ? { attribute: values.until_target_attribute?.trim() } : {}), + }, timeout_ms: values.seek_timeout_ms, ...(values.use_viewport_scroll ? { scroll_scope: 'viewport' } : {}) } : values.until_selector?.trim() ? { until_selector: values.until_selector.trim() } : {}), + }); + if (values.kind === 'wait') Object.assign(action, { state: values.state, timeout_ms: values.timeout_ms, ...(values.state === 'text' ? { text: values.text } : {}), ...(values.state === 'visible' && typeof values.min_count === 'number' ? { min_count: values.min_count, ...(values.wait_scope?.trim() ? { wait_scope: values.wait_scope.trim() } : {}) } : {}) }); + if (values.kind === 'extract') Object.assign(action, { + limit: values.limit, + ...(values.complete_only || values.use_batch ? { complete_only: true } : {}), + ...(values.row_selector?.trim() ? { row_selector: values.row_selector.trim() } : {}), + ...(values.use_batch ? { batch: { scroll_selector: values.batch_scroll_selector?.trim(), pixels: values.batch_pixels, max_batches: values.batch_max_batches, dedupe_field: values.batch_dedupe_field } } : {}), + ...(values.allow_empty && !values.use_batch ? { allow_empty: true, empty_state: { selector: values.empty_state_selector?.trim(), text: values.empty_state_text?.trim() } } : {}), + fields: (values.fields || []).map(field => ({ name: field.name.trim(), source: field.source, required: field.source === 'content' || Boolean(field.required), ...(field.selector?.trim() ? { selector: field.selector.trim() } : {}), ...(field.source === 'attribute' ? { attribute: field.attribute?.trim() } : {}), ...(field.source === 'content' ? { text_selector: field.text_selector?.trim(), media_selector: field.media_selector?.trim(), max_media: field.max_media ?? 4 } : {}) })), + }); + if (dynamic && values.use_target) action.target = { + field: values.target_field || 'text', + ...(values.target_parameter ? { parameter: values.target_parameter } : { value: values.target_value?.trim() }), + ...(values.target_field === 'attribute' ? { attribute: values.target_attribute?.trim() } : {}), + }; + if (await onExecute(action)) form.setFieldValue('selector', ''); + } + + return form={form} layout="vertical" disabled={busy} onFinish={values => void submit(values)} data-testid="learning-action-form" initialValues={{ + kind: 'read', selector: '', value: '', output_name: 'results', direction: 'down', pixels: 600, max_scrolls: 1, + state: 'visible', timeout_ms: 5000, limit: 20, complete_only: false, until_selector: '', fields: [{ name: 'text', source: 'text', required: true }], use_target: false, target_field: 'text', + row_selector: '', use_batch: false, batch_scroll_selector: '', batch_pixels: 400, batch_max_batches: 3, allow_empty: false, empty_state_selector: '', empty_state_text: '', + use_until_target: false, use_viewport_scroll: false, until_target_field: 'text', until_target_selector: '', seek_timeout_ms: 5000, + }}> + + {!parameter && } + } + {kind === 'press' && } + {(kind === 'read' || kind === 'extract') && } + {kind === 'scroll' && <> + } + {useUntilTarget &&
+ 滚动整页寻找容器内目标 + + } + } + + +
} + 限定范围内滚动；到达边界或无法继续移动即停止。 + } + {kind === 'wait' && <> + } + {waitState === 'text' && } + + } + {kind === 'extract' && <> + + + 只读完整可见行 + + { if (event.target.checked) form.setFieldsValue({ complete_only: true, allow_empty: false, use_target: false }); }}>有限多批读取并去重 + {useBatch &&
+ +
+ { if (value && !dedupeOptions.some(option => option.value === value)) throw new Error('去重字段须为必填的 id 或 data-* 属性字段'); } }]}> + +
} + 只提取当前视口内可见的行；输出包含 items、数量、来源和覆盖范围，超过上限会标注截断。 + } + {dynamic && <> + 按当前对象精确匹配 + {useTarget &&
+ } + } + +
} + } + + 首条只读链路。登录资料和发送、关注等动作不进入此流程；条件等待依据页面状态，未承诺平台账号效果。 + ; +} diff --git a/apps/admin/src/components/LoginDetectionNotice.tsx b/apps/admin/src/components/LoginDetectionNotice.tsx new file mode 100644 index 0000000..f686416 --- /dev/null +++ b/apps/admin/src/components/LoginDetectionNotice.tsx @@ -0,0 +1,29 @@ +import { Alert, Space, Tag } from 'antd'; +import type { LoginDetection } from '../types'; + +const states = { + logged_in: { type: 'success', label: '已登录', description: '无需重复扫码，确认当前账号后继续。页面信号不代替账号身份确认。' }, + logged_out: { type: 'warning', label: '未登录', description: '请在官方浏览器窗口自行完成登录，再刷新并确认当前账号。' }, + verification_required: { type: 'warning', label: '需验证码或人工验证', description: '请在官方浏览器窗口自行完成验证，再刷新并确认当前账号。' }, + unknown: { type: 'info', label: '登录状态不确定', description: '当前可见页面信号不足，请在官方浏览器窗口核对账号；确认后可继续。' }, +} as const; +const evidenceLabels: Record = { + visible_current_account_navigation: '可见当前账号导航', + visible_login_dialog: '可见登录窗口', + visible_login_entry: '可见登录入口', + visible_verification_widget: '可见验证控件', + no_conclusive_visible_evidence: '缺少明确页面依据', + unsupported_platform: '此平台尚无检测适配', + inspection_unavailable: '当前页面暂不可检测', +}; + +export function LoginDetectionNotice({ detection, sessionNeedsReopen = false }: { detection?: LoginDetection; sessionNeedsReopen?: boolean }) { + if (!detection) return null; + const state = states[detection.state] || states.unknown; + const evidence = (detection.evidence || []).filter(item => Object.hasOwn(evidenceLabels, item)).slice(0, 8); + return +

{sessionNeedsReopen ? '页面信号仅反映当前登录状态。本次会话已阻断，请先关闭并重新打开浏览器，再核对账号。' : state.description}

+ {evidence.length > 0 && {evidence.map((item, index) => {evidenceLabels[item]})}} + } />; +} diff --git a/apps/admin/src/components/QualityObservations.tsx b/apps/admin/src/components/QualityObservations.tsx new file mode 100644 index 0000000..98dae54 --- /dev/null +++ b/apps/admin/src/components/QualityObservations.tsx @@ -0,0 +1,82 @@ +import { Alert, App, Button, Form, Input, Select, Space, Tag, Typography } from 'antd'; +import { useEffect, useRef, useState } from 'react'; +import { api, errorText } from '../api'; +import { formatTime, StatusTag } from './common'; +import type { PageResult, QualityObservation, Validation } from '../types'; + +const stateLabels: Record = { waiting: '等待复查', clear: '本次范围未见异常', alert: '发现提醒或异常', uncertain: '状态不确定' }; +const reviewLabels: Record = { passed: '操作过程核对通过', failed: '操作过程核对失败', not_reviewed: '尚未核对操作过程' }; +const sourceLabels: Record = { official_message: '官方消息', account_function: '账号功能', manual: '维护者人工检查' }; + +interface ObservationValues { + state: QualityObservation['state']; + note: string; + process_review: QualityObservation['process_review']; + source: QualityObservation['source']; + notice_ref?: string; + occurred_at?: string; +} + +export function QualityObservations({ record, onUpdated }: { record: Validation; onUpdated?: (record: Validation) => void }) { + const [current, setCurrent] = useState(record); + const [busy, setBusy] = useState(false); + const [refreshPending, setRefreshPending] = useState(false); + const generation = useRef(0); + const [form] = Form.useForm(); + const state = Form.useWatch('state', form); + const { message } = App.useApp(); + useEffect(() => { + generation.current += 1; + setCurrent(record); setRefreshPending(false); setBusy(false); form.resetFields(); + return () => { generation.current += 1; }; + }, [record, form]); + async function save(values: ObservationValues) { + const requestGeneration = generation.current; + setBusy(true); + try { + const observation = await api.post(`/learning/validations/${record.id}/quality-observations`, { + state: values.state, note: values.note.trim(), process_review: values.process_review, source: values.source, + ...(values.notice_ref?.trim() ? { notice_ref: values.notice_ref.trim() } : {}), + ...(values.occurred_at ? { occurred_at: new Date(values.occurred_at).toISOString() } : {}), + }); + if (requestGeneration !== generation.current) return; + form.resetFields(); + setRefreshPending(true); + setCurrent(previous => ({ ...previous, quality_observations: [...(previous.quality_observations || []), observation] })); + void message.success('人工观察已追加保存'); + try { + const all = await api.get>('/learning/validations'); + if (requestGeneration !== generation.current) return; + const updated = all.items.find(item => item.id === record.id); + if (updated) { setCurrent(updated); setRefreshPending(false); onUpdated?.(updated); } + else void message.warning('记录已保存，当前质量状态请刷新后查看'); + } catch { if (requestGeneration === generation.current) void message.warning('记录已保存，质量状态刷新失败，请重新打开验证详情'); } + } catch (error) { if (requestGeneration === generation.current) void message.error(errorText(error)); } + finally { if (requestGeneration === generation.current) setBusy(false); } + } + const events = current.quality_observations || []; + const due = current.observation_due_at ? new Date(current.observation_due_at).getTime() : NaN; + const duePassed = Number.isFinite(due) && Date.now() >= due; + const needsUserReview = current.mode === 'runtime' && current.status === 'passed' && current.quality?.state !== 'verified_in_scope'; + return
+

运行质量与任务后观察

{refreshPending ? 质量状态待刷新 : current.quality && }
+ + {needsUserReview && } +
观察到期
{current.observation_due_at ? formatTime(current.observation_due_at) : '未配置'}
运行前基线
{current.baseline_note || '未记录'}
+ {events.length ?
{events.map(event =>
+ {stateLabels[event.state]}{sourceLabels[event.source]} · {reviewLabels[event.process_review]} +

{event.note}

登记于 {formatTime(event.created_at)}{event.occurred_at ? ` · 发生于 ${formatTime(event.occurred_at)}` : ''} + {event.notice_ref &&

关联通知：{event.notice_ref}

} +
)}
:

还没有人工观察记录；未检查不等于没有异常。

} + form={form} layout="vertical" className="quality-observation-form" disabled={busy} onFinish={values => void save(values)} initialValues={{ state: 'waiting', process_review: 'not_reviewed', source: 'manual' }}> +

追加人工观察

+
({ value, label }))} />
+ {state === 'clear' &&

需正式模式试跑成功、观察已到期且操作过程核对通过，服务端才会接受这次结论。当前{duePassed ? '显示已到观察时间' : '尚未到观察时间或未配置'}，最终以服务端时间和检查为准。

} +
+ + + + + {current.mode !== 'runtime' &&

探索试跑可保留等待、提醒和不确定记录；正式质量验收需另建正式模式试跑并配置基线和观察窗口。

} +
; +} diff --git a/apps/admin/src/components/ReadingDemandAnalysis.tsx b/apps/admin/src/components/ReadingDemandAnalysis.tsx new file mode 100644 index 0000000..ffb70a8 --- /dev/null +++ b/apps/admin/src/components/ReadingDemandAnalysis.tsx @@ -0,0 +1,154 @@ +import { Alert, Button, Card, Collapse, Descriptions, Drawer, Select, Space, Table, Tag, Typography } from 'antd'; +import { useEffect, useRef, useState } from 'react'; +import { api } from '../api'; +import type { BusinessBrief, DemandAnalysis, DemandModelConfig, DemandUsage, DiscoveryReading, PageResult } from '../types'; +import { AnalysisReviewPanel, SourceReviewControl } from './AnalysisReviewPanel'; +import { BriefConditions, RegionScope } from './BusinessBriefPanel'; +import { ConsultationDetails } from './ConsultationDetails'; +import { formatTime, NoRows } from './common'; + +const demandLabels: Record = { explicit: '明确需求', possible: '疑似需求', discussion: '普通讨论', insufficient: '信息不足' }; +const missingLabels: Record = { identity_unconfirmed: '身份未核实', region_missing: '地区未提供', budget_missing: '预算未提供', stage_missing: '装修阶段未提供', media_uninterpreted: '媒体内容未解释', reading_partial: '阅读仅部分完成' }; +const errorLabels: Record = { + MODEL_TIMEOUT: '模型请求超时', MODEL_NETWORK: '模型连接未完成', MODEL_AUTH: '模型认证未通过', MODEL_RATE_LIMIT: '模型请求受限', + MODEL_QUOTA: '模型额度不足', MODEL_UNAVAILABLE: '模型服务暂时不可用', MODEL_HTTP_ERROR: '模型服务未完成请求', MODEL_RESPONSE_INVALID: '模型响应结构未通过检查', + MODEL_OUTPUT_INVALID: '需求结果或原文依据未通过检查', MODEL_RESPONSE_TOO_LARGE: '模型响应超出本次上限', MODEL_NOT_CONFIGURED: '服务端模型尚未配置', +}; +const roleLabels: Record = { suspected_demand_side: '疑似需求方', suspected_service_provider: '疑似服务方', participant: '普通参与者', unknown: '角色未知' }; +const expressionLabels: Record = { self_need: '自身需求', question: '询问', seek_advice: '寻求建议', service_offer: '提供服务', advice: '给出建议', case_share: '案例分享', other: '其他表达', unknown: '表达不明' }; +const matchLabels: Record = { matched: '条件初步匹配', possible: '可能匹配', insufficient: '条件不足', not_matched: '不匹配' }; +const nextLabels: Record = { review_author_need: '核对作者自身需求', review_comment_need: '核对本条评论者需求', review_consultation: '核对咨询意向与实际需求', inspect_comments: '查看评论中的需求信号', need_more_context: '补充上下文后再判断', skip: '暂不列入跟进' }; +const matchReasonLabels: Record = { self_need_explicit: '有明确自身需求信号', self_need_possible: '有疑似自身需求信号', no_self_need_signal: '没有自身需求信号', service_relevant: '与服务相关', service_irrelevant: '与服务不相关', service_unclear: '服务相关性不明', region_unspecified: '商家地区未设置', region_missing: '本条原文未提供地区', region_exact: '原文地区与条件字面一致', region_unrestricted: '商家已选择不限地区', region_potential_mismatch: '地区表述可能不匹配，需核对', service_promotion_excluded: '按条件排除服务推广', media_uninterpreted: '媒体内容未解释', reading_partial: '仅有部分阅读结果' }; +function Quotes({ values }: { values: string[] }) { return <>{values.length ? values.map((value, index) => 「{value}」{index + 1 < values.length ? '；' : ''}) : '无原文依据，保留未知'}; } +function safeError(code?: string | null) { return code ? errorLabels[code] || '模型请求或结果未通过检查' : '未提供错误分类'; } +function sourceHref(value: string) { try { const url = new URL(value); return ['http:', 'https:'].includes(url.protocol) && !url.username && !url.password ? url.href : undefined; } catch { return undefined; } } +function tokens(value: number | null | undefined) { return value === null || value === undefined ? '未知' : value.toLocaleString('zh-CN'); } +function Usage({ value }: { value: DemandUsage | null }) { + return 输入 Token：{tokens(value?.prompt_tokens)}输出 Token：{tokens(value?.completion_tokens)}总 Token：{tokens(value?.total_tokens)}缓存命中：{tokens(value?.prompt_cache_hit_tokens)}缓存未命中：{tokens(value?.prompt_cache_miss_tokens)}; +} +function ResultDetails({ analysis, reading }: { analysis: DemandAnalysis; reading: DiscoveryReading }) { + return
+ {analysis.status === 'completed' ? '分析完成' : '分析未完成'}{analysis.prompt_version}{formatTime(analysis.created_at)} + {analysis.status === 'failed' && } + 本次纳入 {analysis.coverage.source_count} 个来源，仅核对已保存的文字；媒体未解释，身份待核实。 + {analysis.context &&

本次商家条件

{analysis.context.conditions.service} · {analysis.context.conditions.target_customer} · {analysis.context.kind === 'confirmed_business_brief' ? '使用已确认商家需求；分析绑定该版本，不随当前表单改变。' : '使用本次初筛条件；旧记录未额外确认商家需求。'}{analysis.context.conditions.exclude_service_promotions ? ' 匹配时排除服务推广信号。' : ''}
} +

逐项需求与原文依据

{analysis.status !== 'completed' ? : analysis.results.map(result => { + const source = analysis.sources.find(item => item.source_ref === result.source_ref); + if (!source) return null; + return
+ {source.kind === 'note' ? '笔记正文' : `首层评论 ${source.comment_index ?? ''}`}{demandLabels[result.demand]}{result.media_uninterpreted && 媒体内容未解释} + + {source.comment_id && 本条评论标识：{source.comment_id}} + 已读原文：{source.text || '无可读文字，媒体内容未解释。'} + + {result.role &&
{roleLabels[result.role]}{result.expressions?.map((expression, index) => {expressionLabels[expression.type]})}{result.match && {matchLabels[result.match]}} + 角色依据： + {result.expressions?.map((expression, index) => {expressionLabels[expression.type]}依据：)} + {result.business_relevance && 服务相关性：{{ relevant: '相关', irrelevant: '不相关', unclear: '不明确' }[result.business_relevance.value]} · } + {result.match_reasons?.length ? 匹配理由：{result.match_reasons.map(value => matchReasonLabels[value] || '条件待核对').join('；')} : null} + {result.next_step && } +
} + 需求依据：{result.evidence.length ? result.evidence.map((quote, index) => 「{quote}」{index + 1 < result.evidence.length ? '；' : ''}) : '没有可核对的需求原文片段'} + ({ key: field, label: { stage: '装修阶段', region: '地区', budget: '预算' }[field], children: result[field] ? <>{result[field].value} · 原文「{result[field].evidence}」 : '未提供，未推断' }))} /> + {result.missing_information.map(item => {missingLabels[item] || '信息待核实'})} + 来源：本次已保存的{source.kind === 'note' ? '笔记正文' : '首层评论'}。{sourceHref(reading.note_url) && 查看笔记页面} +
; + })}
 +
+ {analysis.id} }, + { key: 'time', label: '分析时间', children: formatTime(analysis.created_at) }, { key: 'model', label: '本次配置型号', children: analysis.config.model }, + { key: 'provider', label: '供应商', children: analysis.config.provider }, { key: 'duration', label: '分析耗时', children: `${(analysis.duration_ms / 1000).toFixed(2)} 秒` }, + { key: 'range', label: '实际分析范围', children: `${analysis.coverage.source_count} 个来源，${analysis.coverage.text_chars} 字` }, + { key: 'bounded', label: '原阅读范围', children: analysis.coverage.reading_truncated ? '已截断，仅判断已读内容' : '本次范围未截断' }, + { key: 'comments', label: '评论', children: analysis.coverage.comments_available ? '有已读评论' : '未取得，不代表没有评论' }, + { key: 'price', label: '供应商费用', children: '未知（未接价格结算）' }, + { key: 'versions', label: '规则 / 结果版本', children: `${analysis.prompt_version} / ${analysis.result_version}` }, + ]} /> +

实际 Token 用量

仅汇总已返回用量；失败或超时未返回的用量未知，未计入汇总，费用未结算。
+

主备请求记录

{analysis.attempts.length ?
{analysis.attempts.map((attempt, index) =>
+ {attempt.slot === 'primary' ? '主密钥' : '备用密钥'}{attempt.status === 'succeeded' ? '请求成功' : '请求未通过'}{(attempt.duration_ms / 1000).toFixed(2)} 秒{attempt.http_status !== null && HTTP {attempt.http_status}} + {attempt.response_model && 实际返回型号：{attempt.response_model}} + {attempt.status === 'failed' && {safeError(attempt.error_code)}} + +
)}
: 没有模型调用记录。}
+ 阅读内容摘要：{analysis.reading_digest}
模型配置摘要：{analysis.config.config_digest} +
}]} />
+ ; +} + +export function ReadingDemandAnalysis({ reading }: { reading: DiscoveryReading }) { + const [config, setConfig] = useState(); + const [briefs, setBriefs] = useState([]); + const [briefId, setBriefId] = useState(reading.business_brief_id || ''); + const [history, setHistory] = useState([]); + const [selectedId, setSelectedId] = useState(''); + const [detailsOpen, setDetailsOpen] = useState(false); + const [loading, setLoading] = useState(true); + const [busy, setBusy] = useState(false); + const [loadError, setLoadError] = useState(false); + const [requestError, setRequestError] = useState(false); + const [reused, setReused] = useState(false); + const active = useRef(false); + const mounted = useRef(true); + const generation = useRef(0); + const route = `/discovery/readings/${encodeURIComponent(reading.id)}/analyses`; + useEffect(() => { mounted.current = true; return () => { mounted.current = false; generation.current += 1; }; }, []); + async function refresh() { + const current = ++generation.current; + setLoading(true); setLoadError(false); + const [status, records, business] = await Promise.allSettled([api.get('/analysis/config'), api.get>(route), api.get>('/business/briefs')]); + if (!mounted.current || current !== generation.current) return; + if (status.status === 'fulfilled') setConfig(status.value); else setConfig(undefined); + if (records.status === 'fulfilled') { + const items = records.value.items.filter(item => item.reading_id === reading.id && item.discovery_run_id === reading.discovery_run_id); + setHistory(items); setSelectedId(previous => items.some(item => item.id === previous) ? previous : items[0]?.id || ''); + } + if (business.status === 'fulfilled') setBriefs(business.value.items.filter(item => item.status === 'confirmed')); + setLoadError(status.status === 'rejected' || records.status === 'rejected'); setLoading(false); + } + useEffect(() => { void refresh(); }, [reading.id]); + const binding = reading.note_binding_result; + const eligible = reading.status !== 'failed' && Boolean(reading.detail?.items.length && binding?.href_checked && binding.url_checked && binding.note_id === reading.note_id); + async function analyze() { + if (active.current || !config?.configured || !eligible) return; + active.current = true; setBusy(true); setRequestError(false); setReused(false); + try { + const result = await api.post(route, briefId ? { business_brief_id: briefId } : {}); + if (!mounted.current) return; + setReused(history.some(item => item.id === result.id)); + await refresh(); + if (mounted.current && result.reading_id === reading.id && result.discovery_run_id === reading.discovery_run_id) { setSelectedId(result.id); setDetailsOpen(true); } + } catch { + if (mounted.current) { setRequestError(true); await refresh(); } + } finally { active.current = false; if (mounted.current) setBusy(false); } + } + const selected = history.find(item => item.id === selectedId); + return void refresh()}>刷新分析记录}> +
+ + {loadError && } +

当前服务器模型配置

{config ? <> + {config.configured ? '已配置' : '尚未配置'}{config.provider || '供应商未设置'}{config.model || '型号未设置'}主密钥：{config.key_slots.includes('primary') ? '已配置' : '未配置'}独立备用：{config.key_slots.includes('fallback') ? '已配置' : '未启用'} + 已配置不等于已连通。这里只显示配置状态；相同密钥会去重为一个槽位。备用仅对受支持的临时失败有限尝试，本轮不提供密钥输入或查看。 + : 尚未取得安全配置状态。}
+ {!eligible && } + {requestError && } +

分析所用商家条件

+ ; +} diff --git a/apps/admin/src/components/TestRunHistory.tsx b/apps/admin/src/components/TestRunHistory.tsx index 203098d..96ebbe8 100644 --- a/apps/admin/src/components/TestRunHistory.tsx +++ b/apps/admin/src/components/TestRunHistory.tsx @@ -7,6 +7,7 @@ import { formatTime, StatusTag, useResource } from './common'; import { LIST_PAGE_SIZE, PageControls } from './PageControls'; import { EmptyState, ResourceState } from './States'; import type { PageResult, Validation } from '../types'; +import { validationModeName } from '../learning-view-model'; export function TestRunHistory({ candidateId, currentRunId, onSelect }: { candidateId: string; currentRunId?: string; onSelect: (run: Validation) => void }) { const [open, setOpen] = useState(false); @@ -20,7 +21,7 @@ export function TestRunHistory({ candidateId, currentRunId, onSelect }: { candid
void resource.reload()} /> {resource.data && !resource.loading && !resource.error && <>{items.length ?
- {items.map(run => )}
创建时间测试用例验证类型结果操作
{formatTime(run.created_at)}{run.sample_name || '未命名测试'}{run.kind === 'static' ? '静态校验' : '浏览器试跑'}{run.id === currentRunId ? 当前记录 : }
: } + {items.map(run => {formatTime(run.created_at)}{run.sample_name || '未命名测试'}{validationModeName(run)}{run.id === currentRunId ? 当前记录 : })}
: }
}
diff --git a/apps/admin/src/components/VerificationFailureNotice.tsx b/apps/admin/src/components/VerificationFailureNotice.tsx new file mode 100644 index 0000000..80f4625 --- /dev/null +++ b/apps/admin/src/components/VerificationFailureNotice.tsx @@ -0,0 +1,66 @@ +import { Alert, Typography } from 'antd'; +import { operationName } from '../learning-view-model'; +import type { RequestFailureDiagnostics, RequestPolicyReason, VerificationFailureDiagnostics } from '../types'; + +const reasonLabels: Record = { + missing_policy: '当前动作未建立请求策略', + inactive_policy: '请求策略当前未生效', + invalid_time: '策略时间信息无效', + policy_expired: '请求策略已到期', + automation_inactive: '请求发生时已不在自动执行阶段', + session_out_of_scope: '浏览器会话不在策略范围内', + page_out_of_scope: '页面不在策略范围内', + not_main_frame: '请求来自非主页面框架', + method_not_allowed: '请求方法不在允许范围内', + page_changed: '页面地址在策略生效后发生变化', + policy_not_started: '请求发生时策略尚未开始生效', + invalid_url: '请求地址无效', + request_out_of_scope: '请求地址不在允许范围内', + resource_type_not_allowed: '请求的资源类型不在允许范围内', + navigation_out_of_scope: '页面导航不在允许范围内', +}; +const phaseLabels: Record = { + validate: '校验动作', inspect_current: '检查当前页', capture_before: '操作前取证', resolve_target: '定位目标', + execute: '执行操作', settle: '等待页面稳定', request_guard: '核对请求范围', capture_after: '操作后取证', + failure_capture: '失败取证', batch_capture: '批次取证', +}; +export const failurePhaseName = (phase?: string) => phase && phaseLabels[phase] ? phaseLabels[phase] : '未记录'; + +function safeOrigin(value?: string) { + try { const url = new URL(value || ''); return ['http:', 'https:'].includes(url.protocol) && !url.username && !url.password ? url.origin : '未记录'; } catch { return '未记录'; } +} +function safePath(value?: string) { return value?.startsWith('/') ? value.split(/[?#]/, 1)[0] : '未记录'; } + +/** Only render the API's bounded request summary fields, never raw request data. */ +export function RequestFailureDetails({ diagnostics }: { diagnostics?: RequestFailureDiagnostics }) { + const requests = diagnostics?.blocked_requests || []; + const count = diagnostics?.blocked_request_count; + return
+ 查看请求诊断详情 + 这里只展示受限请求摘要。未记录具体拒因时，不根据错误代码推断原因。 + {Boolean(count && count > 0) && 本地规则阻止请求，不等于平台已发出异常提醒或封号。未知请求的业务副作用尚未确认，不能据此认定发生了关注、私信或评论等写入。} + {requests.length ? <> + 已记录 {requests.length} 条请求摘要{count === undefined ? '' : `，阻止总数 ${count}`}；摘要不一定覆盖所有被阻止请求。 + {requests.map((request, index) =>
+
请求方法
{request.method || '未记录'}
+
来源
{safeOrigin(request.origin)}
+
路径
{safePath(request.pathname)}
+
实际资源类型
{request.resource_type || '未记录'}
+
本地拒绝原因
{request.policy_reason && reasonLabels[request.policy_reason] ? reasonLabels[request.policy_reason] : '未记录具体拒因'}{request.policy_reason && reasonLabels[request.policy_reason] && <>（{request.policy_reason}）}
+
当时策略版本
{request.active_policy_version || '未记录'}
+ {request.expected_resource_types?.length ? <>
规则接受的资源类型
{request.expected_resource_types.map((type, typeIndex) => {typeIndex > 0 ? '、' : ''}{type})}
: null} +
)} + : 本次未记录具体的被阻止请求摘要。} +
; +} + +export function VerificationFailureNotice({ failure }: { failure?: VerificationFailureDiagnostics }) { + if (!failure) return ; + return + 失败阶段：{failurePhaseName(failure.diagnostics.phase)} · 阻止请求总数：{failure.diagnostics.blocked_request_count ?? '未记录'} · 错误代码：{failure.code} + {Boolean(failure.diagnostics.blocked_request_count && failure.diagnostics.blocked_request_count > 0) && 这是执行器本地规则的阻断记录，不代表平台已发出异常提醒或账号被封禁。} + + } />; +} diff --git a/apps/admin/src/components/common.tsx b/apps/admin/src/components/common.tsx index 02655bc..1596800 100644 --- a/apps/admin/src/components/common.tsx +++ b/apps/admin/src/components/common.tsx @@ -34,9 +34,10 @@ const statusLabels: Record = { candidate: '待验证', pending: '待验证', checked: '校验通过', static_passed: '校验通过', validated: '试跑通过', validation_failed: '验证未通过', verified: '已有通过记录', published: '已发布', passed: '通过', failed: '失败', rejected: '未通过', running: '进行中', + experimental: '探索中', awaiting_observation: '等待后续观察', verified_in_scope: '维护范围内已验证', review_required: '待复核', }; export function StatusTag({ status }: { status: string }) { - return {statusLabels[status] || status}; + return {statusLabels[status] || status}; } export const platformLabels: Record = { demo: '本机示例', xiaohongshu: '小红书', douyin: '抖音', other: '其他平台' }; export function formatTime(value?: string) { diff --git a/apps/admin/src/learning-view-model.ts b/apps/admin/src/learning-view-model.ts index 8813333..6ce45d6 100644 --- a/apps/admin/src/learning-view-model.ts +++ b/apps/admin/src/learning-view-model.ts @@ -1,6 +1,6 @@ // Local view contracts for the migrated A2A console components. -// They deliberately contain only fields backed by this workbench's SQLite API. -import type { Candidate } from './types'; +// They contain only fields backed by this workbench's local learning API. +import type { Candidate, RecordedStep, Validation } from './types'; export interface InputGuard { match: 'any' | 'all'; conditions: Array<{ input_key: string; operator: string; value: unknown }> } export interface CapabilityParameterUsage { kind: string; step_id?: string; binding_ref?: string; op?: string; guard?: InputGuard } @@ -20,14 +20,29 @@ export interface CapabilityDetail { input_contract?: { conditional_required?: Array<{ required: string[]; when: InputGuard }>; dependentRequired?: Record; field_relations?: Array<{ left: string; right: string; operator: string }> }; }; } -export function operationName(kind: string) { return ({ fill: '填写参数', click: '点击元素', read: '读取内容' } as Record)[kind] || kind; } +export function operationName(kind: string) { return ({ fill: '填写参数', press: '提交搜索（回车）', click: '点击元素', read: '读取内容', scroll: '有限滚动', wait: '等待页面状态', extract: '结构化提取' } as Record)[kind] || kind; } +export function operationSummary(step: RecordedStep) { + const target = step.target ? ` · 精确匹配${step.target.parameter ? `参数 ${step.target.parameter}` : step.target.value || ''}` : ''; + if (step.kind === 'press') return '搜索框 · Enter'; + if (step.kind === 'scroll') return `${step.scroll_scope === 'viewport' ? '整页 · ' : ''}${step.direction === 'up' ? '向上' : '向下'} ${step.pixels ?? 600}px × 最多 ${step.max_scrolls ?? 1} 次${step.until_target ? ` · 查找${step.until_target.parameter ? `参数 ${step.until_target.parameter}` : '固定目标'} · 最长 ${step.timeout_ms ?? 0}ms` : ''}`; + if (step.kind === 'wait') return `${({ visible: '可见', hidden: '消失', text: '包含文字' })[step.state || 'visible']}${step.state === 'visible' && typeof step.min_count === 'number' ? ` · 至少 ${step.min_count} 项${step.wait_scope ? ` · 范围 ${step.wait_scope}` : ''}` : ''} · 最长 ${step.timeout_ms ?? 5000}ms`; + if (step.kind === 'extract') return `输出 ${step.output_name || 'results'} · ${step.fields?.length || 0} 个字段 · 最多 ${step.limit ?? 20} 项${target}`; + return `${step.parameter ? `参数 ${step.parameter}` : step.output_name ? `输出 ${step.output_name}` : step.value || '固定操作'}${target}`; +} +export function validationModeName(record: Validation) { + if (record.kind === 'route') return '链路组合验证'; + return record.kind === 'static' ? '固定规则' : record.mode === 'runtime' ? '正式模式试跑' : '探索试跑'; +} export function parameterView(candidate: Candidate): CapabilityDetail { return { steps: candidate.manifest.steps.map((step, index) => ({ step_id: String(index + 1), op: step.kind })), parameter_catalog: { execution_usage_complete: true, business_coverage_state: 'unverified', parameters: Object.entries(candidate.manifest.parameters).map(([key, value]) => { - const usages = candidate.manifest.steps.flatMap((step, index) => step.parameter === key ? [{ kind: 'action_input', step_id: String(index + 1), op: step.kind }] : []); + const usages = candidate.manifest.steps.flatMap((step, index) => [ + ...(step.parameter === key ? [{ kind: 'action_input', step_id: String(index + 1), op: step.kind }] : []), + ...(step.target?.parameter === key ? [{ kind: 'target_match', step_id: String(index + 1), op: step.kind }] : []), + ]); return { key, label: key, description: usages.length ? '已绑定到固定操作步骤' : '已登记样本，尚未绑定操作步骤', value_type: typeof value === 'boolean' ? 'boolean' : Number.isInteger(value) ? 'integer' : Array.isArray(value) ? 'array' : value !== null && typeof value === 'object' ? 'object' : 'string', required: usages.length > 0, schema_required: false, execution_required: usages.length > 0, requires_non_empty: false, diff --git a/apps/admin/src/pages/CandidatesPage.tsx b/apps/admin/src/pages/CandidatesPage.tsx index 53bf077..60a906d 100644 --- a/apps/admin/src/pages/CandidatesPage.tsx +++ b/apps/admin/src/pages/CandidatesPage.tsx @@ -1,7 +1,7 @@ // Migrated from A2A console/pages/CapabilitiesPage.tsx: capability table, // fixed-version detail, parameter/operation/evidence tabs and test history. // Local candidate contracts replace generated A2A package and consumption contracts. -import { App, Button, Drawer, Form, Input, Modal, Select, Tabs, Typography } from 'antd'; +import { Alert, App, Button, Drawer, Form, Input, InputNumber, Modal, Select, Tabs, Typography } from 'antd'; import { ChevronLeft, ChevronRight, Play, RefreshCw, Search } from 'lucide-react'; import { useEffect, useMemo, useState } from 'react'; import { api, errorText } from '../api'; @@ -11,12 +11,13 @@ import { EmptyState, ResourceState } from '../components/States'; import { PageControls, LIST_PAGE_SIZE } from '../components/PageControls'; import { ScriptParameterTable } from '../components/ScriptParameterTable'; import { TestRunHistory } from '../components/TestRunHistory'; -import { operationName, parameterView } from '../learning-view-model'; +import { operationName, operationSummary, parameterView } from '../learning-view-model'; import { ValidationDetails, ValidationTable } from './ValidationsPage'; +import { ExecutionContractDetails } from '../components/ExecutionContractDetails'; import type { Candidate, LearningTask, PageResult, PublishCheck, RecordedStep, Validation } from '../types'; function expectedTemplate(candidate: Candidate) { - return JSON.stringify(Object.fromEntries(candidate.manifest.steps.filter(step => step.kind === 'read').map(step => [step.output_name || 'results', ''])), null, 2); + return JSON.stringify(Object.fromEntries(candidate.manifest.steps.filter(step => step.kind === 'read' || step.kind === 'extract').map(step => [step.output_name || 'results', ''])), null, 2); } function CapabilityPage({ candidate, navigate, onRefresh }: { candidate: Candidate; navigate: (path: string) => void; onRefresh: () => unknown }) { @@ -28,6 +29,7 @@ function CapabilityPage({ candidate, navigate, onRefresh }: { candidate: Candida const [record, setRecord] = useState(); const [gate, setGate] = useState(); const [form] = Form.useForm(); + const mode = Form.useWatch('mode', form); const validations = useResource>('/learning/validations'); const task = useResource(`/learning/tasks/${candidate.task_id}`); const records = (validations.data?.items || []).filter(item => item.candidate_id === candidate.id); @@ -45,7 +47,7 @@ function CapabilityPage({ candidate, navigate, onRefresh }: { candidate: Candida const parameters = parseObject(values.parameters, '本次参数'); const expected_outputs = parseObject(values.expected_outputs, '预期输出'); setBusy('verify'); - const result = await api.post(`/learning/candidates/${candidate.id}/verify`, { parameters, expected_outputs, sample_name: values.sample_name }); + const result = await api.post(`/learning/candidates/${candidate.id}/verify`, { parameters, expected_outputs, sample_name: values.sample_name, mode: values.mode || 'exploration', ...(values.mode === 'runtime' ? { baseline_note: values.baseline_note.trim(), observation_window_minutes: values.observation_window_minutes } : {}) }); setVerifyOpen(false); setRecord(result); await validations.reload(); void onRefresh(); void task.reload(); void message[result.status === 'passed' ? 'success' : 'warning'](result.status === 'passed' ? '实际试跑通过，记录已保存' : '试跑未通过，请查看实际结果'); } catch (error) { if (!(error && typeof error === 'object' && 'errorFields' in error)) void message.error(errorText(error)); } @@ -57,17 +59,24 @@ function CapabilityPage({ candidate, navigate, onRefresh }: { candidate: Candida catch (error) { void message.error(errorText(error)); } finally { setBusy(''); } } + function openVerification(mode: 'exploration' | 'runtime') { + void task.reload(); + form.setFieldValue('mode', mode); + setVerifyOpen(true); + } return

{candidate.name}

固定候选版本 v{candidate.version}

- + +
-
状态 创建于 {formatTime(candidate.created_at)}内容摘要 {candidate.digest.slice(0, 16)}
+
状态 运行质量 创建于 {formatTime(candidate.created_at)}内容摘要 {candidate.digest.slice(0, 16)}
+ }, - { key: 'steps', label: '操作链', children:
{candidate.manifest.steps.length} 步 · 固定顺序
{candidate.manifest.steps.map((step, index) => )}
顺序操作参数 / 输出详情
{index + 1}{operationName(step.kind)}{step.parameter ? `参数 ${step.parameter}` : step.output_name ? `输出 ${step.output_name}` : '固定操作'}
}, + { key: 'steps', label: '操作链', children:
{candidate.manifest.steps.length} 步 · 固定顺序
{candidate.manifest.steps.map((step, index) => )}
顺序操作参数 / 输出详情
{index + 1}{operationName(step.kind)}{operationSummary(step)}
}, { key: 'evidence', label: '定位与证据', children:

步骤依据

对应学习时的实际观察
{candidate.manifest.steps.map((step, index) => )}
步骤页面定位证据标识
{index + 1}{step.selector}{step.evidence_id || '未关联'}
}, { key: 'schemas', label: '能力清单', children:

{candidate.digest}

 }, { key: 'tests', label: `测试记录 ${records.length}`, children:
void validations.reload()} />
}, @@ -77,14 +86,20 @@ function CapabilityPage({ candidate, navigate, onRefresh }: { candidate: Candida

按本次参数重放固定步骤，结果与预期逐项对照并留档。

void task.reload()} /> {task.data?.browser_readiness && !task.data.browser_readiness.ready &&

{task.data.browser_readiness.message}

} -
+ + - +
setGate(undefined)} destroyOnHidden>{gate && <>

{gate.ready ? '当前预检条件已满足' : '仍有条件需要补齐'}

当前结果为即时预检；满足条件后才能进入后续发布流程。

{gate.checks.map(item => )}
检查项结论说明
{item.name}{item.detail}
} - setRecord(undefined)} destroyOnHidden>{record && } + setRecord(undefined)} destroyOnHidden>{record && { setRecord(updated); void validations.reload(); void onRefresh(); }} />}
; } @@ -99,7 +114,7 @@ export function CandidatesPage({ selectedId, navigate }: { selectedId?: string; if (selectedId) return selected ? : <> void resource.reload()} />; return

能力实体

{resource.data ? `${all.length} 条候选版本 · 本页 ${items.reduce((sum, item) => sum + item.manifest.steps.length, 0)} 个步骤` : '查看固定版本与操作链'}

} value={search} maxLength={200} allowClear placeholder="搜索能力名称或标识" onChange={event => { setSearch(event.target.value); setOffset(0); }} />
-
void resource.reload()} />{resource.data && !resource.loading && !resource.error && <>{items.length ?
{items.map(candidate => )}
能力名称固定版本步骤验证状态内容摘要操作
{candidate.name}v{candidate.version}{candidate.manifest.steps.length}{candidate.digest.slice(0, 12)}
: } +
void resource.reload()} />{resource.data && !resource.loading && !resource.error && <>{items.length ?
{items.map(candidate => )}
能力名称固定版本步骤验证状态运行质量内容摘要操作
{candidate.name}v{candidate.version}{candidate.manifest.steps.length}{candidate.digest.slice(0, 12)}
: }
}
; diff --git a/apps/admin/src/pages/DiscoveryPage.tsx b/apps/admin/src/pages/DiscoveryPage.tsx new file mode 100644 index 0000000..4a3ba2f --- /dev/null +++ b/apps/admin/src/pages/DiscoveryPage.tsx @@ -0,0 +1,255 @@ +import { Alert, App, Button, Card, Descriptions, Form, Image, Input, InputNumber, Select, Space, Table, Tag, Typography } from 'antd'; +import { useEffect, useMemo, useRef, useState } from 'react'; +import { api, errorText } from '../api'; +import { LoadError, NoRows, Page, formatTime, useResource } from '../components/common'; +import { DetailDrawer } from '../components/DetailDrawer'; +import { ExecutionTimings } from '../components/ExecutionTimings'; +import { BusinessBriefPanel, BriefConditions } from '../components/BusinessBriefPanel'; +import { ReadingDemandAnalysis } from '../components/ReadingDemandAnalysis'; +import { VerificationFailureNotice } from '../components/VerificationFailureNotice'; +import { ExecutionContractDetails } from '../components/ExecutionContractDetails'; +import type { BusinessBrief, Candidate, ContentValue, DiscoveryReading, DiscoveryResult, DiscoveryRun, PageResult, ReadingOutput } from '../types'; + +interface DiscoveryValues { search_candidate_id: string; keyword: string; service: string; target_customer: string; region: string; max_selected: number } +const classificationLabels: Record = { consumer_help: '装修求助信号', related_content: '装修相关内容', peer_case: '同行 / 案例', unrelated: '暂不相关', ambiguous: '信息不足', invalid: '来源无效' }; +const statusLabels: Record = { completed: '初筛完成', search_failed: '搜索未完成', failed: '未完成' }; + +function matchesSearchShape(candidate: Candidate) { + if (!['xiaohongshu', 'demo'].includes(candidate.manifest.platform || '')) return false; + const [fill, press, wait, extract] = candidate.manifest.steps; + if (candidate.manifest.steps.length !== 4 || fill.kind !== 'fill' || fill.parameter !== 'keyword' || press.kind !== 'press' || press.key !== 'Enter' || wait.kind !== 'wait' || wait.state !== 'visible' || !wait.min_count || extract.kind !== 'extract' || extract.output_name !== 'search_results') return false; + const fields = extract.fields || []; + return ['title', 'author'].every(name => fields.some(field => field.name === name && field.source === 'text' && !field.required)) && fields.some(field => field.name === 'url' && field.source === 'href' && field.required); +} +function sourceHref(value?: string | null) { + if (!value) return undefined; + try { const url = new URL(value); return ['http:', 'https:'].includes(url.protocol) && !url.username && !url.password ? url.href : undefined; } catch { return undefined; } +} +function RunStatus({ status }: { status: DiscoveryRun['status'] }) { return {statusLabels[status]}; } + +const readingStatusLabels: Record = { completed: '读取完成', partial: '部分完成', failed: '读取失败' }; +const seekStopLabels: Record = { target_reached: '目标已进入视口', boundary: '到达滚动边界', scroll_budget: '滚动次数预算耗尽', time_budget: '查找时间预算耗尽', ambiguous: '目标重复，无法唯一确定', obscured: '目标被遮挡', target_changed: '目标发生变化', source_changed: '页面来源发生变化', unsafe_target: '目标不符合安全范围', binding_mismatch: '目标来源绑定不符', session_unavailable: '会话不可用', blocked_request: '请求被阻止' }; +function ReadingStatus({ status }: { status: DiscoveryReading['status'] }) { return {readingStatusLabels[status]}; } +function matchesDetailShape(candidate: Candidate, run: DiscoveryRun, candidates: Candidate[]) { + const search = candidates.find(item => item.id === run.search_candidate_id); + const steps = candidate.manifest.steps; + if (!search || candidate.manifest.platform !== search.manifest.platform || ![7, 8].includes(steps.length) || !Object.hasOwn(candidate.manifest.parameters, 'keyword') || !Object.hasOwn(candidate.manifest.parameters, 'note_title')) return false; + const expected = steps.length === 8 ? ['fill', 'press', 'wait', 'scroll', 'click', 'wait', 'extract', 'extract'] : ['fill', 'press', 'wait', 'click', 'wait', 'extract', 'extract']; + if (steps.some((step, index) => step.kind !== expected[index])) return false; + const click = steps[steps.length === 8 ? 4 : 3]; + const [fill, press] = steps; + const [detail, comments] = steps.slice(-2); + if (fill.parameter !== 'keyword' || press.key !== 'Enter' || click.target?.parameter !== 'note_title' || detail.output_name !== 'note_detail' || comments.output_name !== 'comments') return false; + if (steps.length === 8) { + const seek = steps[3]; + if (!seek.until_target || seek.until_selector || seek.selector === 'body' || seek.until_target.selector !== click.selector || seek.until_target.field !== 'text' || seek.until_target.parameter !== 'note_title' || click.target.field !== seek.until_target.field || seek.until_target.value !== click.target.value || seek.until_target.attribute !== click.target.attribute) return false; + } + try { return new URL(candidate.manifest.entry_url).origin === new URL(run.selected[0]?.note_url || '').origin; } catch { return false; } +} +function ReadingFields({ output, testId }: { output: ReadingOutput; testId: string }) { + const fieldLabels: Record = { title: '标题', body: '正文', text: '文字', author: '作者', source: '来源', url: '来源', author_url: '作者来源', comment_id: '评论标识' }; + return
{output.items.map((item, index) =>
{Object.entries(item).map(([key, value]) =>
{fieldLabels[key] || key}
{value === null ? '未提供' : typeof value === 'object' ? : /(?:^|_)(?:url|source)$/.test(key) && sourceHref(value) ? {value} : value}
)}
)}
; +} +function TypedContent({ value }: { value: ContentValue }) { + const labels: Record = { text: '文字内容', media: '媒体内容', mixed: '文字与媒体混合' }; + return
+ {labels[value.kind]}{value.media.length > 0 && 媒体内容未解释}{value.media_truncated && 媒体记录范围已截断} + {value.text ?? '本项没有可读文字；不能据此判断媒体含义或客户需求。'} + {value.media.length > 0 && <>
    {value.media.map((media, index) =>
  • {media.kind === 'image' ? '图片' : '视频'} · {media.selector}
    • )}
这里只记录实际媒体类型与定位，未执行 OCR 或内容解释。请通过本次阅读证据核对真实批次截图；截图不是媒体内容的文字识别结果。} +
; +} +export function ReadingDetails({ reading, navigate }: { reading: DiscoveryReading; navigate: (path: string) => void }) { + const detail = reading.status === 'failed' ? null : reading.detail; + const comments = reading.status === 'failed' ? null : reading.comments; + const binding = reading.note_binding_result; + return } data-testid="discovery-reading-detail"> +
+ {reading.id} + + + {reading.status !== 'completed' && } + {sourceHref(reading.note_url) && 本篇笔记来源} + + +

笔记对象绑定核对

{binding ? {binding.source_url} : '未取得' }, + ]} /> : 本次未形成可用的绑定核对结果，不能据此认定对象已通过核对。}
+ {reading.seek_results &&

目标进入视口记录

+ 这里只记录有界滚动查找阶段，阅读是否完成仍以本条状态、对象绑定及正文和评论结果为准。 + {reading.seek_results.length ? reading.seek_results.map((seek, index) =>
+ 第 {seek.step_position} 步{seek.scrolls} 次滚动{seek.target_reached ? '目标已进入视口' : '目标未确认进入视口'}{seekStopLabels[seek.stop_reason] || seek.stop_reason} + {seek.movements.length ? {seek.movements.map((move, moveIndex) => )}
滚动前位置滚动后位置实际位移容器 / 内容高度
{move.before}{move.after}{move.after - move.before}{move.client_height} / {move.scroll_height}
: 本次没有发生滚动位移。} +
) : 本次尚未记录到动态目标滚动阶段，不代表目标已找到。} +
} + {reading.status !== 'completed' && } + {(reading.errors.length > 0 || reading.error_codes.length > 0) &&
    {reading.errors.map((error, index) =>
  • {error}
    • )}
{reading.error_codes.map((code, index) => {code})}
} +

笔记正文

{detail ? : }
+

有界首层评论

{comments ? <> + {comments.count} 条已读{comments.truncated ? '范围已截断' : '本次范围未截断'}{comments.batch_summary && 读取 {comments.batch_summary.batches_read} 批，去重 {comments.batch_summary.duplicates_removed} 次} + 这里只表示本次读取范围；批数、截断和结束原因不代表已读完平台全部评论。{comments.batch_summary ? ` 结束原因：${comments.batch_summary.stop_reason}` : ''} + {comments.count === 0 && 本次未读到完整评论，不代表平台没有评论，也不算已确认空态。} + {sourceHref(comments.source_url) && 评论页面来源} + + : }
+ {reading.detail_task_id && } + 候选摘要：{reading.candidate_digest || '本次未取得候选摘要'} + + {reading.evidence.length ?
{reading.evidence.map((item, index) =>
{item.screenshot_url && {`阅读证据}{item.title || '阅读页面证据'}{sourceHref(item.url) && 页面来源}
)}
: } + +
+ ; +} +function DiscoveryReadings({ run, candidates, candidatesLoading, selectedReadingId, navigate }: { run: DiscoveryRun; candidates: Candidate[]; candidatesLoading: boolean; selectedReadingId?: string; navigate: (path: string) => void }) { + const available = useMemo(() => candidates.filter(candidate => matchesDetailShape(candidate, run, candidates)), [candidates, run]); + const history = useResource>(`/discovery/runs/${encodeURIComponent(run.id)}/readings`); + const selected = useResource(selectedReadingId ? `/discovery/readings/${encodeURIComponent(selectedReadingId)}` : null); + const [candidateId, setCandidateId] = useState(); + const selectedCandidate = available.find(candidate => candidate.id === candidateId); + const [busyNote, setBusyNote] = useState(''); + const [requestError, setRequestError] = useState(''); + const requestActive = useRef(false); + const mounted = useRef(true); + useEffect(() => { mounted.current = true; return () => { mounted.current = false; }; }, []); + useEffect(() => { if (!candidateId && available.length) setCandidateId(available[0].id); }, [available, candidateId]); + async function read(noteId: string) { + if (requestActive.current || !candidateId) return; + requestActive.current = true; setBusyNote(noteId); setRequestError(''); + try { + const result = await api.post(`/discovery/runs/${run.id}/readings`, { detail_candidate_id: candidateId, note_id: noteId }); + if (!mounted.current) return; + await history.reload(); + if (mounted.current) navigate(`/discovery/${run.id}?reading=${encodeURIComponent(result.id)}`); + } catch (error) { if (mounted.current) { setRequestError(`${errorText(error)}。请先刷新读取历史核对，不会自动重试。`); await history.reload(); } } + finally { requestActive.current = false; if (mounted.current) setBusyNote(''); } + } + const selectedRecord = selected.data; + const reading = selectedRecord && selectedRecord.id === selectedReadingId && selectedRecord.discovery_run_id === run.id ? selectedRecord : undefined; + const foreign = Boolean(selectedRecord && selectedRecord.id === selectedReadingId && selectedRecord.discovery_run_id !== run.id); + return <> + + 按不可变初筛记录的标题与来源，每次读取一篇。相关同行案例仍可作为评论来源；本次初筛使用规则，身份和需求未核实。 + {run.selected.length > 0 &&
({ value: candidate.id, label: `${candidate.name} · v${candidate.version}` }))} /> +
+ +
+ + 使用所选学习任务的已确认浏览器会话。需要登录时前往学习会话，由你在官方页面自行处理；这里不收集平台登录资料。 +
+ + {selectedId && <>{detail.loading && }{detail.data && }} + + + data-testid="discovery-history" loading={history.loading} rowKey="id" dataSource={history.data?.items || []} pagination={{ pageSize: 5, showSizeChanger: false }} scroll={{ x: 700 }} size="small" locale={{ emptyText: '尚无初筛历史，提交后会保留条件、判断和证据。' }} columns={[ + { title: '运行时间', dataIndex: 'created_at', key: 'time', render: (value: string) => formatTime(value) }, + { title: '搜索词', dataIndex: 'keyword', key: 'keyword' }, + { title: '状态', key: 'status', render: (_: unknown, row) => }, + { title: '保留笔记', key: 'selected', render: (_: unknown, row) => `${row.selected.length} / ${row.max_selected}` }, + { title: '详情', key: 'open', render: (_: unknown, row) => }, + ]} /> + +
+ ; +} diff --git a/apps/admin/src/pages/PlatformPathsPage.tsx b/apps/admin/src/pages/PlatformPathsPage.tsx new file mode 100644 index 0000000..bfbddfa --- /dev/null +++ b/apps/admin/src/pages/PlatformPathsPage.tsx @@ -0,0 +1,75 @@ +import { Alert, Button, Card, Input, Select, Space, Spin, Tag, Typography } from 'antd'; +import { useCallback, useEffect, useMemo, useRef, useState } from 'react'; +import { api, errorText } from '../api'; +import { formatTime, LoadError, NoRows, Page } from '../components/common'; +import { buildPathCoverage, isCoverageTask, matchedRouteValidations, PATH_GROUPS, type PathCoverage } from '../platform-paths'; +import type { LearningTask, PageResult, RouteCandidate, Validation } from '../types'; + +interface Snapshot { tasks: LearningTask[]; detailedTasks: LearningTask[]; routes: RouteCandidate[]; validations: Validation[]; detailFailures: number; } +const empty: Snapshot = { tasks: [], detailedTasks: [], routes: [], validations: [], detailFailures: 0 }; + +function FeatureItem({ item, navigate }: { item: PathCoverage; navigate: (path: string) => void }) { + const source = item.evidence[0]; + return
+ {item.path.title}{source ? item.path.scope === 'page' ? '已观察页面' : '已观察入口' : '待探索'} + {item.path.description} + {source ? <> +

{source.pageLabel} · 可见元素 {source.visibleElements} 项 · 本入口匹配 {source.matchingElements} 项 · 已保存截图证据

+ {formatTime(source.observedAt)} · {item.evidence.length} 份观察 + : 尚无本项可核对的页面、可见元素与截图证据。} + {item.path.scope === 'external' &&

外站未进入 · 未学习外站功能

} + {['profile.follow_entry', 'profile.message_entry', 'messages'].includes(item.path.id) &&

这里只记录入口观察，不代表已关注、已发送或链路已固化。

} +
; +} + +export function PlatformPathsPage({ navigate }: { navigate: (path: string) => void }) { + const [snapshot, setSnapshot] = useState(empty); + const [loading, setLoading] = useState(true); + const [error, setError] = useState(''); + const [filter, setFilter] = useState<'all' | 'observed' | 'pending'>('all'); + const [query, setQuery] = useState(''); + const generation = useRef(0); + const reload = useCallback(async () => { + const request = ++generation.current; + setLoading(true); setError(''); setSnapshot(empty); + try { + const [tasks, routes, validations] = await Promise.all([ + api.get>('/learning/tasks'), api.get>('/learning/routes'), api.get>('/learning/validations'), + ]); + const details = await Promise.allSettled(tasks.items.filter(isCoverageTask).map(task => api.get(`/learning/tasks/${encodeURIComponent(task.id)}`))); + if (request !== generation.current) return; + setSnapshot({ tasks: tasks.items, routes: routes.items, validations: validations.items, detailedTasks: details.filter((item): item is PromiseFulfilledResult => item.status === 'fulfilled').map(item => item.value), detailFailures: details.filter(item => item.status === 'rejected').length }); + } catch (err) { if (request === generation.current) setError(errorText(err)); } + finally { if (request === generation.current) setLoading(false); } + }, []); + useEffect(() => { void reload(); return () => { generation.current += 1; }; }, [reload]); + const coverage = useMemo(() => buildPathCoverage(snapshot.detailedTasks), [snapshot.detailedTasks]); + const observed = coverage.filter(item => item.status === 'observed').length; + const reusable = useMemo(() => matchedRouteValidations(snapshot.tasks, snapshot.routes, snapshot.validations), [snapshot]); + const filtered = coverage.filter(item => (filter === 'all' || item.status === filter) && `${item.path.title} ${item.path.id} ${item.path.description}`.toLowerCase().includes(query.trim().toLowerCase())); + function branch(item: PathCoverage, all: PathCoverage[]) { + const children = all.filter(child => child.path.parentId === item.path.id); + return
  • {children.length > 0 &&
      {children.map(child => branch(child, all))}
    }
    • ; + } + return void reload()} data-testid="platform-paths-refresh">刷新证据}> +
    + + + {snapshot.detailFailures > 0 && } + {loading ?

    正在核对已有任务与观察证据…

    : <> + 功能模板 {coverage.length} 项已观察 {observed} 项待探索 {coverage.length - observed} 项已读盘点任务 {snapshot.detailedTasks.length} 个统计的是功能模板，内容对象和账号范围不作全量枚举。页面刷新仅读取账本，不操作平台或调用模型。 + + {reusable.length ? <>搜索 → 笔记正文 → 首层评论：{reusable.length} 条同任务、同固定版本及实际阶段引用一致的探索通过记录。{reusable.slice(0, 3).map((record, index) => )}仅为记录中声明范围的探索通过；其他入口仍按下方实际观察状态展示，未正式发布。 : 尚未读回可对应同任务、固定版本及实际阶段记录的组合探索证据。已有历史可到链路复用中核对。} + + setQuery(event.target.value)} allowClear maxLength={150} style={{ width: 'min(280px, 100%)', maxWidth: '100%' }} /> + {!filtered.length && } + {PATH_GROUPS.map(group => { + const members = filtered.filter(item => item.path.group === group.id); + if (!members.length) return null; + const visibleIds = new Set(members.map(item => item.path.id)); + return
    {group.title}
      {members.filter(item => !item.path.parentId || !visibleIds.has(item.path.parentId)).map(item => branch(item, members))}
    ; + })} + } +
    +     ; +} diff --git a/apps/admin/src/pages/RoutesPage.tsx b/apps/admin/src/pages/RoutesPage.tsx new file mode 100644 index 0000000..02b0b22 --- /dev/null +++ b/apps/admin/src/pages/RoutesPage.tsx @@ -0,0 +1,144 @@ +import { Alert, Button, Card, Collapse, Descriptions, Drawer, Form, Input, Select, Space, Table, Tag, Typography } from 'antd'; +import { useRef, useState } from 'react'; +import { api, errorText } from '../api'; +import { JsonView, LoadError, NoRows, Page, StatusTag, formatTime, platformLabels, useResource } from '../components/common'; +import { ValidationDetails } from './ValidationsPage'; +import type { PageResult, RouteCandidate, RoutePlan, RoutePlatform, RouteValidation, Validation } from '../types'; + +const routeStatusLabels = { completed: '链路完成', partial: '部分完成', failed: '链路未完成' }; +const taskType = 'search_note_comments' as const; +function object(value: unknown): Record | null { return value && typeof value === 'object' && !Array.isArray(value) ? value as Record : null; } +function sourceHref(value: unknown) { + if (typeof value !== 'string') return undefined; + try { const url = new URL(value); return ['http:', 'https:'].includes(url.protocol) && !url.username && !url.password ? url.href : undefined; } catch { return undefined; } +} +function OutputValue({ value }: { value: unknown }) { + const content = object(value); + if (content && ['text', 'media', 'mixed'].includes(String(content.kind))) return <>

    {typeof content.text === 'string' ? content.text : '没有可读文字'}

    {Array.isArray(content.media) && content.media.length > 0 && 媒体内容未解释 · {content.media.length} 项}; + const href = sourceHref(value); + if (href) return {String(value)}; + return <>{value === null || value === undefined ? '未取得' : typeof value === 'object' ? : String(value)}; +} +function BoundedOutput({ title, value, testId }: { title: string; value: unknown; testId: string }) { + const output = object(value); + const rows = Array.isArray(output?.items) ? output.items : []; + const batch = object(output?.batch_summary); + return

    {title}

    {output ? <> + {typeof output.count === 'number' ? output.count : rows.length} 条已读{output.truncated === true ? '范围已截断' : output.truncated === false ? '本次范围未截断' : '截断状态未记录'}{typeof output.coverage === 'string' && 范围：{output.coverage}} + 仅展示本次有界读取，不能据此认定已读完平台内容。{batch && ` 结束原因：${String(batch.stop_reason || '未记录')}；读取批次：${String(batch.batches_read ?? '未记录')}。`} + {sourceHref(output.source_url) && 本次页面来源} + {rows.length ? rows.map((item, index) =>
    {Object.entries(object(item) || {}).map(([name, field]) =>
    {({ title: '标题', body: '正文', text: '文字', author: '作者', url: '来源', source: '来源', comment_id: '评论标识', author_url: '作者来源' } as Record)[name] || name}：
    )}
    ) : } + : }
    ; +} +function RouteResult({ record, navigate, onValidation }: { record: RouteValidation; navigate: (path: string) => void; onValidation: (id: string) => void }) { + const output = record.usable_outputs || {}; + return
    + {routeStatusLabels[record.route_status] || record.route_status} }, + { key: 'keyword', label: '搜索词', children: record.keyword }, + { key: 'id', label: '记录标识', children: record.id, span: { xs: 1, md: 2 } }, + { key: 'note', label: '实际目标', children: record.selected?.note_title || '未取得' }, + { key: 'model', label: '模型调用', children: `${record.model_calls ?? 0} 次` }, + { key: 'published', label: '验证范围', children: '维护探索验证 · 未正式发布', span: { xs: 1, md: 2 } }, + ]} /> + {record.status !== 'passed' && } + {Boolean(record.errors?.length || record.error_codes?.length) &&

    失败原因

    {record.errors?.map((error, index) =>

    {typeof error === 'string' ? error : JSON.stringify(error)}

    )}{record.error_codes?.map(code => {code})}
    } + {sourceHref(record.selected?.note_url) && 实际目标笔记来源} + {record.search_validation_id && }{record.detail_validation_id && }{record.route?.search && }{record.route?.detail && } + + + +

    已锁定链路

    阶段取证输出（可能未通过来源或字段校验）

    此处为失败排查材料，不代表可用正文或评论。上方仅展示通过来源与本项字段校验的输出。 }]} /> +
    ; +} +export function RoutesPage({ selectedValidationId, navigate }: { selectedValidationId?: string; navigate: (path: string) => void }) { + const candidates = useResource>('/learning/routes'); + const validations = useResource>('/learning/validations'); + const [platform, setPlatform] = useState('xiaohongshu'); + const [searchId, setSearchId] = useState(); + const [detailId, setDetailId] = useState(); + const [keyword, setKeyword] = useState(''); + const [noteId, setNoteId] = useState(''); + const [plan, setPlan] = useState(); + const [planInputs, setPlanInputs] = useState<{ keyword: string; noteId: string }>(); + const [busy, setBusy] = useState<'resolve' | 'verify' | null>(null); + const [error, setError] = useState(''); + const operation = useRef(false); + const [latest, setLatest] = useState(); + const [linkedId, setLinkedId] = useState(); + const available = (candidates.data?.items || []).filter(item => item.platform === platform && item.static_status === 'passed' && !item.static_errors?.length); + const searches = available.filter(item => item.ability === 'search_notes'); + const search = searches.find(item => item.candidate_id === searchId); + const details = available.filter(item => item.ability === 'read_note_comments' && search && item.task_id === search.task_id && (item.execution_profile || '') === (search.execution_profile || '') && item.source_origin === search.source_origin); + const detail = details.find(item => item.candidate_id === detailId); + const history = (validations.data?.items || []).filter((item): item is RouteValidation => item.kind === 'route'); + const selected = selectedValidationId ? history.find(item => item.id === selectedValidationId) || (latest?.id === selectedValidationId ? latest : undefined) : undefined; + const linked = validations.data?.items.find(item => item.id === linkedId); + const noteIdValid = !noteId.trim() || /^[a-fA-F0-9]{24}$/.test(noteId.trim()); + const valid = Boolean(search && detail && keyword.trim() && noteIdValid && !candidates.loading && !candidates.error); + const invalidate = () => { setPlan(undefined); setPlanInputs(undefined); setError(''); }; + const resolve = async () => { + if (operation.current || !valid) return; + operation.current = true; setBusy('resolve'); invalidate(); + try { + const result = await api.post('/learning/routes/resolve', { platform, task_type: taskType, search_candidate_id: searchId, detail_candidate_id: detailId }); + setPlan(result); setPlanInputs({ keyword: keyword.trim(), noteId: noteId.trim() }); + } catch (err) { setError(errorText(err)); } + finally { operation.current = false; setBusy(null); } + }; + const verify = async () => { + if (operation.current || !plan || !planInputs || !valid) return; + operation.current = true; setBusy('verify'); setError(''); + try { + const result = await api.post('/learning/routes/verify', { platform: plan.platform, task_type: plan.task_type, search_candidate_id: plan.search.candidate_id, detail_candidate_id: plan.detail.candidate_id, search_digest: plan.search.digest, detail_digest: plan.detail.digest, keyword: planInputs.keyword, ...(planInputs.noteId ? { note_id: planInputs.noteId } : {}) }); + setLatest(result); setPlan(undefined); setPlanInputs(undefined); await validations.reload(); navigate(`/routes?validation=${encodeURIComponent(result.id)}`); + } catch (err) { setError(errorText(err)); setPlan(undefined); setPlanInputs(undefined); } + finally { operation.current = false; setBusy(null); } + }; + const refresh = () => { if (operation.current) return; invalidate(); void candidates.reload(); void validations.reload(); }; + const option = (item: RouteCandidate) => ({ value: item.candidate_id, label: `${item.name} · v${item.version} · ${item.execution_profile || 'legacy'}` }); + return 刷新链路与记录}> +
    + + + +
    +
    + + { setDetailId(value); invalidate(); }} options={details.map(option)} /> + { setKeyword(event.target.value); invalidate(); }} /> + { setNoteId(event.target.value); invalidate(); }} /> +
    + {!candidates.loading && !candidates.error && searches.length === 0 && 当前平台尚无可用搜索能力，请先在“学习批次”学习并完成静态校验。} + {search && details.length === 0 && 尚无同任务、同执行配置的详情与评论能力，请先补学并静态校验。} + {(search || detail) &&
    {[search, detail].filter((item): item is RouteCandidate => Boolean(item)).map(item =>

    {item.ability === 'search_notes' ? '搜索' : '详情与评论'}：最近静态通过最近试跑：{item.trial_status === 'passed' ? '通过' : item.trial_status === 'failed' ? '失败' : '未运行'}维护质量：

    )}
    } + +
    +     + {error && } + {plan && planInputs && +

    {plan.search.digest}

    , span: { xs: 1, md: 2 } }, + { key: 'detail', label: '详情版本', children: <>

    {plan.detail.digest}

    , span: { xs: 1, md: 2 } }, + { key: 'note', label: '笔记选择', children: planInputs.noteId || '本次搜索首条有效笔记' }, { key: 'mode', label: '模式', children: '探索验证 · 未正式发布' }, + ]} />召回只选择固定版本；点击执行后才操作学习浏览器。请先在对应“学习批次”准备会话。修改任一输入会清空本次锁定。{search && }
    +     } + + + rowKey="id" data-testid="route-history" loading={validations.loading} dataSource={history} size="small" pagination={{ pageSize: 5, showSizeChanger: false }} scroll={{ x: 640 }} locale={{ emptyText: '尚无链路复用验证。每次执行独立保存，失败和部分结果保留。' }} columns={[ + { title: '关键词', dataIndex: 'keyword' }, { title: '结果', key: 'status', render: (_: unknown, row) => <>{routeStatusLabels[row.route_status]} }, + { title: '验证时间', key: 'time', render: (_: unknown, row) => formatTime(row.created_at) }, { title: '详情', key: 'details', render: (_: unknown, row) => }, + ]} /> + +
    + { setLinkedId(undefined); navigate('/routes'); }} destroyOnHidden data-testid="route-result-drawer"> + {selected ? : validations.loading ? : } + + setLinkedId(undefined)} destroyOnHidden data-testid="route-linked-validation"> + {linked ? : } + +     ; +} diff --git a/apps/admin/src/pages/TaskDetailPage.tsx b/apps/admin/src/pages/TaskDetailPage.tsx index 61994b9..876ecf0 100644 --- a/apps/admin/src/pages/TaskDetailPage.tsx +++ b/apps/admin/src/pages/TaskDetailPage.tsx @@ -6,18 +6,19 @@ import { JsonView, LoadError, NoRows, Page, StatusTag, formatTime, platformLabel import type { BrowserElement, Candidate, LearningTask, Observation, RecordedStep, Validation } from '../types'; import { ValidationDetails, ValidationTable } from './ValidationsPage'; import { DetailDrawer } from '../components/DetailDrawer'; - -const kindLabels = { fill: '填写参数', click: '点击进入', read: '读取内容' }; -function guessKind(element: BrowserElement): RecordedStep['kind'] { - const type = `${element.kind || ''} ${element.tag || ''}`.toLowerCase(); - if (/input|textarea|fill/.test(type)) return 'fill'; - if (/button|link|click|\ba\b/.test(type)) return 'click'; - return 'read'; -} +import { LearningActionForm } from '../components/LearningActionForm'; +import { operationName, operationSummary } from '../learning-view-model'; +import { ActionFailureNotice } from '../components/ActionFailureNotice'; +import { LoginDetectionNotice } from '../components/LoginDetectionNotice'; +import { ExecutionContractDetails, executionModeLabel } from '../components/ExecutionContractDetails'; export function TaskDetailPage({ taskId, navigate }: { taskId: string; navigate: (path: string) => void }) { const resource = useResource(`/learning/tasks/${taskId}`); const task = resource.data; + const loginDetection = task?.browser_readiness?.login_detection; + const loginNeedsAttention = loginDetection?.state === 'logged_out' || loginDetection?.state === 'verification_required'; + const readiness = task?.browser_readiness; + const sessionNeedsReopen = readiness?.next_action === 'close_and_reopen_browser' || Boolean(readiness?.block_reason) || readiness?.code === 'popup_blocked' || readiness?.code === 'scope_blocked'; const { message } = App.useApp(); const [busy, setBusy] = useState(''); const [readyOpen, setReadyOpen] = useState(false); @@ -25,10 +26,6 @@ export function TaskDetailPage({ taskId, navigate }: { taskId: string; navigate: const [note, setNote] = useState(''); const [selectedObservation, setSelectedObservation] = useState(); const [selectedSelector, setSelectedSelector] = useState(); - const [kind, setKind] = useState('read'); - const [parameter, setParameter] = useState(); - const [value, setValue] = useState(''); - const [outputName, setOutputName] = useState('results'); const [candidateName, setCandidateName] = useState(''); const [candidateOpen, setCandidateOpen] = useState(false); const [record, setRecord] = useState(); @@ -39,12 +36,16 @@ export function TaskDetailPage({ taskId, navigate }: { taskId: string; navigate: const observation = observations.find(item => item.id === selectedObservation) || latest; const latestElements = latest?.elements || []; const selected = latestElements.find(element => element.selector === selectedSelector); - const parameterKeys = Object.keys(task?.parameters || {}); - const observationItems = useMemo(() => observations.map((item, index) => ({ value: item.id, label: `观察 ${index + 1} · ${item.title || formatTime(item.created_at)}` })), [observations]); + const observationItems = useMemo(() => observations.map((item, index) => ({ value: item.id, label: `观察 ${index + 1}${item.source === 'action_failure' ? ' · 动作失败' : ''} · ${item.title || formatTime(item.created_at)}` })), [observations]); const mutation = async (name: string, path: string, body: unknown = {}): Promise => { setBusy(name); try { const result = await api.post(path, body); await resource.reload(); return result; } - catch (error) { void message.error(errorText(error)); return undefined; } + catch (error) { + void message.error(errorText(error)); + if (name === 'action') { setSelectedObservation(undefined); setSelectedSelector(undefined); } + await resource.reload(); + return undefined; + } finally { setBusy(''); } }; const capture = async () => { @@ -56,48 +57,46 @@ export function TaskDetailPage({ taskId, navigate }: { taskId: string; navigate: const result = await mutation('ready', `/learning/tasks/${taskId}/browser/ready`, { identity_label: identity.trim() }); if (result) { setReadyOpen(false); void message.success('人工确认已记录，可以采集页面'); } }; - const execute = async () => { - if (!selected) { void message.warning('请从最新观察中选择一个页面元素'); return; } - if (kind !== 'read' && selected.unique === false) { void message.warning('该定位匹配多个元素，请重新观察或选择唯一元素'); return; } - if (kind === 'fill' && !parameter && !value) { void message.warning('请绑定任务参数或填写本次输入内容'); return; } - if (kind === 'read' && !outputName.trim()) { void message.warning('请为读取内容设置输出名称'); return; } - const result = await mutation('action', `/learning/tasks/${taskId}/browser/action`, { kind, selector: selected.selector, ...(kind === 'fill' ? parameter ? { parameter } : { value } : {}), ...(kind === 'read' ? { output_name: outputName.trim() } : {}) }); + const execute = async (action: RecordedStep): Promise => { + const result = await mutation('action', `/learning/tasks/${taskId}/browser/action`, action); if (result) { setSelectedObservation(undefined); setSelectedSelector(undefined); void message.success('操作与前后证据已记录'); } + return Boolean(result); }; const createCandidate = async () => { const candidate = await mutation('candidate', `/learning/tasks/${taskId}/candidates`, candidateName.trim() ? { name: candidateName.trim() } : {}); if (candidate) { setCandidateOpen(false); navigate(`/candidates/${candidate.id}`); } }; const chooseElement = (element: BrowserElement) => { - setSelectedSelector(element.selector); setKind(guessKind(element)); setParameter(undefined); setValue(''); + setSelectedSelector(element.selector); }; if (!task) return } onClick={() => navigate('/tasks')}>返回任务}>{resource.loading && }; const steps = task.steps || []; return }> - {task.browser_readiness &&

    {task.browser_readiness.message}

    需要试跑时，先打开学习浏览器，在官方页面自行登录，再人工确认会话。
    } + {task.browser_readiness &&

    {task.browser_readiness.message}

    {sessionNeedsReopen ? '请先关闭浏览器，再重新打开并核对新会话；登录确认不能解除当前阻断。' : '需要试跑时，先打开学习浏览器，在官方页面自行登录，再人工确认会话。'}
    }
    {platformLabels[task.platform] || task.platform}创建于 {formatTime(task.created_at)}{task.id}
    {task.entry_url}, span: 2 }, { key: 'parameters', label: '任务参数', children: {JSON.stringify(task.parameters)}, span: 3 }, ]} />
    - 人工登录}>
    在打开的官方页面自行完成扫码或验证码登录。这里仅记录人工确认，尚未接入平台自动身份核验。
    + 人工登录}>
    在打开的官方页面自行完成扫码或验证码登录。这里仅记录人工确认，尚未接入平台自动身份核验。
    + {sessionNeedsReopen && } + +
    setNote(event.target.value)} maxLength={500} />
     {observation ?
    }> +
    {observation.title || '当前页面'}{observation.url}{formatTime(observation.created_at)}{observation.note ? ` · ${observation.note}` : ''}
    {observation.screenshot_url &&
    {`页面观察
    } 
    {observation.text || '本次观察没有可记录的页面文字'}
         {selected ? '已选择定位' : '选择页面元素'}}> -
    ({ value: key, label: `${key} = ${String(task.parameters[key])}` }))} />{!parameter && setValue(event.target.value)} autoComplete="off" placeholder="普通搜索或查询内容" />}} - {kind === 'read' && setOutputName(event.target.value)} placeholder="例如 results 或 comments" />} - -
    定位来自最新观察，执行前重新核对。首版只读链路；登录表单与发送、关注等动作不进入这条验证流程。 +
    : } {latestElements.length} 个定位候选} styles={{ body: { padding: 0 } }}> rowKey={(item, i) => `${item.selector}-${i}`} size="small" pagination={{ pageSize: 8, showSizeChanger: false }} scroll={{ x: 750 }} dataSource={latestElements} locale={{ emptyText: '采集页面后显示可观察的元素' }} rowClassName={row => row.selector === selectedSelector ? 'selected-element' : ''} columns={[ { title: '元素', dataIndex: 'label', key: 'label', render: (label: string, row) =>
    {label || row.tag || '页面元素'}
    {row.tag} · {row.kind || '可读取'}
    }, @@ -108,9 +107,9 @@ export function TaskDetailPage({ taskId, navigate }: { taskId: string; navigate: }, { key: 'steps', label: `链路步骤 ${steps.length}`, children: rowKey={(_, i) => String(i)} dataSource={steps} pagination={false} scroll={{ x: 760 }} locale={{ emptyText: '通过最新页面元素执行操作后，步骤会保存在这里。' }} columns={[ { title: '顺序', key: 'index', width: 65, render: (_: unknown, __: unknown, i: number) => i + 1 }, - { title: '动作', dataIndex: 'kind', width: 100, render: (value: RecordedStep['kind']) => kindLabels[value] }, + { title: '动作', dataIndex: 'kind', width: 120, render: (value: RecordedStep['kind']) => operationName(value) }, { title: '页面定位', dataIndex: 'selector', render: (selector: string) => {selector} }, - { title: '取值或输出', key: 'binding', width: 180, render: (_: unknown, step) => step.parameter ? `参数 ${step.parameter}` : step.output_name ? `输出 ${step.output_name}` : step.value || '固定操作' }, + { title: '取值或输出', key: 'binding', width: 240, render: (_: unknown, step) => operationSummary(step) }, { title: '来源观察', dataIndex: 'evidence_id', width: 130, render: (id: string) => id ? : '—' }, ]} /> }, { key: 'candidates', label: `候选版本 ${task.candidates?.length || 0}`, children: rowKey="id" dataSource={task.candidates || []} pagination={false} locale={{ emptyText: '完成链路操作后，可生成第一个候选版本。' }} columns={[ @@ -119,13 +118,13 @@ export function TaskDetailPage({ taskId, navigate }: { taskId: string; navigate: { key: 'validations', label: `验证记录 ${task.validations?.length || 0}`, children: }, { key: 'parameters', label: '任务参数', children: 候选固化时保存参数快照。试跑可传入不同参数，原候选内容保持不变。 }, ]} /> - setReadyOpen(false)} onOk={() => void markReady()} okText="记录人工确认" confirmLoading={busy === 'ready'}> + setReadyOpen(false)} onOk={() => void markReady()} okText="记录人工确认" confirmLoading={busy === 'ready'} okButtonProps={{ disabled: loginNeedsAttention || sessionNeedsReopen }}>
    setIdentity(event.target.value)} placeholder="填写公开标识，避免手机号等登录信息" maxLength={120} autoComplete="off" />
     setCandidateOpen(false)} onOk={() => void createCandidate()} okText="生成候选" confirmLoading={busy === 'candidate'}> 将当前 {steps.length} 个步骤、参数和页面证据保存为新版本，后续修改不覆盖该版本。
    setCandidateName(event.target.value)} maxLength={120} />
     - setRecord(undefined)} width={800}>{record && } + setRecord(undefined)} width={800}>{record && { setRecord(updated); void resource.reload(); }} />}     ; } diff --git a/apps/admin/src/pages/TasksPage.tsx b/apps/admin/src/pages/TasksPage.tsx index f26aea2..2bc2f74 100644 --- a/apps/admin/src/pages/TasksPage.tsx +++ b/apps/admin/src/pages/TasksPage.tsx @@ -4,6 +4,8 @@ import { useState } from 'react'; import { api, errorText } from '../api'; import { parseObject, platformLabels } from '../components/common'; import type { LearningTask, Platform } from '../types'; +import { TaskExecutionProfileField } from '../components/TaskExecutionProfileField'; +import { NATIVE_EXECUTION_PROFILE } from '../components/ExecutionContractDetails'; const templates: Record = { xiaohongshu: { entry_url: 'https://www.xiaohongshu.com/explore', goal: '搜索笔记 → 读取详情 → 读取评论，整理定位与分支', name: '小红书搜索与评论读取' }, @@ -21,15 +23,17 @@ export function CreateTaskModal({ open, onClose, onCreated }: { open: boolean; o try { const values = await form.validateFields(); const parameters = parseObject(values.parameters, '任务参数'); + const { execution_profile, ...fields } = values; setBusy(true); - const task = await api.post('/learning/tasks', { ...values, parameters }); + const task = await api.post('/learning/tasks', { ...fields, parameters, ...(fields.platform === 'xiaohongshu' && execution_profile === NATIVE_EXECUTION_PROFILE ? { execution_profile } : {}) }); form.resetFields(); onCreated(task); void message.success('学习任务已创建'); } catch (error) { if (!(error && typeof error === 'object' && 'errorFields' in error)) void message.error(errorText(error)); } finally { setBusy(false); } }; return void submit()} okText="创建任务" confirmLoading={busy} width={700} destroyOnHidden> -
    -
    + +
    + diff --git a/apps/admin/src/pages/TestsPage.tsx b/apps/admin/src/pages/TestsPage.tsx index 4d26211..2cee2cb 100644 --- a/apps/admin/src/pages/TestsPage.tsx +++ b/apps/admin/src/pages/TestsPage.tsx @@ -9,6 +9,7 @@ import { EmptyState, ResourceState } from '../components/States'; import { TestRunHistory } from '../components/TestRunHistory'; import { ValidationDetails } from './ValidationsPage'; import type { Candidate, PageResult, Validation } from '../types'; +import { validationModeName } from '../learning-view-model'; export function TestsPage({ navigate }: { navigate: (path: string) => void }) { const resource = useResource>('/learning/validations'); @@ -19,16 +20,16 @@ export function TestsPage({ navigate }: { navigate: (path: string) => void }) { const [selected, setSelected] = useState(); const [kind, setKind] = useState(''); const [status, setStatus] = useState(''); - const all = (resource.data?.items || []).filter(item => (!kind || item.kind === kind) && (!status || item.status === status)); + const all = (resource.data?.items || []).filter(item => item.kind !== 'route' && (!kind || item.kind === kind) && (!status || item.status === status)); const items = all.slice(offset, offset + LIST_PAGE_SIZE); const name = (id: string) => capabilities.data?.items.find(item => item.id === id)?.name || id; const version = (id: string) => capabilities.data?.items.find(item => item.id === id)?.version; return

    脚本测试

    {resource.data ? `${all.length} 条实际记录` : '固定版本测试与来源结果核对'}

    { setStatus(value); setOffset(0); }} style={{ minWidth: 144 }} options={[{ value: '', label: '全部结论' }, { value: 'passed', label: '通过' }, { value: 'failed', label: '失败' }]} />
    -
    void resource.reload()} />{resource.data && !resource.loading && !resource.error && <>{items.length ?
    {items.map(run => )}
    能力 / 样本固定版本验证类型来源结果实测环境创建时间操作
    {name(run.candidate_id)}

    {run.sample_name || '静态校验'}

    		{version(run.candidate_id) === undefined ? '—' : `v${version(run.candidate_id)}`}{run.kind === 'static' ? '静态校验' : '浏览器试跑'}{run.kind === 'static' ? '程序校验' : String(run.environment?.os || run.environment?.platform || '未记录')}{formatTime(run.created_at)}
    : } +
    void resource.reload()} />{resource.data && !resource.loading && !resource.error && <>{items.length ?
    {items.map(run => )}
    能力 / 样本固定版本验证类型来源结果运行质量实测环境创建时间操作
    {name(run.candidate_id)}

    {run.sample_name || '静态校验'}

    		{version(run.candidate_id) === undefined ? '—' : `v${version(run.candidate_id)}`}{validationModeName(run)}{run.kind === 'static' ? '—' : }{run.kind === 'static' ? '程序校验' : String(run.environment?.os || run.environment?.platform || '未记录')}{formatTime(run.created_at)}
    : }
    }
    setPickerOpen(false)} width={480} destroyOnHidden> void capabilities.reload()} />
    -

    图中展示整体设计；学习后台基建已启动，具体实现以 README 为准。真实平台与双端验收待完成。路径播放仅演示设计关系。

    +

    当前按功能模块逐层盘点网页入口，业务分类、完整 A2A 与桌面工程后置。图中包含未来整体设计；真实平台、召回质量与双端交付分别验收。路径播放仅演示设计关系。

    -
    来源：AI 获客助手整体方案 v0.2 · 2026 年 10 月 22 日独立离线文件 · 无外部资源依赖
    +
    来源：AI 获客助手整体方案 v0.2 · 2026 年 10 月 25 日独立离线文件 · 无外部资源依赖
    `); +}); +server.on('connection',socket=>{sockets.add(socket);socket.once('close',()=>sockets.delete(socket));}); +await new Promise(resolve=>server.listen(0,'127.0.0.1',resolve)); +const base=`http://127.0.0.1:${server.address().port}/fixture`; +const read={kind:'read',selector:'#detail',output_name:'detail'}; +const comments={kind:'extract',selector:'#comments-root',row_selector:'.comment-row',output_name:'comments',limit:10,complete_only:true, + fields:[{name:'item_id',source:'attribute',attribute:'data-item-id',required:true},{name:'author',selector:'.comment-author',source:'text',required:true},{name:'body',selector:'.comment-body',source:'text',required:true}], + batch:{scroll_selector:'#comments-window',pixels:80,max_batches:3,dedupe_field:'item_id'}}; +const open=async mode=>{ + const task=`batch-deadline-${++serial}`,url=base+'?mode='+mode; + tasks.add(task); + await dispatch('open',{task_id:task,account_key:'local-batch-deadline',entry_url:url}); + await dispatch('ready',{task_id:task,identity_label:'Local fixture'}); + return {task,url}; +}; +const close=async task=>{await dispatch('close',{task_id:task});tasks.delete(task);}; +const run=async(mode,options={})=>{ + const {task,url}=await open(mode),started=Date.now(); + const result=await verify({task_id:task,entry_url:url,parameters:{},steps:[read,comments],expected_outputs:{detail:'Already read local detail',comments:{state:'nonempty'}}},{batchBudgetMs:4000,...options}); + await fs.writeFile(path.join(runRoot,mode+'.json'),JSON.stringify(result,null,2)); + return {task,result,elapsed_ms:Date.now()-started}; +}; +const assertEvidence=async result=>{ + assert.equal(new Set(result.evidence.map(item=>item.capture_id)).size,result.evidence.length); + for(const capture of result.evidence){ + const png=await fs.readFile(capture.screenshot_path); + assert.equal(png.subarray(1,4).toString(),'PNG'); + } + assert.ok(result.step_timings.every(step=>Object.values(step.phase_ms).reduce((sum,value)=>sum+value,0)===step.duration_ms)); + const saved=JSON.stringify(result);await new Promise(resolve=>setTimeout(resolve,180)); + assert.equal(JSON.stringify(result),saved,'late browser/close callbacks must not mutate returned outputs or evidence'); +}; +const checkClosed=async({task,result,elapsed_ms},code)=>{ + assert.equal(result.passed,false);assert.deepEqual([...result.error_codes].sort(),(Array.isArray(code) ? [...code] : [code]).sort(),JSON.stringify(result.errors)); + assert.equal(result.session_closed,true);assert.equal((await dispatch('status',{task_id:task})).status,'closed'); + assert.ok(elapsed_ms<8500,'internal deadline must cancel the ordinary 10s screenshot wait'); + assert.ok(result.outputs.detail?.[0]?.text.includes('Already read local detail')); + assert.ok(result.step_timings.at(-1).phase_ms.batch_capture>0,'the slow operation must occur in actual batch capture'); + await assertEvidence(result);await close(task); +}; +try{ + let cleaned=0; + assert.equal(await withBatchDeadline(async()=>17,{budgetMs:100,cleanupReserveMs:10,onTimeout:()=>{cleaned+=1;}}),17); + let release; + await assert.rejects(()=>withBatchDeadline(()=>new Promise((_,reject)=>{release=reject;}),{budgetMs:60,cleanupReserveMs:15,onTimeout:()=>{cleaned+=1;release(new Error('cancelled'));}}),error=>error.code==='BATCH_BUDGET'); + assert.equal(cleaned,1); + let finished=false; + await assert.rejects(()=>withBatchDeadline(async()=>{const end=Date.now()+45;while(Date.now(){cleaned+=1;}}),error=>error.code==='BATCH_BUDGET'); + assert.equal(finished,true);assert.equal(cleaned,2,'late resolution is rejected and cleanup occurs once'); + results.push({name:'batch helper normal/cancel/late-resolution semantics'}); + await assert.rejects(()=>withVerifyDeadline(()=>withBatchDeadline(()=>new Promise(()=>{}),{budgetMs:500,cleanupReserveMs:30}),{budgetMs:50,cleanupReserveMs:10,onTimeout:()=>{}}),error=>error.code==='VERIFY_BUDGET'); + results.push({name:'outer VERIFY_BUDGET wins before a longer batch deadline'}); + + const normal=await run('normal'); + assert.equal(normal.result.passed,true,JSON.stringify(normal.result.errors));assert.deepEqual(normal.result.outputs.comments.items.map(item=>item.item_id),['a','b','c','d']); + assert.equal(normal.result.outputs.comments.batch_summary.batches_read,3);await assertEvidence(normal.result);await close(normal.task); + results.push({name:'normal finite batches retain all real PNGs and unique rows',evidence:normal.result.evidence.length}); + + const first=await run('first-font'); + await checkClosed(first,'BATCH_BUDGET');assert.equal(first.result.outputs.comments,undefined,'uncommitted first rows cannot become an empty/success output'); + assert.ok(!first.result.evidence.some(item=>item.note==='有界读取第 1 批'),'unfinished screenshot must not fabricate a first batch'); + results.push({name:'first real batch screenshot cutoff retains detail but no comment output',elapsed_ms:first.elapsed_ms,evidence:first.result.evidence.length}); + + const later=await run('later-font'); + await checkClosed(later,'BATCH_BUDGET'); + assert.deepEqual(later.result.outputs.comments.items.map(item=>item.item_id),['a','b']); + assert.equal(later.result.outputs.comments.truncated,true);assert.deepEqual(later.result.outputs.comments.batch_summary,{batches_read:1,duplicates_removed:0,stop_reason:'time_budget'}); + assert.ok(later.result.evidence.some(item=>item.note==='有界读取第 1 批')); + results.push({name:'later screenshot cutoff exposes only committed prefix with actual first PNG',elapsed_ms:later.elapsed_ms,comments:later.result.outputs.comments.count,evidence:later.result.evidence.length}); + + const laterBlocked=await run('later-font-blocked'); + await checkClosed(laterBlocked,['BATCH_BUDGET','WRITE_REQUEST_BLOCKED']); + assert.equal(laterBlocked.result.outputs.comments,undefined,'a blocked request during the unfinished screenshot must not be hidden by the previous committed prefix'); + assert.ok(laterBlocked.result.evidence.some(item=>item.note==='有界读取第 1 批'),'the known-good first screenshot remains failure evidence'); + assert.ok(laterBlocked.result.step_timings.at(-1).phase_ms.batch_capture>=500,'the delayed request must occur during actual screenshot waiting'); + assert.ok(!JSON.stringify(laterBlocked.result).includes('LOCAL-LATE-QUERY')); + assert.ok(!JSON.stringify(laterBlocked.result).includes('LOCAL-LATE-BODY')); + results.push({name:'unknown POST during a later pending screenshot preserves both errors and suppresses comments',elapsed_ms:laterBlocked.elapsed_ms,evidence:laterBlocked.result.evidence.length,error_codes:laterBlocked.result.error_codes}); + + const global=await run('global-font',{batchBudgetMs:8000,budgetMs:4500,cleanupReserveMs:500}); + await checkClosed(global,'VERIFY_BUDGET');assert.equal(global.result.outputs.comments,undefined); + results.push({name:'real screenshot interrupted by outer deadline keeps VERIFY_BUDGET priority',elapsed_ms:global.elapsed_ms}); + + for(const [mode,code] of [['source','SOURCE_CHANGED'],['root','SOURCE_CHANGED'],['blocked','WRITE_REQUEST_BLOCKED']]){ + const rejection=await run(mode); + assert.equal(rejection.result.passed,false);assert.deepEqual(rejection.result.error_codes,[code],JSON.stringify(rejection.result.errors)); + assert.equal(rejection.result.outputs.comments,undefined,'non-timeout safety failures must not expose the staged/earlier batch as successful comments'); + assert.ok(rejection.result.evidence.some(item=>item.note==='有界读取第 1 批')); + await assertEvidence(rejection.result);await close(rejection.task); + results.push({name:mode+' guard is not hidden by a partial output',evidence:rejection.result.evidence.length}); + } + assert.equal(writeRequests,0,'unknown POST must never reach local fixture server');assert.ok(fontRequests>=4); + const reopen=await open('normal'),capture=await dispatch('capture',{task_id:reopen.task}); + assert.ok(capture.text.includes('retained'),'closed contexts preserve local profile login/cookie data');await close(reopen.task); + results.push({name:'profile survives cancellation and a new explicit ready'}); + for(const invalid of [0,15001,Infinity,1.5]) await assert.rejects(()=>verify({}, {batchBudgetMs:invalid}),error=>error.code==='INVALID_INPUT'); + results.push({name:'internal test budget cannot expand the 15000 ms production cap'}); + await fs.writeFile(path.join(runRoot,'report.json'),JSON.stringify({passed:results.length,results,platform_requests:0,model_calls:0,write_requests:writeRequests,font_requests:fontRequests},null,2)); + console.log(JSON.stringify({passed:results.length,artifacts:runRoot,browser_channel:process.env.ASSISTANT_BROWSER_CHANNEL,platform_requests:0,model_calls:0,write_requests:writeRequests})); +}catch(error){ + await fs.writeFile(path.join(runRoot,'report.json'),JSON.stringify({passed:results.length,results,error:{message:error.message,code:error.code,stack:error.stack},platform_requests:0,model_calls:0,write_requests:writeRequests},null,2)); + console.error(JSON.stringify({passed:results.length,artifacts:runRoot,error:error.message,code:error.code}));process.exitCode=1; +}finally{ + await Promise.allSettled([...tasks].map(task=>dispatch('close',{task_id:task}))); + for(const socket of sockets)socket.destroy(); + await new Promise(resolve=>server.close(resolve)); +} diff --git a/tools/learning/test-browser-worker.mjs b/tools/learning/test-browser-worker.mjs index d1c0ae3..2a6b446 100644 --- a/tools/learning/test-browser-worker.mjs +++ b/tools/learning/test-browser-worker.mjs @@ -5,6 +5,11 @@ import { createServer } from 'node:http'; import path from 'node:path'; import readline from 'node:readline'; import { fileURLToPath } from 'node:url'; +import { redactUrl, validNonemptyExtractOutput, withVerifyDeadline, snapshotSeekResults } from './browser-worker.mjs'; +import { validateAction } from './action-contract.mjs'; +import {contentFixture,contentRegression} from './test-content-scenarios.mjs'; +import {metadataFixture,metadataRegression} from './test-target-metadata.mjs'; +import {viewportFixture,viewportRegression} from './test-viewport-seek.mjs'; const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..'); const TEMP_ROOT = path.join(ROOT, '.tmp'); @@ -49,6 +54,10 @@ function browserClient(home) { if (reply.error) { const error = new Error(reply.error.message); error.code = reply.error.code; + error.before = reply.error.before; + error.after = reply.error.after; + error.diagnostics = reply.error.diagnostics; + error.batch_evidence=reply.error.batch_evidence; request.reject(error); } else request.resolve(reply.result); }); @@ -102,6 +111,196 @@ function browserClient(home) { }; } +async function seekRegression(client,entryUrl,passed,{bindingNoteId,bindingTitle,detailRequests}) { + const task='bounded-seek-regression',base=new URL('/fixture/seek-target',entryUrl).href; + await client.call('open',{task_id:task,account_key:'test-account',entry_url:base}); + await client.call('ready',{task_id:task,identity_label:'隔离有界定位测试'}); + const seek={kind:'scroll',selector:'#seek-window',direction:'down',pixels:160,max_scrolls:5,timeout_ms:2000, + until_target:{selector:'#seek-global a.seek-title',field:'text',value:'目标笔记'}}; + const run=(mode,extra={})=>client.call('verify',{task_id:task,entry_url:base+'?mode='+mode,parameters:{}, + steps:[{...seek,...extra},{kind:'read',selector:'#seek-status',output_name:'status'}],expected_outputs:{status:'实际定位样本'}}); + const actual=await run('offscreen'); + assert.equal(actual.passed,true,JSON.stringify(actual.errors)); + assert.equal(actual.seek_results.length,1); + const summary=actual.seek_results[0]; + assert.deepEqual(Object.keys(summary).sort(),['step_position','scrolls','movements','target_reached','stop_reason'].sort()); + assert.ok(summary.scrolls>0 && summary.scrolls<=5); + assert.equal(summary.scrolls,summary.movements.length); + assert.equal(summary.target_reached,true);assert.equal(summary.stop_reason,'target_reached'); + assert.ok(summary.movements.every(move=>Object.keys(move).length===4 && move.after>move.before)); + assert.equal(actual.verify_budget_ms,undefined,'unbound verify must not get the bound-eight-step budget extension'); + const visible=await run('visible'); + assert.equal(visible.passed,true,JSON.stringify(visible.errors));assert.equal(visible.seek_results[0].scrolls,0); + assert.deepEqual(visible.seek_results[0].movements,[]); + passed('固定容器内全路径CSS只匹配后代，重排离屏目标有限位移到完整可操作；初始可见0次且旧无绑定预算不扩大'); + + for(const [mode,code,reason,extra] of [ + ['missing','SCROLL_BUDGET_EXHAUSTED','scroll_budget',{max_scrolls:1}], + ['missing','SCROLL_BOUNDARY_BEFORE_TARGET','boundary',{pixels:1600,max_scrolls:3}], + ['time','SCROLL_TARGET_TIMEOUT','time_budget',{timeout_ms:1}], + ['duplicate','NON_UNIQUE_TARGET','ambiguous',{}], + ['obscured','TARGET_OBSCURED','obscured',{}], + ['source','SCROLL_SOURCE_CHANGED','source_changed',{}], + ['root','STALE_TARGET','target_changed',{}], + ['blocked','WRITE_REQUEST_BLOCKED','blocked_request',{}], + ]){ + const failure=await run(mode,extra); + assert.equal(failure.passed,false,mode); + assert.deepEqual(failure.error_codes,[code],mode+': '+JSON.stringify(failure.errors)); + assert.equal(failure.seek_results[0].stop_reason,reason,mode); + assert.equal(failure.seek_results[0].target_reached,false,mode); + assert.equal(failure.seek_results[0].scrolls,failure.seek_results[0].movements.length,mode); + assert.ok(failure.evidence.length>=3,'failed seek retains real before/after captures'); + if(['source','root','blocked','missing'].includes(mode)) assert.ok(failure.seek_results[0].scrolls>0,mode+' must retain actual movement'); + assert.equal(failure.outputs.status,undefined,'failed seek must not continue reading'); + } + passed('目标缺失的滚动/边界/时间预算、重名、遮挡、来源变化、根替换及未知写请求明确停止并保留真实有界位移与失败图'); + const lazy=await run('lazy',{pixels:160,max_scrolls:1,timeout_ms:1500}); + // A missing target may populate naturally while the fixed bounded final poll waits. + assert.equal(lazy.passed,true,JSON.stringify(lazy.errors)); + assert.equal(lazy.seek_results[0].scrolls,1); + passed('暂不存在的目标允许在固定短等中自然出现，未重放滚动或主动导航'); + const unsafe=await run('unsafe',{until_target:{...seek.until_target,value:'发送'}}); + assert.equal(unsafe.passed,false);assert.deepEqual(unsafe.error_codes,['WRITE_ACTION_BLOCKED']); + for(const patch of [ + {until_selector:'.seek-title'}, {selector:'body'}, {timeout_ms:0}, {timeout_ms:15001}, + {until_target:{...seek.until_target,parameter:'title'}}, {until_target:{...seek.until_target,field:'attribute',attribute:'token'}}, + {until_target:{...seek.until_target,selector:'body a'}}, {until_target:{...seek.until_target,extra:true}}, + ]) assert.throws(()=>validateAction({...seek,...patch}),error=>error.code==='INVALID_INPUT'); + assert.throws(()=>validateAction({kind:'scroll',selector:'#seek-window',direction:'down',pixels:100,max_scrolls:1,timeout_ms:10}),error=>error.code==='INVALID_INPUT'); + validateAction({...seek,scroll_scope:'viewport'}); + for(const patch of [{scroll_scope:'container'},{scroll_scope:null},{scroll_scope:'viewport',until_target:undefined},{scroll_scope:'viewport',kind:'read'},{selector:'html'},{selector:'HTML'}]) { + assert.throws(()=>validateAction({...seek,...patch}),error=>error.code==='INVALID_INPUT'); + } + passed('目标定位不触发发送/危险动作，合同拒绝混用目标、body根、越界时间、双参数、敏感属性及扩展字段'); + await client.call('close',{task_id:task}); + + const bindingTask='bound-eight-seek',bindingEntry=new URL('/fixture/note-binding',entryUrl).href; + const steps=[ + {kind:'fill',selector:'#binding-search',parameter:'keyword'}, + {kind:'press',selector:'#binding-search',key:'Enter'}, + {kind:'wait',selector:'#binding-results a.binding-title',state:'visible',min_count:1,timeout_ms:1500}, + {kind:'scroll',selector:'#binding-results',direction:'down',pixels:160,max_scrolls:5,timeout_ms:2500,until_target:{selector:'#binding-results a.binding-title',field:'text',parameter:'note_title'}}, + {kind:'click',selector:'#binding-results a.binding-title',target:{field:'text',parameter:'note_title'}}, + {kind:'wait',selector:'#binding-detail',state:'visible',timeout_ms:1500}, + {kind:'extract',selector:'#binding-detail',output_name:'note_detail',limit:1,fields:[ + {name:'title',selector:'.detail-title',source:'text',required:true},{name:'body',selector:'.detail-body',source:'text',required:true},{name:'author',selector:'.detail-author',source:'text',required:true}]}, + {kind:'extract',selector:'#binding-comments-root',row_selector:'.binding-comment',output_name:'comments',limit:3,complete_only:true,fields:[ + {name:'comment_id',source:'attribute',attribute:'id',required:true},{name:'author',selector:'.comment-author',source:'text',required:true},{name:'body',selector:'.comment-body',source:'text',required:true}], + batch:{scroll_selector:'#binding-scroller',pixels:80,max_batches:2,dedupe_field:'comment_id'}}, + ]; + await client.call('open',{task_id:bindingTask,account_key:'test-account',entry_url:bindingEntry}); + await client.call('ready',{task_id:bindingTask,identity_label:'隔离八步绑定'}); + const runBound=(mode,extra={})=>client.call('verify',{task_id:bindingTask,entry_url:bindingEntry+'?mode='+mode,steps, + parameters:{keyword:'装修',note_title:bindingTitle},note_binding:{note_id:bindingNoteId},expected_outputs:{note_detail:{state:'nonempty'},comments:{state:'nonempty'}},...extra}); + const bound=await runBound('seek'); + assert.equal(bound.passed,true,JSON.stringify(bound.errors)); + assert.equal(bound.verify_budget_ms,92500);assert.equal(bound.seek_results[0].step_position,4); + assert.ok(bound.seek_results[0].scrolls>0);assert.equal(bound.note_binding_result.href_checked,true);assert.equal(bound.note_binding_result.url_checked,true); + assert.equal(Object.keys(bound.note_binding_result).length,5);assert.ok(bound.outputs.comments.count>0); + const priorRequests=detailRequests(); + const wrong=await runBound('seek-wrong-id'); + assert.equal(wrong.passed,false);assert.deepEqual(wrong.error_codes,['NOTE_BINDING_MISMATCH']); + assert.equal(wrong.seek_results[0].stop_reason,'binding_mismatch');assert.equal(wrong.seek_results[0].scrolls,0); + assert.equal(detailRequests(),priorRequests);assert.deepEqual(wrong.outputs,{}); + passed('新八步定位与点击同CSS/标题参数，定位及点击分别核对可信ID；同标题错误ID0次滚动0次打开，proof旧5键且仅新绑定链预算增加'); + const viewportSteps=steps.map((step,index)=>index===3 ? {...step,scroll_scope:'viewport'} : step); + const viewportBound=await runBound('seek-viewport',{steps:viewportSteps}); + assert.equal(viewportBound.passed,true,JSON.stringify(viewportBound.errors)); + assert.ok(viewportBound.seek_results[0].scrolls>0);assert.equal(viewportBound.verify_budget_ms,92500); + assert.ok(viewportBound.seek_results[0].movements.every(move=>move.client_height>180 && move.after>move.before)); + assert.equal(viewportBound.note_binding_result.href_checked,true);assert.equal(viewportBound.note_binding_result.url_checked,true); + assert.equal(Object.keys(viewportBound.note_binding_result).length,5);assert.ok(viewportBound.outputs.comments.count>0); + const priorViewportRequests=detailRequests(); + const viewportWrong=await runBound('seek-viewport-wrong-id',{steps:viewportSteps}); + assert.equal(viewportWrong.passed,false);assert.deepEqual(viewportWrong.error_codes,['NOTE_BINDING_MISMATCH']); + assert.equal(viewportWrong.seek_results[0].scrolls,0);assert.equal(detailRequests(),priorViewportRequests); + passed('显式viewport八步在普通固定根内查找并实际滚页面，保留旧proof/预算；错误笔记ID零滚动零打开'); + for(const patch of [ + {steps:steps.map((step,index)=>index===3 ? {...step,until_target:{...step.until_target,selector:'a.binding-title'}} : step)}, + {steps:steps.map((step,index)=>index===3 ? {...step,until_target:{...step.until_target,parameter:'keyword'}} : step)}, + ]) await assert.rejects(()=>runBound('seek',patch),error=>error.code==='NOTE_BINDING_CONTRACT'); + await client.call('close',{task_id:bindingTask}); + passed('八步绑定必须定位与后续点击的CSS和标题参数精确一致，不能套用其它滚动/点击合同'); +} + +async function budgetRegression(entryUrl,home,passed,{bindingNoteId,bindingTitle}) { + let cleaned=0; + assert.equal(await withVerifyDeadline(async()=>17,{budgetMs:100,cleanupReserveMs:10,onTimeout:()=>{cleaned+=1;}}),17); + assert.equal(cleaned,0); + let rejectPending; + await assert.rejects(()=>withVerifyDeadline(()=>new Promise((_,reject)=>{rejectPending=reject;}),{ + budgetMs:50,cleanupReserveMs:10,onTimeout:()=>{cleaned+=1;rejectPending(new Error('cancelled browser operation'));}, + }),error=>error.code==='VERIFY_BUDGET'); + assert.equal(cleaned,1,'cancellation rejection must not outrun the budget error or run cleanup twice'); + await assert.rejects(()=>withVerifyDeadline(async control=>{await new Promise(resolve=>setTimeout(resolve,1));control.check();},{ + budgetMs:100,startedMs:Date.now()-200,cleanupReserveMs:10,onTimeout:()=>{cleaned+=1;}, + }),error=>error.code==='VERIFY_BUDGET'); + assert.equal(cleaned,2); + const active=[{step_position:4,scrolls:1,movements:[{before:0,after:160,scroll_height:1000,client_height:180}],target_reached:false,stop_reason:'time_budget'}]; + const saved=snapshotSeekResults(active),savedJson=JSON.stringify(saved); + active[0].movements[0].after=320; + active[0].movements.push({before:320,after:480,scroll_height:1000,client_height:180}); + active[0].target_reached=true;active.push({...active[0],step_position:5}); + assert.equal(JSON.stringify(saved),savedJson,'a late browser response cannot rewrite a returned seek summary or nested movement'); + passed('总截止计时覆盖未返回的单步，取消异常不能抢先丢失VERIFY_BUDGET，正常完成不误关且预算到期只清理一次'); + const program=` +import assert from 'node:assert/strict'; +import fs from 'node:fs/promises'; +import {dispatch,verify} from ${JSON.stringify(new URL('./browser-worker.mjs',import.meta.url).href)}; +const origin=${JSON.stringify(new URL(entryUrl).origin)},noteId=${JSON.stringify(bindingNoteId)},noteTitle=${JSON.stringify(bindingTitle)}; +const basic=origin+'/fixture/budget-basic'; +let serial=0;const results=[]; +const open=async(url)=>{const task='budget-'+(++serial);await dispatch('open',{task_id:task,account_key:'budget-isolated',entry_url:url});await dispatch('ready',{task_id:task,identity_label:'隔离总预算测试'});return task;}; +const read={kind:'read',selector:'#budget-status',output_name:'detail'}; +const check=async(result,task,started)=>{ + assert.equal(result.passed,false);assert.ok(result.error_codes.includes('VERIFY_BUDGET'),JSON.stringify(result.errors));assert.equal(result.session_closed,true); + assert.ok(Date.now()-started<8500,'short internal deadline must return before an ordinary 10s screenshot or 15s wait'); + assert.equal((await dispatch('status',{task_id:task})).status,'closed'); + assert.ok(result.step_timings.length>0);assert.equal(result.step_timings.at(-1).status,'interrupted'); + assert.ok(result.step_timings.slice(0,-1).every(step=>step.status==='passed')); + assert.ok(result.step_timings.every(step=>Object.values(step.phase_ms).reduce((sum,value)=>sum+value,0)===step.duration_ms)); + assert.ok(Number.isInteger(result.verify_timing.close_ms) && result.verify_timing.close_ms>=0); + assert.ok(result.evidence.length>=2,'already completed real screenshots are retained'); + for(const capture of result.evidence){const png=await fs.readFile(capture.screenshot_path);assert.equal(png.subarray(1,4).toString(),'PNG');} + const saved=JSON.stringify(result);await new Promise(resolve=>setTimeout(resolve,250));assert.equal(JSON.stringify(result),saved,'no late operation may append a successful output or evidence'); +}; +for(const mode of ['wait','font','dom','viewport-dom']){ + const task=await open(basic+'?mode='+mode); + const tail=mode==='wait' ? {kind:'wait',selector:'#budget-pending',state:'text',text:'永不出现',timeout_ms:15000} : mode==='font' ? {kind:'click',selector:'#budget-font-button'} : + {kind:'scroll',selector:'#budget-seek',...(mode==='viewport-dom' ? {scroll_scope:'viewport'} : {}),direction:'down',pixels:160,max_scrolls:10,timeout_ms:15000,until_target:{selector:'a.budget-target',field:'text',value:'目标笔记'}}; + const started=Date.now();const result=await verify({task_id:task,entry_url:basic+'?mode='+mode,parameters:{},steps:[read,tail,{kind:'read',selector:'#budget-reuse',output_name:'late'}],expected_outputs:{detail:'已经读取的真实内容'}},{budgetMs:5000,cleanupReserveMs:500}); + await check(result,task,started);assert.ok(result.outputs.detail[0].text.includes('已经读取'));assert.equal(result.outputs.late,undefined); + if(['dom','viewport-dom'].includes(mode)){assert.equal(result.seek_results[0].stop_reason,'time_budget');assert.equal(result.seek_results[0].target_reached,false);assert.ok(result.seek_results[0].scrolls>0);assert.equal(result.seek_results[0].scrolls,result.seek_results[0].movements.length);} + results.push({mode,evidence:result.evidence.length,elapsed_ms:Date.now()-started}); +} +const reopened=await open(basic);const retained=await dispatch('capture',{task_id:reopened});assert.ok(retained.text.includes('retained'));await dispatch('close',{task_id:reopened}); +const entryTask=await open(origin+'/fixture/budget-entry'); +const entryTimeout=await verify({task_id:entryTask,entry_url:origin+'/fixture/budget-entry',parameters:{},steps:[read],expected_outputs:{detail:'已经读取的真实内容'}},{budgetMs:2500,cleanupReserveMs:500}); +assert.deepEqual(entryTimeout.error_codes,['VERIFY_BUDGET']);assert.equal(entryTimeout.session_closed,true);assert.deepEqual(entryTimeout.step_timings,[]);assert.equal(Object.keys(entryTimeout.outputs).length,0);assert.equal(entryTimeout.evidence.length,0,'unfinished entry capture must not be invented'); +assert.ok(Number.isInteger(entryTimeout.verify_timing.close_ms));assert.ok(entryTimeout.verify_timing.entry_ms+entryTimeout.verify_timing.close_ms<=entryTimeout.verify_timing.duration_ms+10,'entry wall time excludes subsequent closing time'); +results.push({mode:'entry-timeout',timing:entryTimeout.verify_timing}); +const entry=origin+'/fixture/budget-binding',task=await open(entry); +const steps=[{kind:'fill',selector:'#binding-search',parameter:'keyword'},{kind:'press',selector:'#binding-search',key:'Enter'}, + {kind:'wait',selector:'#binding-results a.binding-title',state:'visible',min_count:1,timeout_ms:1500}, + {kind:'scroll',selector:'#binding-results',direction:'down',pixels:160,max_scrolls:5,timeout_ms:2500,until_target:{selector:'#binding-results a.binding-title',field:'text',parameter:'note_title'}}, + {kind:'click',selector:'#binding-results a.binding-title',target:{field:'text',parameter:'note_title'}},{kind:'wait',selector:'#binding-detail',state:'visible',timeout_ms:1500}, + {kind:'extract',selector:'#binding-detail',output_name:'note_detail',limit:1,fields:[{name:'title',selector:'.detail-title',source:'text',required:true},{name:'body',selector:'.detail-body',source:'text',required:true},{name:'author',selector:'.detail-author',source:'text',required:true}]}, + {kind:'extract',selector:'#binding-comments-root',row_selector:'.binding-comment',output_name:'comments',limit:3,complete_only:true,fields:[{name:'comment_id',source:'attribute',attribute:'id',required:true},{name:'author',selector:'.comment-author',source:'text',required:true},{name:'body',source:'content',text_selector:'.comment-body',media_selector:'img.comment-image',required:true}],batch:{scroll_selector:'#binding-scroller',pixels:80,max_batches:2,dedupe_field:'comment_id'}}]; +const started=Date.now();const bound=await verify({task_id:task,entry_url:entry,steps,parameters:{keyword:'装修',note_title:noteTitle},note_binding:{note_id:noteId},expected_outputs:{note_detail:{state:'nonempty'},comments:{state:'nonempty'}}},{budgetMs:7000,cleanupReserveMs:500}); +await check(bound,task,started);assert.equal(bound.verify_budget_ms,92500);assert.equal(bound.outputs.note_detail.items[0].title,noteTitle);assert.equal(bound.outputs.comments,undefined);assert.equal(bound.seek_results[0].target_reached,true);assert.equal(bound.seek_results[0].stop_reason,'target_reached');assert.ok(bound.evidence.length>=9,'completed first batch and current real before capture are retained');assert.equal(bound.note_binding_result.href_checked,true);assert.equal(bound.note_binding_result.url_checked,true); +results.push({mode:'bound-comments',evidence:bound.evidence.length,elapsed_ms:Date.now()-started}); +console.log(JSON.stringify({results,profile_preserved:true,no_late_success:true})); +`; + const child=spawn(process.execPath,['--input-type=module','-e',program],{cwd:ROOT,env:{...process.env,ASSISTANT_LEARNING_HOME:path.join(home,'budget-isolated'),ASSISTANT_BROWSER_CHANNEL:CHANNEL,ASSISTANT_BROWSER_HEADLESS:'1'},windowsHide:true,stdio:['ignore','pipe','pipe']}); + let stdout='',stderr='';child.stdout.on('data',chunk=>{stdout+=chunk;});child.stderr.on('data',chunk=>{stderr+=chunk;}); + const result=await new Promise((resolve,reject)=>{const timer=setTimeout(()=>{child.kill('SIGTERM');reject(new Error('Isolated budget regression did not terminate'));},45000);child.once('error',error=>{clearTimeout(timer);reject(error);});child.once('exit',(code,signal)=>{clearTimeout(timer);resolve({code,signal});});}); + assert.equal(result.code,0,stderr);const report=JSON.parse(stdout.trim()); + await fs.writeFile(path.join(home,'budget-isolated-report.json'),JSON.stringify(report,null,2)); + passed('真实Edge总截止中断等待/截图/容器与viewport阻塞DOM和入口取证，入口不重算关闭时间；已得PNG/时序留存、会话关闭而profile可重开',report); + passed('详情与首批完成后评论截图超时保留真实partial/proof/批次图，已完成seek维持true且不迟到写成功'); +} + async function main() { await fs.mkdir(TEMP_ROOT, { recursive: true }); const runRoot = await fs.mkdtemp(path.join(TEMP_ROOT, 'worker-regression-')); @@ -109,10 +308,119 @@ async function main() { const results = []; let succeeded = false; let slowRequestCount = 0; + let baselineRequestCount = 0; + let blockedWriteReachedServer = 0; + let boundDetailRequests=0; + const bindingNoteId='a'.repeat(24),bindingOtherId='b'.repeat(24),bindingPhoneId='abcdef13800138000abcde12'; + const bindingTitle='装修准备：合成选中笔记'; + const semanticChanges = new Map(); let resolveSlowNavigation; const slowNavigation = new Promise(resolve => { resolveSlowNavigation = resolve; }); const server = createServer((request, response) => { const url = new URL(request.url, 'http://127.0.0.1'); + if(contentFixture(url,request,response)) return; + if(metadataFixture(url,request,response)) return; + if(viewportFixture(url,request,response)) return; + if(url.pathname==='/fixture/budget-font') return; // A real screenshot font wait, interrupted by closing its isolated context. + const stallFont=`const font=new FontFace('budget-hang','url(/fixture/budget-font)');document.fonts.add(font);font.load().catch(()=>{});const span=document.createElement('span');span.textContent='等待字体';span.style.fontFamily='budget-hang';document.body.append(span);`; + if(url.pathname==='/fixture/budget-entry'){ + response.setHeader('Content-Type','text/html; charset=utf-8');response.end(`

    已经读取的真实内容

    `);return; + } + if(url.pathname==='/fixture/budget-basic'){ + response.setHeader('Content-Type','text/html; charset=utf-8'); + response.setHeader('Set-Cookie','budget-session=retained; Max-Age=86400; Path=/; SameSite=Lax'); + const viewportBudget=url.searchParams.get('mode')==='viewport-dom'; + response.end(`总预算隔离样本

    已经读取的真实内容

    ${request.headers.cookie?.includes('budget-session=retained') ? 'retained' : 'fresh'}

    等待文本

    `); + return; + } + if(url.pathname==='/fixture/budget-binding'){ + response.setHeader('Content-Type','text/html; charset=utf-8'); + response.end(`预算绑定样本
    ${bindingTitle}
    `); + return; + } + if(url.pathname==='/fixture/note-binding'){ + const mode=url.searchParams.get('mode') || 'normal',id=mode==='phone' ? bindingPhoneId : mode==='wrong-href' ? bindingOtherId : bindingNoteId; + response.setHeader('Content-Type','text/html; charset=utf-8'); + const viewportSeek=mode.startsWith('seek-viewport'); + response.end(`绑定笔记搜索
    ${mode.startsWith('seek') ? '可见的其它笔记
    ' : ''}${bindingTitle}
    `); + return; + } + if(url.pathname==='/fixture/seek-target'){ + const mode=url.searchParams.get('mode') || 'offscreen',title=mode==='unsafe' ? '发送' : '目标笔记'; + const anchor=`${title}`; + response.setHeader('Content-Type','text/html; charset=utf-8'); + response.end(`有界目标定位
    ${mode==='visible' || mode==='obscured' ? '' : '
    前置内容
    '}${mode==='missing' || mode==='time' || mode==='lazy' ? '' : anchor}${mode==='duplicate' ? anchor : ''}
    目标笔记

    实际定位样本

    ${mode==='obscured' ? '
    遮挡
    ' : ''}`); + return; + } + if(/^\/explore\/[a-f0-9]{24}$/.test(url.pathname)){ + boundDetailRequests+=1; + const mode=url.searchParams.get('mode') || 'normal'; + if(mode==='redirect' && url.pathname.endsWith(bindingNoteId)){ + response.writeHead(302,{Location:`/explore/${bindingOtherId}?mode=redirected`}).end();return; + } + response.setHeader('Content-Type','text/html; charset=utf-8'); + if(mode==='budget-comments') response.write(``); + response.end(`绑定笔记详情

    ${mode==='wrong-title' ? '同ID但标题已变化' : bindingTitle}

    合成正文，仅用于所选笔记读取测试

    合成笔记作者
    ${['one','two','three'].map(id=>`
    评论作者${id}
    合成评论${id}
    `).join('')}
    ${mode==='drift-scroll' ? `` : ''}`); + return; + } + if (url.pathname === '/fixture/semantic-change') { + const key = url.searchParams.get('case'); + semanticChanges.set(key,response); + response.once('close',()=>{ if(semanticChanges.get(key)===response) semanticChanges.delete(key); }); + return; + } + if (url.pathname === '/fixture/search-semantics') { + const mode = url.searchParams.get('mode') || 'placeholder'; + response.setHeader('Content-Type','text/html; charset=utf-8'); + response.end(`搜索固定语义 +
    +
    ${escapeHtml(url.searchParams.get('keyword') || '初始')} 的结果
    +`); + return; + } + if(url.pathname==='/fixture/read-window'){ + const mode=url.searchParams.get('mode') || 'base'; + const row=(name,id)=>`
    作者${name}
    评论完整正文${name}
    ${id==='partial-row' ? '一二' : ''}
    `; + response.setHeader('Content-Type','text/html; charset=utf-8'); + response.end(`长文与完整字段
    ${mode==='lazy' ? '
    长正文
    ' : `${mode==='long' ? '
    长正文
    ' : ''}${row('甲','first-row')}${row('乙','partial-row')}${row('丙','third-row')}`}
    +${mode==='lazy' ? `` : ''}`); + return; + } + if(url.pathname==='/fixture/batch-read'){ + const mode=url.searchParams.get('mode') || 'base'; + const ids=mode==='fit' ? ['a','b'] : ['boundary','lazy'].includes(mode) ? ['a','b','c'] : ['a','b','c','d','e']; + response.setHeader('Content-Type','text/html; charset=utf-8'); + response.end(`有界评论批次
    ${ids.map(id=>`
    作者${id}
    正文${id}
    `).join('')}
    +
    外部行
    +
    无标题首卡
    标题甲
    标题乙
    +`); + return; + } + if(url.pathname==='/fixture/empty-read'){ + const mode=url.searchParams.get('mode') || 'base'; + response.setHeader('Content-Type','text/html; charset=utf-8'); + response.end(`明确空态

    暂无评论

    ${mode==='duplicate' ? '

    暂无评论

    ' : ''}${mode==='rows' ? '
    真实评论
    ' : ''}时间
    `); + return; + } + if(url.pathname==='/fixture/delayed-collection'){ + response.setHeader('Content-Type','text/html; charset=utf-8'); + response.end(`集合延迟加载
    搜索结果加载中
    `); + return; + } + if (url.pathname === '/fixture/baseline') { + baselineRequestCount += 1; + response.writeHead(200, { 'Content-Type': 'application/json' }).end('{}'); + return; + } + if (url.pathname.startsWith('/fixture/write/')) { + blockedWriteReachedServer += 1; + response.writeHead(200).end('unexpected'); + return; + } if (url.pathname === '/fixture/slow') { slowRequestCount += 1; // No response headers: verify is now waiting in its real browser goto, @@ -120,6 +428,76 @@ async function main() { resolveSlowNavigation(); return; } + if (url.pathname === '/fixture/actions') { + const order = url.searchParams.get('order') === 'reverse' ? ['b', 'a'] : ['a', 'b']; + response.setHeader('Content-Type', 'text/html; charset=utf-8'); + response.end(`受限动作测试

    受限动作样本

    +${order.map(id => `笔记 ${id}`).join(' ')} +
    当前 ${escapeHtml(url.searchParams.get('selected') || '无')}
    + +
    等待删除
    +
    +${['a','b','c'].map((id, index) => `
    作者 ${id}材料需求 ${index+1}来源
    `).join('')} +
    +
    遮挡
    +
    尚未尝试
    +
    下方笔记 +
    `); + return; + } + if (url.pathname === '/fixture/viewport') { + response.setHeader('Content-Type', 'text/html; charset=utf-8'); + response.end(`视口候选 +
    ${Array.from({length:140}, (_, i) => `下方 ${i}`).join(' ')}
    +

    透明提示不作为可见提醒

    +
    ${Array.from({length:120},(_,i)=>`可见链接${i}`).join(' ')}
    +`); + return; + } + if (url.pathname === '/fixture/popups') { + response.setHeader('Content-Type', 'text/html; charset=utf-8'); + response.end(`弹窗搜索列表
    +笔记详情 +旧页可关闭的笔记详情 + + + +
    +
    `); + return; + } + if (url.pathname === '/fixture/manual-popup') { + response.setHeader('Content-Type', 'text/html; charset=utf-8'); + const target = url.searchParams.has('foreign') ? `http://localhost:${server.address().port}/fixture/popup-detail?token=MANUAL-POPUP-SECRET` : '/fixture/popup-detail'; + response.end(`人工阶段页面
    等待人工确认 +
    `); + return; + } + if (url.pathname === '/fixture/discovery') { + const duplicate = url.searchParams.has('duplicate'); + response.setHeader('Content-Type', 'text/html; charset=utf-8'); + response.end(`实际可读结构发现

    样本笔记标题

    笔记作者 +

    样本正文：装修材料需求

    隐藏内部提示
    VISIBLE-CREDENTIAL-SECRET

    HIDDEN-ANCESTOR-SECRET

    +
    ${['甲','乙','丙'].map((name,i) => `
    用户${name}
    想了解材料${name}
    `).join('')}
    +
    ${['alpha','beta'].map((id,i)=>`
    ${duplicate ? '同名笔记' : i ? '笔记乙' : '笔记甲'}
    `).join('')}
    +
    ${['alpha','beta'].map((id,i)=>`${''.repeat(9)}`).join('')}
    +
    当前 ${escapeHtml(url.searchParams.get('selected') || '无')}
    +
    验证码登录
    `); + return; + } + if (url.pathname === '/fixture/popup-detail') { + response.setHeader('Content-Type', 'text/html; charset=utf-8'); + response.end(`同源笔记详情
    实际笔记内容 + +

    旧页状态待核对

    `); + return; + } if (url.pathname !== '/fixture/search') { response.writeHead(404).end(); return; } const keyword = escapeHtml(url.searchParams.get('q') || '初始'); response.setHeader('Content-Type', 'text/html; charset=utf-8'); @@ -139,6 +517,71 @@ async function main() { }; const rejectCode = (method, params, code) => assert.rejects(() => client.call(method, params), error => error.code === code); try { + if(process.argv.includes('--metadata-only')){ + await metadataRegression(client,entryUrl,passed,home,CHANNEL,semanticChanges); + await seekRegression(client,entryUrl,passed,{bindingNoteId,bindingTitle,detailRequests:()=>boundDetailRequests}); + succeeded=true; + await fs.writeFile(path.join(runRoot,'report.json'),JSON.stringify({passed:results.length,results},null,2)); + console.log(JSON.stringify({passed:results.length,browser_channel:CHANNEL,artifacts:runRoot,scope:'metadata-only'}));return; + } + if(process.argv.includes('--content-only')){ + await contentRegression(client,entryUrl,passed); + succeeded=true; + await fs.writeFile(path.join(runRoot,'report.json'),JSON.stringify({passed:results.length,results},null,2)); + console.log(JSON.stringify({passed:results.length,browser_channel:CHANNEL,artifacts:runRoot,scope:'content-only'})); + return; + } + if(process.argv.includes('--budget-only')){ + await budgetRegression(entryUrl,home,passed,{bindingNoteId,bindingTitle}); + succeeded=true; + await fs.writeFile(path.join(runRoot,'report.json'),JSON.stringify({passed:results.length,results},null,2)); + console.log(JSON.stringify({passed:results.length,browser_channel:CHANNEL,artifacts:runRoot,scope:'budget-only'})); + return; + } + if(process.argv.includes('--seek-only')){ + await seekRegression(client,entryUrl,passed,{bindingNoteId,bindingTitle,detailRequests:()=>boundDetailRequests}); + succeeded=true; + await fs.writeFile(path.join(runRoot,'report.json'),JSON.stringify({passed:results.length,results},null,2)); + console.log(JSON.stringify({passed:results.length,browser_channel:CHANNEL,artifacts:runRoot,scope:'seek-only'})); + return; + } + if(process.argv.includes('--viewport-seek-only')){ + await viewportRegression(client,entryUrl,passed); + assert.equal(blockedWriteReachedServer,0,'viewport readonly guard prevents the synthetic POST from reaching the fixture'); + succeeded=true; + await fs.writeFile(path.join(runRoot,'report.json'),JSON.stringify({passed:results.length,results},null,2)); + console.log(JSON.stringify({passed:results.length,browser_channel:CHANNEL,artifacts:runRoot,scope:'viewport-seek-only'})); + return; + } + const syntheticNoteId='abcdef13800138000abcde12'; + assert.equal(syntheticNoteId.length,24); + for(const type of ['explore','search_result']){ + const source=`https://www.xiaohongshu.com/${type}/${syntheticNoteId}`; + assert.equal(redactUrl(source),source); + const sensitive=redactUrl(source+'?token=FIXTURE-PRIVATE-TOKEN&phone=13800138000&keyword=13800138000#session=FIXTURE-PRIVATE-SESSION'); + const parsed=new URL(sensitive); + assert.equal(parsed.pathname,`/${type}/${syntheticNoteId}`); + assert.equal(parsed.searchParams.get('token'),'[已隐藏]'); + assert.equal(parsed.searchParams.get('phone'),'[已隐藏]'); + assert.equal(parsed.searchParams.get('keyword'),'[手机号已隐藏]'); + assert.ok(!sensitive.includes('FIXTURE-PRIVATE-TOKEN') && !sensitive.includes('FIXTURE-PRIVATE-SESSION')); + assert.ok(!redactUrl(source+'#contact=13800138000').includes('#contact=13800138000')); + } + for(const source of [ + `https://www.xiaohongshu.com.evil.invalid/explore/${syntheticNoteId}`, + `https://evil.invalid/explore/${syntheticNoteId}`,`https://www.xiaohongshu.com@evil.invalid/explore/${syntheticNoteId}`, + `https://fixture:only@www.xiaohongshu.com/explore/${syntheticNoteId}`, + `http://www.xiaohongshu.com/explore/${syntheticNoteId}`,`https://www.xiaohongshu.com:444/explore/${syntheticNoteId}`, + `https://www.xiaohongshu.com/user/profile/${syntheticNoteId}`,`https://www.xiaohongshu.com/explore/${syntheticNoteId}/`, + `https://www.xiaohongshu.com/explore/${syntheticNoteId}a`, + ]){ + const safe=redactUrl(source); + assert.ok(!safe.includes(syntheticNoteId),'only the precise official public note path may bypass the phone-like substring filter'); + assert.ok(safe.includes('[手机号已隐藏]')); + } + assert.ok(redactUrl('ordinary title 13800138000').includes('[手机号已隐藏]')); + passed('仅官方标准笔记路径保留含手机号形数字的公共hexID，敏感查询/片段和其它来源/账号路径继续脱敏'); + const opened = await client.call('open', { task_id: 'regression-a', account_key: 'test-account', entry_url: entryUrl }); assert.equal(opened.status, 'awaiting_user'); await rejectCode('capture', { task_id: 'regression-a' }, 'AWAITING_USER'); @@ -160,6 +603,16 @@ async function main() { await rejectCode('action', { task_id: 'regression-a', kind: 'click', selector: '[data-testid="send"]' }, 'WRITE_ACTION_BLOCKED'); passed('拒绝发送私信动作'); + await rejectCode('action', { task_id: 'regression-a', kind: 'press', selector: '[data-testid="query"]', key: 'Tab' }, 'INVALID_INPUT'); + await rejectCode('action', { task_id: 'regression-a', kind: 'press', selector: 'input[name="phone"]', key: 'Enter' }, 'UNOBSERVED_SELECTOR'); + const pressSearch = await client.call('verify', { task_id: 'regression-a', entry_url: entryUrl, + steps: [{kind:'fill',selector:'[data-testid="query"]',parameter:'keyword'}, + {kind:'press',selector:'[data-testid="query"]',key:'Enter'}, + {kind:'read',selector:'[data-testid="results"]',output_name:'results'}], + parameters: { keyword: '装修' }, expected_outputs: { results: '装修' } }); + assert.equal(pressSearch.passed, true, JSON.stringify(pressSearch.errors)); + passed('搜索输入框仅允许Enter，实际GET搜索可参数化重放，敏感字段与其他按键拒绝'); + const steps = [ { kind: 'fill', selector: '[data-testid="query"]', parameter: 'keyword' }, { kind: 'click', selector: '[data-testid="search"]' }, @@ -184,6 +637,648 @@ async function main() { parameters: { keyword: '木材' }, expected_outputs: { results: '' } }, 'INVALID_EXPECTED_OUTPUT'); passed('拒绝用空输出预期取得验证通过'); + await client.call('close', { task_id: 'regression-a' }); + const actionUrl = new URL('/fixture/actions', entryUrl).href; + const task = 'restricted-actions'; + await client.call('open', { task_id: task, account_key: 'test-account', entry_url: actionUrl }); + await client.call('ready', { task_id: task, identity_label: '受限动作样本账号' }); + await client.call('capture', { task_id: task }); + const act = params => client.call('action', { task_id: task, ...params }); + const fields = [ + { name: 'author', selector: '.author', source: 'text', required: true }, + { name: 'content', selector: '.content', source: 'text', required: true }, + { name: 'item_id', source: 'attribute', attribute: 'data-item-id', required: true }, + { name: 'url', selector: '.source', source: 'href', required: true }, + { name: 'optional', selector: '.missing', source: 'text' }, + ]; + const extracted = await act({ kind: 'extract', selector: '[data-testid="row"]', fields, limit: 1, output_name: 'rows' }); + assert.equal(extracted.result.count, 1); + assert.equal(extracted.result.items[0].author, '作者 a'); + assert.equal(extracted.result.items[0].optional, null); + assert.equal(extracted.result.coverage, 'limit_reached'); + assert.equal(extracted.result.truncated, true); + assert.equal(extracted.result.source_url, actionUrl); + assert.ok(extracted.before.capture_id !== extracted.after.capture_id); + passed('结构化提取绑定来源、可选空字段、限制及前后证据'); + + const dynamic = await act({ kind: 'extract', selector: '[data-testid="row"]', fields, limit: 10, output_name: 'chosen', + target: { field: 'attribute', attribute: 'data-item-id', value: 'b' } }); + assert.equal(dynamic.result.items[0].author, '作者 b'); + await assert.rejects(() => act({ kind: 'extract', selector: '[data-testid="row"]', fields, limit: 1, output_name: 'missing', + target: { field: 'attribute', attribute: 'data-item-id', value: 'unknown' } }), error => error.code === 'NON_UNIQUE_TARGET'); + await assert.rejects(() => act({ kind: 'click', selector: '[data-testid="post"]' }), error => error.code === 'NON_UNIQUE_TARGET'); + passed('动态对象按实际字段唯一匹配，缺失及多匹配不默选首项'); + + for (const [selector, source, attribute, expected] of [ + ['.secret', 'text', undefined, 'SENSITIVE_OUTPUT'], + ['.js', 'href', undefined, 'UNSAFE_OUTPUT'], + ['.external', 'href', undefined, 'UNSAFE_OUTPUT'], + [undefined, 'attribute', 'data-session-token', 'INVALID_INPUT'], + ['.duplicate', 'text', undefined, 'NON_UNIQUE_FIELD'], + ]) { + await assert.rejects(() => act({ kind: 'extract', selector: '[data-testid="unsafe"]', fields: [{ name: 'value', selector, source, attribute }], + limit: 1, output_name: 'unsafe' }), error => error.code === expected); + } + await assert.rejects(() => act({ kind: 'extract', selector: 'xpath=//article', fields, limit: 1, output_name: 'unsafe' }), error => error.code === 'INVALID_INPUT'); + passed('结构化提取拒绝凭据、脚本/跨站链接、敏感属性、歧义字段及非CSS定位'); + + const scroll = await act({ kind: 'scroll', selector: '[data-testid="scroller"]', direction: 'down', pixels: 80, max_scrolls: 2 }); + assert.ok(scroll.result.movements.some(movement => movement.after > movement.before)); + assert.ok(scroll.result.scrolls <= 2); + const bottom = await act({ kind: 'extract', selector: '[data-testid="row"]', fields, limit: 10, output_name: 'rows' }); + assert.ok(bottom.result.items.some(item => item.author === '作者 c')); + assert.ok(!bottom.result.items.some(item => item.author === '作者 a')); + assert.equal(bottom.result.truncated, true); + assert.equal(bottom.result.coverage, 'bounded', 'viewport omission is distinct from reaching the requested item limit'); + const offscreen = await act({ kind: 'extract', selector: '[data-testid="below"]', + fields: [{ name: 'text', source: 'text' }], limit: 10, output_name: 'offscreen' }); + assert.equal(offscreen.result.count, 0); + assert.equal(offscreen.result.truncated, true); + assert.equal(offscreen.result.coverage, 'empty'); + await assert.rejects(() => act({ kind: 'scroll', selector: 'body', direction: 'down', pixels: 80, max_scrolls: 11 }), error => error.code === 'INVALID_INPUT'); + passed('容器滚动有界、留下位移证据，只提取滚动后可见行'); + + await act({ kind: 'click', selector: '[data-testid="expand"]' }); + const hidden = await act({ kind: 'wait', selector: '[data-testid="pending"]', state: 'hidden', timeout_ms: 1000 }); + assert.equal(hidden.result.satisfied, true); + const waited = await act({ kind: 'wait', selector: '[data-testid="delayed"]', state: 'text', text: '加载完成', timeout_ms: 1500 }); + assert.equal(waited.result.satisfied, true); + await assert.rejects(() => act({ kind: 'wait', selector: '[data-testid="delayed"]', state: 'text', text: '不会出现', timeout_ms: 100 }), error => error.code === 'WAIT_TIMEOUT'); + passed('条件等待支持已移除对象、异步文本及明确超时失败'); + + await assert.rejects(() => act({ kind: 'click', selector: '[data-testid="covered"]' }), error => error.code === 'TARGET_OBSCURED'); + await assert.rejects(() => act({ kind: 'click', selector: '[data-testid="below"]' }), error => error.code === 'TARGET_NOT_IN_VIEWPORT'); + await act({ kind: 'scroll', selector: 'body', direction: 'down', pixels: 1600, max_scrolls: 2 }); + const below = await act({ kind: 'click', selector: '[data-testid="below"]' }); + assert.match(below.result.url, /selected=below/); + passed('点击拒绝遮挡和视口外目标，显式滚动后才能执行'); + + const dynamicSteps = [{ kind: 'click', selector: '[data-testid="post"]', + target: { field: 'attribute', attribute: 'data-item-id', parameter: 'post_id' } }, + { kind: 'read', selector: '[data-testid="selected"]', output_name: 'selected' }]; + const verifyDynamic = await client.call('verify', { task_id: task, entry_url: `${actionUrl}?order=reverse`, steps: dynamicSteps, + parameters: { post_id: 'b' }, expected_outputs: { selected: '当前 b' } }); + assert.equal(verifyDynamic.passed, true, JSON.stringify(verifyDynamic.errors)); + const structuredVerify = await client.call('verify', { task_id: task, entry_url: actionUrl, + steps: [{ kind: 'extract', selector: '[data-testid="row"]', fields, limit: 2, output_name: 'rows' }], + parameters: {}, expected_outputs: { rows: '材料需求 2' } }); + assert.equal(structuredVerify.passed, true, JSON.stringify(structuredVerify.errors)); + const urlOnly = await client.call('verify', { task_id: task, entry_url: actionUrl, + steps: [{ kind: 'extract', selector: '[data-testid="row"]', fields: [fields[3]], limit: 2, output_name: 'rows' }], + parameters: {}, expected_outputs: { rows: '/fixture/actions' } }); + assert.equal(urlOnly.passed, false, 'URL alone cannot satisfy a content assertion'); + passed('重放按变化参数选择实际对象，结构化内容可断言且URL不能冒充内容'); + + const waitVerify = await client.call('verify', { task_id: task, entry_url: actionUrl, + steps: [{ kind: 'click', selector: '[data-testid="expand"]' }, + { kind: 'wait', selector: '[data-testid="pending"]', state: 'hidden', timeout_ms: 1000 }, + { kind: 'wait', selector: '[data-testid="delayed"]', state: 'text', text: '加载完成', timeout_ms: 1500 }, + { kind: 'read', selector: '[data-testid="delayed"]', output_name: 'loaded' }], + parameters: {}, expected_outputs: { loaded: '加载完成' } }); + assert.equal(waitVerify.passed, true, JSON.stringify(waitVerify.errors)); + let failure; + await assert.rejects(() => act({ kind: 'click', selector: '[data-testid="post-write"]' }), error => { + failure = error; + return error.code === 'WRITE_REQUEST_BLOCKED'; + }); + assert.ok(failure.before?.capture_id && failure.after?.capture_id); + assert.ok(failure.after.text.includes('已出现局部变化')); + assert.equal(failure.diagnostics.phase, 'request_guard'); + assert.equal(failure.diagnostics.blocked_request_count, 21); + assert.equal(failure.diagnostics.blocked_requests.length, 2); + assert.equal(failure.diagnostics.blocked_requests.filter(item => item.pathname === '/fixture/write/:redacted/:id').length, 1, '20 identical requests keep one safe summary'); + assert.ok(failure.diagnostics.blocked_requests.some(item => item.pathname === '/fixture/write/unknown-branch'), 'a distinct unknown request must not be buried by repeated background traffic'); + assert.equal(blockedWriteReachedServer, 0); + assert.ok(failure.after.request_observations.some(item => item.blocked && item.automating && item.method === 'POST')); + for (const request of failure.diagnostics.blocked_requests) { + assert.deepEqual(Object.keys(request).sort(), ['method','origin','pathname','policy_reason','resource_type']); + assert.equal(request.policy_reason, 'missing_policy'); + } + const serializedFailure = JSON.stringify({ diagnostics: failure.diagnostics, requests: failure.after.request_observations }); + for (const secret of ['QUERY-SECRET','SECOND-SECRET','HEADER-SECRET','BODY-SECRET','abcdefghijklmnopqrstuvwxyz0123456789abcdef','data-session-token','?token=']) assert.ok(!serializedFailure.includes(secret)); + await client.call('close', { task_id: task }); + passed('条件等待可重放；写请求失败保存真实前后状态与有界脱敏诊断，未重放或发送POST'); + + const viewportTask = 'viewport-observation'; + await client.call('open', { task_id: viewportTask, account_key: 'test-account', entry_url: new URL('/fixture/viewport', entryUrl).href }); + await client.call('ready', { task_id: viewportTask, identity_label: '可见性样本账号' }); + const viewportCapture = await client.call('capture', { task_id: viewportTask }); + assert.ok(viewportCapture.elements.some(item => item.selector === '[data-testid="visible-late"]' && item.in_viewport)); + assert.ok(viewportCapture.elements.some(item => item.selector === '#quota-search' && item.search_input && item.unique),'many read roots/links must not exhaust the search input quota'); + assert.ok(viewportCapture.elements.filter(item => !item.in_viewport).length <= 30); + assert.ok(viewportCapture.elements.length <= 120); + assert.ok(!viewportCapture.text.includes('非真人操作提醒隐藏文本')); + assert.ok(!viewportCapture.text.includes('透明提示不作为可见提醒')); + assert.ok(viewportCapture.viewport_text.includes('搜索当前页面')); + assert.ok(baselineRequestCount > 0); + assert.ok(viewportCapture.request_observations.some(item => item.method === 'POST' && !item.blocked && !item.automating && item.pathname === '/fixture/baseline')); + assert.ok(!JSON.stringify(viewportCapture.request_observations).includes('BASELINE-QUERY-SECRET')); + assert.ok(!JSON.stringify(viewportCapture.request_observations).includes('BASELINE-BODY-SECRET')); + await client.call('close', { task_id: viewportTask }); + passed('视口候选优先且离屏有界，隐藏正文不作可见提示，被动请求摘要区分允许与拦截且不含查询正文'); + + const popupEntry = new URL('/fixture/popups', entryUrl).href; + const popupTask = 'same-origin-popup'; + await client.call('open', { task_id: popupTask, account_key: 'test-account', entry_url: popupEntry }); + await client.call('ready', { task_id: popupTask, identity_label: '同源弹窗样本账号' }); + await client.call('capture', { task_id: popupTask }); + await assert.rejects(() => client.call('action', { task_id: popupTask, kind: 'press', selector: '[data-testid="comment"]', key:'Enter' }), error => error.code === 'WRITE_ACTION_BLOCKED'); + await assert.rejects(() => client.call('action', { task_id: popupTask, kind: 'press', selector: '[data-testid="post-search"]', key:'Enter' }), error => error.code === 'WRITE_ACTION_BLOCKED'); + const popupClicked = await client.call('action', { task_id: popupTask, kind: 'click', selector: '[data-testid="same-popup"]' }); + assert.match(popupClicked.result.url, /\/fixture\/popup-detail$/); + assert.match(popupClicked.after.text, /实际笔记内容/); + assert.equal((await client.call('status', { task_id: popupTask })).status, 'observing'); + await client.call('close', { task_id: popupTask }); + const openerTask = 'popup-retained-opener'; + await client.call('open', { task_id:openerTask, account_key:'test-account', entry_url:popupEntry }); + await client.call('ready', { task_id:openerTask, identity_label:'旧页关闭样本账号' }); + await client.call('capture', { task_id:openerTask }); + await client.call('action', { task_id:openerTask, kind:'click', selector:'[data-testid="opener-popup"]' }); + // A script-opened intermediate page is actually permitted to close. The + // browser forbids closing its original UI-created tab from page scripts. + await client.call('action', { task_id:openerTask, kind:'click', selector:'[data-testid="child-popup"]' }); + await client.call('action', { task_id: openerTask, kind: 'click', selector: '[data-testid="close-opener"]' }); + const closedOpener = await client.call('capture', { task_id: openerTask }); + assert.match(closedOpener.text, /实际笔记内容/); + assert.match(closedOpener.text, /旧页面已关闭/); + await client.call('close', { task_id:openerTask }); + passed('明确点击的同源新页沿用会话及人工确认，旧搜索页关闭不误关详情，评论及POST搜索拒绝Enter'); + + for (const [suffix, kind, selector, expected] of [ + ['foreign', 'click', '[data-testid="foreign-popup"]', ['POPUP_BLOCKED','SCOPE_BLOCKED']], + ['unknown', 'press', '[data-testid="popup-search"]', ['POPUP_BLOCKED']], + ]) { + const id = `popup-${suffix}`; + await client.call('open', { task_id:id, account_key:'test-account', entry_url:popupEntry }); + await client.call('ready', { task_id:id, identity_label:'异常弹窗样本账号' }); + await client.call('capture', { task_id:id }); + await assert.rejects(() => client.call('action', {task_id:id, kind, selector, ...(kind==='press'?{key:'Enter'}:{})}), error => { + assert.ok(error.before && error.after); + assert.ok(!JSON.stringify(error.diagnostics).includes('POPUP-SECRET')); + return expected.includes(error.code); + }); + await assert.rejects(() => client.call('capture', {task_id:id}), error => expected.includes(error.code)); + await client.call('close', {task_id:id}); + } + passed('出域及非明确点击产生的新页关闭并暂停，失败取证不自动追踪或操作未知页面'); + + for (const foreign of [false, true]) { + const id = foreign ? 'manual-foreign-popup' : 'manual-same-origin-popup'; + const manualEntry = new URL(`/fixture/manual-popup${foreign ? '?foreign=1' : ''}`, entryUrl).href; + await client.call('open', { task_id:id, account_key:'test-account', entry_url:manualEntry }); + let state; + const deadline = Date.now() + 6000; + do { + state = await client.call('status', {task_id:id}); + if (foreign ? state.block_reason && state.popup_observations.length : state.url.endsWith('/fixture/popup-detail')) break; + await new Promise(resolve => setTimeout(resolve, 100)); + } while (Date.now() < deadline); + assert.equal(state.status, 'awaiting_user'); + assert.equal(state.manual_confirmation, false); + if (foreign) { + assert.ok(['scope_blocked','popup_blocked'].includes(state.block_reason)); + await assert.rejects(() => client.call('ready', {task_id:id,identity_label:'人工阶段异常样本'}), error => ['SCOPE_BLOCKED','POPUP_BLOCKED'].includes(error.code)); + } else { + assert.equal(state.block_reason, null); + assert.match(state.url, /\/fixture\/popup-detail$/); + await rejectCode('capture', {task_id:id}, 'AWAITING_USER'); + await client.call('ready', {task_id:id,identity_label:'人工阶段同源样本'}); + const manualCapture = await client.call('capture', {task_id:id}); + assert.ok(manualCapture.text.includes('实际笔记内容')); + assert.ok(manualCapture.popup_observations.some(item => item.phase === 'manual' && item.decision === 'adopted' && item.reason === 'same_origin_manual')); + } + state = await client.call('status', {task_id:id}); + assert.ok(state.popup_observations.length <= 10); + assert.ok(!JSON.stringify(state.popup_observations).includes('MANUAL-POPUP-SECRET')); + await client.call('close', {task_id:id}); + passed(foreign ? '人工确认前出域新页仍阻止，status暴露实际阻断原因而非伪确认' : '人工确认前同源新页可接续但仍不可采集，显式确认后留真实事件摘要'); + } + + const discoveryTask = 'readable-discovery'; + const discoveryUrl = new URL('/fixture/discovery', entryUrl).href; + await client.call('open', {task_id:discoveryTask,account_key:'test-account',entry_url:discoveryUrl}); + await client.call('ready', {task_id:discoveryTask,identity_label:'可读结构样本'}); + const discovered = await client.call('capture', {task_id:discoveryTask}); + assert.ok(discovered.elements.some(item=>item.selector==='#noteContainer' && item.kind==='read')); + assert.ok(discovered.elements.some(item=>item.selector==='#detail-desc' && item.kind==='read')); + assert.ok(discovered.elements.some(item=>item.selector.startsWith('#comment-') && item.kind==='read')); + const scroller = discovered.elements.find(item=>item.selector==='#commentsViewport'); + assert.ok(scroller.scroll_y.can_scroll); + const commentRows = discovered.elements.find(item=>item.collection && item.selector_basis==='id_family' && item.selector.includes('comment-')); + assert.ok(commentRows?.source_selectors.length); + const authorHint = commentRows.read_hints.find(item=>item.link && item.label==='用户甲'); + const bodyHint = commentRows.read_hints.find(item=>item.source==='text' && item.label==='想了解材料甲'); + const dateHint = commentRows.read_hints.find(item=>item.source==='attribute' && item.attribute==='datetime'); + assert.ok(authorHint && bodyHint && dateHint, JSON.stringify(commentRows)); + assert.ok(discovered.elements.length<=120 && discovered.elements.filter(item=>!item.in_viewport).length<=30); + assert.ok(!JSON.stringify(discovered.elements).includes('PRIVATE-NOT-READ')); + assert.ok(!JSON.stringify(discovered.elements).includes('隐藏内部提示')); + for (const value of ['VISIBLE-CREDENTIAL-SECRET','HIDDEN-ANCESTOR-SECRET','access-token','ordinary-child','hidden-body']) { + assert.ok(!JSON.stringify(discovered.elements).includes(value)); + assert.ok(!discovered.text.includes(value)); + assert.ok(!discovered.viewport_text.includes(value)); + } + const discoveryAction = params => client.call('action',{task_id:discoveryTask,...params}); + const bodyRead = await discoveryAction({kind:'read',selector:'#detail-desc',output_name:'body'}); + assert.ok(bodyRead.result[0].text.includes('样本正文')); + assert.ok(!bodyRead.result[0].text.includes('SECRET')); + const bodyExtract = await discoveryAction({kind:'extract',selector:'#detail-desc',fields:[{name:'body',source:'text'}],limit:1,output_name:'body'}); + assert.ok(!bodyExtract.result.items[0].body.includes('SECRET')); + const commentFields = [{name:'author',selector:authorHint.selector,source:'text',required:true}, + {name:'content',selector:bodyHint.selector,source:'text',required:true}, + {name:'source',selector:authorHint.selector,source:'href',required:true}, + {name:'comment_id',source:'attribute',attribute:'id',required:true}, + {name:'time',selector:dateHint.selector,source:'attribute',attribute:'datetime',required:true}]; + const firstComments = await discoveryAction({kind:'extract',selector:commentRows.selector,fields:commentFields,limit:3,output_name:'comments'}); + assert.equal(firstComments.result.count,1); + assert.equal(firstComments.result.items[0].author,'用户甲'); + await discoveryAction({kind:'scroll',selector:scroller.selector,direction:'down',pixels:100,max_scrolls:1}); + const nextComments = await discoveryAction({kind:'extract',selector:commentRows.selector,fields:commentFields,limit:3,output_name:'comments'}); + assert.equal(nextComments.result.items[0].content,'想了解材料乙'); + passed('实际id/标题/滚动根及id族可读集合有界发现，用真实相对定位映射评论并只滚动评论容器'); + + const titles = discovered.elements.find(item=>item.collection && item.tag==='a' && item.selector.includes('note-title')); + assert.ok(titles && titles.kind==='click' && !titles.unique && titles.count===2); + await assert.rejects(()=>discoveryAction({kind:'click',selector:titles.selector}), error=>error.code==='NON_UNIQUE_TARGET'); + await assert.rejects(()=>discoveryAction({kind:'click',selector:titles.selector,target:{field:'text',value:'不存在'}}), error=>error.code==='NON_UNIQUE_TARGET'); + const titleClicked = await discoveryAction({kind:'click',selector:titles.selector,target:{field:'text',value:'笔记乙'}}); + assert.match(titleClicked.result.url,/selected=beta/); + const deepTitles = discovered.elements.find(item=>item.selector==='#stable a.title'); + assert.ok(deepTitles && deepTitles.collection && deepTitles.kind==='click' && !deepTitles.unique && deepTitles.count===2); + await assert.rejects(()=>discoveryAction({kind:'click',selector:deepTitles.selector}), error=>error.code==='NON_UNIQUE_TARGET'); + const deepClicked = await discoveryAction({kind:'click',selector:deepTitles.selector,target:{field:'text',value:'深层笔记乙'}}); + assert.match(deepClicked.result.url,/selected=deep-beta/); + await client.call('close',{task_id:discoveryTask}); + const duplicateTask = 'discovery-duplicate-target'; + await client.call('open',{task_id:duplicateTask,account_key:'test-account',entry_url:discoveryUrl+'?duplicate=1'}); + await client.call('ready',{task_id:duplicateTask,identity_label:'重复对象样本'}); + await client.call('capture',{task_id:duplicateTask}); + await assert.rejects(()=>client.call('action',{task_id:duplicateTask,kind:'click',selector:titles.selector,target:{field:'text',value:'同名笔记'}}), error=>error.code==='NON_UNIQUE_TARGET'); + await client.call('close',{task_id:duplicateTask}); + passed('实际重复a安全class集合支持唯一文本目标；无target、零匹配及同名多匹配不点击'); + + const semanticUrl = new URL('/fixture/search-semantics',entryUrl).href; + const placeholderTask = 'placeholder-only-search'; + await client.call('open',{task_id:placeholderTask,account_key:'test-account',entry_url:semanticUrl}); + await client.call('ready',{task_id:placeholderTask,identity_label:'搜索语义样本'}); + const semanticCapture = await client.call('capture',{task_id:placeholderTask}); + const stableSearch = semanticCapture.elements.find(item=>item.selector==='#semantic-search'); + assert.ok(stableSearch.search_input && stableSearch.stable_label.length>0 && stableSearch.stable_label.length<=500); + assert.ok(!stableSearch.stable_label.includes('示例')); + await new Promise(resolve=>setTimeout(resolve,160)); + await client.call('action',{task_id:placeholderTask,kind:'fill',selector:'#semantic-search',value:'装修'}); + await new Promise(resolve=>setTimeout(resolve,160)); + const entered = await client.call('action',{task_id:placeholderTask,kind:'press',selector:'#semantic-search',key:'Enter'}); + assert.ok(entered.after.text.includes('装修 的结果')); + await new Promise(resolve=>setTimeout(resolve,160)); + await assert.rejects(()=>client.call('action',{task_id:placeholderTask,kind:'fill',selector:'#plain-field',value:'不应执行'}),error=>error.code==='STALE_TARGET'); + await client.call('close',{task_id:placeholderTask}); + passed('固定搜索语义不受轮换placeholder影响，实际fill/Enter仍成功；非搜索框保持原严格检查'); + for (const mode of ['aria','intent','missing']) { + const id = `semantic-${mode}`; + await client.call('open',{task_id:id,account_key:'test-account',entry_url:`${semanticUrl}?mode=${mode}`}); + await client.call('ready',{task_id:id,identity_label:'搜索变化拒绝样本'}); + await client.call('capture',{task_id:id}); + const change = semanticChanges.get(mode); + assert.ok(change); + change.writeHead(200,{'Content-Type':'application/json'}).end('{}'); + await new Promise(resolve=>setTimeout(resolve,160)); + await assert.rejects(()=>client.call('action',{task_id:id,kind:'fill',selector:'#semantic-search',value:'不应执行'}),error=>{ + assert.equal(error.diagnostics.phase,'inspect_current'); + return error.code === (mode==='missing' ? 'UNOBSERVED_CURRENT_SELECTOR' : 'STALE_TARGET'); + }); + await client.call('close',{task_id:id}); + } + passed('aria/搜索意图变化继续STALE；当前采集缺失定位单独安全诊断且不放宽执行'); + + const clipTask='complete-clip-fields'; + const windowUrl=new URL('/fixture/read-window',entryUrl).href; + await client.call('open',{task_id:clipTask,account_key:'test-account',entry_url:windowUrl}); + await client.call('ready',{task_id:clipTask,identity_label:'裁剪字段样本'}); + await client.call('capture',{task_id:clipTask}); + const clipAction=params=>client.call('action',{task_id:clipTask,...params}); + const untilBase={kind:'scroll',selector:'#read-window',direction:'down',pixels:240,max_scrolls:3,until_selector:'.comment-item'}; + const zeroScroll=await clipAction(untilBase); + assert.equal(zeroScroll.result.scrolls,0); + assert.equal(zeroScroll.result.until_reached,true); + const clipFields=[{name:'author',selector:'.author',source:'text',required:true},{name:'content',selector:'.content',source:'text',required:true}]; + const complete=await clipAction({kind:'extract',selector:'#read-window .comment-item',fields:clipFields,limit:3,output_name:'comments',complete_only:true}); + assert.equal(complete.result.count,1); + assert.equal(complete.result.items[0].author,'作者甲'); + assert.equal(complete.result.skipped_incomplete_count,1); + assert.equal(complete.result.truncated,true); + assert.equal(complete.result.coverage,'bounded'); + for(const [field,code] of [ + [{name:'missing',selector:'.missing',source:'text',required:true},'REQUIRED_FIELD_MISSING'], + [{name:'hidden',selector:'.hidden-field',source:'text',required:true},'REQUIRED_FIELD_MISSING'], + [{name:'empty',selector:'.empty',source:'text',required:true},'REQUIRED_FIELD_MISSING'], + [{name:'private',selector:'.secret-field',source:'text',required:true},'SENSITIVE_OUTPUT'], + [{name:'duplicate',selector:'.duplicate',source:'text',required:true},'NON_UNIQUE_FIELD'], + ]) await assert.rejects(()=>clipAction({kind:'extract',selector:'#partial-row',fields:[clipFields[1],field],limit:1,output_name:'comments',complete_only:true}),error=>error.code===code); + await assert.rejects(()=>clipAction({...untilBase,until_selector:'body .comment-item'}),error=>error.code==='INVALID_INPUT'); + await assert.rejects(()=>clipAction({...untilBase,until_selector:'.secret-field'}),error=>error.code==='SENSITIVE_OUTPUT'); + await assert.rejects(()=>clipAction({kind:'extract',selector:'#first-row',fields:clipFields,limit:1,output_name:'comments',complete_only:'true'}),error=>error.code==='INVALID_INPUT'); + await client.call('close',{task_id:clipTask}); + passed('until初始完整行可0滚；complete_only只跳过裁剪行，后续真实缺失/隐藏/空值/敏感/歧义仍失败'); + + const longTask='until-long-content'; + await client.call('open',{task_id:longTask,account_key:'test-account',entry_url:windowUrl+'?mode=long'}); + await client.call('ready',{task_id:longTask,identity_label:'长正文有界滚动样本'}); + await client.call('capture',{task_id:longTask}); + const longAction=params=>client.call('action',{task_id:longTask,...params}); + await assert.rejects(()=>longAction({...untilBase,pixels:120,max_scrolls:1}),error=>{ + assert.ok(error.before && error.after);return error.code==='SCROLL_BUDGET_EXHAUSTED'; + }); + const reached=await longAction(untilBase); + assert.ok(reached.result.until_reached && reached.result.scrolls<=3); + const afterLong=await longAction({kind:'extract',selector:'#read-window .comment-item',fields:clipFields,limit:3,output_name:'comments',complete_only:true}); + assert.ok(afterLong.result.count>=1 && afterLong.result.count<=3); + await assert.rejects(()=>longAction({...untilBase,until_selector:'.never',max_scrolls:3}),error=>{ + assert.ok(error.before && error.after);return error.code==='SCROLL_BOUNDARY_BEFORE_TARGET'; + }); + await client.call('close',{task_id:longTask}); + const lazyTask='until-lazy-boundary'; + await client.call('open',{task_id:lazyTask,account_key:'test-account',entry_url:windowUrl+'?mode=lazy'}); + await client.call('ready',{task_id:lazyTask,identity_label:'边界懒加载样本'}); + await client.call('capture',{task_id:lazyTask}); + const lazy=await client.call('action',{task_id:lazyTask,...untilBase}); + assert.ok(lazy.result.until_reached && lazy.result.scrolls<=3); + assert.ok(lazy.after.text.includes('评论完整正文延迟')); + await client.call('close',{task_id:lazyTask}); + passed('长正文按条件有界滚动，预算/边界未到留证失败；边界懒加载固定轮询后成功且不重放滚动'); + + const batchUrl=new URL('/fixture/batch-read',entryUrl).href; + const batchFields=[{name:'item_id',source:'attribute',attribute:'data-item-id',required:true}, + {name:'author',selector:'.batch-author',source:'text',required:true},{name:'content',selector:'.batch-body',source:'text',required:true}]; + const batchParams={kind:'extract',selector:'#batch-container',row_selector:'.batch-row',fields:batchFields,limit:20,output_name:'comments',complete_only:true, + batch:{scroll_selector:'#batch-window',pixels:80,max_batches:3,dedupe_field:'item_id'}}; + const openBatch=async(id,url=batchUrl)=>{ + await client.call('open',{task_id:id,account_key:'test-account',entry_url:url}); + await client.call('ready',{task_id:id,identity_label:'批次读取隔离样本'}); + return client.call('capture',{task_id:id}); + }; + const batchTask='bounded-batches'; + const batchCapture=await openBatch(batchTask); + assert.ok(batchCapture.elements.some(item=>item.selector==='#batch-container' && item.unique)); + assert.ok(batchCapture.elements.some(item=>item.selector==='#batch-window' && item.unique)); + const multiple=await client.call('action',{task_id:batchTask,...batchParams}); + assert.deepEqual(multiple.result.items.map(item=>item.item_id),['a','b','c','d']); + assert.deepEqual(multiple.result.batch_summary,{batches_read:3,duplicates_removed:2,stop_reason:'batch_budget'}); + assert.equal(multiple.result.skipped_incomplete_count,3); + assert.equal(multiple.batch_evidence.length,3); + assert.deepEqual(multiple.batch_evidence.map(item=>[item.batch_index,item.added_count,item.duplicate_count]),[[1,2,0],[2,1,1],[3,1,1]]); + assert.ok(multiple.batch_evidence[0].capture.viewport_text.includes('正文a'),'first batch is captured before scrolling'); + assert.ok(!multiple.batch_evidence[2].capture.viewport_text.includes('正文a'),'later actual viewport is a distinct page capture'); + assert.equal(new Set(multiple.batch_evidence.map(item=>item.capture.capture_id)).size,3); + for(const evidence of multiple.batch_evidence){const bytes=await fs.readFile(evidence.capture.screenshot_path);assert.equal(bytes.subarray(1,4).toString(),'PNG');} + await client.call('close',{task_id:batchTask}); + passed('容器内多批先读首批再滚动，完整行按业务id聚合去重，各批保留独立真实PNG和来源'); + + const boundaryTask='batch-boundary'; + await openBatch(boundaryTask,batchUrl+'?mode=boundary'); + const bounded=await client.call('action',{task_id:boundaryTask,...batchParams,batch:{...batchParams.batch,max_batches:5}}); + assert.deepEqual(bounded.result.items.map(item=>item.item_id),['a','b','c']); + assert.equal(bounded.result.batch_summary.stop_reason,'boundary'); + assert.equal(bounded.result.batch_summary.batches_read,2); + const limitReplay=await client.call('verify',{task_id:boundaryTask,entry_url:batchUrl, + steps:[{...batchParams,limit:3}],parameters:{},expected_outputs:{comments:'正文c'}}); + assert.equal(limitReplay.passed,true,JSON.stringify(limitReplay.errors)); + assert.equal(limitReplay.outputs.comments.batch_summary.stop_reason,'limit_reached'); + assert.equal(limitReplay.evidence.length,4,'entry, both batch pages, and final after evidence'); + assert.deepEqual(limitReplay.error_codes,[]); + await client.call('close',{task_id:boundaryTask}); + passed('批次边界与总条数预算明确结束，重放保留每批证据且不将预算当完整覆盖'); + + for(const [mode,expected] of [['fit',['a','b']],['lazy',['a','b','c','d']]]){ + const id=`batch-${mode}`,observed=await openBatch(id,`${batchUrl}?mode=${mode}`); + if(mode==='fit') assert.equal(observed.elements.find(item=>item.selector==='#batch-window').scroll_y.can_scroll,false); + const read=await client.call('action',{task_id:id,...batchParams,batch:{...batchParams.batch,max_batches:5}}); + assert.deepEqual(read.result.items.map(item=>item.item_id),expected); + assert.equal(read.result.batch_summary.stop_reason,'boundary'); + if(mode==='fit') assert.equal(read.result.batch_summary.batches_read,1); + await client.call('close',{task_id:id}); + } + passed('无溢出的已观察滚动容器仍先读首批再边界结束；边界同高度懒加载新业务id重新读取，不把0位移当内容完整'); + + for(const [mode,code,minimumEvidence] of [['conflict','DUPLICATE_ID_CONFLICT',2],['source','SOURCE_CHANGED',1],['root','SOURCE_CHANGED',1],['blocked','WRITE_REQUEST_BLOCKED',1]]){ + const id=`batch-reject-${mode}`; + await openBatch(id,`${batchUrl}?mode=${mode}`); + await assert.rejects(()=>client.call('action',{task_id:id,...batchParams}),error=>{ + assert.ok(error.before && error.after); + assert.ok(error.batch_evidence.length>=minimumEvidence); + assert.equal(new Set(error.batch_evidence.map(item=>item.capture.capture_id)).size,error.batch_evidence.length); + if(mode==='blocked'){ + assert.ok(error.diagnostics.blocked_requests.some(item=>item.pathname==='/fixture/write/batch-unknown')); + assert.ok(!JSON.stringify(error.diagnostics).includes('BATCH-QUERY-SECRET')); + assert.ok(!JSON.stringify(error.diagnostics).includes('BATCH-BODY-SECRET')); + } + return error.code===code; + }); + if(mode==='blocked'){ + const failedBatchReplay=await client.call('verify',{task_id:id,entry_url:`${batchUrl}?mode=blocked`,steps:[batchParams],parameters:{},expected_outputs:{comments:'正文c'}}); + assert.equal(failedBatchReplay.passed,false); + assert.deepEqual(failedBatchReplay.error_codes,['WRITE_REQUEST_BLOCKED']); + assert.ok(failedBatchReplay.evidence.some(capture=>capture.viewport_text.includes('正文a'))); + assert.ok(failedBatchReplay.evidence.length>=4,'failed replay retains initial, intermediate batch, and actual failure captures'); + } + await client.call('close',{task_id:id}); + } + assert.equal(blockedWriteReachedServer,0); + passed('重复业务id字段冲突、来源/根替换和未知POST均停止，失败保存已得批次图并保持请求保护'); + + const contractTask='batch-contracts'; + const contractCapture=await openBatch(contractTask); + for(const params of [ + {...batchParams,row_selector:undefined}, {...batchParams,complete_only:false}, + {...batchParams,target:{field:'text',value:'任意'}}, {...batchParams,batch:{...batchParams.batch,max_batches:6}}, + {...batchParams,batch:{...batchParams.batch,dedupe_field:'content'}}, {...batchParams,batch:{...batchParams.batch,scroll_selector:'body'}}, + {...batchParams,batch:{...batchParams.batch,unknown:true}}, {...batchParams,row_selector:'body .batch-row'}, + ]) await assert.rejects(()=>client.call('action',{task_id:contractTask,...params}),error=>error.code==='INVALID_INPUT'); + await assert.rejects(()=>client.call('action',{task_id:contractTask,...batchParams,batch:{...batchParams.batch,scroll_selector:'#other-root'}}),error=>error.code==='SOURCE_CHANGED'); + const batchTitles=contractCapture.elements.find(item=>item.selector==='#titles a.title'); + assert.ok(batchTitles && batchTitles.collection); + await assert.rejects(()=>client.call('action',{task_id:contractTask,kind:'wait',selector:batchTitles.selector,state:'visible',timeout_ms:300}),error=>error.code==='NON_UNIQUE_TARGET'); + const collectionWait=await client.call('action',{task_id:contractTask,kind:'wait',selector:batchTitles.selector,state:'visible',min_count:2,timeout_ms:300}); + assert.equal(collectionWait.result.satisfied,true); + await assert.rejects(()=>client.call('action',{task_id:contractTask,kind:'wait',selector:batchTitles.selector,state:'visible',min_count:3,timeout_ms:100}),error=>error.code==='WAIT_TIMEOUT'); + await assert.rejects(()=>client.call('action',{task_id:contractTask,kind:'wait',selector:batchTitles.selector,state:'text',text:'标题甲',min_count:1,timeout_ms:100}),error=>error.code==='INVALID_INPUT'); + await client.call('close',{task_id:contractTask}); + passed('批次合同/范围越界不放宽；无标题首卡后的已观察标题集合显式min_count可等待，默认多匹配仍拒绝'); + + const delayedTask='new-session-delayed-collection',delayedUrl=new URL('/fixture/delayed-collection',entryUrl).href; + await client.call('open',{task_id:delayedTask,account_key:'test-account',entry_url:delayedUrl}); + await client.call('ready',{task_id:delayedTask,identity_label:'无预存集合样本'}); + const delayedReplay=await client.call('verify',{task_id:delayedTask,entry_url:delayedUrl,parameters:{}, + steps:[{kind:'wait',selector:'#delayed-titles a.title',state:'visible',min_count:2,timeout_ms:5000}, + {kind:'read',selector:'#delayed-titles',output_name:'titles'}],expected_outputs:{titles:'标题乙'}}); + assert.equal(delayedReplay.passed,true,JSON.stringify(delayedReplay.errors)); + assert.ok(!delayedReplay.evidence[0].elements.some(element=>element.selector==='#delayed-titles a.title')); + await client.call('close',{task_id:delayedTask}); + passed('全新会话无预存标题集合时，min_count仍从0项等待真实延迟结果，不在等待前误报未观察'); + + const emptyUrl=new URL('/fixture/empty-read',entryUrl).href; + const emptyParams={kind:'extract',selector:'#empty-container',row_selector:'.empty-row', + fields:[{name:'content',selector:'.content',source:'text',required:true}],limit:3,output_name:'comments',complete_only:true, + allow_empty:true,empty_state:{selector:'.empty-state',text:'暂无评论'}}; + const emptyTask='confirmed-empty'; + await openBatch(emptyTask,emptyUrl); + const empty=await client.call('action',{task_id:emptyTask,...emptyParams}); + assert.deepEqual(empty.result,{items:[],count:0,truncated:false,source_url:emptyUrl,coverage:'empty',skipped_incomplete_count:0, + empty_state:{confirmed:true,selector:'.empty-state',text:'暂无评论'}}); + const emptyReplay=await client.call('verify',{task_id:emptyTask,entry_url:emptyUrl,steps:[emptyParams],parameters:{},expected_outputs:{comments:{state:'empty'}}}); + assert.equal(emptyReplay.passed,true,JSON.stringify(emptyReplay.errors)); + const textStillFails=await client.call('verify',{task_id:emptyTask,entry_url:emptyUrl,steps:[emptyParams],parameters:{},expected_outputs:{comments:'暂无评论'}}); + assert.equal(textStillFails.passed,false,'empty proof must not satisfy a content text assertion'); + await assert.rejects(()=>client.call('action',{task_id:emptyTask,kind:'extract',selector:'#empty-container',fields:[{name:'date',selector:'.whitespace-time',source:'attribute',attribute:'datetime',required:true}],limit:1,output_name:'dates'}),error=>error.code==='REQUIRED_FIELD_MISSING'); + for(const bad of [{...emptyParams,empty_state:{selector:'.empty-state',text:'没有结果'}}, {...emptyParams,row_selector:undefined}, {...emptyParams,allow_empty:'true'}, + {...batchParams,allow_empty:true,empty_state:{selector:'.empty-state',text:'暂无评论'}}]) { + await assert.rejects(()=>client.call('action',{task_id:emptyTask,...bad}),error=>['INVALID_INPUT','EMPTY_STATE_UNCONFIRMED'].includes(error.code)); + } + await client.call('close',{task_id:emptyTask}); + passed('唯一完整可见空态经过两次真实复核才能空预期通过；文本预期仍拒空，required空白属性拒绝'); + + for(const mode of ['busy','hidden','duplicate']){ + const id=`empty-reject-${mode}`; + await openBatch(id,`${emptyUrl}?mode=${mode}`); + await assert.rejects(()=>client.call('action',{task_id:id,...emptyParams}),error=>error.code==='EMPTY_STATE_UNCONFIRMED'); + await client.call('close',{task_id:id}); + } + const realRowTask='empty-is-nonempty'; + await openBatch(realRowTask,emptyUrl+'?mode=rows'); + const realRows=await client.call('action',{task_id:realRowTask,...emptyParams}); + assert.equal(realRows.result.count,1); + assert.equal(realRows.result.empty_state,undefined); + await client.call('close',{task_id:realRowTask}); + passed('忙碌、隐藏或多重空态均不确认为零；已有真实行时按普通内容读取而不伪造空态'); + + const nonemptyTask='nonempty-structure-assertion'; + await openBatch(nonemptyTask,actionUrl); + const nonemptyStep={kind:'extract',selector:'[data-testid="row"]',limit:2,output_name:'notes',fields:[ + {name:'title',selector:'.missing-title',source:'text'}, {name:'author',selector:'.missing-author',source:'text'}, + {name:'source',selector:'.source',source:'href',required:true}]}; + const nonempty=await client.call('verify',{task_id:nonemptyTask,entry_url:actionUrl,steps:[nonemptyStep],parameters:{},expected_outputs:{notes:{state:'nonempty'}}}); + assert.equal(nonempty.passed,true,JSON.stringify(nonempty.errors)); + assert.equal(nonempty.outputs.notes.count,2); + assert.ok(nonempty.outputs.notes.items.every(item=>item.title===null && item.author===null && item.source.startsWith(actionUrl)),'only schema nonempty is asserted; optional text can be unavailable'); + const actualEmpty=await client.call('verify',{task_id:nonemptyTask,entry_url:emptyUrl,steps:[emptyParams],parameters:{},expected_outputs:{comments:{state:'nonempty'}}}); + assert.equal(actualEmpty.passed,false,'confirmed zero rows cannot satisfy nonempty'); + const croppedOnly=await client.call('verify',{task_id:nonemptyTask,entry_url:windowUrl,parameters:{}, + steps:[{kind:'extract',selector:'#partial-row',fields:clipFields,limit:1,output_name:'comments',complete_only:true}],expected_outputs:{comments:{state:'nonempty'}}}); + assert.equal(croppedOnly.outputs.comments.count,0); + assert.equal(croppedOnly.outputs.comments.skipped_incomplete_count,1); + assert.equal(croppedOnly.passed,false,'a clipped-only extraction is not a nonempty result'); + const textRead=await client.call('verify',{task_id:nonemptyTask,entry_url:emptyUrl,parameters:{}, + steps:[{kind:'read',selector:'#empty-container',output_name:'notes'}],expected_outputs:{notes:{state:'nonempty'}}}); + assert.equal(textRead.passed,false,'text reads cannot borrow a structured extraction assertion'); + await assert.rejects(()=>client.call('verify',{task_id:nonemptyTask,entry_url:actionUrl,steps:[nonemptyStep],parameters:{},expected_outputs:{notes:{state:'nonempty',minimum:0}}}),error=>error.code==='INVALID_EXPECTED_OUTPUT'); + await client.call('close',{task_id:nonemptyTask}); + passed('nonempty仅确认合法结构化非空；零行/仅裁剪行/普通read和带额外键对象拒绝，不代替语义判断'); + + const realNonempty=nonempty.outputs.notes; + assert.equal(validNonemptyExtractOutput(nonemptyStep,realNonempty),true); + const corrupted=[{...realNonempty,count:1},{...realNonempty,count:true},{...realNonempty,count:1.5},{...realNonempty,truncated:'true'}, + {...realNonempty,coverage:'empty'},{...realNonempty,source_url:'https://user:password@example.invalid/'},{...realNonempty,unexpected:true}, + {...realNonempty,items:[{...realNonempty.items[0],source:' '},realNonempty.items[1]]}, + {...realNonempty,items:[{source:realNonempty.items[0].source},realNonempty.items[1]]}, + {...realNonempty,items:[{...realNonempty.items[0],title:7},realNonempty.items[1]]}]; + for(const output of corrupted) assert.equal(validNonemptyExtractOutput(nonemptyStep,output),false); + assert.equal(validNonemptyExtractOutput({...nonemptyStep,limit:0},realNonempty),false,'an invalid producing contract cannot pass'); + assert.equal(validNonemptyExtractOutput({...nonemptyStep,kind:'read'},realNonempty),false); + assert.equal(validNonemptyExtractOutput(batchParams,multiple.result),true); + assert.equal(validNonemptyExtractOutput(batchParams,{...multiple.result,batch_summary:{...multiple.result.batch_summary,batches_read:6}}),false); + assert.equal(validNonemptyExtractOutput(batchParams,{...multiple.result,items:multiple.result.items.map(()=>multiple.result.items[0])}),false,'duplicate business IDs invalidate a batch output'); + assert.equal(validNonemptyExtractOutput(batchParams,{...multiple.result,skipped_incomplete_count:-1}),false); + passed('nonempty同时核对真实输出字段/计数/必填值/来源及批次形状，损坏结果和非法生产合同不能通过'); + + const bindingEntry=new URL('/fixture/note-binding',entryUrl).href; + const boundSteps=[ + {kind:'fill',selector:'#binding-search',parameter:'keyword'}, + {kind:'press',selector:'#binding-search',key:'Enter'}, + {kind:'wait',selector:'#binding-results a.binding-title',state:'visible',min_count:1,timeout_ms:1500}, + {kind:'click',selector:'#binding-results a.binding-title',target:{field:'text',parameter:'note_title'}}, + {kind:'wait',selector:'#binding-detail',state:'visible',timeout_ms:1500}, + {kind:'extract',selector:'#binding-detail',output_name:'note_detail',limit:1,fields:[ + {name:'title',selector:'.detail-title',source:'text',required:true},{name:'body',selector:'.detail-body',source:'text',required:true},{name:'author',selector:'.detail-author',source:'text',required:true}]}, + {kind:'extract',selector:'#binding-comments-root',row_selector:'.binding-comment',output_name:'comments',limit:3,complete_only:true,fields:[ + {name:'comment_id',source:'attribute',attribute:'id',required:true},{name:'author',selector:'.comment-author',source:'text',required:true},{name:'body',selector:'.comment-body',source:'text',required:true}], + batch:{scroll_selector:'#binding-scroller',pixels:80,max_batches:2,dedupe_field:'comment_id'}}, + ]; + const boundValues={steps:boundSteps,parameters:{keyword:'装修',note_title:bindingTitle},expected_outputs:{note_detail:[bindingTitle,'合成正文'],comments:{state:'nonempty'}}}; + const bindingTask='verify-public-note-binding'; + await openBatch(bindingTask,bindingEntry); + const readBound=(mode,noteId=bindingNoteId,extra={})=>client.call('verify',{task_id:bindingTask,entry_url:bindingEntry+'?mode='+mode,...boundValues,note_binding:{note_id:noteId},...extra}); + const matched=await readBound('normal'); + assert.equal(matched.passed,true,JSON.stringify(matched.errors)); + assert.deepEqual(matched.note_binding_result,{note_id:bindingNoteId,href_checked:true,url_checked:true,source_url:new URL('/explore/'+bindingNoteId,entryUrl).href,title:bindingTitle}); + assert.equal(matched.outputs.note_detail.source_url,matched.note_binding_result.source_url); + assert.equal(matched.outputs.comments.source_url,matched.note_binding_result.source_url); + assert.ok(matched.outputs.comments.count>0); + assert.ok(!JSON.stringify(matched.note_binding_result).includes('FIXTURE-BINDING-TOKEN')); + assert.ok(!JSON.stringify(matched).includes('FIXTURE-BINDING-HASH')); + passed('合法七步详情链唯一标题点击前后核对可信note ID，正文/评论来源及实际标题绑定输出并保留批次证据'); + + const beforeWrongClick=boundDetailRequests; + const wrongHref=await readBound('wrong-href'); + assert.equal(wrongHref.passed,false); + assert.deepEqual(wrongHref.error_codes,['NOTE_BINDING_MISMATCH']); + assert.equal(boundDetailRequests,beforeWrongClick,'a same-title link for another ID must never be opened'); + assert.deepEqual(wrongHref.outputs,{}); + assert.equal(wrongHref.note_binding_result.href_checked,false); + assert.equal(wrongHref.note_binding_result.url_checked,false); + assert.equal(wrongHref.note_binding_result.title,null); + assert.ok(wrongHref.evidence.length>=5); + const redirected=await readBound('redirect'); + assert.equal(redirected.passed,false); + assert.deepEqual(redirected.error_codes,['NOTE_BINDING_MISMATCH']); + assert.equal(redirected.note_binding_result.href_checked,true); + assert.equal(redirected.note_binding_result.url_checked,false); + assert.deepEqual(redirected.outputs,{}); + assert.ok(redirected.evidence.length>=5); + passed('同标题另一ID在点击前拒绝且不请求详情；点击后同站ID重定向拒绝，均无正文/评论输出并保留真实失败图'); + + const changedTitle=await readBound('wrong-title'); + assert.equal(changedTitle.passed,false); + assert.deepEqual(changedTitle.error_codes,['NOTE_BINDING_MISMATCH']); + assert.equal(changedTitle.note_binding_result.title,'同ID但标题已变化'); + assert.equal(changedTitle.outputs.comments,undefined); + const drifted=await readBound('drift-scroll'); + assert.equal(drifted.passed,false); + assert.deepEqual(drifted.error_codes,['NOTE_BINDING_MISMATCH']); + assert.equal(drifted.note_binding_result.url_checked,false); + assert.equal(drifted.outputs.note_detail.items[0].title,bindingTitle); + assert.equal(drifted.outputs.comments,undefined,'URL drift during comment batching cannot produce a completed comment output'); + assert.ok(drifted.evidence.length>matched.evidence.length-2,'already obtained batch and actual failure page evidence are retained'); + passed('实际标题改变停止后续评论；批次滚动期间URL ID漂移停止并保留此前有效正文及已得批次图'); + + const phoneBound=await readBound('phone',bindingPhoneId); + assert.equal(phoneBound.passed,true,JSON.stringify(phoneBound.errors)); + const publicSource=new URL('/explore/'+bindingPhoneId,entryUrl).href; + assert.deepEqual(phoneBound.note_binding_result,{note_id:bindingPhoneId,href_checked:true,url_checked:true,source_url:publicSource,title:bindingTitle}); + assert.equal(phoneBound.outputs.note_detail.source_url,publicSource); + assert.equal(phoneBound.outputs.comments.source_url,publicSource); + const unbound=await readBound('wrong-href',bindingNoteId,{note_binding:undefined}); + assert.equal(unbound.passed,true,JSON.stringify(unbound.errors)); + assert.equal(unbound.note_binding_result,undefined,'legacy unbound verify semantics remain unchanged'); + passed('phone样数字的合成公共hexID按实际URL校验不误伤；显式绑定来源用canonical，旧无绑定语义保持'); + + const beforeBadContract=boundDetailRequests; + for(const [extra,code] of [ + [{note_binding:{note_id:bindingNoteId,extra:true}},'INVALID_NOTE_BINDING'], + [{note_binding:{note_id:'invalid'}},'INVALID_NOTE_BINDING'], + [{steps:boundSteps.slice(0,4)},'NOTE_BINDING_CONTRACT'], + [{steps:boundSteps.map((step,index)=>index===1 ? {...step,selector:'#another-search'} : step)},'NOTE_BINDING_CONTRACT'], + [{steps:boundSteps.map((step,index)=>index===6 ? {...step,fields:step.fields.map(field=>field.name==='comment_id' ? {...field,attribute:'data-item-id'} : field)} : step)},'NOTE_BINDING_CONTRACT'], + ]) await assert.rejects(()=>readBound('normal',bindingNoteId,extra),error=>error.code===code); + assert.equal(boundDetailRequests,beforeBadContract,'invalid binding contracts fail before navigation/click'); + await client.call('close',{task_id:bindingTask}); + passed('绑定参数与七步/动态标题/字段/搜索目标合同严格核对，不能借绑定执行任意click或其它读取链'); + + await seekRegression(client,entryUrl,passed,{bindingNoteId,bindingTitle,detailRequests:()=>boundDetailRequests}); + await viewportRegression(client,entryUrl,passed); + await budgetRegression(entryUrl,home,passed,{bindingNoteId,bindingTitle}); + await contentRegression(client,entryUrl,passed); + await metadataRegression(client,entryUrl,passed,home,CHANNEL,semanticChanges); + await client.call('close', { task_id: 'regression-a' }); const reopened = await client.call('open', { task_id: 'regression-c', account_key: 'test-account', entry_url: entryUrl }); assert.notEqual(reopened.session_id, opened.session_id); diff --git a/tools/learning/test-content-scenarios.mjs b/tools/learning/test-content-scenarios.mjs new file mode 100644 index 0000000..d2cd630 --- /dev/null +++ b/tools/learning/test-content-scenarios.mjs @@ -0,0 +1,100 @@ +import assert from 'node:assert/strict'; +import {validateAction} from './action-contract.mjs'; +import {validContentValue,validNonemptyExtractOutput} from './browser-worker.mjs'; + +const phases=['validate','inspect_current','capture_before','resolve_target','execute','settle','request_guard','capture_after','failure_capture','batch_capture']; + +export function contentFixture(url,request,response){ + if(url.pathname==='/fixture/content-loading'){response.writeHead(404,{'Content-Type':'image/png'}).end('not an image');return true;} + if(url.pathname==='/fixture/content-image'){ + response.writeHead(200,{'Content-Type':'image/svg+xml'}).end('');return true; + } + if(url.pathname!=='/fixture/content-comments') return false; + const mode=url.searchParams.get('mode') || 'text'; + const image=(extra='')=>``; + const row=(id,body)=>``; + const text='真实文字，装修需求'; + let body=mode==='text' ? text : mode==='media' || mode.includes('lazy') || mode==='loading' ? image() : mode==='mixed' ? text+image() : + mode==='hidden' ? image('style="display:none"') : mode==='avatar' ? ''+image()+'' : + mode==='sensitive' ? '
    '+image()+'
    ' : mode==='multiple' ? text+text : mode==='many' ? Array.from({length:5},()=>image()).join('') : + mode==='video' ? '' : ''; + let rows=row('a',body); + if(mode==='cropped-lazy') rows='
    '+rows; + if(mode==='offscreen-lazy') rows=row('first',text)+'
    '+row('a',image()); + response.setHeader('Content-Type','text/html; charset=utf-8'); + response.end(`显式评论内容类型
    ${rows}
    ${mode==='video' ? `` : ''}`); + return true; +} + +function checkTiming(result,steps){ + assert.deepEqual(Object.keys(result.verify_timing).sort(),['duration_ms','entry_ms','close_ms'].sort()); + assert.equal(result.verify_timing.close_ms,null); + assert.ok(Number.isInteger(result.verify_timing.duration_ms) && result.verify_timing.duration_ms>=0); + assert.ok(Number.isInteger(result.verify_timing.entry_ms) && result.verify_timing.entry_ms>=0); + assert.equal(result.step_timings.length,steps.length); + for(const [index,item] of result.step_timings.entries()){ + assert.deepEqual(Object.keys(item).sort(),['step_position','kind','status','duration_ms','phase_ms'].sort()); + assert.equal(item.step_position,index+1);assert.equal(item.kind,steps[index].kind); + assert.ok(Object.keys(item.phase_ms).every(key=>phases.includes(key)));assert.ok(Object.hasOwn(item.phase_ms,'validate')); + assert.equal(Object.hasOwn(item.phase_ms,'capture_before'),false,'verify reuses prior evidence rather than timing an unexecuted screenshot'); + assert.ok(Object.values(item.phase_ms).every(value=>Number.isInteger(value) && value>=0)); + assert.equal(Object.values(item.phase_ms).reduce((sum,value)=>sum+value,0),item.duration_ms,'phase durations are exclusive'); + assert.ok(item.duration_ms<=result.verify_timing.duration_ms+1); + } +} + +export async function contentRegression(client,entryUrl,passed){ + const task='explicit-comment-content',base=new URL('/fixture/content-comments',entryUrl).href; + await client.call('open',{task_id:task,account_key:'test-account',entry_url:base}); + await client.call('ready',{task_id:task,identity_label:'隔离内容类型测试'}); + const fields=[{name:'comment_id',source:'attribute',attribute:'id',required:true}, + {name:'author',selector:'.content-author',source:'text',required:true}, + {name:'body',selector:'.content-root',source:'content',text_selector:'.content-text',media_selector:'.content-media',max_media:2,required:true}]; + const step={kind:'extract',selector:'#content-container',row_selector:'.content-row',complete_only:true,fields,limit:5,output_name:'comments'}; + const run=(mode,extra={})=>client.call('verify',{task_id:task,entry_url:base+'?mode='+mode,parameters:{},steps:[step],expected_outputs:{comments:{state:'nonempty'}},...extra}); + for(const mode of ['text','media','mixed','video']){ + const result=await run(mode);assert.equal(result.passed,true,mode+': '+JSON.stringify(result.errors)); + const body=result.outputs.comments.items[0].body; + assert.equal(body.kind,mode==='video' ? 'media' : mode);assert.equal(validContentValue(fields[2],body),true); + assert.equal(body.text===null,['media','video'].includes(mode));assert.equal(body.media.length,mode==='text' ? 0 : 1); + if(body.media.length) assert.equal(body.media[0].kind,mode==='video' ? 'video' : 'image'); + assert.equal(validNonemptyExtractOutput(step,result.outputs.comments),true);checkTiming(result,[step]); + assert.equal(result.step_timings[0].status,'passed'); + assert.ok(!JSON.stringify(result).includes('RAW-MEDIA-SECRET'),'media source URLs are never returned'); + } + passed('显式content合同从真实可见DOM区分文字/图片/混合/已加载video；正文仅真实text，媒体仅相对CSS且无src/OCR'); + const capture=await client.call('capture',{task_id:task}); + const hints=capture.elements.flatMap(element=>element.read_hints || []).filter(hint=>hint.source==='media'); + assert.ok(hints.some(hint=>hint.media_kind==='video' && hint.loaded===true)); + assert.ok(capture.elements.every(element=>(element.read_hints || []).length<=12)); + const capped=await run('many');assert.equal(capped.passed,true,JSON.stringify(capped.errors)); + assert.equal(capped.outputs.comments.items[0].body.media.length,2);assert.equal(capped.outputs.comments.items[0].body.media_truncated,true);assert.equal(capped.outputs.comments.truncated,true); + const corrupt={...capped.outputs.comments,truncated:false};assert.equal(validNonemptyExtractOutput(step,corrupt),false); + const textAssertion=await run('media',{expected_outputs:{comments:'image'}});assert.equal(textAssertion.passed,false,'kind cannot satisfy ordinary content assertions'); + checkTiming(textAssertion,[step]);assert.equal(textAssertion.step_timings[0].status,'passed','a later output assertion does not rewrite action timing'); + const cssAssertion=await run('media',{expected_outputs:{comments:'img'}});assert.equal(cssAssertion.passed,false,'media CSS cannot satisfy ordinary content assertions'); + passed('有界媒体hint可学习相对CSS与加载事实；max_media截断传播到顶层，类型/定位字符串不能冒充文字断言'); + for(const [mode,code] of [['empty','REQUIRED_FIELD_MISSING'],['loading','CONTENT_MEDIA_NOT_READY'],['hidden','REQUIRED_FIELD_MISSING'],['avatar','UNSAFE_CONTENT_MEDIA'],['sensitive','SENSITIVE_OUTPUT'],['multiple','NON_UNIQUE_FIELD']]){ + const result=await run(mode);assert.equal(result.passed,false,mode);assert.deepEqual(result.error_codes,[code],JSON.stringify(result.errors)); + assert.equal(result.outputs.comments,undefined);checkTiming(result,[step]);assert.equal(result.step_timings[0].status,'failed'); + assert.ok(result.step_timings[0].phase_ms.failure_capture>0); + } + const old=await run('media',{steps:[{...step,fields:fields.map(field=>field.name==='body' ? {name:'body',selector:'.content-text',source:'text',required:true} : field)}]}); + assert.equal(old.passed,false);assert.deepEqual(old.error_codes,['REQUIRED_FIELD_MISSING']); + passed('空/未加载/隐藏/头像/敏感/多文本候选明确失败且保真实failed时序；旧文字required仍不接受纯媒体'); + const cropped=await run('cropped-lazy');assert.equal(cropped.passed,false);assert.deepEqual(cropped.error_codes,[],'clipped lazy media is skipped rather than misreported as missing'); + assert.equal(cropped.outputs.comments.count,0);assert.equal(cropped.outputs.comments.skipped_incomplete_count,1); + const offscreen=await run('offscreen-lazy');assert.equal(offscreen.passed,true,JSON.stringify(offscreen.errors));assert.equal(offscreen.outputs.comments.count,1);assert.equal(offscreen.outputs.comments.skipped_incomplete_count,0); + const realMissing=await run('cropped-lazy',{steps:[{...step,fields:[...fields,{name:'later',selector:'.truly-missing',source:'text',required:true}]}]}); + assert.deepEqual(realMissing.error_codes,['REQUIRED_FIELD_MISSING'],'skipping a clipped content field must not hide a later missing required field'); + passed('complete_only裁剪lazy媒体只记跳过、完全离屏沿旧范围忽略；完整可见未加载失败，裁剪不掩盖后续真实缺失'); + const batchStep={...step,batch:{scroll_selector:'#content-window',pixels:130,max_batches:1,dedupe_field:'comment_id'}}; + const batched=await run('text',{steps:[batchStep]});assert.equal(batched.passed,true,JSON.stringify(batched.errors));assert.ok(batched.step_timings[0].phase_ms.batch_capture>0); + assert.equal(batched.step_timings[0].phase_ms.execute+batched.step_timings[0].phase_ms.batch_capture<=batched.step_timings[0].duration_ms,true); + for(const patch of [ + {required:false},{max_media:0},{max_media:5},{text_selector:'body'},{media_selector:'javascript:alert(1)'},{attribute:'src'}, + ]) assert.throws(()=>validateAction({...step,fields:[...fields.slice(0,2),{...fields[2],...patch}]}),error=>error.code==='INVALID_INPUT'); + assert.throws(()=>validateAction({...step,fields:[{name:'body',source:'text',required:true,media_selector:'img'}]}),error=>error.code==='INVALID_INPUT'); + await client.call('close',{task_id:task}); + passed('批次截图单独独占计时，10phase合计真实step耗时；媒体配置严格有界且不能混入旧字段源'); +} diff --git a/tools/learning/test-control.mjs b/tools/learning/test-control.mjs new file mode 100644 index 0000000..eab0bd6 --- /dev/null +++ b/tools/learning/test-control.mjs @@ -0,0 +1,184 @@ +import assert from 'node:assert/strict'; +import { createServer } from 'node:http'; +import { Readable, Writable } from 'node:stream'; +import { LearningControlClient, main, readControlInput } from './control.mjs'; + +// Isolated transport fixtures only: no browser, real accounts or development API. +const calls = []; +const secret = 'Transport-fixture-password-51'; +let csrf = 'fixture-csrf-before-login'; +let actions = 0; +let redirect = false; +const server = createServer(async (request, response) => { + const buffers = []; + for await (const chunk of request) buffers.push(chunk); + const body = buffers.length ? JSON.parse(Buffer.concat(buffers).toString()) : undefined; + calls.push({ method: request.method, path: request.url }); + const send = (status, data, cookie) => { + response.writeHead(status, { 'Content-Type': 'application/json', ...(cookie ? { 'Set-Cookie': cookie } : {}) }); + response.end(JSON.stringify(data)); + }; + if (request.url === '/api/health') return send(200, { status: 'ok' }); + if (request.url === '/api/auth/status') return send(200, { csrf_token: csrf, authenticated: false }, 'assistant_maintainer=anonymous; HttpOnly; SameSite=Strict'); + if (request.url === '/api/auth/login') { + assert.equal(request.headers['x-csrf-token'], csrf); + assert.match(request.headers.cookie, /assistant_maintainer=anonymous/); + assert.equal(body.password, secret); + if (redirect) { response.writeHead(302, { Location: '/unexpected-redirect' }); response.end(); return; } + csrf = 'fixture-csrf-after-login'; + return send(200, { csrf_token: csrf, authenticated: true }, 'assistant_maintainer=authenticated; HttpOnly; SameSite=Strict'); + } + assert.equal(request.headers.origin, `http://127.0.0.1:${server.address().port}`); + assert.match(request.headers.cookie, /assistant_maintainer=authenticated/); + if (request.method === 'POST') assert.equal(request.headers['x-csrf-token'], csrf); + if (request.url === '/api/learning/tasks/fail/browser/action') return send(409, { detail: secret }); + if (request.url === '/api/learning/tasks/fail_code/browser/action') return send(409, { detail: { code: body.fixture_code, message: secret, csrf_token: csrf } }); + if (request.url.endsWith('/browser/action')) { + actions += 1; + return send(200, { result: { kind: body.kind } }); + } + return send(200, { items: [] }); +}); +await new Promise(resolve => server.listen(0, '127.0.0.1', resolve)); +const baseUrl = `http://127.0.0.1:${server.address().port}`; +const environment = { ASSISTANT_CONTROL_URL: baseUrl, ASSISTANT_MAINTAINER_USERNAME: 'fixture-maintainer', ASSISTANT_MAINTAINER_PASSWORD: secret }; +function outputBuffer() { + let value = ''; + return { output: new Writable({ write(chunk, encoding, next) { value += chunk; next(); } }), text: () => value }; +} +try { + for (const base of ['https://127.0.0.1:8810', 'http://example.com', 'http://user:pass@127.0.0.1:8810', baseUrl + '/api', baseUrl + '?token=secret']) { + assert.throws(() => new LearningControlClient({ baseUrl: base }), error => error.code === 'INVALID_BASE_URL'); + } + const client = new LearningControlClient({ baseUrl, username: 'fixture-maintainer', password: secret }); + assert.deepEqual(await client.execute({ operation: 'health' }), { status: 'ok' }); + const beforeInvalid = calls.length; + await assert.rejects(client.execute({ operation: 'shell.run' }), error => error.code === 'UNKNOWN_OPERATION'); + await assert.rejects(client.execute({ operation: 'tasks.get', task_id: '../auth/status' }), error => error.code === 'INVALID_REQUEST'); + await assert.rejects(client.execute({ operation: 'discovery.get', run_id: '../auth/status' }), error => error.code === 'INVALID_REQUEST'); + await assert.rejects(client.execute({ operation: 'readings.list', run_id: '../auth/status' }), error => error.code === 'INVALID_REQUEST'); + await assert.rejects(client.execute({ operation: 'readings.create', run_id: '../auth/status', data: {} }), error => error.code === 'INVALID_REQUEST'); + await assert.rejects(client.execute({ operation: 'readings.get', reading_id: '../auth/status' }), error => error.code === 'INVALID_REQUEST'); + await assert.rejects(client.execute({ operation: 'analyses.list', reading_id: '../auth/status' }), error => error.code === 'INVALID_REQUEST'); + await assert.rejects(client.execute({ operation: 'analyses.create', reading_id: '../auth/status', data: {} }), error => error.code === 'INVALID_REQUEST'); + await assert.rejects(client.execute({ operation: 'analyses.get', analysis_id: '../auth/status' }), error => error.code === 'INVALID_REQUEST'); + for (const operation of ['briefs.get', 'briefs.confirm', 'briefs.discover']) { + await assert.rejects(client.execute({ operation, brief_id: '../auth/status', data: {} }), error => error.code === 'INVALID_REQUEST'); + } + for (const operation of ['reviews.list', 'reviews.create', 'reviews.summary']) { + await assert.rejects(client.execute({ operation, analysis_id: '../auth/status', data: {} }), error => error.code === 'INVALID_REQUEST'); + } + await assert.rejects(client.execute({ operation: 'reviews.get', review_id: '../auth/status' }), error => error.code === 'INVALID_REQUEST'); + assert.equal(calls.length, beforeInvalid); + const discoveryStart = calls.length; + await client.execute({ operation: 'discovery.list' }); + await client.execute({ operation: 'discovery.get', run_id: 'discovery_fixture' }); + await client.execute({ operation: 'discovery.create', data: { search_candidate_id: 'candidate_fixture', keyword: '装修', max_selected: 2 } }); + assert.deepEqual(calls.slice(discoveryStart).map(({method, path}) => ({method, path})), [ + { method: 'GET', path: '/api/auth/status' }, + { method: 'POST', path: '/api/auth/login' }, + { method: 'GET', path: '/api/discovery/runs' }, + { method: 'GET', path: '/api/discovery/runs/discovery_fixture' }, + { method: 'POST', path: '/api/discovery/runs' }, + ]); + const routeStart = calls.length; + await client.execute({ operation: 'routes.list' }); + await client.execute({ operation: 'routes.resolve', data: { platform: 'demo', task_type: 'search_note_comments' } }); + await client.execute({ operation: 'routes.verify', data: { platform: 'demo', task_type: 'search_note_comments', keyword: '换参样例' } }); + assert.deepEqual(calls.slice(routeStart), [ + { method: 'GET', path: '/api/learning/routes' }, + { method: 'POST', path: '/api/learning/routes/resolve' }, + { method: 'POST', path: '/api/learning/routes/verify' }, + ]); + const actualReadingStart = calls.length; + await client.execute({ operation: 'readings.list', run_id: 'discovery_fixture' }); + await client.execute({ operation: 'readings.get', reading_id: 'reading_fixture' }); + await client.execute({ operation: 'readings.create', run_id: 'discovery_fixture', data: { detail_candidate_id: 'candidate_fixture', note_id: 'a'.repeat(24) } }); + assert.deepEqual(calls.slice(actualReadingStart), [ + { method: 'GET', path: '/api/discovery/runs/discovery_fixture/readings' }, + { method: 'GET', path: '/api/discovery/readings/reading_fixture' }, + { method: 'POST', path: '/api/discovery/runs/discovery_fixture/readings' }, + ]); + const analysisStart = calls.length; + await client.execute({ operation: 'model.config' }); + await client.execute({ operation: 'analyses.list', reading_id: 'reading_fixture' }); + await client.execute({ operation: 'analyses.create', reading_id: 'reading_fixture', data: {} }); + await client.execute({ operation: 'analyses.get', analysis_id: 'analysis_fixture' }); + assert.deepEqual(calls.slice(analysisStart), [ + { method: 'GET', path: '/api/analysis/config' }, + { method: 'GET', path: '/api/discovery/readings/reading_fixture/analyses' }, + { method: 'POST', path: '/api/discovery/readings/reading_fixture/analyses' }, + { method: 'GET', path: '/api/discovery/analyses/analysis_fixture' }, + ]); + const briefStart = calls.length; + await client.execute({ operation: 'briefs.list' }); + await client.execute({ operation: 'briefs.create', data: { request_text: '小红书寻找装修业主' } }); + await client.execute({ operation: 'briefs.get', brief_id: 'brief_fixture' }); + await client.execute({ operation: 'briefs.confirm', brief_id: 'brief_fixture', data: {} }); + await client.execute({ operation: 'briefs.discover', brief_id: 'brief_fixture', data: { search_candidate_id: 'candidate_fixture', max_selected: 2 } }); + assert.deepEqual(calls.slice(briefStart), [ + { method: 'GET', path: '/api/business/briefs' }, + { method: 'POST', path: '/api/business/briefs' }, + { method: 'GET', path: '/api/business/briefs/brief_fixture' }, + { method: 'POST', path: '/api/business/briefs/brief_fixture/confirm' }, + { method: 'POST', path: '/api/business/briefs/brief_fixture/discover' }, + ]); + const reviewStart = calls.length; + await client.execute({ operation: 'reviews.list', analysis_id: 'analysis_fixture' }); + await client.execute({ operation: 'reviews.create', analysis_id: 'analysis_fixture', data: { source_ref: 'note', previous_review_id: null, decision: 'uncertain', labels: null, note: '需要更多上下文' } }); + await client.execute({ operation: 'reviews.summary', analysis_id: 'analysis_fixture' }); + await client.execute({ operation: 'reviews.get', review_id: 'review_fixture' }); + assert.deepEqual(calls.slice(reviewStart), [ + { method: 'GET', path: '/api/discovery/analyses/analysis_fixture/reviews' }, + { method: 'POST', path: '/api/discovery/analyses/analysis_fixture/reviews' }, + { method: 'GET', path: '/api/discovery/analyses/analysis_fixture/review-summary' }, + { method: 'GET', path: '/api/discovery/reviews/review_fixture' }, + ]); + await client.execute({ operation: 'browser.action', task_id: 'fixture_task', data: { kind: 'scroll', selector: 'body', direction: 'down', pixels: 500, max_scrolls: 1 } }); + assert.equal(actions, 1); + const rejection = await client.execute({ operation: 'browser.action', task_id: 'fail', data: { kind: 'click', selector: 'button' } }).catch(error => error); + assert.equal(rejection.code, 'API_REJECTED'); + assert.equal(rejection.status, 409); + assert.equal(rejection.apiCode, undefined); + assert.ok(!rejection.message.includes(secret)); + assert.equal(actions, 1, 'A rejected write must not be retried'); + const codedRejection = await client.execute({ operation: 'browser.action', task_id: 'fail_code', data: { kind: 'click', selector: 'button', fixture_code: 'WRITE_REQUEST_BLOCKED' } }).catch(error => error); + assert.equal(codedRejection.code, 'API_REJECTED'); + assert.equal(codedRejection.status, 409); + assert.equal(codedRejection.apiCode, 'WRITE_REQUEST_BLOCKED'); + assert.equal(codedRejection.message, rejection.message); + assert.ok(!JSON.stringify(codedRejection).includes(secret) && !JSON.stringify(codedRejection).includes(csrf)); + const coded = outputBuffer(); + assert.equal(await main(Readable.from([JSON.stringify({ operation: 'browser.action', task_id: 'fail_code', data: { kind: 'click', selector: 'button', fixture_code: 'WRITE_REQUEST_BLOCKED' } })]), coded.output, environment), 1); + assert.deepEqual(JSON.parse(coded.text()).error, { code: 'API_REJECTED', message: rejection.message, status: 409, api_code: 'WRITE_REQUEST_BLOCKED' }); + for (const fixtureCode of [null, 409, {}, 'X', 'A'.repeat(65), 'browser_session_required', 'WRITE REQUEST BLOCKED', 'WRITE_REQUEST_BLOCKED\n', 'ＷRITE_REQUEST_BLOCKED', secret, 'API_SECRET_LEAK', 'PASSWORD_123456', 'CSRF_TOKEN_VALUE']) { + const unsafe = outputBuffer(); + assert.equal(await main(Readable.from([JSON.stringify({ operation: 'browser.action', task_id: 'fail_code', data: { kind: 'click', selector: 'button', fixture_code: fixtureCode } })]), unsafe.output, environment), 1); + const error = JSON.parse(unsafe.text()).error; + assert.deepEqual(error, { code: 'API_REJECTED', message: rejection.message, status: 409 }); + assert.ok(!unsafe.text().includes(secret) && !unsafe.text().includes(csrf)); + } + assert.equal(actions, 1, 'Diagnostic failures must not retry writes'); + await assert.rejects(readControlInput(Readable.from(['x'.repeat(64 * 1024 + 1)])), error => error.code === 'INPUT_TOO_LARGE'); + await assert.rejects(readControlInput(Readable.from(['[]'])), error => error.code === 'INVALID_REQUEST'); + const batch = outputBuffer(); + assert.equal(await main(Readable.from([JSON.stringify([ + { operation: 'tasks.list' }, + { operation: 'browser.action', task_id: 'fail', data: { kind: 'click', selector: 'button' } }, + { operation: 'browser.action', task_id: 'must_not_execute', data: { kind: 'click', selector: 'button' } }, + ])]), batch.output, environment), 1); + const results = JSON.parse(batch.text()); + assert.equal(results.length, 2); + assert.equal(results[0].ok, true); + assert.equal(results[1].error.status, 409); + assert.ok(!batch.text().includes(secret) && !batch.text().includes(csrf) && !batch.text().includes('authenticated;')); + assert.ok(!calls.some(call => call.path.includes('must_not_execute'))); + redirect = true; + const redirected = new LearningControlClient({ baseUrl, username: 'fixture-maintainer', password: secret }); + await assert.rejects(redirected.execute({ operation: 'tasks.list' }), error => error.code === 'CONNECTION_FAILED'); + assert.ok(!calls.some(call => call.path === '/unexpected-redirect')); + process.stdout.write(JSON.stringify({ result: 'passed', local_only: true, csrf_rotation: true, operation_allowlist: true, + no_redirect: true, no_write_retry: true, no_credentials_in_output: true, bounded_input: true, safe_api_code: true }) + '\n'); +} finally { + await new Promise(resolve => server.close(resolve)); +} diff --git a/tools/learning/test-native-ui-profile.mjs b/tools/learning/test-native-ui-profile.mjs new file mode 100644 index 0000000..f943b21 --- /dev/null +++ b/tools/learning/test-native-ui-profile.mjs @@ -0,0 +1,515 @@ +// Synthetic contract tests only. This test must never navigate to an official +// platform or expose an origin override in the production execution profile. +import assert from 'node:assert/strict'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import {fileURLToPath} from 'node:url'; +import {createServer} from 'node:http'; +import {ActionInputError} from './action-contract.mjs'; + +const root=path.resolve(path.dirname(fileURLToPath(import.meta.url)),'../..'); +await fs.mkdir(path.join(root,'.tmp'),{recursive:true}); +const output=await fs.mkdtemp(path.join(root,'.tmp','native-ui-profile-')); +process.env.ASSISTANT_LEARNING_HOME=path.join(output,'worker-learning'); +process.env.ASSISTANT_BROWSER_HEADLESS='1'; +process.env.ASSISTANT_BROWSER_CHANNEL=process.platform==='win32' ? 'msedge' : 'chromium'; +const results=[]; +let assertions=0; +let browserFixture=null; +const eq=(actual,expected,message)=>{assert.deepEqual(actual,expected,message);assertions+=1;}; +const ok=(value,message)=>{assert.ok(value,message);assertions+=1;}; +const throws=(fn,predicate,message)=>{assert.throws(fn,predicate,message);assertions+=1;}; +const report=async(extra={})=>fs.writeFile(path.join(output,'report.json'),JSON.stringify({ + result:'passed',passed:results.length,assertions,results, + environment:'synthetic execution-profile, action, and private request-capability helpers', + official_browser_navigation:false,live_platform_requests:0,application_api_calls:0,model_calls:0, + browser_fixture:browserFixture || 'not executed; no production origin or network bypass is introduced',...extra, +},null,2)); +const group=async(name,run)=>{ + const start=assertions; + try{const details=await run();results.push({name,assertions:assertions-start,...details});await report();} + catch(error){await report({result:'failed',failed_group:name,error:{name:error.name,message:error.message,code:error.code}});throw error;} +}; + +const official='https://www.xiaohongshu.com'; +const noteId='a'.repeat(24),otherNoteId='b'.repeat(24); +const noteUrl=`${official}/explore/${noteId}?xsec_token=LOCAL_NATIVE_FIXTURE_TOKEN`; +const profile=await import('./execution-profile.mjs'); +const {NATIVE_UI_PROFILE,NATIVE_UI_CONTRACT,resolveExecutionProfile,validNativePage,nativeNoteId, + nativeSearchKeyword,nativeReadRootsValid,assertNativeUiAction,assertNativeUiResultLocation, + executionContractMatches,executionMetadata}=profile; + +await group('exact versioned profile and immutable execution contract',()=>{ + eq(NATIVE_UI_PROFILE,'playwright-xhs-native-ui-v1'); + eq(NATIVE_UI_CONTRACT,{engine:'playwright',action_policy:'xhs-bounded-readonly-v1',network_policy:'xhs-page-native-v1'}); + eq(Object.isFrozen(NATIVE_UI_CONTRACT),true); + eq(resolveExecutionProfile(undefined),null,'only an omitted profile selects legacy'); + eq(resolveExecutionProfile(NATIVE_UI_PROFILE),NATIVE_UI_CONTRACT); + eq(executionMetadata(undefined),{},'legacy omits the new contract fields'); + eq(executionContractMatches({...NATIVE_UI_CONTRACT}),true); + for(const value of [null,undefined,[],{},NATIVE_UI_PROFILE, + {...NATIVE_UI_CONTRACT,engine:'browser-use'}, + {...NATIVE_UI_CONTRACT,action_policy:'legacy'}, + {...NATIVE_UI_CONTRACT,network_policy:'unknown'}, + {...NATIVE_UI_CONTRACT,extra:'fixture'}, + {engine:'playwright',action_policy:'xhs-bounded-readonly-v1'}, + Object.create(NATIVE_UI_CONTRACT)]) eq(executionContractMatches(value),false); + const metadata=executionMetadata(NATIVE_UI_PROFILE); + eq(metadata,{execution_profile:NATIVE_UI_PROFILE,execution_contract:NATIVE_UI_CONTRACT}); + metadata.execution_contract.engine='LOCAL_FIXTURE_MUTATION'; + eq(executionMetadata(NATIVE_UI_PROFILE).execution_contract.engine,'playwright','returned DTOs cannot mutate the shared descriptor'); + for(const value of [null,false,true,0,1,'','legacy','playwright-xhs-native-ui-v2',NATIVE_UI_PROFILE+' ', + ' '+NATIVE_UI_PROFILE,NATIVE_UI_PROFILE.toUpperCase(),[],{},NATIVE_UI_CONTRACT]){ + throws(()=>resolveExecutionProfile(value),error=>error instanceof ActionInputError,'unknown or object-valued profiles never silently downgrade'); + } +}); + +await group('official page shape uses HTTPS exact host and declared paths',()=>{ + for(const url of [official+'/explore',official+'/search_result',official+'/search_result_ai', + official+'/search_result?keyword=%E8%A3%85%E4%BF%AE',official+'/search_result_ai?keyword=fixture', + noteUrl,`${official}/search_result/${otherNoteId}`]) eq(validNativePage(url),true,url); + for(const url of [null,undefined,{},'', 'not a url','http://www.xiaohongshu.com/explore', + 'https://xiaohongshu.com/explore','https://www.xiaohongshu.com.evil.invalid/explore', + 'https://evilxiaohongshu.com/explore','https://www.xiaohongshu.com:8443/explore', + 'https://user:pass@www.xiaohongshu.com/explore','https://www.xiaohongshu.com@evil.invalid/explore', + 'https://so.xiaohongshu.com/explore','http://127.0.0.1/explore',official+'/',official+'/explore/', + official+'/message',official+'/notification',official+'/user/profile/'+noteId, + official+'/api/sns/web/v2/comment/post',official+'/search_result_ai/'+noteId, + official+'/explore/'+noteId+'/',official+'/explore/'+noteId.slice(1), + official+'/explore/'+'g'.repeat(24),official+'/explore/%2f'+noteId]) eq(validNativePage(url),false,String(url)); +}); + +await group('public note identity and bounded decoding preserve the selected search',()=>{ + eq(nativeNoteId(noteUrl),noteId); + eq(nativeNoteId(`${official}/search_result/${otherNoteId.toUpperCase()}?from=fixture`),otherNoteId); + for(const value of [official+'/search_result_ai/'+noteId,official+'/explore/'+noteId+'/', + 'https://evil.invalid/explore/'+noteId,'https://u:p@www.xiaohongshu.com/explore/'+noteId, + official+'/user/profile/'+noteId,null,undefined]) eq(nativeNoteId(value),null); + const keyword='装修 案例'; + eq(nativeSearchKeyword(official+'/search_result?keyword='+encodeURIComponent(keyword)),keyword); + eq(nativeSearchKeyword(official+'/search_result_ai?keyword='+encodeURIComponent(encodeURIComponent(keyword))),keyword); + eq(nativeSearchKeyword(official+'/search_result?keyword=fixture%20value'),'fixture value'); + for(const value of [official+'/explore?keyword=fixture',official+'/search_result', + official+'/search_result?keyword=fixture&keyword=other', + official+'/search_result?keyword=%25E0%25A4%25A','https://evil.invalid/search_result?keyword=fixture']) eq(nativeSearchKeyword(value),null); + const threeTimes=encodeURIComponent(encodeURIComponent(encodeURIComponent(keyword))); + ok(nativeSearchKeyword(official+'/search_result?keyword='+threeTimes)!==keyword,'the helper never repeatedly decodes until a desired keyword happens to match'); +}); + +const searchObserved={kind:'fill',tag:'input',unique:true,search_input:true,sensitive:false,stable_label:'Search'}; +const searchActual={tag:'input',search_input:true,sensitive:false,visible:true,in_viewport:true,unobscured:true, + native_context:{search_input:true,note_link:false,read_root:false}}; +const searchAction={kind:'fill',pageUrl:official+'/explore',selector:'#search-input-in-feeds', + observed:searchObserved,actualRoots:[searchActual],matchCount:1,keyword:'装修案例'}; +const rejectsAction=(context,code='WRITE_ACTION_BLOCKED')=>{ + const snapshot=JSON.stringify(context); + throws(()=>assertNativeUiAction(context),error=>error instanceof ActionInputError && error.code===code); + eq(JSON.stringify(context),snapshot,'rejected action metadata stays unchanged'); +}; + +await group('only observed usable exact search controls accept fill and Enter',()=>{ + const snapshot=JSON.stringify(searchAction); + for(const selector of ['#search-input-in-feeds','#search-input-ai']){ + eq(assertNativeUiAction({...searchAction,selector}),true); + eq(assertNativeUiAction({...searchAction,selector,kind:'press',key:'Enter'}),true); + } + eq(JSON.stringify(searchAction),snapshot); + for(const patch of [ + {selector:'#generic-search'},{selector:'#comment-input'},{selector:'input[type=search]'}, + {matchCount:0},{matchCount:2},{actualRoots:[]},{actualRoots:[searchActual,searchActual]}, + {observed:null},{observed:{...searchObserved,kind:'read'}},{observed:{...searchObserved,unique:false}}, + {observed:{...searchObserved,search_input:false}},{observed:{...searchObserved,sensitive:true}}, + {actualRoots:[{...searchActual,tag:'button'}]},{actualRoots:[{...searchActual,search_input:false}]}, + {actualRoots:[{...searchActual,native_context:{...searchActual.native_context,search_input:false}}]}, + {actualRoots:[{...searchActual,native_context:undefined}]},{actualRoots:[{...searchActual,sensitive:true}]}, + {actualRoots:[{...searchActual,visible:false}]},{actualRoots:[{...searchActual,in_viewport:false}]}, + {actualRoots:[{...searchActual,unobscured:false}]},{kind:'press',key:'Tab'}, + {kind:'press',key:'Control+Enter'},{kind:'press'}, + ]) rejectsAction({...searchAction,...patch}); + for(const keyword of [undefined,null,'',' ','x'.repeat(2001),'手机号 13800138000','验证码:123456','password:fixture']){ + rejectsAction({...searchAction,keyword},'SENSITIVE_INPUT'); + } + rejectsAction({...searchAction,pageUrl:official+'/message'},'SCOPE_BLOCKED'); +}); + +const noteObserved={kind:'click',tag:'a',unique:true,sensitive:false}; +const noteActual={tag:'a',href:noteUrl,sensitive:false,visible:true,in_viewport:true,unobscured:true, + native_context:{search_input:false,note_link:true,read_root:false}}; +const noteAction={kind:'click',pageUrl:official+'/search_result?keyword=fixture',selector:'#global a.title', + observed:noteObserved,actualRoots:[noteActual],matchCount:1,href:noteUrl,noteId}; +await group('clicks require a fixed actual standard note link and selected identity',()=>{ + const snapshot=JSON.stringify(noteAction); + eq(assertNativeUiAction(noteAction),true); + eq(assertNativeUiAction({...noteAction,observed:{...noteObserved,unique:false}}),true,'the worker may supply a uniquely resolved collection member'); + eq(JSON.stringify(noteAction),snapshot); + for(const patch of [ + {kind:'click',observed:null},{matchCount:0},{matchCount:2},{actualRoots:[]}, + {observed:{...noteObserved,kind:'read'}},{observed:{...noteObserved,tag:'button'}}, + {observed:{...noteObserved,sensitive:true}}, + {actualRoots:[{...noteActual,tag:'button',href:''}]}, + {actualRoots:[{...noteActual,native_context:{...noteActual.native_context,note_link:false}}]}, + {actualRoots:[{...noteActual,native_context:undefined}]}, + {actualRoots:[{...noteActual,sensitive:true}]},{actualRoots:[{...noteActual,visible:false}]}, + {actualRoots:[{...noteActual,in_viewport:false}]},{actualRoots:[{...noteActual,unobscured:false}]}, + {actualRoots:[{...noteActual,href:`${official}/explore/${otherNoteId}`}]}, + {noteId:otherNoteId}, + ]) rejectsAction({...noteAction,...patch}); + for(const href of [official+'/message',official+'/explore',official+'/user/profile/'+noteId, + official+'/explore/not-a-note',official+'/explore/'+noteId+'/', + official+'/api/sns/web/v2/comment/post','https://evil.invalid/explore/'+noteId, + 'javascript:void(0)','data:text/html,fixture','']){ + rejectsAction({...noteAction,href,actualRoots:[{...noteActual,href}]}); + } +}); + +const readRoot={tag:'section',readonly_root:true,sensitive:false,forbidden:false, + native_context:{read_root:true,note_link:false,search_input:false}}; +await group('read wait extract and bounded scroll require fixed ordinary roots',()=>{ + for(const kind of ['read','wait','extract','scroll']) eq(assertNativeUiAction({kind,pageUrl:noteUrl,actualRoots:[readRoot]}),true); + eq(nativeReadRootsValid([readRoot]),true); + eq(nativeReadRootsValid(Array.from({length:500},()=>readRoot)),true); + for(const actualRoots of [null,undefined,[],[null],[readRoot,null],Array.from({length:501},()=>readRoot), + [{...readRoot,readonly_root:false}],[{...readRoot,sensitive:true}],[{...readRoot,forbidden:true}], + [{...readRoot,native_context:undefined}],[{...readRoot,native_context:{...readRoot.native_context,read_root:false}}]]){ + eq(nativeReadRootsValid(actualRoots),false); + rejectsAction({kind:'read',pageUrl:noteUrl,actualRoots},'SENSITIVE_OUTPUT'); + } + for(const kind of ['evaluate','fetch','request','goto','send_message','follow','comment','arbitrary_js']){ + rejectsAction({kind,pageUrl:noteUrl,actualRoots:[readRoot]},'SENSITIVE_OUTPUT'); + } +}); + +await group('completed actions require the actual expected final source rather than only permitted in-flight traffic',()=>{ + const failLocation=(context,pageUrl,code)=>throws(()=>assertNativeUiResultLocation(context,pageUrl), + error=>error instanceof ActionInputError && error.code===code); + const press={...searchAction,kind:'press',key:'Enter'}; + for(const path of ['/search_result','/search_result_ai']){ + eq(assertNativeUiResultLocation(press,official+path+'?keyword='+encodeURIComponent(press.keyword)),true); + } + failLocation(press,press.pageUrl,'SEARCH_TRANSITION_NOT_READY'); + failLocation(press,official+'/search_result?keyword=other','SEARCH_TRANSITION_NOT_READY'); + failLocation(press,official+'/search_result?keyword='+encodeURIComponent(press.keyword)+'&keyword=other','SEARCH_TRANSITION_NOT_READY'); + failLocation(press,official+'/message','SOURCE_CHANGED'); + eq(assertNativeUiResultLocation(noteAction,noteUrl),true); + eq(assertNativeUiResultLocation(noteAction,official+'/search_result/'+noteId),true); + failLocation(noteAction,noteAction.pageUrl,'NOTE_BINDING_MISMATCH'); + failLocation(noteAction,official+'/explore/'+otherNoteId,'NOTE_BINDING_MISMATCH'); + failLocation(noteAction,'https://evil.invalid/explore/'+noteId,'SOURCE_CHANGED'); + for(const kind of ['fill','read','extract','wait','scroll']){ + const context={kind,pageUrl:searchAction.pageUrl}; + eq(assertNativeUiResultLocation(context,context.pageUrl),true); + failLocation(context,context.pageUrl+'?fixture=changed','SOURCE_CHANGED'); + failLocation(context,context.pageUrl+'#changed','SOURCE_CHANGED'); + failLocation(context,noteUrl,'SOURCE_CHANGED'); + } +}); + +const {beginNativeUiRequestPolicy,evaluatePlatformRequest,matchPlatformRequest,endPlatformRequestPolicy, + NATIVE_UI_POLICY_VERSION}=await import('./platform-policy.mjs'); +const policyContext={...searchAction,sessionOrigin:official,now:10000}; +const nativeRequest={automating:true,sessionOrigin:official,pageUrl:searchAction.pageUrl,activeMainFrame:true, + method:'POST',resourceType:'xhr',url:'https://as.xiaohongshu.com/api/sec/v1/fixture',now:10001}; +const allowed={allowed:true,policy_version:'xhs-page-native-v1',policy_rule:'native_page_traffic'}; +const nativePolicies=new Set(); +const mint=context=>{ + const snapshot=JSON.stringify(context),policy=beginNativeUiRequestPolicy(context); + eq(JSON.stringify(context),snapshot,'minting never changes action evidence'); + ok(policy,'a permitted finite UI action can mint its private request policy'); + nativePolicies.add(policy);return policy; +}; +const denied=(request,reason)=>{ + const actual=evaluatePlatformRequest(request); + eq(actual.allowed,false); + if(reason!==undefined) eq(actual.reason,reason); + eq(matchPlatformRequest(request),null,'the public matcher uses the same rejection'); +}; + +await group('only valid exact native actions mint private fifteen-second capabilities',()=>{ + eq(NATIVE_UI_POLICY_VERSION,'xhs-page-native-v1'); + const policy=mint(policyContext); + eq(policy,{version:NATIVE_UI_POLICY_VERSION,scope:'native_ui',expiresAt:25000}); + eq(Object.isFrozen(policy),true); + for(const patch of [ + {sessionOrigin:'https://xiaohongshu.com'},{sessionOrigin:'http://127.0.0.1'}, + {pageUrl:'https://evil.invalid/explore'},{pageUrl:official+'/message'}, + {now:NaN},{now:Infinity},{now:'10000'},{kind:'evaluate'}, + {selector:'#comment-input'},{actualRoots:[]},{observed:null}, + {actualRoots:[{...searchActual,native_context:{search_input:false}}]}, + ]) eq(beginNativeUiRequestPolicy({...policyContext,...patch}),null); + eq(evaluatePlatformRequest({...nativeRequest,policy}),allowed); + denied({...nativeRequest,policy:{...policy}},'inactive_policy'); + denied({...nativeRequest,policy:null},'missing_policy'); + endPlatformRequestPolicy(policy); + denied({...nativeRequest,policy},'inactive_policy'); +}); + +await group('native page POST permits only exact first-party HTTPS main-frame xhr or fetch',()=>{ + const policy=mint(policyContext); + const start=assertions; + for(const host of ['www.xiaohongshu.com','xiaohongshu.com','as.xiaohongshu.com', + 'apm-fe.xiaohongshu.com','so.xiaohongshu.com','nested.fixture.xiaohongshu.com']){ + for(const resourceType of ['xhr','fetch']){ + const request={...nativeRequest,policy,resourceType,url:`https://${host}/unknown-native-page-traffic?fixture=1`}; + eq(evaluatePlatformRequest(request),allowed); + eq(matchPlatformRequest(request),{policy_version:NATIVE_UI_POLICY_VERSION,policy_rule:'native_page_traffic'}); + } + } + for(const url of ['http://as.xiaohongshu.com/native','https://u:p@as.xiaohongshu.com/native', + 'https://as.xiaohongshu.com:8443/native','https://as.xiaohongshu.com.evil.invalid/native', + 'https://evilxiaohongshu.com/native','https://evil.invalid/xiaohongshu.com/native', + 'https://xiaohongshu.com@evil.invalid/native','https://xiaohongshu.com./native', + 'http://127.0.0.1/native','https://127.0.0.1/native','file:///fixture','data:text/plain,fixture','invalid']){ + denied({...nativeRequest,policy,url}); + } + for(const resourceType of ['document','script','stylesheet','image','font','media','websocket','eventsource','other','',null]){ + const request={...nativeRequest,policy,resourceType}; + denied(request,'resource_type_not_allowed'); + eq(evaluatePlatformRequest(request).expected_resource_types,['xhr','fetch']); + } + for(const method of ['GET','HEAD','OPTIONS','PUT','PATCH','DELETE','CONNECT','TRACE','post','',null]){ + denied({...nativeRequest,policy,method},'method_not_allowed'); + } + denied({...nativeRequest,policy,activeMainFrame:false},'not_main_frame'); + denied({...nativeRequest,policy,automating:false},'automation_inactive'); + denied({...nativeRequest,policy,sessionOrigin:'https://evil.invalid'},'session_out_of_scope'); + denied({...nativeRequest,policy,pageUrl:'https://evil.invalid/search_result'},'page_out_of_scope'); + return {request_boundary_assertions:assertions-start, + scope:'POST policy only; ordinary GET/HEAD/OPTIONS routing is not represented as an issued native permission', + http_side_effect_free_claim:false}; +}); + +await group('native capabilities reject early exact-expiry late invalid and revoked use',()=>{ + const policy=mint(policyContext); + eq(evaluatePlatformRequest({...nativeRequest,policy,now:10000}),allowed); + eq(evaluatePlatformRequest({...nativeRequest,policy,now:24999}),allowed); + denied({...nativeRequest,policy,now:9999},'policy_not_started'); + denied({...nativeRequest,policy,now:25000},'policy_expired'); + denied({...nativeRequest,policy,now:25001},'policy_expired'); + for(const now of [NaN,Infinity,-Infinity,'10001',null]) denied({...nativeRequest,policy,now},'invalid_time'); + endPlatformRequestPolicy(policy); + denied({...nativeRequest,policy,now:10001},'inactive_policy'); +}); + +await group('fill and ordinary reads cannot borrow page-transition authority',()=>{ + const fillPolicy=mint(policyContext); + for(const pageUrl of [official+'/search_result?keyword='+encodeURIComponent(searchAction.keyword), + official+'/explore?other=1',official+'/explore#changed',noteUrl]){ + denied({...nativeRequest,policy:fillPolicy,pageUrl},'page_changed'); + } + for(const kind of ['read','wait','extract','scroll']){ + const pageUrl=official+'/search_result?keyword=fixture'; + const policy=mint({kind,pageUrl,actualRoots:[readRoot],sessionOrigin:official,now:10000}); + eq(evaluatePlatformRequest({...nativeRequest,policy,pageUrl}),allowed); + for(const changed of [pageUrl+'&page=2',pageUrl+'#changed',official+'/search_result_ai?keyword=fixture',noteUrl]){ + denied({...nativeRequest,policy,pageUrl:changed},'page_changed'); + } + } +}); + +await group('Enter allows only the selected keyword search-result transition',()=>{ + const keyword='装修 案例'; + const policy=mint({...policyContext,kind:'press',key:'Enter',keyword}); + eq(evaluatePlatformRequest({...nativeRequest,policy}),allowed); + for(const pathname of ['/search_result','/search_result_ai']){ + for(const encoded of [encodeURIComponent(keyword),encodeURIComponent(encodeURIComponent(keyword))]){ + eq(evaluatePlatformRequest({...nativeRequest,policy,pageUrl:official+pathname+'?keyword='+encoded}),allowed); + } + } + for(const pageUrl of [official+'/search_result?keyword=other',official+'/search_result', + official+'/search_result?keyword='+encodeURIComponent(keyword)+'&keyword=other', + official+'/explore?keyword='+encodeURIComponent(keyword),noteUrl,official+'/message', + 'https://www.xiaohongshu.com.evil.invalid/search_result?keyword='+encodeURIComponent(keyword)]){ + denied({...nativeRequest,policy,pageUrl}); + } + denied({...nativeRequest,policy,pageUrl:official+'/search_result?keyword='+encodeURIComponent(keyword),now:25000},'policy_expired'); +}); + +await group('note click permits only its bound public note identity transition',()=>{ + const policy=mint({...noteAction,sessionOrigin:official,now:10000}); + eq(evaluatePlatformRequest({...nativeRequest,policy,pageUrl:noteAction.pageUrl}),allowed); + for(const pageUrl of [noteUrl,official+'/search_result/'+noteId,official+'/explore/'+noteId+'?from=fixture']){ + eq(evaluatePlatformRequest({...nativeRequest,policy,pageUrl}),allowed); + } + for(const pageUrl of [official+'/explore/'+otherNoteId,official+'/search_result/'+otherNoteId, + noteAction.pageUrl+'&changed=1',official+'/search_result?keyword=other',official+'/explore', + official+'/user/profile/'+noteId,official+'/search_result_ai/'+noteId]){ + denied({...nativeRequest,policy,pageUrl},'page_changed'); + } +}); + +await group('issued policy stores immutable source facts and never accepts fabricated contracts',()=>{ + const context=structuredClone(policyContext),policy=mint(context); + context.pageUrl=official+'/search_result?keyword=other'; + context.keyword='other';context.actualRoots[0].native_context.search_input=false; + eq(evaluatePlatformRequest({...nativeRequest,policy}),allowed); + denied({...nativeRequest,policy,pageUrl:context.pageUrl},'page_changed'); + for(const fake of [{...policy},{version:NATIVE_UI_POLICY_VERSION,scope:'native_ui',expiresAt:25000}, + {version:NATIVE_UI_POLICY_VERSION,scope:'search',expiresAt:25000}, + {execution_profile:NATIVE_UI_PROFILE,execution_contract:NATIVE_UI_CONTRACT}]){ + denied({...nativeRequest,policy:fake},'inactive_policy'); + } +}); + +await group('real loopback Edge DOM produces restricted search link and ordinary-root facts',async()=>{ + const {inspectElements}=await import('./browser-worker.mjs'); + const {chromium}=await import('playwright'); + let browser,fixtureRequests=0,unexpectedNetwork=0; + const observations=[]; + const server=createServer((request,response)=>{ + fixtureRequests+=1; + if(request.method!=='GET' || request.url!=='/') {response.writeHead(404).end();return;} + response.writeHead(200,{'content-type':'text/html; charset=utf-8'}).end('Local native UI DOM fixture

    Loopback only

    '); + }); + await new Promise(resolve=>server.listen(0,'127.0.0.1',resolve)); + const loopback=`http://127.0.0.1:${server.address().port}`; + try{ + browser=await chromium.launch({channel:process.platform==='win32' ? 'msedge' : 'chromium',headless:true}); + const context=await browser.newContext({viewport:{width:1280,height:840},serviceWorkers:'block'}); + await context.route('**/*',async route=>{ + const url=new URL(route.request().url()); + if(url.origin!==loopback){unexpectedNetwork+=1;await route.abort();return;} + await route.continue(); + }); + const page=await context.newPage(); + await page.goto(loopback+'/',{waitUntil:'domcontentloaded'}); + const render=body=>page.setContent(`Local native UI contract fixture

    Local native UI DOM fixture

    Only metadata is inspected; no search or link is activated.

    ${body}`); + const inspect=async(selector,nativeProfile=true)=>{ + const facts=await page.locator(selector).evaluate(inspectElements,{origin:official,nativeProfile}); + eq(facts.length,1);return facts[0]; + }; + const remember=(name,state,accepted)=>observations.push({name,accepted, + tag:state.tag,sensitive:state.sensitive,search_input:state.search_input, + readonly_root:state.readonly_root,native_context:state.native_context}); + const searchFixtures=[ + ['known-feed-search','','#search-input-in-feeds',true], + ['known-ai-search','','#search-input-ai',true], + ['generic-search-name','','#generic-search',false], + ['comment-input','','#comment-input',false], + ['post-form-search','
    ','#search-input-in-feeds',false], + ['editable-search','
    ','#search-input-in-feeds',false], + ['button-role-search','','#search-input-in-feeds',false], + ['checkbox-role-search','','#search-input-in-feeds',false], + ['button-ancestor-search','
    ','#search-input-in-feeds',false], + ['searchbox-role-search','','#search-input-in-feeds',true], + ]; + for(const [name,html,selector,accepted] of searchFixtures){ + await render(html);const state=await inspect(selector); + eq(state.native_context.search_input,accepted,name); + const action={...searchAction,selector,actualRoots:[state]}; + if(accepted){eq(assertNativeUiAction(action),true);eq(assertNativeUiAction({...action,kind:'press',key:'Enter'}),true);} + else rejectsAction(action); + remember(name,state,accepted); + } + const linkFixtures=[ + ['standard-note',`Synthetic note`,true], + ['self-target-note',`Synthetic note`,true], + ['download-note',`Synthetic note`,false], + ['blank-target-note',`Synthetic note`,false], + ['named-target-note',`Synthetic note`,false], + ['editable-note',`Synthetic note`,false], + ['plaintext-editable-note',``,false], + ['form-note',`
    Synthetic note
    `,false], + ['button-role-note',`Synthetic note`,false], + ['checkbox-role-note',`Synthetic note`,false], + ['switch-role-note',`Synthetic note`,false], + ['menuitem-role-note',`Synthetic note`,false], + ['button-ancestor-note',``,false], + ['checkbox-ancestor-note',``,false], + ['link-role-note',`Synthetic note`,true], + ]; + for(const [name,html,accepted] of linkFixtures){ + await render(html);const state=await inspect('#fixture'); + eq(state.native_context.note_link,accepted,name); + const action={...noteAction,selector:'#fixture',actualRoots:[state]}; + if(accepted) eq(assertNativeUiAction(action),true);else rejectsAction(action); + remember(name,state,accepted); + } + const rootFixtures=[ + ['ordinary-section','
    Plain result text
    ',true], + ['editable-section','
    Plain result text
    ',false], + ['plaintext-editable-section','
    Plain result text
    ',false], + ['inherited-editable-section','
    Plain result text
    ',false], + ['button-root','',false], + ['button-role-root','
    Plain result text
    ',false], + ['checkbox-role-root','',false], + ['button-ancestor-root','
    Plain result text
    ',false], + ['checkbox-ancestor-root','
    Plain result text
    ',false], + ['form-root','
    Plain result text
    ',false], + ['anchor-root',`Synthetic note`,false], + ]; + for(const [name,html,accepted] of rootFixtures){ + await render(html);const state=await inspect('#fixture'); + eq(state.native_context.read_root,accepted,name); + const action={kind:'read',pageUrl:noteUrl,selector:'#fixture',actualRoots:[state]}; + if(accepted) eq(assertNativeUiAction(action),true);else rejectsAction(action,'SENSITIVE_OUTPUT'); + remember(name,state,accepted); + } + await render('
    Plain result text
    '); + eq(Object.hasOwn(await inspect('#fixture',false),'native_context'),false,'legacy inspection omits the new metadata'); + const screenshot=path.join(output,'loopback-native-dom.png'); + await page.screenshot({path:screenshot,fullPage:true}); + const png=await fs.readFile(screenshot);eq(png.subarray(1,4).toString(),'PNG'); + eq(unexpectedNetwork,0,'synthetic links are never navigated or fetched'); + browserFixture={browser:process.platform==='win32' ? 'Edge' : 'Chromium',origin:'loopback only', + inspected_cases:observations.length,fixture_requests:fixtureRequests,unexpected_network_attempts:unexpectedNetwork, + actual_png:true,screenshot,worker_native_profile_end_to_end:false}; + await fs.writeFile(path.join(output,'loopback-dom-observations.json'),JSON.stringify(observations,null,2)); + return browserFixture; + }finally{ + if(browser) await browser.close(); + await new Promise(resolve=>server.close(resolve)); + } +}); + +await group('real legacy worker DTOs omit native fields and reject profile overrides before navigation',async()=>{ + const {dispatch}=await import('./browser-worker.mjs'); + const task='native-ui-profile-legacy-shape'; + let requests=0,opened=false; + const server=createServer((request,response)=>{ + requests+=1; + response.writeHead(200,{'content-type':'text/html; charset=utf-8'}).end('Local legacy shape fixture
    Loopback only
    '); + }); + await new Promise(resolve=>server.listen(0,'127.0.0.1',resolve)); + const entry=`http://127.0.0.1:${server.address().port}/`; + const omitted=value=>{ + eq(Object.hasOwn(value,'execution_profile'),false); + eq(Object.hasOwn(value,'execution_contract'),false); + }; + const rejectedAsync=async(run,code)=>{ + await assert.rejects(run,error=>error.code===code);assertions+=1; + }; + try{ + await rejectedAsync(()=>dispatch('open',{task_id:task,account_key:'isolated-native-profile-shape',entry_url:entry, + execution_profile:NATIVE_UI_PROFILE}),'EXECUTION_PROFILE_INVALID'); + eq(requests,0,'native mode never introduces a loopback origin override'); + const initial=await dispatch('open',{task_id:task,account_key:'isolated-native-profile-shape',entry_url:entry});opened=true; + omitted(initial); + omitted(await dispatch('ready',{task_id:task,identity_label:'Local legacy shape'})); + omitted(await dispatch('status',{task_id:task})); + const capture=await dispatch('capture',{task_id:task});omitted(capture); + eq(capture.elements.some(element=>element.selector==='#local-read'),true); + const action=await dispatch('action',{task_id:task,kind:'read',selector:'#local-read'}); + omitted(action);omitted(action.before);omitted(action.after); + const verifyParams={task_id:task,schema_version:2,entry_url:entry, + steps:[{kind:'read',selector:'#local-read',output_name:'note'}],parameters:{},expected_outputs:{note:'Loopback only'}}; + const previous=requests; + await rejectedAsync(()=>dispatch('verify',{...verifyParams,execution_profile:NATIVE_UI_PROFILE, + execution_contract:{...NATIVE_UI_CONTRACT}}),'EXECUTION_PROFILE_MISMATCH'); + await rejectedAsync(()=>dispatch('verify',{...verifyParams,execution_contract:{...NATIVE_UI_CONTRACT}}),'EXECUTION_PROFILE_MISMATCH'); + await rejectedAsync(()=>dispatch('verify',{...verifyParams,execution_profile:'unknown-mode'}),'EXECUTION_PROFILE_INVALID'); + eq(requests,previous,'profile failures cannot navigate or replay the legacy session'); + const verified=await dispatch('verify',verifyParams); + eq(verified.passed,true);omitted(verified); + ok(verified.evidence.length>0);for(const evidence of verified.evidence) omitted(evidence); + return {fixture_requests:requests,legacy_worker_native_fields_absent:true, + mismatched_or_unknown_profile_before_navigation:true,native_loopback_override:false}; + }finally{ + if(opened) await dispatch('close',{task_id:task}).catch(()=>{}); + await new Promise(resolve=>server.close(resolve)); + } +}); + +for(const policy of nativePolicies) endPlatformRequestPolicy(policy); +await report({status:'pure profile/action/request contract checks complete; worker wiring and platform execution are separate'}); +console.log(JSON.stringify({passed:results.length,assertions,output, + official_browser_navigation:false,live_platform_requests:0,application_api_calls:0,model_calls:0})); diff --git a/tools/learning/test-navigation-profile.mjs b/tools/learning/test-navigation-profile.mjs new file mode 100644 index 0000000..01e5375 --- /dev/null +++ b/tools/learning/test-navigation-profile.mjs @@ -0,0 +1,643 @@ +// Synthetic contracts and a loopback-only DOM fixture. Official-looking links +// are inspected but never clicked, fetched, or exposed as an origin override. +import assert from 'node:assert/strict'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import {fileURLToPath} from 'node:url'; +import {createServer} from 'node:http'; +import {ActionInputError} from './action-contract.mjs'; + +const root=path.resolve(path.dirname(fileURLToPath(import.meta.url)),'../..'); +await fs.mkdir(path.join(root,'.tmp'),{recursive:true}); +const output=await fs.mkdtemp(path.join(root,'.tmp','navigation-profile-')); +process.env.ASSISTANT_LEARNING_HOME=path.join(output,'worker-learning'); +process.env.ASSISTANT_BROWSER_HEADLESS='1'; +process.env.ASSISTANT_BROWSER_CHANNEL=process.platform==='win32' ? 'msedge' : 'chromium'; +let assertions=0; +const results=[]; +let browserFixture=null; +const eq=(actual,expected,message)=>{assert.deepEqual(actual,expected,message);assertions+=1;}; +const ok=(value,message)=>{assert.ok(value,message);assertions+=1;}; +const throws=(fn,code)=>{assert.throws(fn,error=>error instanceof ActionInputError && error.code===code);assertions+=1;}; +const report=async(extra={})=>fs.writeFile(path.join(output,'report.json'),JSON.stringify({ + result:'passed',passed:results.length,assertions,results,official_browser_navigation:false, + live_platform_requests:0,application_api_calls:0,model_calls:0, + browser_fixture:browserFixture || 'not executed',...extra, +},null,2)); +const group=async(name,run)=>{ + const start=assertions; + try{const details=await run();results.push({name,assertions:assertions-start,...details});await report();} + catch(error){await report({result:'failed',failed_group:name,error:{name:error.name,message:error.message,code:error.code}});throw error;} +}; + +const official='https://www.xiaohongshu.com'; +const noteId='a'.repeat(24),otherId='b'.repeat(24),profileId='c'.repeat(24); +const note=`${official}/explore/${noteId}?xsec_token=LOCAL_NAVIGATION_FIXTURE`; +const profileUrl=`${official}/user/profile/${profileId}?source=LOCAL_FIXTURE`; +const profile=await import('./execution-profile.mjs'); +const {NATIVE_UI_PROFILE,NATIVE_UI_CONTRACT,NAVIGATION_UI_PROFILE,NAVIGATION_UI_CONTRACT, + isNativeUiProfile,resolveExecutionProfile,executionContractMatches,executionMetadata, + validNativePage,validNavigationPage,navigationTarget,navigationPopupMatches,navigationComparison,assertNativeUiAction,assertNativeUiResultLocation}=profile; + +await group('navigation profile is explicit immutable and never silently changes the old profile',()=>{ + eq(NATIVE_UI_PROFILE,'playwright-xhs-native-ui-v1'); + eq(NATIVE_UI_CONTRACT,{engine:'playwright',action_policy:'xhs-bounded-readonly-v1',network_policy:'xhs-page-native-v1'}); + eq(NAVIGATION_UI_PROFILE,'playwright-xhs-navigation-ui-v1'); + eq(NAVIGATION_UI_CONTRACT,{engine:'playwright',action_policy:'xhs-observed-navigation-readonly-v1',network_policy:'xhs-page-navigation-v1'}); + eq(Object.isFrozen(NATIVE_UI_CONTRACT),true);eq(Object.isFrozen(NAVIGATION_UI_CONTRACT),true); + eq(resolveExecutionProfile(undefined),null); + eq(resolveExecutionProfile(NATIVE_UI_PROFILE),NATIVE_UI_CONTRACT); + eq(resolveExecutionProfile(NAVIGATION_UI_PROFILE),NAVIGATION_UI_CONTRACT); + eq(executionContractMatches({...NAVIGATION_UI_CONTRACT},NAVIGATION_UI_PROFILE),true); + eq(executionContractMatches({...NATIVE_UI_CONTRACT},NATIVE_UI_PROFILE),true); + eq(executionContractMatches({...NATIVE_UI_CONTRACT},NAVIGATION_UI_PROFILE),false); + eq(executionContractMatches({...NAVIGATION_UI_CONTRACT},NATIVE_UI_PROFILE),false); + for(const value of [null,undefined,[],{},Object.create(NAVIGATION_UI_CONTRACT), + {...NAVIGATION_UI_CONTRACT,extra:'fixture'}, {...NAVIGATION_UI_CONTRACT,engine:'browser-use'}, + {...NAVIGATION_UI_CONTRACT,network_policy:NATIVE_UI_CONTRACT.network_policy}]){ + eq(executionContractMatches(value,NAVIGATION_UI_PROFILE),false); + } + eq(isNativeUiProfile(NATIVE_UI_PROFILE),true);eq(isNativeUiProfile(NAVIGATION_UI_PROFILE),true); + for(const value of [null,undefined,false,0,'',{},'unknown']) eq(isNativeUiProfile(value),false); + eq(executionMetadata(undefined),{}); + const metadata=executionMetadata(NAVIGATION_UI_PROFILE); + eq(metadata,{execution_profile:NAVIGATION_UI_PROFILE,execution_contract:NAVIGATION_UI_CONTRACT}); + metadata.execution_contract.engine='LOCAL_MUTATION'; + eq(executionMetadata(NAVIGATION_UI_PROFILE).execution_contract.engine,'playwright'); + for(const value of [null,false,'',{},NAVIGATION_UI_PROFILE+' ',NAVIGATION_UI_PROFILE.toUpperCase(),'playwright-xhs-navigation-ui-v2']){ + throws(()=>resolveExecutionProfile(value),'EXECUTION_PROFILE_INVALID'); + } +}); + +await group('destination parser allows only observed official page shapes and keeps the exact href',()=>{ + for(const [href,kind,objectId] of [[note,'note',noteId],[profileUrl,'profile',profileId], + [`${official}/search_result/${otherId}`,'note',otherId],[official+'/notification','notification',null], + [official+'/explore','explore',null],[official+'/search_result?keyword=fixture','search',null], + [official+'/search_result_ai?keyword=fixture','search',null]]){ + eq(validNavigationPage(href),true,href); + eq(navigationTarget(href),{kind,object_id:objectId,href:new URL(href).href}); + } + for(const href of [null,undefined,{},'', 'not a url','/explore', + 'http://www.xiaohongshu.com/explore','https://xiaohongshu.com/explore', + 'https://www.xiaohongshu.com:8443/explore','https://u:p@www.xiaohongshu.com/explore', + 'https://www.xiaohongshu.com.evil.invalid/explore','https://www.xiaohongshu.com@evil.invalid/explore', + 'https://so.xiaohongshu.com/explore',official+'/',official+'/explore/', + official+'/message',official+'/messages',official+'/notification/',official+'/notification/send', + official+'/user/profile/'+profileId+'/',official+'/user/profile/'+profileId.slice(1), + official+'/user/profile/'+'g'.repeat(24),official+'/user/profile/%2f'+profileId, + official+'/user/profile/'+profileId+'/follow',official+'/api/sns/web/v2/comment/post', + official+'/api/sns/web/v1/follow',official+'/explore/'+noteId+'/delete', + 'javascript:void(0)','data:text/html,fixture','file:///fixture']){ + eq(validNavigationPage(href),false,String(href));eq(navigationTarget(href),null); + } + for(const href of [profileUrl,official+'/notification']) eq(validNativePage(href),false,'old v1 page scope remains unchanged'); +}); + +const observed={kind:'click',tag:'a',unique:true,sensitive:false}; +const actual=(href,kind)=>({tag:'a',href,sensitive:false,visible:true,in_viewport:true,unobscured:true, + native_context:{search_input:false,note_link:kind==='note',profile_link:kind==='profile',navigation_link:['notification','explore','search'].includes(kind),read_root:false}}); +const action=(href,kind,pageUrl=note)=>({executionProfile:NAVIGATION_UI_PROFILE,kind:'click',pageUrl, + selector:'#fixture',observed:{...observed},actualRoots:[actual(href,kind)],matchCount:1,href}); +const reject=(context,code='WRITE_ACTION_BLOCKED')=>{ + const before=JSON.stringify(context);throws(()=>assertNativeUiAction(context),code);eq(JSON.stringify(context),before); +}; + +await group('new navigation accepts fixed note profile notification and return links only',()=>{ + for(const [href,kind] of [[profileUrl,'profile'],[official+'/notification','notification'], + [official+'/explore','explore'],[official+'/search_result?keyword=fixture','search'],[note,'note']]){ + const context=action(href,kind); + eq(assertNativeUiAction(context),true); + eq(assertNativeUiAction({...context,pageUrl:profileUrl}),true); + eq(assertNativeUiAction({...context,pageUrl:official+'/notification'}),true); + eq(assertNativeUiResultLocation(context,href),true); + } + const collection=action(profileUrl,'profile'); + reject({...collection,observed:{...observed,unique:false}}); + eq(assertNativeUiAction({...collection,observed:{...observed,unique:false},targetResolvedUnique:true}),true); + reject({...collection,observed:{...observed,unique:false},targetResolvedUnique:true,matchCount:2}); +}); + +await group('click permission requires actual usable unique immutable anchor evidence',()=>{ + const context=action(profileUrl,'profile'),root=context.actualRoots[0]; + for(const patch of [ + {observed:null},{observed:{...observed,kind:'read'}},{observed:{...observed,tag:'button'}}, + {observed:{...observed,sensitive:true}},{matchCount:0},{matchCount:2},{actualRoots:[]}, + {actualRoots:[root,root]},{actualRoots:[{...root,sensitive:true}]}, + {actualRoots:[{...root,visible:false}]},{actualRoots:[{...root,in_viewport:false}]}, + {actualRoots:[{...root,unobscured:false}]},{actualRoots:[{...root,tag:'button'}]}, + {actualRoots:[{...root,native_context:undefined}]}, + {actualRoots:[{...root,native_context:{...root.native_context,profile_link:false}}]}, + {actualRoots:[{...root,href:`${official}/user/profile/${otherId}`}]}, + {href:`${official}/user/profile/${otherId}`}, + ]) reject({...context,...patch}); + for(const tag of ['button','input','textarea','select','form']){ + reject({...context,observed:{...observed,tag},actualRoots:[{...root,tag}]}); + } + for(const kind of ['follow','unfollow','send_message','comment','evaluate','fetch','request','goto']){ + reject({...context,kind},'SENSITIVE_OUTPUT'); + } +}); + +await group('fixed profile cannot be borrowed by a note or arbitrary page link',()=>{ + for(const [href,kind] of [[profileUrl,'note'],[note,'profile'],[profileUrl,'notification'], + [official+'/notification','profile'],[official+'/message','notification'], + [official+'/api/sns/web/v1/follow','profile'],['https://evil.invalid/user/profile/'+profileId,'profile'], + ['https://www.xiaohongshu.com:8443/user/profile/'+profileId,'profile'], + ['https://u:p@www.xiaohongshu.com/user/profile/'+profileId,'profile']]) reject(action(href,kind)); + for(const executionProfile of [undefined,NATIVE_UI_PROFILE]){ + const context=action(profileUrl,'profile',official+'/explore'); + reject({...context,executionProfile}); + reject({...action(official+'/notification','notification',official+'/explore'),executionProfile}); + eq(assertNativeUiAction({...action(note,'note',official+'/explore'),executionProfile}),true); + } + for(const executionProfile of [null,'unknown',false,{},NAVIGATION_UI_PROFILE+' ']){ + reject({...action(profileUrl,'profile'),executionProfile},'EXECUTION_PROFILE_INVALID'); + } +}); + +await group('navigation result binds complete destination and object rather than a merely allowed page',()=>{ + const context=action(profileUrl,'profile'); + for(const changed of [note,context.pageUrl,official+'/notification', + `${official}/user/profile/${otherId}?source=LOCAL_FIXTURE`, + `${official}/user/profile/${profileId}`,profileUrl+'&changed=1',profileUrl+'#changed']){ + throws(()=>assertNativeUiResultLocation(context,changed),'NAVIGATION_BINDING_MISMATCH'); + } + for(const href of [official+'/message','https://evil.invalid/user/profile/'+profileId]){ + throws(()=>assertNativeUiResultLocation(context,href),'SOURCE_CHANGED'); + } + for(const kind of ['read','extract','wait','scroll','fill']){ + const context={executionProfile:NAVIGATION_UI_PROFILE,kind,pageUrl:profileUrl}; + eq(assertNativeUiResultLocation(context,profileUrl),true); + throws(()=>assertNativeUiResultLocation(context,profileUrl+'&changed=1'),'SOURCE_CHANGED'); + } +}); + +await group('navigation diagnostics expose only the fixed comparison schema and no original identifiers or values',()=>{ + const keys=['expected_kind','actual_kind','same_origin','same_pathname','same_object_id','same_query','same_query_pairs','same_hash','xsec_token','xsec_source','other_query_equal'].sort(); + const boolKeys=['same_origin','same_pathname','same_object_id','same_query','same_query_pairs','same_hash','other_query_equal']; + const kinds=new Set(['note','profile','notification','explore','search','invalid']); + const validate=value=>{ + eq(Object.keys(value).sort(),keys); + ok(kinds.has(value.expected_kind));ok(kinds.has(value.actual_kind)); + for(const key of boolKeys) eq(typeof value[key],'boolean',key); + for(const key of ['xsec_token','xsec_source']){ + eq(Object.keys(value[key]).sort(),['actual_count','expected_count','values_equal']); + for(const name of ['actual_count','expected_count']) ok(Number.isInteger(value[key][name]) && value[key][name]>=0 && value[key][name]<=1000); + eq(typeof value[key].values_equal,'boolean'); + } + }; + const token='LOCAL_SENSITIVE_TOKEN_8f31',source='LOCAL_TRACKING_SOURCE_c82f',unknown='LOCAL_UNKNOWN_QUERY_SECRET_4a17'; + const left=`${official}/user/profile/${profileId}?xsec_token=${token}&xsec_source=${source}&private_fixture_key=${unknown}#LOCAL_HASH_MARKER`; + const right=`${official}/user/profile/${otherId}?xsec_token=DIFFERENT_LOCAL_TOKEN&xsec_source=${source}&private_fixture_key=${unknown}#OTHER_LOCAL_HASH`; + const comparison=navigationComparison(left,right);validate(comparison); + eq(comparison.expected_kind,'profile');eq(comparison.actual_kind,'profile'); + eq(comparison.same_origin,true);eq(comparison.same_pathname,false);eq(comparison.same_object_id,false); + eq(comparison.same_query,false);eq(comparison.same_query_pairs,false);eq(comparison.same_hash,false); + eq(comparison.xsec_token,{expected_count:1,actual_count:1,values_equal:false}); + eq(comparison.xsec_source,{expected_count:1,actual_count:1,values_equal:true});eq(comparison.other_query_equal,true); + const serialized=JSON.stringify(comparison); + for(const marker of [official,profileId,otherId,token,source,unknown,'private_fixture_key','LOCAL_HASH_MARKER','OTHER_LOCAL_HASH','DIFFERENT_LOCAL_TOKEN']){ + eq(serialized.includes(marker),false,'fixed diagnostics cannot disclose '+marker); + } + const invalid=navigationComparison(null,'not a url');validate(invalid); + eq(invalid.expected_kind,'invalid');eq(invalid.actual_kind,'invalid'); + for(const key of boolKeys) eq(invalid[key],false); + const many=Array.from({length:1001},()=>['xsec_token','LOCAL_COUNT_ONLY']); + const repeated=profileUrl+'&'+new URLSearchParams(many).toString(); + const bounded=navigationComparison(repeated,repeated);validate(bounded); + eq(bounded.xsec_token,{expected_count:1000,actual_count:1000,values_equal:true}); +}); + +await group('encoding tracking order and unknown query differences stay diagnostic and never authorize new navigation',()=>{ + const base=`${official}/user/profile/${profileId}`; + const encoded=navigationComparison(base+'?q=local+fixture&xsec_token=%2F',base+'?q=local%20fixture&xsec_token=/'); + eq(encoded.same_query,false);eq(encoded.same_query_pairs,true);eq(encoded.xsec_token.values_equal,true);eq(encoded.other_query_equal,true); + const tracking=navigationComparison(base+'?q=fixture&xsec_token=LEFT&xsec_source=one',base+'?q=fixture&xsec_token=RIGHT&xsec_source=two'); + eq(tracking.same_object_id,true);eq(tracking.same_query,false);eq(tracking.other_query_equal,true); + eq(tracking.xsec_token.values_equal,false);eq(tracking.xsec_source.values_equal,false); + const duplicate=navigationComparison(base+'?xsec_token=one&xsec_token=two',base+'?xsec_token=one'); + eq(duplicate.xsec_token,{expected_count:2,actual_count:1,values_equal:false}); + const order=navigationComparison(base+'?a=one&b=two',base+'?b=two&a=one'); + eq(order.same_query_pairs,false);eq(order.other_query_equal,false); + const unknown=navigationComparison(base+'?unknown=one',base+'?unknown=two'); + eq(unknown.other_query_equal,false);eq(unknown.xsec_token.values_equal,true);eq(unknown.xsec_source.values_equal,true); + const hash=navigationComparison(base+'?q=fixture#one',base+'?q=fixture#two'); + eq(hash.same_query,true);eq(hash.same_query_pairs,true);eq(hash.same_hash,false); + for(const [left,right] of [[base+'?q=local+fixture',base+'?q=local%20fixture'], + [base+'?xsec_token=one',base+'?xsec_token=two'],[base+'?unknown=one',base+'?unknown=two'], + [base+'?a=one&b=two',base+'?b=two&a=one'],[base+'#one',base+'#two']]){ + const context=action(left,'profile');let error; + try{assertNativeUiResultLocation(context,right);}catch(value){error=value;} + ok(error instanceof ActionInputError);eq(error.code,'NAVIGATION_BINDING_MISMATCH'); + eq(error.navigation_comparison,navigationComparison(left,right)); + eq(navigationPopupMatches(left,right),false,'diagnostic equality facts never weaken full profile URL binding'); + } +}); + +await group('only new navigation mismatch errors receive comparison fields and old error shapes stay unchanged',()=>{ + const expected=`${official}/user/profile/${profileId}?xsec_token=LOCAL_ERROR_SECRET&unknown=LOCAL_UNKNOWN_SECRET`; + const context=action(expected,'profile');let mismatch; + try{assertNativeUiResultLocation(context,expected+'&changed=1');}catch(error){mismatch=error;} + ok(mismatch instanceof ActionInputError);eq(mismatch.code,'NAVIGATION_BINDING_MISMATCH'); + eq(mismatch.navigation_comparison.same_object_id,true);eq(mismatch.navigation_comparison.other_query_equal,false); + for(const marker of ['LOCAL_ERROR_SECRET','LOCAL_UNKNOWN_SECRET',profileId,official,'unknown']){ + eq((JSON.stringify(mismatch)+' '+mismatch.message).includes(marker),false); + } + const cases=[ + [{executionProfile:NATIVE_UI_PROFILE,kind:'click',pageUrl:official+'/explore',href:note},`${official}/explore/${otherId}`,'NOTE_BINDING_MISMATCH'], + [{kind:'click',pageUrl:official+'/explore',href:note},`${official}/explore/${otherId}`,'NOTE_BINDING_MISMATCH'], + [context,'https://evil.invalid/user/profile/'+profileId,'SOURCE_CHANGED'], + [{executionProfile:NAVIGATION_UI_PROFILE,kind:'read',pageUrl:profileUrl},profileUrl+'&changed=1','SOURCE_CHANGED'], + ]; + for(const [current,href,code] of cases){ + let error;try{assertNativeUiResultLocation(current,href);}catch(value){error=value;} + ok(error instanceof ActionInputError);eq(error.code,code);eq(Object.hasOwn(error,'navigation_comparison'),false); + } +}); + +await group('popup continuation binds complete non-note destinations and the fixed note identity',()=>{ + for(const href of [profileUrl,official+'/notification',official+'/explore',official+'/search_result?keyword=fixture']){ + eq(navigationPopupMatches(href,href),true); + for(const changed of [href+'#changed',href+(href.includes('?') ? '&changed=1' : '?changed=1')]){ + eq(navigationPopupMatches(href,changed),false); + eq(navigationPopupMatches(changed,href),false); + } + } + for(const expected of [undefined,null,'',{},'about:blank',official+'/message','https://evil.invalid/explore']){ + eq(navigationPopupMatches(expected,profileUrl),false); + eq(navigationPopupMatches(profileUrl,expected),false); + } + for(const actualUrl of [`${official}/user/profile/${otherId}?source=LOCAL_FIXTURE`, + `${official}/user/profile/${profileId}`,note,official+'/notification', + `https://u:p@www.xiaohongshu.com/user/profile/${profileId}?source=LOCAL_FIXTURE`, + `https://www.xiaohongshu.com:8443/user/profile/${profileId}?source=LOCAL_FIXTURE`, + `https://www.xiaohongshu.com.evil.invalid/user/profile/${profileId}?source=LOCAL_FIXTURE`, + `http://www.xiaohongshu.com/user/profile/${profileId}?source=LOCAL_FIXTURE`, + official+'/api/sns/web/v1/follow']) eq(navigationPopupMatches(profileUrl,actualUrl),false); +}); + +const {beginNativeUiRequestPolicy,evaluatePlatformRequest,matchPlatformRequest,endPlatformRequestPolicy}=await import('./platform-policy.mjs'); +const policies=new Set(); +const mint=context=>{const policy=beginNativeUiRequestPolicy(context);ok(policy);policies.add(policy);return policy;}; +const request={automating:true,sessionOrigin:official,pageUrl:note,activeMainFrame:true,method:'POST',resourceType:'xhr', + url:'https://so.xiaohongshu.com/api/synthetic-native-request',now:10001}; +const allow={allowed:true,policy_version:'xhs-page-navigation-v1',policy_rule:'native_page_traffic'}; +const denied=(context,reason)=>{ + const value=evaluatePlatformRequest(context);eq(value.allowed,false);if(reason) eq(value.reason,reason);eq(matchPlatformRequest(context),null); +}; + +await group('navigation request capabilities bind the original source and complete target',()=>{ + const context={...action(profileUrl,'profile'),sessionOrigin:official,now:10000}; + const policy=mint(context); + eq(policy,{version:'xhs-page-navigation-v1',scope:'native_ui',expiresAt:25000});eq(Object.isFrozen(policy),true); + eq(evaluatePlatformRequest({...request,policy}),allow); + eq(evaluatePlatformRequest({...request,policy,pageUrl:profileUrl}),allow); + for(const pageUrl of [profileUrl+'&changed=1',profileUrl+'#changed',`${official}/user/profile/${otherId}`, + official+'/notification',note+'&changed=1',official+'/explore']) denied({...request,policy,pageUrl},'page_changed'); + context.pageUrl=official+'/explore';context.href=official+'/notification';context.actualRoots[0].href=context.href; + eq(evaluatePlatformRequest({...request,policy,pageUrl:profileUrl}),allow,'issued source facts cannot be mutated by the caller'); + for(const href of [official+'/explore',official+'/notification',official+'/search_result?keyword=fixture']){ + const kind=navigationTarget(href).kind; + const policy=mint({...action(href,kind),sessionOrigin:official,now:10000}); + eq(evaluatePlatformRequest({...request,policy,pageUrl:href}),allow); + denied({...request,policy,pageUrl:profileUrl},'page_changed'); + } +}); + +await group('note results allow only the same fixed identity across standard aliases and tracking changes',()=>{ + const context=action(note,'note',official+'/search_result?keyword=fixture'); + const policy=mint({...context,sessionOrigin:official,now:10000}); + const aliases=[note,`${official}/search_result/${noteId}`,`${official}/explore/${noteId}`, + `${official}/explore/${noteId}?xsec_token=OTHER_LOCAL_TRACKING&source=fixture`, + `${official}/search_result/${noteId.toUpperCase()}?source=changed#fixture`]; + for(const pageUrl of aliases){ + eq(assertNativeUiResultLocation(context,pageUrl),true); + eq(evaluatePlatformRequest({...request,policy,pageUrl}),allow); + eq(navigationPopupMatches(note,pageUrl),true); + eq(navigationPopupMatches(pageUrl,note),true); + } + // Aliases are accepted only after a pinned actual anchor was activated. + // The click's precondition still binds the exact href observed on that node. + reject({...context,actualRoots:[actual(`${official}/search_result/${noteId}`,'note')]}); + reject({...context,href:`${official}/search_result/${noteId}`}); + for(const pageUrl of [`${official}/explore/${otherId}`,`${official}/search_result/${otherId}`, + `${official}/user/profile/${noteId}`,official+'/explore',official+'/notification', + `${official}/explore/${noteId}/delete`,`${official}/search_result_ai/${noteId}`, + `https://evil.invalid/explore/${noteId}`,`https://www.xiaohongshu.com:8443/explore/${noteId}`, + `https://u:p@www.xiaohongshu.com/explore/${noteId}`]){ + const inScope=validNavigationPage(pageUrl); + throws(()=>assertNativeUiResultLocation(context,pageUrl),inScope ? 'NAVIGATION_BINDING_MISMATCH' : 'SOURCE_CHANGED'); + denied({...request,policy,pageUrl}); + eq(navigationPopupMatches(note,pageUrl),false); + } + return {note_equivalence:'official standard note route and identical note ID only; non-note target hrefs stay exact', + pre_click_href_exact:true}; +}); + +await group('profile hydration may only omit the two observed single channel context parameters',async()=>{ + const base=`${official}/user/profile/${profileId}`; + const fixed='xsec_token=LOCAL_FIXED_TOKEN&xsec_source=LOCAL_FIXED_SOURCE'; + const expected=base+'?'+fixed+'&channel_type=LOCAL_CHANNEL&parent_page_channel_type=LOCAL_PARENT&other=LOCAL_OTHER#fixed'; + const context=action(expected,'profile',note); + const policy=mint({...context,sessionOrigin:official,now:10000}); + const allowedUrls=[expected, + base+'?'+fixed+'&parent_page_channel_type=LOCAL_PARENT&other=LOCAL_OTHER#fixed', + base+'?'+fixed+'&channel_type=LOCAL_CHANNEL&other=LOCAL_OTHER#fixed', + base+'?'+fixed+'&other=LOCAL_OTHER#fixed']; + for(const pageUrl of allowedUrls){ + eq(assertNativeUiResultLocation(context,pageUrl),true); + eq(evaluatePlatformRequest({...request,policy,pageUrl}),allow); + eq(navigationPopupMatches(expected,pageUrl),true); + } + // An existing empty value is still a single observed pair, and can only + // disappear; no arbitrary new channel pair may be created during hydration. + eq(navigationPopupMatches(base+'?channel_type=',base),true); + eq(navigationPopupMatches(base+'?parent_page_channel_type=',base),true); + const changedUrls=[ + base+'?'+fixed+'&channel_type=CHANGED&parent_page_channel_type=LOCAL_PARENT&other=LOCAL_OTHER#fixed', + base+'?'+fixed+'&parent_page_channel_type=CHANGED&other=LOCAL_OTHER#fixed', + base+'?'+fixed+'&channel_type=LOCAL_CHANNEL&channel_type=LOCAL_CHANNEL&other=LOCAL_OTHER#fixed', + base+'?'+fixed+'&parent_page_channel_type=LOCAL_PARENT&parent_page_channel_type=LOCAL_PARENT&other=LOCAL_OTHER#fixed', + base+'?xsec_token=CHANGED&xsec_source=LOCAL_FIXED_SOURCE&other=LOCAL_OTHER#fixed', + base+'?xsec_token=LOCAL_FIXED_TOKEN&xsec_source=CHANGED&other=LOCAL_OTHER#fixed', + base+'?xsec_source=LOCAL_FIXED_SOURCE&other=LOCAL_OTHER#fixed', + base+'?'+fixed+'&other=CHANGED#fixed',base+'?'+fixed+'#fixed', + base+'?'+fixed+'&other=LOCAL_OTHER&added=NEW#fixed', + base+'?other=LOCAL_OTHER&'+fixed+'#fixed', + base+'?'+fixed+'&other=LOCAL_OTHER#changed',base+'?'+fixed+'&other=LOCAL_OTHER', + `${official}/user/profile/${otherId}?${fixed}&other=LOCAL_OTHER#fixed`, + `${official}/explore/${profileId}?${fixed}&other=LOCAL_OTHER#fixed`, + `https://evil.invalid/user/profile/${profileId}?${fixed}&other=LOCAL_OTHER#fixed`, + `https://www.xiaohongshu.com:8443/user/profile/${profileId}?${fixed}&other=LOCAL_OTHER#fixed`, + `https://u:p@www.xiaohongshu.com/user/profile/${profileId}?${fixed}&other=LOCAL_OTHER#fixed`, + ]; + for(const pageUrl of changedUrls){ + eq(navigationPopupMatches(expected,pageUrl),false); + throws(()=>assertNativeUiResultLocation(context,pageUrl),validNavigationPage(pageUrl) ? 'NAVIGATION_BINDING_MISMATCH' : 'SOURCE_CHANGED'); + denied({...request,policy,pageUrl}); + } + for(const [left,right] of [ + [base+'?'+fixed,base+'?'+fixed+'&channel_type=NEW'], + [base+'?'+fixed,base+'?'+fixed+'&parent_page_channel_type=NEW'], + [base+'?channel_type=one',base+'?parent_page_channel_type=NEW'], + [base+'?parent_page_channel_type=one',base+'?channel_type=NEW'], + [base+'?channel_type=one&channel_type=two',base], + [base+'?parent_page_channel_type=one&parent_page_channel_type=two',base], + [base+'?channel_type=one&channel_type=one',base+'?channel_type=one'], + ]) eq(navigationPopupMatches(left,right),false); + const duplicateExact=base+'?channel_type=one&channel_type=one'; + eq(navigationPopupMatches(duplicateExact,duplicateExact),true,'an unchanged complete href keeps the existing exact-match contract'); + // The extra rule applies to profiles only. Notification and search targets + // must not inherit profile-specific channel omission behavior. + for(const pathname of ['/explore','/notification','/search_result']){ + eq(navigationPopupMatches(official+pathname+'?channel_type=one',official+pathname),false); + } + const {nativePopupRequestContext}=await import('./browser-worker.mjs'); + const sourceFrame={},popupFrame={}; + const source={url:()=>note,mainFrame:()=>sourceFrame},popup={url:()=>allowedUrls[3],mainFrame:()=>popupFrame}; + sourceFrame.page=()=>source;popupFrame.page=()=>popup; + const permission={opener:source,sourceUrl:note,expectedUrl:expected,pendingPopup:popup,adopted:false}; + const popupContext={executionProfile:NAVIGATION_UI_PROFILE,automating:true,policy,policyPage:source,currentPage:source, + permission,requestPage:popup,requestFrame:popupFrame,popupOpener:source,now:10001}; + eq(nativePopupRequestContext(popupContext),{activeMainFrame:true,pageUrl:allowedUrls[3]}); + eq(nativePopupRequestContext({...popupContext,popupOpener:null}),null); + eq(nativePopupRequestContext({...popupContext,now:25000}),null); + eq(nativePopupRequestContext({...popupContext,executionProfile:NATIVE_UI_PROFILE}),null); + eq(policy.expiresAt,25000); + reject({...context,actualRoots:[actual(allowedUrls[3],'profile')]},'WRITE_ACTION_BLOCKED'); + return {fixed_omittable_keys:['channel_type','parent_page_channel_type'],other_query_values_and_order_preserved:true, + popup_expiry_and_opener_unchanged:true,pre_click_href_exact:true}; +}); + +await group('fifteen second capabilities reject early exact expiry revoked and forged use',()=>{ + const policy=mint({...action(profileUrl,'profile'),sessionOrigin:official,now:10000}); + eq(evaluatePlatformRequest({...request,policy,now:10000}),allow); + eq(evaluatePlatformRequest({...request,policy,now:24999}),allow); + denied({...request,policy,now:9999},'policy_not_started'); + for(const now of [25000,25001]) denied({...request,policy,now},'policy_expired'); + for(const now of [NaN,Infinity,null,'10001']) denied({...request,policy,now},'invalid_time'); + denied({...request,policy:{...policy}},'inactive_policy'); + endPlatformRequestPolicy(policy);denied({...request,policy},'inactive_policy'); +}); + +await group('request authority remains first party HTTPS xhr fetch and requires a permitted UI action',()=>{ + const context={...action(profileUrl,'profile'),sessionOrigin:official,now:10000}; + const policy=mint(context); + for(const host of ['www.xiaohongshu.com','xiaohongshu.com','so.xiaohongshu.com','as.xiaohongshu.com']){ + for(const resourceType of ['xhr','fetch']) eq(evaluatePlatformRequest({...request,policy,resourceType,url:`https://${host}/synthetic-native`}),allow); + } + for(const patch of [{activeMainFrame:false},{automating:false},{sessionOrigin:'https://evil.invalid'}, + {pageUrl:official+'/message'},{pageUrl:'https://evil.invalid/explore'}, + {url:'https://www.xiaohongshu.com.evil.invalid/synthetic-native'}, + {url:'https://u:p@so.xiaohongshu.com/synthetic-native'}, + {url:'https://so.xiaohongshu.com:8443/synthetic-native'}, + {url:'http://so.xiaohongshu.com/synthetic-native'}, + {url:'http://127.0.0.1/synthetic-native'}, + {resourceType:'document'},{resourceType:'websocket'},{method:'GET'},{method:'DELETE'}]) denied({...request,policy,...patch}); + for(const patch of [{executionProfile:'unknown'},{executionProfile:null},{sessionOrigin:'https://evil.invalid'}, + {kind:'follow'},{kind:'send_message'},{observed:null},{href:official+'/message'}, + {actualRoots:[actual(official+'/message','notification')]},{now:NaN}]){ + eq(beginNativeUiRequestPolicy({...context,...patch}),null); + } +}); + +await group('old v1 cannot receive the navigation page scope via copied or missing profile metadata',()=>{ + const readRoot={tag:'section',readonly_root:true,sensitive:false,forbidden:false,native_context:{read_root:true}}; + for(const pageUrl of [profileUrl,official+'/notification']){ + for(const executionProfile of [undefined,NATIVE_UI_PROFILE]){ + reject({executionProfile,kind:'read',pageUrl,actualRoots:[readRoot]},'SCOPE_BLOCKED'); + eq(beginNativeUiRequestPolicy({executionProfile,kind:'read',pageUrl,actualRoots:[readRoot],sessionOrigin:official,now:10000}),null); + } + eq(assertNativeUiAction({executionProfile:NAVIGATION_UI_PROFILE,kind:'read',pageUrl,actualRoots:[readRoot]}),true); + } +}); + +await group('popup request handoff needs one fixed opener target and genuine main frame without renewing policy',async()=>{ + const {nativePopupRequestContext}=await import('./browser-worker.mjs'); + const sourceFrame={},popupFrame={},otherFrame={}; + const source={url:()=>note,mainFrame:()=>sourceFrame}; + const popup={url:()=>profileUrl,mainFrame:()=>popupFrame}; + sourceFrame.page=()=>source;popupFrame.page=()=>popup; + const otherPopup={url:()=>profileUrl,mainFrame:()=>({})}; + const policy=mint({...action(profileUrl,'profile'),sessionOrigin:official,now:10000}); + const permission={opener:source,sourceUrl:note,expectedUrl:profileUrl,adopted:false,pendingPopup:popup}; + const context={executionProfile:NAVIGATION_UI_PROFILE,automating:true,policy,policyPage:source,currentPage:source, + permission,requestPage:popup,requestFrame:popupFrame,popupOpener:source,now:10001}; + const accepted={activeMainFrame:true,pageUrl:profileUrl}; + const before=JSON.stringify({permission:{sourceUrl:permission.sourceUrl,expectedUrl:permission.expectedUrl,adopted:permission.adopted},policy}); + eq(nativePopupRequestContext(context),accepted); + eq(evaluatePlatformRequest({...request,policy,...nativePopupRequestContext(context)}),allow); + eq(JSON.stringify({permission:{sourceUrl:permission.sourceUrl,expectedUrl:permission.expectedUrl,adopted:permission.adopted},policy}),before,'request qualification changes neither facts nor policy'); + eq(policy.expiresAt,25000); + const adopted={...context,policyPage:popup,currentPage:popup,permission:{...permission,adopted:true}}; + eq(nativePopupRequestContext(adopted),accepted); + eq(adopted.policy,policy,'adoption reuses the original issued capability'); + eq(adopted.policy.expiresAt,25000,'adoption cannot start a new fifteen second window'); + for(const patch of [ + {executionProfile:NATIVE_UI_PROFILE},{executionProfile:undefined},{executionProfile:'unknown'}, + {automating:false},{policy:null},{policy:{...policy,scope:'search'}}, + {policy:{...policy,version:'xhs-page-native-v1'}},{policy:{...policy,expiresAt:10000}}, + {requestPage:source},{requestFrame:sourceFrame},{requestFrame:otherFrame},{requestFrame:null}, + {popupOpener:null},{popupOpener:otherPopup},{permission:null}, + {permission:{...permission,manual:true}},{permission:{...permission,opener:otherPopup}}, + {permission:{...permission,sourceUrl:note+'&changed=1'}}, + {permission:{...permission,expectedUrl:profileUrl+'&changed=1'}}, + {permission:{...permission,expectedUrl:official+'/message'}}, + {permission:{...permission,pendingPopup:otherPopup}}, + {permission:{...permission,adopted:true}}, + {policyPage:otherPopup},{currentPage:otherPopup}, + {requestPage:{url:()=>official+'/message',mainFrame:()=>popupFrame}}, + {requestPage:{url:()=>profileUrl+'&changed=1',mainFrame:()=>popupFrame}}, + {requestPage:{url:()=>`https://evil.invalid/user/profile/${profileId}`,mainFrame:()=>popupFrame}}, + {now:25000},{now:25001},{now:NaN},{now:Infinity},{now:'10001'}, + ]) eq(nativePopupRequestContext({...context,...patch}),null); + for(const patch of [{policyPage:source},{currentPage:source},{permission:{...permission,adopted:true,pendingPopup:otherPopup}}]){ + eq(nativePopupRequestContext({...adopted,...patch}),null); + } + // Even apparently correct public fields cannot fabricate the private issued + // policy. Qualification is followed by the existing capability evaluator. + const clone={...policy}; + denied({...request,policy:clone,...(nativePopupRequestContext({...context,policy:clone}) || {})},'inactive_policy'); + eq(nativePopupRequestContext({...context,now:24999}),accepted); + denied({...request,policy,now:25000,...accepted},'policy_expired'); + endPlatformRequestPolicy(policy); + denied({...request,policy,...accepted},'inactive_policy'); + return {pre_adoption_and_post_adoption:true,child_frames_and_unrequested_popups_rejected:true, + original_expiry:25000,reissued_policy:false,production_network:false}; +}); + +await group('real loopback Edge popup frame and opener objects exercise the handoff predicate',async()=>{ + const {nativePopupRequestContext}=await import('./browser-worker.mjs'); + const {chromium}=await import('playwright'); + let browser,fixtureRequests=0,unexpectedNetwork=0; + const server=createServer((req,res)=>{ + fixtureRequests+=1; + const html=req.url==='/source' ? 'Open local profile fixture' : + req.url==='/destination' ? '

    Local popup

    ' : '

    Local child frame

    '; + res.writeHead(200,{'content-type':'text/html; charset=utf-8'}).end(''+html); + }); + await new Promise(resolve=>server.listen(0,'127.0.0.1',resolve)); + const loopback=`http://127.0.0.1:${server.address().port}`; + try{ + browser=await chromium.launch({channel:process.platform==='win32' ? 'msedge' : 'chromium',headless:true}); + const browserContext=await browser.newContext({viewport:{width:1280,height:840},serviceWorkers:'block'}); + let postRequest; + browserContext.on('request',req=>{if(req.method()==='POST' && req.url()===loopback+'/local-post') postRequest=req;}); + await browserContext.route('**/*',async route=>{ + if(new URL(route.request().url()).origin!==loopback){unexpectedNetwork+=1;await route.abort();return;}await route.continue(); + }); + const actualSource=await browserContext.newPage();await actualSource.goto(loopback+'/source'); + const [actualPopup]=await Promise.all([actualSource.waitForEvent('popup'),actualSource.locator('#open').click()]); + await actualPopup.waitForLoadState('load'); + eq(await actualPopup.opener(),actualSource,'the popup has a real explicit opener'); + ok(postRequest,'local popup emitted an actual main-frame POST'); + eq(postRequest.frame(),actualPopup.mainFrame());eq(postRequest.frame().page(),actualPopup); + const child=actualPopup.frames().find(frame=>frame!==actualPopup.mainFrame());ok(child);eq(child.parentFrame(),actualPopup.mainFrame()); + // Only the fixed URL reads are synthetic. Frame and opener identity were + // obtained from the actual loopback browser above; no official page is + // navigated, intercepted, fetched, or accepted via an origin override. + const source=actualSource,popup=actualPopup; + const realSourceUrl=source.url.bind(source),realPopupUrl=popup.url.bind(popup); + // Project only the two URL accessors after local navigation completed; + // retain genuine Page, Frame, frame.page() and opener object identities. + source.url=()=>note;popup.url=()=>profileUrl; + const policy=mint({...action(profileUrl,'profile'),sessionOrigin:official,now:10000}); + const permission={opener:source,sourceUrl:note,expectedUrl:profileUrl,pendingPopup:popup,adopted:false}; + const context={executionProfile:NAVIGATION_UI_PROFILE,automating:true,policy,policyPage:source,currentPage:source, + permission,requestPage:popup,requestFrame:postRequest.frame(),popupOpener:source,now:10001}; + const accepted={activeMainFrame:true,pageUrl:profileUrl}; + try{ + eq(nativePopupRequestContext(context),accepted); + eq(evaluatePlatformRequest({...request,policy,...accepted}),allow); + eq(nativePopupRequestContext({...context,requestFrame:child}),null); + eq(nativePopupRequestContext({...context,popupOpener:null}),null); + eq(nativePopupRequestContext({...context,permission:null}),null); + eq(nativePopupRequestContext({...context,executionProfile:NATIVE_UI_PROFILE}),null); + const adopted={...context,currentPage:popup,policyPage:popup,permission:{...permission,adopted:true}}; + eq(nativePopupRequestContext(adopted),accepted);eq(adopted.policy,policy);eq(policy.expiresAt,25000); + eq(nativePopupRequestContext({...adopted,now:25000}),null); + }finally{source.url=realSourceUrl;popup.url=realPopupUrl;} + eq(nativePopupRequestContext(context),null,'unprojected real loopback URL cannot become an official profile'); + eq(unexpectedNetwork,0);endPlatformRequestPolicy(policy); + return {browser:process.platform==='win32' ? 'Edge' : 'Chromium',loopback_requests:fixtureRequests,local_main_frame_post:true, + real_opener_and_frame_identity:true,official_urls_synthetic_metadata_only:true,unexpected_network_attempts:unexpectedNetwork, + production_origin_override:false,worker_navigation_profile_end_to_end:false}; + }finally{if(browser) await browser.close();await new Promise(resolve=>server.close(resolve));} +}); + +await group('fixed loopback DOM distinguishes ordinary navigation from editable and write controls',async()=>{ + const {inspectElements}=await import('./browser-worker.mjs'); + const {chromium}=await import('playwright'); + const observations=[]; + let browser,fixtureRequests=0,unexpectedNetwork=0; + const server=createServer((req,res)=>{fixtureRequests+=1;res.writeHead(200,{'content-type':'text/html; charset=utf-8'}).end('Local navigation DOM fixture');}); + await new Promise(resolve=>server.listen(0,'127.0.0.1',resolve)); + const loopback=`http://127.0.0.1:${server.address().port}`; + try{ + browser=await chromium.launch({channel:process.platform==='win32' ? 'msedge' : 'chromium',headless:true}); + const browserContext=await browser.newContext({viewport:{width:1280,height:840},serviceWorkers:'block'}); + await browserContext.route('**/*',async route=>{ + if(new URL(route.request().url()).origin!==loopback){unexpectedNetwork+=1;await route.abort();return;}await route.continue(); + }); + const page=await browserContext.newPage();await page.goto(loopback+'/',{waitUntil:'domcontentloaded'}); + const fixtures=[ + ['ordinary-profile',`Profile`,'profile',true], + ['ordinary-notification',`Notifications`,'notification',true], + ['return-explore',`Explore`,'explore',true], + ['return-search',`Search`,'search',true], + ['self-profile',`Profile`,'profile',true], + ['download-profile',`Profile`,'profile',false], + ['blank-profile',`Profile`,'profile',true], + ['blank-note',`Note`,'note',true], + ['blank-notification',`Notifications`,'notification',true], + ['named-profile',`Profile`,'profile',false], + ['editable-profile',`Profile`,'profile',false], + ['inherited-editable',``,'profile',false], + ['form-profile',`
    Profile
    `,'profile',false], + ['role-button-profile',`Profile`,'profile',false], + ['button-ancestor-profile',``,'profile',false], + ['hidden-profile',``,'profile',false], + ['follow-button','','profile',false], + ['message-button','','profile',false], + ['message-input','','profile',false], + ['external-profile',`Profile`,'profile',false], + ['message-link',`Message`,'profile',false], + ]; + for(const [name,html,kind,accepted] of fixtures){ + await page.setContent(`Local only

    Navigation DOM metadata

    ${html}`); + const states=await page.locator('#fixture').evaluate(inspectElements,{origin:official,nativeProfile:true,navigationProfile:true}); + eq(states.length,1);const state=states[0]; + const href=typeof state.href==='string' ? state.href : profileUrl; + const context={...action(href,kind),actualRoots:states}; + if(accepted){eq(assertNativeUiAction(context),true);eq(state.native_context[kind==='note' ? 'note_link' : kind==='profile' ? 'profile_link' : 'navigation_link'],true);} + else reject(context); + observations.push({name,accepted,tag:state.tag,visible:state.visible,sensitive:state.sensitive,native_context:state.native_context}); + } + // Metadata is profile-specific: allowing one explicitly bound popup in the + // new profile does not change the old profile's same-window-only anchors. + await page.setContent(`Note`); + const blankOld=await page.locator('#fixture').evaluate(inspectElements,{origin:official,nativeProfile:true}); + eq(blankOld[0].native_context.note_link,false); + reject({...action(note,'note',official+'/explore'),executionProfile:NATIVE_UI_PROFILE,actualRoots:blankOld}); + const blankNew=await page.locator('#fixture').evaluate(inspectElements,{origin:official,nativeProfile:true,navigationProfile:true}); + eq(blankNew[0].native_context.note_link,true); + eq(assertNativeUiAction({...action(note,'note',official+'/explore'),actualRoots:blankNew}),true); + await page.setContent(`Profile`); + const legacy=await page.locator('#fixture').evaluate(inspectElements,{origin:official}); + eq(Object.hasOwn(legacy[0],'native_context'),false); + const old=await page.locator('#fixture').evaluate(inspectElements,{origin:official,nativeProfile:true}); + eq(Object.hasOwn(old[0].native_context,'profile_link'),false,'new navigation metadata is absent under old inspection'); + const screenshot=path.join(output,'loopback-navigation-dom.png');await page.screenshot({path:screenshot,fullPage:true}); + eq((await fs.readFile(screenshot)).subarray(1,4).toString(),'PNG'); + eq(unexpectedNetwork,0,'official-looking links were never navigated or fetched'); + await fs.writeFile(path.join(output,'loopback-dom-observations.json'),JSON.stringify(observations,null,2)); + browserFixture={browser:process.platform==='win32' ? 'Edge' : 'Chromium',origin:'loopback only',inspected_cases:observations.length, + fixture_requests:fixtureRequests,unexpected_network_attempts:unexpectedNetwork,screenshot,actual_png:true, + worker_navigation_profile_end_to_end:false};return browserFixture; + }finally{if(browser) await browser.close();await new Promise(resolve=>server.close(resolve));} +}); + +for(const policy of policies) endPlatformRequestPolicy(policy); +await report(); +console.log(JSON.stringify({passed:results.length,assertions,output,official_browser_navigation:false, + live_platform_requests:0,application_api_calls:0,model_calls:0})); diff --git a/tools/learning/test-observation-cache.mjs b/tools/learning/test-observation-cache.mjs new file mode 100644 index 0000000..d98b35d --- /dev/null +++ b/tools/learning/test-observation-cache.mjs @@ -0,0 +1,173 @@ +import assert from 'node:assert/strict'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import { pathToFileURL } from 'node:url'; +import { createHash } from 'node:crypto'; +import { chromium } from 'playwright'; +import { describePage } from './browser-worker.mjs'; + +// Compare against an explicitly saved pre-change evaluate function, never a +// production/RPC switch. All fixtures are synthetic and network is blocked. +const baselinePath = process.argv[2]; +assert.ok(baselinePath, 'Pass the saved baseline module path'); +const { describeBaseline } = await import(pathToFileURL(path.resolve(baselinePath)).href); +const runDirectory = path.resolve('.tmp', `observation-cache-${Date.now()}`); +await fs.mkdir(runDirectory, { recursive: true }); +const report = { browser: process.platform === 'win32' ? 'msedge' : 'chromium', real_platform_requests: 0, + baseline_sha256: createHash('sha256').update(await fs.readFile(path.resolve(baselinePath))).digest('hex'), comparisons: [], performance: [] }; +const browser = await chromium.launch({ headless: true, ...(process.platform === 'win32' ? { channel: 'msedge' } : {}) }); +try { + const page = await browser.newPage({ viewport: { width: 1120, height: 760 } }); + await page.route('**/*', route => route.abort()); + const cards = Array.from({ length: 230 }, (_, index) => `
    Author ${index}
    Question ${index}

    Ordinary renovation sample ${index}

    Nested ${index}
    `).join(''); + await page.setContent(`Observation equivalence

    Fixture

    +
    ${cards}

    Clipped ordinary text

    Clipped away content

    +

    Ordinary heading

    Fresh text

    Contact 13800138000

    Transparent content

    Concealed content

    +
    SENSITIVE_ANCESTOR

    SENSITIVE_FORM

    +

    Offscreen retained loaded text

    Visible overlay
    +
    `); + await page.evaluate(() => { + const costs = () => ({ computed_style: 0, bounding_rect: 0, query_selector: 0, query_selector_all: 0, element_from_point: 0 }); + window.observationCosts = costs(); + window.resetObservationCosts = () => { window.observationCosts = costs(); }; + const wrap = (owner, name, cost) => { + const original = owner[name]; + owner[name] = function (...args) { window.observationCosts[cost] += 1; return original.apply(this, args); }; + }; + wrap(window, 'getComputedStyle', 'computed_style'); + wrap(Element.prototype, 'getBoundingClientRect', 'bounding_rect'); + for (const owner of [Document.prototype, Element.prototype]) { + wrap(owner, 'querySelector', 'query_selector'); + wrap(owner, 'querySelectorAll', 'query_selector_all'); + } + wrap(Document.prototype, 'elementFromPoint', 'element_from_point'); + }); + async function sample(read, request) { + await page.evaluate(() => window.resetObservationCosts()); + const started = performance.now(); + const value = await read(page, request); + const duration_ms = performance.now() - started; + return { value, duration_ms, costs: await page.evaluate(() => window.observationCosts) }; + } + async function compare(name, mutate, request = null) { + if (mutate) await page.evaluate(mutate); + const baseline = await sample(describeBaseline, request), optimized = await sample(describePage, request); + assert.deepEqual(optimized.value, baseline.value, name); + report.comparisons.push({ name, exact: true, elements: optimized.value.elements.length, ...(optimized.value.masks ? { masks: optimized.value.masks.length } : {}) }); + return { baseline, optimized }; + } + const initial = await compare('initial full observation with overlay, clipping, sensitive subtrees and output caps'); + assert.ok(!initial.optimized.value.text.includes('SENSITIVE_ANCESTOR')); + assert.ok(!initial.optimized.value.text.includes('SENSITIVE_FORM')); + assert.ok(!initial.optimized.value.text.includes('13800138000')); + assert.ok(initial.optimized.value.masks.includes('#phone-leaf')); + assert.ok(initial.optimized.value.elements.length <= 120); + assert.equal(initial.optimized.value.viewport.scroll_y, 0); + await compare('canonical metadata', null, { selector: '#search-field', requireCanonical: true }); + await compare('collection metadata', null, { selector: '#scroller a.name', collection: true }); + await compare('noncanonical metadata rejected', null, { selector: 'input[type=search]', requireCanonical: true }); + await compare('sensitive metadata rejected', null, { selector: '#sensitive-child' }); + await compare('bounded metadata rejects more than 500 nodes', () => { + const root = document.createElement('div'); root.id = 'many'; + root.innerHTML = 'X'.repeat(501); document.body.append(root); + }, { selector: '.many-node', collection: true }); + await compare('remove large fixture and overlay; text and source attributes change', () => { + document.querySelector('#many').remove(); document.querySelector('#overlay').remove(); + document.querySelector('#mutable').textContent = 'Changed source after previous call'; + document.querySelector('#search-field').setAttribute('placeholder', 'Find content'); + document.querySelector('#search-field').setAttribute('name', 'query'); + }); + await compare('scroll root changes viewport and current scroll metadata', () => { document.querySelector('#scroller').scrollTop = 630; }); + const scrolled = await compare('metadata refreshed after root scroll', null, { selector: '#scroller', requireCanonical: true }); + assert.equal(scrolled.optimized.value.elements[0].scroll_y.scroll_top, 630); + await compare('window scroll changes clipping', () => window.scrollTo(0, 460)); + await compare('style changes are not cached across calls', () => { + window.scrollTo(0, 0); + document.querySelector('#hidden-parent').classList.remove('hidden'); + document.querySelector('#opacity').style.opacity = '1'; + document.querySelector('#concealed').style.visibility = 'visible'; + document.querySelector('#ordinary').style.visibility = 'collapse'; + document.querySelector('#scroller').style.height = '240px'; + }); + await compare('ancestor sensitivity changes are not cached across calls', () => { + document.querySelector('#ordinary').style.visibility = 'visible'; + document.querySelector('#sensitive-ancestor').setAttribute('aria-label', 'Ordinary section'); + document.querySelector('#private-field').setAttribute('type', 'text'); + document.querySelector('#private-form label').firstChild.textContent = 'Ordinary'; + document.querySelector('#private-form').id = 'ordinary-form'; + document.querySelector('#private-text').textContent = 'Now ordinary form'; + }); + await compare('new private field causes current form exclusion', () => { + const form = document.querySelector('#ordinary-form'); + const field = document.createElement('input'); field.autocomplete = 'one-time-code'; form.append(field); + }); + await compare('disabled field and changed sensitive field label', () => { + document.querySelector('#search-field').disabled = true; + document.querySelector('#search-field').setAttribute('aria-label', 'verification code'); + }); + await compare('delete and recreate nodes with same IDs and different geometry', () => { + document.querySelector('#scroller').remove(); + const section = document.createElement('section'); section.id = 'scroller'; section.style.height = '300px'; + section.innerHTML = ''; + document.querySelector('#main').prepend(section); + }); + await compare('metadata refreshed after node replacement', null, { selector: '#scroller', requireCanonical: true }); + const duplicate = await compare('duplicate target metadata preserves ambiguity', () => { + const other = document.querySelector('#scroller').cloneNode(true); document.querySelector('#main').append(other); + }, { selector: '#scroller', requireCanonical: true }); + assert.equal(duplicate.optimized.value.elements[0].unique, false); + assert.equal(duplicate.optimized.value.elements[0].count, 2); + await page.evaluate(async () => { + document.body.innerHTML = '

    Media observation

    Ordinary image and video

    '; + const pixel = 'data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///ywAAAAAAQABAAACAUwAOw=='; + const images = [...document.images]; + await Promise.all(images.map(image => new Promise(resolve => { image.onload = image.onerror = resolve; image.src = pixel; }))); + window.scrollTo(0, 0); + }); + const media = await compare('media hints preserve loaded state, avatar exclusion and hidden media exclusion'); + const hints = media.optimized.value.elements.find(element => element.selector === '#media-root').read_hints; + assert.ok(hints.some(hint => hint.selector === '#content-image' && hint.media_kind === 'image' && hint.loaded === true)); + assert.ok(hints.some(hint => hint.selector === '#content-video' && hint.media_kind === 'video' && hint.loaded === false)); + assert.ok(!hints.some(hint => hint.selector === '#avatar-image' || hint.selector === '#hidden-image')); + const long = await compare('raw walk reuse preserves normalization and per-root text truncation', () => { + document.body.innerHTML = '

    Tail must be truncated only at the outer root

    '; + document.querySelector('#long-text').textContent = ' \n Leading whitespace\t' + '中文 Long text '.repeat(1600) + ' 13800138000'; + }); + assert.equal(long.optimized.value.text.length, 16000); + assert.ok(long.optimized.value.text.startsWith('Leading whitespace 中文 Long text')); + const limited = await compare('candidate/control/read limits and order remain identical in a dense visible page', () => { + document.body.innerHTML = '
    '; + const root = document.querySelector('#dense-root'); + root.innerHTML = Array.from({ length: 130 }, (_, index) => `
    Read ${index}
    `).join('') + + Array.from({ length: 40 }, (_, index) => ``).join('') + + Array.from({ length: 25 }, (_, index) => ``).join(''); + }); + assert.ok(limited.optimized.value.elements.length <= 120); + assert.equal(limited.optimized.value.elements.filter(element => !element.collection && element.kind === 'fill').length, 16); + assert.equal(limited.optimized.value.elements.filter(element => !element.collection && element.tag === 'button').length, 24); + assert.equal(limited.optimized.value.elements.filter(element => !element.collection && element.kind === 'read').length, 48); + + // Measure deterministic DOM work on the large unmodified fixture as well as + // elapsed time. Elapsed time is reported, never used as a flaky pass threshold. + report.performance.push({ name: 'initial_large_fixture', baseline_ms: initial.baseline.duration_ms, optimized_ms: initial.optimized.duration_ms, + baseline_costs: initial.baseline.costs, optimized_costs: initial.optimized.costs }); + for (const name of ['computed_style', 'bounding_rect', 'element_from_point']) { + assert.ok(initial.optimized.costs[name] < initial.baseline.costs[name], `${name} must reuse current-call observations`); + } + assert.equal(initial.optimized.costs.query_selector_all, initial.baseline.costs.query_selector_all, 'Candidate discovery queries are unchanged'); + report.status = 'passed'; +} catch (error) { + report.status = 'failed'; report.error = { name: error.name, message: error.message.slice(0, 2000) }; + throw error; +} finally { + await browser.close(); + await fs.writeFile(path.join(runDirectory, 'report.json'), JSON.stringify(report, null, 2) + '\n'); + console.log(JSON.stringify({ report: path.join(runDirectory, 'report.json'), status: report.status, comparisons: report.comparisons.length, real_platform_requests: 0 })); +} diff --git a/tools/learning/test-platform-policy.mjs b/tools/learning/test-platform-policy.mjs new file mode 100644 index 0000000..0f895a1 --- /dev/null +++ b/tools/learning/test-platform-policy.mjs @@ -0,0 +1,296 @@ +import assert from 'node:assert/strict'; +import { beginSearchRequestPolicy, beginNoteOpenRequestPolicy, beginReadonlyRequestPolicy, endPlatformRequestPolicy, matchPlatformRequest, SEARCH_POLICY_VERSION, NOTE_POLICY_VERSION, READONLY_POLICY_VERSION, SEARCH_SCROLL_POLICY_VERSION } from './platform-policy.mjs'; + +const official = 'https://www.xiaohongshu.com'; +const observed = { kind: 'fill', tag: 'input', unique: true, search_input: true, sensitive: false }; +const actual = { tag: 'input', search_input: true, sensitive: false, visible: true, in_viewport: true, unobscured: true }; +const action = { sessionOrigin: official, pageUrl: `${official}/explore`, kind: 'fill', observed, actual, matchCount: 1 }; +const policy = beginSearchRequestPolicy(action); +assert.ok(policy); +const request = { policy, automating: true, sessionOrigin: official, pageUrl: `${official}/search_result_ai?keyword=fixture`, activeMainFrame: true, + method: 'POST', resourceType: 'xhr', url: 'https://so.xiaohongshu.com/api/sns/web/v2/search/notes' }; +let assertions = 1; +function check(value, expected = null) { assert.deepEqual(value, expected); assertions += 1; } + +for (const [rule, url, resourceType = 'xhr'] of [ + ['collect', 'https://t2.xiaohongshu.com/api/v2/collect'], + ['search_history', 'https://so.xiaohongshu.com/api/sns/web/search/history/sync'], + ['search_notes', 'https://so.xiaohongshu.com/api/sns/web/v2/search/notes'], + ['search_onebox', 'https://edith.xiaohongshu.com/api/sns/web/v1/search/onebox'], + ['search_dqa', 'https://so.xiaohongshu.com/api/sns/web/v1/dqa/instant'], + ['apm_data', 'https://apm-fe.xiaohongshu.com/api/data', 'fetch'], + ['apm_data_xhr', 'https://apm-fe.xiaohongshu.com/api/data', 'xhr'], +]) check(matchPlatformRequest({ ...request, url, resourceType }), { policy_version: SEARCH_POLICY_VERSION, policy_rule: rule }); + +const enter = beginSearchRequestPolicy({ ...action, kind: 'press', key: 'Enter' }); +check(matchPlatformRequest({ ...request, policy: enter }), { policy_version: SEARCH_POLICY_VERSION, policy_rule: 'search_notes' }); +for (const changed of [ + {sessionOrigin:'http://127.0.0.1'}, {sessionOrigin:'https://xiaohongshu.com'}, {pageUrl:'https://www.xiaohongshu.com.evil.test/explore'}, + {pageUrl:'https://user:password@www.xiaohongshu.com/explore'}, {pageUrl:'http://www.xiaohongshu.com/explore'}, {pageUrl:'invalid'}, + {kind:'click'}, {kind:'read'}, {kind:'extract'}, {kind:'wait'}, {kind:'scroll'}, {kind:'press',key:'Tab'}, + {matchCount:0}, {matchCount:2}, {observed:{...observed,unique:false}}, {observed:{...observed,search_input:false}}, + {observed:{...observed,sensitive:true}}, {observed:{...observed,tag:'button'}}, {observed:null}, + {actual:{...actual,sensitive:true}}, {actual:{...actual,search_input:false}}, {actual:{...actual,visible:false}}, + {actual:{...actual,in_viewport:false}}, {actual:{...actual,unobscured:false}}, {actual:null}, +]) check(beginSearchRequestPolicy({ ...action, ...changed })); + +for (const changed of [ + {policy:null}, {policy:{version:SEARCH_POLICY_VERSION,expiresAt:Date.now()+15000}}, {automating:false}, {activeMainFrame:false}, + {sessionOrigin:'https://xiaohongshu.com'}, {pageUrl:'https://example.com/'}, {pageUrl:'https://user:password@www.xiaohongshu.com/'}, + {method:'GET'}, {method:'PUT'}, {method:'DELETE'}, {resourceType:'fetch'}, {resourceType:'document'}, {resourceType:'other'}, + {url:'https://so.xiaohongshu.com.evil.test/api/sns/web/v2/search/notes'}, {url:'http://so.xiaohongshu.com/api/sns/web/v2/search/notes'}, + {url:'https://user:password@so.xiaohongshu.com/api/sns/web/v2/search/notes'}, {url:'https://so.xiaohongshu.com:8443/api/sns/web/v2/search/notes'}, + {url:'https://so.xiaohongshu.com/api/sns/web/v2/search/notes/'}, {url:'https://so.xiaohongshu.com/api/sns/web/v2/search/notes/delete'}, + {url:'https://so.xiaohongshu.com/api/sns/web/v1/search/notes'}, {url:'https://edith.xiaohongshu.com/api/sns/web/v2/search/notes'}, + {url:'https://so.xiaohongshu.com/api/sns/web/v2/comment/post'}, {url:'https://apm.xiaohongshu.com/api/data'}, {url:'invalid'}, + {url:'https://apm-fe.xiaohongshu.com/api/data/',resourceType:'xhr'}, + {url:'https://apm-fe.xiaohongshu.com.evil.test/api/data',resourceType:'xhr'}, + {url:'https://apm-fe.xiaohongshu.com/api/data',resourceType:'xhr',method:'GET'}, + {url:'https://edith.xiaohongshu.com/api/sns/web/v1/feed'}, +]) check(matchPlatformRequest({ ...request, ...changed })); + +const noteHref = `${official}/search_result/${'a'.repeat(24)}?xsec_token=FIXTURE-SECRET`; +const noteObserved = { kind:'click', tag:'a', unique:true, sensitive:false }; +const noteActual = { tag:'a', sensitive:false, visible:true, in_viewport:true, unobscured:true, href:noteHref }; +const noteAction = { sessionOrigin:official, pageUrl:`${official}/search_result_ai?keyword=fixture`, kind:'click', + observed:noteObserved, actual:noteActual, matchCount:1, href:noteHref }; +const notePolicy = beginNoteOpenRequestPolicy(noteAction); +assert.ok(notePolicy); assertions += 1; +const noteRequest = { ...request, policy:notePolicy, url:'https://edith.xiaohongshu.com/api/sns/web/v1/feed' }; +for (const [rule, url, resourceType = 'xhr'] of [ + ['collect','https://t2.xiaohongshu.com/api/v2/collect'], + ['note_share_code','https://edith.xiaohongshu.com/api/sns/web/share/code'], + ['note_widgets','https://edith.xiaohongshu.com/api/sns/web/v2/widgets'], + ['note_feed','https://edith.xiaohongshu.com/api/sns/web/v1/feed'], + ['note_seo','https://edith.xiaohongshu.com/api/sns/web/worldcup/note/seo'], + ['note_metrics','https://edith.xiaohongshu.com/api/sns/web/v1/note/metrics_report'], + ['note_history_report','https://edith.xiaohongshu.com/api/sns/v1/history/report_web'], + ['apm_data','https://apm-fe.xiaohongshu.com/api/data','fetch'], +]) check(matchPlatformRequest({ ...noteRequest, url, resourceType }), { policy_version:NOTE_POLICY_VERSION, policy_rule:rule }); +const targetedCollectionPolicy = beginNoteOpenRequestPolicy({...noteAction,observed:{...noteObserved,unique:false},targetResolvedUnique:true}); +check(matchPlatformRequest({...noteRequest,policy:targetedCollectionPolicy}), {policy_version:NOTE_POLICY_VERSION,policy_rule:'note_feed'}); +check(beginNoteOpenRequestPolicy({...noteAction,observed:{...noteObserved,unique:false},targetResolvedUnique:true,matchCount:2})); +check(matchPlatformRequest({...request,url:'https://edith.xiaohongshu.com/api/sns/v1/history/report_web'})); +check(matchPlatformRequest({...noteRequest,url:'https://edith.xiaohongshu.com/api/sns/v1/history/report_web',resourceType:'fetch'})); +const exploreHref = `${official}/explore/${'b'.repeat(24)}`; +const explorePolicy = beginNoteOpenRequestPolicy({ ...noteAction, href:exploreHref, actual:{...noteActual,href:exploreHref} }); +check(matchPlatformRequest({...noteRequest,policy:explorePolicy}), { policy_version:NOTE_POLICY_VERSION,policy_rule:'note_feed' }); +for (const changed of [ + {kind:'fill'}, {kind:'press'}, {kind:'read'}, {kind:'extract'}, {kind:'scroll'}, {kind:'wait'}, + {sessionOrigin:'http://127.0.0.1'}, {pageUrl:'https://example.com/'}, {pageUrl:'https://user:pass@www.xiaohongshu.com/'}, + {matchCount:0}, {matchCount:2}, {observed:{...noteObserved,unique:false}}, {observed:{...noteObserved,sensitive:true}}, + {observed:{...noteObserved,tag:'button'}}, {observed:{...noteObserved,kind:'read'}}, {observed:null}, + {actual:{...noteActual,tag:'button'}}, {actual:{...noteActual,sensitive:true}}, {actual:{...noteActual,visible:false}}, + {actual:{...noteActual,in_viewport:false}}, {actual:{...noteActual,unobscured:false}}, {actual:{...noteActual,href:exploreHref}}, +]) check(beginNoteOpenRequestPolicy({...noteAction,...changed})); +for (const href of [ + `${official}/explore`, `${official}/explore/${'b'.repeat(23)}`, `${official}/explore/${'g'.repeat(24)}`, + `${official}/Explore/${'b'.repeat(24)}`, `${official}/explore/${'b'.repeat(24)}/`, `${official}/explore/${'b'.repeat(24)}/delete`, + `${official}/user/profile/${'b'.repeat(24)}`, `https://www.xiaohongshu.com.evil.test/explore/${'b'.repeat(24)}`, + `https://user:pass@www.xiaohongshu.com/explore/${'b'.repeat(24)}`, `javascript:alert(1)`, +]) check(beginNoteOpenRequestPolicy({...noteAction,href,actual:{...noteActual,href}})); +for (const changed of [ + {automating:false}, {activeMainFrame:false}, {pageUrl:'https://example.com/'}, {sessionOrigin:'https://example.com'}, + {method:'GET'}, {method:'DELETE'}, {resourceType:'fetch'}, + {url:'https://edith.xiaohongshu.com/api/sns/web/v1/feed/delete'}, {url:'https://so.xiaohongshu.com/api/sns/web/v1/feed'}, + {url:'https://edith.xiaohongshu.com.evil.test/api/sns/web/v1/feed'}, + {url:'https://user:pass@edith.xiaohongshu.com/api/sns/web/v1/feed'}, + {url:'https://so.xiaohongshu.com/api/sns/web/v2/search/notes'}, {url:'https://apm-fe.xiaohongshu.com/api/data',resourceType:'xhr'}, +]) check(matchPlatformRequest({...noteRequest,...changed})); + +const expired = beginSearchRequestPolicy({ ...action, now:Date.now()-16000 }); +check(matchPlatformRequest({ ...request, policy:expired })); +const expiredNote = beginNoteOpenRequestPolicy({...noteAction,now:Date.now()-16000}); +check(matchPlatformRequest({...noteRequest,policy:expiredNote})); +endPlatformRequestPolicy(policy); +endPlatformRequestPolicy(enter); +endPlatformRequestPolicy(notePolicy); +endPlatformRequestPolicy(explorePolicy); +endPlatformRequestPolicy(targetedCollectionPolicy); +check(matchPlatformRequest(request)); +check(matchPlatformRequest({ ...request, policy:enter })); +check(matchPlatformRequest(noteRequest)); +check(matchPlatformRequest({...noteRequest,policy:explorePolicy})); + +const readRoot = {tag:'div',readonly_root:true,sensitive:false,forbidden:false}; +const readonlyAction = {sessionOrigin:official,pageUrl:`${official}/explore/${'a'.repeat(24)}`,kind:'scroll',selector:'#comments',selectorObserved:true,actualRoots:[readRoot]}; +for (const kind of ['read','extract','scroll','wait']) { + const cap = beginReadonlyRequestPolicy({...readonlyAction,kind}); + const readRequest = {...request,policy:cap,url:'https://t2.xiaohongshu.com/api/v2/collect'}; + check(matchPlatformRequest(readRequest),{policy_version:READONLY_POLICY_VERSION,policy_rule:'collect'}); + check(matchPlatformRequest({...readRequest,url:'https://apm-fe.xiaohongshu.com/api/data',resourceType:'fetch'}),{policy_version:READONLY_POLICY_VERSION,policy_rule:'apm_data'}); + for (const changed of [ + {url:'https://so.xiaohongshu.com/api/sns/web/v2/search/notes'}, {url:'https://edith.xiaohongshu.com/api/sns/web/v1/feed'}, + {url:'https://edith.xiaohongshu.com/api/sns/v1/history/report_web'}, {url:'https://so.xiaohongshu.com/api/sns/web/search/history/sync'}, + {url:'https://t2.xiaohongshu.com/api/v2/collect/post'}, {method:'PUT'}, {resourceType:'fetch'}, {activeMainFrame:false}, {automating:false}, + {pageUrl:'https://example.com/'}, {url:'https://apm-fe.xiaohongshu.com/api/data',resourceType:'xhr'}, + ]) check(matchPlatformRequest({...readRequest,...changed})); + endPlatformRequestPolicy(cap); + check(matchPlatformRequest(readRequest)); +} +for (const changed of [ + {kind:'click'}, {kind:'fill'}, {kind:'press'}, {selectorObserved:false}, {selector:''}, {selector:'body'}, + {sessionOrigin:'http://127.0.0.1'}, {pageUrl:'https://example.com/'}, {pageUrl:'https://user:pass@www.xiaohongshu.com/'}, + {actualRoots:[]}, {actualRoots:null}, {actualRoots:[{...readRoot,readonly_root:false}]}, + {actualRoots:[{...readRoot,sensitive:true}]}, {actualRoots:[{...readRoot,forbidden:true}]}, + ...['input','textarea','select','button','form','script','iframe','body'].map(tag=>({actualRoots:[{...readRoot,tag}]})), +]) check(beginReadonlyRequestPolicy({...readonlyAction,...changed})); +for (const kind of ['scroll','wait']) { + const cap = beginReadonlyRequestPolicy({...readonlyAction,kind,selector:'body',selectorObserved:false,actualRoots:[{...readRoot,tag:'body'}]}); + check(matchPlatformRequest({...request,policy:cap,url:'https://t2.xiaohongshu.com/api/v2/collect'}),{policy_version:READONLY_POLICY_VERSION,policy_rule:'collect'}); + endPlatformRequestPolicy(cap); +} +for (const kind of ['read','extract']) check(beginReadonlyRequestPolicy({...readonlyAction,kind,selector:'body',actualRoots:[{...readRoot,tag:'body'}]})); +const expiredRead = beginReadonlyRequestPolicy({...readonlyAction,now:Date.now()-16000}); +check(matchPlatformRequest({...request,policy:expiredRead,url:'https://t2.xiaohongshu.com/api/v2/collect'})); + +// A seek within a known search-results root receives only the pagination rule, +// bound to the exact page and the finite action window. These tests make no +// browser, API, model or network calls. +check(SEARCH_SCROLL_POLICY_VERSION, 'xhs-search-scroll-experimental-v1'); +const scrollNow = 10_000; +const scrollPage = `${official}/search_result_ai?keyword=fixture&source=web`; +const scrollObserved = {kind:'read',tag:'div',unique:true,sensitive:false}; +const scrollRoot = {...readRoot,visible:true,in_viewport:true}; +const untilTarget = {selector:'section.note-item a.title',field:'text',value:'准备开始装修了，主打一个听劝！'}; +const scrollAction = {sessionOrigin:official,pageUrl:scrollPage,kind:'scroll',selector:'#search-results',selectorObserved:true, + observed:scrollObserved,actualRoots:[scrollRoot],scrollScope:'viewport',untilTarget,now:scrollNow}; +const scrollPolicy = beginReadonlyRequestPolicy(scrollAction); +check(scrollPolicy, {version:SEARCH_SCROLL_POLICY_VERSION,scope:'search_scroll',expiresAt:25_000}); +check(Object.isFrozen(scrollPolicy), true); +check(Object.keys(scrollPolicy).sort(), ['expiresAt','scope','version']); +const scrollRequest = {policy:scrollPolicy,automating:true,sessionOrigin:official,pageUrl:scrollPage,activeMainFrame:true, + method:'POST',resourceType:'xhr',url:'https://so.xiaohongshu.com/api/sns/web/v2/search/notes',now:scrollNow}; +for (const [pagePath,tag] of [['/search_result','section'],['/search_result_ai','div']]) { + const pageUrl = `${official}${pagePath}?keyword=fixture&source=web`; + const cap = beginReadonlyRequestPolicy({...scrollAction,pageUrl,observed:{...scrollObserved,tag},actualRoots:[{...scrollRoot,tag}]}); + check(cap?.scope, 'search_scroll'); + for (const [rule,url,resourceType] of [ + ['collect','https://t2.xiaohongshu.com/api/v2/collect','xhr'], + ['apm_data','https://apm-fe.xiaohongshu.com/api/data','fetch'], + ['search_notes','https://so.xiaohongshu.com/api/sns/web/v2/search/notes','xhr'], + ]) check(matchPlatformRequest({...scrollRequest,policy:cap,pageUrl,url,resourceType}), + {policy_version:SEARCH_SCROLL_POLICY_VERSION,policy_rule:rule}); + endPlatformRequestPolicy(cap); + check(matchPlatformRequest({...scrollRequest,policy:cap,pageUrl})); +} +const boundedTextPolicy = beginReadonlyRequestPolicy({...scrollAction,untilTarget:{...untilTarget,value:'文'.repeat(2000)}}); +check(boundedTextPolicy?.scope, 'search_scroll'); +endPlatformRequestPolicy(boundedTextPolicy); + +// Invalid new-scope intent can legitimately retain the old readonly UI policy. +// The invariant is that it cannot authorize search pagination. +function noSearchScroll(changed, {fallback = false} = {}) { + const cap = beginReadonlyRequestPolicy({...scrollAction,...changed}); + check(cap?.scope === 'search_scroll', false); + check(matchPlatformRequest({...scrollRequest,policy:cap})); + if (fallback) { + check(cap?.scope, 'readonly_ui'); + check(matchPlatformRequest({...scrollRequest,policy:cap,url:'https://t2.xiaohongshu.com/api/v2/collect'}), + {policy_version:READONLY_POLICY_VERSION,policy_rule:'collect'}); + } + endPlatformRequestPolicy(cap); +} +for (const changed of [ + ...['read','extract','wait'].map(kind=>({kind})), + {scrollScope:undefined}, {scrollScope:'container'}, {scrollScope:'Viewport'}, + {observed:undefined}, {observed:null}, {observed:{...scrollObserved,kind:'scroll'}}, + {observed:{...scrollObserved,kind:'click'}}, {observed:{...scrollObserved,unique:false}}, + {observed:{...scrollObserved,unique:undefined}}, {observed:{...scrollObserved,sensitive:true}}, + {observed:{...scrollObserved,sensitive:undefined}}, {observed:{...scrollObserved,tag:undefined}}, + {observed:{...scrollObserved,tag:''}}, {observed:{...scrollObserved,tag:'DIV'}}, + {observed:{...scrollObserved,tag:'section'}}, + {actualRoots:[{...scrollRoot,visible:false}]}, {actualRoots:[{...scrollRoot,visible:undefined}]}, + {actualRoots:[{...scrollRoot,in_viewport:false}]}, {actualRoots:[{...scrollRoot,in_viewport:undefined}]}, + {actualRoots:[scrollRoot,{...scrollRoot}]}, + {actualRoots:[{...scrollRoot,tag:'section'}]}, + {observed:{...scrollObserved,tag:'a'},actualRoots:[{...scrollRoot,tag:'a'}]}, + {untilTarget:undefined}, {untilTarget:null}, {untilTarget:[]}, {untilTarget:'title'}, + {untilTarget:{...untilTarget,field:'href'}}, {untilTarget:{...untilTarget,field:'attribute'}}, + {untilTarget:{...untilTarget,field:undefined}}, {untilTarget:{...untilTarget,value:''}}, + {untilTarget:{...untilTarget,value:' \n\t '}}, {untilTarget:{...untilTarget,value:undefined}}, + {untilTarget:{...untilTarget,value:42}}, {untilTarget:{...untilTarget,value:'文'.repeat(2001)}}, + {untilTarget:{selector:untilTarget.selector,field:'text',parameter:'note_title'}}, + {untilTarget:{...untilTarget,parameter:'note_title'}}, {untilTarget:{...untilTarget,attribute:'title'}}, + {untilTarget:{...untilTarget,unexpected:true}}, + ...['','body','html',':root','body .note','html > section','section, body','xpath=//a','text=title', + 'role=link','section >> a','a:has-text("title")','javascript:alert(1)'].map(selector=>({untilTarget:{...untilTarget,selector}})), + ...['html',':root','body .note','section, body','xpath=//section','text=results','css=#results', + '#results >> section','section:has-text("results")','javascript:alert(1)'].map(selector=>({selector})), + ...['/explore','/explore/'+ 'a'.repeat(24),'/search_result/','/search_result_ai/', + '/search_result/'+ 'a'.repeat(24),'/search_result_ai/more','/Search_result','/user/profile', + '/search_result_other'].map(path=>({pageUrl:`${official}${path}?keyword=fixture`})), +]) noSearchScroll(changed, {fallback:true}); +for (const changed of [ + ...['click','fill','press'].map(kind=>({kind})), + {sessionOrigin:'http://127.0.0.1'}, {sessionOrigin:'https://xiaohongshu.com'}, + {sessionOrigin:'https://www.xiaohongshu.com.evil.test'}, + {pageUrl:'http://www.xiaohongshu.com/search_result?keyword=fixture'}, + {pageUrl:'https://www.xiaohongshu.com.evil.test/search_result?keyword=fixture'}, + {pageUrl:'https://user:password@www.xiaohongshu.com/search_result?keyword=fixture'}, + {pageUrl:'https://www.xiaohongshu.com:8443/search_result?keyword=fixture'}, {pageUrl:'invalid'}, + {selectorObserved:false}, {selectorObserved:undefined}, {selector:''}, {selector:null}, {selector:'x'.repeat(1001)}, + {actualRoots:[]}, {actualRoots:null}, {actualRoots:[null]}, + {actualRoots:[{...scrollRoot,readonly_root:false}]}, {actualRoots:[{...scrollRoot,sensitive:true}]}, + {actualRoots:[{...scrollRoot,forbidden:true}]}, + ...['input','textarea','select','button','form','script','style','noscript','iframe','body','html'].map(tag=> + ({observed:{...scrollObserved,tag},actualRoots:[{...scrollRoot,tag}]})), + {selector:'body',selectorObserved:false,observed:{...scrollObserved,tag:'body'},actualRoots:[{...scrollRoot,tag:'body'}]}, + {now:NaN}, {now:Infinity}, +]) noSearchScroll(changed); + +for (const changed of [ + {policy:null}, {policy:{}}, {policy:{...scrollPolicy}}, + {policy:Object.freeze({version:SEARCH_SCROLL_POLICY_VERSION,scope:'search_scroll',expiresAt:25_000,pageUrl:scrollPage,startsAt:scrollNow})}, + {automating:false}, {automating:undefined}, {activeMainFrame:false}, {activeMainFrame:undefined}, + {sessionOrigin:'https://xiaohongshu.com'}, {sessionOrigin:'https://www.xiaohongshu.com.evil.test'}, + {pageUrl:`${official}/search_result?keyword=fixture&source=web`}, + {pageUrl:`${official}/search_result_ai?keyword=other&source=web`}, + {pageUrl:`${official}/search_result_ai?source=web&keyword=fixture`}, + {pageUrl:`${scrollPage}#results`}, {pageUrl:`${scrollPage}&page=2`}, + {pageUrl:`${official}/explore?keyword=fixture&source=web`}, + {pageUrl:'https://www.xiaohongshu.com.evil.test/search_result_ai?keyword=fixture&source=web'}, + {pageUrl:'https://user:password@www.xiaohongshu.com/search_result_ai?keyword=fixture&source=web'}, + {pageUrl:'http://www.xiaohongshu.com/search_result_ai?keyword=fixture&source=web'}, + {pageUrl:'https://www.xiaohongshu.com:8443/search_result_ai?keyword=fixture&source=web'}, + {method:'GET'}, {method:'PUT'}, {method:'DELETE'}, {method:'PATCH'}, {method:'post'}, + {resourceType:'fetch'}, {resourceType:'document'}, {resourceType:'other'}, + {url:'https://so.xiaohongshu.com/api/sns/web/v2/search/notes/'}, + {url:'https://so.xiaohongshu.com/api/sns/web/v2/search/notes/delete'}, + {url:'https://so.xiaohongshu.com/api/sns/web/v1/search/notes'}, + {url:'https://edith.xiaohongshu.com/api/sns/web/v2/search/notes'}, + {url:'https://so.xiaohongshu.com.evil.test/api/sns/web/v2/search/notes'}, + {url:'https://user:password@so.xiaohongshu.com/api/sns/web/v2/search/notes'}, + {url:'http://so.xiaohongshu.com/api/sns/web/v2/search/notes'}, + {url:'https://so.xiaohongshu.com:8443/api/sns/web/v2/search/notes'}, + {url:'https://so.xiaohongshu.com/api/sns/web/search/history/sync'}, + {url:'https://edith.xiaohongshu.com/api/sns/web/v1/search/onebox'}, + {url:'https://so.xiaohongshu.com/api/sns/web/v1/dqa/instant'}, + {url:'https://edith.xiaohongshu.com/api/sns/web/v1/feed'}, + {url:'https://edith.xiaohongshu.com/api/sns/v1/history/report_web'}, + {url:'https://t2.xiaohongshu.com/api/v2/collect/post'}, + {url:'https://apm-fe.xiaohongshu.com/api/data',resourceType:'xhr'}, + {url:'https://so.xiaohongshu.com/api/sns/web/v2/comment/post'}, + {url:'https://edith.xiaohongshu.com/api/sns/web/v1/user/follow'}, + {url:'https://edith.xiaohongshu.com/api/sns/web/v1/im/message/send'}, + {url:'https://so.xiaohongshu.com/api/sns/web/v2/unknown'}, + {url:'https://so.xiaohongshu.com/api/security/verify'}, + {url:'javascript:alert(1)'}, {url:'invalid'}, + {now:scrollNow-1}, {now:25_000}, {now:25_001}, {now:NaN}, {now:Infinity}, +]) check(matchPlatformRequest({...scrollRequest,...changed})); +for (const now of [scrollNow,24_999]) check(matchPlatformRequest({...scrollRequest,now}), + {policy_version:SEARCH_SCROLL_POLICY_VERSION,policy_rule:'search_notes'}); + +// Mutating the caller's action after creation cannot move the internal page pin. +const mutableAction = {...scrollAction}; +const pinnedPolicy = beginReadonlyRequestPolicy(mutableAction); +mutableAction.pageUrl = `${official}/search_result_ai?keyword=changed`; +check(matchPlatformRequest({...scrollRequest,policy:pinnedPolicy,pageUrl:mutableAction.pageUrl})); +check(matchPlatformRequest({...scrollRequest,policy:pinnedPolicy}), + {policy_version:SEARCH_SCROLL_POLICY_VERSION,policy_rule:'search_notes'}); +endPlatformRequestPolicy(pinnedPolicy); +check(matchPlatformRequest({...scrollRequest,policy:pinnedPolicy})); +endPlatformRequestPolicy(scrollPolicy); +check(matchPlatformRequest(scrollRequest)); +console.log(JSON.stringify({ result:'passed', assertions, search_rules:7, note_rules:8, readonly_rules:2, search_scroll_rules:3, + scoped_page_binding:true, live_requests:0, localhost_exception:false, expiry_and_revocation:true })); diff --git a/tools/learning/test-request-diagnostics.mjs b/tools/learning/test-request-diagnostics.mjs new file mode 100644 index 0000000..57d8e5b --- /dev/null +++ b/tools/learning/test-request-diagnostics.mjs @@ -0,0 +1,207 @@ +// Request diagnostics regression: official URLs are pure metadata only. +// Every browser navigation and page request uses our own isolated loopback page. +import assert from 'node:assert/strict'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import {createServer} from 'node:http'; +import {fileURLToPath} from 'node:url'; +import {setTimeout as delay} from 'node:timers/promises'; +import {beginSearchRequestPolicy,beginNoteOpenRequestPolicy,beginReadonlyRequestPolicy, + evaluatePlatformRequest,matchPlatformRequest,endPlatformRequestPolicy, + SEARCH_POLICY_VERSION,NOTE_POLICY_VERSION,READONLY_POLICY_VERSION,SEARCH_SCROLL_POLICY_VERSION} from './platform-policy.mjs'; + +const root=path.resolve(path.dirname(fileURLToPath(import.meta.url)),'../..'); +await fs.mkdir(path.join(root,'.tmp'),{recursive:true}); +const output=await fs.mkdtemp(path.join(root,'.tmp','request-diagnostics-')); +process.env.ASSISTANT_LEARNING_HOME=path.join(output,'learning'); +process.env.ASSISTANT_BROWSER_CHANNEL=process.platform==='win32' ? 'msedge' : 'chromium'; +process.env.ASSISTANT_BROWSER_HEADLESS='1'; +const {dispatch,verify}=await import('./browser-worker.mjs'); +const results=[],tasks=new Set(),sockets=new Set(); +let assertions=0,fixtureRequests=0,fixtureWrites=0,fontRequests=0,serial=0; +const eq=(actual,expected,message)=>{assert.deepEqual(actual,expected,message);assertions+=1;}; +const ok=(actual,message)=>{assert.ok(actual,message);assertions+=1;}; +const save=()=>fs.writeFile(path.join(output,'report.json'),JSON.stringify({passed:results.length,assertions,results, + live_platform_requests:0,official_browser_navigation:false,application_api_calls:0,model_calls:0, + fixture_requests:fixtureRequests,fixture_writes:fixtureWrites,font_requests:fontRequests},null,2)); +const passed=async(name,details={})=>{results.push({name,...details});await save();}; + +const official='https://www.xiaohongshu.com',pageUrl=official+'/search_result_ai?keyword=synthetic'; +const now=10_000,originPage=value=>{try{const url=new URL(value);return url.origin===official && !url.username && !url.password;}catch{return false;}}; +// Independent copy of the pre-diagnostic matcher. Rules and authorization +// tracking are test-owned, so this does not use the new evaluator as its oracle. +const baselineRules={ + search:[['collect','https://t2.xiaohongshu.com','/api/v2/collect','xhr'],['search_history','https://so.xiaohongshu.com','/api/sns/web/search/history/sync','xhr'],['search_notes','https://so.xiaohongshu.com','/api/sns/web/v2/search/notes','xhr'],['search_onebox','https://edith.xiaohongshu.com','/api/sns/web/v1/search/onebox','xhr'],['search_dqa','https://so.xiaohongshu.com','/api/sns/web/v1/dqa/instant','xhr'],['apm_data','https://apm-fe.xiaohongshu.com','/api/data','fetch'],['apm_data_xhr','https://apm-fe.xiaohongshu.com','/api/data','xhr']], + note_open:[['collect','https://t2.xiaohongshu.com','/api/v2/collect','xhr'],['note_share_code','https://edith.xiaohongshu.com','/api/sns/web/share/code','xhr'],['note_widgets','https://edith.xiaohongshu.com','/api/sns/web/v2/widgets','xhr'],['note_feed','https://edith.xiaohongshu.com','/api/sns/web/v1/feed','xhr'],['note_seo','https://edith.xiaohongshu.com','/api/sns/web/worldcup/note/seo','xhr'],['note_metrics','https://edith.xiaohongshu.com','/api/sns/web/v1/note/metrics_report','xhr'],['note_history_report','https://edith.xiaohongshu.com','/api/sns/v1/history/report_web','xhr'],['apm_data','https://apm-fe.xiaohongshu.com','/api/data','fetch']], + readonly_ui:[['collect','https://t2.xiaohongshu.com','/api/v2/collect','xhr'],['apm_data','https://apm-fe.xiaohongshu.com','/api/data','fetch']], + search_scroll:[['collect','https://t2.xiaohongshu.com','/api/v2/collect','xhr'],['apm_data','https://apm-fe.xiaohongshu.com','/api/data','fetch'],['search_notes','https://so.xiaohongshu.com','/api/sns/web/v2/search/notes','xhr']], +}; +const issued=new WeakMap(); +const remember=policy=>{ok(policy,'fixture mints an actual issued policy');issued.set(policy,{pageUrl,startsAt:now});return policy;}; +function baselineMatch({policy,automating,sessionOrigin,pageUrl:page,activeMainFrame,method,resourceType,url,now:time}){ + if(!policy || !issued.has(policy) || time>policy.expiresAt || !Number.isFinite(time) || automating!==true || sessionOrigin!==official || !originPage(page) || activeMainFrame!==true || method!=='POST')return null; + if(policy.scope==='search_scroll'){const bound=issued.get(policy);if(page!==bound.pageUrl || time=policy.expiresAt)return null;} + let parsed;try{parsed=new URL(url);}catch{return null;} + if(parsed.protocol!=='https:' || parsed.username || parsed.password)return null; + const rule=(baselineRules[policy.scope] || []).find(([,origin,pathname,type])=>parsed.origin===origin && parsed.pathname===pathname && resourceType===type); + return rule ? {policy_version:policy.version,policy_rule:rule[0]} : null; +} +const input={kind:'fill',tag:'input',unique:true,search_input:true,sensitive:false}; +const inputActual={tag:'input',search_input:true,sensitive:false,visible:true,in_viewport:true,unobscured:true}; +const link=official+'/explore/'+'a'.repeat(24),observedLink={kind:'click',tag:'a',unique:true,sensitive:false}; +const ordinary={tag:'section',readonly_root:true,sensitive:false,forbidden:false,visible:true,in_viewport:true}; +const readContext={sessionOrigin:official,pageUrl,kind:'wait',selector:'#global',selectorObserved:true, + observed:{kind:'read',tag:'section',unique:true,sensitive:false},actualRoots:[ordinary],now}; +const policies=[ + remember(beginSearchRequestPolicy({sessionOrigin:official,pageUrl,kind:'fill',observed:input,actual:inputActual,matchCount:1,now})), + remember(beginNoteOpenRequestPolicy({sessionOrigin:official,pageUrl,kind:'click',observed:observedLink,actual:{...observedLink,visible:true,in_viewport:true,unobscured:true,href:link},matchCount:1,href:link,now})), + remember(beginReadonlyRequestPolicy(readContext)), + remember(beginReadonlyRequestPolicy({...readContext,kind:'scroll',scrollScope:'viewport',untilTarget:{selector:'#global a.title',field:'text',value:'Synthetic title'}})), +]; +const request={policy:policies[2],automating:true,sessionOrigin:official,pageUrl,activeMainFrame:true,method:'POST',resourceType:'xhr',url:'https://t2.xiaohongshu.com/api/v2/collect',now:now+1}; +const deny=(patch,reason,extra={})=>{ + const context={...request,...patch}; + eq(evaluatePlatformRequest(context),{allowed:false,reason,...(context.policy && issued.has(context.policy) ? {policy_version:context.policy.version} : {}),...extra}); + eq(matchPlatformRequest(context),null); +}; + +const server=createServer((req,res)=>{ + fixtureRequests+=1; + if(req.method!=='GET'){fixtureWrites+=1;res.writeHead(405).end();return;} + const url=new URL(req.url,'http://127.0.0.1'); + if(url.pathname==='/pending-font'){fontRequests+=1;return;} + if(url.pathname!=='/fixture'){res.writeHead(404).end();return;} + const mode=url.searchParams.get('mode'); + res.writeHead(200,{'Content-Type':'text/html; charset=utf-8'}); + res.end(`Local request diagnostic fixture

    Already read synthetic detail

    Synthetic result container${['success','final-font'].includes(mode) ? 'Synthetic title oneSynthetic title two' : ''}
    `); +}); +server.on('connection',socket=>{sockets.add(socket);socket.once('close',()=>sockets.delete(socket));}); +let listening=false; +const open=async mode=>{ + const task='request-diagnostics-'+(++serial),url=`http://127.0.0.1:${server.address().port}/fixture?mode=${mode}`; + tasks.add(task); + await dispatch('open',{task_id:task,account_key:'isolated-request-diagnostics',entry_url:url}); + await dispatch('ready',{task_id:task,identity_label:'Local request diagnostic fixture'}); + await dispatch('capture',{task_id:task});return {task,url}; +}; +const close=async task=>{await dispatch('close',{task_id:task});tasks.delete(task);}; +const waiting={kind:'wait',selector:'#global a.title',wait_scope:'#global',state:'visible',min_count:2,timeout_ms:8000}; +const read={kind:'read',selector:'#detail',output_name:'detail'}; +const noPrivate=value=>{const text=JSON.stringify(value);for(const sentinel of ['LOCAL_PRIVATE_QUERY','LOCAL_PRIVATE_BODY','LOCAL_PRIVATE_HEADER'])eq(text.includes(sentinel),false,'diagnostics omit query, body and headers');}; +const png=async capture=>{ok(capture?.screenshot_path,'actual capture exists');const bytes=await fs.readFile(capture.screenshot_path);eq(bytes.subarray(1,4).toString(),'PNG');}; +const unchanged=async value=>{const before=JSON.stringify(value);await delay(180);eq(JSON.stringify(value),before,'returned failure and evidence remain immutable after late callbacks');}; +const blockedSummary=diagnostics=>{ + eq(diagnostics.phase,'execute');eq(diagnostics.blocked_request_count,16);eq(diagnostics.blocked_requests.length,12,'bounded unique diagnostic requests'); + for(const summary of diagnostics.blocked_requests){eq(summary.policy_reason,'missing_policy');eq(summary.active_policy_version,undefined);eq(summary.expected_resource_types,undefined);} + noPrivate(diagnostics); +}; + +try{ + eq(policies.map(policy=>policy.version),[SEARCH_POLICY_VERSION,NOTE_POLICY_VERSION,READONLY_POLICY_VERSION,SEARCH_SCROLL_POLICY_VERSION]); + const endpoints=[...new Set(Object.values(baselineRules).flat().map(([,origin,pathname])=>origin+pathname)), + 'https://as.xiaohongshu.com/api/sec/v1/shield/webprofile','https://example.invalid/unknown-write','http://t2.xiaohongshu.com/api/v2/collect', + 'https://t2.xiaohongshu.com.evil.invalid/api/v2/collect','https://t2.xiaohongshu.com/api/v2/collect/','https://user:pass@t2.xiaohongshu.com/api/v2/collect','invalid']; + let comparisons=0; + for(const policy of policies){ + const allowedRules=new Set(); + for(const url of endpoints)for(const resourceType of ['xhr','fetch','document','other'])for(const method of ['POST','GET','PUT'])for(const time of [now-1,now+1,now+15000,now+15001]){ + const context={...request,policy,url,resourceType,method,now:time}; + const expected=baselineMatch(context);eq(matchPlatformRequest(context),expected,'pre-diagnostic allow behavior is unchanged'); + const decision=evaluatePlatformRequest(context);eq(decision.allowed,expected!==null); + if(expected){eq(decision,{allowed:true,...expected});if(time===now+1)allowedRules.add(decision.policy_rule);} + comparisons+=1; + } + eq([...allowedRules].sort(),baselineRules[policy.scope].map(rule=>rule[0]).sort(),'the exact original scope rule set is retained'); + } + await passed('independent pre-change matcher equivalence across all four scopes',{comparisons,rules:[7,8,2,3]}); + + deny({policy:null},'missing_policy');deny({policy:{...policies[2]}},'inactive_policy'); + deny({now:NaN},'invalid_time');deny({now:now+15001},'policy_expired');deny({automating:false},'automation_inactive'); + deny({sessionOrigin:'https://example.invalid'},'session_out_of_scope');deny({pageUrl:'https://example.invalid'},'page_out_of_scope'); + deny({activeMainFrame:false},'not_main_frame');deny({method:'PUT'},'method_not_allowed'); + deny({policy:policies[3],pageUrl:pageUrl+'&changed=yes'},'page_changed');deny({policy:policies[3],now:now-1},'policy_not_started'); + deny({policy:policies[3],now:now+15000},'policy_expired');deny({url:'not-a-url'},'invalid_url');deny({url:'http://t2.xiaohongshu.com/api/v2/collect'},'invalid_url'); + deny({url:'https://user:pass@t2.xiaohongshu.com/api/v2/collect'},'invalid_url'); + deny({url:'https://as.xiaohongshu.com/api/sec/v1/shield/webprofile'},'request_out_of_scope'); + deny({url:'https://apm-fe.xiaohongshu.com/api/data'},'resource_type_not_allowed',{expected_resource_types:['fetch']}); + const privateRequest={...request,url:request.url+'?token=LOCAL_PRIVATE_QUERY#LOCAL_PRIVATE_QUERY'}; + Object.defineProperties(privateRequest,{headers:{get(){throw new Error('headers must not be inspected');}},body:{get(){throw new Error('body must not be inspected');}}}); + eq(evaluatePlatformRequest(privateRequest),{allowed:true,policy_version:READONLY_POLICY_VERSION,policy_rule:'collect'});noPrivate(evaluatePlatformRequest(privateRequest)); + const revoked=remember(beginReadonlyRequestPolicy(readContext));endPlatformRequestPolicy(revoked);issued.delete(revoked); + deny({policy:revoked},'inactive_policy'); + await passed('all fixed rejection reasons, inactive-policy privacy and private input exclusion'); + + const saved=await fs.readFile(path.join(root,'.tmp','peer-scoped-loading-evidence.json'),'utf8').then(JSON.parse).catch(error=>{if(error.code==='ENOENT')return null;throw error;}); + if(saved){ + const requests=[...new Map(saved.flatMap(item=>item.request_observations || []).filter(item=>item.blocked).map(item=>[item.sequence,item])).values()]; + eq(requests.length,2,'preserved run contains two distinct blocked requests'); + for(const observed of requests){const decision=evaluatePlatformRequest({...request,method:observed.method,resourceType:observed.resource_type,url:observed.origin+observed.pathname}); + eq(decision.allowed,false);eq(decision.reason,observed.resource_type==='xhr' && observed.origin==='https://apm-fe.xiaohongshu.com' ? 'resource_type_not_allowed' : 'request_out_of_scope');} + await passed('preserved actual request metadata remains rejected under unchanged policy',{synthetic_metadata_only:true}); + } + if(process.argv.includes('--pure')){ + await save();console.log(JSON.stringify({passed:results.length,assertions,output,pure_only:true,live_platform_requests:0,application_api_calls:0,model_calls:0})); + }else{ + await new Promise((resolve,reject)=>{server.once('error',reject);server.listen(0,'127.0.0.1',resolve);});listening=true; + const actionFixture=await open('blocked');let actionError; + try{await dispatch('action',{task_id:actionFixture.task,...waiting});}catch(error){actionError=error;} + eq(actionError?.code,'WRITE_REQUEST_BLOCKED');blockedSummary(actionError.diagnostics); + const observations=actionError.after.request_observations.filter(item=>item.blocked); + eq(observations.length,16);for(const observed of observations)eq(observed.policy_reason,'missing_policy'); + for(const capture of [actionError.before,actionError.after])await png(capture); + const actionRecord={code:actionError.code,diagnostics:actionError.diagnostics,before:actionError.before,after:actionError.after}; + noPrivate(actionRecord);await unchanged(actionRecord);await fs.writeFile(path.join(output,'action-blocked.json'),JSON.stringify(actionRecord,null,2));await close(actionFixture.task); + await passed('real loopback action refusal preserves bounded reasons and request observation diagnostics'); + + const failing=await open('blocked'); + const result=await verify({task_id:failing.task,entry_url:failing.url,parameters:{},steps:[read,waiting],expected_outputs:{detail:'Already read synthetic detail'}}); + eq(result.passed,false);eq(result.error_codes,['WRITE_REQUEST_BLOCKED']);eq(result.failure_diagnostics.step_position,2);eq(result.failure_diagnostics.kind,'wait');eq(result.failure_diagnostics.code,'WRITE_REQUEST_BLOCKED'); + blockedSummary(result.failure_diagnostics.diagnostics);ok(result.outputs.detail?.[0]?.text.includes('Already read synthetic detail')); + for(const capture of result.evidence)await png(capture); + noPrivate(result);await unchanged(result);await fs.writeFile(path.join(output,'verify-blocked.json'),JSON.stringify(result,null,2));await close(failing.task); + await passed('verify captures actual failed step, bounded reasons, real evidence and completed prefix'); + + const success=await open('success'); + const completed=await verify({task_id:success.task,entry_url:success.url,parameters:{},steps:[read,waiting],expected_outputs:{detail:'Already read synthetic detail'}}); + eq(completed.passed,true,JSON.stringify(completed.errors));eq(Object.hasOwn(completed,'failure_diagnostics'),false,'success never fabricates a failure diagnostic'); + await fs.writeFile(path.join(output,'verify-success.json'),JSON.stringify(completed,null,2));await close(success.task); + await passed('successful verification has no failure_diagnostics field'); + + for(const mode of ['poll-dom-hang','final-font']){ + const fixture=await open(mode),started=Date.now(); + const cutoff=await verify({task_id:fixture.task,entry_url:fixture.url,parameters:{},steps:[read,waiting],expected_outputs:{detail:'Already read synthetic detail'}},{budgetMs:3500,cleanupReserveMs:500}); + const elapsed=Date.now()-started; + eq(cutoff.passed,false);eq(cutoff.error_codes,['VERIFY_BUDGET']);eq(cutoff.session_closed,true);eq((await dispatch('status',{task_id:fixture.task})).status,'closed'); + eq(cutoff.failure_diagnostics.step_position,2);eq(cutoff.failure_diagnostics.kind,'wait');eq(cutoff.failure_diagnostics.code,'VERIFY_BUDGET'); + eq(cutoff.failure_diagnostics.diagnostics.phase,mode==='final-font' ? 'capture_after' : 'execute'); + ok(elapsed<6000,'diagnostic cloning adds no post-deadline browser wait'); + ok(cutoff.step_timings.at(-1).phase_ms[mode==='final-font' ? 'capture_after' : 'execute']>0); + if(mode==='final-font')ok(fontRequests>0,'real final capture waits for controlled local font'); + for(const capture of cutoff.evidence)await png(capture); + await unchanged(cutoff);await fs.writeFile(path.join(output,mode+'.json'),JSON.stringify(cutoff,null,2));await close(fixture.task); + await passed(mode+' diagnostics remain inside the actual global cutoff',{elapsed_ms:elapsed}); + } + eq(fixtureWrites,0,'no unknown POST reaches our fixture server');await save(); + console.log(JSON.stringify({passed:results.length,assertions,output,live_platform_requests:0,application_api_calls:0,model_calls:0,fixture_writes:fixtureWrites})); + } +}catch(error){ + await save();const report=JSON.parse(await fs.readFile(path.join(output,'report.json'),'utf8')); + report.error={message:error.message,code:error.code,stack:error.stack};await fs.writeFile(path.join(output,'report.json'),JSON.stringify(report,null,2)); + console.error(JSON.stringify({passed:results.length,assertions,output,error:error.message,code:error.code}));process.exitCode=1; +}finally{ + for(const policy of policies)endPlatformRequestPolicy(policy); + await Promise.allSettled([...tasks].map(task=>dispatch('close',{task_id:task}))); + for(const socket of sockets)socket.destroy(); + if(listening)await new Promise(resolve=>server.close(resolve)); +} diff --git a/tools/learning/test-scoped-wait.mjs b/tools/learning/test-scoped-wait.mjs new file mode 100644 index 0000000..ec5f4dd --- /dev/null +++ b/tools/learning/test-scoped-wait.mjs @@ -0,0 +1,250 @@ +// Finite scoped-wait regression. Browser navigation is restricted to our own +// loopback fixture; official-origin cases below use synthetic helper metadata. +import assert from 'node:assert/strict'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import {createServer} from 'node:http'; +import {fileURLToPath} from 'node:url'; +import {setTimeout as delay} from 'node:timers/promises'; +import {READONLY_POLICY_VERSION,endPlatformRequestPolicy,matchPlatformRequest} from './platform-policy.mjs'; + +const root=path.resolve(path.dirname(fileURLToPath(import.meta.url)),'../..'); +await fs.mkdir(path.join(root,'.tmp'),{recursive:true}); +const output=await fs.mkdtemp(path.join(root,'.tmp','scoped-wait-')); +process.env.ASSISTANT_LEARNING_HOME=path.join(output,'learning'); +process.env.ASSISTANT_BROWSER_HEADLESS='1'; +process.env.ASSISTANT_BROWSER_CHANNEL=process.platform==='win32' ? 'msedge' : 'chromium'; +const {dispatch,verify,preflightReadonlyWait,withWaitDeadline,withVerifyDeadline}=await import('./browser-worker.mjs'); +const results=[],tasks=new Set(),pending=new Map(),polled=new Set(),sockets=new Set(); +let serial=0,fixtureRequests=0,fixtureWrites=0,fontRequests=0,assertions=0; +const eq=(actual,expected,message)=>{assert.deepEqual(actual,expected,message);assertions+=1;}; +const ok=(value,message)=>{assert.ok(value,message);assertions+=1;}; +const save=()=>fs.writeFile(path.join(output,'report.json'),JSON.stringify({passed:results.length,assertions,results, + live_platform_requests:0,official_browser_navigation:false,application_api_calls:0,model_calls:0, + fixture_requests:fixtureRequests,fixture_writes:fixtureWrites,font_requests:fontRequests},null,2)); +const passed=async(name,details={})=>{results.push({name,...details});await save();}; + +const server=createServer((request,response)=>{ + fixtureRequests+=1; + const url=new URL(request.url,'http://127.0.0.1'); + if(request.method!=='GET'){fixtureWrites+=1;response.writeHead(405).end();return;} + if(url.pathname==='/pending-font'){fontRequests+=1;return;} + if(url.pathname==='/release'){pending.set(url.searchParams.get('id'),response);return;} + if(url.pathname==='/poll-ready'){polled.add(url.searchParams.get('id'));response.end('observed');return;} + if(url.pathname!=='/fixture'){response.writeHead(404).end();return;} + const id=url.searchParams.get('id'),mode=url.searchParams.get('mode'); + response.writeHead(200,{'Content-Type':'text/html; charset=utf-8'}); + response.end(`Local scoped wait fixture

    Local loading transition

    +${mode==='missing-root' ? '
    Container is not yet observed
    ' : mode==='interactive-root' ? '' : '
    Results pending
    '} +
    Outside the fixed scope
    `); +}); +server.on('connection',socket=>{sockets.add(socket);socket.once('close',()=>sockets.delete(socket));}); +await new Promise(resolve=>server.listen(0,'127.0.0.1',resolve)); +const base=`http://127.0.0.1:${server.address().port}/fixture`; +const open=async mode=>{ + const task=`scoped-wait-${++serial}`,url=`${base}?id=${serial}&mode=${mode}`; + tasks.add(task); + await dispatch('open',{task_id:task,account_key:'isolated-scoped-wait',entry_url:url}); + await dispatch('ready',{task_id:task,identity_label:'Local scoped wait'}); + const before=await dispatch('capture',{task_id:task}); + for(let index=0;!pending.has(String(serial)) && index<100;index+=1)await delay(10); + ok(pending.has(String(serial)),'fixture release is pending before an action'); + return {task,id:String(serial),before,url}; +}; +const close=async task=>{await dispatch('close',{task_id:task});tasks.delete(task);}; +const release=fixture=>{const response=pending.get(fixture.id);ok(response,'release is controlled by the loopback fixture');response.end('release');pending.delete(fixture.id);}; +const png=async capture=>{ok(capture?.screenshot_path,'a real capture path exists');const file=await fs.readFile(capture.screenshot_path);eq(file.subarray(1,4).toString(),'PNG');}; +const waitParams={kind:'wait',wait_scope:'#global',selector:'#global a.title',state:'visible',min_count:2,timeout_ms:3000}; +const action=async(fixture,patch={},beforeAction=false)=>{ + if(beforeAction){release(fixture);await delay(100);} + let settled=false; + const started=Date.now(); + const promise=dispatch('action',{task_id:fixture.task,...waitParams,...patch}); + void promise.then(()=>{settled=true;},()=>{settled=true;}); + if(!beforeAction){ + // Synchronize on the fixture's first actual root-relative query rather + // than a wall-clock guess while capture_before may still be running. + if(Object.hasOwn(patch,'wait_scope') && patch.wait_scope===undefined)await delay(150); + else{ + for(let index=0;!polled.has(fixture.id) && !settled && index<250;index+=1)await delay(10); + ok(polled.has(fixture.id),'the first real scoped collection query has executed'); + } + eq(settled,false,'initial zero rows do not satisfy the wait');release(fixture); + } + try{return {value:await promise,elapsed_ms:Date.now()-started};} + catch(error){return {error,elapsed_ms:Date.now()-started};} +}; +const errorRecord=error=>({code:error.code,message:error.message,session_closed:error.session_closed===true, + ...(error.error_codes ? {error_codes:error.error_codes} : {}),before:error.before,after:error.after,diagnostics:error.diagnostics}); +const rejected=async(mode,code,{patch={},beforeAction=false,closed=false}={})=>{ + const fixture=await open(mode),result=await action(fixture,patch,beforeAction); + if(result.error)await fs.writeFile(path.join(output,mode+'.json'),JSON.stringify({...errorRecord(result.error),elapsed_ms:result.elapsed_ms},null,2)); + ok(result.error,`${mode} must fail`);eq(result.error.code,code,`${mode} code`); + if(closed){eq(result.error.session_closed,true,'only actual context close sets session_closed');eq((await dispatch('status',{task_id:fixture.task})).status,'closed');ok(result.elapsed_ms<2500,'scoped wait cancels instead of waiting for a 10s operation');} + await png(result.error.before); + if(result.error.after)await png(result.error.after); + const record=errorRecord(result.error),saved=JSON.stringify(record); + await delay(150);eq(JSON.stringify(record),saved,'late callbacks do not mutate the returned evidence'); + await fs.writeFile(path.join(output,mode+'.json'),JSON.stringify({...record,elapsed_ms:result.elapsed_ms},null,2)); + await close(fixture.task);return result; +}; + +try{ + let cleanup=0; + eq(await withWaitDeadline(async()=>17,{budgetMs:100,cleanupReserveMs:10,onTimeout:()=>{cleanup+=1;}}),17); + let rejectPending; + await assert.rejects(()=>withWaitDeadline(()=>new Promise((_,reject)=>{rejectPending=reject;}),{budgetMs:60,cleanupReserveMs:10,onTimeout:()=>{cleanup+=1;rejectPending(new Error('cancelled fixture'));}}),error=>error.code==='WAIT_TIMEOUT');assertions+=1; + eq(cleanup,1); + await assert.rejects(()=>withWaitDeadline(async()=>{const until=Date.now()+45;while(Date.now(){cleanup+=1;}}),error=>error.code==='WAIT_TIMEOUT');assertions+=1; + eq(cleanup,2,'late resolution is rejected with exactly one cleanup'); + await assert.rejects(()=>withVerifyDeadline(()=>withWaitDeadline(()=>new Promise(()=>{}),{budgetMs:500,cleanupReserveMs:20}),{budgetMs:50,cleanupReserveMs:10,onTimeout:()=>{}}),error=>error.code==='VERIFY_BUDGET');assertions+=1; + await passed('wait helper normal, cancellation, late resolution and outer budget priority'); + + const official='https://www.xiaohongshu.com',pageUrl=official+'/search_result_ai?keyword=synthetic'; + const rootState={tag:'section',readonly_root:true,sensitive:false,forbidden:false,visible:true,in_viewport:true}; + const context={sessionOrigin:official,pageUrl,kind:'wait',selector:'#global',selectorObserved:true, + observed:{kind:'read',tag:'section',unique:true,sensitive:false},actualRoots:[rootState],now:10000}; + const policy=preflightReadonlyWait(context); + eq(policy?.scope,'readonly_ui');eq(policy?.version,READONLY_POLICY_VERSION); + const request={policy,automating:true,sessionOrigin:official,pageUrl,activeMainFrame:true,method:'POST',resourceType:'xhr',now:10001}; + eq(matchPlatformRequest({...request,url:'https://t2.xiaohongshu.com/api/v2/collect'}),{policy_version:READONLY_POLICY_VERSION,policy_rule:'collect'}); + eq(matchPlatformRequest({...request,url:'https://apm-fe.xiaohongshu.com/api/data',resourceType:'fetch'}),{policy_version:READONLY_POLICY_VERSION,policy_rule:'apm_data'}); + for(const patch of [ + {url:'https://apm-fe.xiaohongshu.com/api/data'}, + {url:'https://as.xiaohongshu.com/api/sec/v1/shield/webprofile'}, + {url:'https://so.xiaohongshu.com/api/sns/web/v2/search/notes'}, + {url:'https://edith.xiaohongshu.com/api/sns/web/v2/comment/post'}, + {url:'https://example.invalid/unknown-write'}, + {url:'https://t2.xiaohongshu.com/api/v2/collect',activeMainFrame:false}, + {url:'https://t2.xiaohongshu.com/api/v2/collect',now:25001}, + ])eq(matchPlatformRequest({...request,...patch}),null); + eq(matchPlatformRequest({...request,policy:{...policy},url:'https://t2.xiaohongshu.com/api/v2/collect'}),null); + endPlatformRequestPolicy(policy); + eq(matchPlatformRequest({...request,url:'https://t2.xiaohongshu.com/api/v2/collect'}),null); + await passed('actual root policy metadata retains only the original two readonly request rules'); + + for(const [mode,selector] of [['normal','#global a.title'],['relative',':scope a.title'],['outside-plus-inside','a.title']]){ + const fixture=await open(mode); + const observed=fixture.before.elements.find(element=>element.selector==='#global'); + eq(observed?.kind,'read');eq(observed?.unique,true); + eq(fixture.before.elements.some(element=>element.selector==='#global a.title'),false,'a zero-item collection is not invented'); + const {value,error}=await action(fixture,{selector}); + ok(!error,`${mode}: ${error?.code} ${error?.message}`);eq(value.result.satisfied,true);eq(value.result.min_count,2); + eq(value.result.wait_scope,'#global'); + ok(value.after.elements.some(element=>element.selector==='#global a.title' && element.count===2),'full final capture establishes the new real collection'); + for(const capture of [value.before,value.after])await png(capture); + const read=await dispatch('action',{task_id:fixture.task,kind:'read',selector:'#global a.title',output_name:'titles'}); + eq(read.result.length,2,'subsequent read consumes the newly observed collection'); + await fs.writeFile(path.join(output,mode+'.json'),JSON.stringify(value,null,2)); + await close(fixture.task); + await passed(mode+' zero-to-two transition and subsequent read',{selector}); + } + for(const mode of ['outside-only','one-inside']){ + const result=await rejected(mode,'WAIT_TIMEOUT',{patch:{selector:'a.title',timeout_ms:1200},closed:true}); + await passed(mode+' outside matches never satisfy min_count',{elapsed_ms:result.elapsed_ms}); + } + for(const mode of ['replace-root','source']){ + await rejected(mode,'SOURCE_CHANGED');await passed(mode+' during wait is pinned and rejected'); + } + await rejected('sensitive-row','SENSITIVE_OUTPUT');await passed('sensitive descendant is rejected'); + await rejected('too-many','READ_LIMIT');await passed('more than 100 scoped rows is rejected'); + for(const mode of ['missing-root','hide-root','interactive-root']){ + await rejected(mode,'UNOBSERVED_CURRENT_SELECTOR',{beforeAction:true});await passed(mode+' cannot become an observed ordinary root'); + } + await rejected('sensitive-root','SENSITIVE_OUTPUT');await passed('root becoming sensitive during wait is rejected'); + const blocked=await rejected('blocked','WRITE_REQUEST_BLOCKED'); + ok(!JSON.stringify(errorRecord(blocked.error)).includes('LOCAL_SCOPED_QUERY')); + ok(!JSON.stringify(errorRecord(blocked.error)).includes('LOCAL_SCOPED_BODY')); + eq(fixtureWrites,0,'unknown POST never reaches the fixture server'); + await passed('unknown POST is blocked, aborts the wait and remains redacted'); + + const oldFixture=await open('normal'); + const old=await action(oldFixture,{wait_scope:undefined}); + ok(!old.error,JSON.stringify(old.error));eq(old.value.result.satisfied,true); + eq(old.value.result.wait_scope,undefined);await png(old.value.after);await close(oldFixture.task); + await passed('existing unscoped loopback demo wait remains compatible'); + + const timeout=await rejected('never','WAIT_TIMEOUT',{patch:{timeout_ms:1200},closed:true}); + await passed('real empty wait reaches its hard deadline and actually closes context',{elapsed_ms:timeout.elapsed_ms}); + + const dom=await rejected('poll-dom-block','WAIT_TIMEOUT',{patch:{timeout_ms:2000},closed:true}); + await passed('in-flight scoped row DOM evaluation is cancelled by the wait deadline',{elapsed_ms:dom.elapsed_ms}); + + // Post-condition evidence uses the existing general screenshot pipeline, + // outside the wait policy. Its separate outer verification budget remains + // authoritative while an actual local font stalls the final screenshot. + const fontFixture=await open('final-font'); + const started=Date.now(); + const fontTimer=setInterval(()=>{if(polled.has(fontFixture.id) && pending.has(fontFixture.id))release(fontFixture);},20); + let font; + try{ + font=await verify({task_id:fontFixture.task,entry_url:fontFixture.url,parameters:{}, + steps:[{...waitParams,timeout_ms:2500},{kind:'read',selector:'#global a.title',output_name:'titles'}], + expected_outputs:{titles:'Synthetic title'}},{budgetMs:4000,cleanupReserveMs:500}); + }finally{clearInterval(fontTimer);} + eq(font.passed,false);eq(font.error_codes,['VERIFY_BUDGET']);eq(font.session_closed,true); + eq((await dispatch('status',{task_id:fontFixture.task})).status,'closed'); + ok(fontRequests>0,'the actual screenshot waits on our controlled local font'); + ok(Date.now()-started<6500,'verification cancels the 10s final screenshot wait'); + ok(font.step_timings.some(timing=>timing.phase_ms.capture_after>0),'the stalled operation is final evidence capture'); + for(const capture of font.evidence)await png(capture); + const stable=JSON.stringify(font);await delay(150);eq(JSON.stringify(font),stable,'late screenshot completion never changes the verification result'); + await fs.writeFile(path.join(output,'final-font.json'),JSON.stringify(font,null,2)); + await close(fontFixture.task); + await passed('final actual screenshot remains covered by the outer verification deadline',{elapsed_ms:Date.now()-started}); + eq(fixtureWrites,0);await save(); + console.log(JSON.stringify({passed:results.length,assertions,output,live_platform_requests:0,model_calls:0,fixture_writes:fixtureWrites})); +}catch(error){ + await save(); + const report=JSON.parse(await fs.readFile(path.join(output,'report.json'),'utf8')); + report.error={message:error.message,code:error.code,stack:error.stack}; + await fs.writeFile(path.join(output,'report.json'),JSON.stringify(report,null,2)); + console.error(JSON.stringify({passed:results.length,assertions,output,error:error.message,code:error.code}));process.exitCode=1; +}finally{ + await Promise.allSettled([...tasks].map(task=>dispatch('close',{task_id:task}))); + for(const response of pending.values())response.end('closed'); + for(const socket of sockets)socket.destroy(); + await new Promise(resolve=>server.close(resolve)); +} diff --git a/tools/learning/test-session-detection.mjs b/tools/learning/test-session-detection.mjs new file mode 100644 index 0000000..7284888 --- /dev/null +++ b/tools/learning/test-session-detection.mjs @@ -0,0 +1,48 @@ +import assert from 'node:assert/strict'; +import { chromium } from 'playwright'; +import { classifyLoginSignals, detectSession, inspectVisibleSessionSignals } from './session-detection.mjs'; + +// New empty browser + synthetic documents only; never any user account/profile. +const browser = await chromium.launch({ headless: true, channel: process.env.ASSISTANT_TEST_BROWSER_CHANNEL || (process.platform === 'win32' ? 'msedge' : 'chromium') }); +const page = await browser.newPage({ viewport: { width: 1280, height: 840 } }); +const navigation = ''; +const results = []; +async function check(name, html, state) { + await page.setContent(`${html}`); + // Reading input values or local session data would make the fixture fail. + await page.evaluate(() => { + Object.defineProperty(HTMLInputElement.prototype, 'value', { get() { throw Error('form value read'); }, configurable: true }); + Object.defineProperty(document, 'cookie', { get() { throw Error('cookie read'); }, configurable: true }); + }); + const signals = await inspectVisibleSessionSignals(page); + assert.equal(classifyLoginSignals(signals).state, state, name); + assert.ok(!JSON.stringify(signals).includes('FakePrivateValue91')); + results.push(name); +} +try { + await check('visible account in main navigation', navigation, 'logged_in'); + await check('public note author is not viewer login', '', 'unknown'); + await check('visible login entry', '', 'logged_out'); + await check('hidden login dialog ignored', navigation + '
    扫码登录
    ', 'logged_in'); + await check('hidden ancestor ignored', navigation + '
    手机号登录
    ', 'logged_in'); + await check('offscreen login ignored', navigation + '
    扫码登录
    ', 'logged_in'); + await check('active login dialog wins over old navigation', navigation + '
    扫码登录
    ', 'logged_out'); + await check('verification widget wins', navigation + '
    请完成安全验证
    ', 'verification_required'); + await check('hidden verification ignored', navigation + '', 'logged_in'); + await check('hidden login text inside visible dialog ignored', navigation + '

    其他说明

    ', 'logged_in'); + await check('hidden challenge text inside visible widget ignored', navigation + '
    ', 'logged_in'); + await check('contradictory entry remains unknown', navigation + '', 'unknown'); + await check('no stored-login inference', '

    欢迎

    ', 'unknown'); + assert.equal((await detectSession(page)).state, 'unknown'); + assert.deepEqual((await detectSession(page)).evidence, ['unsupported_platform']); + // The URL facade tests official-host gating against the same synthetic DOM; + // there is no network navigation to a real platform. + await page.setContent(navigation); + const facade = { url: () => 'https://www.xiaohongshu.com/explore', evaluate: (...args) => page.evaluate(...args) }; + const detected = await detectSession(facade); + assert.equal(detected.state, 'logged_in'); + assert.equal(detected.basis, 'visible_page'); + assert.ok(Number.isFinite(Date.parse(detected.observed_at))); + assert.equal((await detectSession({ ...facade, url: () => 'https://xiaohongshu.com.evil.invalid/' })).state, 'unknown'); + console.log(JSON.stringify({ result: 'passed', visible_page_cases: results.length, host_boundary: true, no_cookies_or_form_values: true, real_platform: false })); +} finally { await browser.close(); } diff --git a/tools/learning/test-target-metadata.mjs b/tools/learning/test-target-metadata.mjs new file mode 100644 index 0000000..9459e52 --- /dev/null +++ b/tools/learning/test-target-metadata.mjs @@ -0,0 +1,73 @@ +import assert from 'node:assert/strict'; +import path from 'node:path'; +import {chromium} from 'playwright'; +import {describePage} from './browser-worker.mjs'; + +export function metadataFixture(url,request,response){ + if(url.pathname!=='/fixture/target-metadata') return false; + response.setHeader('Content-Type','text/html; charset=utf-8'); + response.end(`局部当前对象检查
    第一条内容
    第二条内容
    标题甲标题乙
    不应返回的凭据容器
    ${Array.from({length:500},(_,index)=>'

    与目标无关的已加载文本'+index+'

    ').join('')}`); + return true; +} + +export async function metadataRegression(client,entryUrl,passed,home,channel,semanticChanges){ + const context=await chromium.launchPersistentContext(path.join(home,'target-metadata-isolated'),{channel,headless:true,viewport:{width:1280,height:840}}); + try{ + const page=context.pages()[0] || await context.newPage();await page.goto(new URL('/fixture/target-metadata',entryUrl).href); + const full=await describePage(page); + const keys=['selector','tag','label','kind','unique','count','sensitive','in_viewport','search_input','stable_label','scroll_y']; + const core=value=>Object.fromEntries(keys.filter(key=>Object.hasOwn(value,key)).map(key=>[key,value[key]])); + await page.evaluate(()=>{ + const original=document.querySelectorAll; + document.querySelectorAll=function(selector){ + if(['body *','input,textarea,button,a,[data-testid],[data-learning-read],main,article,[role=main],[role=button]','body [id],h1,h2,h3,h4,h5,h6,section,[role=heading],[role=region],[role=list],[role=listitem]'].includes(selector)) throw Error('FULL_PAGE_DISCOVERY_FORBIDDEN'); + return original.call(this,selector); + }; + }); + for(const selector of ['#metadata-search','#metadata-root','#metadata-scroll','#metadata-root a.title']){ + const observed=full.elements.find(element=>element.selector===selector);assert.ok(observed,selector+' must be observed by normal full capture'); + const current=await describePage(page,{selector,collection:observed.collection===true,requireCanonical:!observed.collection}); + assert.deepEqual(core(current.elements[0]),core(observed),selector+' keeps identical metadata without global discovery'); + } + assert.deepEqual((await describePage(page,{selector:'#not-present'})).elements,[]); + assert.deepEqual((await describePage(page,{selector:'#access-token'})).elements,[]); + await page.evaluate(()=>document.querySelector('#metadata-search').placeholder='另一个搜索示例'); + const rotated=(await describePage(page,{selector:'#metadata-search',requireCanonical:true})).elements[0]; + assert.equal(rotated.stable_label,full.elements.find(element=>element.selector==='#metadata-search').stable_label); + await page.evaluate(()=>document.querySelector('#metadata-search').setAttribute('data-testid','new-search-target')); + assert.deepEqual((await describePage(page,{selector:'#metadata-search',requireCanonical:true})).elements,[],'canonical locator changes remain unobserved'); + await page.evaluate(()=>{const root=document.querySelector('#metadata-root');root.setAttribute('aria-label','secret credential');}); + assert.deepEqual((await describePage(page,{selector:'#metadata-root',requireCanonical:true})).elements,[],'newly sensitive roots cannot retain prior ordinary metadata'); + }finally{await context.close();} + passed('已观察selector局部元数据与整页原构造一致，禁止全页枚举仍可检查；固定搜索语义/普通滚动根/集合/敏感与canonical变化保持'); + + const semanticUrl=new URL('/fixture/search-semantics',entryUrl).href; + for(const mode of ['placeholder','aria','intent','missing']){ + const task='narrow-current-'+mode; + await client.call('open',{task_id:task,account_key:'test-account',entry_url:semanticUrl+'?mode='+mode}); + await client.call('ready',{task_id:task,identity_label:'局部元数据搜索回归'});await client.call('capture',{task_id:task}); + if(mode==='placeholder'){ + await new Promise(resolve=>setTimeout(resolve,160)); + await client.call('action',{task_id:task,kind:'fill',selector:'#semantic-search',value:'装修'}); + const entered=await client.call('action',{task_id:task,kind:'press',selector:'#semantic-search',key:'Enter'});assert.ok(entered.after.text.includes('装修 的结果')); + }else{ + const response=semanticChanges.get(mode);assert.ok(response);response.writeHead(200,{'Content-Type':'application/json'}).end('{}');await new Promise(resolve=>setTimeout(resolve,160)); + await assert.rejects(()=>client.call('action',{task_id:task,kind:'fill',selector:'#semantic-search',value:'不可执行'}),error=>error.code===(mode==='missing' ? 'UNOBSERVED_CURRENT_SELECTOR' : 'STALE_TARGET')); + } + await client.call('close',{task_id:task}); + } + const delayedTask='narrow-zero-visible-wait',delayed=new URL('/fixture/delayed-collection',entryUrl).href; + await client.call('open',{task_id:delayedTask,account_key:'test-account',entry_url:delayed});await client.call('ready',{task_id:delayedTask,identity_label:'局部零集合等待'}); + const waiting=await client.call('verify',{task_id:delayedTask,entry_url:delayed,parameters:{},steps:[{kind:'wait',selector:'#delayed-titles a.title',state:'visible',min_count:1,timeout_ms:5000},{kind:'read',selector:'#delayed-titles',output_name:'titles'}],expected_outputs:{titles:'标题甲'}}); + assert.equal(waiting.passed,true,JSON.stringify(waiting.errors));await client.call('close',{task_id:delayedTask}); + passed('局部检查后placeholder仍可fill/Enter，aria/意图/canonical变化仍拒绝；初始0项集合由原wait轮询实际等待出现'); + + const batchTask='narrow-current-batch',batchUrl=new URL('/fixture/batch-read',entryUrl).href; + await client.call('open',{task_id:batchTask,account_key:'test-account',entry_url:batchUrl});await client.call('ready',{task_id:batchTask,identity_label:'局部元数据批次根'}); + const extraction={kind:'extract',selector:'#batch-container',row_selector:'.batch-row',output_name:'comments',complete_only:true,limit:5,fields:[{name:'comment_id',source:'attribute',attribute:'data-item-id',required:true},{name:'author',selector:'.batch-author',source:'text',required:true},{name:'body',selector:'.batch-body',source:'text',required:true}],batch:{scroll_selector:'#batch-window',pixels:160,max_batches:3,dedupe_field:'comment_id'}}; + const normal=await client.call('verify',{task_id:batchTask,entry_url:batchUrl,parameters:{},steps:[extraction],expected_outputs:{comments:{state:'nonempty'}}});assert.equal(normal.passed,true,JSON.stringify(normal.errors));assert.ok(normal.outputs.comments.count>=3); + const replaced=await client.call('verify',{task_id:batchTask,entry_url:batchUrl+'?mode=root',parameters:{},steps:[extraction],expected_outputs:{comments:{state:'nonempty'}}}); + assert.equal(replaced.passed,false);assert.ok(replaced.error_codes.includes('SOURCE_CHANGED') || replaced.error_codes.includes('STALE_TARGET'));assert.equal(replaced.outputs.comments,undefined);assert.ok(replaced.evidence.length>=3); + await client.call('close',{task_id:batchTask}); + passed('滚动selector独立局部复核保留readonly根检查与真实批次截图；固定容器替换继续中断并保留已得证据'); +} diff --git a/tools/learning/test-viewport-seek.mjs b/tools/learning/test-viewport-seek.mjs new file mode 100644 index 0000000..820d0c1 --- /dev/null +++ b/tools/learning/test-viewport-seek.mjs @@ -0,0 +1,104 @@ +import assert from 'node:assert/strict'; + +// All pages and mutations here are isolated fixtures. The ordinary fixed root +// deliberately has no scrolling overflow; only the document viewport can move. +export function viewportFixture(url,request,response){ + if(url.pathname!=='/fixture/viewport-seek') return false; + const mode=url.searchParams.get('mode') || 'offscreen'; + const initialTarget=['visible','obscured'].includes(mode) ? 80 : 1100; + const anchor=(top=initialTarget)=>`目标笔记`; + const noTarget=['missing','time','lazy'].includes(mode); + const change=mode==='source' ? `history.replaceState({},'',location.pathname+'?changed=1')` : + mode==='root' ? `root.replaceWith(root.cloneNode(true))` : + mode==='scroller' ? `Object.defineProperty(document,'scrollingElement',{configurable:true,get:()=>document.body})` : + mode==='disappear' ? `root.querySelector('.viewport-title').remove()` : + mode==='blocked' ? `fetch('/fixture/write/viewport-seek',{method:'POST'}).catch(()=>{})` : + mode==='lazy' ? `setTimeout(()=>root.insertAdjacentHTML('beforeend',${JSON.stringify(anchor(500))}),250)` : ''; + response.setHeader('Content-Type','text/html; charset=utf-8'); + response.end(`固定根内的视口有界定位

    实际视口定位样本

    普通已加载内容，根容器不独立滚动

    ${noTarget ? '' : anchor()}${mode==='duplicate' ? anchor(1140) : ''}
    目标笔记
    ${mode==='obscured' ? '
    遮挡目标
    ' : ''}`); + return true; +} + +function checkSummary(result,{passed,reason,code,movement=false,zero=false}){ + assert.equal(result.passed,passed,JSON.stringify(result.errors)); + assert.deepEqual(result.outputs,{status:[{text:'实际视口定位样本'}]},'the real read prefix is retained even when the later seek fails'); + assert.equal(result.seek_results.length,1); + const summary=result.seek_results[0]; + assert.deepEqual(Object.keys(summary).sort(),['step_position','scrolls','movements','target_reached','stop_reason'].sort()); + assert.equal(summary.step_position,2);assert.equal(summary.target_reached,passed);assert.equal(summary.stop_reason,reason); + assert.equal(summary.scrolls,summary.movements.length); + assert.ok(summary.movements.every(move=>Object.keys(move).sort().join(',')==='after,before,client_height,scroll_height' && + Object.values(move).every(Number.isFinite) && move.before>=0 && move.after>=move.before && move.client_height>180 && move.scroll_height>move.client_height)); + for(let index=1;index0);assert.ok(summary.movements.some(move=>move.after>move.before));} + if(zero){assert.equal(summary.scrolls,0);assert.deepEqual(summary.movements,[]);} + assert.equal(result.verify_budget_ms,undefined,'explicit viewport scope does not expand the ordinary verify budget'); + assert.deepEqual(result.error_codes,code ? [code] : [],JSON.stringify(result.errors)); + assert.equal(result.step_timings[0].status,'passed'); + assert.equal(result.step_timings[1].status,passed ? 'passed' : 'failed'); + assert.ok(result.evidence.length>=(passed ? 3 : 4),'real entry, prefix and final/failed captures are retained'); + return summary; +} + +export async function viewportRegression(client,entryUrl,passed){ + const task='viewport-seek-regression',base=new URL('/fixture/viewport-seek',entryUrl).href; + await client.call('open',{task_id:task,account_key:'test-account',entry_url:base}); + await client.call('ready',{task_id:task,identity_label:'隔离视口定位测试'}); + const seek={kind:'scroll',selector:'#viewport-root',scroll_scope:'viewport',direction:'down',pixels:320,max_scrolls:5,timeout_ms:3000, + until_target:{selector:'#viewport-global a.viewport-title',field:'text',value:'目标笔记'}}; + // A required output is read before seeking, since after a genuine document + // movement this first-screen status is intentionally no longer in view. + const run=(mode,extra={})=>client.call('verify',{task_id:task,entry_url:base+'?mode='+mode,parameters:{}, + steps:[{kind:'read',selector:'#viewport-status',output_name:'status'},{...seek,...extra}],expected_outputs:{status:'实际视口定位样本'}}); + try{ + const actual=await run('offscreen'); + const summary=checkSummary(actual,{passed:true,reason:'target_reached',movement:true}); + assert.ok(summary.scrolls<=seek.max_scrolls); + assert.equal(summary.movements[0].before,0);assert.equal(summary.movements[0].after,seek.pixels); + assert.ok(summary.movements.every(move=>move.after-move.before<=seek.pixels),'fixed viewport movements cannot jump directly to a target'); + const final=actual.evidence.at(-1); + assert.equal(final.viewport.scroll_y,summary.movements.at(-1).after,'evidence records the same actual window displacement'); + assert.equal(final.elements.find(element=>element.selector==='#viewport-root')?.scroll_y,undefined,'ordinary root never becomes an implicit inner scroll container'); + checkSummary(await run('visible'),{passed:true,reason:'target_reached',zero:true}); + checkSummary(await run('offscreen',{scroll_scope:undefined}),{passed:false,code:'UNOBSERVED_CURRENT_SELECTOR',reason:'target_changed',zero:true}); + passed('显式viewport在固定普通根内匹配并实际固定步长滚document，证据与真实metrics一致；已可见零移动，旧未声明scope仍拒绝非滚动根'); + + for(const [mode,code,reason,extra] of [ + ['missing','SCROLL_BUDGET_EXHAUSTED','scroll_budget',{max_scrolls:1}], + ['missing','SCROLL_BOUNDARY_BEFORE_TARGET','boundary',{pixels:1600,max_scrolls:3}], + ['disappear','SCROLL_BUDGET_EXHAUSTED','scroll_budget',{max_scrolls:1}], + ['time','SCROLL_TARGET_TIMEOUT','time_budget',{timeout_ms:1}], + ]){ + const failure=await run(mode,extra); + const item=checkSummary(failure,{passed:false,code,reason,movement:['missing','disappear'].includes(mode)}); + if(reason==='boundary') assert.equal(item.movements.at(-1).after,item.movements.at(-1).before,'boundary summary records the real no-movement attempt'); + } + const lazy=await run('lazy',{max_scrolls:1,timeout_ms:2000}); + const lazySummary=checkSummary(lazy,{passed:true,reason:'target_reached',movement:true}); + assert.equal(lazySummary.scrolls,1); + passed('视口目标缺失或消失不接受根外可见同名链接，滚动/边界/时间预算分别停止；短等允许根内自然出现且不重复滚动'); + + for(const [mode,code,reason,movement] of [ + ['duplicate','NON_UNIQUE_TARGET','ambiguous',false], + ['obscured','TARGET_OBSCURED','obscured',false], + ['source','SCROLL_SOURCE_CHANGED','source_changed',true], + ['root','STALE_TARGET','target_changed',true], + ['scroller','STALE_TARGET','target_changed',true], + ['blocked','WRITE_REQUEST_BLOCKED','blocked_request',true], + ]){ + const failure=await run(mode); + checkSummary(failure,{passed:false,code,reason,movement,zero:!movement}); + } + passed('视口重名/遮挡不滚动，来源/固定根/实际scrollingElement身份变化和未知POST中断；失败保留真实阅读前缀、位移与截图'); + }finally{ + await client.call('close',{task_id:task}); + } +} diff --git a/tools/learning/test-wait-preflight.mjs b/tools/learning/test-wait-preflight.mjs new file mode 100644 index 0000000..46cafd8 --- /dev/null +++ b/tools/learning/test-wait-preflight.mjs @@ -0,0 +1,142 @@ +import assert from 'node:assert/strict'; +import fs from 'node:fs/promises'; +import path from 'node:path'; +import { createServer } from 'node:http'; +import { fileURLToPath } from 'node:url'; +import { setTimeout as delay } from 'node:timers/promises'; +import { READONLY_POLICY_VERSION, endPlatformRequestPolicy, matchPlatformRequest } from './platform-policy.mjs'; + +// Official-origin cases are pure helper calls with synthetic metadata. The only +// browser entry below is a new isolated profile on our own loopback HTTP page. +const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..'); +await fs.mkdir(path.join(root,'.tmp'),{recursive:true}); +const output = await fs.mkdtemp(path.join(root, '.tmp', 'wait-preflight-')); +process.env.ASSISTANT_LEARNING_HOME = path.join(output, 'learning'); +process.env.ASSISTANT_BROWSER_HEADLESS = '1'; +process.env.ASSISTANT_BROWSER_CHANNEL = process.platform === 'win32' ? 'msedge' : 'chromium'; +const { preflightReadonlyWait, dispatch } = await import('./browser-worker.mjs'); + +const official = 'https://www.xiaohongshu.com'; +const pageUrl = official + '/search_result_ai?keyword=fixture'; +const ordinary = { tag:'a', readonly_root:true, sensitive:false, forbidden:false, visible:true, in_viewport:true }; +const base = { sessionOrigin:official, pageUrl, kind:'wait', selector:'#global a.title', selectorObserved:true, + observed:{kind:'click',tag:'a',unique:false,sensitive:false}, actualRoots:[{...ordinary},{...ordinary}], now:10000 }; +const request = { automating:true, sessionOrigin:official, pageUrl, activeMainFrame:true, method:'POST', + resourceType:'xhr', url:'https://t2.xiaohongshu.com/api/v2/collect', now:10001 }; +let assertions = 0; +let officialCases = 0; +const equal = (actual, expected, message) => { assert.deepEqual(actual, expected, message); assertions += 1; }; +const allowed = (policy, rule) => ({policy_version:READONLY_POLICY_VERSION,policy_rule:rule}); +function rejected(patch, code = 'READONLY_WAIT_NOT_READY') { + let returned; + const context = {...base,...patch}; + const before = JSON.stringify(context); + assert.throws(() => { returned = preflightReadonlyWait(context); }, error => error.code === code); assertions += 1; + equal(returned, undefined, 'preflight failure never returns an active capability'); + equal(JSON.stringify(context), before, 'preflight does not mutate caller state or start an action'); + officialCases += 1; +} + +// Empty current collection and a visible but unobserved collection are two +// distinct missing preconditions. Neither may borrow the previous search scope. +for (const patch of [ + {actualRoots:[]}, {actualRoots:[],selectorObserved:false}, + {selectorObserved:false}, {selectorObserved:undefined}, {selectorObserved:null}, + {actualRoots:null}, {actualRoots:undefined}, {actualRoots:[null]}, + {actualRoots:[{...ordinary,readonly_root:false}]}, + {actualRoots:[{...ordinary,readonly_root:undefined}]}, + {selector:''}, {selector:'body',actualRoots:[ordinary]}, + {pageUrl:'https://example.invalid/search_result'}, + {actualRoots:[{...ordinary,tag:'input'}]}, +]) rejected(patch); +for (const patch of [ + {actualRoots:[{...ordinary,sensitive:true}]}, + {actualRoots:[{...ordinary,forbidden:true}]}, + {actualRoots:[ordinary,{...ordinary,sensitive:true}]}, + {actualRoots:[{...ordinary,sensitive:true}],selectorObserved:false}, +]) rejected(patch,'SENSITIVE_OUTPUT'); + +const unchanged = JSON.stringify(base); +const policy = preflightReadonlyWait(base); officialCases += 1; +equal(policy?.scope,'readonly_ui'); +equal(policy?.version,READONLY_POLICY_VERSION); +equal(JSON.stringify(base),unchanged); +equal(matchPlatformRequest({...request,policy}),allowed(policy,'collect')); +equal(matchPlatformRequest({...request,policy,url:'https://apm-fe.xiaohongshu.com/api/data',resourceType:'fetch'}),allowed(policy,'apm_data')); +for (const patch of [ + {url:'https://apm-fe.xiaohongshu.com/api/data',resourceType:'xhr'}, + {url:'https://as.xiaohongshu.com/api/sec/v1/shield/webprofile'}, + {url:'https://so.xiaohongshu.com/api/sns/web/v2/search/notes'}, + {url:'https://example.invalid/unknown-write'}, + {url:'https://edith.xiaohongshu.com/api/sns/web/v2/comment/post'}, + {activeMainFrame:false}, {method:'PUT'}, {now:25001}, +]) equal(matchPlatformRequest({...request,policy,...patch}),null); +endPlatformRequestPolicy(policy); +equal(matchPlatformRequest({...request,policy}),null,'returned wait policy is revoked with the existing end path'); +equal(matchPlatformRequest({...request,policy:{...policy}}),null,'a copied policy is never an issued capability'); + +const bodyPolicy = preflightReadonlyWait({...base,selector:'body',selectorObserved:false,actualRoots:[{...ordinary,tag:'body'}]}); officialCases += 1; +equal(bodyPolicy?.scope,'readonly_ui','only the existing wait body exception is retained'); +equal(matchPlatformRequest({...request,policy:bodyPolicy}),allowed(bodyPolicy,'collect')); +endPlatformRequestPolicy(bodyPolicy); +equal(matchPlatformRequest({...request,policy:bodyPolicy}),null); +for (const kind of ['fill','press','click','read','extract','scroll']) equal(preflightReadonlyWait({...base,kind}),null); +for (const sessionOrigin of ['http://127.0.0.1:8765','https://example.invalid','https://www.xiaohongshu.com.evil.invalid']) { + equal(preflightReadonlyWait({...base,sessionOrigin,actualRoots:[],selectorObserved:false}),null,'non-official behavior is unchanged'); +} + +let pendingRelease; +let fixtureRequests = 0; +let fixtureWrites = 0; +const server = createServer((req,res) => { + fixtureRequests += 1; + if (req.method !== 'GET') { fixtureWrites += 1; res.writeHead(405).end(); return; } + const url = new URL(req.url,'http://127.0.0.1'); + if (url.pathname === '/release') { pendingRelease = res; return; } + if (url.pathname !== '/') { res.writeHead(404).end(); return; } + res.writeHead(200,{'Content-Type':'text/html; charset=utf-8'}); + res.end(`隔离等待前置回归

    本机等待样本

    结果稍后出现
    `); +}); +let opened = false; +let listening = false; +try { + await new Promise((resolve,reject) => { server.once('error',reject); server.listen(0,'127.0.0.1',resolve); }); listening = true; + const entry = `http://127.0.0.1:${server.address().port}/`; + const taskId = 'wait-preflight-demo'; + await dispatch('open',{task_id:taskId,account_key:'isolated-wait-preflight',entry_url:entry}); opened = true; + await dispatch('ready',{task_id:taskId,identity_label:'隔离等待测试'}); + const before = await dispatch('capture',{task_id:taskId}); + equal(before.elements.some(element=>element.selector==='#global a.title'),false,'latest capture does not invent a zero-item collection'); + assert.ok(before.elements.some(element=>element.selector==='#global')); assertions += 1; + for (let index=0;!pendingRelease && index<100;index+=1) await delay(10); + assert.ok(pendingRelease,'fixture owns the pending loopback GET before testing the wait'); assertions += 1; + let settled = false; + const waiting = dispatch('action',{task_id:taskId,kind:'wait',selector:'#global a.title',state:'visible',min_count:2,timeout_ms:4000}); + void waiting.then(()=>{settled=true;},()=>{settled=true;}); + await delay(300); + equal(settled,false,'unchanged demo wait does not fail on an initially missing collection'); + pendingRelease.writeHead(200,{'Content-Type':'text/plain'}).end('release'); pendingRelease = null; + const result = await waiting; + equal(result.result.satisfied,true); + equal(result.result.min_count,2); + assert.ok(result.after.elements.some(element=>element.selector==='#global a.title' && element.count===2)); assertions += 1; + for (const evidence of [result.before,result.after]) { + const png = await fs.readFile(evidence.screenshot_path); equal(png.subarray(1,4).toString(),'PNG'); + } + equal(result.after.request_observations.filter(item=>item.blocked).length,0); + equal(fixtureWrites,0); + const summary = {result:'passed',assertions,official_preflight_cases:officialCases,official_case_environment:'pure helper only', + official_browser_navigation:false,live_platform_requests:0,application_api_calls:0,model_calls:0, + empty_or_unobserved_precondition_rejected:true,sensitive_precondition_distinct:true,old_readonly_rules_only:true, + copied_or_ended_policy_rejected:true,caller_state_unchanged:true, + loopback_edge_demo_wait_passed:true,initially_unobserved_collection:true,actual_pngs:true, + fixture_requests:fixtureRequests,fixture_writes:fixtureWrites,production_official_wiring:'reviewed separately; not official browser end-to-end'}; + await fs.writeFile(path.join(output,'summary.json'),JSON.stringify(summary,null,2)); + console.log(JSON.stringify({...summary,output})); +} finally { + if (opened) await dispatch('close',{task_id:'wait-preflight-demo'}).catch(()=>{}); + if (pendingRelease) pendingRelease.end('closed'); + if (listening) await new Promise(resolve=>server.close(resolve)); +}