import { createHash, randomUUID } from 'node:crypto'; import fs from 'node:fs/promises'; import path from 'node:path'; import readline from 'node:readline'; import process from 'node:process'; import { pathToFileURL } from 'node:url'; import { ACTION_KINDS, ActionInputError, validateAction, validateSelector } from './action-contract.mjs'; import { detectSession } from './session-detection.mjs'; import { beginSearchRequestPolicy, beginNoteOpenRequestPolicy, beginReadonlyRequestPolicy, beginNativeUiRequestPolicy, endPlatformRequestPolicy, evaluatePlatformRequest, NAVIGATION_UI_POLICY_VERSION } from './platform-policy.mjs'; import {NAVIGATION_UI_PROFILE,isNativeUiProfile,validNavigationPage,navigationPopupMatches,resolveExecutionProfile,executionMetadata,executionContractMatches,validNativePage,assertNativeUiAction,assertNativeUiResultLocation,nativeSearchKeyword,nativeReadRootsValid} from './execution-profile.mjs'; const HOME = path.resolve(process.env.ASSISTANT_LEARNING_HOME || '.runtime/learning'); const CHANNEL = process.env.ASSISTANT_BROWSER_CHANNEL || 'chromium'; const HEADLESS = process.env.ASSISTANT_BROWSER_HEADLESS === '1'; const TIMEOUT = 8_000; const MAX_STEPS = 30; const MAX_BLOCKED_REQUESTS = 12; const sessions = new Map(); const contexts = new Set(); const accountOwners = new Map(); const closingAccounts = new Map(); let chromiumPromise; let shuttingDown = false; let shutdownPromise; export class WorkerError extends Error { constructor(message, code = 'WORKER_ERROR') { super(message); this.code = code; } } const TIMING_PHASES=['validate','inspect_current','capture_before','resolve_target','execute','settle','request_guard','capture_after','failure_capture','batch_capture']; function actionTrace(){ const start=performance.now(),costs=Object.fromEntries(TIMING_PHASES.map(phase=>[phase,0])); const visited=new Set(['validate']); let phase='validate',last=start; const trace={before:null,after:null,snapshot(position,kind,status){ const now=performance.now(),values={...costs};values[phase]+=now-last; const duration=Math.max(0,Math.round(now-start)),rounded=Object.fromEntries([...visited].map(key=>[key,Math.max(0,Math.floor(values[key]))])); rounded[phase]+=duration-Object.values(rounded).reduce((sum,value)=>sum+value,0); return {step_position:position,kind,status,duration_ms:duration,phase_ms:rounded}; }}; Object.defineProperty(trace,'phase',{get:()=>phase,set:value=>{ if(!TIMING_PHASES.includes(value)) throw new WorkerError('未知的受限执行计时阶段','INVALID_TIMING_PHASE'); const now=performance.now();costs[phase]+=now-last;last=now;phase=value;visited.add(value); }}); return trace; } function requireString(value, name, max = 2_000) { if (typeof value !== 'string' || !value.trim() || value.length > max) { throw new WorkerError(`${name} 必须是非空文本,最多 ${max} 字符`, 'INVALID_INPUT'); } return value.trim(); } function httpUrl(value) { let url; try { url = new URL(requireString(value, 'entry_url', 4_096)); } catch { throw new WorkerError('仅支持明确的 http 或 https 页面地址', 'INVALID_URL'); } if (!['http:', 'https:'].includes(url.protocol) || url.username || url.password) { throw new WorkerError('仅支持不含账号密码的 http 或 https 地址', 'INVALID_URL'); } return url; } const digest = value => createHash('sha256').update(value).digest('hex'); const redactText = value => String(value || '') .replace(/(?:\+?86[-\s]?)?1[3-9]\d{9}(?!\d)/g, '[手机号已隐藏]') .replace(/((?:验证码|校验码|动态码|短信码|password|verification\s*code)\s*[::]?\s*)[\w-]{4,12}/gi, '$1[已隐藏]'); export function redactUrl(value) { try { const url = new URL(value); for (const key of [...url.searchParams.keys()]) { if (/password|passwd|phone|mobile|captcha|verification|(?:^|_)code$|token|secret|session|credential/i.test(key)) { url.searchParams.set(key, '[已隐藏]'); } } if (/token|password|secret|session|credential/i.test(url.hash)) url.hash = '[已隐藏]'; // A public note ID is an opaque hexadecimal identifier, even when it // happens to contain eleven digits matching the phone-number pattern. // Preserve only the verified official note path; all other URL parts keep // the existing sensitive-key and ordinary-text redaction rules. if(url.origin==='https://www.xiaohongshu.com' && !url.username && !url.password && /^\/(?:(?:explore|search_result)\/[a-fA-F0-9]{24}|user\/profile\/[a-fA-F0-9]{24})$/.test(url.pathname)) { return url.origin+url.pathname+redactText(url.search+url.hash); } return redactText(url.href); } catch { return redactText(value); } } async function chromium() { if (!chromiumPromise) chromiumPromise = import('playwright').then(module => module.chromium).catch(() => { throw new WorkerError('缺少 Playwright 依赖,请先在项目中安装依赖', 'MISSING_DEPENDENCY'); }); return chromiumPromise; } function sessionFor(taskId, requireReady = true) { const key = requireString(taskId, 'task_id', 128); const session = sessions.get(key); if (!session || session.closed || session.page.isClosed()) { throw new WorkerError('浏览器会话已关闭,请重新打开并人工确认当前页面', 'SESSION_CLOSED'); } if (session.blockedNavigation) { throw new WorkerError('页面尝试离开本次学习的站点范围,请关闭会话后检查入口', 'SCOPE_BLOCKED'); } if (session.popupBlocked) throw new WorkerError('出现未知或站点范围外的弹窗,已关闭并暂停,请维护者核对', 'POPUP_BLOCKED'); if (requireReady && !session.ready) { throw new WorkerError('请先在官方页面完成登录,再明确开始观察;登录阶段不采集证据', 'AWAITING_USER'); } if(requireReady && isNativeUiProfile(session.executionProfile)) requireOrigin(session,session.page.url()); return session; } function requireOrigin(session, url) { let parsed; try { parsed = new URL(url); } catch { throw new WorkerError('当前页面地址不可核验', 'SCOPE_BLOCKED'); } if (parsed.origin !== session.origin) { throw new WorkerError('当前页面不属于本次学习允许的站点', 'SCOPE_BLOCKED'); } if(isNativeUiProfile(session.executionProfile) && !(session.executionProfile===NAVIGATION_UI_PROFILE ? validNavigationPage(url) : validNativePage(url))) throw new WorkerError('当前页面不在此执行模式声明的搜索或笔记范围','SCOPE_BLOCKED'); } function boundNoteSource(value, origin) { try { const url=new URL(value),match=/^\/(?:explore|search_result)\/([a-fA-F0-9]{24})$/.exec(url.pathname); return url.origin===origin && !url.username && !url.password && match ? {note_id:match[1].toLowerCase(),source_url:url.origin+url.pathname} : null; }catch{return null;} } function requireBoundNotePage(session,binding) { if(!binding?.opened) return; const actual=boundNoteSource(session.page.url(),session.origin); if(!actual || actual.note_id!==binding.note_id){ binding.url_checked=false; throw new WorkerError('当前笔记来源与本次已选笔记绑定不一致,停止读取','NOTE_BINDING_MISMATCH'); } binding.url_checked=true; binding.source_url=actual.source_url; } function makeNoteBinding(value,entry,steps,parameters) { if(!value || typeof value!=='object' || Array.isArray(value) || Object.keys(value).length!==1 || !Object.hasOwn(value,'note_id') || typeof value.note_id!=='string' || !/^[a-fA-F0-9]{24}$/.test(value.note_id)) throw new WorkerError('笔记绑定必须是明确的公共笔记编号','INVALID_NOTE_BINDING'); const fail=()=>{throw new WorkerError('笔记绑定仅支持已固化的七步或有界定位八步详情与首层评论合同','NOTE_BINDING_CONTRACT');}; const official=entry.origin==='https://www.xiaohongshu.com',demo=['127.0.0.1','localhost','[::1]'].includes(entry.hostname); if(!official && !demo) fail(); const seek=steps.length===8 ? steps[3] : null; const readingSteps=seek ? steps.filter((_,index)=>index!==3) : steps; if(readingSteps.length!==7 || readingSteps.some((step,index)=>step?.kind!==['fill','press','wait','click','wait','extract','extract'][index])) fail(); try{steps.forEach(validateAction);}catch{fail();} const [fill,press,waiting,click,detailWaiting,detail,comments]=readingSteps; if(seek && (seek.kind!=='scroll' || !seek.until_target || /^(?:body|html)$/i.test(seek.selector.trim()) || seek.until_target.selector!==click.selector || seek.until_target.field!==click.target?.field || seek.until_target.parameter!==click.target?.parameter || seek.until_target.parameter!=='note_title')) fail(); if(fill.parameter!=='keyword' || press.selector!==fill.selector || press.key!=='Enter' || waiting.state!=='visible' || !Number.isInteger(waiting.min_count) || waiting.min_count<1 || click.target?.field!=='text' || click.target?.parameter!=='note_title' || detailWaiting.state!=='visible' || typeof parameters.keyword!=='string' || !parameters.keyword.trim() || typeof parameters.note_title!=='string' || !parameters.note_title.trim() || parameters.note_title.length>2000) fail(); const fieldsMatch=(step,names,content=false)=>step.fields.length===names.length && names.every(name=>{ const field=step.fields.find(field=>field.name===name); return field?.required===true && (name==='comment_id' ? field.source==='attribute' && field.attribute==='id' : field.source==='text' || (content && name==='body' && field.source==='content')); }); if(detail.output_name!=='note_detail' || detail.limit!==1 || detail.target || detail.batch || detail.allow_empty===true || !fieldsMatch(detail,['title','body','author']) || comments.output_name!=='comments' || !comments.row_selector || comments.complete_only!==true || !comments.batch || comments.batch.dedupe_field!=='comment_id' || comments.allow_empty===true || !fieldsMatch(comments,['comment_id','author','body'],true)) fail(); return {note_id:value.note_id.toLowerCase(),href_checked:false,url_checked:false,source_url:null,title:null,opened:false,note_title:parameters.note_title.trim(),seek_timeout_ms:seek?.timeout_ms || 0}; } const bindingResult=binding=>({note_id:binding.note_id,href_checked:binding.href_checked,url_checked:binding.url_checked,source_url:binding.source_url,title:binding.title}); function safePathname(pathname) { return pathname.split('/').map(segment => { let decoded; try { decoded = decodeURIComponent(segment); } catch { return ':redacted'; } if (/password|passwd|credential|token|secret|session|cookie|authorization|captcha|verification|phone|mobile|验证码|密码|手机号/i.test(decoded)) return ':redacted'; if (/(?:\+?86[-\s]?)?1[3-9]\d{9}(?!\d)/.test(decoded) || decoded.length > 64 || /^[a-f0-9-]{16,}$/i.test(decoded) || /^[A-Za-z0-9_=-]{32,}$/.test(decoded) || decoded.includes('@')) return ':id'; return segment; }).join('/').slice(0, 300); } function observeRequest(session, request) { if (!['document', 'fetch', 'xhr'].includes(request.resourceType())) return null; let url; try { url = new URL(request.url()); } catch { return null; } if (!['http:', 'https:'].includes(url.protocol)) return null; const record = { sequence: ++session.requestSequence, recorded_at: new Date().toISOString(), method: request.method().slice(0, 16), origin: url.origin, pathname: safePathname(url.pathname), resource_type: request.resourceType().slice(0, 32), automating: session.automating, blocked: false }; session.requestObservations.push(record); if (session.requestObservations.length > 30) session.requestObservations.shift(); return record; } function recordBlockedRequest(session, request, observation, decision) { const policyDiagnostic=decision ? {policy_reason:decision.reason, ...(decision.policy_version ? {active_policy_version:decision.policy_version} : {}), ...(decision.expected_resource_types ? {expected_resource_types:[...decision.expected_resource_types]} : {})} : {}; if (observation) Object.assign(observation,{blocked:true},policyDiagnostic); const diagnostic = session.actionDiagnostics; if (!diagnostic) return; diagnostic.blocked_request_count += 1; if (diagnostic.blocked_requests.length >= MAX_BLOCKED_REQUESTS) return; let url; try { url = new URL(request.url()); } catch { return; } // Deliberately omit query/hash, request bodies, headers and credentials. const summary = { method: request.method().slice(0, 16), origin: url.origin, pathname: safePathname(url.pathname), resource_type: request.resourceType().slice(0, 32),...policyDiagnostic }; if (!diagnostic.blocked_requests.some(previous => JSON.stringify(previous)===JSON.stringify(summary))) { diagnostic.blocked_requests.push(summary); } } function recordPopup(session, popup, phase, decision, reason) { const record = { sequence: ++session.popupSequence, recorded_at: new Date().toISOString(), phase, decision, reason }; let url; try { url = new URL(popup.url()); } catch { url = null; } if (url && ['http:', 'https:'].includes(url.protocol)) { record.origin = url.origin; record.pathname = safePathname(url.pathname); } session.popupObservations.push(record); if (session.popupObservations.length > 10) session.popupObservations.shift(); } function watchPage(session, page) { page.on('close', () => { // Closing a former search tab must not invalidate the adopted detail tab. if (session.page !== page || session.closed || session.popupPermission?.opener === page || (!session.ready && !session.automating && session.pendingPopups.size > 0)) return; session.closed = true; session.ready = false; session.lastCapture = null; void closeContext(session); }); } // Only fixed Page/Frame objects obtained from Playwright are passed here. This // decides ownership of the existing capability; it never issues or renews one. export function nativePopupRequestContext({executionProfile,automating,policy,policyPage,currentPage,permission,requestPage,requestFrame,popupOpener,now=Date.now()}){ try{ if(executionProfile!==NAVIGATION_UI_PROFILE || automating!==true || !policy || policy.version!==NAVIGATION_UI_POLICY_VERSION || policy.scope!=='native_ui' || !Number.isFinite(policy.expiresAt) || !Number.isFinite(now) || now>=policy.expiresAt || !permission || permission.manual || !permission.opener || popupOpener!==permission.opener || requestPage===permission.opener || requestFrame!==requestPage.mainFrame() || requestFrame.page()!==requestPage || !validNavigationPage(permission.sourceUrl) || permission.opener.url()!==permission.sourceUrl || !navigationPopupMatches(permission.expectedUrl,requestPage.url()) || (permission.pendingPopup && permission.pendingPopup!==requestPage)) return null; const before=permission.adopted===false && currentPage===permission.opener && policyPage===permission.opener; const after=permission.adopted===true && currentPage===requestPage && policyPage===requestPage && permission.pendingPopup===requestPage; return before || after ? {activeMainFrame:true,pageUrl:requestPage.url()} : null; }catch{return null;} } function receivePopup(session, popup) { const manual = !session.ready && !session.automating; const phase = manual ? 'manual' : session.automating ? 'automation' : 'confirmed'; const permission = session.popupPermission || (manual ? { opener: session.page, adopted: false, manual: true } : null); const pending = (async () => { const opener = await popup.opener().catch(() => null); if (!permission || permission.adopted || (opener && opener !== permission.opener) || session.closed || shuttingDown) { session.popupBlocked = true; recordPopup(session, popup, phase, 'blocked', 'unrequested_popup'); await popup.close().catch(() => {}); return; } if(session.executionProfile===NAVIGATION_UI_PROFILE && !permission.manual){ if(permission.pendingPopup && permission.pendingPopup!==popup){ session.popupBlocked=true; recordPopup(session,popup,phase,'blocked','unrequested_popup'); await popup.close().catch(()=>{}); return; } permission.pendingPopup=popup; } // The context route checks every top-level navigation, including this // popup's first request, before this event can adopt it. // about:blank may already have fired domcontentloaded when context emits // page. Wait for the actual bounded navigation before checking the URL. await popup.waitForURL(url => ['http:', 'https:'].includes(url.protocol), { waitUntil: 'domcontentloaded', timeout: 3_000 }).catch(() => {}); let url; try { url = new URL(popup.url()); } catch { url = null; } if (!url || !['http:', 'https:'].includes(url.protocol) || url.origin !== session.origin || (session.executionProfile===NAVIGATION_UI_PROFILE && (!validNavigationPage(url.href) || (!permission.manual && !navigationPopupMatches(permission.expectedUrl,url.href)))) || url.username || url.password || session.blockedNavigation || permission.adopted || (!permission.manual && !opener && (session.executionProfile===NAVIGATION_UI_PROFILE ? !navigationPopupMatches(permission.expectedUrl,url.href) : (!permission.expectedUrl || url.href !== permission.expectedUrl)))) { session.popupBlocked = true; recordPopup(session, popup, phase, 'blocked', !url || !['http:', 'https:'].includes(url.protocol) ? 'unconfirmed_origin' : url.origin !== session.origin || session.blockedNavigation ? 'outside_scope' : 'unrequested_popup'); await popup.close().catch(() => {}); return; } const transferred=nativePopupRequestContext({executionProfile:session.executionProfile,automating:session.automating, policy:session.platformRequestPolicy,policyPage:session.platformRequestPage,currentPage:session.page, permission,requestPage:popup,requestFrame:popup.mainFrame(),popupOpener:opener}); permission.adopted = true; watchPage(session, popup); session.page = popup; if(transferred) session.platformRequestPage=popup; session.lastCapture = null; // A same-origin page opened during manual login remains a page awaiting // confirmation; adopting the page never authorizes automated actions. if (permission.manual) session.ready = false; recordPopup(session, popup, phase, 'adopted', permission.manual ? 'same_origin_manual' : 'same_origin_click'); })(); session.pendingPopups.add(pending); void pending.finally(() => { session.pendingPopups.delete(pending); if (!session.ready && !session.automating && !session.pendingPopups.size && session.page.isClosed() && !session.closed) { session.closed = true; session.lastCapture = null; void closeContext(session); } }).catch(() => {}); } async function settlePopups(session) { while (session.pendingPopups.size) await Promise.all([...session.pendingPopups]); if (session.popupBlocked) throw new WorkerError('出现未获明确点击授权或站点范围外的弹窗,已关闭并暂停', 'POPUP_BLOCKED'); if (session.blockedNavigation) throw new WorkerError('页面尝试离开本次学习站点,已阻止并暂停', 'SCOPE_BLOCKED'); } async function settle(page) { await page.waitForLoadState('domcontentloaded', { timeout: 3_000 }).catch(() => {}); await page.waitForTimeout(180); } async function refreshLoginDetection(session) { session.loginDetection = await detectSession(session.page); if (['logged_out', 'verification_required'].includes(session.loginDetection.state)) { session.ready = false; session.lastCapture = null; } return session.loginDetection; } async function open(params) { const taskId = requireString(params.task_id, 'task_id', 128); const accountKey = requireString(params.account_key, 'account_key', 256); const entry = httpUrl(params.entry_url); resolveExecutionProfile(params.execution_profile); if(isNativeUiProfile(params.execution_profile) && !(params.execution_profile===NAVIGATION_UI_PROFILE ? validNavigationPage(entry.href) : validNativePage(entry.href))) throw new WorkerError('新执行模式仅支持已声明的小红书官网搜索或笔记页面','EXECUTION_PROFILE_INVALID'); const existing = sessions.get(taskId); if (existing && !existing.closed && !existing.page.isClosed()) { if (existing.accountKey !== accountKey || existing.origin !== entry.origin || existing.executionProfile!==params.execution_profile) { throw new WorkerError('当前任务已经持有不同的账号或站点会话,请先关闭', 'SESSION_CONFLICT'); } await refreshLoginDetection(existing); return sessionInfo(existing); } const accountId = digest(`${entry.origin}\0${accountKey}`); if (closingAccounts.has(accountId)) await closingAccounts.get(accountId); const owner = accountOwners.get(accountId); if (owner && owner !== taskId) { throw new WorkerError('同一平台账号正在被另一学习任务使用,请先关闭原会话', 'ACCOUNT_BUSY'); } const profile = path.join(HOME, 'profiles', accountId); await fs.mkdir(profile, { recursive: true }); accountOwners.set(accountId, taskId); let context; try { const browserType = await chromium(); if (shuttingDown) throw new WorkerError('学习工作进程正在退出', 'WORKER_CLOSING'); context = await browserType.launchPersistentContext(profile, { channel: CHANNEL, headless: HEADLESS, viewport: { width: 1280, height: 840 }, acceptDownloads: false, timeout: 30_000, }); contexts.add(context); context.once('close', () => contexts.delete(context)); // An EOF may arrive while launching. Never expose a newly launched context // after shutdown has already taken its snapshot of active contexts. if (shuttingDown) throw new WorkerError('学习工作进程正在退出', 'WORKER_CLOSING'); const page = context.pages()[0] || await context.newPage(); await Promise.allSettled(context.pages().filter(other => other !== page).map(other => other.close())); page.setDefaultTimeout(TIMEOUT); const session = { taskId, accountKey, accountId, context, page, origin: entry.origin,executionProfile:params.execution_profile, entryUrl: entry.href, sessionId: randomUUID(), ready: false, closed: false, identityLabel: null, lastCapture: null, blockedNavigation: null, loginDetection: null, observedSelectors: new Set(), automating: false, blockedWrites: 0, actionDiagnostics: null, popupPermission: null, pendingPopups: new Set(), popupBlocked: false, requestSequence: 0, requestObservations: [], popupSequence: 0, popupObservations: [], platformRequestPolicy: null, platformRequestPage: null, }; await context.route('**/*', async route => { const request = route.request(); const observation = observeRequest(session, request); let topLevelNavigation = false; if (request.isNavigationRequest()) { // A popup's first navigation can arrive before Playwright creates its // Frame. Treat that case as top-level; never bypass its scope check. try { topLevelNavigation = request.frame().parentFrame() === null; } catch { topLevelNavigation = true; } } if (topLevelNavigation) { let target; try { target = new URL(request.url()); } catch { target = null; } if (!target || target.origin !== session.origin || (session.executionProfile===NAVIGATION_UI_PROFILE && !validNavigationPage(target.href))) { session.blockedNavigation = true; recordBlockedRequest(session, request, observation,{reason:'navigation_out_of_scope'}); await route.abort('blockedbyclient').catch(() => {}); return; } } // Unknown non-GET requests remain blocked. Only the finite experimental // official UI action policy can allow its observed auxiliary requests. // Human login does not use the automation flag. if (session.automating && !['GET', 'HEAD', 'OPTIONS'].includes(request.method())) { let activeMainFrame = false; let policyPageUrl=session.page.url(); try { const frame=request.frame(),requestPage=frame.page(),permission=session.popupPermission; if(session.executionProfile===NAVIGATION_UI_PROFILE && permission && requestPage!==permission.opener){ const popupOpener=await requestPage.opener(); if(session.popupPermission===permission){ const popupContext=nativePopupRequestContext({executionProfile:session.executionProfile,automating:session.automating, policy:session.platformRequestPolicy,policyPage:session.platformRequestPage,currentPage:session.page, permission,requestPage,requestFrame:frame,popupOpener}); if(popupContext){ if(!permission.pendingPopup) permission.pendingPopup=requestPage; activeMainFrame=popupContext.activeMainFrame;policyPageUrl=popupContext.pageUrl; } } }else activeMainFrame=session.platformRequestPage===session.page && frame===session.page.mainFrame(); } catch {} const policyDecision = evaluatePlatformRequest({ policy: session.platformRequestPolicy, automating: session.automating, sessionOrigin: session.origin, pageUrl: policyPageUrl, activeMainFrame, method: request.method(), resourceType: request.resourceType(), url: request.url() }); if (policyDecision.allowed) { if (observation) Object.assign(observation,{policy_version:policyDecision.policy_version,policy_rule:policyDecision.policy_rule}); await route.continue().catch(() => {}); return; } session.blockedWrites += 1; recordBlockedRequest(session, request, observation,policyDecision); await route.abort('blockedbyclient').catch(() => {}); return; } await route.continue().catch(() => {}); }); context.on('page', popup => { if (popup !== session.page) receivePopup(session, popup); }); const markClosed = () => { session.closed = true; session.ready = false; session.lastCapture = null; if (accountOwners.get(accountId) === taskId) accountOwners.delete(accountId); }; context.on('close', markClosed); watchPage(session, page); sessions.set(taskId, session); await page.goto(entry.href, { waitUntil: 'domcontentloaded', timeout: 25_000 }); requireOrigin(session, page.url()); await settle(page); await refreshLoginDetection(session); return sessionInfo(session); } catch (error) { if (context) await context.close().catch(() => {}); sessions.delete(taskId); if (accountOwners.get(accountId) === taskId) accountOwners.delete(accountId); if (/executable|browser.*not found|distribution.*not found/i.test(error.message)) { throw new WorkerError(`浏览器不可用。请安装 Playwright Chromium,或配置本机已安装的浏览器通道。当前通道 ${CHANNEL}`, 'BROWSER_UNAVAILABLE'); } throw error; } } function sessionInfo(session) { return { ...executionMetadata(session.executionProfile), session_id: session.sessionId, status: session.closed ? 'closed' : session.ready ? 'observing' : 'awaiting_user', url: redactUrl(session.page.url()), manual_confirmation: session.ready, identity_label: session.identityLabel, login_detection: session.loginDetection, block_reason: session.blockedNavigation ? 'scope_blocked' : session.popupBlocked ? 'popup_blocked' : null, popup_observations: session.popupObservations.map(record => ({ ...record })), }; } async function ready(params) { const session = sessionFor(params.task_id, false); await settlePopups(session); requireOrigin(session, session.page.url()); const detection = await refreshLoginDetection(session); if (detection.state === 'logged_out') throw new WorkerError('当前可见页面仍要求登录,请在官方页面完成后重新确认', 'LOGIN_REQUIRED'); if (detection.state === 'verification_required') throw new WorkerError('当前可见页面需要人工验证,请在官方页面处理后重新确认', 'VERIFICATION_REQUIRED'); const label = requireString(params.identity_label, 'identity_label', 100); if (/(?:\+?86[-\s]?)?1[3-9]\d{9}|验证码|password|密码/i.test(label)) { throw new WorkerError('确认标签请使用测试账号昵称或代号,不填写手机号、密码或验证码', 'SENSITIVE_INPUT'); } session.ready = true; session.identityLabel = label; session.lastCapture = null; return { ...sessionInfo(session), identity_verification: 'operator_confirmed', note: '此记录是维护者人工确认,不代表自动完成平台账号身份核验' }; } export async function describePage(page,request=null) { return page.evaluate(request => { const metadataOnly=Boolean(request?.selector); const PRIVATE = /password|passwd|credential|token|secret|cookie|authorization|session|one.time.code|verification|captcha|phone|mobile|(?:^|[-_])tel(?:$|[-_])|sms|验证码|校验码|动态码|短信码|手机号|手机号码|密码|口令|登录|登陆|sign.?in|log.?in/i; const phone = /(?:\+?86[-\s]?)?1[3-9]\d{9}(?!\d)/g; const clean = text => String(text || '').replace(phone, '[手机号已隐藏]').trim(); // This evaluate callback is synchronous and makes no DOM writes. Reuse the // same node facts only within this observation; scrolling, style changes // and later captures always start with fresh caches. const memo = read => { const values = new WeakMap(); return element => { if (!values.has(element)) values.set(element, read(element)); return values.get(element); }; }; const rectFor = memo(element => element.getBoundingClientRect()); const styleFor = memo(element => getComputedStyle(element)); const fieldLabel = memo(element => { const id = element.id; const explicit = id ? document.querySelector(`label[for="${CSS.escape(id)}"]`)?.textContent : ''; return [element.getAttribute('aria-label'), element.getAttribute('placeholder'), explicit, element.getAttribute('title'), element.getAttribute('name'), element.getAttribute('autocomplete'), element.id] .filter(Boolean).join(' ').trim(); }); const stableFieldLabel = memo(element => { const explicit = element.id ? document.querySelector(`label[for="${CSS.escape(element.id)}"]`)?.textContent : ''; const fields = [element.getAttribute('aria-label'),element.getAttribute('title'),element.getAttribute('name'),explicit,element.id, element.getAttribute('type') || (element.tagName==='INPUT' ? 'text' : 'textarea'),element.getAttribute('autocomplete')] .map(value=>clean(value).replace(/\s+/g,' ')); const result = JSON.stringify(fields); return fields.some(Boolean) && result.length<=500 ? result : ''; }); const isSensitive = memo(element => { if (element.matches('input[type=password],input[type=tel],[autocomplete="one-time-code"]')) return true; if (PRIVATE.test(fieldLabel(element))) return true; for (let parent = element.parentElement; parent; parent = parent.parentElement) { if (PRIVATE.test([parent.id, parent.getAttribute('aria-label'), parent.getAttribute('title')].filter(Boolean).join(' '))) return true; } const form = element.closest('form,[role=dialog]'); return Boolean(form && (form.querySelector('input[type=password],input[type=tel],[autocomplete="one-time-code"]') || PRIVATE.test([form.getAttribute('id'), form.getAttribute('class'), form.getAttribute('aria-label'), form.textContent?.slice(0, 1200)].join(' ')))); }); const visible = memo(element => { const rect = rectFor(element); if (rect.width <= 0 || rect.height <= 0) return false; for (let current = element; current; current = current.parentElement) { const style = styleFor(current); if (style.display === 'none' || ['hidden', 'collapse'].includes(style.visibility) || Number(style.opacity) === 0) return false; } return true; }); const inViewport = memo(element => { const rect = rectFor(element); let top = Math.max(0, rect.top), bottom = Math.min(innerHeight, rect.bottom); let left = Math.max(0, rect.left), right = Math.min(innerWidth, rect.right); for (let parent = element.parentElement; parent; parent = parent.parentElement) { const style = styleFor(parent), box = rectFor(parent); if (/(?:auto|scroll|hidden|clip)/.test(style.overflowY)) { top = Math.max(top, box.top); bottom = Math.min(bottom, box.bottom); } if (/(?:auto|scroll|hidden|clip)/.test(style.overflowX)) { left = Math.max(left, box.left); right = Math.min(right, box.right); } } return visible(element) && bottom > top && right > left; }); const selectorFor = element => { if (element.getAttribute('data-testid')) return `[data-testid=${JSON.stringify(element.getAttribute('data-testid'))}]`; if (element.id) return `#${CSS.escape(element.id)}`; if (element.getAttribute('data-learning-read')) return `[data-learning-read=${JSON.stringify(element.getAttribute('data-learning-read'))}]`; const segments = []; let current = element; for (let level = 0; current && current !== document.body && level < 7; level += 1, current = current.parentElement) { if (current.id) { segments.unshift(`#${CSS.escape(current.id)}`); break; } const tag = current.tagName.toLowerCase(); const siblings = current.parentElement ? Array.from(current.parentElement.children).filter(node => node.tagName === current.tagName) : []; segments.unshift(siblings.length > 1 ? `${tag}:nth-of-type(${siblings.indexOf(current) + 1})` : tag); } return segments.join(' > '); }; const rawTextCaches = [new WeakMap(), new WeakMap()]; const displayedText = (root, viewportOnly = false) => { const cache = rawTextCaches[Number(viewportOnly)]; const walk = node => { if (cache.has(node)) return cache.get(node); let text = ''; if (node.nodeType === Node.TEXT_NODE) text = node.textContent; else if (node.nodeType === Node.ELEMENT_NODE && !node.matches('script,style,noscript,iframe,input,textarea,select,[contenteditable=true]') && !isSensitive(node) && visible(node) && (!viewportOnly || inViewport(node))) { text = Array.from(node.childNodes, walk).join(' '); } cache.set(node, text); return text; }; return clean(walk(root)).replace(/\s+/g, ' ').slice(0, 16_000); }; const text = metadataOnly ? '' : displayedText(document.body); const viewportText = metadataOnly ? '' : displayedText(document.body, true); const elements = [], seen = new Set(), masks = new Set(); for (const element of metadataOnly ? [] : document.querySelectorAll('body *')) { if (PRIVATE.test([element.id, element.getAttribute('aria-label'), element.getAttribute('title')].filter(Boolean).join(' '))) { masks.add(selectorFor(element)); } if (element.children.length === 0 && /(?:\+?86[-\s]?)?1[3-9]\d{9}(?!\d)/.test(element.textContent || '')) { masks.add(selectorFor(element)); } } const safeToken = value => /^[A-Za-z][A-Za-z0-9_-]{0,95}$/.test(value || '') && !PRIVATE.test(value) && !/(?:\+?86[-\s]?)?1[3-9]\d{9}(?!\d)/.test(value); const readable = memo(element => !element.matches('script,style,noscript,iframe,input,textarea,select,[contenteditable=true]') && !isSensitive(element)); const scrollInfo = memo(element => { if (!readable(element) || !/(?:auto|scroll)/.test(styleFor(element).overflowY) || element.clientHeight <= 0) return null; return { can_scroll: element.scrollHeight > element.clientHeight + 1, scroll_top: element.scrollTop, scroll_height: element.scrollHeight, client_height: element.clientHeight }; }); const relativeSelector = (root, element) => { if (safeToken(element.id)) return `#${CSS.escape(element.id)}`; for (const name of [...element.classList].filter(safeToken).slice(0, 3)) { const selector = `${element.tagName.toLowerCase()}.${CSS.escape(name)}`; if (root.querySelectorAll(selector).length === 1) return selector; } const parts = []; for (let current = element; current && current !== root && parts.length < 6; current = current.parentElement) { const tag = current.tagName.toLowerCase(); const siblings = [...current.parentElement.children].filter(node => node.tagName === current.tagName); parts.unshift(siblings.length > 1 ? `${tag}:nth-of-type(${siblings.indexOf(current) + 1})` : tag); if (current.parentElement === root) return `:scope > ${parts.join(' > ')}`; } return null; }; const readHints = root => { const hints = [], seenHints = new Set(); const media=[...root.querySelectorAll('img,video')].slice(0,40).filter(element=>visible(element) && readable(element) && !element.closest('a') && ![element,...function*(node){for(let parent=node.parentElement;parent;parent=parent.parentElement) yield parent;}(element)].some(node=>/avatar|头像/i.test([node.id,...node.classList].join(' ')))) .sort((a,b)=>Number(inViewport(b))-Number(inViewport(a))).slice(0,4); for(const child of media){ const selector=relativeSelector(root,child);if(!selector || selector.length>350 || seenHints.has(selector)) continue; const image=child.tagName==='IMG',count=root.querySelectorAll(selector).length; seenHints.add(selector);hints.push({selector,tag:child.tagName.toLowerCase(),label:image ? '图片元素(未解释)' : '视频元素(未解释)',unique:count===1,count, in_viewport:inViewport(child),link:false,source:'media',media_kind:image ? 'image' : 'video',loaded:image ? child.complete && child.naturalWidth>0 : child.readyState>=2 && child.videoWidth>0}); } const children = [...root.querySelectorAll('a,h1,h2,h3,h4,h5,h6,p,time,span,div[id],div[class]')].slice(0, 200) .filter(element => visible(element) && readable(element)) .sort((a, b) => Number(inViewport(b)) - Number(inViewport(a)) || Number(b.children.length === 0) - Number(a.children.length === 0)); for (const child of children) { if (hints.length >= 12) break; const selector = relativeSelector(root, child); if (!selector || selector.length > 350 || seenHints.has(selector)) continue; const count = root.querySelectorAll(selector).length; const label = displayedText(child).slice(0, 100); if (!label) continue; seenHints.add(selector); const link = child.tagName === 'A'; const datetime = child.tagName === 'TIME' && child.hasAttribute('datetime'); hints.push({ selector, tag: child.tagName.toLowerCase(), label, unique: count === 1, count, in_viewport: inViewport(child), link, source: datetime ? 'attribute' : link ? 'href' : 'text', ...(datetime ? { attribute: 'datetime' } : {}) }); } return hints; }; const elementMetadata=(element,selector,count)=>{ const tag=element.tagName.toLowerCase(); const kind=['input','textarea'].includes(tag) ? 'fill' : (tag==='button' || tag==='a' || element.getAttribute('role')==='button') ? 'click' : 'read'; if(!visible(element) || isSensitive(element) || element.hasAttribute('disabled') || (kind==='fill' && !['text','search','url',''].includes((element.getAttribute('type') || '').toLowerCase()))) return null; let label=kind==='fill' ? fieldLabel(element) : displayedText(element); label=clean(label || element.getAttribute('aria-label') || tag).slice(0,180) || tag; const searchDescription=[element.getAttribute('type'),element.id,fieldLabel(element)].filter(Boolean).join(' '); const form=element.form || element.closest('form'); const searchForm=form && (form.getAttribute('method') || 'get').toLowerCase()==='get' && (/search|query|搜索|查询|检索/i.test([form.getAttribute('action'),form.getAttribute('role')].join(' ')) || [...form.querySelectorAll('button,input[type=submit]')].some(button=>/search|搜索|查询|检索/i.test([button.textContent,button.getAttribute('aria-label'),button.getAttribute('value')].join(' ')))); const search_input=kind==='fill' && !/comment|reply|message|私信|评论|回复|发言|留言|发送/i.test(searchDescription) && (!form || (form.getAttribute('method') || 'get').toLowerCase()==='get') && (/search|query|keyword|搜索|查询|检索/i.test(searchDescription) || Boolean(searchForm)); const scroll_y=scrollInfo(element); return {selector,tag,label,kind,unique:count===1,count,sensitive:false,in_viewport:inViewport(element),search_input, ...(search_input ? {stable_label:stableFieldLabel(element)} : {}), selector_basis:safeToken(element.id) ? 'id' : scroll_y ? 'scroll_container' : 'semantic',...(scroll_y ? {scroll_y} : {})}; }; if(metadataOnly){ const matches=[...document.querySelectorAll(request.selector)]; if(matches.length>500) return {elements:[],read_limit:true}; if(!matches.length || matches.some(element=>isSensitive(element))) return {elements:[]}; if(request.requireCanonical && matches.some(element=>selectorFor(element)!==request.selector)) return {elements:[]}; const first=matches.find(element=>visible(element) && inViewport(element)) || (!request.collection && matches.find(visible)); if(!first || matches.some(element=>element.tagName!==first.tagName) || (request.collection && matches.some(element=>!readable(element)))) return {elements:[]}; const metadata=elementMetadata(first,request.selector,matches.length); return {elements:metadata ? [metadata] : []}; } const base = [...document.querySelectorAll('input,textarea,button,a,[data-testid],[data-learning-read],main,article,[role=main],[role=button]')]; const semantic = [...document.querySelectorAll('body [id],h1,h2,h3,h4,h5,h6,section,[role=heading],[role=region],[role=list],[role=listitem]')] .slice(0, 2000).filter(element => readable(element) && (safeToken(element.id) || element.matches('h1,h2,h3,h4,h5,h6,section,[role=heading],[role=region],[role=list],[role=listitem]'))); const scrollRoots = new Set(); for (const element of [...semantic, ...base].filter(inViewport).slice(0, 240)) { for (let parent = element, depth = 0; parent && parent !== document.body && depth < 7; parent = parent.parentElement, depth += 1) { if (scrollInfo(parent)) scrollRoots.add(parent); } } const foreground = memo(element => { if (!inViewport(element)) return false; const rect = rectFor(element); const hit = document.elementFromPoint(Math.max(0, Math.min(innerWidth - 1, (Math.max(0, rect.left) + Math.min(innerWidth, rect.right)) / 2)), Math.max(0, Math.min(innerHeight - 1, (Math.max(0, rect.top) + Math.min(innerHeight, rect.bottom)) / 2))); return Boolean(hit && (hit === element || element.contains(hit))); }); const controlRank = element => element.matches('input,textarea') ? 0 : element.matches('button,[role=button]') ? 1 : element.tagName==='A' ? 3 : 2; const candidates = [...new Set([...semantic, ...scrollRoots, ...base])] .sort((a, b) => Number(inViewport(b)) - Number(inViewport(a)) || controlRank(a)-controlRank(b) || Number(foreground(b))-Number(foreground(a))); let loadedOffscreen = 0; let readCount = 0, hintedRoots = 0, fillCount = 0, buttonCount = 0; for (const element of candidates) { if (!visible(element)) continue; const selector = selectorFor(element); if (!selector) continue; if (isSensitive(element)) { const form = element.closest('form,[role=dialog]'); masks.add(selectorFor(form || element)); continue; } if (seen.has(selector) || elements.length >= 108) continue; const in_viewport = inViewport(element); if (!in_viewport && loadedOffscreen >= 30) continue; const tag = element.tagName.toLowerCase(); const kind = ['input', 'textarea'].includes(tag) ? 'fill' : (tag === 'button' || tag === 'a' || element.getAttribute('role') === 'button') ? 'click' : 'read'; if (kind === 'read' && readCount >= 48) continue; if ((kind==='fill' && fillCount>=16) || (kind==='click' && tag!=='a' && buttonCount>=24)) continue; if (kind === 'fill' && !['text','search','url',''].includes((element.getAttribute('type') || '').toLowerCase())) continue; if (element.hasAttribute('disabled')) continue; let count = 0; try { count = document.querySelectorAll(selector).length; } catch { continue; } seen.add(selector); if (kind === 'read') readCount += 1; if (kind === 'fill') fillCount += 1; if (kind === 'click' && tag!=='a') buttonCount += 1; if (!in_viewport) loadedOffscreen += 1; const metadata=elementMetadata(element,selector,count); if(!metadata) continue; const structure = kind === 'read' && hintedRoots < 16 ? readHints(element) : []; if (structure.length) hintedRoots += 1; elements.push({...metadata,...(structure.length ? {read_hints:structure} : {})}); } const addCollection = (selector, basis, sources) => { if (elements.length >= 120 || seen.has(selector) || selector.length > 1000) return; let matches; try { matches = [...document.querySelectorAll(selector)]; } catch { return; } if (!matches.length || matches.length > 200 || matches.some(element => !readable(element))) return; const first = matches.find(element => visible(element) && inViewport(element)); if (!first || matches.some(element => element.tagName !== first.tagName)) return; const tag = first.tagName.toLowerCase(), kind = tag === 'a' ? 'click' : 'read'; if (kind === 'click' && matches.some(element => !(element instanceof HTMLAnchorElement))) return; seen.add(selector); elements.push({ selector, tag, label: displayedText(first).slice(0,180) || tag, kind, unique: matches.length === 1, count: matches.length, sensitive:false, in_viewport:true, search_input:false, collection:true, selector_basis:basis, source_selectors:sources.slice(0,3), read_hints:readHints(first) }); }; for (const element of candidates.filter(element => readable(element) && visible(element) && inViewport(element)).slice(0, 400)) { if (elements.length >= 120) break; const family = safeToken(element.id) && element.id.match(/^([A-Za-z][A-Za-z_-]{1,47}[-_])(?:[a-fA-F0-9]{16,32}|\d{1,12})$/); const parent = element.parentElement; if (family && parent && parent !== document.body) { const scope = selectorFor(parent); if (scope && document.querySelectorAll(scope).length === 1) { const selector = `${scope} > ${element.tagName.toLowerCase()}[id^=${JSON.stringify(family[1])}]`; const members = [...document.querySelectorAll(selector)]; if (members.every(member => member.id.match(/^([A-Za-z][A-Za-z_-]{1,47}[-_])(?:[a-fA-F0-9]{16,32}|\d{1,12})$/)?.[1] === family[1])) { addCollection(selector, 'id_family', members.slice(0,3).map(selectorFor)); } } } for (const name of [...element.classList].filter(safeToken).slice(0, 2)) { for (let scopeNode = parent, depth = 0; scopeNode && scopeNode !== document.body && depth < 12; scopeNode = scopeNode.parentElement, depth += 1) { if (!safeToken(scopeNode.id) && !scopeNode.matches('main,article,section,[role=main],[role=region]')) continue; const scope = selectorFor(scopeNode), selector = `${scope} ${element.tagName.toLowerCase()}.${CSS.escape(name)}`; if (!scope || document.querySelectorAll(scope).length !== 1) continue; const members = [...document.querySelectorAll(selector)]; if (members.length >= 2 && members.length <= 200) { addCollection(selector, 'sibling_class', members.slice(0,3).map(selectorFor)); break; } } } } return { title: clean(document.title).slice(0, 250), text, viewport_text: viewportText, text_scope: 'displayed_loaded_dom', viewport: { width: innerWidth, height: innerHeight, scroll_x: scrollX, scroll_y: scrollY }, elements, masks: [...masks].filter(Boolean) }; },request); } async function describeCurrentSelector(page,selector,observed){ const current=await describePage(page,{selector,collection:observed?.collection===true,requireCanonical:Boolean(observed && !observed.collection)}); if(current.read_limit) throw new WorkerError('当前定位超过500个对象,请重新限定范围','READ_LIMIT'); return current.elements[0]; } async function captureSession(session, note = '') { if (shuttingDown) throw new WorkerError('学习工作进程正在退出', 'WORKER_CLOSING'); if(session.closed || session.page.isClosed()) throw new WorkerError('浏览器会话已关闭,不能继续采集','SESSION_CLOSED'); requireOrigin(session, session.page.url()); const description = await describePage(session.page); if(session.closed || session.page.isClosed()) throw new WorkerError('会话已中断,不保存迟到的页面描述','SESSION_CLOSED'); const captureId = randomUUID(); const directory = path.join(HOME, 'evidence', digest(session.taskId).slice(0, 24)); await fs.mkdir(directory, { recursive: true }); const screenshotPath = path.join(directory, `${captureId}.png`); const mask = description.masks.map(selector => session.page.locator(selector)); mask.push(session.page.locator('input,textarea,[contenteditable=true]')); await session.page.screenshot({ path: screenshotPath, fullPage: false, mask, maskColor: '#1d252b', timeout: 10_000 }); if(session.closed || session.page.isClosed()) throw new WorkerError('浏览器会话已关闭,不追加迟到的采集','SESSION_CLOSED'); const snapshot = { ...executionMetadata(session.executionProfile), capture_id: captureId, url: redactUrl(session.page.url()), title: redactText(description.title), text: redactText(description.text), viewport_text: redactText(description.viewport_text), text_scope: description.text_scope, viewport: description.viewport, elements: description.elements, screenshot_path: screenshotPath, relative_screenshot_path: path.relative(HOME, screenshotPath).split(path.sep).join('/'), captured_at: new Date().toISOString(), note: redactText(note).slice(0, 1000), identity_verification: 'operator_confirmed', request_observations: session.requestObservations.map(record => ({ ...record })), popup_observations: session.popupObservations.map(record => ({ ...record })), }; session.lastCapture = snapshot; for (const element of snapshot.elements) if (!element.sensitive) session.observedSelectors.add(element.selector); return snapshot; } async function capture(params) { const session = sessionFor(params.task_id); return captureSession(session, params.note || ''); } const DANGEROUS = /关注|私信|发送|提交|支付|购买|删除|移除|注销|退出登录|确认订单|下单|转账|follow|unfollow|send|submit|purchase|delete|remove|logout|sign.out|pay(?:ment)?/i; const READ_ONLY_BUTTON = /搜索|查询|检索|筛选|展开|收起|更多|下一页|上一页|刷新|加载|search|filter|expand|collapse|more|next|previous|refresh|load/i; // Runs only the fixed inspection below, never caller-provided JavaScript. export function inspectElements(elements, { target, fields, origin, completeOnly, untilSelector, rowSelector, fixedRootSelector, fixedSourceUrl,nativeProfile=false,navigationProfile=false }) { if (!Array.isArray(elements)) elements = [elements]; if(fixedRootSelector && (location.href!==fixedSourceUrl || elements.length!==1 || !elements[0].isConnected || document.querySelectorAll(fixedRootSelector).length!==1 || document.querySelector(fixedRootSelector)!==elements[0])) return [{scope_changed:true}]; if(rowSelector){ elements=elements.flatMap(element=>[...element.querySelectorAll(rowSelector)]); if(elements.length>500) return [{row_limit:true}]; } const PRIVATE = /password|passwd|credential|token|secret|cookie|authorization|session|one.time.code|verification|captcha|phone|mobile|(?:^|[-_])tel(?:$|[-_])|sms|验证码|校验码|动态码|短信码|手机号|手机号码|密码|口令|登录|登陆|sign.?in|log.?in/i; const excluded = 'script,style,noscript,iframe,input,textarea,select,[contenteditable=true]'; const sensitive = element => { const description = [element.getAttribute('aria-label'), element.getAttribute('placeholder'), element.getAttribute('title'), element.getAttribute('name'), element.getAttribute('autocomplete'), element.id].filter(Boolean).join(' '); if (element.matches('input[type=password],input[type=tel],[autocomplete="one-time-code"]') || PRIVATE.test(description)) return true; for (let parent = element.parentElement; parent; parent = parent.parentElement) { if (PRIVATE.test([parent.id, parent.getAttribute('aria-label'), parent.getAttribute('title')].filter(Boolean).join(' '))) return true; } const form = element.closest('form,[role=dialog]'); return Boolean(form && (form.querySelector('input[type=password],input[type=tel],[autocomplete="one-time-code"]') || PRIVATE.test([form.id, form.className, form.getAttribute('aria-label'), form.textContent?.slice(0, 1200)].join(' ')))); }; const visible = element => { const rect = element.getBoundingClientRect(); if (rect.width<=0 || rect.height<=0) return false; for (let current=element;current;current=current.parentElement) { const style=getComputedStyle(current); if (style.display==='none' || ['hidden','collapse'].includes(style.visibility) || Number(style.opacity)===0) return false; } return true; }; const viewport = element => { const rect = element.getBoundingClientRect(); let top = Math.max(0, rect.top), bottom = Math.min(innerHeight, rect.bottom); let left = Math.max(0, rect.left), right = Math.min(innerWidth, rect.right); for (let parent = element.parentElement; parent; parent = parent.parentElement) { const style = getComputedStyle(parent), box = parent.getBoundingClientRect(); if (/(?:auto|scroll|hidden|clip)/.test(style.overflowY)) { top = Math.max(top, box.top); bottom = Math.min(bottom, box.bottom); } if (/(?:auto|scroll|hidden|clip)/.test(style.overflowX)) { left = Math.max(left, box.left); right = Math.min(right, box.right); } } return visible(element) && bottom > top && right > left; }; const fullyViewport = (element, container = null) => { if (!visible(element)) return false; const rect=element.getBoundingClientRect(); let top=0,bottom=innerHeight,left=0,right=innerWidth; for(let parent=element.parentElement;parent;parent=parent.parentElement){ const style=getComputedStyle(parent),box=parent.getBoundingClientRect(); if (/(?:auto|scroll|hidden|clip)/.test(style.overflowY)) {top=Math.max(top,box.top+parent.clientTop);bottom=Math.min(bottom,box.top+parent.clientTop+parent.clientHeight);} if (/(?:auto|scroll|hidden|clip)/.test(style.overflowX)) {left=Math.max(left,box.left+parent.clientLeft);right=Math.min(right,box.left+parent.clientLeft+parent.clientWidth);} } if(container){const box=container.getBoundingClientRect();top=Math.max(top,box.top+container.clientTop);bottom=Math.min(bottom,box.top+container.clientTop+container.clientHeight);left=Math.max(left,box.left+container.clientLeft);right=Math.min(right,box.left+container.clientLeft+container.clientWidth);} return rect.top>=top-0.5 && rect.bottom<=bottom+0.5 && rect.left>=left-0.5 && rect.right<=right+0.5; }; const clean = value => String(value || '').replace(/\s+/g, ' ').trim().slice(0, 4_000); const stableFieldLabel = element => { const explicit = element.id ? document.querySelector(`label[for="${CSS.escape(element.id)}"]`)?.textContent : ''; const fields = [element.getAttribute('aria-label'),element.getAttribute('title'),element.getAttribute('name'),explicit,element.id, element.getAttribute('type') || (element.tagName==='INPUT' ? 'text' : 'textarea'),element.getAttribute('autocomplete')] .map(value=>String(value || '').replace(/(?:\+?86[-\s]?)?1[3-9]\d{9}(?!\d)/g,'[手机号已隐藏]').replace(/\s+/g,' ').trim()); const result = JSON.stringify(fields); return fields.some(Boolean) && result.length<=500 ? result : ''; }; const textOf = root => { if (root.matches(excluded) || sensitive(root)) return null; const walk = node => { if (node.nodeType === Node.TEXT_NODE) return node.textContent; if (node.nodeType !== Node.ELEMENT_NODE || node.matches(excluded) || sensitive(node) || !visible(node)) return ''; return Array.from(node.childNodes, walk).join(' '); }; return clean(walk(root)); }; const hrefOf = element => { if (!(element instanceof HTMLAnchorElement) || !element.hasAttribute('href')) return null; const url = new URL(element.href, document.baseURI); if (!['http:', 'https:'].includes(url.protocol) || url.origin !== origin || url.username || url.password) return { error: 'UNSAFE_OUTPUT' }; return url.href; }; const contentOf=(root,field,row)=>{ const textNodes=[...root.querySelectorAll(field.text_selector)],mediaNodes=[...root.querySelectorAll(field.media_selector)]; if(textNodes.length>1) return {error:'NON_UNIQUE_FIELD'}; if(mediaNodes.length>20) return {error:'READ_LIMIT'}; if(textNodes.some(node=>node.matches(excluded) || sensitive(node))) return {error:'SENSITIVE_OUTPUT'}; const textNode=textNodes[0],text=textNode && visible(textNode) ? textOf(textNode) || null : null; if(textNode && visible(textNode) && !fullyViewport(textNode) && !completeOnly) return {error:'TARGET_NOT_IN_VIEWPORT'}; let incomplete=Boolean(completeOnly && (!fullyViewport(row) || !fullyViewport(root) || (textNode && visible(textNode) && !fullyViewport(textNode)))); const media=[]; for(const node of mediaNodes){ if(!['IMG','VIDEO'].includes(node.tagName) || sensitive(node)) return {error:'SENSITIVE_OUTPUT'}; if(node.closest('a')) return {error:'UNSAFE_CONTENT_MEDIA'}; for(let parent=node;parent;parent=parent.parentElement) if(/avatar|头像/i.test([parent.id,...parent.classList].join(' '))) return {error:'UNSAFE_CONTENT_MEDIA'}; const shown=visible(node),complete=shown && fullyViewport(node); const loaded=node.tagName==='IMG' ? node.complete && node.naturalWidth>0 : node.readyState>=2 && node.videoWidth>0; if(completeOnly && shown && !complete) incomplete=true; if(!shown) continue; if(!loaded){if(incomplete) continue;return {error:'CONTENT_MEDIA_NOT_READY'};} if(!complete && !completeOnly) return {error:'TARGET_NOT_IN_VIEWPORT'}; const safeToken=value=>/^[A-Za-z][A-Za-z0-9_-]{0,95}$/.test(value || '') && !PRIVATE.test(value) && !/(?:\+?86[-\s]?)?1[3-9]\d{9}(?!\d)/.test(value); let selector=safeToken(node.id) ? '#'+CSS.escape(node.id) : null; for(const name of [...node.classList].filter(safeToken).slice(0,3)) if(!selector && root.querySelectorAll(node.tagName.toLowerCase()+'.'+CSS.escape(name)).length===1) selector=node.tagName.toLowerCase()+'.'+CSS.escape(name); if(!selector){ const parts=[]; for(let current=node;current && current!==root && parts.length<8;current=current.parentElement){ const tag=current.tagName.toLowerCase(),siblings=[...current.parentElement.children].filter(child=>child.tagName===current.tagName); parts.unshift(siblings.length>1 ? tag+':nth-of-type('+(siblings.indexOf(current)+1)+')' : tag); if(current.parentElement===root){selector=':scope > '+parts.join(' > ');break;} } } if(!selector || selector.length>1000) return {error:'UNSAFE_CONTENT_MEDIA'}; media.push({kind:node.tagName==='IMG' ? 'image' : 'video',selector}); } if(!text && !media.length){ if(incomplete && mediaNodes.length) return {incomplete:true,value:null}; return {error:'REQUIRED_FIELD_MISSING'}; } return {incomplete,value:{kind:text ? media.length ? 'mixed' : 'text' : 'media',text,media:media.slice(0,field.max_media ?? 4),media_truncated:media.length>(field.max_media ?? 4)}}; }; return elements.map((element, index) => { const rect = element.getBoundingClientRect(); const x = Math.max(0, Math.min(innerWidth - 1, (Math.max(0, rect.left) + Math.min(innerWidth, rect.right)) / 2)); const y = Math.max(0, Math.min(innerHeight - 1, (Math.max(0, rect.top) + Math.min(innerHeight, rect.bottom)) / 2)); const hit = document.elementFromPoint(x, y); const searchDescription = [element.getAttribute('type'), element.id, element.getAttribute('name'), element.getAttribute('aria-label'), element.getAttribute('placeholder'), element.getAttribute('title'), element.id ? document.querySelector(`label[for="${CSS.escape(element.id)}"]`)?.textContent : ''].filter(Boolean).join(' '); const form = element.form || element.closest('form'); const searchForm = form && (form.getAttribute('method') || 'get').toLowerCase() === 'get' && (/search|query|搜索|查询|检索/i.test([form.getAttribute('action'), form.getAttribute('role')].join(' ')) || [...form.querySelectorAll('button,input[type=submit]')].some(button => /search|搜索|查询|检索/i.test([button.textContent, button.getAttribute('aria-label'), button.getAttribute('value')].join(' ')))); const searchInput = element.matches('input[type=text],input[type=search],input:not([type]),textarea') && !sensitive(element) && !/comment|reply|message|私信|评论|回复|发言|留言|发送/i.test(searchDescription) && (!form || (form.getAttribute('method') || 'get').toLowerCase() === 'get') && (/search|query|keyword|搜索|查询|检索/i.test(searchDescription) || Boolean(searchForm)); const state = { index, tag: element.tagName.toLowerCase(), role:element.getAttribute('role'), sensitive: sensitive(element), forbidden: element.matches(excluded), search_input: searchInput, stable_label:searchInput ? stableFieldLabel(element) : '', readonly_root: !sensitive(element) && !element.matches(`${excluded},button,form`) && !element.closest('form'), visible: visible(element), in_viewport: viewport(element), unobscured: Boolean(hit && (hit === element || element.contains(hit))), fully_in_viewport:fullyViewport(element), busy:element.closest('[aria-busy="true"]')!==null || [...element.querySelectorAll('[aria-busy="true"],[role="progressbar"]')].some(visible), text: textOf(element), href: hrefOf(element), label: clean([element.textContent, element.getAttribute('aria-label'), element.getAttribute('title')].filter(Boolean).join(' ')), form_method: element.closest('form')?.getAttribute('method') || 'get' }; if(nativeProfile){ const role=element.getAttribute('role'),editable=element.isContentEditable || Boolean(element.closest('[contenteditable]')?.isContentEditable); const inForm=Boolean(element.closest('form')); const interactiveAncestor=Boolean(element.parentElement?.closest('button,a,[role="button"],[role="link"],[role="checkbox"],[role="radio"],[role="switch"],[role="menuitem"]')); state.native_context={ search_input:searchInput && !editable && (!form || (form.getAttribute('method') || 'get').toLowerCase()==='get') && ['search-input-in-feeds','search-input-ai'].includes(element.id) && !interactiveAncestor && [null,'searchbox','textbox','combobox'].includes(role), note_link:element instanceof HTMLAnchorElement && !inForm && !editable && !interactiveAncestor && [null,'link'].includes(role) && !element.hasAttribute('download') && ['', '_self'].includes(element.getAttribute('target') || ''), read_root:state.readonly_root && !editable && !['a','body','html'].includes(state.tag) && !['button','link','checkbox','radio','switch','combobox','textbox','menuitem'].includes(role) && !interactiveAncestor, }; if(navigationProfile){ // Fixed worker metadata only. Caller input cannot assert an anchor is // a profile/navigation link, and editable/nested controls stay out. const safeAnchor=element instanceof HTMLAnchorElement && !inForm && !editable && !interactiveAncestor && [null,'link'].includes(role) && !element.hasAttribute('download') && ['', '_self','_blank'].includes(element.getAttribute('target') || ''); let destination=null; try{ const url=new URL(element.href); if(url.origin===origin && !url.username && !url.password && !url.port) destination=url; }catch{} state.native_context.note_link=Boolean(safeAnchor && destination && /^\/(?:explore|search_result)\/[a-fA-F0-9]{24}$/.test(destination.pathname)); state.native_context.profile_link=Boolean(safeAnchor && destination && /^\/user\/profile\/[a-fA-F0-9]{24}$/.test(destination.pathname)); state.native_context.navigation_link=Boolean(safeAnchor && destination && ['/explore','/search_result','/search_result_ai','/notification'].includes(destination.pathname)); } } if (target) state.match_value = target.field === 'text' ? state.text : target.field === 'href' ? state.href : element.getAttribute(target.attribute); if (untilSelector) { const matches=[...element.querySelectorAll(untilSelector)]; state.until={count:matches.length,complete_count:0}; if(matches.length>200) state.until.error='READ_LIMIT'; else if(matches.some(child=>sensitive(child) || child.matches(`${excluded},button,form`) || child.closest('form'))) state.until.error='SENSITIVE_OUTPUT'; else state.until.complete_count=matches.filter(child=>fullyViewport(child,element)).length; } if (fields) { state.values = Object.create(null); for (const field of fields) { const matches = field.selector ? Array.from(element.querySelectorAll(field.selector)) : [element]; if (matches.length > 1) { state.error = 'NON_UNIQUE_FIELD'; break; } const child = matches[0]; let value = null; if (child && (child.matches(excluded) || sensitive(child))) { state.error = 'SENSITIVE_OUTPUT'; break; } const inView=child && viewport(child); const actualVisible=child && visible(child); if(field.source==='content'){ if(!child || !actualVisible) {state.error='REQUIRED_FIELD_MISSING';break;} if(child.matches('button,form') || child.closest('form')) {state.error='SENSITIVE_OUTPUT';break;} const content=contentOf(child,field,element); if(content.error){state.error=content.error;break;} if(content.incomplete) state.incomplete_fields=true; state.values[field.name]=content.value; continue; } if (child && actualVisible && (inView || (completeOnly && field.required))) value = field.source === 'text' ? textOf(child) : field.source === 'href' ? hrefOf(child) : child.getAttribute(field.attribute); if (value && typeof value === 'object') { state.error = value.error; break; } if (field.required && (value === null || clean(value) === '')) { state.error = 'REQUIRED_FIELD_MISSING'; break; } if (completeOnly && field.required && actualVisible && !fullyViewport(child)) { state.incomplete_fields=true; state.values[field.name]=null; continue; } if (!inView) value=null; state.values[field.name] = value === null ? null : clean(value); } } return state; }); } async function inspectLocator(session, selector, params) { // CSS is validated before the locator is constructed so Playwright's other // selector engines cannot be reached through this interface. return session.page.locator(selector).evaluateAll(inspectElements, { nativeProfile:isNativeUiProfile(session.executionProfile),navigationProfile:session.executionProfile===NAVIGATION_UI_PROFILE, origin: session.origin, target: params.target, fields: params.kind === 'extract' ? params.fields : undefined, completeOnly:params.complete_only===true,untilSelector:params.kind==='scroll' ? params.until_selector : undefined, }); } function resolveTarget(states, target) { if (!target) return states; if (typeof target.value !== 'string' || !target.value.trim()) throw new WorkerError('动态目标缺少实际匹配值', 'MISSING_PARAMETER'); const matches = states.filter(state => typeof state.match_value === 'string' && state.match_value === target.value.trim()); if (matches.length !== 1) throw new WorkerError(`动态目标当前匹配 ${matches.length} 个对象,必须唯一核对`, 'NON_UNIQUE_TARGET'); return matches; } function requireOperable(state, { viewport = true, obstruction = false, data = false } = {}) { if (state.sensitive || (data && state.forbidden)) throw new WorkerError('不能操作或读取敏感元素、脚本及表单值', 'SENSITIVE_OUTPUT'); if (!state.visible || (viewport && !state.in_viewport)) throw new WorkerError('目标当前不可见或不在视口,请先显式滚动并重新采集', 'TARGET_NOT_IN_VIEWPORT'); if (obstruction && !state.unobscured) throw new WorkerError('目标被其他元素遮挡,不能执行操作', 'TARGET_OBSCURED'); } function renderExtractValue(field,value){ if(value===null) return null; if(field?.source==='content') return {kind:value.kind,text:value.text===null ? null : redactText(value.text),media:value.media.map(item=>({...item})),media_truncated:value.media_truncated}; return field?.source==='href' ? redactUrl(value) : redactText(value); } function selectExtractRows(states, params, { completeRows = false } = {}) { if(states.some(state=>state.row_limit)) throw new WorkerError('单次最多检查 500 个候选对象,请缩小范围','READ_LIMIT'); const visibleStates=states.filter(state=>state.visible && state.in_viewport),selected=[]; let skippedIncomplete=0; for(const state of visibleStates){ if(selected.length>=params.limit) break; requireOperable(state,{data:true}); if(params.row_selector && state.readonly_root!==true) throw new WorkerError('容器内行不能属于交互或表单区域','SENSITIVE_OUTPUT'); if(state.error) throw new WorkerError(`结构化字段读取失败:${state.error}`,state.error); if(params.complete_only===true && (state.incomplete_fields || (completeRows && !state.fully_in_viewport))){skippedIncomplete+=1;continue;} selected.push(state); } const items=selected.map(state=>Object.fromEntries(Object.entries(state.values).map(([key,value])=> [key,renderExtractValue(params.fields.find(field=>field.name===key),value)]))); return {items,selected,visibleStates,skippedIncomplete,truncated:states.length>selected.length || items.some(item=>params.fields.some(field=>field.source==='content' && item[field.name]?.media_truncated))}; } async function extractInContainer(session, params, { observed, currentObserved, enableReadonlyPolicy, trace, initialBlockedWrites, noteBinding, batchBudgetMs=15_000 }) { const page=session.page,sourceUrl=page.url(),rootLocator=page.locator(params.selector); if(!observed || observed.sensitive || observed.unique!==true || !currentObserved || currentObserved.unique!==true || currentObserved.tag!==observed.tag || await rootLocator.count()!==1) { throw new WorkerError('容器提取必须使用最近观察到且当前仍唯一的普通容器','UNOBSERVED_CURRENT_SELECTOR'); } const root=await rootLocator.elementHandle({timeout:TIMEOUT}); const scrollSelector=params.batch?.scroll_selector; let scrollRoot; if(scrollSelector){ const scrollObserved=trace.before.elements.find(element=>element.selector===scrollSelector); const currentScroll=currentObserved.selector===scrollSelector ? currentObserved : await describeCurrentSelector(page,scrollSelector,scrollObserved); if(!scrollObserved || scrollObserved.sensitive || scrollObserved.unique!==true || !scrollObserved.scroll_y || !currentScroll || currentScroll.unique!==true || !currentScroll.scroll_y || await page.locator(scrollSelector).count()!==1) { throw new WorkerError('批次滚动根必须为最近观察到且当前唯一的普通容器','UNOBSERVED_CURRENT_SELECTOR'); } scrollRoot=await page.locator(scrollSelector).elementHandle({timeout:TIMEOUT}); } const rootState=async()=>{ requireBoundNotePage(session,noteBinding); if(session.page!==page || page.isClosed() || page.url()!==sourceUrl) throw new WorkerError('批次读取来源已变化,停止混合不同来源','SOURCE_CHANGED'); sessionFor(session.taskId); requireOrigin(session,page.url()); if(!root || !await root.evaluate((element,selector)=>element.isConnected && document.querySelectorAll(selector).length===1 && document.querySelector(selector)===element,params.selector)) { throw new WorkerError('读取容器对象已替换,必须重新观察','STALE_TARGET'); } const states=await root.evaluate(inspectElements,{origin:session.origin,nativeProfile:isNativeUiProfile(session.executionProfile)}); requireOperable(states[0],{data:true}); if(states[0].readonly_root!==true) throw new WorkerError('读取容器不能是敏感、交互或表单区域','SENSITIVE_OUTPUT'); if(isNativeUiProfile(session.executionProfile) && !nativeReadRootsValid(states)) throw new WorkerError('新模式不读取可交互或可编辑区域','SENSITIVE_OUTPUT'); if(scrollRoot){ const stable=await scrollRoot.evaluate((element,{selector,rootSelector,rowSelector})=>{ const container=document.querySelector(rootSelector),matches=document.querySelectorAll(selector),rows=container?.querySelectorAll(rowSelector); return element.isConnected && matches.length===1 && matches[0]===element && container?.contains(element) && rows.length<=500 && [...rows].every(row=>element.contains(row)); },{selector:scrollSelector,rootSelector:params.selector,rowSelector:params.row_selector}); if(!stable) throw new WorkerError('批次滚动容器或行的所属范围已变化','SOURCE_CHANGED'); const scrolling=await scrollRoot.evaluate(inspectElements,{origin:session.origin,nativeProfile:isNativeUiProfile(session.executionProfile)}); requireOperable(scrolling[0],{data:true}); if(scrolling[0].readonly_root!==true) throw new WorkerError('批次滚动根不能是敏感、交互或表单区域','SENSITIVE_OUTPUT'); if(isNativeUiProfile(session.executionProfile) && !nativeReadRootsValid(scrolling)) throw new WorkerError('新模式不滚动可交互或可编辑区域','SENSITIVE_OUTPUT'); if(!await scrollRoot.evaluate(element=>/(?:auto|scroll)/.test(getComputedStyle(element).overflowY) && element.clientHeight>0)) { throw new WorkerError('批次滚动根不再是已观察的纵向滚动容器','STALE_TARGET'); } } return states[0]; }; const inspectRows=()=>root.evaluate(inspectElements,{origin:session.origin,rowSelector:params.row_selector,fields:params.fields,completeOnly:params.complete_only===true}); const guardRequests=()=>{ if(session.blockedWrites>initialBlockedWrites) throw new WorkerError('页面尝试产生写请求,已阻止并停止本次只读步骤','WRITE_REQUEST_BLOCKED'); }; const initialRoot=await rootState(); enableReadonlyPolicy([initialRoot]); if(params.batch){ const startedMs=Date.now(),policyEnd=session.platformRequestPolicy?.expiresAt; const budgetMs=Math.max(1,Math.min(batchBudgetMs,15_000,Number.isFinite(policyEnd) ? policyEnd-startedMs : 15_000)); const reserve=Math.min(2_000,Math.max(1,Math.floor(budgetMs/4))); const deadline=startedMs+budgetMs-reserve,items=[],seen=new Map(); let duplicateCount=0,skippedIncomplete=0,stopReason='batch_budget'; let committedBatches=0; trace.batch_evidence=[]; const output=reason=>({items:structuredClone(items),count:items.length,truncated:true,source_url:noteBinding?.opened ? noteBinding.source_url : redactUrl(sourceUrl), coverage:!items.length ? 'empty' : items.length===params.limit ? 'limit_reached' : 'bounded',skipped_incomplete_count:skippedIncomplete, batch_summary:{batches_read:committedBatches,duplicates_removed:duplicateCount,stop_reason:reason}}); return withBatchDeadline(async control=>{ for(let index=1;index<=params.batch.max_batches;index+=1){ if(index>1 && Date.now()>=deadline){stopReason='time_budget';break;} control.check(); await rootState(); await refreshLoginDetection(session); sessionFor(session.taskId); const batchStates=await inspectRows(); const batch=selectExtractRows(batchStates,{...params,limit:100},{completeRows:true}); control.check(); let added=0,duplicates=0; const stagedItems=[...items],stagedSeen=new Map(seen); let conflict=false; for(const item of batch.items){ const id=item[params.batch.dedupe_field]; if(typeof id!=='string' || !id.trim() || id.includes('[已隐藏]') || id.includes('[手机号已隐藏]')) throw new WorkerError('批次业务标识为空或不适合稳定去重','UNSAFE_DEDUPE_ID'); if(stagedSeen.has(id)){ duplicates+=1; if(JSON.stringify(stagedSeen.get(id))!==JSON.stringify(item)) conflict=true; }else if(stagedItems.length=params.limit){stopReason='limit_reached';break;} if(Date.now()>=deadline){stopReason='time_budget';break;} const boundary=await scrollRoot.evaluate(element=>element.scrollTop+element.clientHeight>=element.scrollHeight-1); if(boundary){ const previousHeight=await scrollRoot.evaluate(element=>element.scrollHeight),pollEnd=Math.min(deadline,Date.now()+1_000); const fingerprint=states=>JSON.stringify(states.map(state=>({index:state.index,values:state.values,visible:state.in_viewport}))); const initialRows=fingerprint(batchStates); let rowsChanged=false; while(Date.now()element.scrollHeight)!==previousHeight) break; } if(rowsChanged && indexelement.scrollTop+element.clientHeight>=element.scrollHeight-1)) {stopReason='boundary';break;} } if(index===params.batch.max_batches) break; await rootState(); control.check(); await scrollRoot.evaluate((element,pixels)=>element.scrollBy({top:pixels,behavior:'instant'}),params.batch.pixels); await page.waitForTimeout(80); guardRequests(); } return output(stopReason); },{budgetMs,startedMs,cleanupReserveMs:reserve,onTimeout:()=>{ trace.batch_timeout=true; trace.batch_request_blocked=session.blockedWrites>initialBlockedWrites; session.closed=true;session.ready=false;session.lastCapture=null; endPlatformRequestPolicy(session.platformRequestPolicy); session.platformRequestPolicy=null;session.platformRequestPage=null;session.automating=false; const closeStart=performance.now(); session.context.once('close',()=>{trace.batch_context_closed=true;}); return closeContext(session).finally(()=>{trace.batch_close_ms=Math.max(0,Math.round(performance.now()-closeStart));}); }}); } const rows=await inspectRows(); if(rows.length===0 && params.allow_empty===true){ const proof=async()=>{ const state=await rootState(); await refreshLoginDetection(session); sessionFor(session.taskId); if(state.busy || !state.fully_in_viewport) throw new WorkerError('空态容器仍忙碌或未完整可见','EMPTY_STATE_UNCONFIRMED'); if((await inspectRows()).length!==0) throw new WorkerError('行集合已变化,不能确认空态','EMPTY_STATE_UNCONFIRMED'); const markers=await root.evaluate(inspectElements,{origin:session.origin,rowSelector:params.empty_state.selector}); if(markers.length!==1 || markers[0].readonly_root!==true || !markers[0].fully_in_viewport || markers[0].busy || markers[0].text!==params.empty_state.text.trim()) { throw new WorkerError('未观察到唯一、完整可见且文本精确一致的普通空态','EMPTY_STATE_UNCONFIRMED'); } if(redactText(markers[0].text)!==markers[0].text) throw new WorkerError('空态证据不能包含敏感信息','SENSITIVE_OUTPUT'); }; await proof(); await page.waitForTimeout(180); await proof(); guardRequests(); return {items:[],count:0,truncated:false,source_url:redactUrl(sourceUrl),coverage:'empty', ...(params.complete_only===true ? {skipped_incomplete_count:0} : {}), empty_state:{confirmed:true,selector:params.empty_state.selector,text:params.empty_state.text.trim()}}; } const selection=selectExtractRows(rows,params); return {items:selection.items,count:selection.items.length,truncated:selection.truncated,source_url:redactUrl(sourceUrl), coverage:!selection.items.length ? 'empty' : selection.truncated && selection.items.length===params.limit ? 'limit_reached' : 'bounded', ...(params.complete_only===true ? {skipped_incomplete_count:selection.skippedIncomplete} : {})}; } const seekStopReason=code=>({SCROLL_BOUNDARY_BEFORE_TARGET:'boundary',SCROLL_BUDGET_EXHAUSTED:'scroll_budget',SCROLL_TARGET_TIMEOUT:'time_budget',NON_UNIQUE_TARGET:'ambiguous',TARGET_OBSCURED:'obscured',STALE_TARGET:'target_changed',UNOBSERVED_CURRENT_SELECTOR:'target_changed',SCROLL_SOURCE_CHANGED:'source_changed',SENSITIVE_OUTPUT:'unsafe_target',UNSAFE_OUTPUT:'unsafe_target',WRITE_ACTION_BLOCKED:'unsafe_target',ACTION_MISMATCH:'unsafe_target',READ_LIMIT:'unsafe_target',NOTE_BINDING_MISMATCH:'binding_mismatch',WRITE_REQUEST_BLOCKED:'blocked_request'}[code] || 'session_unavailable'); export const snapshotSeekResults=results=>results.map(item=>({...item,movements:item.movements.map(movement=>({...movement}))})); const validViewport=value=>Boolean(value && Number.isFinite(value.width) && value.width>0 && Number.isFinite(value.height) && value.height>0 && Number.isFinite(value.scroll_x) && Number.isFinite(value.scroll_y) && value.scroll_y>=0); async function seekScroll(session,params,{observed,currentObserved,observedViewport,currentViewport,trace,enableReadonlyPolicy,initialBlockedWrites,noteBinding}) { const page=session.page,source=page.url(),rootLocator=page.locator(params.selector),deadline=Date.now()+params.timeout_ms; const viewport=params.scroll_scope==='viewport'; const summary=trace.seek_result={scrolls:0,movements:[],target_reached:false,stop_reason:'time_budget'}; if(!observed || observed.sensitive || observed.unique!==true || !currentObserved || currentObserved.sensitive || currentObserved.unique!==true || currentObserved.tag!==observed.tag || ['html','body'].includes(currentObserved.tag) || (viewport ? observed.kind!=='read' || currentObserved.kind!=='read' || !validViewport(observedViewport) || !validViewport(currentViewport) : typeof observed.scroll_y?.can_scroll!=='boolean' || typeof currentObserved.scroll_y?.can_scroll!=='boolean')) { throw new WorkerError('有界目标滚动必须使用已观察的唯一普通容器及对应滚动元信息','UNOBSERVED_CURRENT_SELECTOR'); } if(typeof params.until_target.value!=='string' || !params.until_target.value.trim()) throw new WorkerError('有界目标尚未提供实际匹配值','MISSING_PARAMETER'); const root=await rootLocator.elementHandle({timeout:TIMEOUT}); if(!root) throw new WorkerError('滚动容器已离开页面','STALE_TARGET'); const scroller=viewport ? (await page.evaluateHandle(()=>document.scrollingElement)).asElement() : null; if(viewport && !scroller) throw new WorkerError('页面没有可固定的实际视口滚动对象','ACTION_MISMATCH'); const checkViewport=async()=>{ const valid=await scroller.evaluate(element=>element.isConnected && element.ownerDocument===document && document.scrollingElement===element && (element===document.documentElement || element===document.body) && Number.isFinite(innerWidth) && innerWidth>0 && Number.isFinite(innerHeight) && innerHeight>0 && Number.isFinite(scrollX) && Number.isFinite(scrollY) && scrollY>=0 && Number.isFinite(element.scrollTop) && element.scrollTop>=0 && Number.isFinite(element.scrollHeight) && Number.isFinite(element.clientHeight) && element.clientHeight>0 && element.scrollHeight>=element.clientHeight); if(!valid) throw new WorkerError('视口滚动对象或实际元信息已变化','STALE_TARGET'); }; const check=async()=>{ if(session.page!==page || page.url()!==source) throw new WorkerError('有界滚动过程中页面来源已变化','SCROLL_SOURCE_CHANGED'); await refreshLoginDetection(session); sessionFor(session.taskId); requireBoundNotePage(session,noteBinding); if(session.blockedWrites>initialBlockedWrites) throw new WorkerError('页面尝试产生未知写请求,停止本次有界定位','WRITE_REQUEST_BLOCKED'); const same=await root.evaluate((element,selector)=>element.isConnected && document.querySelectorAll(selector).length===1 && document.querySelector(selector)===element,params.selector); if(!same) throw new WorkerError('滚动容器对象已经变化','STALE_TARGET'); if(viewport) await checkViewport(); const roots=await root.evaluate(inspectElements,{origin:session.origin,nativeProfile:isNativeUiProfile(session.executionProfile)}); if(roots.length!==1 || roots[0].readonly_root!==true) throw new WorkerError('滚动根不是普通读取区域','SENSITIVE_OUTPUT'); if(isNativeUiProfile(session.executionProfile) && !nativeReadRootsValid(roots)) throw new WorkerError('新模式不滚动可交互或可编辑区域','SENSITIVE_OUTPUT'); requireOperable(roots[0],{data:true}); if(!viewport && !await root.evaluate(element=>/^(?:auto|scroll)$/.test(getComputedStyle(element).overflowY))) throw new WorkerError('有界定位必须使用实际滚动容器','ACTION_MISMATCH'); enableReadonlyPolicy(roots); const candidates=await root.evaluate(inspectElements,{origin:session.origin,rowSelector:params.until_target.selector,target:params.until_target}); if(candidates.some(state=>state.row_limit)) throw new WorkerError('有界目标候选超过500个','READ_LIMIT'); const matches=candidates.filter(state=>typeof state.match_value==='string' && state.match_value===params.until_target.value.trim()); if(matches.length>1) throw new WorkerError('有界目标当前存在多个精确匹配,停止定位','NON_UNIQUE_TARGET'); if(!matches.length) return false; const target=matches[0]; if(target.sensitive || target.forbidden) throw new WorkerError('有界目标不是普通可操作元素','SENSITIVE_OUTPUT'); if(!['a','button'].includes(target.tag) && target.role!=='button') throw new WorkerError('有界目标不是链接或读取按钮','ACTION_MISMATCH'); if(target.href && typeof target.href!=='string') throw new WorkerError('有界目标来源不是允许的同站链接','UNSAFE_OUTPUT'); if(DANGEROUS.test(target.label) || DANGEROUS.test(target.href || '') || (target.tag!=='a' && (!READ_ONLY_BUTTON.test(target.label) || target.form_method.toLowerCase()!=='get'))) { throw new WorkerError('有界定位仅支持普通同站链接或明确读取按钮','WRITE_ACTION_BLOCKED'); } if(noteBinding){ const actual=boundNoteSource(target.href,session.origin); if(target.tag!=='a' || !actual || actual.note_id!==noteBinding.note_id) throw new WorkerError('有界定位目标与已选笔记编号不一致','NOTE_BINDING_MISMATCH'); } if(!target.fully_in_viewport) return false; requireOperable(target,{data:true,obstruction:true}); return true; }; const poll=async(ms)=>{ const until=Math.min(deadline,Date.now()+ms); do { if(await check()) return true; if(Date.now()>=until) return false; await page.waitForTimeout(Math.min(100,Math.max(1,until-Date.now()))); }while(true); }; let reached=await check(); for(let index=0;!reached && index=deadline) throw new WorkerError('有界定位超过本轮时间预算','SCROLL_TARGET_TIMEOUT'); // Recheck the pinned root and candidate immediately before each fixed movement. if(await check()){reached=true;break;} const movement=viewport ? await scroller.evaluate((element,{root,selector,direction,pixels})=>{ if(!element.isConnected || document.scrollingElement!==element || !root.isConnected || document.querySelectorAll(selector).length!==1 || document.querySelector(selector)!==root) return {error:true}; const before=element.scrollTop; window.scrollBy({top:direction==='down' ? pixels : -pixels,behavior:'instant'}); return {before,after:element.scrollTop,scroll_height:element.scrollHeight,client_height:element.clientHeight}; },{root,selector:params.selector,direction:params.direction,pixels:params.pixels}) : await root.evaluate((element,{direction,pixels})=>{ const before=element.scrollTop; element.scrollBy({top:direction==='down' ? pixels : -pixels,behavior:'instant'}); return {before,after:element.scrollTop,scroll_height:element.scrollHeight,client_height:element.clientHeight}; },params); if(movement.error) throw new WorkerError('滚动前视口或普通目标根已经变化','STALE_TARGET'); summary.movements.push(movement);summary.scrolls=summary.movements.length; await page.waitForTimeout(Math.min(80,Math.max(1,deadline-Date.now()))); reached=await check(); if(!reached && movement.before===movement.after){ reached=await poll(1000); if(!reached) throw new WorkerError('已到滚动边界,限定等待后目标仍未完整可操作','SCROLL_BOUNDARY_BEFORE_TARGET'); } } if(!reached) reached=await poll(1000); if(!reached) throw new WorkerError('有界定位尚未找到完整目标',Date.now()>=deadline ? 'SCROLL_TARGET_TIMEOUT' : 'SCROLL_BUDGET_EXHAUSTED'); summary.target_reached=true;summary.stop_reason='target_reached'; return {direction:params.direction,pixels:params.pixels,requested_max_scrolls:params.max_scrolls,scrolls:summary.scrolls,movements:summary.movements, brought_into_view:false,target_reached:true,stop_reason:'target_reached',timeout_ms:params.timeout_ms,source_url:redactUrl(page.url())}; } // Preflight the already existing policy before a wait can enable automation. // A missing observed collection is a learning precondition, not permission to // carry search authority into another step or to widen network exceptions. export function preflightReadonlyWait(context) { if(context.kind!=='wait' || context.sessionOrigin!=='https://www.xiaohongshu.com') return null; if(Array.isArray(context.actualRoots) && context.actualRoots.some(root=>root?.sensitive || root?.forbidden)) { throw new WorkerError('不能等待敏感元素或表单区域','SENSITIVE_OUTPUT'); } const policy=beginReadonlyRequestPolicy(context); if(!policy) throw new WorkerError('等待目标尚未满足已观察普通根与只读策略前置条件,请先观察并学习加载过渡','READONLY_WAIT_NOT_READY'); return policy; } async function prepareScopedWait(session,params,previous,noteBinding) { const page=session.page,source=page.url(),selector=params.wait_scope; const observed=previous.elements.find(element=>element.selector===selector); if(!observed || observed.kind!=='read' || observed.unique!==true || observed.sensitive || !observed.in_viewport || /^(?:body|html)$/i.test(observed.tag || '') || await page.locator(selector).count()!==1) throw new WorkerError('范围等待须使用最新观察中唯一可见的普通容器','UNOBSERVED_CURRENT_SELECTOR'); const current=await describeCurrentSelector(page,selector,observed); if(!current || current.unique!==true || current.kind!=='read' || current.tag!==observed.tag) throw new WorkerError('范围等待根已变化,请重新观察','UNOBSERVED_CURRENT_SELECTOR'); const root=await page.locator(selector).elementHandle({timeout:TIMEOUT}); const check=async()=>{ sessionFor(session.taskId);requireBoundNotePage(session,noteBinding); if(session.page!==page || page.isClosed() || page.url()!==source) throw new WorkerError('范围等待页面来源已变化','SOURCE_CHANGED'); if(!root || !await root.evaluate((element,css)=>element.isConnected && document.querySelectorAll(css).length===1 && document.querySelector(css)===element,selector)) { throw new WorkerError('范围等待的固定容器已替换','SOURCE_CHANGED'); } const states=await root.evaluate(inspectElements,{origin:session.origin,fixedRootSelector:selector,fixedSourceUrl:source,nativeProfile:isNativeUiProfile(session.executionProfile)}); if(states.some(state=>state.scope_changed) || session.page!==page || page.url()!==source) throw new WorkerError('范围等待的固定容器或来源已变化','SOURCE_CHANGED'); requireOperable(states[0],{data:true}); if(states[0].readonly_root!==true) throw new WorkerError('范围等待根不能是表单或交互区域','SENSITIVE_OUTPUT'); if(isNativeUiProfile(session.executionProfile) && !nativeReadRootsValid(states)) throw new WorkerError('新模式等待根不能是可交互或可编辑区域','SENSITIVE_OUTPUT'); return states[0]; }; await check(); return {observed,check,async rows(){ const rows=await root.evaluate(inspectElements,{origin:session.origin,rowSelector:params.selector,fixedRootSelector:selector,fixedSourceUrl:source}); if(rows.some(row=>row.scope_changed)) throw new WorkerError('范围等待的固定容器或来源已变化','SOURCE_CHANGED'); if(rows.some(row=>row.row_limit) || rows.length>100) throw new WorkerError('范围等待最多核对100个集合对象','READ_LIMIT'); if(rows.some(row=>row.sensitive || row.forbidden || row.readonly_root!==true)) throw new WorkerError('范围等待集合包含敏感或交互对象','SENSITIVE_OUTPUT'); return rows; }}; } async function waitInScope(session,params,scope,{trace,initialBlockedWrites,noteBinding}) { const startedMs=Date.now(),policyEnd=session.platformRequestPolicy?.expiresAt; const budgetMs=Math.max(1,Math.min(params.timeout_ms,Number.isFinite(policyEnd) ? policyEnd-startedMs : params.timeout_ms)); const guard=()=>{if(session.blockedWrites>initialBlockedWrites) throw new WorkerError('范围等待期间已阻止未知请求','WRITE_REQUEST_BLOCKED');}; return withWaitDeadline(async control=>{ for(;;){ control.check();await scope.check(); await refreshLoginDetection(session);sessionFor(session.taskId); const rows=await scope.rows(); await scope.check();guard();control.check(); if(rows.filter(row=>row.visible && row.in_viewport && !row.busy).length>=params.min_count){ return {satisfied:true,state:'visible',selector:params.selector,wait_scope:params.wait_scope,min_count:params.min_count,timeout_ms:params.timeout_ms}; } await session.page.waitForTimeout(80); } },{budgetMs,startedMs,cleanupReserveMs:Math.min(2_000,Math.max(1,Math.floor(budgetMs/4))),onTimeout:()=>{ trace.wait_timeout=true;trace.wait_request_blocked=session.blockedWrites>initialBlockedWrites; session.closed=true;session.ready=false;session.lastCapture=null; endPlatformRequestPolicy(session.platformRequestPolicy); session.platformRequestPolicy=null;session.platformRequestPage=null;session.automating=false; const closeStart=performance.now(); session.context.once('close',()=>{trace.wait_context_closed=true;}); return closeContext(session).finally(()=>{trace.wait_close_ms=Math.max(0,Math.round(performance.now()-closeStart));}); }}); } async function performActionSession(session, params, { captureBefore = true, trace, noteBinding, batchBudgetMs=15_000 } = {}) { if (shuttingDown) throw new WorkerError('学习工作进程正在退出', 'WORKER_CLOSING'); const kind = params.kind; validateAction(params); requireBoundNotePage(session,noteBinding); const selector = requireString(params.selector, 'selector', 1_000); const previous = session.lastCapture; if (!previous || previous.url !== redactUrl(session.page.url())) { throw new WorkerError('页面已变化或尚未采集,请先重新采集当前页面', 'FRESH_CAPTURE_REQUIRED'); } const observed = previous.elements.find(element => element.selector === selector); if (kind === 'wait' && params.state === 'hidden' && selector !== 'body' && !session.observedSelectors.has(selector)) { throw new WorkerError('消失等待需要本会话已采集过的普通元素', 'UNOBSERVED_SELECTOR'); } const flexible = ['wait', 'extract'].includes(kind) || Boolean(params.target) || (kind === 'scroll' && selector === 'body'); if (!flexible && (!observed || observed.sensitive)) throw new WorkerError('定位必须来自最近一次页面采集的可用元素', 'UNOBSERVED_SELECTOR'); if (kind === 'fill' && observed?.kind !== 'fill') throw new WorkerError('该元素不是可填写的普通输入框', 'ACTION_MISMATCH'); if (kind === 'press' && (observed?.kind !== 'fill' || !observed.search_input)) throw new WorkerError('Enter 仅适用于已观察的普通搜索输入框,不能提交评论或消息', 'WRITE_ACTION_BLOCKED'); if (kind === 'click' && !params.target && observed?.kind !== 'click') throw new WorkerError('该元素不是已观察到的链接或按钮', 'ACTION_MISMATCH'); let locator = session.page.locator(selector); const count = params.wait_scope ? 0 : await locator.count(); if (kind !== 'wait' && ((!count && kind !== 'extract') || (!['read', 'extract'].includes(kind) && !params.target && count !== 1))) { throw new WorkerError(`定位当前匹配 ${count} 个元素,操作要求唯一目标`, 'NON_UNIQUE_TARGET'); } if (kind === 'read' && count > 200) throw new WorkerError('单次读取最多 200 个对象,请缩小定位范围', 'READ_LIMIT'); if (count > 500) throw new WorkerError('单次最多检查 500 个候选对象,请缩小范围', 'READ_LIMIT'); trace.phase = 'inspect_current'; const currentObserved = params.wait_scope ? null : await describeCurrentSelector(session.page,selector,observed); const currentViewport=kind==='scroll' && params.scroll_scope==='viewport' ? await session.page.evaluate(()=>({width:innerWidth,height:innerHeight,scroll_x:scrollX,scroll_y:scrollY})) : null; if (!flexible && !currentObserved) throw new WorkerError('本次页面采集未返回该定位,请重新采集并核对当前控件', 'UNOBSERVED_CURRENT_SELECTOR'); const searchAction = ['fill','press'].includes(kind) && observed?.search_input===true; const sameStableSearch = searchAction && currentObserved?.search_input===true && observed.unique===true && currentObserved.unique===true && currentObserved.tag===observed.tag && typeof observed.stable_label==='string' && observed.stable_label.length>0 && currentObserved.stable_label===observed.stable_label; if (!flexible && ((!['read', 'scroll'].includes(kind) && (currentObserved.label !== observed.label || currentObserved.tag !== observed.tag) && !sameStableSearch) || (searchAction && observed.stable_label && !sameStableSearch))) { throw new WorkerError('目标语义已变化,请重新采集后确认', 'STALE_TARGET'); } if(captureBefore) trace.phase = 'capture_before'; const before = trace.before = captureBefore ? await captureSession(session) : previous; trace.phase = 'resolve_target'; requireBoundNotePage(session,noteBinding); let states = params.wait_scope ? [] : resolveTarget(await inspectLocator(session, selector, params.row_selector ? {...params,kind:'read'} : params), params.target); requireBoundNotePage(session,noteBinding); if (searchAction && observed.stable_label && (states.length!==1 || states[0].search_input!==true || states[0].stable_label!==observed.stable_label)) { throw new WorkerError('搜索控件的固定语义已变化,请重新采集并核对', 'STALE_TARGET'); } if (params.target) locator = locator.nth(states[0].index); let objectHandle; let targetResolvedUnique = false; if (['fill', 'press', 'click'].includes(kind) || (kind === 'read' && params.target)) { // Pin the object itself: resolving an nth locator again during a rerender // must never silently redirect a click to a different row. objectHandle = await locator.elementHandle({ timeout: TIMEOUT }); if (!objectHandle) throw new WorkerError('目标对象已离开页面,请重新采集', 'STALE_TARGET'); const pinned = await objectHandle.evaluate(inspectElements, { origin: session.origin, target: params.target,nativeProfile:isNativeUiProfile(session.executionProfile),navigationProfile:session.executionProfile===NAVIGATION_UI_PROFILE }); if (params.target) { resolveTarget(pinned, params.target); targetResolvedUnique = states.length === 1 && pinned.length === 1; } else if (pinned[0].tag !== states[0].tag || pinned[0].label !== states[0].label || (searchAction && pinned[0].stable_label!==states[0].stable_label)) throw new WorkerError('目标在执行前发生变化', 'STALE_TARGET'); states = pinned; } if (['fill', 'press', 'click'].includes(kind)) requireOperable(states[0], { obstruction: true }); if (kind === 'press' && !states[0].search_input) throw new WorkerError('当前目标已不再是普通搜索输入框', 'WRITE_ACTION_BLOCKED'); if (kind === 'click' && !['a', 'button'].includes(states[0].tag)) { const role = await locator.getAttribute('role'); if (role !== 'button') throw new WorkerError('动态目标不是链接或按钮', 'ACTION_MISMATCH'); } if (kind === 'extract') states.forEach(state => { if (state.sensitive || state.forbidden) throw new WorkerError('结构化行不能是敏感元素、脚本或表单控件', 'SENSITIVE_OUTPUT'); }); sessionFor(session.taskId); const scopedWait=params.wait_scope ? await prepareScopedWait(session,params,previous,noteBinding) : null; const nativeProfile=isNativeUiProfile(session.executionProfile); let nativeContext; if(nativeProfile){ const nativeRoots=['fill','press','click'].includes(kind) ? states : scopedWait ? [await scopedWait.check()] : await inspectLocator(session,selector,{kind:'read'}); nativeContext={executionProfile:session.executionProfile,sessionOrigin:session.origin,targetResolvedUnique,pageUrl:session.page.url(),kind,selector,key:params.key,observed, actualRoots:nativeRoots,matchCount:params.target ? 1 : count,href:kind==='click' ? states[0]?.href : undefined, noteId:noteBinding?.note_id,keyword:kind==='fill' ? params.value : undefined}; if(kind==='press'){ // Check the exact ordinary search control before reading its business // search value; never inspect arbitrary form inputs for this purpose. assertNativeUiAction({...nativeContext,keyword:'ordinary_search_precheck'}); nativeContext.keyword=await objectHandle.inputValue({timeout:TIMEOUT}); } assertNativeUiAction(nativeContext); } const initialBlockedWrites = session.blockedWrites; const readonlyContext = roots=>({sessionOrigin:session.origin,pageUrl:session.page.url(),kind,selector:params.wait_scope || selector, observed:scopedWait?.observed || observed,scrollScope:params.scroll_scope,untilTarget:params.until_target, selectorObserved:scopedWait ? true : Boolean(observed && !observed.sensitive) || (kind==='wait' && params.state==='hidden' && session.observedSelectors.has(selector)),actualRoots:roots}); const enableReadonlyPolicy = roots => { if(nativeProfile) assertNativeUiAction({...nativeContext,actualRoots:roots}); if (session.platformRequestPolicy) return; session.platformRequestPolicy = nativeProfile ? beginNativeUiRequestPolicy({...nativeContext,actualRoots:roots}) : beginReadonlyRequestPolicy(readonlyContext(roots)); if(nativeProfile && !session.platformRequestPolicy) throw new WorkerError('当前动作未满足有界网页操作合同','NATIVE_UI_POLICY_NOT_READY'); session.platformRequestPage = session.platformRequestPolicy ? session.page : null; }; if(nativeProfile || kind==='wait'){ // Authorize the already checked action before enabling automation. There // must be no interval in which normal page traffic sees a missing policy. session.platformRequestPolicy=nativeProfile ? beginNativeUiRequestPolicy(nativeContext) : preflightReadonlyWait(readonlyContext(scopedWait ? [await scopedWait.check()] : states)); if(nativeProfile && !session.platformRequestPolicy) throw new WorkerError('当前动作未满足有界网页操作合同','NATIVE_UI_POLICY_NOT_READY'); session.platformRequestPage=session.platformRequestPolicy ? session.page : null; } session.automating = true; let result; try { trace.phase = 'execute'; requireBoundNotePage(session,noteBinding); if (kind === 'fill') { if (typeof params.value !== 'string' || params.value.length > 2_000) throw new WorkerError('填写值必须是最多 2000 字符的文本', 'INVALID_INPUT'); if (/(?:\+?86[-\s]?)?1[3-9]\d{9}|(?:验证码|密码|password)\s*[::]/i.test(params.value)) { throw new WorkerError('此学习步骤不接受平台手机号、密码或验证码', 'SENSITIVE_INPUT'); } if(!nativeProfile) session.platformRequestPolicy = beginSearchRequestPolicy({ sessionOrigin: session.origin, pageUrl: session.page.url(), kind, observed, actual: states[0], matchCount: count }); session.platformRequestPage = session.platformRequestPolicy ? session.page : null; await objectHandle.fill(params.value, { timeout: TIMEOUT }); result = { filled: true, selector }; } else if (kind === 'press') { if(!nativeProfile) session.platformRequestPolicy = beginSearchRequestPolicy({ sessionOrigin: session.origin, pageUrl: session.page.url(), kind, key: params.key, observed, actual: states[0], matchCount: count }); session.platformRequestPage = session.platformRequestPolicy ? session.page : null; await objectHandle.press('Enter', { timeout: TIMEOUT }); trace.phase = 'settle'; await settle(session.page); await settlePopups(session); requireOrigin(session, session.page.url()); if(nativeProfile){ const deadline=Date.now()+TIMEOUT; while(nativeSearchKeyword(session.page.url())!==nativeContext.keyword.trim()){ sessionFor(session.taskId); if(session.blockedWrites>initialBlockedWrites) throw new WorkerError('搜索过渡期间有请求超出当前动作范围','WRITE_REQUEST_BLOCKED'); if(Date.now()>=deadline) throw new WorkerError('搜索未在预算内进入对应关键词结果页','SEARCH_TRANSITION_NOT_READY'); await session.page.waitForTimeout(80); } } result = { pressed: true, key: 'Enter', selector, url: redactUrl(session.page.url()) }; } else if (kind === 'click') { const target = await objectHandle.evaluate(element => ({ tag: element.tagName.toLowerCase(), label: [element.textContent, element.getAttribute('aria-label'), element.getAttribute('title')].filter(Boolean).join(' ').trim(), href: element instanceof HTMLAnchorElement ? element.href : '', formMethod: element.closest('form')?.getAttribute('method') || 'get', })); if(nativeProfile){ const pinned=await objectHandle.evaluate(inspectElements,{origin:session.origin,nativeProfile:true,navigationProfile:session.executionProfile===NAVIGATION_UI_PROFILE}); assertNativeUiAction({...nativeContext,actualRoots:pinned,href:target.href}); if(target.href!==nativeContext.href) throw new WorkerError('实际笔记链接在点击前已变化','STALE_TARGET'); } if(noteBinding){ const actual=target.tag==='a' ? boundNoteSource(target.href,session.origin) : null; if(!actual || actual.note_id!==noteBinding.note_id) throw new WorkerError('唯一标题当前链接不是本次已选笔记,未执行点击','NOTE_BINDING_MISMATCH'); noteBinding.href_checked=true; } if (DANGEROUS.test(target.label) || DANGEROUS.test(target.href)) { throw new WorkerError('首版学习验证不执行关注、私信、发送、提交、支付或删除动作', 'WRITE_ACTION_BLOCKED'); } if (target.tag === 'a' && target.href) { const url = httpUrl(target.href); requireOrigin(session, url.href); } else if (!READ_ONLY_BUTTON.test(target.label) || target.formMethod.toLowerCase() !== 'get') { throw new WorkerError('首版仅允许同站点链接或明确的搜索、查询、展开、加载按钮', 'WRITE_ACTION_BLOCKED'); } session.popupPermission = { opener: session.page, expectedUrl: target.tag === 'a' ? target.href : null, adopted: false, ...(session.executionProfile===NAVIGATION_UI_PROFILE ? {sourceUrl:session.page.url()} : {}) }; if(!nativeProfile) session.platformRequestPolicy = beginNoteOpenRequestPolicy({ sessionOrigin: session.origin, pageUrl: session.page.url(), kind, observed, actual: states[0], matchCount: targetResolvedUnique ? 1 : count, href: target.href, targetResolvedUnique }); session.platformRequestPage = session.platformRequestPolicy ? session.page : null; await objectHandle.click({ timeout: TIMEOUT }); trace.phase = 'settle'; await settle(session.page); await settlePopups(session); await settle(session.page); requireOrigin(session, session.page.url()); if(noteBinding){noteBinding.opened=true;requireBoundNotePage(session,noteBinding);} result = { clicked: true, selector, url: redactUrl(session.page.url()) }; } else if (kind === 'read') { result = states.map(state => { requireOperable(state, { data:true, viewport:Boolean(params.target) }); if (state.href && typeof state.href !== 'string') throw new WorkerError('读取来源不是允许的同源链接', 'UNSAFE_OUTPUT'); return { text:redactText(state.text || ''), ...(state.href ? {href:redactUrl(state.href)} : {}) }; }); enableReadonlyPolicy(states); } else if (kind === 'scroll' && params.until_target) { result=await seekScroll(session,params,{observed,currentObserved,observedViewport:previous.viewport,currentViewport,trace,enableReadonlyPolicy,initialBlockedWrites,noteBinding}); } else if (kind === 'scroll') { if(params.until_selector && states.some(state=>state.until?.error)) { const invalid=states.find(state=>state.until?.error); throw new WorkerError('滚动目标不是可读取的普通有界元素',invalid.until.error); } if (selector !== 'body') { requireOperable(states[0], { viewport: false, data: true }); enableReadonlyPolicy(states); await locator.scrollIntoViewIfNeeded({ timeout: TIMEOUT }); } else enableReadonlyPolicy(states); const movements = []; const untilReached=async(pollMs=0)=>{ const deadline=Date.now()+pollMs; do { const current=await inspectLocator(session,selector,params); if(current.length!==1) throw new WorkerError('滚动容器已变化,必须保持唯一','STALE_TARGET'); if(current[0].readonly_root!==true) throw new WorkerError('滚动容器已变为敏感或表单区域','SENSITIVE_OUTPUT'); const until=current[0].until; if(until?.error) throw new WorkerError('滚动目标不是可读取的普通有界元素',until.error); if(until?.complete_count>0) return true; if(Date.now()>=deadline) return false; await session.page.waitForTimeout(Math.min(100,Math.max(1,deadline-Date.now()))); } while(true); }; let reached=params.until_selector ? await untilReached() : false; for (let index = 0; index < params.max_scrolls && !reached; index += 1) { const movement = await locator.evaluate((element, { direction, pixels, pageScroll }) => { const root = pageScroll ? document.scrollingElement : element; const before = root.scrollTop; root.scrollBy({ top: direction === 'down' ? pixels : -pixels, behavior: 'instant' }); return { before, after: root.scrollTop, scroll_height: root.scrollHeight, client_height: root.clientHeight }; }, { ...params, pageScroll: selector === 'body' }); movements.push(movement); await session.page.waitForTimeout(80); if(params.until_selector) reached=await untilReached(); if (movement.before === movement.after) { if(params.until_selector && !reached) reached=await untilReached(1000); if(params.until_selector && !reached) throw new WorkerError('已到滚动边界,限定等待后目标仍未完整可见','SCROLL_BOUNDARY_BEFORE_TARGET'); break; } } if(params.until_selector && !reached) reached=await untilReached(1000); if(params.until_selector && !reached) throw new WorkerError('本次滚动预算已耗尽,目标尚未完整可见','SCROLL_BUDGET_EXHAUSTED'); result = { direction: params.direction, pixels: params.pixels, requested_max_scrolls: params.max_scrolls, scrolls: movements.length, movements, brought_into_view: selector !== 'body' && !states[0].in_viewport, ...(params.until_selector ? {until_selector:params.until_selector,until_reached:reached} : {}), source_url: redactUrl(session.page.url()) }; } else if (kind === 'wait' && scopedWait) { result=await waitInScope(session,params,scopedWait,{trace,initialBlockedWrites,noteBinding}); } else if (kind === 'wait') { if (states.some(state=>state.sensitive || state.forbidden)) throw new WorkerError('不能观察敏感元素或表单值', 'SENSITIVE_OUTPUT'); if(params.min_count!==undefined && states.length>100) throw new WorkerError('集合等待最多核对100个候选对象','READ_LIMIT'); if (params.state!=='hidden' && params.min_count===undefined && states.length>1) throw new WorkerError('条件等待必须定位唯一对象', 'NON_UNIQUE_TARGET'); enableReadonlyPolicy(states); const deadline = Date.now() + params.timeout_ms; let satisfied = false; do { const currentStates = await inspectLocator(session, selector, params); if (currentStates.length > 500) throw new WorkerError('等待目标范围过大', 'READ_LIMIT'); if (currentStates.some(state => state.sensitive || state.forbidden)) throw new WorkerError('不能观察敏感元素或表单值', 'SENSITIVE_OUTPUT'); if (params.state === 'hidden') satisfied = currentStates.length === 0 || currentStates.every(state => !state.visible); else { if(params.min_count!==undefined){ if(currentStates.length>100) throw new WorkerError('集合等待最多核对100个候选对象','READ_LIMIT'); satisfied=currentStates.filter(state=>state.visible && state.in_viewport).length>=params.min_count; }else { if (currentStates.length > 1) throw new WorkerError('条件等待必须定位唯一对象', 'NON_UNIQUE_TARGET'); const state = currentStates[0]; satisfied = Boolean(state?.visible && (params.state !== 'text' || state.text?.includes(params.text))); } } enableReadonlyPolicy(currentStates); if (satisfied) break; if (Date.now() >= deadline) throw new WorkerError('条件等待超时,当前页面尚未达到预期状态', 'WAIT_TIMEOUT'); await session.page.waitForTimeout(Math.min(100, Math.max(1, deadline - Date.now()))); } while (true); result = { satisfied, state: params.state, selector, timeout_ms: params.timeout_ms, ...(params.min_count!==undefined ? {min_count:params.min_count} : {}) }; } else if (kind === 'extract') { if(params.row_selector){ result=await extractInContainer(session,params,{observed,currentObserved,enableReadonlyPolicy,trace,initialBlockedWrites,noteBinding,batchBudgetMs}); }else { const visibleStates = states.filter(state => state.visible && state.in_viewport); if (params.target && !visibleStates.length) throw new WorkerError('动态行当前不在视口,请先显式滚动', 'TARGET_NOT_IN_VIEWPORT'); const selected=[]; let skippedIncomplete=0; for (const state of visibleStates) { if(selected.length>=params.limit) break; requireOperable(state, { data: true }); if (state.error) throw new WorkerError(`结构化字段读取失败:${state.error}`, state.error); if(params.complete_only===true && state.incomplete_fields){skippedIncomplete+=1;continue;} selected.push(state); } const items = selected.map(state => Object.fromEntries(Object.entries(state.values).map(([key, value]) => [key,renderExtractValue(params.fields.find(field=>field.name===key),value)]))); enableReadonlyPolicy(selected.length ? selected : visibleStates); const truncated = states.length > selected.length || items.some(item=>params.fields.some(field=>field.source==='content' && item[field.name]?.media_truncated)); result = { items, count: items.length, truncated, source_url: redactUrl(session.page.url()), coverage: !items.length ? 'empty' : truncated && items.length === params.limit ? 'limit_reached' : 'bounded', ...(params.complete_only===true ? {skipped_incomplete_count:skippedIncomplete} : {}) }; } } if(noteBinding?.opened){ requireBoundNotePage(session,noteBinding); if(result && typeof result==='object' && Object.hasOwn(result,'source_url')) result.source_url=noteBinding.source_url; if(kind==='extract' && params.output_name==='note_detail'){ noteBinding.title=typeof result.items?.[0]?.title==='string' ? result.items[0].title : null; if(noteBinding.title?.trim()!==noteBinding.note_title) throw new WorkerError('实际读取笔记标题与本次已选标题不一致,停止后续评论读取','NOTE_BINDING_MISMATCH'); } } trace.phase = 'settle'; await session.page.waitForTimeout(120); await settlePopups(session); requireBoundNotePage(session,noteBinding); if(nativeProfile) assertNativeUiResultLocation(nativeContext,session.page.url()); trace.phase = 'request_guard'; if (session.blockedWrites > initialBlockedWrites) throw new WorkerError('页面尝试产生写请求,已阻止并停止本次只读步骤', 'WRITE_REQUEST_BLOCKED'); } finally { endPlatformRequestPolicy(session.platformRequestPolicy); session.platformRequestPolicy = null; session.platformRequestPage = null; session.automating = false; } trace.phase = 'capture_after'; const after = trace.after = await captureSession(session); requireBoundNotePage(session,noteBinding); if(nativeProfile) assertNativeUiResultLocation(nativeContext,session.page.url()); if(scopedWait){ trace.phase='inspect_current'; await scopedWait.check(); const rows=await scopedWait.rows(); if(rows.filter(row=>row.visible && row.in_viewport && !row.busy).length null); } if (trace.before) error.before = trace.before; if (trace.after) error.after = trace.after; if(trace.batch_evidence) error.batch_evidence=trace.batch_evidence; if(trace.batch_context_closed===true || trace.wait_context_closed===true) error.session_closed=true; if(trace.batch_request_blocked===true || trace.wait_request_blocked===true) error.error_codes=['WRITE_REQUEST_BLOCKED']; if(trace.seek_result){ error.seek_result={...trace.seek_result,target_reached:false,stop_reason:seekStopReason(error.code)}; } error.diagnostics = structuredClone(diagnostics); throw error; } finally { endPlatformRequestPolicy(session.platformRequestPolicy); session.platformRequestPolicy = null; session.platformRequestPage = null; session.automating = false; session.popupPermission = null; session.actionDiagnostics = null; if (session.page.isClosed() && !session.closed) { session.closed = true; session.ready = false; session.lastCapture = null; void closeContext(session); } } } async function action(params) { return actionSession(sessionFor(params.task_id), params); } async function close(params) { const taskId = requireString(params.task_id, 'task_id', 128); const session = sessions.get(taskId); if (!session) return { closed: true, profile_preserved: true }; session.closed = true; session.ready = false; session.lastCapture = null; await closeContext(session); sessions.delete(taskId); if (accountOwners.get(session.accountId) === taskId) accountOwners.delete(session.accountId); return { closed: true, profile_preserved: true }; } async function closeContext(session) { if (closingAccounts.has(session.accountId)) return closingAccounts.get(session.accountId); const promise = session.context.close().catch(() => {}).finally(() => { if (accountOwners.get(session.accountId) === session.taskId) accountOwners.delete(session.accountId); closingAccounts.delete(session.accountId); }); closingAccounts.set(session.accountId, promise); return promise; } const emptyExpectation=value=>value && typeof value==='object' && !Array.isArray(value) && Object.keys(value).length===1 && value.state==='empty'; const nonemptyExpectation=value=>value && typeof value==='object' && !Array.isArray(value) && Object.keys(value).length===1 && value.state==='nonempty'; export function validContentValue(field,value){ if(!value || typeof value!=='object' || Array.isArray(value) || Object.keys(value).length!==4 || Object.keys(value).some(key=>!['kind','text','media','media_truncated'].includes(key)) || !['text','media','mixed'].includes(value.kind) || (value.text!==null && (typeof value.text!=='string' || !value.text.trim())) || !Array.isArray(value.media) || value.media.length>(field.max_media ?? 4) || typeof value.media_truncated!=='boolean') return false; if(value.kind!==(value.text ? value.media.length ? 'mixed' : 'text' : 'media') || (!value.text && !value.media.length) || (value.media_truncated && value.media.length!==(field.max_media ?? 4))) return false; const selectors=new Set(); for(const media of value.media){ if(!media || typeof media!=='object' || Array.isArray(media) || Object.keys(media).length!==2 || !['image','video'].includes(media.kind) || !Object.hasOwn(media,'selector') || selectors.has(media.selector)) return false; try{validateSelector(media.selector,{relative:true});}catch{return false;} selectors.add(media.selector); } return true; } // This checks the extraction contract and result structure only. It deliberately // makes no claim that a title, author or comment matches a business requirement. export function validNonemptyExtractOutput(step,output){ if(!step || step.kind!=='extract') return false; try {validateAction(step);}catch{return false;} if(!output || typeof output!=='object' || Array.isArray(output)) return false; const keys=['items','count','truncated','source_url','coverage',...(step.complete_only===true ? ['skipped_incomplete_count'] : []),...(step.batch ? ['batch_summary'] : [])]; if(Object.keys(output).length!==keys.length || Object.keys(output).some(key=>!keys.includes(key)) || !Array.isArray(output.items) || !Number.isInteger(output.count) || output.count<1 || output.count!==output.items.length || output.count>step.limit || typeof output.truncated!=='boolean' || !['bounded','limit_reached'].includes(output.coverage)) return false; try {httpUrl(output.source_url);}catch{return false;} if(output.coverage==='limit_reached' && (!output.truncated || output.count!==step.limit)) return false; if(step.complete_only===true && (!Number.isInteger(output.skipped_incomplete_count) || output.skipped_incomplete_count<0 || (output.skipped_incomplete_count>0 && !output.truncated))) return false; const names=step.fields.map(field=>field.name); if(output.items.some(item=>!item || typeof item!=='object' || Array.isArray(item) || Object.keys(item).length!==names.length || Object.keys(item).some(key=>!names.includes(key)) || step.fields.some(field=>field.source==='content' ? !validContentValue(field,item[field.name]) : (item[field.name]!==null && typeof item[field.name]!=='string') || (field.required && (typeof item[field.name]!=='string' || !item[field.name].trim()))))) return false; if(output.truncated!==true && output.items.some(item=>step.fields.some(field=>field.source==='content' && item[field.name]?.media_truncated))) return false; if(step.batch){ const summary=output.batch_summary; if(output.truncated!==true || !summary || typeof summary!=='object' || Array.isArray(summary) || Object.keys(summary).length!==3 || Object.keys(summary).some(key=>!['batches_read','duplicates_removed','stop_reason'].includes(key)) || !Number.isInteger(summary.batches_read) || summary.batches_read<1 || summary.batches_read>step.batch.max_batches || !Number.isInteger(summary.duplicates_removed) || summary.duplicates_removed<0 || !['limit_reached','boundary','batch_budget','time_budget'].includes(summary.stop_reason) || (summary.stop_reason==='limit_reached' && output.count!==step.limit)) return false; const identities=output.items.map(item=>item[step.batch.dedupe_field]); if(identities.some(value=>typeof value!=='string' || !value.trim()) || new Set(identities).size!==identities.length) return false; } return true; } function assertExpected(expected, outputs, steps = []) { if (!expected || typeof expected !== 'object' || Array.isArray(expected) || !Object.keys(expected).length) { throw new WorkerError('实际试跑必须提供至少一个明确的输出预期', 'EXPECTED_OUTPUT_REQUIRED'); } const errors = []; for (const [name, value] of Object.entries(expected)) { if(nonemptyExpectation(value)){ const step=steps.find(step=>step.kind==='extract' && step.output_name===name); if(!validNonemptyExtractOutput(step,outputs[name])) errors.push(`输出 ${name} 没有符合提取契约的非空结构化结果`); continue; } if(emptyExpectation(value)){ const step=steps.find(step=>step.kind==='extract' && step.output_name===name),output=outputs[name]; if(!step || step.allow_empty!==true || !step.row_selector || !output || output.count!==0 || output.items?.length!==0 || output.coverage!=='empty' || output.truncated!==false || output.empty_state?.confirmed!==true || output.empty_state.selector!==step.empty_state?.selector || output.empty_state.text!==step.empty_state?.text?.trim()) { errors.push(`输出 ${name} 没有当前可见页面的明确空态证据`); } continue; } const samples = Array.isArray(value) ? value : [value]; if (!samples.length || samples.some(item => typeof item !== 'string' || !item.trim())) { errors.push(`输出 ${name} 的预期必须是非空文本或非空文本列表`); continue; } if (!Object.hasOwn(outputs, name)) { errors.push(`没有读回预期输出 ${name}`); continue; } const output = outputs[name]; // URLs never satisfy content assertions, including structured extraction. const fields = steps.find(step => step.kind === 'extract' && step.output_name === name)?.fields || []; const contentFields = fields.filter(field => field.source !== 'href').map(field => field.name); const actual = Array.isArray(output) ? output.map(item => item.text || '').join('\n') : output?.items ? output.items.flatMap(item => contentFields.map(name => fields.find(field=>field.name===name)?.source==='content' ? item[name]?.text || '' : item[name] || '')).join('\n') : ''; if (!actual.trim()) { errors.push(`输出 ${name} 的实际读取结果为空`); continue; } for (const sample of samples) if (!actual.includes(sample)) errors.push(`输出 ${name} 未包含预期内容 ${redactText(sample).slice(0, 120)}`); } return errors; } // Only module callers can shorten this deadline for isolated regression tests; // the JSONL RPC exposes neither a budget override nor a cleanup configuration. async function withDeadline(run,{budgetMs,startedMs=Date.now(),cleanupReserveMs=2000,onTimeout,errorCode}) { const reserve=Math.min(cleanupReserveMs,Math.max(1,budgetMs-1)); const cutoff=startedMs+budgetMs-reserve; const control={expired:false,check(){ if(this.expired || Date.now()>=cutoff) throw budgetError(); }}; let timer,cleanupTimer,cleanupPromise; const budgetError=()=>new WorkerError(errorCode==='BATCH_BUDGET' ? '评论批次已达到时间预算,已中断会话并保留已完成批次证据' : errorCode==='WAIT_TIMEOUT' ? '范围等待已达到时间预算,已中断会话并保留已取得证据' : '试跑已达到本轮总时间预算,已中断当前会话并保留已取得证据',errorCode); const expire=()=>{ control.expired=true; if(!cleanupPromise){ const remaining=Math.max(1,startedMs+budgetMs-Date.now()); cleanupPromise=Promise.race([Promise.resolve().then(()=>onTimeout?.()),new Promise(resolve=>{cleanupTimer=setTimeout(resolve,remaining);})]).catch(()=>{}); } return cleanupPromise; }; const execution=Promise.resolve().then(()=>run(control)); const expiration=new Promise((_,reject)=>{ timer=setTimeout(()=>{ expire().then(()=>reject(budgetError())); },Math.max(0,cutoff-Date.now())); }); try{const value=await Promise.race([execution,expiration]);control.check();return value;} catch(error){ if(control.expired || error.code===errorCode){await expire();throw budgetError();} throw error; } finally{clearTimeout(timer);clearTimeout(cleanupTimer);} } export const withVerifyDeadline=(run,options)=>withDeadline(run,{...options,errorCode:'VERIFY_BUDGET'}); export const withBatchDeadline=(run,options)=>withDeadline(run,{...options,errorCode:'BATCH_BUDGET'}); export const withWaitDeadline=(run,options)=>withDeadline(run,{...options,errorCode:'WAIT_TIMEOUT'}); export async function verify(params,{budgetMs:internalBudgetMs,cleanupReserveMs=2000,batchBudgetMs=15_000}={}) { if(!Number.isInteger(batchBudgetMs) || batchBudgetMs<1 || batchBudgetMs>15_000) throw new WorkerError('内部批次测试预算只能缩短既有上限','INVALID_INPUT'); const startedMs=Date.now(); const startedAt = new Date().toISOString(); const session = sessionFor(params.task_id); resolveExecutionProfile(params.execution_profile); if(params.execution_profile!==session.executionProfile || (isNativeUiProfile(session.executionProfile) ? !executionContractMatches(params.execution_contract,session.executionProfile) : params.execution_contract!==undefined)) throw new WorkerError('候选执行模式与当前浏览器会话不一致,未导航或回放','EXECUTION_PROFILE_MISMATCH'); const outputs = Object.create(null), evidence = [], errors = [],errorCodes=[],seekResults=[],stepTimings=[]; let currentStepPosition=0; const expected = params.expected_outputs; // Validate assertions before any browser movement. assertExpected(expected, {}); for (const value of Object.values(expected)) { if(emptyExpectation(value) || nonemptyExpectation(value)) continue; const samples = Array.isArray(value) ? value : [value]; if (!samples.length || samples.some(item => typeof item !== 'string' || !item.trim())) { throw new WorkerError('每个输出预期必须是非空文本或非空文本列表', 'INVALID_EXPECTED_OUTPUT'); } } if (!Array.isArray(params.steps) || !params.steps.length || params.steps.length > MAX_STEPS) { throw new WorkerError(`试跑要求 1–${MAX_STEPS} 个固定步骤`, 'INVALID_STEPS'); } const entry = httpUrl(params.entry_url); requireOrigin(session, entry.href); const parameters = params.parameters && typeof params.parameters === 'object' ? params.parameters : {}; const noteBinding=params.note_binding===undefined ? null : makeNoteBinding(params.note_binding,entry,params.steps,parameters); const verifyBudgetMs=90_000+(noteBinding?.seek_timeout_ms || 0); const effectiveBudget=Number.isInteger(internalBudgetMs) && internalBudgetMs>0 ? Math.min(verifyBudgetMs,internalBudgetMs) : verifyBudgetMs; let currentTrace=null,failureDiagnostics=null,budgetExpired=false,budgetContextClosed=false,batchContextClosed=false,batchCloseMs=null,entryFinished=null,closeStarted=null,closeMs=null; const retainFailure=(code,diagnostics=currentTrace?.diagnostics,phase=diagnostics?.phase)=>{ const kind=params.steps[currentStepPosition-1]?.kind; if(!currentTrace || !diagnostics || !ACTION_KINDS.includes(kind) || typeof code!=='string' || !/^[A-Z][A-Z0-9_]{0,63}$/.test(code)) return; failureDiagnostics=structuredClone({step_position:currentStepPosition,kind,code, diagnostics:{...diagnostics,phase:phase || currentTrace.phase}}); }; const retainTiming=status=>{ if(currentTrace && !stepTimings.some(item=>item.step_position===currentStepPosition)) stepTimings.push(currentTrace.snapshot(currentStepPosition,params.steps[currentStepPosition-1].kind,status)); }; const retainTrace=trace=>{ for(const snapshot of [trace?.before,...(trace?.batch_evidence || []).map(item=>item.capture),trace?.after]) { if(snapshot && !evidence.some(item=>item.capture_id===snapshot.capture_id)) evidence.push(snapshot); } }; try { await withVerifyDeadline(async control=>{ try { await session.page.goto(entry.href, { waitUntil: 'domcontentloaded', timeout: 20_000 }); control.check(); await settle(session.page); control.check(); evidence.push(await captureSession(session, '变参试跑入口')); entryFinished=Date.now(); control.check(); for (let index = 0; index < params.steps.length; index += 1) { currentStepPosition=index+1; if (shuttingDown) throw new WorkerError('学习工作进程正在退出', 'WORKER_CLOSING'); control.check(); const step = params.steps[index]; if (!step || typeof step !== 'object' || !ACTION_KINDS.includes(step.kind)) throw new WorkerError(`第 ${index + 1} 步类型无效`, 'INVALID_STEPS'); const actionParams = { ...step }; if (step.kind === 'fill' && step.parameter) { if (!Object.hasOwn(parameters, step.parameter)) throw new WorkerError(`缺少运行参数 ${step.parameter}`, 'MISSING_PARAMETER'); actionParams.value = String(parameters[step.parameter]); } if (step.target?.parameter) { if (!Object.hasOwn(parameters, step.target.parameter)) throw new WorkerError(`缺少动态目标参数 ${step.target.parameter}`, 'MISSING_PARAMETER'); actionParams.target = { ...step.target, value: requireString(parameters[step.target.parameter], '动态目标参数', 2_000) }; } if(step.until_target?.parameter){ if(!Object.hasOwn(parameters,step.until_target.parameter)) throw new WorkerError(`缺少有界目标参数 ${step.until_target.parameter}`,'MISSING_PARAMETER'); actionParams.until_target={...step.until_target,value:requireString(parameters[step.until_target.parameter],'有界目标参数',2_000)}; delete actionParams.until_target.parameter; } if (['read', 'extract'].includes(step.kind)) requireString(step.output_name, 'output_name', 80); const result = await actionSession(session, actionParams, { captureBefore: false,noteBinding,batchBudgetMs,onTrace:trace=>{currentTrace=trace;} }); control.check(); if(step.until_target) seekResults.push({step_position:index+1,scrolls:result.result.scrolls,movements:result.result.movements,target_reached:result.result.target_reached,stop_reason:result.result.stop_reason}); for(const batch of result.batch_evidence || []) if(!evidence.some(item=>item.capture_id===batch.capture.capture_id)) evidence.push(batch.capture); evidence.push(result.after); if (['read', 'extract'].includes(step.kind)) outputs[step.output_name] = result.result; retainTiming('passed'); currentTrace=null; } errors.push(...assertExpected(expected, outputs, params.steps)); requireBoundNotePage(session,noteBinding); } catch (error) { if(control.expired || error.code==='VERIFY_BUDGET') throw error; retainFailure(error.code,error.diagnostics || currentTrace?.diagnostics); if(error.code==='BATCH_BUDGET' && currentTrace?.batch_timeout){ const step=params.steps[currentStepPosition-1],partial=currentTrace.batch_partial_result; if(currentTrace.batch_request_blocked!==true && step?.kind==='extract' && partial?.count>0 && validNonemptyExtractOutput(step,partial)) outputs[step.output_name]=structuredClone(partial); if(currentTrace.batch_request_blocked===true){errors.push('批次截止前已阻止未知请求,不提供该步部分输出');errorCodes.push('WRITE_REQUEST_BLOCKED');} batchContextClosed=currentTrace.batch_context_closed===true;batchCloseMs=currentTrace.batch_close_ms ?? null; } if(error.code==='WAIT_TIMEOUT' && currentTrace?.wait_timeout){ batchContextClosed=currentTrace.wait_context_closed===true;batchCloseMs=currentTrace.wait_close_ms ?? null; if(currentTrace.wait_request_blocked===true){errors.push('范围等待截止前已阻止未知请求');errorCodes.push('WRITE_REQUEST_BLOCKED');} } if(error.seek_result) seekResults.push({step_position:currentStepPosition,...error.seek_result}); errors.push(redactText(error.message)); if(typeof error.code==='string' && /^[A-Z][A-Z0-9_]{0,63}$/.test(error.code)) errorCodes.push(error.code); for (const snapshot of [error.before, ...(error.batch_evidence || []).map(item=>item.capture),error.after]) { if (snapshot && !evidence.some(item => item.capture_id === snapshot.capture_id)) evidence.push(snapshot); } if (!session.closed && !session.page.isClosed()) { if(currentTrace) currentTrace.phase='failure_capture'; await captureSession(session, '失败后页面状态').then(snapshot => evidence.push(snapshot)).catch(() => {}); } if(control.expired) throw new WorkerError('总预算期间已中断取证','VERIFY_BUDGET'); retainTiming('failed'); } },{budgetMs:effectiveBudget,startedMs,cleanupReserveMs,onTimeout:()=>{ budgetExpired=true; retainFailure('VERIFY_BUDGET',currentTrace?.diagnostics,currentTrace?.phase); if(entryFinished===null) entryFinished=Date.now(); retainTiming('interrupted'); retainTrace(currentTrace); if(currentTrace?.seek_result && !seekResults.some(item=>item.step_position===currentStepPosition)) { seekResults.push({step_position:currentStepPosition,...currentTrace.seek_result,target_reached:false,stop_reason:'time_budget'}); } // Closing the context cancels pending DOM inspections, screenshots, // navigation and action waits. Merely racing them would leave late work // running after a timeout response and could append a false success. session.closed=true;session.ready=false; closeStarted=performance.now(); session.context.once('close',()=>{budgetContextClosed=true;}); const last=session.lastCapture; if(last && Date.parse(last.captured_at)>=startedMs && !evidence.some(item=>item.capture_id===last.capture_id)) evidence.push(last); session.lastCapture=null; endPlatformRequestPolicy(session.platformRequestPolicy); session.platformRequestPolicy=null;session.platformRequestPage=null;session.automating=false; return closeContext(session).finally(()=>{closeMs=Math.max(0,Math.round(performance.now()-closeStarted));}); }}); } catch(error) { if(error.code!=='VERIFY_BUDGET') throw error; retainTrace(currentTrace); if(!errorCodes.includes('VERIFY_BUDGET')) {errors.push(redactText(error.message));errorCodes.push('VERIFY_BUDGET');} } return { ...executionMetadata(session.executionProfile), passed: errors.length === 0, outputs, errors, evidence,error_codes:errorCodes, os: process.platform === 'win32' ? 'windows' : process.platform === 'darwin' ? 'macos' : process.platform, browser_version: session.context.browser()?.version() || 'unknown', identity_verification: 'operator_confirmed', started_at: startedAt, finished_at: new Date().toISOString(), model_calls: 0, ...(params.steps.some(step=>step?.until_target) ? {seek_results:snapshotSeekResults(seekResults)} : {}), ...(noteBinding?.seek_timeout_ms ? {verify_budget_ms:verifyBudgetMs} : {}), ...((budgetExpired && budgetContextClosed) || batchContextClosed ? {session_closed:true} : {}), ...(noteBinding ? {note_binding_result:bindingResult(noteBinding)} : {}), ...(failureDiagnostics ? {failure_diagnostics:structuredClone(failureDiagnostics)} : {}), step_timings:stepTimings.map(item=>({...item,phase_ms:{...item.phase_ms}})), verify_timing:{duration_ms:Math.max(0,Date.now()-startedMs),entry_ms:Math.max(0,(entryFinished ?? Date.now())-startedMs), close_ms:closeStarted===null ? batchCloseMs : closeMs ?? Math.max(0,Math.round(performance.now()-closeStarted))}, }; } const methods = { open, ready, capture, action, close, verify, status: async params => { const session = sessions.get(requireString(params.task_id, 'task_id', 128)); if (session && !session.closed && !session.page.isClosed()) await refreshLoginDetection(session); return session && !session.closed ? sessionInfo(session) : { status: 'closed' }; }, }; export async function dispatch(method, params = {}) { if (shuttingDown) throw new WorkerError('学习工作进程正在退出', 'WORKER_CLOSING'); if (!Object.hasOwn(methods, method)) throw new WorkerError('不支持的工作进程方法', 'UNKNOWN_METHOD'); return methods[method](params); } function shutdown() { if (shutdownPromise) return shutdownPromise; shuttingDown = true; for (const session of sessions.values()) { session.closed = true; session.ready = false; session.lastCapture = null; } shutdownPromise = (async () => { // Closing the context interrupts a pending goto/click/capture immediately; // it must not wait for the serial RPC loop to finish the current request. await Promise.allSettled([...contexts].map(context => context.close())); await Promise.allSettled([...closingAccounts.values()]); sessions.clear(); accountOwners.clear(); })(); return shutdownPromise; } async function main() { await fs.mkdir(HOME, { recursive: true }); const input = readline.createInterface({ input: process.stdin, crlfDelay: Infinity }); const stop = () => { input.close(); process.stdin.pause(); void shutdown(); }; input.once('close', stop); process.stdin.once('error', stop); // A terminated parent may close its read pipe before closing our stdin. // Treat EPIPE as loss of the owner, never an unhandled stream exception. process.stdout.on('error', error => { if (error.code !== 'EPIPE') process.exitCode = 1; stop(); }); process.on('SIGINT', stop); process.on('SIGTERM', stop); const reply = value => { if (!shuttingDown && !process.stdout.destroyed) process.stdout.write(JSON.stringify(value) + '\n'); }; try { for await (const line of input) { if (shuttingDown) break; if (!line.trim()) continue; let request; try { if (line.length > 1_000_000) throw new WorkerError('请求超过大小限制', 'REQUEST_TOO_LARGE'); request = JSON.parse(line); if (!request || typeof request !== 'object' || request.id === undefined) throw new WorkerError('请求缺少 id', 'INVALID_REQUEST'); const result = await dispatch(request.method, request.params || {}); reply({ id: request.id, result }); } catch (error) { const code = error.code || 'BROWSER_ERROR'; reply({ id: request?.id ?? null, error: { message: redactText(error.message).slice(0, 1_200), code, ...(error.before ? { before: error.before } : {}), ...(error.after ? { after: error.after } : {}), ...(error.batch_evidence ? {batch_evidence:error.batch_evidence} : {}), ...(error.session_closed===true ? {session_closed:true} : {}), ...(error.error_codes ? {error_codes:error.error_codes} : {}), ...(error.diagnostics ? { diagnostics: error.diagnostics } : {}) } }); } } } finally { input.close(); process.stdin.pause(); await shutdown(); } } if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) { main().catch(error => { process.stderr.write(`学习工作进程启动失败:${redactText(error.message)}\n`); process.exitCode = 1; }); }