Project HELIX — Personal Browser Employee Runtime

Implement Shadow Mode: HELIX passively predicts user activity/workflow/next-action without executing, for validation

Implement Workflow Skills conversion (purpose, trigger, inputs, outcome, constraints, verification, exceptions, approval requirements, confidence, versioning)

Build Personal Browser Operating Model (navigation style, interaction style, attention patterns, workflow patterns, preferences)

Build workflow confirmation UX ("I noticed this pattern..." with confirm/modify/ignore/keep-observing)

Implement workflow discovery engine (probabilistic pattern detection over repeated site/action sequences)

Build activity timeline generation from raw observation stream

Implement browser activity observation with sensitive-data minimization (exclude passwords, payment info, auth secrets)

Build Chrome extension with Learning Mode (visibility into status, included/excluded sites, captured data categories, retention, pause control)

Define human handoff triggers and take-over/return-control flow (CAPTCHA, auth, destructive/financial actions, low confidence, ambiguity, limits)

Implement action verification (state-change confirmation beyond command acceptance: notifications, URL/element/record changes)

Build Recovery Engine to handle page changes, missing elements, crashes, model failures, session expiration, and network problems

Implement task checkpointing (objective, context, completed/remaining work, last action, recovery info) for interrupted-task resume

Design persistence layer (tasks, projects, runs, browser states, observations, actions, checkpoints, skills, policies, events, audit logs)

Build context management, embeddings, memory retrieval, and skill retrieval infrastructure

Evaluate and integrate local vision-language models (e.g. Qwen-family VLMs) for perception/reasoning

Integrate local OpenAI-compatible inference (Ollama/vLLM) with a replaceable model abstraction layer

Implement hybrid perception combining visual (screenshots), accessibility tree, DOM, and browser events

Build abstract browser state representation (URL, tabs, page load state, interactive elements, a11y tree, visual state, task association)

Implement persistent browser sessions with isolated credential/auth handling (browser controller owns sensitive material, not model)

Stand up Browser Runtime (Chrome, Playwright control, persistent profiles, tabs/contexts, events, screenshots)

Implement human-like operating behavior (priority-driven, non-random task execution emerging from workload/context)

Design semantic Agent Action Model (intent, target, preconditions, parameters, expected outcome, validation, result, error state) for navigate/click/type/scroll/select/etc.

Implement core loop: Observe → Understand → Plan → Validate → Act → Verify → Remember → Decide

Implement context-switching engine mapping browser tabs to responsibilities/work state

Implement multi-task management: unified workload intake from instructions, schedules, events, monitored conditions, unfinished work

Build attention model: current focus, active/waiting/deferred/upcoming work, background monitoring, human-required states

Implement system startup sequence (reload config, permissions, task state, recurring/unfinished work, sessions, events, build & prioritize workload)

Implement unified planner/scheduler supporting one-time, recurring, scheduled, event-based, monitoring, and persistent-responsibility work

Build Work Management layer: tasks, projects, responsibilities, dependencies, priorities, deadlines

Model the Employee → Responsibility → Work → Context → Activity hierarchy

Design Personal Operating Model schema (workflow patterns, navigation preferences, learned skills, user preferences)

Define Employee Identity layer (role, permissions, policies, working schedule, connected services)