Keep the two systems separate and connect only what is required.
Home Assistant should remain the house-control system. The personal AI assistant should remain the work-and-life system. Connect them through one controlled interface with explicit state, actions, permissions, logs, denial behavior, revocation, and rollback.
Home Assistant: devices, sensors, automations, speakers, media, voice endpoints, and physical state.
Personal AI assistant: projects, notes, tasks, messages, calendar, files, computer tools, decisions, preferences, and memory.
Controlled interface: Home Assistant’s MCP server or another narrow API that exposes only approved state and actions.
Optional input and output: phone, room microphone, wearable microphone, continuous transcription, speakers, and proactive alerts.
Scott’s reference system also uses one wearable microphone for two concurrent paths. Addressed requests route to either Home Assistant or the personal AI assistant. Approved speech also enters timestamped context that can be reconciled with work telemetry, notes, messages, projects, and history. Replies return through an approved speaker or phone.
Observed unattended behavior: a delivery person asked the house where to leave a package, the house answered from an approved delivery instruction, and the package was placed correctly. Scott was not present or part of the conversation. The visitor-facing behavior does not expose names, schedules, occupancy, messages, or broader household context.
What you need before you start.
Start with a running Home Assistant environment, a separate personal AI assistant environment, one useful cross-system workflow, and one candidate controlled interface.
A capable implementation agent can inventory devices and software, verify exact-model compatibility, write configuration and glue code, inspect logs, run tests, and repair failures. The owner still defines the goal, permissions, local/cloud boundary, pass conditions, and rollback.
Build one bounded phase at a time. Require an observable result, preserve rollback, and do not add voice, continuous capture, or broader authority until the core connection passes.
No major vendor is strongly incentivized to ship the full integration.
This is not an intelligence problem. Amazon, Apple, Google, Anthropic, and OpenAI can all supply capable models. The incentive and liability structure is the constraint.
A mass-market vendor must support millions of households, devices, jurisdictions, consent expectations, failure modes, and lawsuits. The vendor owns most of that exposure while the individual owner receives most of the benefit from broader permissions, cross-platform context, proactive judgment, and physical action. The rational consumer product is therefore narrower than the system many owners would choose for themselves.
An informed owner can make a different choice for one household: keep the environments separate, expose only selected state and actions, choose what stays local, inspect logs, test denial and rollback, and increase authority one capability at a time. This is closer to a bounded home-improvement project than a consumer feature worth waiting for. Open-source home infrastructure, common standards such as Matter, and capable implementation agents make the project practical for a nonengineer to direct.
Minimum security, household, and rollback requirements
- Do not port-forward Home Assistant directly onto the public internet.
- Use Home Assistant Cloud or a reputable VPN for remote access.
- Install updates and keep automatic backups. Make a fresh backup before each migration.
- Use strong unique credentials and multi-factor authentication where supported.
- Keep secrets out of model conversations, code, screenshots, transcripts, and logs.
- Prefer maintained official or local integrations; review custom integrations before installing them.
- Expose only the devices, state, and actions the personal assistant actually needs.
- Keep an action log and a quick way to revoke access.
A Home Assistant instance kept on the private home network is not waiting openly on the internet. A serious compromise generally needs the network, credentials, an exposed integration, or the underlying computer and operating system. That is a tractable problem for one owner to reason about.
Definitions used throughout this guide.
Use these terms consistently in the page, copied prompts, generated plan, and implementation logs.
Home Assistant, personal AI assistant, controlled interface, optional voice/capture, and implementation agent
Check these five things before buying a smart device.
Run this test on the exact model before buying a plug, light, microphone, camera, sensor, speaker, hub, or radio.
Most households already own more smart devices than they would list from memory: speakers, televisions, streaming boxes, phones, watches, cameras, thermostats, plugs, lights, bridges, appliances, vehicles, and network equipment. Inventory before shopping. Scott began with more than 70 devices spread across several ecosystems; the work was coordination and testing, not replacing the house.
Prefer devices that support Matter, Zigbee, Thread, a documented local API, or a mature Home Assistant integration—when the exact product passes the full test. Matter can make an inexpensive device easier to carry forward, but interoperability does not turn poor radios, microphones, power supplies, firmware, or physical construction into quality hardware. “Open” does not automatically mean secure, reliable, or good.
- Can it operate locally if the vendor cloud disappears?
- Does Home Assistant support this exact model well today?
- Is there a documented local or open protocol?
- Can its data and controls be selectively exposed?
- Can it be migrated or replaced without rebuilding the system?
Run the five compatibility checks
Choose a hardware baseline for the calculator.
Selecting a baseline sets the hardware calculator and synchronizes the generator’s hardware-baseline answer below. The personal AI assistant and controlled interface are configured separately and may require no new hardware.
Reuse existing hardware. Buy only missing components.
This calculator covers Home Assistant, radio, voice, and optional wearable-capture hardware. It does not price the personal AI assistant, model subscription or API usage, transcription, storage, or other software services.
If Home Assistant and a suitable personal AI assistant already exist, the core controlled-interface connection may require $0 in new hardware.
Rough official U.S. reference prices checked July 23, 2026. Tax, shipping, availability, controllable devices, and regional variation are not included. Home Assistant Voice Preview Edition is one dedicated endpoint option; phones, existing speakers, and other supported hardware may cover some zones. It does not include its USB-C power supply or cable. One ZBT-2 must be dedicated to Zigbee or Thread; it cannot run both simultaneously.
Do not infer endpoint count from a starter bundle. A wearable microphone solves one person’s mobile input. It does not guarantee that visitors can address the house, that the house can speak back in every useful area, or that noisy and outdoor zones are covered. Start with the endpoints you own, run a room-by-room and outdoor-zone coverage test, then add units only where the test fails.
| Component | What it does | Current price | Your status | Line total |
|---|---|---|---|---|
| Home Assistant Green Official page |
Simple dedicated Home Assistant host. | $199 | $0 | |
| Dedicated voice endpoint — first unit Home Assistant Voice Preview Edition example |
First dedicated Home Assistant voice endpoint. | $69 | $0 | |
| Dedicated voice endpoint — second test unit Home Assistant Voice Preview Edition example |
Second test endpoint for a separate acoustic zone; not a whole-house ceiling. | $69 | $0 | |
| Additional dedicated voice endpoints Voice Preview Edition example |
Additional units after coverage testing. Enter only the number still missing after reusing phones, speakers, and supported endpoints. | $69 each | $0 | |
| Home Assistant Connect ZBT-2 #1 Official page |
One Zigbee or Thread radio, not both at once. | $49 | $0 | |
| Optional ZBT-2 #2 Official page |
Needed only if both Zigbee and Thread require new dedicated radios. | $49 | $0 | |
| DJI Mic 3 kit Official page |
Optional 2-transmitter / 1-receiver wearable capture kit with charging case. | $259 reference | $0 |
The baseline selector is shared; the detailed edits are not. Choosing a hardware baseline here or in the generator synchronizes both. Manual status or quantity changes in the table above change its subtotal and copied list only; the generated plan uses its own inventory, coverage-test, ecosystem, and other answers.
Historical build receipt: roughly $284 in new Home Assistant-side gear plus small accessories; roughly $543 including the DJI microphone kit. Scott reused an old computer, the existing network, more than 70 smart devices, speakers and Echoes, Hue/Govee and other equipment, and an iPhone. His coverage required more than two fixed voice endpoints even though he wore a microphone. Historical spend and today’s replacement cost are different numbers.
Integrate existing devices and hubs into Home Assistant.
Do not throw away a working house. Home Assistant can absorb what you own while you gradually move coordination back under one roof.
Build the first inventory from exports and discovery wherever possible: Home Assistant entities, router clients, vendor apps, hubs, bridges, phones, speakers, televisions, cameras, sensors, and existing automations. A manual memory dump will miss things.
Map rooms, outdoor zones, devices, and voice coverage
Give the implementation agent a floor plan, sketch, photo, or room-and-zone list. Name rooms and outdoor areas. Place devices, sensors, speakers, microphones, doors, network equipment, and important boundaries. Mark noisy areas, thick walls, private zones, and anywhere a visitor may need to speak to the house.
The map is bootstrap context, not permanent truth. Once the system has entity state, logs, failed-command reports, latency, device availability, and room corrections, it should maintain the working topology and propose map updates. The original drawing matters less as observed behavior gets better.
Philips Hue
Keep the Hue bridge initially and add its local Home Assistant integration. Test state, control, restart recovery, and clean disable. Move bulbs away from the bridge later only for a concrete reason.
Echo / Alexa devices
Keep useful speakers and displays. Verify the exact output or control path before depending on it. Replacement is not the objective; owner-controlled coordination is.
Sonos, Cast, and Apple TV
Add one media platform at a time as an announcement or playback target. Test room selection, volume, failure behavior, and mute before relying on proactive speech.
Kasa / Tapo
Check the exact model in current Home Assistant documentation. Prefer supported local control. Do not assume every product under one brand uses the same integration path.
Govee
Integration quality and locality vary by model. Verify before buying more. Test state, action, restart, and vendor-cloud loss.
Matter, Thread, and Zigbee
Inventory existing border routers and radios before buying another. A ZBT-2 can serve Zigbee or Thread, not both simultaneously. If you need both, use two dedicated radios or combine one with compatible coverage you already own.
Apple Home, Google Home, and SmartThings
Coexist first. Bring devices into Home Assistant gradually and preserve the current app or hub until the new path passes for everyone who relies on it.
An unknown device
Find the exact model in the official Home Assistant integration catalog, then check current compatibility reports. Apply the five compatibility checks before replacing it or buying more from the same line.
Build and test the connection in five phases.
The implementation agent should complete one phase at a time. Every phase has a prerequisite, observable result, pass test, and rollback.
Phase 1 · Inventory both systems and choose one workflow
Build Phase 1 only. Interview me about: - the current Home Assistant host, version, integrations, devices, entities, backups, network, and remote-access method; - the approximate device count and every available inventory source, including Home Assistant exports, router clients, hubs, bridges, and vendor apps; - a floor plan, sketch, photo, or room-and-outdoor-zone list; device placement; network boundaries; noisy areas; private zones; and desired input/output coverage; - the personal AI assistant environment, approved accounts, tools, context sources, logs, and current permissions; - one useful workflow that requires both systems; - the selected local/cloud boundary and household constraints. Propose one controlled interface, using Home Assistant's MCP server or another narrow API only if current official documentation and my requirements support it. Specify the exact state and actions that would be exposed. Return a written architecture, normalized device/entity inventory, room-and-zone topology, initial voice-coverage test, data flow, credential boundary, permission level, allowed and denied examples, action log, pass condition, revocation method, and rollback. Flag duplicates, unknown devices, stale entities, missing rooms, and conflicting names. Do not purchase hardware or change either production environment in Phase 1.
Phase 2 · Verify one Home Assistant state and one action
Build Phase 2 only. Create a fresh Home Assistant backup. Select one existing entity or sensor and one low-consequence action from the approved Phase 1 workflow. Prefer an official or mature integration and local control when the exact device supports it. Test: 1. correct state reading; 2. one approved action; 3. readable logging; 4. restart recovery; 5. a denied or unavailable action; 6. manual fallback; 7. clean disable and rollback. Do not connect the personal AI assistant yet. Do not remove a working bridge, app, or fallback.
Phase 3 · Connect the personal AI assistant through the controlled interface
Build Phase 3 only. Create one narrow, auditable controlled interface between the personal AI assistant and Home Assistant. Home Assistant's built-in MCP server may be used if current official documentation and my requirements support it. Use a separate credential and expose only the state and action approved in Phase 1. Apply the selected permission to each capability. Define visible scope, allowed and denied examples, log, success and denial tests, immediate revocation, and rollback or safe state. Test one allowed read, one allowed action, one action requiring confirmation, and one unexposed action that must fail safely. Show me the log and exact revocation method before proposing broader access. Rollback: revoke the controlled-interface credential or permission and disable the exposed action set.
Phase 4 · Complete one workflow across the house and work/life tools
Build Phase 4 only. Use the Phase 1 workflow and the Phase 3 controlled interface. Connect exactly one approved work or life tool, such as a task inbox, notes store, calendar, message-draft queue, or project thread. Define the trigger, required Home Assistant state, required personal context, permitted action, owner-visible result, attribution, duplicate prevention, uncertainty handling, log, undo, and revocation. Run one real example. Then test stale Home Assistant state, missing information, ambiguity, a duplicate event, an action beyond permission, tool failure, revocation, and undo. Pause and ask one concise question instead of guessing when required information is missing. Rollback: revoke the work/life tool access, disable the workflow, and undo or remove test records.
Phase 5 · Add optional voice, wearable capture, and proactive return
Build Phase 5 only. Ask which optional interfaces I want: room voice, phone voice, wearable capture, continuous transcription, spoken clarification, or proactive alerts. Ask about household expectations, regular visitors, jurisdiction, retention, and the exact local/cloud path before enabling broad recording. Inventory every room and outdoor zone that needs microphone input, visitor access, or spoken output. A wearable microphone covers the person wearing it, not the rest of the house. Test the endpoints, phones, and speakers I already own. Do not assume one endpoint per room, and do not treat two endpoints as a ceiling. Recommend additional fixed endpoints only for coverage failures the test can identify. If I choose one microphone for both addressed requests and longer capture, use one capture process. Add a local address detector that packages the relevant transcript window with a timestamp and places it in a queue. Route addressed requests to either: - the personal AI assistant for computer work, live threads, context questions, tasks, messages, or other approved tools; or - Home Assistant for selected state, devices, media, and routines. Return results through an approved speaker or phone. Add transcription confirmation, self-voice suppression, stale-event protection, logs, automatic restart, mute, stop, revocation, and the written retention rule. Test one personal-assistant request, one Home Assistant request, ordinary conversation, speaker audio, the system's own voice, stale input, microphone loss, host restart, mute, retention, and revocation. Rollback: stop the optional interface services, revoke their permissions, and delete test data under the written retention rule.
Make the system show its work and improve from observed failures.
A large home system should not depend on the owner remembering every device, room, integration, test, and exception. Build a live operating view, record corrections, and rerun bounded tests whenever the system changes.
Scott started with more than 70 smart devices and gave the implementation agents a physical map of the house and outdoor space with named rooms and device locations. That map made the first integration faster. Over time, entity state, availability, logs, latency, failed commands, room corrections, and actual use become the better source of truth.
Create the device, room, outdoor-zone, and coverage map
Output: a normalized inventory and maintainable topology, not a decorative floor-plan image.
Create a device and topology inventory for this Home Assistant environment. Start with machine-readable discovery: Home Assistant devices, entities, areas, integrations, automations, router clients, hubs, bridges, and available vendor exports. Then reconcile that list against my floor plan, sketch, photos, or room-and-outdoor-zone list. For every device or entity, record: - exact model and integration; - room or outdoor zone; - physical purpose and owner; - network, hub, bridge, radio, and power dependencies; - last-seen and current availability; - whether it can be locally controlled; - whether it is exposed through the controlled interface; - fallback, replacement, and rollback path. Map microphone input, spoken-output, sensor, network, and visitor coverage. Flag duplicates, stale entities, conflicting names, unknown devices, dead integrations, untested zones, and privacy-sensitive areas. Store the result as editable structured data. Treat my original drawing as bootstrap evidence, not permanent truth. Propose map changes when observed state and corrections disagree with it; do not silently rewrite physical meaning.
Create the self-correcting QA loop
Loop: observe → classify → reproduce → repair → rerun → promote or roll back.
Build a bounded QA loop for the Home Assistant and personal-AI-assistant connection. For every tested behavior, define the expected state, action, room or zone, response, maximum latency, permission result, log entry, fallback, and rollback. Capture failures and owner corrections. Classify each as device availability, integration, entity naming, room mapping, speech recognition, address routing, stale context, permission, model judgment, network, output coverage, or unknown. For a failure: 1. preserve the event and relevant logs; 2. reproduce it safely; 3. change one bounded variable; 4. rerun the original test plus adjacent regression tests; 5. compare the result with the prior baseline; 6. promote only after the pass threshold is met; 7. roll back automatically when the change creates a regression. Never let the loop broaden permissions, purchase hardware, rename physical areas, or change irreversible behavior without the authority assigned to that capability. Surface unresolved failures and conflicting evidence on the dashboard.
Create operational and personal KPI dashboards
Start with decisions: every metric should answer a question or trigger a bounded action.
Build two dashboards from approved data sources. DASHBOARD 1 — SYSTEM OPERATION Show device and entity availability, stale sensors, automation failures, bridge and queue health, voice/capture process health, endpoint coverage failures, latency, last successful backup, restart recovery, unresolved corrections, and regressions. Every red state must link to the relevant evidence, owner, fallback, and next test. DASHBOARD 2 — OWNER-DEFINED KPIs Interview me about the measures I actually use to run my work, health, household, calendar, communication, obligations, or finances. Use only approved read sources. For each KPI define source, calculation, freshness, target or threshold, confidence, correction path, and what decision it supports. Do not fill either dashboard with metrics merely because they are available. Show source freshness and uncertainty. Preserve later corrections over earlier transcripts. Let me drill from a summary number to the underlying events without exposing unrelated private context.
Copyable prompts for ten specific behaviors.
These are behavior contracts for an implementation agent. They are not universal configuration files; the agent must adapt them to the actual house and tools.
1 · Single spoken command with logging
Behavior: One low-consequence action with accepted phrases, target, success reply, failure reply, log, and disable path.
Implement one reliable spoken command in Home Assistant. Interview me for the target and phrases. Keep the action deterministic and local where useful. Define success reply, failure reply, action log, and disable path. Test ten times, record every miss, and do not add variants until the base command passes.
2 · Speech to task
Behavior: A short spoken note becomes a task with timing, context, source, and uncertainty. Nothing is sent externally.
Build a speech-to-task behavior. Capture a short intentional spoken note, transcribe it, extract task title, timing clue, project context, and unresolved ambiguity, then place it in my chosen task inbox. Do not send a message. Log source, extracted task, and confidence. Ask one concise question when required information is missing.
3 · Speech to message draft with approval
Behavior: Ordinary speech becomes a draft. The system follows the selected outbound authority.
Build a speech-to-draft behavior for email or text. Extract intent, recipient, key facts, and tone. Stop and ask if the recipient or a material fact is ambiguous. Create a draft and follow the outbound-message permission I selected. Log the draft, approval state, send result if any, and revocation path.
4 · Spoken question when a workflow is blocked
Behavior: Missing information pauses the workflow and produces one concise spoken question.
Implement spoken clarification for blocked workflows. Define required information before action. When it is missing, pause instead of guessing, choose an approved available speaker, ask one concise question, record the answer, and resume only when the answer resolves the block. Log question, answer, resumed step, and result. Provide mute and revocation.
5 · Wearable recording to notes, tasks, and drafts
Behavior: Intentional wearable capture returns notes, decisions, drafts, tasks, and unresolved questions.
Build an intentional wearable-capture workflow for a run or walk. Ask about capture device, upload path, retention, and local/cloud processing. Transcribe a real recording and return five sections: notes, decisions, drafts, tasks, and unresolved questions. Preserve attribution and uncertainty. Do not send drafts. Test raw-audio deletion.
6 · Automatic capture recovery
Behavior: Restart the process, reset the input, test the path, and interrupt the owner only if recovery fails.
Add self-healing only after simple capture works. Detect an unhealthy capture path. Attempt in order: restart the process, reset the input, run a short test phrase, verify the downstream transcript, and notify me only if recovery still fails. Log every recovery attempt and provide a command that disables the loop.
7 · Interruption thresholds: urgent, normal, or silent
Behavior: Route events into urgent spoken interruption, normal spoken question, or silent log using owner-tuned thresholds.
Create three explicit proactivity levels: urgent spoken interruption, normal spoken question, and silent log. Interview me for examples, attention rules, and quiet behavior. Make thresholds editable and logged. Test one event in each class, one borderline event, mute behavior, and restart recovery.
8 · Visitor delivery answers with restricted context
Behavior: A visitor can ask where a package goes without gaining private household context.
Build a visitor-facing delivery behavior with a small context boundary. It may answer where a package should go using only the approved delivery instruction. It must not reveal names, schedules, occupancy, device state, private messages, or broader household context. Test approved questions, unrelated questions, repeated attempts, mute behavior, and safe failure.
9 · Visitor answers in other languages with the same restrictions
Behavior: Use a chosen language path while preserving the same visitor boundary.
Extend the bounded visitor behavior to selected languages. Detect language only when confidence is adequate; otherwise ask. Route speech recognition and voice output through the approved language path. Preserve the restricted delivery context in every language. Log language, confidence, and outcome without inventing conversations that were not captured and verified.
10 · Daily reconciliation of notes, tasks, and messages
Behavior: Reconcile spoken notes with tasks and messages, then surface only unresolved items needing judgment.
Build a daily reconciliation sweep. Compare approved spoken notes with the task inbox, draft queue, and sent-message log. Identify duplicates, resolved items, contradictions, and unresolved obligations. Surface only items requiring my judgment. Do not treat an earlier transcript as truth when a later correction or action supersedes it. Log reconciliation and provide undo.
Set an authority level for each capability.
Every granted capability needs visible scope, an action log, a pass test, immediate revocation, and a rollback or safe state.
Choose a starting rung for each capability. These are not permanent ceilings. Every granted capability still needs visible scope, an action log, a test, immediate revocation, and a rollback or safe state.
Your current permission block
OWNER-SELECTED PERMISSION BLOCK Lights, media, and reminders: Act and notify Notes, tasks, and internal organization: Act after confirmation Outbound messages: Draft or propose Purchases and financial actions: Observe / read only Microphones, cameras, and recording: Observe / read only Doors, alarms, ovens, and irreversible actions: Observe / read only For each capability, define visible scope, log, allowed and denied examples, pass test, immediate revocation, and rollback or safe state.
Generate a plan from your hardware and goals.
Answer a short inventory. The page will generate a reuse-first budget, migration order, phases, permission starting point, and complete project prompt.
Copy the full prompt into an implementation agent.
Paste this into a capable coding agent. The generator appends your answers and derived plan to this complete prompt.
Read and copy the complete implementation prompt
You are my implementation agent. Help me connect Home Assistant to a separate personal AI assistant through one narrow, logged, revocable interface. OPERATING GOAL Help me build the capable version around who I am, what I am trying to do, which burdens I want a computer to carry, and what I would rather spend my time doing. Treat the home layer, personal data structure, logs, and permission model as infrastructure I own. Treat the frontier model as replaceable. Do not imitate the limitations of mass-market assistants by silently removing useful capability. Present scoped choices, tradeoffs, tests, logs, revocation, and rollback; let me choose. I may not be an engineer. Do the implementation work a coding agent can safely do: inventory, research, configuration, code, deployment, log inspection, testing, repair, and documentation. Give me exact physical or account actions only when I must perform them. START WITH AN INTERVIEW Start a durable project. Do not recommend hardware before interviewing me. Ask about: - who I am and what I am trying to accomplish; - the first administrative, memory, communication, or physical burden I want the computer to carry; - my current home, approximate device count, exact device models, hubs, bridges, speakers, and voice endpoints; - machine-readable inventory sources: Home Assistant devices and entities, router clients, hubs, bridges, vendor apps, and existing automations; - a floor plan, sketch, photos, or room-and-outdoor-zone list; device locations; network boundaries; noisy areas; privacy-sensitive areas; and desired microphone and spoken-output coverage; - computers, phones, operating systems, network, and spare hardware; - notes, projects, tasks, messages, calendar, and personal-context systems; - the people who live in or regularly visit the home; - jurisdiction and recording-consent requirements; - retention preferences and the exact local/cloud boundary I want; - budget, comfort level, accessibility needs, and maintenance tolerance; - which actions I want the system to observe, propose, confirm, perform and notify, or perform silently inside an explicit rule. Inventory and reuse what I own before recommending purchases. Normalize the inventory and physical topology into editable structured data. Flag duplicates, stale entities, unknown devices, conflicting names, missing areas, and untested voice zones. Treat an initial drawing as bootstrap evidence; update the working topology only through observed state, logs, and explicit corrections. CORE ARCHITECTURE Keep the two runtime environments separate: - HOME ASSISTANT controls devices, sensors, automations, speakers, media, voice endpoints, and physical state. - THE PERSONAL AI ASSISTANT can use only the approved projects, notes, tasks, messages, calendar, files, computer tools, decisions, preferences, and memory. Connect them through one CONTROLLED INTERFACE. Home Assistant's built-in MCP server is one possible implementation; use it only if current official documentation and my requirements support it. Expose only selected state and actions. Use a separate credential. Define scope, logs, allowed and denied examples, immediate revocation, and rollback. Keep immediate deterministic Home Assistant behavior local where useful. Do not route a light switch through deep reasoning. Use the personal AI assistant when the workflow requires work or life context, computer use, tools, or judgment. OPTIONAL VOICE AND CAPTURE INTERFACES Do not require new voice or capture hardware for the core connection. If I choose room voice, phone voice, wearable capture, continuous transcription, spoken clarification, or proactive alerts, add them only after the controlled interface and one cross-system workflow pass. Separate mobile-owner input from fixed-zone coverage. A wearable microphone may cover me while I move, but visitors still need an approved way to address the house and the house still needs approved spoken-output paths. Test each named room and outdoor zone using the endpoints, phones, and speakers I already own. Do not assume one endpoint per room or treat a two-unit starter as whole-house coverage. Recommend more fixed endpoints only after the coverage test identifies a gap. If one microphone feeds both addressed requests and longer capture, it should be owned by one capture process so parallel listeners do not fight for the audio device. Produce stable timestamped transcript chunks. PATH A — ADDRESSED REQUEST ROUTING A local address detector watches the transcript. When I address the assistant, it packages the relevant transcript window with a timestamp and places it in a queue. The addressed name is a router, not a Home Assistant-only wake command. Route each request to the correct environment: - THE PERSONAL AI ASSISTANT for computer work, live threads, context questions, tasks, messages, or other approved tools; or - HOME ASSISTANT for selected physical state, devices, media, and routines. Return the result through an approved speaker or phone. Support both directions: I can send a request, and the system can initiate a question or alert under an explicit proactivity rule. PATH B — CONTEXT RETENTION All approved speech may become timestamped context even when it is not addressed to the assistant. Reconcile it with the sources I choose, such as work telemetry, screenshot/OCR summaries, notes, messages, projects, history, calendar, and operating rules. Use the context only within the selected permissions, retention, and local/cloud boundary. DEVICE COMPATIBILITY TEST Before recommending any purchase, ask: Will this exact model remain usable in an owner-controlled Home Assistant system? Check: 1. Can it operate locally if the vendor cloud disappears? 2. Does Home Assistant support the exact model well today? 3. Is there a documented local or open protocol? 4. Can its data and controls be selectively exposed? 5. Can it be migrated or replaced without rebuilding the system? Prefer Matter, Zigbee, Thread, documented local APIs, or mature Home Assistant integrations when the exact product passes the full test. Matter improves interoperability; it does not make poor hardware, firmware, radios, microphones, or power supplies good. Do not assume open source or an open protocol is automatically secure or well implemented. Research current compatibility, limitations, and pricing from official sources before recommending a purchase. State the date and region of every price. SECURITY, HOUSEHOLD, AND DATA BOUNDARIES - Never expose Home Assistant directly to the public internet. - Use Home Assistant Cloud or a reputable VPN for remote access when needed. - Keep secrets out of prompts, code, screenshots, recordings, transcripts, logs, and public files. - Use strong unique credentials and multi-factor authentication where supported. - Keep the host operating system and Home Assistant updated. - Create and verify backups before every migration phase. - Prefer official and local integrations; review custom integrations before installing them. - Make local versus cloud processing explicit for every component and data type. - Ask about household expectations, regular visitors, jurisdiction, and notice before broad recording or visitor-facing behavior. - Define raw-audio and transcript retention, deletion, and access. - Keep an action log and a fast revocation path for every granted capability. - Do not design this as emergency, medical, life-safety, security, or commercial monitoring. PERMISSION MODEL Present these five owner-selectable rungs for each capability: 1. Observe / read only 2. Draft or propose 3. Act after confirmation 4. Act and notify 5. Act silently within an explicit rule Do not permanently lock a capability because a mass-market vendor would. Suggest tighter starting defaults for outbound messages, purchases, doors, alarms, ovens, financial actions, and irreversible actions, while allowing me to promote capability after tests. For every granted capability define: - visible scope; - allowed and denied examples; - action log; - pass/fail test; - immediate revocation; - rollback or safe state; - promotion criteria. BUILD IN FIVE PHASES Phase 1 — Inventory both systems and choose one workflow Document Home Assistant, the personal AI assistant, one useful cross-system workflow, interface options, credentials, data flow, permissions, pass condition, revocation, and rollback. Do not change production or buy hardware. Phase 2 — Verify one Home Assistant state and one action Use one existing entity or sensor and one low-consequence action. Test correct state, action, logging, restart, denial, manual fallback, clean disable, and rollback. Phase 3 — Connect the personal AI assistant through the controlled interface Expose only the selected Home Assistant state and action through a separate credential. Test one allowed read, one allowed action, one confirmation, one denied action, logging, immediate revocation, and restoration. Phase 4 — Complete one workflow across the house and work/life tools Connect exactly one approved work or life tool. Complete the Phase 1 workflow. Test success, stale state, missing information, ambiguity, duplicate prevention, unauthorized action, tool failure, revocation, and undo. Phase 5 — Add optional voice, wearable capture, and proactive return Add only the interfaces I choose after Phase 4 passes. Apply household, jurisdiction, retention, local/cloud, mute, stop, recovery, self-voice suppression, stale-event, logging, revocation, and rollback rules. OPERATIONS, DASHBOARDS, AND SELF-CORRECTION After the first behavior passes, create a system-health view for device and entity availability, stale sensors, automation failures, queue and bridge health, voice/capture process health, endpoint coverage, latency, backups, restart recovery, unresolved corrections, and regressions. If I want owner-defined KPI dashboards, interview me about the measures I actually use to run my work, health, household, calendar, communication, obligations, or finances. Use only approved read sources. Define source, calculation, freshness, target, uncertainty, correction path, and the decision each KPI supports. Build a bounded QA loop: 1. define expected state, action, zone, response, latency, permission result, log, fallback, and rollback; 2. capture failures and owner corrections; 3. classify and safely reproduce the failure; 4. change one bounded variable; 5. rerun the original and adjacent regression tests; 6. promote only after the pass threshold; 7. roll back if the change creates a regression. The QA loop may repair configuration and code within its authority. It may not silently broaden permissions, purchase hardware, rename physical areas, or change irreversible behavior. FOR EVERY PHASE PROVIDE - goal; - prerequisites; - exact observable outcome; - current official sources; - commands or configuration appropriate to my actual environment; - the redacted log information I should return when something fails; - pass/fail test; - rollback; - what remains unproven; - the next reversible step. IMPLEMENTATION BEHAVIOR - Inspect logs and repair under my direction. - Never invent private paths, IP addresses, entity IDs, credentials, transcripts, topology, or a capability that has not been mechanically proven. - Do not claim all processing is local unless every component has been verified as local. - Add self-healing only after the simple path works and can be disabled. - When blocked, ask one concise question instead of guessing. - Keep public documentation sanitized. - End each work session with what changed, what passed, what failed, what was rolled back, what remains unproven, and the next reversible step. Begin with the interview. Then propose the smallest Phase 1 plan for one useful cross-system workflow. OWNER PROFILE If no structured OWNER PROFILE appears below, obtain it through the interview before designing or purchasing anything.
Validate the system before expanding permissions.
This is working infrastructure, not a maintenance-free consumer appliance.
Validation checklist
- Restore a backup successfully at least once.
- Confirm remote access uses Home Assistant Cloud or a VPN, not a public port.
- Search prompts, code, screenshots, transcripts, and logs for exposed secrets.
- Write down the local/cloud boundary and retention rule.
- Test every allowed action and at least one denied action.
- Verify every permission has scope, log, revocation, test, and rollback.
- Test restart recovery and the manual fallback.
- Address other residents, regular visitors, notice, and recording rules before broad capture.
- Keep high-consequence actions at the chosen rung until their promotion tests pass.
Dated build results and official documentation.
- May 31–June 1, 2026: Home Assistant Green and Voice Preview Edition completed the wake → speech recognition → action/agent → spoken response loop.
- June 9–10, 2026: custom wake and continuous-capture path mechanically proven. The wake name was “Adrian” because the rabbit hole began with Scott wanting to yell “Yo, Adrian” at the house. That is the Philadelphia/Rocky footnote, not part of the architecture.
- Measured test period: roughly 0.5-second wake latency and zero false activations per hour after transcription confirmation.
- July 23, 2026: system still in use; public facts and prices checked.
Facts and prices checked July 23, 2026 · Complete public build kit: this page