Skip to content
SearchSubscribe
Expert · Lesson 14 — AR research sprints when hardware isn't shipped
E14Expert
Expert · Lesson 14● live

AR research sprints when hardware isn’t shipped

Shipping Phase-0 R&D against hardware you can’t hold. The Parley approach to Everysight Maverick AI before the device ships.

20 min read · 60 min applyprereq: Expert 12 (custom agent files)

Unreleased hardware problem

Parley is building an AR-glasses product for real-time bidirectional transcription between deaf and hearing users. The canonical hardware target is Everysight Maverick AI. As of May 2026, the Maverick AI hasn’t shipped commercially. The public spec sheet is limited. You can’t hold the device and run your pipeline against it.

I think this is more common than people admit. Most serious AR teams work against hardware that’s either not shipped or not in their hands. The question isn’t “how do we wait?” It’s “what can we build now that will still be valid when the hardware ships?”

Parley’s Phase-0 approach: ship hardware-invariant research (ASR architecture benchmarking, signer-holdout methodology, failure taxonomy) while explicitly flagging hardware-dependent assumptions (on-device latency budget, HUD rendering pipeline, microphone placement) as unvalidated. The Phase-1 COMPLETE milestone as of April 26, Notebooks 00, 01, and 02 all shipped with top architecture landing at 0.4467 ± 0.0097 on signer-holdout, shows what’s possible before the device is in hand.

The signer-holdout gap, roughly 35 percentage points below the random-split Kaggle leaderboard, is the most important finding from Phase 1. It exposes the benchmark’s structural optimism. It’s hardware-invariant. The gap exists regardless of which AR glasses are eventually targeted. That kind of finding compounds. Hardware-dependent work comes later.

Can-do vs can’t-do

Partitioning is the first discipline. Before any sprint planning, every work item gets categorized as hardware-invariant or hardware-dependent.

CategoryExamplesPublication readiness
Hardware-invariantASR model accuracy on benchmark and holdout datasets; signer generalization methodology; failure taxonomy; architecture comparison; dataset curation approachesPublish now — findings hold regardless of specific hardware
Hardware-inferredLatency budget estimated from comparable glasses; rendering pipeline designed for inferred HUD spec; audio preprocessing designed for assumed microphone placementPublish with explicit caveat — conclusions may change when hardware specs confirmed
Hardware-dependentOn-device inference timing; HUD-specific rendering optimization; real microphone placement effects on ASR; thermal throttling under sustained loadFlag for Phase 2+ — requires device in hand to measure

Parley’s community publication standard is higher than for internal research, specifically because the Deaf community deserves accurate capability claims. The r/deaf subreddit is on explicit hold until a Deaf-community advisor has reviewed Phase-2 numbers. “Our Kaggle benchmark shows 95% accuracy” and “our product works 95% of the time for real conversations” are very different claims. The hardware assumptions register prevents the conflation.

Parley Phase-0 sprint structure

Research question hierarchy

Phase 0 established four research questions, ordered by hardware-invariance. Q1 and Q2 are almost entirely hardware-invariant. Q3 and Q4 have hardware-dependent components that are flagged but not blocked.

Q1, architecture ceiling.What’s the best accuracy achievable on the Google ISLR dataset under a rigorous signer-holdout protocol? Entirely hardware-invariant. The answer (0.4467 ± 0.0097 for frame_transformer) benchmarks the field’s quiet failure mode and establishes Parley’s baseline. Doesn’t need a device.

Q1.5, recipe vs architecture.Notebook 02’s finding that 5 of 7 architectures collapse or lottery-ticket under a shared training recipe raises a sub-question. Is the ceiling set by the architecture or by the recipe? It emerged from Phase 1 and is now the Notebook 03 candidate. Still hardware-invariant.

Q2, signer diversity. Does the architecture ceiling improve with signer-diverse training data, and how? This is where hardware-inferred work enters. Real-world signer diversity includes signers using AR glasses, which may produce different video properties. Phase-2 work addresses WLASL as a secondary dataset (kicked off in commit 926b7b6) before device-specific signer data is available.

Q3 and Q4, audio-visual sync and latency.Hardware-dependent. They need the Maverick AI specifically. Microphone placement, camera sync timing, edge compute profile. Planned for Phase 3–4 when device access becomes possible. Sprint work that touches these questions is flagged in the notebook.

Recorder spec as a bridge artifact

The Recorder, scope-locked in commit abda055 on April 25, is a Phase-3 capture application that will collect labeled video data from real signers using real hardware. It’s the bridge artifact. Built now (hardware-invariant in its architecture), deployed later (hardware-dependent in its AR-glasses integration).

The Recorder spec is the right shape for hardware-dependent work under uncertainty. Design the architecture now (capture format, labeling schema, upload pipeline to Supabase project gjwpepeyywqxuekwqtix). Defer the device-specific integration until the device is in hand. The work done now doesn’t need the device. The work deferred does.

Agent collaboration during AR research sprints

The Parley internal agent team (HELM for sprint orchestration, CORTEX for data pipeline, LEXICON for ASR/NLP, PLANNER for roadmap, BEACON for QA, CIPHER for IP, SCRIBE for documentation) runs under the same hardware partition discipline. CORTEX’s work on signlang/data/landmarks.pyis hardware-invariant (landmark extraction from existing video datasets). PRISM’s rendering work (Phase 6) is deferred until device specs are confirmed.

Each agent’s scope includes a hardware-dependent flag. When a session drifts toward hardware-dependent work before the hardware is available, the agent flags it and routes to PLANNER for deferral rather than producing speculative conclusions.

Three failure patterns

Where AR research sprints fail when hardware isn’t shipped. Each failure is a variant of the same root cause: treating hardware assumptions as confirmed facts.

№ 01

Building to an unknown display spec

claim looks like“A team designs an AR transcription overlay that requires 1280×720 HUD resolution, 60Hz refresh, and full-color rendering. The target hardware (Everysight Maverick AI) ships with a different display specification. The rendering pipeline must be redesigned. Three months of UI work requires redo.”
what’s missingDisplay specification as an explicit design constraint from day one. When the hardware isn't shipped, the display spec is unknown. The design should explicitly flag every assumption about display capability and design for the lowest confirmed specification — or design the pipeline to be display-agnostic until hardware specs are confirmed.
the moveAt the start of any AR UI sprint, create a hardware assumptions register. List every display property the design depends on: resolution, color depth, refresh rate, FOV, luminance range, text rendering method. For each: mark it as 'confirmed in hardware docs,' 'assumed from similar device,' or 'unknown.' Design decisions that depend on 'unknown' properties must be reversible. Never build an irreversible pipeline on an unknown hardware spec.
№ 02

ASR model validation without representative audio

claim looks like“A team validates their ASR model (Whisper large-v3) on standard benchmark datasets and achieves 95% WER. They assume this translates to the Maverick AI use case: real-time transcription of conversational speech rendered on an AR HUD for a deaf wearer. In field testing, WER drops to 68% because the benchmark data doesn't include the specific failure modes of AR glasses audio: wind noise, body movement acoustics, and microphone placement on the frame.”
what’s missingHardware-realistic audio conditions in the validation set. AR glasses have specific microphone placements and acoustic properties that standard ASR benchmarks don't model. Without device-realistic audio, validation numbers don't predict field performance.
the moveBuild a hardware-realistic audio validation set before trusting ASR numbers. For Maverick AI specifically: collect audio samples with a similarly-placed microphone (head-mounted, body-movement-present), in the target environments (indoor conversation, mild outdoor). Notebook 00's signer-holdout finding — 35pp gap between benchmark and real performance — is the ASR equivalent for sign language recognition. Design for the gap.
№ 03

Latency budget designed for server compute

claim looks like“An ASR + rendering pipeline is validated on a MacBook Pro and achieves 120ms end-to-end latency. The team designs the product UX around sub-200ms response. The Maverick AI edge compute budget is constrained. Porting the pipeline to on-device compute produces 800ms latency. The UX assumptions collapse.”
what’s missingAn on-device compute budget as a first-class design constraint. AR glasses have limited power and thermal envelopes. Any pipeline designed for server or laptop compute will not port to glasses without significant redesign. The latency budget must be designed against the target device's constraint, not the development machine's capability.
the moveEstablish an explicit latency budget for the target device class before designing the pipeline. For Everysight Maverick AI: estimate the on-device compute budget from public specs and comparable device benchmarks. Design the pipeline to fit that budget, not the development machine's budget. Flag any component that is currently running on server compute and will require porting as a red item in the sprint plan.

Common thread: hardware assumptions treated as facts before the hardware ships. The hardware assumptions register, built into every notebook with hardware-dependent conclusions, is the structural fix.

Hardware-assumption audit

Run this on any research notebook or sprint plan that involves unreleased or constrained hardware. It surfaces every conclusion that depends on an unconfirmed hardware property and grades whether the work is reversible.

The hardware-assumption audit
Run a hardware-assumption audit on this research notebook
or sprint plan.

Context: [describe the AR/hardware platform you're targeting]
Hardware availability: [shipped / pre-release specs only / limited docs]
Notebook or plan contents: [paste or summarize]

For every conclusion, result, or design decision in the content:

1. HARDWARE DEPENDENCY: Does this conclusion depend on
   a specific hardware property?
   (display resolution, compute budget, microphone placement,
   sensor spec, form factor, OS/runtime)

2. STATUS OF DEPENDENCY: Is that property:
   a) Confirmed in public hardware documentation
   b) Inferred from comparable devices
   c) Unknown — no public data

3. IF WRONG: What happens to this conclusion if the
   hardware property is different from your assumption?
   Is the conclusion reversible (can be updated when
   hardware ships) or irreversible (requires rework)?

Output: a hardware assumptions register table + a list
of any conclusions that should be flagged or caveated
in publication.

Produces a hardware assumptions register: every hardware dependency, its status, and the implication if the assumption is wrong. Run before publishing any AR research notebook.

For Parley specifically: Notebooks 00, 01, and 02 are hardware-invariant and need no caveats on their core findings. Notebook 03 and beyond will start touching hardware-inferred territory as the work moves toward real-time inference. The audit runs on each notebook before publication.

Annotated transcript

Structuring Parley’s Phase-0 research sprint against Everysight Maverick AI hardware. Two turns, two answers. The second turn addresses the trickiest publication question: representing benchmark findings honestly when the deployment gap isn’t yet measured.

Structuring Parley Phase-0 research against unreleased hardware

Two operator turns, two agent turns. Result: hardware-invariant work identified, hardware-dependent work flagged, publication caveats for the signer-holdout finding.

  1. operator

    Parley is building for Everysight Maverick AI AR glasses. They're not shipped yet — we only have limited public specs. We need to ship Phase-0 research notebooks on ASR accuracy. How do we structure this so the work is valid when the hardware ships?

  2. agent

    Structure the research around what's hardware-invariant and explicitly flag everything that's hardware-dependent. Here's the partition:

    Hardware-invariant work (ship now):
    - ASR model accuracy on sign-language-adjacent audio: Whisper large-v3 vs other models on the same validation set
    - Failure taxonomy: what classes of input does each model fail on, and why
    - Signer-holdout methodology: building an eval set that tests generalization across signers, not just accuracy on training distribution
    - The architecture ceiling finding: what's the best possible WER you can achieve before the hardware bottleneck, so you know where the ceiling is

    Hardware-dependent work (flag explicitly, design reversibly):
    - On-device latency: you don't know the Maverick AI compute budget
    - Rendering pipeline: you don't know the exact HUD spec
    - Microphone placement: this affects ASR performance significantly
    - Audio preprocessing: wind rejection, body-movement noise — device-specific

    The Notebook 00 signer-holdout finding (35pp gap between benchmark and real performance) is actually perfect Phase-0 work. That finding is hardware-invariant — the benchmark gap exists regardless of which AR glasses you're targeting. Publish it. The hardware-dependent work comes in Phase 2-3 when device specs are confirmed.

    Write a hardware assumptions register into every notebook that has hardware-dependent conclusions. It's a table: assumption, status (confirmed/inferred/unknown), implication if wrong.

  3. operator

    What's the biggest risk of the signer-holdout methodology for Parley specifically? We're publishing on Kaggle with public data (Google ISLR) where the benchmark already exists.

  4. agent

    The biggest risk is publishing a finding that's accurate for the benchmark but misleading for the product use case.

    The Google ISLR benchmark is a controlled dataset. The Parley use case is real-time bidirectional transcription on AR glasses in conversation — ambient noise, variable lighting, signers who weren't in ISLR, and the specific failure modes of AR frame-mounted cameras (angle variation, motion blur from head movement).

    The signer-holdout finding — that 0.4467 accuracy on signer-holdout is ~35pp below the random-split leaderboard — is important precisely because it exposes the benchmark's optimism. But it doesn't fully characterize the Parley deployment gap, because the deployment conditions are harder than the benchmark in ways the holdout test doesn't capture.

    The right way to publish: state the signer-holdout finding accurately, note the conditions under which it was measured (ISLR data, controlled video), and explicitly flag the additional deployment gaps that aren't measured yet: AR frame camera angle variation, real-time audio-visual sync, ambient noise in conversation settings. Those become the research questions for Phase 2 and 3.

    Never represent a benchmark finding as a deployment guarantee when the deployment conditions are harder than the benchmark. r/deaf is on hold for exactly this reason — the community deserves accurate numbers, not optimistic benchmark-to-deployment projections.

AR research sprint template

A template for structuring a research sprint when the target hardware isn’t in hand. Fill this out at sprint planning, before any work begins.

AR RESEARCH SPRINT TEMPLATE
(for use when target hardware is pre-release or unavailable)

SPRINT GOAL: [what you're trying to produce]
TARGET HARDWARE: [name the device — even if not shipped]
AVAILABLE HARDWARE INFO: [public specs, SDK docs, comparable devices]
HARDWARE AVAILABILITY DATE: [estimated or unknown]

HARDWARE ASSUMPTIONS REGISTER
(fill before sprint begins; update as hardware info arrives)

| Property          | Value assumed   | Status*  | Risk if wrong |
|-------------------|----------------|----------|---------------|
| Display resolution| [value]        | [status] | [impact]      |
| Compute budget    | [value]        | [status] | [impact]      |
| Mic placement     | [value]        | [status] | [impact]      |
| Camera spec       | [value]        | [status] | [impact]      |
| [add rows...]     |                |          |               |

*Status: confirmed = in hardware docs | inferred = from comparable device
        unknown = no data

WORK PARTITION

Hardware-invariant (ship this sprint):
[ ] [work item 1]
[ ] [work item 2]

Hardware-inferred (ship with caveat):
[ ] [work item — note which assumption it depends on]

Hardware-dependent (defer to Phase N):
[ ] [work item — will require device in hand]

PUBLICATION CHECKLIST
[ ] Every finding states the conditions under which
    it was measured
[ ] Hardware-inferred conclusions have explicit caveats
[ ] Hardware-dependent conclusions are NOT presented as
    confirmed findings
[ ] Community-facing publications (r/deaf, etc.) reviewed
    by domain advisor before posting
[ ] Deployment gap between test conditions and real
    deployment is acknowledged

HARDWARE-CONFIRMED GATE
When hardware ships: revisit [list the hardware-inferred
and hardware-dependent items above] with device in hand.

Prompt kit

Three prompts. Audit hardware assumptions, partition sprint work, write caveated findings. I keep mine in the research project CLAUDE.md.

The hardware-assumption audit
Run a hardware-assumption audit on this research notebook
or sprint plan.

Context: [describe the AR/hardware platform you're targeting]
Hardware availability: [shipped / pre-release specs only / limited docs]
Notebook or plan contents: [paste or summarize]

For every conclusion, result, or design decision in the content:

1. HARDWARE DEPENDENCY: Does this conclusion depend on
   a specific hardware property?
   (display resolution, compute budget, microphone placement,
   sensor spec, form factor, OS/runtime)

2. STATUS OF DEPENDENCY: Is that property:
   a) Confirmed in public hardware documentation
   b) Inferred from comparable devices
   c) Unknown — no public data

3. IF WRONG: What happens to this conclusion if the
   hardware property is different from your assumption?
   Is the conclusion reversible (can be updated when
   hardware ships) or irreversible (requires rework)?

Output: a hardware assumptions register table + a list
of any conclusions that should be flagged or caveated
in publication.
Partition a research sprint into hardware-invariant vs hardware-dependent work
Partition this research sprint into hardware-invariant
and hardware-dependent work.

Sprint goal: [describe what you're trying to ship]
Target hardware: [name the device — even if not shipped]
Available hardware info: [what you know about the device]

For each work item in the sprint:
1. INVARIANT or DEPENDENT?
   - Invariant: result holds regardless of specific
     hardware properties; can be published now
   - Dependent: result requires specific hardware
     properties to be meaningful; flag for later

2. For DEPENDENT items:
   - What specific hardware property does it depend on?
   - Is that property confirmed, inferred, or unknown?
   - Can the work be done reversibly now (so it can
     be updated when hardware ships)?

Output: two lists — ship-now work and flagged work —
with the specific dependency noted for each flagged item.
Write a hardware-caveated research finding
Rewrite this research finding with appropriate
hardware caveats.

Original finding: [paste the finding as written]
Hardware target: [name the device]
Hardware availability: [shipped / pre-release specs only]
Known gaps between test conditions and deployment: [list them]

The rewritten finding must:
1. State the result accurately under the conditions tested
2. Name the specific conditions (dataset, hardware, setup)
3. Flag the deployment gap: what's different between
   test conditions and real deployment conditions?
4. Distinguish: what's likely to hold, what's uncertain,
   what requires hardware-in-hand validation

Tone: direct, honest, not hedged to the point of being
useless. A finding with appropriate caveats is more
valuable than a finding with false confidence.

Apply this — structure one pre-hardware sprint

~60-minute exercise. Audit, partition, register. Ship the invariant work; flag the rest.

Your first hardware-assumptions-registered sprint

Each step takes 10-20 minutes. Progress saves automatically.

0/5
  1. 01Create a hardware assumptions register for your current AR research sprint. List every hardware property your conclusions depend on.Use three status categories: confirmed, inferred, unknown. Every 'unknown' is a flag — the conclusion that depends on it needs a caveat or needs to be redesigned as reversible.
  2. 02Run the hardware-assumption audit on your most recent notebook or sprint deliverable. Find at least one conclusion that needs a caveat.The most common finding: a latency or accuracy number presented without the test conditions that make it meaningful. Adding the conditions converts a number into a reproducible finding.
  3. 03Partition your next sprint plan into hardware-invariant and hardware-dependent work. Move all hardware-invariant work to the front of the sprint.Hardware-invariant work compounds — findings from it remain valid as the hardware picture clarifies. Hardware-dependent work done on assumptions may need to be redone. Front-load the durable work.
  4. 04Write one published finding with explicit deployment caveats. Use the prompt in the kit.The standard for publications involving community stakeholders (like r/deaf for Parley) is higher than the standard for internal research notes. Publishing caveated findings is more valuable than publishing confident-sounding numbers that don't hold in deployment.
  5. 05Define the hardware-confirmed gate for your sprint: at what point do hardware specs become available, and what work will you revisit at that point?Without this gate, hardware-dependent work just accumulates. The gate converts it from deferred debt into a planned revision — you know exactly which conclusions to revisit when the device ships.
Expert tier · what's next

After this lesson