Prototype · live demo · Apache-2.0
ayaka-hand
A hand-prosthesis control system that pre-shapes the grip before contact. A wrist camera looks at the object being reached for and forms the grasp on the way in; the muscle signal is reduced to a single binary GO.
The user decides when to close. The vision system decides how.
- status
- prototype · public demo
- license
- Apache-2.0
- student
- 2,232,839 params
- weights
- 8.9 MB ONNX
- runtime
- in-browser · ORT Web
- grasp budget
- ≤125 ms
The problem
A recurring reason is cognitive load. Conventional control asks the user to consciously modulate muscle contractions to pick a grip and meter force, for every object, all day. ayaka-hand moves that work to a vision system and leaves the user exactly one decision.
How it works
Tier 1 · teacher · offline
Cortex
- model
- Qwen3-VL-32B-Instruct
- license
- Apache-2.0
- in the loop
- never
A vision-language model that reasons about an object's appearance and state and emits a structured grasp label. It labels the training set offline; in the demo it appears only as a slower second-opinion panel.
Tier 2 · reflex · on-device
Student
- model
- MobileNetV2 · grip + force heads
- size
- 2,232,839 params · 8.9 MB
- input
- wrist-camera pixels only
Distilled from the teacher's labels, this is what runs in real time, in the browser via ONNX-Runtime Web. No object detector tells it what it sees.
A grasp must be committed inside ≤125 ms to feel like part of the body. A cloud model cannot live in that budget, so it doesn't.
Results
The baseline is deliberately generous: it gets the ground-truth object class name (a perfect detector) and looks up a canonical grip for it. The student gets only pixels.
| Metric | Lookup baseline gets the true class name |
Student 2.23M · pixels only |
|---|---|---|
| Grip accuracy | 70.5 % | 72.3 % |
| Grip macro-F1 | 0.581 | 0.659 |
| Force accuracy | 65.4 % | 81.1 % |
| Force macro-F1 | 0.595 | 0.806 |
Grip accuracy
Grip macro-F1
Force accuracy
Force macro-F1
Pinch grip · macro-F1
0.000 → 0.323
No object class maps cleanly to a pinch; you have to look at the object. The baseline cannot score here. The student can.
Force accuracy · gain
+15.7 pts
Force depends on an object's state (a full cup versus an empty one), which a class name cannot encode.
What it is not
- 1A research prototype from a hackathon submission. Not a medical device. No injury-prevention claims are made.
- 2No physical hand has been built. The demo renders a simulated hand in the browser.
- 3The teacher can emit advisory hazard flags (hot, sharp, fragile). They are display-only: not distilled into the student, not wired to actuation. The system does not detect or prevent harm and must not be relied on to.
Data and licensing
Trained on 11,004 teacher-labelled COCO 2017 crops, validated on 613. Images were license-filtered to CC BY, CC BY-SA and no-known-restriction sources; everything NonCommercial or NoDerivatives was excluded, because the crops are derivative works. Code and documentation are Apache-2.0.