Offline pocket AI assistant

Speculative concept
AI illustration · Illustrative scene, not a photograph of the specific device or experiment.

This is an idea, not a product. Nothing here has been built or tested unless a linked technology profile says so.

The problem

Some people may want a voice device that handles a few simple requests without sending their speech to a remote server. This is an untested hypothesis, not a measured need.

The idea

A possible first version is a battery-powered handheld with a microphone and a low-power processor that recognises a short, fixed list of spoken commands, such as setting a timer or a reminder. The concept assumes a neuromorphic processor, but no specific chip has been selected. The device would process speech locally, would not hold open-ended conversations and would not run a general language model. Its scope would be narrow so that its behaviour can be checked against a fixed test set.

Enabling technology

  • Neuromorphic computing chips: Neuromorphic chips are processors designed around ideas from the brain, such as many simple neurons that communicate with short spikes and memory placed right next to computation. Research chips and large research systems exist and report large energy savings on some tasks under specific test conditions, but they are not yet a mainstream alternative to GPUs.

What stands in the way

  • Software tools and programming models for neuromorphic hardware are far less mature than those for CPUs and GPUs, so few developers can build on it.
  • Training spiking networks to match the accuracy of conventional deep learning is still an active research problem, so a command model may need to be trained on a conventional computer and then mapped to the chip.
  • The neuromorphic profile reports no speech or spoken-command result, so the accuracy of such a device on this task is unknown.
  • The low-power figures in the neuromorphic profile come from video, image classification and simulation tasks, not speech, so they cannot be carried over to this device.
  • Reported efficiency gains are usually measured on selected benchmarks and are often supplied by the chip makers, so they may not hold for a speech task on a small battery.
  • The large systems that show the strongest results, such as Hala Point, are research platforms rather than commercially deployed products, and no small shipped device is identified in the profile.
  • The sources reviewed for the neuromorphic profile test on-device learning on benchmark tasks only, so the device cannot be assumed to adapt safely to a user's own voice.

A cheap way to test it

On a bench, with no human trials, run a fixed set of pre-recorded spoken commands through a prototype board. The recordings should be made by the team or with each speaker's consent. Record the share of commands recognised correctly, and the average battery power drawn over a fixed run time, measured with a power meter. Run the board with its radio disabled and monitor its network-facing interfaces, recording any attempt to send data off the board. Write the pass thresholds down before the run. The prototype passes only if recognition meets its threshold, power draw meets its threshold, and no audio leaves the device. Repeat the same clips on a conventional low-power microcontroller as a baseline.