Cloud Overview

What the cloud tier does, what leaves the device, and how it differs from on-device.

neccessory ships in two tiers. Both return the same measurement result and both are driven by the same SDK call. What differs is where the inference runs — and therefore what leaves the device.

The cloud tier transmits images of the face

On-device, raw camera frames never leave the device. This is not true of the cloud tier. In cloud mode the SDK sends cropped images of the face to our servers, and the metrics are computed there. The crops live only in the memory of the running session — they are never written to disk, to logs, or to object storage — but they are transmitted, which makes the cloud tier processing of biometric personal data. Read Cloud privacy before you ship it.

The two tiers

	Cloud	On-device
Where metrics are computed	Our servers	The phone or the browser
What leaves the device	Face crops during the measurement	Nothing but the finished result, and only if you submit it
Model weights on the client	No	Yes
Client assets to download	~6.9 MB (face tracker only)	~45 MB (full engine + models)
Works offline	No	Yes
Latency	One network round-trip per event	Real time, no network
Attested result	Yes	No
Batch processing of recorded video	Yes	No
Access	Self-serve	On request — see Obtaining keys

The tiers are selected by the key mode and by one argument in the SDK:

createMeasurement({ videoEl, mode: 'cloud',     sessionToken });
createMeasurement({ videoEl, mode: 'on-device', modelGrant });

Everything downstream — callbacks, the result shape, the REST API — is identical. Moving from cloud to on-device after your access request is approved is a one-line change.

Who does what

Stage	Cloud	On-device
Camera capture	Client	Client
Face detection and cropping	Client (server for `batch_video`)	Client
Pulse recovery, heart rate, HRV, breathing, stress	Server	Client
Blood pressure, SpO2	Server	Client
Attestation of the result	Server	Not available

Face detection stays on the client for the streaming and frame-batch modes, so a cloud client still needs a face tracker — but only the public Google models (~6.9 MB over the wire with brotli), not the full measurement engine.

Three intake modes

Mode	What you send	Transport	Live events	Face detection
`stream`	36×36 face crops, up to 30 per second	WebSocket	Yes	Client
`batch_frames`	The same frames, packed into one file	Presigned `PUT`	No	Client
`batch_video`	An mp4 / mov file	Presigned `PUT`	No	Server

stream is the interactive measurement: live heart rate, live quality, live prompts. This is what a browser or a mobile app normally uses. See Streaming protocol.
batch_frames is the same client SDK on a bad network: frames are buffered locally and uploaded as one file at the end. Degradation instead of a dropped measurement.
batch_video is an integration with no browser at all — your backend posts a recorded file. This is the only mode where face detection and video decoding happen on our side. See Batch processing.

Delivery is your choice, expressed as one option:

createMeasurement({ videoEl, mode: 'cloud', sessionToken, delivery: 'auto' });
// delivery: 'stream' | 'batch' | 'auto'   (default: 'auto')

With auto the SDK streams and keeps a local buffer. If the socket never opens, or dies mid-measurement, the buffer is uploaded as batch_frames and the measurement still produces a result. Live metrics stop at that point, so the SDK reports onStage('uploading') and then onStage('processing') — show that instead of a frozen pulse.

Metric availability

Metric	Cloud	On-device
Heart rate, HRV, breathing rate, stress	Yes	Yes
Blood pressure, SpO2 (Beta)	Yes, on the server	Yes, locally
Emotion, gaze, attention (Beta)	Yes, when ordered in `features`	Yes
Fatigue, blink rate	Not in cloud yet	Yes

Emotion is a separate, optional module

Emotion inference is not a physiological metric, it is optional, and it is legally restricted in the EU for workplace and education use. In the cloud it runs only when the session ordered the emotion set — see the protocol for the events and Metrics for the restriction.

Choosing a tier

Choose cloud when you want to start today without an approval step, when you cannot ship tens of megabytes of models to the client, when you need a result the end user cannot fabricate, or when you have recorded video to process — and when transmitting face crops to us is acceptable for your users and your legal basis.

Choose on-device when your privacy posture requires that no image ever leaves the device, or when the client must work offline or at zero latency. On-device access is granted by application — see Obtaining keys.

The costs, stated plainly

Bandwidth. The pulse crop alone is 3 900 bytes per frame — about 117 KB/s at 30 fps, roughly 7 MB for a 60-second measurement. Enabling the optional auxiliary crop raises the total to roughly 160–180 KB/s.
Session stickiness. A streaming session keeps recurrent model state in the memory of one node, so it is pinned to that node for its whole duration. The batch modes have no such constraint.
No resume. If the WebSocket drops, the streaming session is finished (aborted). delivery: 'auto' is what turns that into a completed measurement instead of a lost one.

Next: Cloud quickstart.