How it works

Both SDKs run the same conceptual pipeline. You never configure the internal models — you feed the camera, and you receive a measurement result. The stages below describe what happens between those two points.

Where each stage runs depends on the tier. In the on-device tier every stage below runs on the phone or in the browser. In the cloud tier stage 1 runs on the client and stages 2–6 run on our servers, which means the cropped face images from stage 1 are transmitted to us — see Cloud overview. Each stage is marked accordingly.

1. Face tracking

On-device: local. Cloud: on the client (except batch_video, where we decode the video and detect the face ourselves).

Each frame is scanned for a face. When one is found, a stable region on the face is tracked frame-to-frame — this is where the color signal is strongest and most reliable. In the on-device tier this happens locally and no image is sent anywhere; in the cloud tier the cropped region is what gets sent, and nothing else from the frame is.

2. Quality gating

Before the pulse signal is trusted, each frame is checked against quality conditions — is a face present, is the scene bright enough, is the person holding still, is the camera running fast enough. Frames that fail are not accumulated, which keeps noisy data out of the result. The overall outcome is condensed into a single signal-quality rating you can react to in real time.

3. Pulse recovery

On-device: local. Cloud: on our servers.

Accepted frames feed the pulse-recovery stage, which reconstructs the pulse waveform from the tiny color changes in the tracked region. Neural networks and signal processing clean this signal and turn it into the raw material for every metric. This is the stage that needs the model weights, and it is the reason the tiers differ at all: on-device those weights are on the client, in the cloud they never leave our servers.

4. Metric estimation

• Heart rate comes from the rhythm of the recovered pulse, with a confidence value.
• HRV (SDNN, RMSSD, pNN50) is computed from the beat-to-beat timing. It needs many clean beats — typically ~15 seconds of good signal before a first estimate.
• Breathing rate is derived from the slow modulation of the pulse over several seconds.
• Stress index condenses the variability pattern into a single 0–100 value.
• Blood pressure and SpO2 (experimental, Beta) are estimated from the pulse and are clearly demoted as wellness-only.

Each metric only appears once there is enough clean signal to support it. Until then its value is withheld rather than shown as a noisy guess.

5. Best-result selection

On-device: local. Cloud: on our servers.

Throughout the measurement the signal is continuously scored and the best window is kept. When the measurement completes, the strongest result is returned as the final value, together with the signal-quality rating. This is the value you present to the user.

In the cloud tier this selection happens server-side, so the result arrives with origin: "cloud" and attested: true — the client received the number, it did not choose it. See Attestation.

6. Signal quality on every result

Every completed measurement carries a required quality object (good / fair / poor, plus a 0–1 score and a usable flag). Design your UI to respect it: when quality is poor, hold or hide values rather than presenting an unreliable number. See Signal quality.

Next: Requirements & limitations.