A talking mascot in one Rive file

Mascotbot↗ sells talking avatars for voice agents. A small cartoon sits in a circle in the corner of the page. When you call him, he stands up out of the circle, talks, and his mouth follows every word.

I wanted one of my own, so I built Freddy as a single Rive file. Move your cursor, try the buttons, and press "say hello".

Each button sets one property on Freddy's view model: standing up is a boolean, each mouth shape is a number, and the nod is a trigger. This page has no API key. "Say hello" plays a recording through the same lip sync a live call uses.

What's inside theirs

I opened Mascotbot's site in Chrome DevTools and loaded their .riv files into a bare Rive runtime to see what they contain. The mouth is fifteen number inputs named 100 to 118, one per mouth shape, and each one blends a pose on a separate mouth artboard. Around them are is_speaking, is_listening, a gesture trigger, a reveal trigger, and idle and speaking loops.

Drawing him

Freddy started in Codex as one front view. Then I asked for the same character turned three quarters in four poses: arms crossed, one hand explaining, a raised finger, and open palms. Each pose was an edit of the first, so the head and torso stay in the same place and only the arms change.

A script traces the PNGs into vectors. My first tracer filled each flat colour region and drew a uniform outline around it. Wherever the original line was thicker than my outline, the background showed through: between the neck and the shirt, in the hair, between the fingers.

So the tracer now keeps the line art as drawn. Each colour region grows a few pixels under the black lines, and the line art becomes one filled shape with holes, so every line keeps its real weight and no seam can show through.

Rigging in text

I didn't build him in the Rive editor. The Rive CLI↗ compiles .riv files from RML, a text format, so a Python script writes the whole scene, about 8,600 objects. Each head motion (breathing, the sway while talking, the listening tilt, the nod) gets its own nested node, so no two of them animate the same transform.

The CLI can also render a frame without a window and set view model values from the command line, so I checked every pose with a screenshot before loading it in a browser:

rive rive/freddy --screenshot=ah.png --data=isConnected=true --data=v101=100

Sixteen mouths

Every mouth shape is the same four-point path: two corners, an upper lip and a lower lip. Because each shape moves the same four points, any two shapes can blend, and halfway between "oh" and "ee" is a real mouth shape.

The shapes follow a hand-drawn reference sheet. Open vowels are a D: a nearly flat upper lip over a deep round bottom, with a band of top teeth and a tongue. "Oh" is a tall oval, "oo" a pucker, "ee" and "s" a wide grin with both rows of teeth, and "m" is pressed lips. His face is turned, so the far corner sits a little closer to the middle.

Rive applies blend animations in order, each one a lerp over the result so far. Give it two shapes at half weight each and the second one wins. So the weights are converted before they're written:

// convex weights c → Rive's sequential mixes, last one first
let remaining = 1;
for (let i = VISEMES.length - 1; i >= 0; i--) {
  const c = weight[i];
  mix[i] = remaining > 1e-4 ? c / remaining : 0;
  remaining -= c;
}

Lip sync from sound

The mouth follows the audio. Gemini's voice arrives a little before it plays, so each chunk is analysed as soon as it's queued, in 10 ms steps, and stamped with the time it will be heard. The mouth reads that timeline 35 ms early, about two frames, because a mouth that trails the sound looks late.

Each weight moves on a critically damped spring that settles in about 60 ms, so shapes ease into each other without overshooting. It isn't a trained model and it gets some sounds wrong. At talking speed you mostly see the rhythm. The mouth closes between syllables, opens wide on stressed ones and rounds on rounded vowels.

The call

In the full widget, pressing "Call Freddy" rings like a phone while a small server mints a single-use Gemini Live↗ token, locked to his voice and persona. The browser then talks to Gemini directly. The ringing stops when his first audio arrives, a short chime plays, and only then does the microphone open, so it never picks up the ring. If you talk over him, Gemini stops him and the queued audio is dropped.

This page doesn't make calls, so it has no key. The rest of Freddy is in the file.

Use him

Download freddy.riv (224 KB). It works with any Rive runtime↗: web, React, iOS, Android, Flutter.

The file has one artboard, Character, one state machine, mascotStateMachine, and a view model called Freddy. Blinking, breathing and the hover reaction run inside the file. You control the rest by setting these properties:

On the web, install @rive-app/webgl2 and point it at a canvas:

import { Rive } from "@rive-app/webgl2";
 
const freddy = new Rive({
  src: "freddy.riv",
  canvas: document.querySelector("canvas"),
  stateMachine: "mascotStateMachine",
  autoBind: true,
  autoplay: true,
  onLoad: () => {
    const vm = freddy.viewModelInstance;
    vm.boolean("isConnected").value = true; // stand up
    vm.boolean("isSpeaking").value = true;  // start gesturing
    vm.number("v101").value = 100;          // open on "ah"
  },
});

To make him talk, set isSpeaking while audio plays and move the v numbers every frame. One shape at 100 holds that mouth. Weights between 0 and 100 blend shapes, and all of them at 0 is his resting smile.