SonicFrame v1.3.2

The wave in words

For people who cannot see the wave: a short description of the sound, read by their own screen reader when they reach the player. It is shown here as text too. Off until you turn it on.

Loading the sound…

Try it

Keys

  • D says the description again
  • P and Shift + P step between the parts
  • The speech bubble reads it with your browser’s own voice

In code

On a frame of your own, it is the soundDescription plugin:

import {
    PlayButton,
    SonicFrame,
    loadWaveform,
    soundDescription,
} from 'sonicframe';

const box = document.querySelector('#words-frame');
box.style.display = 'flex';
box.style.alignItems = 'center';
box.style.gap = '0.75rem';
const media = new Audio('./sounds/little-elf.mp3');

PlayButton.create({ container: box, media });
const wave = document.createElement('div');
wave.style.flex = '1';
wave.style.height = '4rem';
box.append(wave);
const frame = new SonicFrame({
    container: wave,
    media,
    peaks: () => loadWaveform('./sounds/little-elf.dat'),
});

frame.use(soundDescription({ lang: 'pt', keys: true }));

What it says

How long the sound is, where the sound and the pauses are, and where it is loudest: "33 seconds. Sound until 20 seconds, a pause of 4 seconds, then sound to the end. Loudest at 12 seconds." A sound with many pauses is summed up by its longest one, and the silence at the start and the end is told apart.

It says "sound", never "speech" or "music": peaks cannot tell them apart, and they are relative, so it describes the shape, never how loud the recording is.

Who narrates

The person's own screen reader, in their voice and at their speed: the text goes in the wave's aria-describedby, next to any ids your page put there. Nothing is ever spoken on its own.

For people without a screen reader, speech: true lets a click read it with the browser's own voice (speechSynthesis), with nothing to download. Only voices on the device are used, because some browsers' online voices send the text to a server; speech: { voices: 'any' } allows them.

Options

Option Default What it does
lang the frame's 'en', 'pt', 'pt-PT', 'es', 'fr' (see Languages), 'auto', or any tag with your own words. Without one, the frame's a11y.lang, English unless it has one.
words built in Your own phrases, templates with {duration}, {at}, {count}…
timeFormat 'words' Positions as words ("12 seconds"), or 'clock' says them as 0:12.
text from the peaks Your own description, written ahead.
minPause automatic The shortest pause, in seconds (3 % of the sound, from 0.75 to 5).
threshold 0.06 Below this share of the loudest peak counts as quiet, from 0.001 to 0.45; anything that is not a number is 0.06.
describedBy true Put the text in the wave's aria-describedby.
target none An element of yours that shows the text too.
announce false Say it once if it arrives while the wave has focus.
keys false D says it again, P / Shift + P step through the parts.
pauseInValue false Add "pause" or "silence" to the wave's value.
speech false Let speak() read it with the browser's voice; { voices, rate } (the rate from 0.5 to 2, 1 by default).
onChange none Called with the new text.

The plugin it gives can be read and driven:

Member What it does
text The description, or null while the peaks or the length are unknown.
outline The shape it was written from, or null (also while your own text is used).
locale The language tag the text is in.
set(text) Uses your own text; null goes back to the one from the peaks.
refresh() Writes it again, after the page's language changed, say.
say() Says it through the live region.
speak(), stopSpeaking(), speaking Reads it with the browser's voice (it needs speech; call it from a person's action), stops it (Escape on the wave does too), and tells whether it is reading.

Written ahead of time

outlineSound and describeSound are pure functions on sonicframe/analysis, so a server can write the description when a file is uploaded and pass it as text, ready at the first focus.

import { describeSound, outlineSound } from 'sonicframe/analysis';

const text = describeSound(outlineSound(peaks, duration), { lang: 'en' });

describeSound takes the same timeFormat: 'words' by default, or 'clock'.