SonicFrame v1.3.2

Peaks

Peaks are the loudest point of each slice of a sound, as numbers from 0 to 1. They are all the frame needs to draw: SonicFrame never decodes audio just to draw it.

The idea

Compute the peaks once, when a sound is uploaded, and serve them as a small file. A few hundred to a few thousand values are plenty: the frame folds or stretches them to the number of bars that fit.

A peaks file is either a list or an object with a list:

[0.02, 0.31, 0.77, 0.54, 0.12]
{ "peaks": [0.02, 0.31, 0.77, 0.54, 0.12] }

Giving peaks to a frame

peaks takes any of these:

Value What happens
a list of numbers drawn at once.
{ peaks, duration } drawn at once; the duration shows until the media knows its own.
a function returning a promise of either called the first time the frame comes near the screen.
null a thin line until peaks arrive (setPeaks).
import { SonicFrame, loadPeaks } from 'sonicframe';

new SonicFrame({
    container,
    media,
    peaks: () => loadPeaks('/voice.peaks.json'),
});

loadPeaks(url) fetches a file once per page, however many frames use it, and refuses a file with more than 20,000 values. For larger and binary files, use ready-made waveform files.

loadPeaks(url, options) Default What it does
shared true Fetch the same address once per page; false fetches every time.
fetcher fetch Your own Fetcher: it takes the address and gives a response with ok, status and arrayBuffer(), as for loadWaveform and loadChapters.
maxPeaks 20000 The most peaks read, up to 1,000,000 (10 hours at about 27 a second).
maxBytes 2000000 The largest file read, in bytes, up to 64,000,000. Past either end of maxPeaks or maxBytes, a RangeError names the option.
const peaks = await loadPeaks('/voice.peaks.json', {
    shared: false,
    fetcher: (url) => fetch(url, { credentials: 'include' }),
});

Making peaks

On your server

Write an audiowaveform file with writeWaveform from sonicframe/analysis (it runs in Node), or use the audiowaveform tool itself. Both are covered in Ready-made peaks.

In the browser

For a recording just made, compute them from the decoded samples:

import { peaksFromChannel } from 'sonicframe';

const peaks = peaksFromChannel(audioBuffer.getChannelData(0), 400);

Let the player do it

The ready-made player, given only src, decodes the sound in the browser the first time it comes near the screen. It is the easiest start, and the most expensive for long sounds (see Long sounds); decode on the player sets its limits.

Long sounds

A browser decodes a whole file before it gives a single sample: an hour takes about 635 MB, and ten hours more than a tab can hold. Pick the way by the sound's length:

Sound Way What it costs
any length peaks made on your server A few kilobytes per sound, and the wave shows at once. The best way.
up to hours, no server work decode: { stream: true } on the player The file is read in pieces as it downloads, keeping only the peaks: the memory stays small, and the wave grows while it comes.
up to about 30 minutes the whole decode (the default) The whole sound in memory for a moment.

Reading in pieces

import { createPlayer } from 'sonicframe/player';

createPlayer({
    container: document.querySelector('#peaks-stream'),
    src: './sounds/music/november-snow.mp3',
    decode: { stream: true },
});

It reads MP3, AAC (.aac), WAV, MP4 and M4A, Ogg (Opus and Vorbis), FLAC and WebM, known by their first bytes, never by the address. The frames go to the browser's own decoder (WebCodecs) where there is one; without it, MP3, AAC, Ogg and FLAC are decoded about 30 seconds at a time. Any other sound, and MP4 or WebM without WebCodecs, is decoded whole as before, within decode.maxBytes and decode.maxSeconds.

The sound is downloaded twice: once by the <audio> that plays it, and once for the peaks. An hour of MP3 at 128 kbit/s is about 58 MB more for each listener. Peaks from your server save that too. See the player's options for the stream's ceilings.

Peaks of hours on your server

createPeakStream from sonicframe/analysis takes samples in pieces of any size and keeps only a few values per peak, so a ten-hour sound needs no more memory than a short one. Here it reads what ffmpeg decodes, in Node:

import { spawn } from 'node:child_process';
import { writeFile } from 'node:fs/promises';
import { createPeakStream } from 'sonicframe/analysis';

const peaks = createPeakStream(20000);
// One channel of 32-bit floats; 8,000 a second is plenty for a shape.
const ffmpeg = spawn('ffmpeg', ['-i', 'episode.mp3', '-ac', '1', '-ar', '8000', '-f', 'f32le', '-']);
let rest = Buffer.alloc(0);

for await (const piece of ffmpeg.stdout) {
    const bytes = Buffer.concat([rest, piece]);
    const whole = bytes.length - (bytes.length % 4);
    peaks.push(new Float32Array(bytes.buffer.slice(bytes.byteOffset, bytes.byteOffset + whole)));
    rest = bytes.subarray(whole);
}

await writeFile('episode.peaks.json', JSON.stringify({ peaks: peaks.peaks() }));

Then give the file to the player as peaks, or to the tag as <sonic-player src="episode.mp3" peaks="episode.peaks.json">. While a sound is still coming, peaks.peaks(totalSamples) draws the part not read yet as silence.

Shape, not loudness

Peaks are relative: a quiet recording and a loud one can draw the same wave. The bar style's curve (0.7 by default) lifts quiet parts so speech reads well; 1 draws the peaks as they are. See Drawings.