Peaks
Peaks are the loudest point of each slice of a sound, as numbers from 0 to 1. They are all the frame needs to draw: SonicFrame never decodes audio just to draw it.
The idea
Compute the peaks once, when a sound is uploaded, and serve them as a small file. A few hundred to a few thousand values are plenty: the frame folds or stretches them to the number of bars that fit.
A peaks file is either a list or an object with a list:
[0.02, 0.31, 0.77, 0.54, 0.12]
{ "peaks": [0.02, 0.31, 0.77, 0.54, 0.12] }
Giving peaks to a frame
peaks takes any of these:
| Value | What happens |
|---|---|
| a list of numbers | drawn at once. |
{ peaks, duration } |
drawn at once; the duration shows until the media knows its own. |
| a function returning a promise of either | called the first time the frame comes near the screen. |
null |
a thin line until peaks arrive
(setPeaks).
|
import { SonicFrame, loadPeaks } from 'sonicframe';
new SonicFrame({
container,
media,
peaks: () => loadPeaks('/voice.peaks.json'),
});
loadPeaks(url) fetches a file once per
page, however many frames use it, and refuses a file
with more than 20,000 values. For larger and binary
files, use
ready-made waveform files.
loadPeaks(url, options)
|
Default | What it does |
|---|---|---|
shared |
true |
Fetch the same address once per page;
false fetches every time.
|
fetcher |
fetch |
Your own Fetcher: it takes
the address and gives a response with
ok, status and
arrayBuffer(), as for
loadWaveform and
loadChapters.
|
maxPeaks |
20000 |
The most peaks read, up to 1,000,000 (10 hours at about 27 a second). |
maxBytes |
2000000 |
The largest file read, in bytes, up to
64,000,000. Past either end of
maxPeaks or
maxBytes, a
RangeError names the
option.
|
const peaks = await loadPeaks('/voice.peaks.json', {
shared: false,
fetcher: (url) => fetch(url, { credentials: 'include' }),
});
Making peaks
On your server
Write an audiowaveform file with
writeWaveform from
sonicframe/analysis (it runs in Node), or
use the audiowaveform tool itself. Both are covered in
Ready-made peaks.
In the browser
For a recording just made, compute them from the decoded samples:
import { peaksFromChannel } from 'sonicframe';
const peaks = peaksFromChannel(audioBuffer.getChannelData(0), 400);
Let the player do it
The ready-made player, given only src,
decodes the sound in the browser the first time it comes
near the screen. It is the easiest start, and the most
expensive for long sounds (see
Long sounds);
decode on the
player sets its
limits.
Long sounds
A browser decodes a whole file before it gives a single sample: an hour takes about 635 MB, and ten hours more than a tab can hold. Pick the way by the sound's length:
| Sound | Way | What it costs |
|---|---|---|
| any length | peaks made on your server | A few kilobytes per sound, and the wave shows at once. The best way. |
| up to hours, no server work |
decode: { stream: true } on
the player
|
The file is read in pieces as it downloads, keeping only the peaks: the memory stays small, and the wave grows while it comes. |
| up to about 30 minutes | the whole decode (the default) | The whole sound in memory for a moment. |
Reading in pieces
import { createPlayer } from 'sonicframe/player';
createPlayer({
container: document.querySelector('#peaks-stream'),
src: './sounds/music/november-snow.mp3',
decode: { stream: true },
});
It reads MP3, AAC (.aac), WAV, MP4 and M4A,
Ogg (Opus and Vorbis), FLAC and WebM, known by their
first bytes, never by the address. The frames go to the
browser's own decoder (WebCodecs) where there is one;
without it, MP3, AAC, Ogg and FLAC are decoded about 30
seconds at a time. Any other sound, and MP4 or WebM
without WebCodecs, is decoded whole as before, within
decode.maxBytes and
decode.maxSeconds.
The sound is downloaded twice: once by the
<audio> that plays it, and once for
the peaks. An hour of MP3 at 128 kbit/s is about 58 MB
more for each listener. Peaks from your server save that
too. See the
player's options
for the stream's ceilings.
Peaks of hours on your server
createPeakStream from
sonicframe/analysis takes samples in pieces
of any size and keeps only a few values per peak, so a
ten-hour sound needs no more memory than a short one.
Here it reads what ffmpeg decodes, in Node:
import { spawn } from 'node:child_process';
import { writeFile } from 'node:fs/promises';
import { createPeakStream } from 'sonicframe/analysis';
const peaks = createPeakStream(20000);
// One channel of 32-bit floats; 8,000 a second is plenty for a shape.
const ffmpeg = spawn('ffmpeg', ['-i', 'episode.mp3', '-ac', '1', '-ar', '8000', '-f', 'f32le', '-']);
let rest = Buffer.alloc(0);
for await (const piece of ffmpeg.stdout) {
const bytes = Buffer.concat([rest, piece]);
const whole = bytes.length - (bytes.length % 4);
peaks.push(new Float32Array(bytes.buffer.slice(bytes.byteOffset, bytes.byteOffset + whole)));
rest = bytes.subarray(whole);
}
await writeFile('episode.peaks.json', JSON.stringify({ peaks: peaks.peaks() }));
Then give the file to the player as
peaks, or to the tag as
<sonic-player src="episode.mp3"
peaks="episode.peaks.json">. While a sound is still coming,
peaks.peaks(totalSamples)
draws the part not read yet as silence.
Shape, not loudness
Peaks are relative: a quiet recording and a loud one can
draw the same wave. The bar style's curve
(0.7 by default) lifts quiet parts so speech reads well;
1 draws the peaks as they are. See
Drawings.