WebCodecs: low-level, hardware-accelerated audio and video encoding

In one line: “The WebCodecs API enables web developers to encode and decode video and audio in the browser efficiently (using hardware acceleration) and with very low-level control (processing on a per-frame basis),” per MDN.

What it’s for

MDN says it “is useful for web applications that do heavy media processing, or which require low-level control over the way media is encoded. This includes browser-based video and audio editing, as well as live-streaming and video conferencing.” A codec, per MDN, “is a specific algorithm for encoding (compressing) and decoding (decompressing) video and audio.”

The core interfaces

MDN’s WebCodecs overview lists the interfaces this way: VideoEncoder “encodes VideoFrame objects” and VideoDecoder “decodes EncodedVideoChunk objects”; AudioEncoder “encodes AudioData objects” and AudioDecoder “decodes EncodedAudioChunk objects.” A VideoFrame “represents a video frame, and is tied to actual pixel data on the device’s graphics memory,” while an EncodedVideoChunk “represents the encoded (compressed) version of the same frame,” typically storing “10 to 100 times less data than its corresponding raw VideoFrame.” The parallel audio pair is AudioData (“a number of individual audio samples”) and EncodedAudioChunk (“the encoded (compressed) version of an AudioData object”).

Encoding a frame

const encoder = new VideoEncoder({
  output(chunk, meta) {
    // Do something with chunk, typically send to muxing library
  },
  error(e) {
    console.warn(e);
  },
});

encoder.configure({
  codec: "vp09.00.40.08.00", // fully specified codec string, not just "vp9"
  width: 1280,
  height: 720,
  bitrate: 1_000_000,
  framerate: 30,
});

const frame = new VideoFrame(canvas, { timestamp: 0 });
encoder.encode(frame, { keyFrame: true });

MDN’s processing-model description explains why output/error are callbacks rather than a return value: “The WebCodecs API uses an asynchronous processing model. Each instance of an encoder or decoder maintains an internal, independent processing queue.” configure(), encode(), decode(), and flush() “operate asynchronously by appending control messages to the end of the queue,” while reset() and close() “synchronously abort all pending work and purge the processing queue” — and close() “is a permanent operation.”

Codec strings must be fully specified

MDN is explicit: “Encoders and decoders must be configured with fully specified codec strings (such as 'vp09.00.40.08.00' for VP9 or 'avc1.4d0034' for H.264) instead of ambiguous codec names like 'vp9' or 'h264'.”

No built-in muxing or demuxing

MDN notes a real gap developers hit: “The WebCodecs API only deals with encoding and decoding, with encoded chunks just representing binary data. It does not provide a built-in way to read EncodedVideoChunk objects from a video file, or write them to a playable video file.” Pair it with a separate muxing/demuxing library to produce or parse an actual container file (e.g. MP4 or WebM).

Available in a Dedicated Worker

MDN flags this directly on the API overview: “This feature is available in Dedicated Web Workers.”

Feature detection and fallback

function canUseWebCodecs() {
  return typeof VideoEncoder !== "undefined" && typeof VideoDecoder !== "undefined";
}

function createCanvasRecorder(canvas, canvasStream) {
  if (!canUseWebCodecs()) {
    // WebCodecs unsupported here — fall back to MediaRecorder, which records
    // to a container file format instead of exposing individual frames.
    return new MediaRecorder(canvasStream);
  }
  const encoder = new VideoEncoder({
    output: (chunk) => {
      // hand chunk to a muxing library
    },
    error: (e) => console.warn(e),
  });
  encoder.configure({ codec: "vp09.00.40.08.00", width: canvas.width, height: canvas.height });
  return encoder;
}

Where it is supported

MDN documents VideoEncoder’s browser-compatibility table on its own reference page; check that table (and the equivalent pages for VideoDecoder, AudioEncoder, and AudioDecoder) for the current per-browser support position before shipping.

Practical checklist

• Use fully specified codec strings (e.g. "vp09.00.40.08.00", "avc1.4d0034") in configure() — MDN states ambiguous names like "vp9" or "h264" are not accepted.
• Remember WebCodecs is documented as available in a Dedicated Web Worker.
• Pair WebCodecs with a separate muxing/demuxing library: per MDN it has no built-in way to read encoded chunks from, or write them to, a playable container file.
• Call close() only when truly done — MDN describes it as a permanent operation, unlike reset(), after which more work can still be queued via configure().
• Feature-detect VideoEncoder/VideoDecoder before use and fall back to MediaRecorder, which MDN documents as recording to a configurable container file type, when they are unavailable.

Cross-references

• WebGPU
• Web Share API