Frontend Masters Web Audio Synthesis & Visualization TUTORIAL

By | January 16, 2022

Frontend Masters Web Audio Synthesis & Visualization TUTORIAL

Frontend Masters Web Audio Synthesis & Visualization TUTORIAL

Learn to create sounds using nothing but code! Synthesize and visualize audio, and add fun effects with JavaScript. Use these skills to build custom audio into games, web applications, or art projects in the browser. You’ll appreciate the richness of sound that is only possible with the web audio API!

This course and others like it are available as part of our Frontend Masters video subscription.

Published: December 7, 2021

Table of Contents

Introduction

  • Introduction
    Matt DesLauriers introduces the course by providing an overview of the course material, a walkthrough of the course repo, and some prerequisites. The demos provided in this segment involve mp3, buffered mp3, gain, waveform, meter, and frequency.
  • How Audio Works
    Matt discusses the basics of how audio and digital audio works including a description of sound, waveforms, and frequency. A closer look into what a waveform represents is also covered in this segment.
  • Web Audio in Art
    Matt walks through some artistic examples that involve the use of web audio. Personal projects with web audio including art, audio visualization, a game, and work from other developers are provided to showcase web audio’s many applications.

Web Audio API

  • Web Audio API Overview
    Matt provides a brief overview of what the Web Audio API is, a graph of how audio data flows from input to output, and how the API works.
  • Playing an Audio File
    Matt walks through the JavaScript being used to stream an mp3 file through the browser overtime using the Web Audio API. Streaming audio is a better option for audio files with a longer track length to allow the file to be played as it downloads instead of waiting for the file to be completely loaded in. A brief walkthrough of a typical course sketch is also provided in this segment.
  • Buffering Audio
    Matt discusses buffering an audio file to fully download and decode the file to allow the audio to immediately be played. To help avoid errors from edge cases a best practice is to make sure the audioContext is resumed before playing audio.
  • GainNode
    Matt demonstrates passing an AudioNode through a second GainNode before the end AudioDestinationNode using the createGain function to allow for simple adjustments to the audio’s volume. Gain values should be adjusted gradually to avoid pops and clicks in the audio.

Visualization

  • Visualizing Audio Overview
    Matt discusses mapping a waveform into a digital space with arrays and JavaScript using an analyserNode and the Web Audio API function getFloatTimeDomainData. The getFloatTimeDomainData takes in a dataArray which designates the window of time ahead of the playhead to be analyzed.
  • Visualizing a Waveform
    Matt walks through the code being used in the waveform demo to visualize a waveform including creating a Float32Array to store the waveform data. A brief discussion on the advanced waveform demo, which builds on the first example with extra features and browser support, is also provided in this segment.
  • Volume Meter
    Matt demonstrates using the waveform data to create a visualization based on the volume of the audio track. The advanced meter demo uses the Web Audio API to help split the audio track into different colored frequency bands to help create a more interesting visualization. A brief discussion regarding the BiquadFilterNode is also covered in this segment.

Frequency

  • Frequency Analysis Overview
    Matt discusses what it means to analyze the frequency, provides multiple visualizations of the frequency spectrum, and discusses how frequencies can be used. A demonstration of using the tool spectrum to visualize the frequency graph and display a chosen decibel and hertz value of any audio file passed to it is also provided in this segment.
  • Frequency Analysis Demo
    Matt walks through the code provided in the frequency demos and discusses how to use frequencyToIndex, indexToFrequency, and audioSignal to help visualize parts of an audio track without using a the biquad filter. A student’s question regarding if fft is the only frequency analysis supported by the Web Audio API is also covered in this segment..
  • Advanced Frequency and Frequency Q&A
    Matt discusses he advanced filter demo which builds on the previous frequency demo by using the frequency to map to a generative artwork. Student questions regarding how to adjust the fixed parameters for a live audio source and how to treat files at different bit depths for dB FS are also covered in this section.

Tone

  • Synth & Tone Generation
    Matt discusses synthesizing audio from code without using any source files and demonstrates some examples of synthesizing audio with code. Examples of using Tone.js for interactive and procedural sound is also covered in this segment.
  • Tone.js Demos
    Matt discusses the open source JavaScript library Tone.js, how the base code is laid out for creating a Tone.synth, and provides the API documentation. A walkthrough of each part of code involved in synthesis in the tone demo and a demonstration of a game that uses a procedurally generated soundtrack is also provided in this segment.
  • Tone.js Effects
    Matt discusses the various effects included in Tone.js such as FeedbackDelay and walks through how to apply those effects to an audio track. A walkthrough of a demonstration using FeedbackDelay, Reverb, Filter, and editing the envelope is also covered in this segment.
  • Sequencer Demo
    Matt discusses creating a sequencer using PolySynth to create a richer sound with multiple oscillators, Transport for timing events, and a randomizeSequencer function to randomize the played notes. A walkthrough of the code used to create the sequencer is also provided in this segment.

Resources

  • Learning Synths with Ableton
    Matt walks through the various resources provided in the learning synths playground by Ableton including some material covered in this course such as amplitude, pitch, and effects. A demonstration of the synth recipes section, which provides settings for some common synth sounds, is also provided in this segment.
  • Inspiration
    Matt provides some inspiration for future web audio projects by providing various artists works that involve or could possibly be made with web audio.
  • Further Reading
    Matt discusses some links and provides a section on the course repository for further reading about web audio including GitHub pages that collect web audio resources, math, learning audio, fun web audio sites, and tools.

Wrapping Up

  • Wrapping Up
    Matt wraps up the course by providing well wishes on continued learning of web audio and answering student questions regarding web audio limitations, kick or beat detection, and tools to combine audio and a mobile device’s gyroscope. Student questions regarding using web sockets along with audio to sync users in real time, routing signals to and from outside of the browser, and how to find inspiration for future projects.

Links

Leave a Reply