Audio Package

Audio processing and encoding utilities for AIRI

Overview

The @proj-airi/audio package provides audio processing utilities including AudioContext management, audio encoding/decoding, and worklet-based audio processing.

Installation

npm install @proj-airi/audio

Package Structure

The package provides three main export paths:

import * as audio from '@proj-airi/audio'
import * as audioContext from '@proj-airi/audio/audio-context'
import * as encoding from '@proj-airi/audio/encoding'

Audio Context Management

Manage Web Audio API context with automatic initialization and cleanup.

initializeAudioContext()

Initialize or get the shared AudioContext.

import { initializeAudioContext } from '@proj-airi/audio/audio-context'

const context = await initializeAudioContext(48000)
console.log('Sample rate:', context.sampleRate)

requestedSampleRate · number · default: "48000"

Desired sample rate in Hz. The actual sample rate will be at least 48000Hz for high quality.

context · AudioContext

The initialized AudioContext instance

Audio Context State

Get current audio context state:

import {
  getAudioContextState,
  isAudioContextReady,
  getAudioContext,
  getCurrentTime
} from '@proj-airi/audio/audio-context'

const state = getAudioContextState()
// {
//   isReady: boolean
//   sampleRate: number
//   error: string
//   isInitializing: boolean
//   workletLoaded: boolean
//   currentTime: number
//   state: AudioContextState
// }

if (isAudioContextReady()) {
  const context = getAudioContext()
  const time = getCurrentTime()
}

State Interface

interface State {
  isReady: boolean                    // Context is initialized and ready
  sampleRate: number                  // Actual sample rate
  error: string                       // Error message if initialization failed
  isInitializing: boolean             // Currently initializing
  workletLoaded: boolean              // Audio worklets are loaded
  currentTime: number                 // Current audio time
  state: AudioContextState            // Native context state
}

Subscribe to State Changes

import { subscribeToAudioContext } from '@proj-airi/audio/audio-context'

const unsubscribe = subscribeToAudioContext((state) => {
  console.log('Audio context state:', state)
})

// Later: clean up subscription
unsubscribe()

Audio Node Creation

createAudioSource()

Create an audio source from a MediaStream.

import { createAudioSource } from '@proj-airi/audio/audio-context'

const stream = await navigator.mediaDevices.getUserMedia({ audio: true })
const source = createAudioSource(stream)

mediaStream · MediaStream · required

MediaStream to create source from

source · MediaStreamAudioSourceNode

Created audio source node

createAudioAnalyser()

Create an analyser node for audio visualization.

import { createAudioAnalyser } from '@proj-airi/audio/audio-context'

const analyser = createAudioAnalyser({
  fftSize: 2048,
  smoothingTimeConstant: 0.8,
  minDecibels: -90,
  maxDecibels: -10
})

// Connect to source
source.connect(analyser)

// Get frequency data
const dataArray = new Uint8Array(analyser.frequencyBinCount)
analyser.getByteFrequencyData(dataArray)

options.fftSize · number

FFT size for frequency analysis (power of 2)

options.smoothingTimeConstant · number

Smoothing between frames (0-1)

options.minDecibels · number

Minimum decibel value for FFT scaling

options.maxDecibels · number

Maximum decibel value for FFT scaling

createAudioGainNode()

Create a gain node for volume control.

import { createAudioGainNode } from '@proj-airi/audio/audio-context'

const gainNode = createAudioGainNode(0.5)  // 50% volume

// Connect source -> gain -> destination
source.connect(gainNode)
gainNode.connect(context.destination)

// Change volume
gainNode.gain.value = 0.8

initialGain · number · default: "1"

Initial gain value (0-1)

Audio Resampling

createResamplingWorkletNode()

Create a worklet node for real-time audio resampling.

import { createResamplingWorkletNode } from '@proj-airi/audio/audio-context'

const workletNode = createResamplingWorkletNode(source, {
  inputSampleRate: 48000,
  outputSampleRate: 16000,
  channels: 1,
  converterType: 2,  // SRC_SINC_MEDIUM_QUALITY
  bufferSize: 4096
})

workletNode.port.onmessage = (event) => {
  const { audio } = event.data
  // Process resampled audio
}

inputNode · AudioNode · required

Source audio node to resample

options.inputSampleRate · number · default: "48000"

Input sample rate in Hz

options.outputSampleRate · number · default: "16000"

Output sample rate in Hz

options.channels · number · default: "1"

Number of audio channels

options.converterType · number · default: "2"

Resampling quality (0-4, higher is better)

options.bufferSize · number · default: "4096"

Processing buffer size

Audio Encoding

WAV Encoding

Encode audio data to WAV format.

import { toWav, toWAVBase64 } from '@proj-airi/audio/encoding'

// Encode to WAV ArrayBuffer
const wavBuffer = toWav(audioData, 16000, 1)

// Encode to base64 WAV
const base64Wav = toWAVBase64(audioData, 16000)

// Use as data URL
const audio = new Audio(`data:audio/wav;base64,${base64Wav}`)
audio.play()

buffer · ArrayBufferLike · required

Float32 audio samples

sampleRate · number · required

Sample rate in Hz

channels · number · default: "1"

Number of audio channels

wavBuffer · ArrayBuffer

Complete WAV file as ArrayBuffer

base64Wav · string

Base64-encoded WAV file

Resource Management

Node Cleanup

Remove nodes when done:

import {
  removeAudioSource,
  removeAudioGainNode,
  removeAudioAnalyser,
  removeWorkletNode
} from '@proj-airi/audio/audio-context'

// Clean up nodes
removeAudioSource(source)
removeAudioGainNode(gainNode)
removeAudioAnalyser(analyser)
removeWorkletNode(workletNode)

Context Suspension

Suspend and resume audio context:

import {
  suspendAudioContext,
  resumeAudioContext
} from '@proj-airi/audio/audio-context'

// Suspend to save CPU
await suspendAudioContext()

// Resume when needed
await resumeAudioContext()

Complete Cleanup

Clean up all resources:

import { cleanupAudioContext } from '@proj-airi/audio/audio-context'

// Clean up all nodes and close context
await cleanupAudioContext()

Complete Example

Microphone recording with resampling:

import {
  initializeAudioContext,
  createAudioSource,
  createResamplingWorkletNode,
  removeAudioSource,
  removeWorkletNode,
  cleanupAudioContext
} from '@proj-airi/audio/audio-context'
import { toWAVBase64 } from '@proj-airi/audio/encoding'

// Initialize
const context = await initializeAudioContext(48000)

// Get microphone access
const stream = await navigator.mediaDevices.getUserMedia({
  audio: {
    sampleRate: 48000,
    channelCount: 1,
    echoCancellation: true,
    noiseSuppression: true
  }
})

// Create audio pipeline
const source = createAudioSource(stream)
const worklet = createResamplingWorkletNode(source, {
  inputSampleRate: 48000,
  outputSampleRate: 16000,
  channels: 1
})

// Collect audio chunks
const chunks: Float32Array[] = []

worklet.port.onmessage = (event) => {
  if (event.data.audio) {
    chunks.push(new Float32Array(event.data.audio))
  }
}

// Start recording
console.log('Recording...')

// Stop after 5 seconds
setTimeout(async () => {
  // Stop recording
  stream.getTracks().forEach(track => track.stop())
  
  // Combine chunks
  const totalLength = chunks.reduce((sum, chunk) => sum + chunk.length, 0)
  const combined = new Float32Array(totalLength)
  let offset = 0
  for (const chunk of chunks) {
    combined.set(chunk, offset)
    offset += chunk.length
  }
  
  // Encode to WAV
  const wav = toWAVBase64(combined.buffer, 16000)
  console.log('Recording complete:', wav.length, 'bytes')
  
  // Clean up
  removeAudioSource(source)
  removeWorkletNode(worklet)
  await cleanupAudioContext()
}, 5000)

Best Practices

Initialize early

Initialize AudioContext early in your app, preferably on user interaction due to browser autoplay policies.

Use high sample rates

Use 48kHz or higher for input to ensure quality. Resample down as needed for processing.

Clean up resources

Always remove nodes and clean up the context when done to prevent memory leaks.

Handle errors

Wrap audio operations in try-catch blocks and check the error state.

Suspend when inactive

Suspend the audio context when not actively processing to save CPU.

Browser Compatibility

The package uses Web Audio API features:

• AudioContext
• AudioWorklet
• MediaStream

Supported in all modern browsers:

• Chrome/Edge 66+
• Firefox 76+
• Safari 14.1+

Next Steps

Stage UI Package

Integrate audio with UI components

Server SDK

Stream audio to server runtime