---
title: Audio Playback & Streaming
description: Learn how to play sounds and stream microphone audio using Python and ALSA on WendyOS
language: python
---
## Audio Playback & Streaming
**Source Code**: The complete source code for this example is available at [github.com/wendylabsinc/samples/python/audio](https://github.com/wendylabsinc/samples/tree/main/python/audio)
In this guide, we'll build an audio application that demonstrates two key capabilities:
1. **Audio Playback**: Triggering sound effects on the device from a web interface, played through GStreamer to an ALSA speaker.
2. **Microphone Streaming**: Capturing live PCM audio from the device's microphone with GStreamer and streaming it to a web client over a WebSocket, where it's rendered as a waveform.
This demonstrates how to drive audio capture and playback with GStreamer and ALSA inside a Python container.
### Prerequisites
- Wendy CLI installed
- A WendyOS device with a speaker and microphone (or a USB audio interface)
**Recommended Hardware**: For the best experience, we recommend using a USB speakerphone like the Anker PowerConf plugged into your NVIDIA Jetson via USB. It provides high-quality audio capture and playback in a single device.
## Setting Up Your Project
### Initialize the Project
```bash
wendy init audio --target wendyos --language python --template audio --var APP_ID=audio --var PORT=3004 --assistant skip --git-init no
cd audio
```
The template creates the Wendy config, Dockerfile, frontend, and backend files with the `audio` entitlement already wired. The sections below explain the generated project.
### Run on WendyOS
```bash
wendy run
```
Wendy will build the app, ask you to select a device if one is not already configured, deploy the app, and print the app URL.
## Code Breakdown
## Project Structure
The template generates a single-file frontend (`index.html`) and a single-file backend (`app.py`), plus a few sample WAV files under `assets/`:
```
audio/
├── Dockerfile
├── wendy.json
├── requirements.txt
├── app.py # FastAPI + GStreamer backend
├── index.html # Waveform visualizer & controls (Tailwind via CDN)
└── assets/ # Bundled sound effects + logo
├── car-horn.wav
├── magic.wav
├── piano.wav
├── voice.wav
└── wendy-logo.svg
```
## Setting Up the Backend
The backend uses [FastAPI](https://fastapi.tiangolo.com/) for the HTTP API and WebSockets. Audio capture and playback are driven by GStreamer pipelines using the ALSA plugins (`alsasrc` / `alsasink`). ALSA devices are enumerated by parsing `arecord -l` / `aplay -l`.
### 1. Dependencies
The `requirements.txt` is just FastAPI and Uvicorn — GStreamer comes from the system packages installed in the Dockerfile:
```text
fastapi
uvicorn[standard]
```
### 2. Audio Playback
To play a sound, the backend builds a GStreamer pipeline that reads a WAV file from `assets/` and sends it to the selected ALSA speaker (falling back to the first detected speaker, then `autoaudiosink`):
```python
@app.post("/play/{filename}")
async def play_sound(filename: str):
"""Play a wav file from ./assets on the device speaker via GStreamer."""
filepath = _assets_dir / filename
if not filepath.exists() or not filename.endswith(".wav"):
return JSONResponse(content={"error": "not found"}, status_code=404)
if _current_speaker:
sink = f'alsasink device="{_current_speaker}"'
else:
speakers = _list_speakers()
sink = f'alsasink device="{speakers[0]["id"]}"' if speakers else "autoaudiosink"
desc = f'filesrc location="{filepath}" ! wavparse ! audioconvert ! audioresample ! {sink}'
pipeline = Gst.parse_launch(desc)
pipeline.set_state(Gst.State.PLAYING)
# ... a background thread watches the bus for EOS/ERROR, then tears the pipeline down ...
return JSONResponse(content={"status": "playing", "file": filename})
```
The UI discovers playable sounds via `GET /sounds`, and available output devices via `GET /speakers`. Selecting a speaker posts to `/speaker/{device_id}`.
### 3. Microphone Streaming
Microphone capture is an `alsasrc` pipeline resampled to S16LE mono 16 kHz, ending in an `appsink`. Each captured buffer is pushed to every connected client's queue:
```python
def _start_pipeline(self) -> Gst.Pipeline | None:
appsink = "appsink name=sink emit-signals=true max-buffers=4 drop=true sync=false"
pcm_caps = "audio/x-raw,format=S16LE,channels=1,rate=16000"
# tries the selected device, then each device from `arecord -l`, then a generic alsasrc
desc = f'alsasrc device="{dev}" ! audioconvert ! audioresample ! {pcm_caps} ! {appsink}'
# ... parse_launch, probe PAUSED state, return the first pipeline that prerolls ...
def _on_new_sample(self, sink):
sample = sink.emit("pull-sample")
buf = sample.get_buffer()
ok, mapinfo = buf.map(Gst.MapFlags.READ)
data = bytes(mapinfo.data)
buf.unmap(mapinfo)
with self._lock:
for q in self.queues.values():
try:
q.put_nowait(data)
except asyncio.QueueFull:
pass
return Gst.FlowReturn.OK
```
The WebSocket endpoint streams those raw PCM bytes to the browser and accepts a `switch_microphone` command to change the active input device:
```python
@app.websocket("/stream")
async def websocket_stream(websocket: WebSocket):
await websocket.accept()
q = await audio.add_client(websocket)
async def send_audio():
while True:
data = await q.get()
await websocket.send_bytes(data)
async def recv_commands():
try:
while True:
msg = json.loads(await websocket.receive_text())
if "switch_microphone" in msg:
await audio.switch_microphone(msg["switch_microphone"])
except WebSocketDisconnect:
pass
try:
done, pending = await asyncio.wait(
[asyncio.create_task(send_audio()), asyncio.create_task(recv_commands())],
return_when=asyncio.FIRST_COMPLETED,
)
for t in pending:
t.cancel()
finally:
audio.remove_client(websocket)
```
The backend also exposes `GET /microphones`, `GET /logs`, and `GET /debug`, and serves the UI at `GET /`.
## Frontend Implementation
The frontend is a single `index.html` (Tailwind via CDN). It draws a fullscreen waveform on a `