---
title: Webcam Streaming
description: Stream a live webcam feed from your WendyOS device to the browser using GStreamer, WebSockets, and hardware-accelerated JPEG encoding
language: python
---

import { Callout } from 'fumadocs-ui/components/callout';
import { Steps, Step } from 'fumadocs-ui/components/steps';
import { Accordions, Accordion } from 'fumadocs-ui/components/accordion';

![Webcam Setup](../../../images/webcam-setup.png)

## Low-Latency Webcam Streaming with GStreamer

This guide walks you through building a real-time webcam streaming application that runs on a WendyOS device. The app uses GStreamer for hardware-accelerated video capture and JPEG encoding, WebSockets for low-latency frame delivery, and a simple HTML5 frontend that renders frames on a canvas.

The complete sample is available in the [samples repository](https://github.com/wendylabsinc/samples/tree/main/python/webcam).

### What You'll Build

- A FastAPI backend using GStreamer to capture MJPEG frames from a USB webcam
- WebSocket-based binary JPEG streaming with automatic client management and lazy start/stop
- An HTML5 frontend with an `<img>` viewer, camera picker, FPS counter, and connection status
- A Docker container built on a slim Debian base with GStreamer installed

### Prerequisites

- Wendy CLI installed on your development machine
- A WendyOS device (NVIDIA Jetson Orin Nano, Jetson AGX, etc.)
- A USB webcam connected to your WendyOS device

<Callout type="info">
**Recommended Webcams**: We recommend a standard USB webcam such as the Logitech C920 or C270.
</Callout>

<Steps>
  <Step>
    ### Initialize the Project

    Start from the Wendy camera-feed template:

    ```bash
    wendy init webcam --target wendyos --language python --template camera-feed --var APP_ID=webcam --var PORT=3003 --assistant skip --git-init no
    cd webcam
    ```

    <CliShot flow="init" step="wizard" alt="wendy init scaffolding the webcam streaming project from the camera-feed template" />

    The template creates the Wendy config, Dockerfile, backend, frontend, and camera entitlements. The sections below explain the generated project.
  </Step>

  <Step>
    ### Run on WendyOS

    ```bash
    wendy run
    ```

    <CliShot flow="run" step="deploy" alt="wendy run building and deploying the webcam streaming app to a WendyOS device" />

    Wendy will build the app, ask you to select a device if one is not already configured, deploy the app, and print the app URL.
  </Step>
</Steps>

## Code Breakdown

### Understanding the Architecture

This sample uses a **slim Debian base image** (`debian:bookworm-slim`) with GStreamer and its Python bindings installed. Most USB webcams output **MJPEG natively**, so the app streams those JPEG frames straight through to the browser without re-encoding.

The streaming pipeline works as follows:

1. **GStreamer** captures frames from the webcam via V4L2 (`v4l2src`)
2. The app tries several pipelines in order, preferring native MJPEG passthrough and falling back to `jpegenc` software encoding when needed
3. **FastAPI** serves a WebSocket endpoint that broadcasts JPEG frames to all connected browsers
4. The **HTML5 frontend** renders each frame by setting the `src` of an `<img>` element to a base64 data URL

The camera pipeline starts lazily when the first client connects and stops when the last client disconnects, saving resources when nobody is watching. The app also enumerates available cameras (`GET /cameras`) so the frontend can offer a camera picker, and supports switching cameras over the WebSocket.


<Steps>
  <Step>
    ### Understanding the Backend

    The FastAPI backend (`app.py`) initializes GStreamer, runs a GLib main loop in a background thread, and serves both the UI and the WebSocket stream:

    ```python
    import asyncio
    import json
    import logging
    import threading
    from pathlib import Path

    import gi

    gi.require_version("Gst", "1.0")
    gi.require_version("GstApp", "1.0")

    from gi.repository import Gst, GLib
    from fastapi import FastAPI, WebSocket, WebSocketDisconnect
    from fastapi.responses import FileResponse, JSONResponse
    from fastapi.staticfiles import StaticFiles

    Gst.init(None)

    _glib_loop = GLib.MainLoop()
    threading.Thread(target=_glib_loop.run, daemon=True).start()

    app = FastAPI()
    ```

    **GStreamer Pipeline**

    The `MJPEGCamera` class builds a capture pipeline. It prefers passing the camera's native MJPEG straight through and falls back to software JPEG encoding, trying each candidate until one reaches the `PAUSED` state:

    ```python
    class MJPEGCamera:
        """Captures MJPEG frames from a camera using GStreamer appsink."""

        def _start_pipeline(self, device_id: str | None = None) -> Gst.Pipeline | None:
            src = build_source(device_id)  # v4l2src on Linux, avfvideosrc on macOS
            appsink = "appsink name=sink emit-signals=true max-buffers=2 drop=true sync=false"
            pipelines = [
                f"{src} ! image/jpeg ! jpegdec ! jpegenc quality=85 ! {appsink}",
                f"{src} ! image/jpeg,width=640,height=480 ! jpegdec ! jpegenc quality=85 ! {appsink}",
                f"{src} ! videoconvert ! jpegenc quality=70 ! {appsink}",
                f"{src} ! image/jpeg ! {appsink}",
                f"{src} ! image/jpeg,width=640,height=480 ! {appsink}",
            ]

            for p_str in pipelines:
                try:
                    pipeline = Gst.parse_launch(p_str)
                    ret = pipeline.set_state(Gst.State.PAUSED)
                    if ret == Gst.StateChangeReturn.FAILURE:
                        pipeline.set_state(Gst.State.NULL)
                        continue
                    # ... wait for ASYNC preroll, check for errors ...
                    logger.info("Pipeline ready: %s", p_str)
                    return pipeline
                except Exception as e:
                    logger.info("Pipeline exception: %s — %s", p_str, e)
            return None
    ```

    **How it works:**
    - `v4l2src` captures frames from the USB webcam (the device defaults to `/dev/video0`)
    - The last two pipelines pass the camera's native `image/jpeg` straight through with no re-encoding
    - The earlier pipelines decode and re-encode (`jpegdec ! jpegenc`) or convert raw frames (`videoconvert ! jpegenc`) as fallbacks
    - `appsink` with `max-buffers=2 drop=true` prevents frame buildup if clients are slow

    **WebSocket Frame Broadcasting**

    Each connected client gets a small queue. When GStreamer produces a new JPEG frame, it is pushed to every client's queue:

    ```python
    def _on_new_sample(self, sink):
        sample = sink.emit("pull-sample")
        if not sample:
            return Gst.FlowReturn.OK
        buf = sample.get_buffer()
        ok, mapinfo = buf.map(Gst.MapFlags.READ)
        if not ok:
            return Gst.FlowReturn.OK
        data = bytes(mapinfo.data)
        buf.unmap(mapinfo)

        with self._lock:
            for q in self.queues.values():
                try:
                    q.put_nowait(data)
                except asyncio.QueueFull:
                    pass

        return Gst.FlowReturn.OK
    ```

    **Lazy Start/Stop**

    The camera starts when the first client connects (via a restart task) and stops when the last disconnects:

    ```python
    async def add_client(self, ws: WebSocket) -> asyncio.Queue:
        self._loop = asyncio.get_running_loop()
        q = asyncio.Queue(maxsize=2)
        with self._lock:
            self.queues[ws] = q
            should_restart = self.pipeline is None
        if should_restart:
            self._ensure_restart_task("client connected")
        return q

    def remove_client(self, ws: WebSocket):
        with self._lock:
            self.queues.pop(ws, None)
            if not self.queues:
                self._clear_pipeline_locked()
                logger.info("Camera stopped (no clients)")
    ```

    **WebSocket Endpoint**

    The endpoint runs a sender task (frames out) and a receiver task (commands in, such as switching cameras) concurrently:

    ```python
    @app.websocket("/stream")
    async def websocket_stream(websocket: WebSocket):
        await websocket.accept()
        try:
            q = await camera.add_client(websocket)
        except Exception as e:
            logger.error(f"Failed to start camera: {e}")
            await websocket.close(code=1011)
            return

        async def send_frames():
            while True:
                data = await q.get()
                await websocket.send_bytes(data)

        async def recv_commands():
            try:
                while True:
                    msg = json.loads(await websocket.receive_text())
                    if "switch_camera" in msg:
                        await camera.switch_camera(msg["switch_camera"])
            except WebSocketDisconnect:
                pass

        try:
            done, pending = await asyncio.wait(
                [asyncio.create_task(send_frames()), asyncio.create_task(recv_commands())],
                return_when=asyncio.FIRST_COMPLETED,
            )
            for t in pending:
                t.cancel()
        finally:
            camera.remove_client(websocket)
    ```

    **Other endpoints**

    The backend also serves the UI and a few helpers:
    - `GET /` — serves `index.html`
    - `GET /cameras` — lists detected cameras (`{"id": ..., "name": ...}`) for the picker
    - `GET /logs` — recent log lines (handy for debugging)
    - `GET /debug` — current mode, cameras, pipeline state, and client count
  </Step>

  <Step>
    ### Understanding the Frontend

    The frontend (`index.html`) is a single HTML file (Tailwind via CDN) that connects to the WebSocket stream and renders each JPEG frame into a fullscreen `<img>` element:

    ```html
    <div class="fixed inset-0 z-0 flex items-center justify-center bg-black">
      <img
        id="video-stream"
        class="max-w-full max-h-full w-full h-full object-contain"
        alt=""
      />
    </div>
    ```

    **WebSocket Connection**

    Incoming binary frames are converted to a base64 data URL and assigned to the image's `src`:

    ```javascript
    function connect() {
      if (ws) ws.close();
      const proto = location.protocol === "https:" ? "wss:" : "ws:";
      ws = new WebSocket(`${proto}//${location.host}/stream`);
      ws.binaryType = "arraybuffer";
      ws.onopen = () => setStatus("Connected", "yellow");
      ws.onmessage = (e) => {
        if (e.data instanceof ArrayBuffer) {
          imgEl.src = arrayBufferToDataUrl(e.data);
          frameCount++;
          overlay.classList.add("opacity-0", "pointer-events-none");
          setStatus("Live", "green");
        }
      };
      ws.onclose = () => {
        setStatus("Disconnected", "red");
        setTimeout(connect, reconnectDelay);
        reconnectDelay = Math.min(reconnectDelay * 1.5, 10000);
      };
    }
    ```

    **Camera Picker**

    The page fetches `/cameras` to populate a dropdown. Selecting a camera sends a `switch_camera` command over the open WebSocket:

    ```javascript
    async function loadCameras() {
      const cameras = await (await fetch("/cameras")).json();
      camSelect.innerHTML = "";
      cameras.forEach((c, i) => {
        const o = document.createElement("option");
        o.value = c.id; o.textContent = c.name || `Camera ${i}`;
        camSelect.appendChild(o);
      });
    }

    camSelect.addEventListener("change", () => {
      if (camSelect.value && ws?.readyState === WebSocket.OPEN) {
        setStatus("Switching...", "yellow");
        ws.send(JSON.stringify({ switch_camera: camSelect.value }));
      }
    });
    ```

    **UI Features:**
    - Connection status indicator (yellow/green/red) with text
    - Live FPS counter updated every second
    - Camera picker dropdown for switching between connected webcams
    - Fullscreen toggle button
    - Loading spinner overlay while connecting
    - Automatic reconnection on disconnect, with backoff
  </Step>

  <Step>
    ### Understanding the Dockerfile

    The Dockerfile uses a slim Debian base image with GStreamer and its Python bindings installed, and creates a virtualenv (with system site packages so the GStreamer bindings are visible):

    ```dockerfile
    FROM debian:bookworm-slim

    RUN apt-get update && apt-get install -y --no-install-recommends \
        python3 \
        python3-pip \
        python3-venv \
        python3-gi \
        gir1.2-gst-plugins-base-1.0 \
        gir1.2-gst-plugins-bad-1.0 \
        gir1.2-gstreamer-1.0 \
        gstreamer1.0-tools \
        gstreamer1.0-plugins-base \
        gstreamer1.0-plugins-good \
        gstreamer1.0-plugins-bad \
        gstreamer1.0-plugins-ugly \
        gstreamer1.0-libav \
        gstreamer1.0-nice \
        v4l-utils \
        && rm -rf /var/lib/apt/lists/*

    WORKDIR /app

    RUN python3 -m venv --system-site-packages /app/venv
    ENV PATH="/app/venv/bin:$PATH"

    COPY requirements.txt .
    RUN pip install --no-cache-dir -r requirements.txt

    COPY app.py index.html ./
    COPY assets/ ./assets/

    ENV PYTHONUNBUFFERED=1

    ARG WENDY_DEVICE_TYPE
    ARG WENDY_DEBUG=false
    ENV WENDY_DEVICE_TYPE=${WENDY_DEVICE_TYPE}
    ENV WENDY_DEBUG=${WENDY_DEBUG}
    RUN if [ "$WENDY_DEBUG" = "true" ]; then pip install --no-cache-dir debugpy; fi

    EXPOSE 3003

    CMD ["/app/venv/bin/uvicorn", "app:app", "--host", "0.0.0.0", "--port", "3003"]
    ```

    The matching `requirements.txt` is just FastAPI and Uvicorn (GStreamer comes from the system packages):

    ```txt
    fastapi
    uvicorn[standard]
    ```

    **Key points:**
    - The Debian base ships the GStreamer plugins needed for MJPEG capture and software JPEG encoding
    - GStreamer Python bindings (`python3-gi`, `gir1.2-*`) enable pipeline control from Python
    - `v4l-utils` provides Video4Linux tools for webcam discovery and access
    - No frontend build step is needed since the UI is a single HTML file using Tailwind via CDN
  </Step>

  <Step>
    ### Configure Entitlements

    The generated `wendy.json` specifies the required device permissions:

    ```json
    {
        "appId": "webcam",
        "platform": "linux",
        "version": "0.1.0",
        "entitlements": [
            {
                "type": "network",
                "mode": "host"
            },
            {
                "type": "camera"
            },
            {
                "type": "gpu"
            }
        ],
        "readiness": {
            "tcpSocket": { "port": 3003 },
            "timeoutSeconds": 30
        },
        "hooks": {
            "postStart": {
                "cli": "wendy utils open-browser http://${WENDY_HOSTNAME}:3003"
            }
        }
    }
    ```

    - **network (host mode)**: Allows binding to ports directly on the device and enables WebSocket connections
    - **camera**: Grants access to `/dev/video*` webcam devices
    - **gpu**: Allows hardware-accelerated paths when the device and GStreamer plugins support them
    - **readiness**: Waits for port `3003` to accept connections before proceeding
    - **postStart**: Automatically opens the stream in your browser once ready
  </Step>
</Steps>

## Run Again on WendyOS

Connect your USB webcam to the Jetson, then run:

```bash
wendy run
```

The CLI will:
1. Build the Docker image (cross-compiling for ARM64)
2. Push the image to your device's local registry
3. Start the container with the configured entitlements

```
wendy run
✔︎ Searching for WendyOS devices [5.0s]
✔︎ Which device?: wendyos-zestful-stork.local [USB, LAN]
✔︎ Builder ready [0.2s]
✔︎ Container built and uploaded successfully! [22.1s]
✔ Success
  Started app
INFO:     Uvicorn running on http://0.0.0.0:3003
INFO:     Discovered 1 camera(s) via V4L2: /dev/video0 (HD Pro Webcam C920)
INFO:     Pipeline ready: v4l2src device=/dev/video0 ! image/jpeg ! appsink name=sink emit-signals=true max-buffers=2 drop=true sync=false
```

Your browser will open automatically once the stream is ready, thanks to the `postStart` hook. If it doesn't, navigate to:

```
http://wendyos-zestful-stork.local:3003
```

Replace the hostname with your device's actual hostname.

## Troubleshooting

<Accordions>
  <Accordion title="Webcam not detected">
    Ensure your webcam is connected before running the app. Check that the `camera` entitlement is in your `wendy.json`. You can verify the webcam is visible on the device:

    ```bash
    ssh wendy@wendyos-zestful-stork.local "ls /dev/video*"
    ```

    The app discovers cameras automatically (see `GET /cameras` and the picker in the UI). Common webcam devices are `/dev/video0` or `/dev/video1`. You can also hit `GET /debug` to see what the app detected.
  </Accordion>

  <Accordion title="No frames / pipeline failing">
    The app tries several GStreamer pipelines and logs `Pipeline ready: ...` for the one it selects. If none work, you'll see `Pipeline failed` / `Pipeline exception` lines. Check `GET /logs` or the app logs, confirm the `camera` entitlement is present, and make sure the webcam outputs MJPEG (most USB webcams do).
  </Accordion>

  <Accordion title="High latency or choppy video">
    The WebSocket stream delivers JPEG frames. If you experience latency:

    - Use a **USB 3.0 cable** for the webcam (not USB 2.0)
    - Lower the requested resolution by editing the `width`/`height` caps in the pipeline strings in `app.py`
    - Lower the `jpegenc quality=` value to reduce frame size when re-encoding
    - Ensure you're on a fast network (USB-C direct connection or wired LAN)
  </Accordion>

  <Accordion title="WebSocket disconnects frequently">
    The frontend automatically reconnects after 2 seconds. If disconnects are frequent, check your network stability. The server sends a ping every 30 seconds to keep the connection alive.
  </Accordion>
</Accordions>

## Next Steps

- Add the [YOLOv8 Webcam Detection](../../../../guides/tutorials/python/yolov8-webcam-detection/) guide to layer object detection on top of this stream
- Add recording to save video clips to disk
- Implement snapshot endpoints to capture still images on demand
- Add multiple camera support by parameterizing the device path