Skip to content

Tutorial - Whisper

Let's run OpenAI's Whisper , pre-trained model for automatic speech recognition on Jetson!

What you need

  1. One of the following Jetson devices:

    Jetson AGX Orin (64GB) Jetson AGX Orin (32GB) Jetson Orin NX (16GB) Jetson Orin Nano (8GB)

  2. Running one of the following versions of JetPack :

    JetPack 5 (L4T r35.x) JetPack 6 (L4T r36.x)

  3. Sufficient storage space (preferably with NVMe SSD).

    • 6.1 GB for whisper container image
    • Space for checkpoints

  4. Clone and setup jetson-containers : 

    git clone https://github.com/dusty-nv/jetson-containers
bash jetson-containers/install.sh

How to start

Use run.sh and autotag script to automatically pull or build a compatible container image. 

jetson-containers run $(autotag whisper)

The container has a default run command ( CMD ) that will automatically start the Jupyter Lab server, with SSL enabled.

Open your browser and access https://<IP_ADDRESS>:8888 .

Attention

Note it is https (not http ).

HTTPS (SSL) connection is needed to allow ipywebrtc widget to have access to your microphone (for record-and-transcribe.ipynb ).

You will see a warning message like this.

Press " Advanced " button and then click on " Proceed to (unsafe) " link to proceed to the Jupyter Lab web interface.

The default password for Jupyter Lab is nvidia .

Run Jupyter notebooks

Whisper repo comes with demo Jupyter notebooks, which you can find under /notebooks/ directory.

jetson-containers also adds one convenient notebook ( record-and-transcribe.ipynb ) to record your audio sample on Jupyter notebook in order to run transcribe on your recorded audio.

record-and-transcribe.ipynb

This notebook is to let you record your own audio sample using your PC's microphone and apply Whisper's medium model to transcribe the audio sample.

It uses Jupyter notebook/lab's ipywebrtc extension to record an audio sample on your web browser.

Attention

When you click the ⏺ botton, your web browser may show a pop-up to ask you to allow it to use your microphone. Be sure to allow the access.

Final check

Once done, if you click on the " ⚠ Not secure " part in the URL bar, you should see something like this.

Result

Once you go through all the steps, you should see the transcribe result in text like this.