Skip to content

Jetson AI Lab Research Group

The Jetson AI Lab Research Group is a global collective for advancing open-source Edge ML, open to anyone to join and collaborate with others from the community and leverage each other's work. Our goal is using advanced AI for good in real-world applications in accessible and responsible ways. By coordinating together as a group, we can keep up with the rapidly evolving pace of AI and more quickly arrive at deploying intelligent multimodal agents and autonomous robots into the field.

There are virtual meetings that anyone is welcome to join, offline discussion on the Jetson Projects forum, and guidelines for upstreaming open-source contributions .

Next Meeting - 1/7

With holiday schedules soon taking effect, we will reconvene in 2025 - thank you everyone for an amazing year!

In the meantime, enjoy the time with your families, and feel welcome to keep in touch through the forums, Discord, or LinkedIn.
The next team meeting is on Tuesday, January 7 th at 9am PST - see the invite below or click here to join the meeting in progress.

Topics of Interest

These are some initial research topics for us to discuss and investigate. This list will vary over time as experiments evolve and the SOTA progresses:

• Controller LLMs for dynamic pipeline code generation • Fine-tuning LLM/VLM onboard Jetson AGX Orin 64GB
• HomeAssistant.io integration for smart home [ 1 ] [ 2 ] • Continuous multi-image VLM streaming and change detection
• Recurrent LLM architectures (Mamba, RKVW, ect) [ 1 ] • Lightweight low-memory streaming ASR/TTS models
• Diffusion models for image processing and enhancement • Time Series Forecasting with Transformers [ 1 ] [ 2 ]
• Guidance, grammars, and guardrails for constrained output • Inline LLM function calling / plugins from API definitions
• ML DevOps, edge deployment, and orchestration • Robotics, IoT, and cyberphysical systems integration

New topics can be raised to the group either during the meetings or on the forums (people are welcome to work on whatever they want of course)

Contribution Guidelines

When experiments are successful, ideally the results will be packaged in such a way that they are easily reusable for others to integrate into their own projects:

Ongoing technical discussions are encouraged to occur on the forums or GitHub Issues, with status updates on projects given during the meetings.

Meeting Schedule

We'll aim to meet monthly or bi-weekly as a team in virtual meetings that anyone is welcome to join and speak during. We'll discuss the latest updates and experiments that we want to explore. Please remain courteous to others during the calls. We'll stick around after for anyone who has questions or didn't get the chance to be heard.

Tuesday January 7 th at 9am PST (1/7/24)

The agenda will be listed here beforehand - post to the forum to add agenda items. The meetings will be recorded so anyone unable to attend live can watch them after.

Past Meetings

Recordings Archive

Due to a backlog of editing/posting the previous meetings, here is a link containing the raw footage:
November 12, 2024

Topics Covered:

October 29, 2024

Topics Covered:

October 15, 2024

Topics Covered:

October 1, 2024

Topics Covered:

September 17, 2024

Topics Covered:

September 3, 2024

Topics Covered:

August 20, 2024

Topics Covered:

August 6, 2024

Topics Covered:

  • OpenVLA Fine-Tuning
  • Gemma-2-2b (Asier Arranz)
  • Ultralytics YOLOv8 (Lakshantha Dissanayake)
July 23, 2024

Topics Covered:

July 9, 2024

Topics Covered:

June 25, 2024

Topics Covered:

June 11, 2024

Topics Covered:

  • Agent Studio
  • HomeAssistant 2024.6
  • AWS IoT Greengrass (Romil Shah)
May 29, 2024

Topics Covered:

May 15, 2024

Topics Covered:

May 1, 2024

Topics Covered:

April 17, 2024

Topics Covered:

April 3, 2024

Project Kickoffs:

Team Members

Below are shown some of the sustaining members of the group who have been working on generative AI in edge computing:


Dustin Franklin, NVIDIA
Principal Engineer | Pittsburgh, PA
( jetson-inference , jetson-containers )

Nurgaliyev Shakhizat
Institute of Smart Systems and AI | Kazakhstan
( Assistive Devices , Vision2Audio , HPC )

Kris Kersey, Kersey Fabrications
Embedded Software Engineer | Atlanta, GA
( The OASIS Project , AR/VR, 3D Fabrication)

Johnny Núñez Cano
PhD Researcher in CV/AI | Barcelona, Spain
(Recurrent LLMs, Pose & Behavior Analysis)

Doruk Sönmez, ConnectTech
Intelligent Video Analytics Engineer | Turkey
(NVIDIA DLI Certified Instructor, IVA, VLM)

Akash James, Spark Cognition
AI Architect, UC Berkeley Researcher | Oakland
(NVIDIA AI Ambassador, Personal Assistants )

Mieszko Syty, MS/1 Design
AI/ML Engineer | Warsaw, Poland
(LLM, Home Assistants, ML DevOps)

Jim Benson, JetsonHacks
DIY Extraordinaire | Los Angeles, CA
(AI in Education, RACECAR/J )

Chitoku Yato, NVIDIA
Jetson AI DevTech | Santa Clara, CA
( JetBot , JetRacer , MinDisk , Containers)

Dana Sheahen, NVIDIA
DLI Curriculum Developer | Santa Clara, CA
(AI in Education, Jetson AI Fundamentals )

Sammy Ochoa, NVIDIA
Jetson AI DevTech | Austin, TX
( Metropolis Microservices )

John Welsh, NVIDIA
( NanoOWL , NanoSAM , JetBot , JetRacer , torch2trt , trt_pose , Knowledge Distillation )

Dave Niewinski
Dave's Armoury | Waterloo, Ontario
( GLaDOS , Fetch , Offroad La-Z-Boy , KUKA Bot )

Gary Hilgemann, REBOTNIX
CEO & AI Roboticist | Lünen, Germany
( GUSTAV , SPIKE , VisionTools , GenAI )

Elaine Wu, Seeed Studio
AI & Robotics Partnerships | Shenzhen, China
( reComputer , YOLOv8 , LocalJARVIS , Voice Bot )

Patty Delafuente, NVIDIA
Data Scientist & UMBC PhD Student | MD
(AI in Education, DLI Robotics Teaching Kit )

Song Han, MIT HAN Lab
NVIDIA Research | Cambridge, MA
( Efficient Large Models , AWQ , VILA )

Bryan Hughes, Mimzy AI
Founder, Entrepreneur | SF Bay Area
(Multimodal Assistants, AI at the Edge)

Tianqi Chen, CMU Catalyst
OctoML , CTO | Seattle, WA
( MLC , Apache TVM , XGBoost )

Michael Grüner, RidgeRun
Team Lead / Architect | Costa Rica
(Embedded Vision & AI, Multimedia)

Jesse Flot, CMU Robotics Academy
Co-Director | Pittsburgh, PA
( Applied AI & Robotics , Autonomy Foundations )

Paul DeCarlo, Microsoft
Professor | University of Houston
( Azure IoT , Video Analytics , Microsoft JARVIS )

Mike Hansen, Nabu Casa
Voice Engineering Lead | Urbandale, Iowa
( Home Assistant , Piper TTS , Wyoming )

Lakshantha Dissanayake, Ultralytics
Embedded CV Engineer | Vancouver, BC
( YOLOv8 , TensorRT , DeepStream )

Kerry Shih, GenAI Nerds
Founder, CEO | Los Angeles, CA
( Gapi )

Ziad Youssfi, CMU
ECE Professor | Pittsburgh, PA
(ML in Robotics & Embedded Systems)

Walter Lucetti, Stereolabs
Robotics & Vision Engineer | Paris, France
( MyzharBot , ROS2 , GStreamer )

Raffaello Bonghi, NVIDIA
AI & Robotics Engineer | Manchester, UK
( Nanosaur , Panther , jetson-stats )

Alvaro Costa, ANS Group
AI & Robotics Lead | Epsom, UK
( TeknTrash , StipraPOD )

David Pearson, ConnectTech
Embedded Systems Engineer | Ontario, CA
(Edge AI Systems, Vision/Language Models)

Jason Seawall, Numurus
CEO | Seattle, WA
( NEPI , Edge AI & Automation)

Martin Cerven
AI Researcher | Germany
(Autonomous Robotics, Voicecraft)

Romil Shah, Amazon
GenAI IIoT @ AWS | San Jose, CA
( aws-samples/genai-at-edge )

Kabilan Kb, Roboticist
NVIDIA DLI Ambassador | Tamil Nadu, IN
( ROS2 Tutorials , Autonomous Wheelchair )