Awesome Physical AI

A curated list of open-source tools for physical AI and robotics — foundation models, world models, simulators, learning frameworks, benchmarks, and runtime.

Maintained by Backblaze.

Related Lists

• Awesome ML Data Pipelines
• Awesome Multimodal Data
• Awesome Agent Infrastructure
• Awesome Image Generation
• Awesome Video Generation
• Awesome Audio Generation

Contents

• Robotics Foundation Models
• World Models
• Simulation and Physics Engines
• Robot Learning Frameworks
• Datasets and Benchmarks
• Robot Middleware and Runtime
• Teleop and Data Collection
• SDKs and Developer Tooling
• Templates and Example Projects

---

Robotics Foundation Models

Vision-language-action (VLA) and embodied foundation models you can fine-tune and deploy.

• OpenVLA – 7B-parameter open vision-language-action model trained on 970k demonstrations from Open X-Embodiment. Strong generalist manipulation baseline. Docs
• Octo – Transformer-based generalist robot policy pretrained on 800k trajectories. Flexible conditioning on goal images or language. Docs
• CrossFormer – Transformer policy trained on 900k trajectories across 30 embodiments. Single model weights control arms, wheeled robots, quadcopters, and quadrupeds via language or goal-image conditioning. Docs
• HoloMotion – Foundation model for humanoid whole-body control from Horizon Robotics. End-to-end pipeline for motion retargeting, distributed training, and ROS 2 deployment on real hardware (Unitree G1). Pre-trained weights on HuggingFace. Docs
• HuggingFace SmolVLA – Compact VLA model from HuggingFace designed to run on consumer hardware while retaining generalist behaviour. Docs
• openpi – Open-source code and weights for π0, π0-FAST, and π0.5 VLA models from Physical Intelligence. Fine-tuning recipes for ALOHA, DROID, and custom platforms. Docs
• Physical Intelligence π-0 – General-purpose robot foundation model from Physical Intelligence. Weights partially released; commercial access via partners.
• RT-X / RT-2 – Google DeepMind's RT-X family of generalist robotics transformers and the Open X-Embodiment dataset that underpins them.

World Models

Generative world models for physical simulation, planning, and synthetic data.

• V-JEPA 2 – Meta FAIR's non-generative video predictive model. Learns world representations useful for planning and robot perception. Docs
• NVIDIA Cosmos – World foundation-model platform for physical AI. Cosmos-Predict generates physics-aware video from text, image, video, or sensor inputs. Docs
• 1X World Model – Humanoid-centric world model from 1X. Generates high-fidelity first-person rollouts for policy evaluation in simulation.

Simulation and Physics Engines

Physics simulators and training environments for robotics and embodied AI.

• Genesis – Pure-Python physics platform for generalist embodied AI. Unifies rigid, soft, fluid, and differentiable simulation. Docs | SDK: Python (pip install genesis-world)
• PyBullet – Python bindings for the Bullet physics engine. Still widely used for manipulation and locomotion baselines. Docs | SDK: Python (pip install pybullet)
• MuJoCo – DeepMind's fast multi-joint dynamics engine. Standard for contact-rich manipulation and locomotion research. Docs | SDK: Python (pip install mujoco), C
• NVIDIA Isaac Lab – Unified robot-learning framework on Isaac Sim. GPU-accelerated training with rich sensor and contact simulation. Docs
• Drake – MIT/TRI's model-based design toolbox. Rigid-body dynamics, trajectory optimization, and system modeling for robotics. Docs
• Brax – Google's differentiable physics engine in JAX. Massively parallel RL on a single accelerator. SDK: Python (pip install brax)
• Gazebo – Open-source robotics simulator with tight ROS 2 integration. Modern Gazebo (Harmonic/Ionic) is the successor to Gazebo Classic. Docs
• Genie Sim – Humanoid-focused simulation platform built on Isaac Sim. LLM-driven scene generation, 5,100+ validated assets, 200+ benchmark tasks, and zero-shot sim-to-real transfer toolchain. Docs
• Newton – GPU-accelerated physics simulation engine built on NVIDIA Warp, co-developed by NVIDIA, Google DeepMind, and Disney Research. Integrates MuJoCo Warp as its primary backend. Docs | SDK: Python (pip install newton)
• NVIDIA Isaac Sim – Open-source robotics simulation platform on NVIDIA Omniverse. GPU-accelerated physics, photorealistic rendering, synthetic data generation, and ROS 2 bridge. Underlying runtime for Isaac Lab. Docs
• robosuite – MuJoCo-based simulation framework and benchmark suite for robot learning. Supports humanoids, custom robot composition, and photorealistic rendering. Docs | SDK: Python (pip install robosuite)
• RoboVerse – Unified simulation platform, synthetic dataset, and benchmark suite for scalable robot learning. MetaSim abstraction wraps 8+ physics engines (Isaac Lab, MuJoCo, SAPIEN, Genesis, PyBullet) under one API. Accepted RSS 2025. Docs

Robot Learning Frameworks

Imitation learning, RL, and policy training toolkits targeting robot control.

• HuggingFace LeRobot – End-to-end robot-learning library from HuggingFace. Datasets on the Hub, policies, and low-cost reference hardware (SO-100). Docs | SDK: Python (pip install lerobot)
• Stable-Baselines3 – Reliable PyTorch implementations of popular RL algorithms. De-facto baseline for reproducible RL research. Docs | SDK: Python (pip install stable-baselines3)
• Robomimic – Research framework for imitation learning from human demonstrations. Standardized dataset format and algorithm zoo. Docs
• Ark – Python-first robot-learning framework. Gym-style interface for collecting data, training ACT/Diffusion Policy policies, and switching between simulation and real hardware with minimal code changes. Docs
• HumanoidVerse – Multi-simulator locomotion training framework for humanoid robots supporting IsaacGym, IsaacSim, and Genesis. Enables sim-to-sim and sim-to-real transfer with domain randomization.
• MuJoCo Playground – GPU-accelerated suite of robot learning environments built on MJX. Supports locomotion, manipulation, and dexterous hands with zero-shot sim-to-real transfer. Docs | SDK: Python (pip install playground)
• OpenVLA-OFT – Fine-tuning recipe for VLA models combining parallel decoding, action chunking, and continuous-action regression. Achieves 25-50x inference speedup and 97.1% on LIBERO; tested on bimanual ALOHA hardware. Docs
• RLlib (Ray) – Scalable RL library part of Ray. Distributes training across clusters; supports most standard algorithms. SDK: Python (pip install ray[rllib])

Datasets and Benchmarks

Public robot datasets, embodied AI benchmarks, and evaluation suites.

• ManiSkill – GPU-parallel manipulation benchmark built on SAPIEN. Millions of env-steps per second on a single GPU. Docs
• Meta-World – 50-task manipulation benchmark for meta-learning and multitask RL. Docs
• RLBench – Large-scale benchmark for robot learning with 100+ tasks built on CoppeliaSim. Docs
• LIBERO – Benchmark for lifelong robot learning. 130 tasks across four skill categories, with standard splits and evaluation protocols. Docs
• AgiBot World – Large-scale bimanual manipulation dataset with 1M+ trajectories from 100 robots across 100+ real-world scenarios. Includes the GO-1 foundation model and LeRobot-based toolchain. Docs
• DROID – 76k in-the-wild Franka manipulation trajectories (350 h) collected across 564 scenes at 13 institutions. Dataset, policy-learning code, and hardware setup guide are open source. Docs
• Gymnasium-Robotics – Farama Foundation's collection of MuJoCo-based RL environments for robotics. Includes Fetch arm, Shadow Hand, Franka Kitchen, Adroit, and multi-agent variants. Gymnasium API compatible. Docs | SDK: Python (pip install gymnasium-robotics)
• Open X-Embodiment – Collaborative dataset of 22 embodiments, 1M+ episodes, 527 skills. Standard training corpus for generalist policies. Docs
• PHUMA – Physically-grounded humanoid locomotion dataset from KAIST. Physics-constrained retargeting pipeline (PhySINK) adapts large-scale human motion capture to Unitree G1 and H1-2 while enforcing joint limits and eliminating foot skating. Docs
• RoboCasa – Large-scale simulation framework for household robot training. RoboCasa365 ships 365 tasks, 2,500+ kitchen scenes, and 2,200+ hours of demonstration data. Docs

Robot Middleware and Runtime

Messaging, scheduling, and dataflow layers for running robots in the real world.

• ROS 2 – Standard robotics middleware. DDS-based messaging, real-time support, and a massive ecosystem of drivers and tools. Docs
• Dora-rs – Low-latency dataflow robotics runtime written in Rust. Python-friendly; often faster than ROS 2 for perception pipelines. Docs | SDK: Python (pip install dora-rs), Rust

Teleop and Data Collection

Hardware kits and software for collecting demonstration data on real and simulated robots.

• LeRobot SO-100 Arm – Open-source 6-DOF arm co-developed with HuggingFace LeRobot. ~$120 BOM; standard entry-point for homegrown robot data.
• Mobile ALOHA – Low-cost bimanual mobile-manipulation platform with open hardware and teleop code. Reference data-collection rig. Docs
• GELLO – Low-cost, 3D-printable intuitive teleoperation system for arm manipulators. Popular for collecting demonstration data. Docs

SDKs and Developer Tooling

Libraries for 3D, kinematics, manipulation, and general robotics development.

• Open3D – 3D data processing library with point-cloud registration, reconstruction, and a modern ML module. Docs | SDK: Python (pip install open3d), C++
• PyTorch3D – Reusable 3D components in PyTorch from Meta FAIR. Differentiable rendering, mesh ops, and point-cloud primitives. Docs | SDK: Python (pip install pytorch3d)
• NVIDIA GR00T – NVIDIA's open humanoid foundation-model toolkit. Training recipes, policies, and data-generation pipelines for humanoids. Docs
• GR00T-WholeBodyControl – Unified platform for training and deploying humanoid whole-body controllers. Includes SONIC, a behavior foundation model trained on large-scale motion-capture data for walking, manipulation, and VR teleoperation. Docs

Templates and Example Projects

Reference implementations, demos, and starter projects.

• Isaac Lab Examples – Official template repo for building Isaac Lab tasks and training environments.
• LeRobot Tutorials – Step-by-step tutorials for recording robot data, training policies, and evaluating on real hardware.

---

Contributing

Contributions are welcome. See CONTRIBUTING.md. One entry per PR — edit entries.yaml only and let the maintainers regenerate README.md.

License

Released under CC0 1.0 Universal. You may copy, modify, and redistribute without attribution.

About Backblaze B2

Backblaze B2 Cloud Storage is S3-compatible object storage designed for AI and media workloads. This list is maintained as part of our work making B2 a convenient storage layer for AI workflows.