Skip to main content

Jupyter Notebook Guide

Overview​

This guide provides instructions for using the interactive Jupyter notebooks included with the Physical AI & Humanoid Robotics course. The notebooks are designed to provide hands-on experience with robotics concepts using both real hardware and simulation environments.

Notebook Structure​

Each module includes a corresponding notebook:

  • chapter-01-ros2.ipynb - ROS 2 fundamentals and implementation
  • chapter-02-simulation.ipynb - Simulation environments and tools
  • chapter-03-isaac.ipynb - NVIDIA Isaac platform applications
  • chapter-04-vla.ipynb - Vision-Language-Action models

Running Notebooks​

Prerequisites​

Before running any notebook, ensure:

  1. Python virtual environment is activated
  2. Required packages are installed
  3. Jupyter is installed and accessible
# Activate virtual environment
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Verify Jupyter
jupyter --version

Starting Jupyter​

# From the project root directory
jupyter notebook

This will open the Jupyter interface in your default web browser.

Hardware vs. Simulation Modes​

Hardware Mode​

When running on systems with appropriate hardware (NVIDIA RTX GPU, robotics platform), notebooks will:

  • Use real sensors and actuators
  • Connect to physical robots
  • Provide real-time performance
  • Access full computational capabilities

Simulation Mode​

For systems without specialized hardware, notebooks include:

  • Software-based simulation environments
  • Mock sensor data
  • Fallback algorithms
  • Reduced computational requirements

Most notebooks have a configuration cell at the beginning that allows you to select the execution mode:

# Execution mode configuration
EXECUTION_MODE = "simulation"  # Options: "hardware", "simulation"

Notebook Execution Guidelines​

1. Sequential Execution​

Execute cells in order to maintain state and dependencies:

  • Run initialization cells first
  • Execute setup cells before using functions
  • Follow the notebook's suggested execution order

2. Expected Execution Time​

Each notebook is designed to complete within 30 minutes:

  • ROS 2 notebook: ~20 minutes
  • Simulation notebook: ~25 minutes
  • Isaac notebook: ~30 minutes (with hardware) or ~20 minutes (simulation)
  • VLA notebook: ~30 minutes (with hardware) or ~25 minutes (simulation)

3. Resource Management​

To manage computational resources:

  • Close unused notebooks
  • Restart kernels periodically for memory management
  • Use simulation mode if experiencing performance issues

Troubleshooting Common Issues​

1. Memory Issues​

If you encounter memory errors:

# Clear variables and reset kernel
import gc
import torch

# Clear Python garbage
gc.collect()

# Clear GPU memory if using PyTorch
if torch.cuda.is_available():
    torch.cuda.empty_cache()

2. GPU Memory Issues​

For GPU memory problems:

# Reduce batch sizes in model operations
BATCH_SIZE = 1  # Instead of default batch size

3. Connection Issues​

For hardware connection problems:

  • Verify hardware is properly connected
  • Check network configuration
  • Ensure correct drivers are installed
  • Use simulation mode as fallback

Notebook-Specific Instructions​

Chapter 1: ROS 2​

This notebook covers:

  • Node creation and management
  • Publisher/subscriber patterns
  • Service and action implementation
  • TF transforms and coordinate frames
# ROS 2 specific setup
import rclpy
from std_msgs.msg import String

# Initialize ROS context
rclpy.init()

Chapter 2: Simulation​

This notebook covers:

  • Gazebo simulation environment setup
  • Robot model loading and configuration
  • Sensor simulation and data processing
  • Controller implementation
# Simulation mode activation
SIMULATION_MODE = True

Chapter 3: NVIDIA Isaac​

This notebook covers:

  • Isaac ROS package usage
  • GPU-accelerated perception
  • Navigation stack implementation
  • Isaac Sim integration
# Isaac-specific imports
import isaac_ros_python

Chapter 4: Vision-Language-Action​

This notebook covers:

  • VLA model loading and configuration
  • Multimodal input processing
  • Action generation from vision-language inputs
  • Integration with robotics platforms
# VLA model configuration
MODEL_NAME = "vla_model"  # or "vla_model_simulation"

Best Practices​

1. Code Modification​

  • Experiment with code modifications in separate cells
  • Keep original code for reference
  • Document your changes and observations

2. Data Handling​

  • Use provided sample data initially
  • Gradually increase complexity
  • Save intermediate results when needed

3. Performance Monitoring​

  • Monitor execution time of each cell
  • Track memory usage
  • Note performance differences between modes

Safety Considerations​

When using real hardware:

  • Ensure safe operating environment
  • Monitor robot behavior continuously
  • Have emergency stop procedures ready
  • Follow all safety protocols

Customization​

Adding Custom Code​

You can add custom code cells to experiment with:

# Custom experiment cell
# Add your modifications here
# Document your findings

Data Input​

Notebooks accept custom data inputs:

# Custom data path configuration
CUSTOM_DATA_PATH = "path/to/your/data"

Evaluation and Grading​

Completion Criteria​

Each notebook includes:

  • Code execution without errors
  • Expected output verification
  • Understanding questions (in markdown cells)
  • Optional challenges for advanced learners

Self-Assessment​

Check your understanding by:

  • Running all cells successfully
  • Understanding the code purpose
  • Completing optional exercises
  • Documenting your observations

Integration with Course​

Module Alignment​

  • Chapter 1 notebook aligns with Module 1: ROS 2
  • Chapter 2 notebook aligns with Module 2: Simulation
  • Chapter 3 notebook aligns with Module 3: NVIDIA Isaac
  • Chapter 4 notebook aligns with Module 4: Vision-Language-Action

Assignment Submission​

For assignment submissions:

  • Ensure all cells execute without errors
  • Include your custom modifications
  • Document your findings in markdown cells
  • Submit the completed notebook file

Support and Resources​

Getting Help​

  • Check the FAQ section in each notebook
  • Use the discussion forum for complex issues
  • Contact instructors during office hours

Additional Resources​

  • ROS 2 documentation
  • Simulation platform documentation
  • NVIDIA Isaac documentation
  • VLA model documentation

Advanced Usage​

Batch Processing​

For advanced users, notebooks support batch processing:

# Batch processing configuration
BATCH_PROCESSING = True
BATCH_SIZE = 10

Distributed Computing​

Some notebooks support distributed computing:

# Distributed computing setup
DISTRIBUTED = True
NUM_WORKERS = 4

Version Compatibility​

Notebooks are tested with:

  • Jupyter Notebook 7.0+
  • Python 3.10+
  • Compatible with JupyterLab interface

File Management​

Saving Work​

  • Save notebooks frequently using Ctrl+S
  • Create copies before major modifications
  • Use version control for significant changes

Export Options​

Notebooks can be exported to:

  • Python scripts
  • HTML documents
  • PDF files
  • Markdown format
# Export to Python script
jupyter nbconvert --to python chapter-01-ros2.ipynb

This guide should help you make the most of the interactive learning experience provided by the course notebooks. Remember to start with simulation mode if you're unsure about hardware requirements, and don't hesitate to experiment and explore the concepts in a safe environment.