AIDino

Overview

AIDino is a reinforcement learning project that automates gameplay of the Chrome Dino game using deep reinforcement learning. The system connects to Chrome's DevTools Protocol via WebSocket, captures the game state through efficient screenshot processing, and makes intelligent decisions using a trained Proximal Policy Optimization (PPO) model.

Features

• Chrome DevTools Protocol integration for direct browser communication
• Efficient screenshot capture and processing using MSS
• Custom OpenAI Gym environment for reinforcement learning
• PPO model implementation via Stable Baselines3
• Image preprocessing with Sobel edge detection
• Progressive model checkpoints for tracking training progress

Technical Architecture

Chrome Integration

The system connects to Chrome's DevTools Protocol via WebSocket, allowing for:

• Programmatic control of the Dino game
• Precise input simulation (keyboard events)
• Real-time game state monitoring
• Screenshot capture of the game area

Custom Gym Environment

The project implements a custom OpenAI Gym environment (DinoEnv) that:

• Defines a discrete action space (jump, duck, do nothing)
• Processes screenshots into suitable observations
• Provides appropriate rewards based on game progress
• Handles game reset and initialization

Reinforcement Learning

The training process uses:

• Proximal Policy Optimization (PPO) algorithm
• MLP policy network architecture
• Reward function that encourages longer survival
• Periodic model checkpoints to track training progress

Model Checkpoints

The repository includes 10 progressive model checkpoints from training:

• ppo_dino_100k.zip through ppo_dino_1000k.zip
• Each checkpoint represents increased training (100k to 1M timesteps)

Getting Started

Environment Setup

1. Create and activate a virtual environment:

   # Create virtual environment
   python -m venv venv
   
   # Activate on Linux/macOS
   source venv/bin/activate
   
   # Activate on Windows
   # venv\Scripts\activate
   

2. Install dependencies:

   pip install -r requirements.txt
   

Running the Project

1. Launch Chrome with remote debugging:

   # Linux
   google-chrome --remote-debugging-port=1234
   
   # Windows
   # chrome.exe --remote-debugging-port=1234
   

2. Train the model:

   python rf.py
   

3. Use a pre-trained model by modifying the load path in rf.py:

   # Uncomment and modify this line in rf.py
   # model = PPO.load("models/ppo_dino_1000k", env=env)
   

How It Works

1. The environment connects to Chrome via DevTools Protocol
2. The game is initialized and the initial state is captured
3. For each timestep:
   • The current game state is captured through screenshots
   • Images are processed using edge detection
   • The model selects an action (jump, duck, nothing)
   • The action is executed through the Chrome connection
   • Rewards are calculated based on survival and game progress
4. Training continues until the model achieves optimal performance

Performance

The model demonstrates progressive improvement across training checkpoints, with later checkpoints showing significantly better game performance and higher average scores.

Contributing

Feel free to contribute to this project! Here are some ways you can help:

• Improve the reinforcement learning model or try different algorithms
• Enhance the image processing for better feature detection
• Optimize the performance for faster training
• Report bugs or suggest features by opening an issue