Raspberry Pi AI HAT+ 2 adds 40 TOPS accelerator to the single-board computer

Add-on brings chatbots and LLMs to the edge

by · TechSpot

Serving tech enthusiasts for over 25 years.
TechSpot means tech analysis and advice you can trust.

TL;DR: Raspberry Pi products have been used in a wide range of custom computing applications, from industrial automation to IoT and everything in between. Thanks to a recently introduced add-on, the boards can now also handle and manage complex generative AI workloads.

Raspberry Pi has started selling the AI HAT+ 2, an add-on board that represents a significant upgrade over the AI HAT+ model launched in 2024. While the original board could only accelerate certain types of neural network models, the Raspberry Pi AI HAT+ 2 is powerful enough to manage and run a popular subset of generative AI models.

When paired with a Raspberry Pi 5 base board, the AI HAT+ 2 can help close the GenAI performance gap that the single-board computer platform has faced so far. Chatbot interactions and local LLM workloads are now easier than ever, although the Raspberry Pi Foundation notes that this is not on the same scale as ChatGPT in terms of parameter size and model complexity.

The AI HAT+ 2 includes a Hailo-10H neural network accelerator, a chip capable of delivering 40 TOPS of inferencing performance. By comparison, the first AI HAT was available with either a Hailo-8 (26 TOPS) or Hailo-8L (13 TOPS) accelerator and was primarily designed to enhance the performance of vision-based neural network models.

Raspberry Pi said that a Pi 5 board equipped with an AI HAT+ 2 module can provide developers with a low-latency, cost-effective device for locally performed GenAI tasks. The AI HAT+ 2 is well-suited for managing smaller inference workloads, thanks in large part to its 8GB of dedicated on-board RAM and an improved hardware architecture.

Vision-based models should perform similarly to the original AI HAT+, with the added benefit of Raspberry Pi's integrated camera software stack. The foundation encourages developers already using the original board to upgrade, noting that existing software tools are easy to adapt to the new model.

// Related Stories

Alongside the AI HAT+ 2, the foundation is providing several LLMs that can be readily installed and run on the device. Model complexity ranges from 1 billion parameters (Llama 3.2) to 1.5 billion parameters, supporting popular GenAI tasks such as chatbot queries (Qwen2), code generation (Qwen2.5-Coder), language translation, and more.

Raspberry Pi cautions that cloud-based LLMs like ChatGPT or Gemini have been trained on 500 billion to 2 trillion parameters, allowing for far more complex interactions. However, by fine-tuning or retraining the smaller models with custom datasets, developers can overcome these limitations to power specific GenAI applications.

The Raspberry Pi AI HAT+ 2 add-on is available for $130. The UK-based organization is also providing a comprehensive AI HAT guide, with step-by-step instructions on how to install and use the product.