Real-time Hazard Prediction in Connected Autonomous Vehicles: A Digital Twin Approach

Abstract

The growing interest in connected autonomous vehicles (CAVs) has intensified the focus on technologies and algorithms that enhance behavior, comfort, and safety. Among these, the concept of Digital Twins (DT) represents an emerging field of research that is now beginning to be applied to autonomous systems. Traditional Advanced Driver-Assistance Systems (ADAS) can prevent real-time collisions using sensor data. However, we propose that employing a DT can enable the accounting for complex, simulated decisions before they occur in reality. This paper introduces an initial model of a Digital Twin, founded on an internal simulator aligned with vehicle control architecture, for real-time hazard prediction and effective decision-making. Our DT synchronizes with the vehicle’s state to simulate various hazardous scenarios in advance, allowing for preemptive actions. To support our hypothesis, we introduce an algorithm for the early detection of potential collisions between CAVs and pedestrians through the unsupervised simulation of diverse traffic scenarios. This solution integrates the CORTEX cognitive architecture with CARLA for internal simulation, leveraging probabilistic models to select optimal scenarios. Employing data from external pedestrian cameras, a particle filter predicts the most probable pedestrian trajectories via DT simulations, thereby informing safe maneuvers. Although the algorithm itself is established, the novelty of our approach lies in incorporating a simulator within the digital twin. This simulator, informed by real-time data on the vehicle’s and environment’s state, facilitates appropriate responses to unpredictable behaviors. We have conducted extensive tests with an actual autonomous electric vehicle on a university campus to validate the system’s predictive and adaptive functions.