Associated Reality: A cognitive Human–Machine Layer for autonomous driving

Advanced Driver Assistance Systems (ADAS) and Automated and Autonomous Vehicles (AV) are cooperative systems and processes that use: artificial intelligence, cognitive methods, cloud technologies, cooperative vehicle-to-everything-communications (V2X), software–hardwareplatforms, sensor platforms and incipient intelligent transport infrastructures, to get self-driving systems and smart connected mobility services. This paper, to support automated driving systems (assisted, semi-autonomous and fully autonomous vehicles), introduces a cognitive layer called Associated Reality to enhance the involved information, knowledge and communication processes. The architecture defined includes an augmented Local Dynamic Map, with complementary layers, and an augmented Graph Database, with complementary semantic–cognitive relations, for the considered purpose, in cooperative human–machine and machine–machine systems. Virtual augmented landmarks are defined to improve the connectivity and intelligence of the involved spatial-information systems. Different structure landmarks and sequence landmarks (which includes regular, repetitive and periodic landmarks) are defined, categorized and used in diverse visual localization and mapping scenarios, for autonomous driving. In this paper, it is also shown, as a proof-of-concept for vehicle localization and mapping in road tunnels, the visual detection of different sequences of periodic luminaires, using YOLO v3 for the corresponding LED lights detection, or a specific alternative procedure developed with very low computational cost. •Specification of the cognitive layer Associated Reality, for autonomous driving systems.•Integration of an augmented Local Dynamic Map and an augmented Graph Database.•Definition and application of Structure Landmarks for spatial intelligent systems.•Micro and macro-localization with regular landmarks for autonomous driving.•Visual localization and mapping in vehicle tunnels using periodic luminaires.