nuScenes Knowledge Graph

Ontologies and Knowledge Graphs of the ICCV 2023 workshop paper "nuScenes Knowledge Graph - A comprehensive semantic representation of traffic scenes for trajectory prediction". Content nSKG (nuScenes Knowledge Graph): knowledge graph for the nuScenes dataset, that models all scene participants and road elements, as well as their semantic and spatial relationships nSTP (nuScenes Trajectory Prediction Graph): heterogeneous graph of the nuScenes dataset for trajectory prediction in PyTorch Geometric (PyG) format. It extends nSKG for example by transformation into agents' local coordinate systems, relevant agent extraction, semantic relationships between agents. nuScenes_agent_onto: ontology for the traffic participants (agents) nuScenes_map_onto: ontology for the extended map stardog_rules: SPARQL rules for mapping nuScenes concepts to the agent ontology How to use nSKG represents the KG created on the basis of the nuScenes ontologies nuScenes_agent_onto.ttl and nuScenes_map_onto.ttl and materializing the nuScenes annotation dataset. It can be used for applications where relational information between entities are important. The ontologies are in Turtle format and can be viewed by ontology editors such as Protege. nSTP represents the extended version of nSKG, where agents are represented in local coordinate systems to enforce shift- and rotation-invariance. It also includes semantic relationships between agents, e.g. whether agents are on neighboring lanes, the same lane or might intersect. This is done based on the semantic scene graph describe in (Towards Traffic Scene Description: The Semantic Scene Graph). The data is provided in PyTorch Geometric format and directly be used to train graph neural network for trajectory prediction. Acknowledgements Special thanks to Motional for the permission to distribute this modified version of the nuScenes dataset. Citation If you use this work please cite @inproceedings{ title={nuScenes Knowledge Graph - A comprehensive semantic representation of traffic scenes for trajectory prediction}, author={Leon Mlodzian and Zhigang Sun and Hendrik Berkemeyer and Sebastian Monka and Zixu Wang and Stefan Dietze and Lavdim Halilaj and Juergen Luettin},booktitle={International Conference on Computer Vision (ICCV), Workhsop on Scene Graphs and Graph Representation Learning (SG2RL)}, year={2023} }