Weisfeiler-Leman in the Bamboo : Novel AMR Graph Metrics and a Benchmark for AMR Graph Similarity

Several metrics have been proposed for assessing the similarity of (abstract) meaning representations (AMRs), but little is known about how they relate to human similarity ratings. Moreover, the current metrics have complementary strengths and weaknesses: Some emphasize speed, while others make the alignment of graph structures explicit, at the price of a costly alignment step. In this work we propose new that unify the strengths of previous metrics, while mitigating their weaknesses. Specifically, our new metrics are able to match contextualized substructures and induce n:m alignments between their nodes. Furthermore, we introduce a enchmark for MR etrics ased on vert bjectives ( ), the first benchmark to support empirical assessment of graph-based MR similarity metrics. maximizes the interpretability of results by defining multiple that range from to that probe a metric’s robustness against meaning-altering and meaning- preserving graph transformations. We show the benefits of by profiling previous metrics and our own metrics. Results indicate that our novel metrics may serve as a strong baseline for future work.