Some metrological aspects of preferences expressed by prioritization of alternatives

•A distance metric between two prioritization chains is proposed.•A metrological approach to interpretation of prioritization chain data is proposed.•Metrological ideas for studying experimental data such as R&R are applied.•Main ideas are demonstrated using illustrative examples. With the growth of industrial process sophistication, new horizons have opened up for modern metrology. Here, we focus on metrological aspects of multi-agent assessment, expressed by prioritization. Specifically, we examine the case of a group of agents expressing their preferences over a fixed set of alternatives by means of weak orders. Preference chains are compared via simple distance metrics based on cosine similarity. Following a metrological approach, the agents’ precision in performing prioritization tasks is evaluated in terms of repeatability and reproducibility. This is done by analyzing the preference variations in different trials of the same prioritization task and among different agents involved in the prioritization process. The paper introduces a median concept for aggregating multi-agent preference orderings to obtain a collective solution. The proposed strategy is illustrated through a case study of consumers expressing their preferences regarding a set of alternatives. Some further research opportunities are also discussed.