A One-Size-Fits-All Approach to Improving Randomness in Paper Assignment
The assignment of papers to reviewers is a crucial part of the peer review processes of large publication venues, where organizers (e.g., conference program chairs) rely on algorithms to perform automated paper assignment. As such, a major challenge for the organizers of these processes is to specif...
Gespeichert in:
Hauptverfasser: | , , , |
---|---|
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext bestellen |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
Zusammenfassung: | The assignment of papers to reviewers is a crucial part of the peer review
processes of large publication venues, where organizers (e.g., conference
program chairs) rely on algorithms to perform automated paper assignment. As
such, a major challenge for the organizers of these processes is to specify
paper assignment algorithms that find appropriate assignments with respect to
various desiderata. Although the main objective when choosing a good paper
assignment is to maximize the expertise of each reviewer for their assigned
papers, several other considerations make introducing randomization into the
paper assignment desirable: robustness to malicious behavior, the ability to
evaluate alternative paper assignments, reviewer diversity, and reviewer
anonymity. However, it is unclear in what way one should randomize the paper
assignment in order to best satisfy all of these considerations simultaneously.
In this work, we present a practical, one-size-fits-all method for randomized
paper assignment intended to perform well across different motivations for
randomness. We show theoretically and experimentally that our method
outperforms currently-deployed methods for randomized paper assignment on
several intuitive randomness metrics, demonstrating that the randomized
assignments produced by our method are general-purpose. |
---|---|
DOI: | 10.48550/arxiv.2310.05995 |