Exploring the design elements of open evaluation

PurposeFirms increasingly integrate a wide range of actors in the early ideation and concept creation phases of innovation processes leading to the collection of a large number of ideas. This creates the challenge of filtering the most promising ideas from a large number of submissions. The use of external stakeholders into the evaluation and selection of submissions (i.e. open evaluation (OE)) might be a viable alternative. The purpose of this paper is to provide a state-of-the-art analysis on how such OE systems are designed and structured.Design/methodology/approachSince OE is a new phenomenon, an exploratory qualitative research approach is adopted. In all, 122 instances of OE in 90 innovation contest cases are examined for their design elements.FindingsThis research reveals that OE systems are configured in many different ways. In total, 32 design elements and their respective parameters are identified and described along the six socio-technical system components of an OE system. This study allows for a comprehensive understanding of what OE is and what factors need to be taken into consideration when designing an OE system.Practical implicationsScholars and professionals may draw insights on what design choices to make when implementing OE.Originality/valueThe comprehensive analysis performed in this study contributes to research on open and user innovation by examining the concept of OE. In particular, it extends knowledge on design elements of OE systems.