Consensual Assessment in the New Domain of E-Textiles: Comparing Insights From Expert, Quasi-Expert, and Novice Judges

Establishing what constitutes creativity in a domain is something for which we often look to experts-individuals versed in a domain's history and able to identify timeworn ideas from fresh ones. Such valuations of creative merit are tied to a familiarity with past and present trends and, therefore, opinions of newcomers are often ignored. However, what about domains that build upon new, unexplored practices? This study examines the creativity ratings of judges with varying expertise in the emergent domain of electronic textiles (or e-textiles). E-textiles are fabrics that have programmable electronics such as sensors and actuators embedded in them toward a variety of expressive and functional ends. Judges included domain pioneers ("experts"), individuals with over 20 hr of nonprofessional experience in the domain ("quasi-experts"), and individuals untrained in the domain ("novices"). Each group evaluated the creativity of e-textile artifacts from an online gallery using the Consensual Assessment Technique (CAT). Our analyses found high interjudge reliability within all groups and between quasi-experts and experts, suggesting that quasi-experts could be sufficiently trained to judge the creativity of artifacts on par with experts. Furthermore, larger panels of novice judges may serve as an alternative, but it would be with the caveat that novice scores represent the opinions of general audiences that might not understand technical practices of e-textiles. Findings offer alternative considerations for how creativity is assessed in emergent, technology-rich domains and have implications for judge recruitment.