Determining similarity using human generated data

Subjective user-generated data can be utilized to determine visually similar items. Various item descriptors can be determined for a pair of items, which can provide an objective measure of visual aspects of those items, such as how similar those items are in color, style, material, or texture. The ways in which users interact with data for those items, either explicitly or implicitly through user behavior, can provide a level of perceived visual similarity on behalf of these users. The perceived, subjective visual similarity data from the users can be used to adjust a weighting of the various item descriptor factors for a pair of items, or otherwise adjust a visual similarity score, such that items selected as being visually similar more accurately reflect the subjective opinions of the users.