Nouns for visual objects: A hypothesis of the vision-language interface

We propose an interpretation of the vision process and a structural analysis of nouns and nominal reference which make it possible to relate the visual/cognitive and the linguistic encapsulation of objecthood in a rigorous way. The result of this integrated hypothesis is a predictive account of possible and impossible nouns lexicalizing visual objects. Visual objects are indexed relations between stimuli interpreted via visual properties, such as [round], and what we define as object concepts: a red ball is the relation between the red and spherical features and the object concept of a ball. In language, nouns identify object concepts, semantically modelled as kinds, and the noun phrases they head can refer to instances of those kinds. No aspect of grammatical structure links up to visual properties directly, so no noun in natural language can denote an arbitrary subset of visual properties; the interaction is only at the level of objects, whether an abstract concept or a fully specified referent (the latter expressed by a full noun phrase). We formalize the relation between the two by means of an infomorphism, a formal representation of information flow between systems. This translates the objects of the visual and linguistic systems in terms of information types and tokens, constraining the possible lexicalization of object concepts. For instance, a visual property cannot be identified by a choice of noun unless it is interpreted as instantiating an object concept, because nouns can denote object concepts but not directly properties. •The article proposes a novel hypothesis of the relation between nouns and visual objects.•It proposes a unified strategy to operationalize the investigation of linguistic and visual mental constructions.•It provides a model for the vision-language interface.•It explores a novel approach to concepts, and their relations to visual objects and nouns.•It offers a novel, unified account regarding recalcitrant data and problems (e.g. null objects, the gavagai conundrum).