Electronic device with polarized ambient light sensor

An electronic device may be provided with an ambient light sensor. The ambient light sensor may be a color ambient light sensor or a monochrome ambient light sensor. The electronic device may have a light-emitting component such as a display. During operation of the display, the display emits light. To reduce noise due to the emitted light while measuring ambient light, the ambient light sensor may have optical structures such as wave plates and polarizers. Theses optical structures may overlap light detectors. The optical structures may be configured to prevent ambient light from reaching a first of the light detectors while allowing ambient light to reach a second of the light detectors. The ambient light sensor may be configured to receive ambient light that has passed thorough an inactive area of a display or that has passed through a pixel array in an active area of a display.