Color Constancy and The Dress: Why Two People See Different Colors

Your visual system does not report the light in your eye. It reports its best guess about the surface, and the guess needs an assumption.

Short answer: the light reaching your eye from a surface is the surface's reflectance multiplied by the illumination, and your visual system has to separate those two from a single measurement. It cannot do that without an assumption about the light. Most of the time the scene supplies enough cues that everyone lands on the same answer. The famous dress photograph supplied almost none, so different observers used different assumptions and arrived at genuinely different, genuinely stable percepts. Nobody in that argument was seeing it wrongly.

The problem constancy solves

Consider a sheet of white paper. Outdoors on a clear day it is lit by a mixture of direct sun and blue skylight; indoors it may be lit by a warm lamp. The spectral composition of the light coming off that paper differs enormously between those two situations, and so does its intensity, by orders of magnitude. Yet the paper looks white in both, and a lemon next to it looks yellow in both.

That is color constancy, and it is not a minor correction. It is closer to the main job. A visual system that reported the light arriving at the eye would be a photometer, and it would be nearly useless for recognising objects, because the same object would produce a different reading every hour of the day. What you want to know is what the surface is, and that requires discounting the illuminant.

The catch is that the problem is underdetermined. One measurement, two unknowns. The visual system solves it by exploiting regularities: the average color of a scene, the brightest surface in view, specular highlights that carry the illuminant's color directly, mutual reflections between surfaces, and the statistics of natural daylight. A comprehensive review of the field lays out how many partial cues are involved and how far constancy actually gets, which is good but well short of perfect.

Why the dress broke it

In February 2015 a badly exposed photograph of a striped dress produced an argument that ran round the world. The image is unusual in a specific and interesting way: the lighting is ambiguous, the background is blown out, there is no recognisable object of known color anywhere in the frame, and the pixel colors of the dress lie close to the axis along which natural daylight itself varies, from bluish shade to yellowish direct sun.

That last point is the crux. Because the visual system routinely discounts variation along the daylight axis, an image whose colors sit on that axis is exactly the case where the discounting becomes a guess. Assume the dress is in bluish shade and your visual system subtracts blue, leaving white and gold. Assume it is under warm artificial light and it subtracts yellow, leaving blue and black. Both operations are the normal machinery working correctly on an image that does not constrain the answer.

The empirical work supports that reading. A free-response survey of 1401 respondents found three modal groups rather than two, blue and black, white and gold, and blue and brown, and reported that white and gold answers were more common among older respondents and among women. Experimental work manipulating observers' implicit assumptions about the illumination found that those assumptions predicted the reported colors, and concluded that implicit illumination assumptions are the most reasonable explanation available. A separate study of more than 13,000 respondents found that self-reported early risers were significantly more likely to see white and gold than late risers, which fits the idea that lifetime exposure to daylight rather than artificial light shifts the prior. That last result is correlational and rests on self-report, so treat it as a suggestive pattern across a large sample rather than a fact about you.

What the disagreement actually shows

The deeper lesson is one that runs through this whole site: perception is inference, not measurement. Two observers with identical retinas can arrive at different answers from the same input if they bring different assumptions, and neither of them is choosing. This is the same reason a patch changes appearance when you change its surround, described in simultaneous contrast, and the same reason your memory for a color drifts, described in color working memory.

Constancy in the games, and why we nag about lighting

Every color you see in a game here is emitted light, and your visual system will still try to discount an illuminant. It will use your screen's white point, the light in your room, and whatever you were looking at just before. Three consequences follow.

Your adaptation state is part of the stimulus. Play after twenty minutes on a page with a warm background and the first few trials will be shifted. This is why we suggest a neutral background and a short pause before starting.

Automatic warm-tint modes are a moving target. Night-shift features change the display white point continuously through the evening. Your visual system partially compensates, but only partially, and the compensation depends on how much of your field of view is the screen.

Warm and cool judgements are constancy judgements. Huember asks whether one color is warmer or cooler than another, and that question is unavoidably relative to what your visual system currently treats as neutral. Our page on color temperature and warm versus cool covers why that intuition is more slippery than it looks.

Huekle itself is less exposed, because the target and your guesses are on screen together, but the surround still matters. If you want your daily numbers to be comparable across weeks, hold the conditions steady, exactly as our guide to recording and evaluating your scores sets out.

How to use ambiguity deliberately

There is a practical skill hiding in all of this. When you need to judge a color accurately, you can reduce the number of assumptions your visual system has to make.

Put a known neutral in view. A grey card, or a patch of known white, gives the visual system an anchor and removes most of the guessing.

Compare, do not judge alone. Two colors side by side under the same light is a far more reliable comparison than one color judged against your idea of what it should be.

Change the light and look again. If a color reads differently under two light sources, that difference is information about the surface, not a failure.

Beware of the memory route. Once you look away, you are no longer doing constancy, you are doing recall, which has its own biases described in color memory and recall.

Keep reading

• Color temperature, warm and cool: why the warm and cool intuition does not match the physics.
• Simultaneous contrast: the local version of the same inference.
• How eyes see color: the three signals everything above is built on.

Sources: Foster, D. H. (2011), Color constancy, Vision Research 51(7), 674; Lafer-Sousa, R., Hermann, K. L. and Conway, B. R. (2015), Striking individual differences in color perception uncovered by 'the dress' photograph, Current Biology 25(13), R545; Brainard, D. H. and Hurlbert, A. C. (2015), Colour vision: understanding #TheDress, Current Biology 25(13), R551; Witzel, C., Racey, C. and O'Regan, J. K. (2017), The most reasonable explanation of "the dress": implicit assumptions about illumination, Journal of Vision 17(2), 1; Wallisch, P. (2017), Illumination assumptions account for individual differences in the perceptual interpretation of a profoundly ambiguous stimulus in the color domain: "The dress", Journal of Vision 17(4), 5.