Chroma Subsampling Explained: 4:4:4 vs 4:2:0 and When It Shows

Almost every photo on the internet is missing three quarters of its color information, and you have probably never noticed. That is not a bug. It is a deliberate trade called chroma subsampling, and for most photographic content it is close to free.

The catch is the word "most." The same trade that is invisible on a sunset is obvious on red text, on a saturated logo edge, on a UI screenshot. Knowing where the line sits is the difference between a smart saving and a visible defect nobody signed off on.

This piece explains what 4:4:4 and 4:2:0 actually mean, why the eye lets you get away with it, and how to decide which one an asset needs.

Key Takeaways

• Chroma subsampling stores full brightness detail but reduces color detail. 4:2:0 keeps one color sample for every four pixels.
• It works because human vision resolves luminance far more sharply than color, a fact codecs have exploited since the analog TV era.
• The damage concentrates in specific places: fine colored text, hard saturated edges, and interface graphics.
• Many quality metrics score luminance only, so chroma damage can pass a benchmark while looking wrong to a person.
• The right answer is per asset, not one global setting: photographs tolerate 4:2:0, graphics and text often need 4:4:4.

What is chroma subsampling?

Chroma subsampling splits an image into brightness (luma) and color (chroma), then stores fewer color samples than brightness samples. In 4:4:4 every pixel keeps its own color. In the common 4:2:0 scheme, one color sample covers a 2x2 block, so three of every four color values are discarded.

The notation looks cryptic but describes a sampling grid. The first digit is a reference block width of four pixels. The second and third digits count how many chroma samples survive in the top and bottom rows of that block (Poynton, "Chroma Subsampling Notation"). So 4:2:0 halves color resolution horizontally and vertically at once, and 4:2:2 halves it horizontally only.

Luma is left untouched in all of these schemes. Every pixel keeps its full brightness value, which is why the loss is so much less visible than a naive "throw away 75% of the data" description would suggest. For the broader question of what "quality" even means here, see why file size is the wrong metric.

Why can the eye tolerate losing color detail?

The human visual system resolves fine brightness detail far better than fine color detail. The retina carries roughly 20 times more luminance-sensitive rod cells than any single color-sensitive cone type, and the neural pathways that carry color information are lower in spatial bandwidth (Wandell, "Foundations of Vision", 1995). Compression exploits exactly that gap.

This is not a modern trick. Analog color television was engineered around the same asymmetry: broadcasters transmitted a sharp black-and-white luminance signal and squeezed color into a narrower band, so existing sets stayed compatible and no bandwidth was wasted on detail nobody could see (Poynton, "Digital Video and HD"). JPEG inherited the idea. Most JPEG encoders default to 4:2:0 for photographs.

The key point for practitioners is that this trade is tuned to a statistical average of human perception. It is not tuned to your specific image. When your content sits away from that average, the assumption breaks, which is the whole subject of the next section.

Where does 4:2:0 damage actually show?

The damage concentrates wherever meaningful detail lives in color rather than brightness. Three cases dominate: small colored text on a contrasting background, hard edges between saturated colors, and synthetic graphics like logos and UI. In these, 4:2:0 produces smearing, fringing, and a soft "bleed" around edges that looked crisp in the source.

Red is the worst offender. Because red carries relatively little luminance, a red edge lives almost entirely in the chroma channels, precisely the channels 4:2:0 halves in both directions. Fine red text on a dark background can turn muddy or develop colored halos. Designers hit this constantly with saturated brand colors, and it is a common reason a compressed logo looks subtly wrong.

Photographs mostly escape because their color transitions are gradual and their important detail is in luminance texture: skin, foliage, stone, cloth. That is why a 4:2:0 landscape looks pristine while a 4:2:0 screenshot of colored code looks broken. The failure mode is content dependent, which is the argument in why blanket compression hurts your CMS.

When should you keep 4:4:4?

Keep 4:4:4 when color carries information, not just mood. That means screenshots, diagrams, charts, slide exports, product renders on flat backgrounds, images with overlaid text, and anything with a hard saturated edge. For these, the extra bytes of full chroma are cheap insurance against a defect that is obvious to a viewer and invisible to a byte count.

Use 4:2:0 freely for natural photographs, where it is genuinely close to free. This is the ordinary case for hero images, editorial photography, and most stock. The savings are real and the perceptual cost is near zero, which is exactly the situation subsampling was designed for.

The mistake almost nobody catches is mixed content. A marketing image that is 90% photograph with a small block of colored caption text will look fine to a spot check and wrong to the reader who tries to read the caption. One global subsampling setting cannot serve both regions well, which is why the decision belongs at the asset level, not the pipeline level.

How do you even measure chroma damage?

You measure it carefully, because the default tools hide it. Peak signal to noise ratio is very often computed on the luma channel alone, which means an image can lose most of its chroma resolution and still post a strong PSNR number. A benchmark that scores luminance only will rate a smeared red edge as near perfect.

In our own testing, the assets that most embarrassed a compression setting were never the ones the headline metric flagged. They were saturated graphics that passed a luma PSNR check comfortably and then looked visibly fringed to anyone who opened them. That gap between what the number says and what the eye sees is the entire reason we treat metric choice as a design decision, not a default. The full argument sits in PSNR vs SSIM vs looks good to humans and why SSIM falls short of human vision.

The honest approach is to measure error in a way that includes color, on the kind of content where color matters, and to compare against the original rather than against a convenient proxy. That discipline is the subject of our benchmark methodology.

How does per-asset routing handle subsampling?

By deciding per asset instead of picking one scheme for everything. A photograph and a colored screenshot have different chroma needs, so a system that evaluates each asset can preserve full color where it matters and subsample where it does not. Our Neural Media Orchestrator selects the optimal path from 352 possibilities, and chroma handling is one axis of that decision.

For image mode this runs all-intra, meaning every image is coded on its own rather than predicted from neighbors, which keeps the color decision self-contained and verifiable per asset. Anything that would ship visible chroma damage does not clear the perceptual quality floor of structural similarity at or above 0.975 against the original, so it falls back rather than shipping degraded.

The result is Pareto-safe by routing: a saturated logo keeps its full color, a photograph takes the efficient path, and neither is ever larger than the strongest adaptive baseline. This is the same per-asset logic behind neural versus adaptive compression and the broader perceptual media optimization guide. It also connects to how you read the numbers, covered in rate-distortion theory for practitioners and how to read a BD-rate curve.

Frequently Asked Questions

What does 4:2:0 mean in simple terms?

4:2:0 means the image keeps full brightness detail for every pixel but stores just one color sample for each 2x2 block of pixels. Three of every four color values are discarded. Brightness stays sharp, so most photographs look unchanged despite losing most of their color resolution.

Is 4:4:4 always better than 4:2:0?

No. 4:4:4 preserves all color detail but costs more bytes, and on natural photographs the extra detail is invisible to human vision. 4:4:4 is worth it for text, graphics, screenshots, and saturated edges. For editorial photography, 4:2:0 delivers the same perceived quality at a smaller size.

Why does red look worse under chroma subsampling?

Red carries little luminance, so a red edge lives almost entirely in the color channels that 4:2:0 halves in both directions. With brightness contributing little to define the edge, the reduced color resolution shows directly as fringing, smearing, or muddy halos around fine red text and saturated red shapes.

Does chroma subsampling affect PNG images?

No. PNG stores full RGB color per pixel and does not subsample chroma, which is one reason PNG suits graphics, logos, and text. Chroma subsampling is a feature of formats built on luma and chroma separation, such as JPEG and most video codecs, where 4:2:0 is common.

How do I know if subsampling damaged my image?

Look at fine colored text, saturated edges, and thin colored lines at full resolution, not at the whole image thumbnail. Damage appears as color bleeding beyond edges or muddy text. Beware luma-only metrics like standard PSNR, which can rate a chroma-damaged image as near perfect.

The point

Chroma subsampling is one of the oldest and best trades in imaging: give up color detail the eye cannot resolve, keep the brightness detail it can, and pay far fewer bytes for it. On photographs it is close to a free lunch.

The failure is treating it as a universal setting. The same 4:2:0 that is invisible on a landscape is a visible defect on red text and brand graphics, and a luma-only metric will wave both through. The fix is not a better global default. It is deciding per asset, measuring color where color matters, and refusing to ship anything that cannot clear the bar. That is the discipline behind the perceptual optimization guide.