virtual {mo}ritz

Several blogs and mailing lists I frequent linked to an article investigating the validity of Apple’s claims that the iPhone 4′s display has a ‘terminal’ resolution for the application at hand: namely a human reading its display at an ‘average’ distance (which, being unspecified in Apple’s press release, offers quite a bit of latitude for interpretation).

The article ignores several important facts.

From my humble outlook of someone with a background in print (from another life), graphics design with emphasis on typography (a long time ago) and signal processing theory (working in visual effects for 15 years) this misses the point as it ignores context. It solely limits its view to the the projected size of that pixel, on the retina, at the average reading distance. But something people — and geeks specifically — tend to overlook: Math ≠ reality.

For the sake of making a point, I will limit my view to type (text, not images) being displayed on the iPhone 4′s screen.

As Apple’s claim is actually not too far off for images I will briefly touch on rasterization of such mostly continuous data to (hopefully) make it obvious why I think it doesn’t hold well for discrete data (type) and, therefore, in general.

Some Facts

Common computer-to-film and computer-to-plate techniques in printing use 2,400dpi and up. Actually, 4,800dpi are not uncommon these days. 4,800dpi are really fucking crisp™. Consider that plates used in chemical milling tiny mechanical parts use less than three times as much — about 8,000dpi.

For type (not images), 2,400–4,800dpi is the actual resolution at which it ends up rasterized on paper (and ignoring any defects, loss of accuracy or even local variance in resolution that the final process of applying the ink to the paper may introduce).

For images (not type), the input signal’s resolution is usually around 300–400ppi. The sampling resolution (called lpi) is around half of that (sampling theorem — if you wonder about the mathematical relationship between ppi, dpi and lpi I suggest reading an old and badly written post of mine about that very subject).

The reason is screening. The number of levels that can be displayed with a common print screening process are (dpi÷lpi)².

So if we take a 300ppi image printed at 150lpi on a 2,400dpi device, screening can produce (2,400÷150)²=256 levels. Curiously this is the reason 8bit images are fine for final output in print, most of the time.

Again, this is images, not type.

Resolutions Used in Print

As mentioned above, type is rasterized at the full dpi of the device. Read: ≧2,400dpi. People often get confused over this because ‘glossy magazines’ are supposedly “printed at 300dpi“. This is bollocks™ of course. The magazine is printed at ≧2.400dpi and the images fed into the printing device are ≧300ppi, for above reasons. Type, however, is printed with ≧2,400dpi.

If you wanted to send the printing device images that have the type burned in, the image would need to be ≧2.400ppi. This is required to get the same crispness as sending the type to the rasterizer inside the printing device. That is the reason why type is commonly sent separated from images, as curves (or ‘vector graphics’) rather than pre-rasterized, as pixels.

Now does this actually make a difference? It does indeed. Look at small type (text sizes) printed by a 300dpi laser printer and then look at some from one with 600dpi. The 300dpi simply does not look not as crisp and as smooth.

I dare say I can even tell the difference between a 600dpi and 1,200dpi laser printout. Many people working in graphic design/typography will be able to. And it actually makes sense.

A good use case to make the point is the colouring of graphical novels (comic books). This is mostly done in Photoshop these days. The thing to note is that this process is done at ≧1,000dpi. Now recall the 300–400dpi commonly used for images mentioned above? The panels in most graphical novels are not images in the traditional sense. They are what professionals oftern refer to as ‘line art’. Line art requires crisp contours and edges to look good.

If a little bit over 300dpi would be the resolution the human eye can resolve at reading distance, why would someone go through the hoop of editing images at roughly nine times the resolution? After all that does require nine times as much memory. And memory is always scarce in Photoshop (because of its completely lame and destructive way of applying a user’s input to an image at the full resolution all the time, but that’s a rant for another post). And finally the resulting files will be larger too; having implications for the systems used to process them before and during printing.

So why do a little bit over 300ppi not suffice for anything that should look really crisp? There are several reasons.

Mapping of Pixels to Photoreceptors

Firstly, there is no 1:1 mapping between photoreceptors in a user’s retina and pixels on her iPhone’s display. The iPhone is not (yet!) ‘connected’ to the retina.

This means that the projection of the iPhone’s display onto the retina may, for example, map the tiny gap between two individual pixels on the display to the center of a photoreceptor (and not to the gap between photoreceptors, as one would prefer, if one worked for Apple’s marketing department). Pixels are arranged in a rectangular grid, photoreceptors aren’t. You get the idea.

For type this means that while you may not be able to tell why it doesn’t look as smooth & crisp (i.e. spot the individual jaggies that cause this) this doesn’t follow that your brain’s image processing isn’t good enough for you to not spot the difference at all. This is because we do have sharpening filters build into our visual postprocessing ‘wetware’.

Any discontinuity in the input signal (the image you see) will be exaggerated once the information enters the visual cortex. So if a jaggy ends up being picked up, it will be exaggerated by the way your brain works. As for projection: at 300ppi the probability that a jaggy is picked up by your visual system is 4 times as high as at 600ppi. And so on. This is the reason type does look crisper and ‘higher quality’ at higher resolutions even though these resolutions, in theory, exceed that of an average human’s retina at the projected reading distance.

Another thing that must be considered for type is the fact that a letter will not only have its outline undergo discretization but also its stroke width, as a direct result of that.

Enter your brain’s shape recognition software. Again, while some people may not be able to spot the jaggies in the outline, most people can tell that the letters don’t look as ‘smooth’ because the stroke width jumps in steps that are up to four times more obvious at 300ppi than they are at 600ppi (600÷300)². The most likely reason is that certain shapes the discretization yields will be extrapolated to slightly different shapes by your brain’s ‘wetware’.

This is the reason printing does (and always has been) making use of the resp. available process’s full resolution, for rasterizing any type.

In the early days, this did make a difference for technology because rasterization was slow(er) (CPU & memory access) and memory was expensive.

Think about it: if it was true that 300ppi were enough, why did printer manufacturers ever bother to produce b&w laser printers with more than 600dpi resolution? Most people using b&w laser printers do not print (grayscale) images but text. So the resolution available for screening (dpi vs lpi) is completely 2ndary. If Apple’s claims were true, this would make no sense whatsoever. And this includes Apple’s own (discontinued) range of LaserWriter printers.

Since we’re on a non-monochromatic display on the iPhone, using filtering (antialiasing) can be used to aid the text rendering.

This is of course not available in monochromatic printing so this does make my comparison slightly contestable. But it does not invalidate the very first point, namely the mapping of the rectangular laid out pixel grid to the (Poisson–like) packing of photoreceptors.

Antialiasing requires a careful choice of reconstruction filter. A filtering method that works fine for one typeface may very well destroy the typographer’s work for another. There is a reason Photoshop offers four different filtering modes for antialiased type. So while this sounds like this may a be a point for Apple, what I said above about stroke width modulation due to discretization in general make it a dubious one. Additionally antialiasing can’t help much once a letter’s feature gets smaller that a pixel in itself (e.g. a serif or, worse, an accent) the improvement it yields will be hardly quantize-able for a user anymore. For antialiasing to ‘work’ you need connected shapes of antialiased pixels.

Lastly it doesn’t change the fact that Apple’s claim, even if it may hold truth in specific circumstances (photographic images), is, in general, refutable.

And I am happy to bet a large sum of money that we will see mobile device displays with much higher resolutions than the iPhone 4′s very soon.

Again, if Apple’s marketing were speaking the truth, this would never happen as it made no sense. Lets see what resolution the iPhone 5′s screen will have and what fancy name Apple will give it (“hubble display”?).

---

P.S.: it doesn’t help that Apple, in general, does not use per-color filter offsets (aka ‘Clear Type’) for text rendering. This technique essentially triples the ‘visual improvement’ that filtering brings on one axis on displays with resp. sub-pixel layouts.

I always notice that when I switch between OSX and Linux on my LCD. Linux’ type just looks thrice as crisp (yet not at all jaggy). OSX’ looks almost blurry after a few hours in Linux. It is most apparent when I run OSX in a VirtualBox on a Linux desktop — or vice versa.

---

P.P.S: This is an edited reply to a thread on the very subject matter on the 3D-Pro mailing list, on the same day the article, linked at the beginning, was published.