I should note at the outset that this isn't a tutorial. I don't go into detail about software, file formats, etc.; these are general remarks on a process of creating a conventional illustration with a vintage microcomputer as a target medium.
START SMALL · STAY SMALL
When your target platforms have low display resolution, fine lines and textures will distort and vanish beneath the Nyquist frequency* of these graphics systems, so begin with that limitation in mind and start small.
I began an image with a pencil sketch, about three inches wide.
After digitizing the sketch, I scaled it down to a low working resolution. My active area was 640×480 pixels* – good ol' VGA resolution. This is higher than the resolution for both of my target systems, but not by a large amount. I don't want to waste effort creating image detail that will be lost when I scale/interpolate/dither to lower resolution/depth.
From there, I produced some nice, crisp line-art. At another time I could discuss tonal artwork but penmanship is my forté and my chief interest, so I went down that road for this test.
Before discussing color, here are the target platform/mode specs:
Atari 800/XL/XE, ANTIC mode E: 160×192, 4 colors
Atari ST, Medium resolution: 640×200, 4 colors
I don't own an ST, but I wanted to include it in the tests because it's also a four-color mode with a non-square pixel aspect ratio; it's got quadruple the horizontal resolution of the XL, but the vertical rez is the same.
As these are both four-color modes, doing any work with a lot of local color is out of the question. You can do nice monochromatic shading with four value levels though.
I took two stabs at colors: first, a flat-shaded (cel-shaded, in some circles) four-color approach (black, white, and two gray levels)—
Then, a more tonal version with smooth gradations of value.
I took two approaches because I was curious, especially in the case of the Atari XL version, about the extent to which tonal dithering would be pleasing to look at. My initial assumption was that in very low resolutions (that is, with very large pixels) dithering restricted to areas of high contrast – edges and edge-like transitions, which is what you'd get with a flat-shaded input image – would be better-looking than the spray of dots you'd get from dithering a tonal piece.
WITHERING DITHERING
Now, when de-rezzing these things to get them to their target dimensions/depths, the obvious first step is to do a full-color rescale to your target dimensions using a nice interpolation algorithm. Next, you've got to do the color conversion.
You can either take the hard-and-flat palette-mapping approach (something like: calculate n average values of all the colors in the image, then change each pixel's color to whichever of those averages it most closely matches), or use some dithering (something like: find the darkest and lightest colors in the image, and n in-between colors, then use an error-diffusion algorithm to generate a kind-of halftone pattern for the image). You can certainly get interesting results with the former, but images tend to appear more flat. Dithering creates noise, but it gives the appearance of greater dynamic range. In my professional (high-res, print) work I enjoy the effects you can generate with posterization/palette-mapping, but in very limited graphics systems like these, I prefer dithering.
The trouble with dithering is that it can produce such wildly different results depending on the value range of the input image. Worse, the parameters of the dithering process in most programs are totally out of your control, and, in the case of flat-shaded images especially, you can experience the terribly irritating situation of the dithering algorithm failing to identify any dominant colors in your image, thus not using them as your palette, so that flat areas of color are unnecessarily dithered.
A few years ago I wrote a handy tool for interactive "tuning" of a monochrome Floyd-Steinberg dither; something like that would have been really useful here, but I never wrote a color version of that tool. Alas, I resorted to doing indexed-color mode conversions and color curve manipulations in GIMP, and it was not an enjoyable process. I need to write that tool.
Nevertheless, I found my way to some adequate 4-color output images.
Now, keeping things strictly monochromatic – that is, grayscale – is not at all necessary. The important thing with tweaking the dither is simply that you get the image down to your target color depth with the pixels in a good-looking distribution pattern. The specific colors used, though, is where you can zazz things up a bit.
I used Tebe/Mad Team's Graph2Font to manipulate the colors for the XL version and produce an .xex file so that I could run it on my Atari. Here's the output image, and a photo of it displaying on my machine which, at the moment, has a monochrome monitor. I know, I know; why have an Atari – the most colorful home computer of its time – without a color monitor? Well, because aperture grilles are stupid, and these little 5-inch monochrome screens are nice and crisp.
I was wrong, as it turned out, about the flat-shaded art looking better on the XL. On the contrary, I preferred the flat-shading on the higher-resolution system, the ST. That style of shading is all about crispness and clarity of line, and the 8-bit Atari simply doesn't have the resolution to suit it. Here are some ST-medium versions of the image, flat-shaded and tonal:
What did I learn? Not much, as this was a path I'd already trod somewhat. I initially explored a similar workflow a few years ago to produce a couple of small pieces. Those pieces, however, were tight little pencil drawings that I only cleaned-up on the computer. This experiment was closer to my typical illustration process, and confirmed for me that nicely anti-aliased VGA line art with a bit of tinting can convert respectably to ANTIC mode E. What about the ST? I figured it would look fine, and it does.
Time, then, to start working on my Atari 8-bit visual novel!