http://www.majid.info/mylos/weblog/2004/05/02-1.htmlIs the Nikon D70 NEF (RAW) format truly lossless?
Many digital photographers (including myself) prefer shooting in so-called RAW mode. In theory, the camera saves the data exactly as it is read off the sensor, in a proprietary format that can later be processed on a PC or Mac to extract every last drop of performance, dynamic range and detail from the captured image, something the embedded processor on board the camera is hard-pressed to do when it is trying to cook the raw data into a JPEG file in real time.
The debate rages between proponents of JPEG and RAW workflows. What it really reflects is two different approaches to photography, both equally valid.
For people who favor JPEG, the creative moment is when you press the shutter release, and they would rather be out shooting more images than slaving in a darkroom or in front of a computer doing post-processing. This was Henri Cartier-Bresson's philosophy — he was notoriously ignorant of the details of photographic printing, preferring to rely on a trusted master printmaker. This group also includes professionals like wedding photographers or photojournalists for whom the productivity of a streamlined workflow is an economic necessity (even though the overhead of a RAW workflow diminishes with the right software, it is still there).
Advocates of RAW tend to be perfectionists, almost to the point of becoming image control freaks. In the age of film, they would spend long hours in the darkroom getting their prints just right. This is the approach of Ansel Adams, who used every trick in the book (he invented quite a few of them, like the Zone System) to obtain the creative results he wanted. In his later days, he reprinted many of his most famous photographs in ways that made them darker and filled with foreboding. For RAW aficionados, the RAW file is the negative, and the finished output file, which could well be a JPEG file, the equivalent of a print.
Implicit is the assumption that the RAW file is pristine and have not been tampered with, unlike JPEGs that had post-processing such as white balance or Bayer interpolation applied to them, and certainly no lossy compression. This is why the debate can get emotional when a controversy erupts, such as whether a specific camera's RAW format is lossless or not.
The new Nikon D70's predecessor, the D100, had the option of using uncompressed or compressed NEFs. Uncompressed NEFs were about 10MB in size, compressed NEF between 4.5MB and 6MB. In comparison, the Canon 10D lossless CRW format images are around 6MB to 6.5MB in size. In practice, compressed NEFs were not an option as they were simply too slow (the camera would lock up for 20 seconds or so while compressing).
The D70 only offers compressed NEFs as an option, but mercifully they have improved the performance. Ken Rockwell asserts D70 compressed NEFs are lossless, while Thom Hogan claims:
Leaving off Uncompressed NEF is potentially significant--we've been limited in our ability to post process highlight detail, since some of it is destroyed in compression.
To find out which one is correct, I read the C language source code for Dave Coffin's excellent reverse-engineered, open-source RAW converter, dcraw, which supports the D70. The camera has a 12-bit analog to digital converter (ADC) that digitizes the analog signal coming out of the Sony ICX413AQ CCD sensor. In theory a 12-bit sensor should yield up to 212 = 4096 possible values, but the RAW conversion reduces these 4096 values into 683 by applying a quantization curve. These 683 values are then encoded using a variable number of bits (1 to 10) with a tree structure similar to the lossless Huffmann or Lempel-Ziv compression schemes used by programs like ZIP.
The decoding curve is embedded in the NEF file (and could thus be changed by a firmware upgrade without having to change NEF converters), I used a D70 NEF file made available by Uwe Stei
