ACR 4.3.1 hasn’t solved A700 high ISO mush problems

After reading Andrea Nivini’s article in Italian Tutti Fotografi, December 2007, which launches an attack on Adobe’s Camera Raw plug-in and its handling of many camera types – but specifically, the Sony Alpha 700 – I decided to check out whether the December 5th release, ACR 4.3.1, fixed the problems.

The short answer is that it doesn’t. Nivini uses examples from the source Bayer-pattern decoder used by nearly all raw converters, Dave Coffin’s donation to the world – DCRaw. Coffin not only pioneered open source raw file conversion, but has continued on a voluntary basis to add every new camera just as soon as possible. DCRaw includes several different algorithms for extracting an RGB image from the mosaic of RGRGRGRGRG, BGBGBGBGBG monochromatic pixels which make up a raw Bayer pattern sensor image file.

Nivini argues that Adobe often select entirely the wrong type of conversion for the camera involved, and sometimes will add a new camera just by changing a few lines of code from the previous model; ACR can be made to convert unknown models of camera (such as those with a European name embedded, Dynax instead of Maxxum, and so on) just by changing the name in the program’s code. It seems that if Adobe then encounter unexpectedly crude conversion, instead of trying to establish the best algorithm for extracting detail from the file cleanly, they alter the hidden presets for noise reduction, colour smoothing, sharpening and contrast masking so that the result is not immediately visible as a total disaster.

Except, of course, if the camera itself produces JPEGs which are apparently far superior to the hashed up Adobe ACR result. Then users begin to get critical. Raw development normally beats an in-camera JPEG on all counts, especially detail extraction and sharpness, ideally on noise control and colour as well. From the start, users have found the Fine quality JPEG from the Sony Alpha 700 is better than the ACR processed raw at any ISO setting over 400. It seems that the programmers have got the DCRaw choices wrong (even Adobe rely on Dave Coffin’s free expertise). This was known to be the case with the Olympus E-3, where the chosen algorithm created ‘maze-pattern’ grain resembling a parquet floor, and version 4.3.1 was released rapidly to pacify Olympus.

Maze patterns were not the issue with the Alpha 700. Instead, we get coarse high ISO ‘grain’ with large clouds of smoothed colour retaining its RGB separation at high saturation; fine edge detail is destroyed even at default (25) sharpening, and the overall result increasingly resembles a 3 megapixel consumer camera file as you go beyond 800 ISO. No fine pixel-for-pixel image information is retained to allow post processing. Even with all NR and sharpening turned to minimum, detail is reduced to what appear to be blocks of 4×4 or even 8×8 pixels. The point of the 12 megapixel DSLR is lost. This does not happen with the Nikon D300, which uses a nearly identical IMX021 sensor from Sony. It’s almost as if Adobe is deliberately treating the two raw formats differently when the Sony files should be processed just as for the Nikon data.

This effect starts around ISO 800 and is extreme by ISO 3200. However, the subtle differences are more easily shown at ISO 800, a speed which with a modern DSLR should be 100 per cent usable. It is with the Olympus E-3, Nikon D300 and Canon 40D.

Here is a quick night-time snap at ISO 800, Christmas lights in Faro city centre, Portugal, after a day of torrential rain and flooding (we do pick the right times to go for a couple of days!):

It’s a hand-held 1/40th at f2, the full aperture of the 28mm f2 Minolta AF lens. Focus in on the baroque building beyond the car. This result is as processed using RawDeveloper, a low-cost specialised raw processing program from Iridient, downloadable as a demo from http://www.iridientdigital.com/products/rawdeveloper_download.html. It is a Mac-only program, but offers a comprehensive set of NR and sharpening controls along with a completely different algorithm for image extraction, compared to Adobe Camera Raw. You do not have access to specific variants, as with some other programs. Here is the result using Sony A700 default colour profile and settings, with ‘Smooth Look’ enabled for the image noise, and noise reduction and sharpening auto-set by RawDeveloper:

This is a 100 per cent clip of the image. Why do I pick such a dark, murky section? Easy – that’s where noise appears most. Blue sky is also one of the worst colours for showing noise patterns. And you will notice there are some sharp details, top left and bottom right, which help define the detail extraction of the program. That 28mm lens at full bore with SSS and 1/45th has no problems… It also has some very low contrast detail in the palm leaves. This is the type of information easily lost for good with a bad conversion.

Now take a look at the Adobe Camera Raw 4.3.1 equivalent:

While the differences are subtle and in some ways ACR does a good job, there is a certain look to the structure of the ACR image which does not go down well. To see better what is happening, I used another Mac-only utility, Andrey Tverdokhleb’s RawPhotoProcessor, which is donationware. See http://www.raw-photo-processor.com. This program does not use camera colour profiles, so it produces very flat looking colours initially. But it allows the choice of different interpolation methods (deconvolution from Bayer) including AHD, which the DCRaw option Andrea Nivini chose to show the potential of the Alpha 700 better. I used RawPhotoProcessor with AHD to get to an almost ‘pure’ version of the detail:

Here, you begin to see pixel by pixel detail (along with pixel by pixel noise, of course) and the absence of the coarse coloured graininess which affects RawDeveloper and ACR. It is still far from perfect, and from Nivini’s article, it looks as if the DCRaw ‘option -f’ conversion produces something a little more like the Nikon D300 ACR conversion, less scratchy in structure and more globular for the noise grains.

The options provided by RawPhotoProcessor for the Mac can be found in RawTherapee for PC, which is a similarly independent donationware venture – see http://www.rawtherapee.com. You can pick an ‘EAHD’ conversion; if I took the trouble to find out exactly what all these acronyms mean, I’d probably end up with a brain as scrambled as a bad raw conversion, but they are there to ‘suck and see’ and I have no doubt at all that the Sony Alpha 700 is capable of producing high ISO results which are not softened by Adobe ACR.

There are several other raw developer options. Here is the result from LightZone, a multiplatform program from LightCrafts Inc which is fully featured but crashes consistently on my Mac when in the company of any other programs:

Overall this is not too bad, very clean noise level on default settings, and reasonable muddy-contrast dark detail rendering. A similar result is achieved using the latest version 3.0 of SilkyPix, a Japanese program which is supposed to have been used in part or all by some of the own-software packages for camera makers. Here, I have used the option which they call ‘Pure Detail’, a special noise reduction and sharpening setting, rather than the Default ‘Normal’:

This is impressive for the smoothness of the sky noise, visibility of the palm fronds, and overall definition without excessive grainy noise or disruption. There is a hint of false detail creation present in the hardest edges, like the finial on top of the glass cupola; this is not an entirely artefact-free conversion choice. But it looks good. If SilkyPix can do it, why can’t Adobe?

However, I would ask you to step back from the ACR result and consider its microcontrast. Despite the smudging of some detail, Adobe does have some expertise which allows hard-to-see edges and information to come through clearly. Mainly, this is because ACR opens up the lower midtones greatly. It applies a brightness, by default, which reveals more in most pictures (and also leads to normal Sony A700 images appearing over-exposed, the other big complaint levelled at ACR). In some ways the ACR default images is the best. When printed, it certainly works well. The question is whether this enhancement of detail tonally could be accomplished without the coarsening of the grain/noise structure.

Using Raw Photo Processor, it is possible to choose between two different conversion algorithms:

On the left is the method called ‘VNG’, and on the right, ‘AHD’. Since no noise reduction is possible in RPP apart from ‘Chroma Blur’ and no sharpening apart from ‘Local Contrast’ the results both represent almost unprocessed image data, compared to Adobe Camera Raw.

Does Sony come close? With the default settings of the v2 firmware/software upgraded Image Data Converter SR v2, here is the result:

While the grain structure is good, the softening created in low contrast details is enough to remove those palm leaves from sharp view. It is possible to use the somewhat obscure controls of IDC SR to enhance this a little, and turning off the NR entirely does some good, but you are not going to be peeking into shadows and seeing stray hairs on the carpet with this converter. Does the default really match the in-camera settings (Standard look, sRGB, all settings at 0 or Normal/Medium including High ISO NR)? Here’s the same clip as processed by the camera – in-camera FINE JPEG:

There’s more colour noise, less luminance noise, and better definition of those difficult dark low contrast details. Overall it’s not a very clean output but you might well decide that the in-camera JPEG is still the best compromise conversion despite some of the odd effects, like the watermarks in the sky.

We have not yet seen what can be done with Alpha 700 raw files, because somewhere out there is a combination of de-Bayer routine, colour profile, tone curve, local contrast enhancement or masking, sharpening and noise reduction which combines the best aspects of all these conversions. I have not used LightRoom for this test, since it is functionally identically to ACR 4.3.1 in raw conversion when set to similar defaults. I have been unable to use Capture One v4 as it does not support the A700 (at least in the beta 4 version I have right now), same goes for DxO Optics Pro, same goes for Apple’s Aperture program, all of which I have. Nor have I tested Bibble, which has consistently proved buggy and prone to unwanted output artefacts (like tartan stripes) when resizing images. I do not generally pay for the programs I use, if I did I would be bankrupt as I have to use as many as possible for review purposes – Adobe, Nikon, Canon, Apple, DxO, Phase One and others send press NFR version or provide unlock keys. Bibble Labs has never been able to do so, nor has SilkyPix; in the case of SilkyPix, each new version allows a trial period. This is sufficient for my purposes. In the case of Bibble, one trial period of an early version prevents the use of later versions for ever (there may be some preference or file left behind on the Mac which can fix this, but I do not have the time to find it).

There is also the possible issue of differences in behaviour between Mac and PC. We do use PCs at Icon Publications Ltd, but generally don’t have duplicate PC/Mac versions of the same software unless that is a standard issue by the makers on their installation CD. Nor do we have the identical graphics card/identical monitor/shared calibrator situation really necessary to make comparisons. So, for the majority of you, my apologies – this report is Mac based, and does omit some PC-only programs like ACDSee, BreezeBrowser Pro, Picasa, and Raw Therapee which may produce different results.

I also apologise for picking such a difficult image to view under bad conditions. If your room lighting is excessive or your monitor not well calibrated, it will be very difficult to assess. But that is also exactly the type of image which says most about raw file processing and noise levels.

‘Could do better’ is the end of term report of Adobe Camera Raw. With Adobe’s muscle behind it, it should be the best raw converter of all. I would hesitate to abandon it because it’s the fastest, most intuitive single or small batch processor where individual adjustments are needed picture by picture, and Photoshop will be used for further editing. But I’m worried about the opinions it gives me about my Sony Alpha 700, including an unwillingness to use the camera at settings over ISO 800 under any conditions, where I found the Nikon D300 to be acceptable.

Since this article was written, I have posted further raw conversions at different high ISO settings using different converters. My conclusions are mixed; it was Andrea Nivini’s article which prompted this chain of investigation, but since then (writing here in February) many others have become involved in researching the problems with ACR.

See: http://photoclubalpha.com/2008/02/21/a700-6400-iso-seven-raw-processors/

and http://photoclubalpha.com/2008/02/12/apple-aperture-20-processes-a700-raw/

and http://photoclubalpha.com/2008/02/08/capture-one-v4-cures-a700-high-iso-confetti/

– David Kilpatrick