Secure Computing: Sec-C

In my previous blog posting, I mentioned how some people really do "get it" when it comes to digital manipulation and photo fakery. However, others like "photographer" Nicholas Routzen and BP's Marc Morrison still don't understand why representing modified photos as if they were "real" is nothing other than fraud.

BP was heavily criticized in the media for releasing edited photos. In fact on 22-July-2010, White House Press Secretary Robert Gibbs even commented that it was sheer stupidity:

"I think it's genuinely on the stupidity part of the transparency scale," Gibbs said this afternoon at the White House daily briefing. "I mean, if you want to show a picture of what the room looks like, just take a picture."

Upon the discovery of BP's digital manipulation, BP decided to come clean. Sort of. It was actually more of a "throw the photographer under the bus" than an actual correction:

BP cast the blame entirely on a hired photographer and claimed to have no part in the decision to alter the photos. "One of BP's contract photographers used Photoshop to edit images posted on the bp.com Gulf of Mexico Response web site," the company said, adding, "[W]e've instructed the photographer who created the images to refrain from cutting-and-pasting in the future and to adhere to standard photo journalistic best practices."

Too bad this isn't an isolated incident... and it still has not stopped.

As part of their corrections, BP created a special Flickr set where they show the before and after photos of the three pictures that America Blog and Gizmodo identified as modified. However, BP is only showing the three outed photos.

Standard is Better than Better

I really like that phrase, "Standard Photo Journalistic Best Practices". There is no such standard. As I detailed last year, different organizations have different rules about acceptable manipulation. However, there are some generalizations that can be made.

For Photographers

In general, if the photo is supposed to represent something real then the person providing the photo to the media should abide by these guidelines (a combination of rules from Reuters, Associated Press, Getty Images, and other photo providers including China's Xinhua news agency):

• No splicing, no drawing. Whether it is for removal or enhancement does not matter. You never splice images and you never alter content. If Billy blinked during the photo, then the photo must have Billy's eyes closed -- don't draw in eyes or splice them from a different photo.
  
• Minor cropping. A little cropping from an edge ("little" as in "up to 5% from an edge") is acceptable if it does not remove a subject from the image. Major cropping, such as Morrison's removal of the entire upper half of the photo, chairs, and two people (before and after) is not permitted.
  
• Minor dust and speck removal. Minor dust and speck removal from a non-critical section of the photo is generally permitted. However, this really depends on the photo provider. Some providers (like the AP) are more critical than others. In general, if there is a tiny speck of dust on the lens that ends up looking like a distant UFO in the sky, then you can remove it. But if there are lots of specks, then you must suffer with a dirty picture (next time, clean your lens!). And if the speck is located on the subject matter (like Hillary Clinton's shoulder), then don't touch it!
  
• Minor color enhancements. Color corrections that do not alter the subject are permitted. As the AP described:
  

  Minor adjustments in PhotoShop are acceptable. These include cropping, dodging and burning, conversion into grayscale, and normal toning and color adjustments that should be limited to those minimally necessary for clear and accurate reproduction (analogous to the burning and dodging often used in darkroom processing of images) and that restore the authentic nature of the photograph. Changes in density, contrast, color and saturation levels that substantially alter the original scene are not acceptable. Backgrounds should not be digitally blurred or eliminated by burning down or by aggressive toning.

  
  In general, any acceptable, minor color enhancements should be equally applied over the entire image and not isolated to a specific region. Highly focused or region-specific color alterations are no different than drawing. ("Select all" or "select none", but if you touch the selection tool or magic wand then you are drawing.)
  
• After effects. Blur, sharpen, smudge, liquify, rotate, blend, and other enhancements are not permitted. Basically, if it is not found in the real picture then this is considered "drawing".
  
• Acceptable drawing. There are only two situations where drawing, such as blurring or using a black censor box, are permitted: (1) to protect anonymity, and (2) to prevent advertising. A blurred face, blurred logo on a hat or shirt, or black box over a license plate is acceptable. However, the blurring/drawing must be blatant and obvious.

For Media Outlets

The photographers who provide the photos to the media must abide by much stricter rules than the media outlets. In contrast, outlets are permitted to perform manipulations that match their medium and format. These include:

• Scaling and cropping. While the photographer must provide the whole picture, the media is permitted to crop it to show a specific subject matter. For example, the photo may show President Obama surrounded by thirty people, but USA Today may crop it to just show his head.
  
• Color adjustment. The media, particularly printed media, are permitted to color correct an image. The photographer's picture may be too dark or not print well for a magazine. The media outlet can, and usually does, color correct the image. In this case, the pictures may actually be color corrected in specific regions or specific color bands, and not applied uniformly across the image.
  
• Acceptable drawing. The media may also choose to blur faces, censor logos, or annotate features with arrows or circles that highlight specific items. However, these modifications must be blatant. The subtle removal of a logo, when discovered, can lead to sharp criticism (or worse).

BP used to take photos and use them in their advertising campaigns; anything goes in advertisements. However, that role has changed. Since the Gulf disaster, BP has been providing photos that document recovery and cleanup efforts to the mass media. As someone who provides photos to the media, BP is expected to adhere to the higher standard. BP should not be making modifications reserved for media outlets.

BP: Best Practices

Unfortunately, BP seems to be making up their "Standard Photo Journalistic Best Practices" as they go. While I have not seen any splicing in the last few days, some of their photographers are still taking liberties with the crop tool and recoloring. Here are a few examples from BP's Flickr feed. (Click on the photo to see the full picture.)

Creative Cropping


This photo by Marc Morrison is dated 26-July-2010 but was last modified on 27-July-2010. The full picture is 3981x1496. The problem is, the Canon EOS-1Ds Mark II does not take photos at these dimensions. The closest it gets is 4992x3328. This means that Marc cropped nearly 20% from the horizontal and over 55% from the vertical. So what did Marc not want us to see?

A few years ago I was told a story about a photo from China. It appeared to show a government vehicle with people standing around it cheering. But the uncropped photo showed the crowd throwing stones; the people were not cheering, they were yelling. Creative cropping can alter the meaning of a picture. For this reason, "Standard Photo Journalistic Best Practices" requires the photographer to submit the whole picture and not something with creative cropping. For all we know, there could be a dead whale on the right, and that gray structure in the top-left could actually be pollution filling the sky. If the picture has too much sky, then BP needs to let the media outlets decide what to crop.

BP's True Colors
Here's a very colorful photo by BP:



This photo by Harrison McClary is dated 26-July-2010 and last modified a day later. The image itself measures 3600x2400. That is close to a native resolution for the Canon EOS-1D Mark III, which can take pictures at 3888x2592 (cropped or scaled 7% horizontal and 7% vertical). However, McClary over-applied the color correction. We can see this in the color histogram (graphing HSV).



There are two things that really stand out as abnormal: (1) the clusters of blue and yellow at the top shows a blown-out color space, and (2) the wide color blobs are too wide, too tall, and too blended for a natural picture. This is not a typical color space for a Canon EOS-1D Mark III.

For a comparison, consider this sample photo from the same model camera (and not provided by BP):




Notice how the unmodified photo does not blow out colors at the extreme intensities, and has less-blended color bands. This is very typical for a digital camera, including cameras made by Canon, Olympus, Nikon, Ricoh, and other manufacturers.

So why would BP's Harrison McClary over-correct the color space? Perhaps he is inexperienced with cameras. Or maybe he really wanted that brown water to look blue. By blowing out the color spectrum, he has given the image a "clean" look -- the sand is white, the sky and water are blue, the tractor does not look dirty, and even the brown grasses look green.

Here's another example from Harrison McClary:




Again, the blue and green are blown out (blobs at the upper intensities). Also, notice how the orange spike actually curves with intensity (vertical). That's why they call it a "color curve adjustment".

Of course, McClary isn't the only one tweaking colors. BP's Robert Seale also did some color corrections.




Notice how Robert's dark red, blue, and green all lean toward the left at the top? While he didn't blow out the color range, he did adjust the sky, grass, and maroon stripe on the bookmobile (the RV in the background-right that says "Vermilion Parish Public Library").

Seeing Red

Dear British Petroleum,

If you want to us to believe that the pictures are real, then please release real pictures. Don't crop out stuff you don't want us to see. Don't make the sky and water look bluer. And most importantly, don't think that we won't notice.

Having been caught splicing images, BP promised to adhere to "Standard Photo Journalistic Best Practices". However, this is clearly not the case. While BP claimed that the modifications were limited to one photographer, the actual problem is more systemic. BP's photographers may no longer be splicing, but they are still striving to literally show that the grass is always greener. This isn't a problem with BP's photographers; this is a problem with BP.

This week really gave me a thrill. Readers, models, and even large companies have taken steps against digital photo manipulation in the media.

The first big congrats goes to Domino's Pizza. They recently announced a promise to use real photos of real pizzas in their advertisements. No more cardboard, glue, and partially-cooked food that looks "better" when photographed.

Our Photo Promise
Here at Domino's, we don't think our inspired Domino's pizza needs the "extra" things typically done to food at photo shoots to look mouth watering. Our pizza is good enough to stand on its own. That's why we're making the following promises about how we photograph our pizzas from this day forward. Did we just buck the food photography trend? Oh yes we did.

1. We will only photograph real, honest-to-goodness pizzas.
That means fresh from our own ovens, with exactly the same ingredients we deliver to your doorstep. Nothing else added.

2. Our employees will make the pizza we shoot.
Not an art director or model maker or food stylist. A Domino's employee trained to make pizzas the only way they know how: by hand.

3. We will not artificially manipulate the food we shoot.
No tweezers, no steam guns, no model knives cutting perfect perforations in the cheese. The only thing that will touch the pizzas we shoot is the pizza-maker's hands and a standard Domino's pizza cutter.

Russell Weiner, Chief Marketing Officer

Bravo! I've looked at some of the pizza photos on their web site and I must say: no detectable manipulation (beyond scaling and cropping, which does not modify the look of the food). Moreover, the food actually looks good! (Good enough for me to now have a pizza craving.)

Pizza Photo by Makena B. from Houston, TX

Worth the Wait

Not to be outdone, plus-size model (and super hottie) Crystal Renn just went on the record saying that she is offended by some photoshopping done to her picture. As she said in her Today Show interview this morning, "When I first saw the photos, I would have to say I was absolutely shocked." The photographer turned this well-known size-10 into much thinner version. (But at least he didn't give her noodle arms, right Ralphie?)

The photographer, Nicholas Routzen, has this reply:

I want to reiterate that I feel Crystal looks amazing in both images and the minimal retouching that I did do - it's nothing you wouldn't see in any magazine today. There is nothing hidden about this.

This tells me three things: (1) he sways to peer pressure (everyone else is doing it...), (2) he does not listen to the models that he shoots (Renn has been a strong voice against the unhealthy, unrealistic anorexic female shape that most of the fashion companies strive for), and (3) he photoshops his pictures. It makes me want to take a much closer look and see if he also does splicing, smoothing, and other common forms of deceptive manipulation.

However, I would not recommend browsing Routzen's blog. Some of his photos could easily pass for child pornography. (Full frontal nudity of a minor.)

Feeling Pumped

But I am saving my largest applause for America Blog and Gizmodo. These people have been looking at the media photos released by British Petroleum (BP).

It isn't enough that BP's runaway deep-sea oil well poisoned the Gulf of Mexico, after they lied to the United States by claiming that they knew how to handle any deep-sea accidents. Or when they repeatedly underestimated the amount of oil and would not assist scientists in creating an accurate estimate (we still don't know how much oil was leaked). Or that they only provided low resolution video feeds to the public while they had high resolution footage available. Or that they tried to stop the media from documenting the disaster. No... they also have to doctor pictures. (Is anyone really surprised?)

One photo has the title "Aerials over Gulf of Mexico". With a name like "aerials", one would think it would be taken from the air...

The problem is, the view out the window has been photoshopped. I noticed many things in this picture, but the people on Reddit just shredded the photo. Some of the findings:

• The display clearly says that the door is open, ramp is open, rotor brake is on, and parking break is on. There is no way this helicopter is in the air.
  
  
• The radar shows something to the far left, but nothing in front of him. Thus, no boats.
  
• There is a light that says APU GEN ON. This is the alternate power unit. It provider power until the engines are started.
  
• The pilot is holding a pre-flight checklist. (Ironic that his fingers are crossed.)
  
• Neither pilot is holding the flight stick!
  
• There is a waterbottle resting in the handhold above the guy on the right. The water in the bottle is smooth and flat -- no vibrations at all.
  
• The pilot on the left is wearing glasses. The glasses are reflecting some type of straight-line object. This is likely a runway or edge for the helipad.
  
• The outside water goes from clear blue to smokey. You can clearly see the waves in the blue and smokey areas, but the waves are fuzzy/blended where the two meet.
  
• The water is also blurry around the pilot on the left and near the top of the right window.
  
• The edge of the boats on the left are precisely in the fuzzy section.
  
• The first boat in the right window has a very visible shadow. So all boats should have shadows. However, none of the boats in the left window have shadows.
  
• The boat with shadow indicates that the sun in in front of the helicopter. However, the entire copter is in shadow and so is the tower structure (top left).

This isn't even the entire list. It is suffice to say that this is not an "aerial" photo and it has been grossly modified.

Another photo shows people in front of some monitors. The problem is, the image shown in some of the monitors was changed. Technically, content from three screens was replicated into the three off-line screens. Oh, and the picture has an internal timestamp indicating that it was created in 2001 (2001-03-06 15:16:50.25) and not 2010 (EXIF data modified time 2010-07-19 18:54:04.25). In either case, the timestamps do not match the "HIVE at Houston Command Center 16 July 2010" as BP captioned the picture.

Modified

Allegedly Unmodified

The final picture (so far) shows people in a meeting room. However, the splicing of the content on the screen was done very poorly.

Here's a closeup of some of the splicing:

Frankly, I'm not sure what is more offensive -- the fact that the picture was modified, or the quality of the modification. In either case, this should be a firing offense.

Of course, I began to do what everyone else is probably doing -- poring over bp.com and looking for more doctored photos. That's when I noticed something. All of the modified photos appear to have something in common. The meta data and associated credits identify the photographer as "Marc Morrison".

Hello, Marc

According to his bio, Marc has been a photographer for 26 years and works for BP. A significant number of photos released by BP were taken by Marc.

Marc prefers Canon cameras like the EOS-1Ds Mark II or EOS 5D. While these cameras usually take very good photos, Marc's pictures always have a large mount of sensor noise and discoloration. (I can actually pick out Marc's photos on BP's site just by looking for the sensor noise and grainy coloring. Not every picture has had content modifications, but all look grainy and noisy.)

When it comes to manipulation, Marc seems to rely on overlaying and blending. He primarily targets flat surfaces like monitors or windows. His non-grainy photos appear to have color enhancements to make bright colors pop -- look for things that are red or yellow (his favorite bright colors). I have not seen him advance to people splicing, reflections, or lighting. He also appears to be fond of image cropping; I have yet to see any of his photos that are anywhere near close to a native camera resolution size. Oh, and Marc likes to use something called Photoshelter. (Since I have no experience with it, I can't tell if it is a program for editing or only web creations and annotations... In either case, many of his photos were modified by it.)

Two photos by Morrison. The left is "AP Photo/BP, Marc Morrison". The right is "EPA/Marc Morrison/BP Handout". Both show the same room and same people but at slightly different times. Some monitors are the same, some are different. Is either unmodified?

Now, for clarity, there appears to be many photographers named "Marc Morrison". One lives in Steamboat Springs, Colorado -- I really don't think it is him. Another lives in Houston, Texas. The Houston guy seems to take some celebrity photos as well as plenty of oil rig and related industrial photos. However, I haven't seen anything that says the guy in Houston works for BP. (This Marc could be a different Marc.)

In any case, many of the photos provided by BP's Marc Morrison were credited as "AP Photos/BP, Marc Morrison" and "Marc Morrison - AP". (Example: Washington Post, look at the slide show.) However, I cannot find any of Marc's photos at AP's web site. I wonder if they already booted him for altering images...

(Thanks to the 11 people who sent me links to this BP story. Keep 'em coming!)

The hardest parts of forensic analysis isn't the tools; it's the training. Anyone can buy rubber gloves, swabs for collecting blood samples, and plastic evidence bags. But if you are not trained to properly collect, handle, and evaluate evidence, then the tools and methods are meaningless.

The learning curve is the hardest part. To address this, I've been working on documentation and worksheets for digital image analysis and photo forensics. While there is still a steep learning curve, the investigator can review the worksheets as a checklist for common things to evaluate. The associated documentation provides details regarding the checklist items, in case the investigator needs to review how a particular system works.

An Eye For Details

While luminance gradient and error level analysis draw pretty pictures, the most important tool is basic observation. It is one thing to see the big and obvious signs of manipulation. It is something else to remember all of the fine details.

The folks at Photoshop Disasters recently posted a couple of amazingly bad shopped pictures that clearly illustrate the power of observation for detecting image modifications.

The first picture comes from an ad campaign for fingernail polish. The picture is supposed to show a model and some nail polish. The magical stars that go from her elbow to the picture frame are just artistic. However, it is the fine details that make this such an obvious disaster... Just using your eyes, what stands out as abnormal and not intentionally artistic? Give yourself a minute to look over it, then scroll down and see how many things you noticed.



If you only saw the disconnected leg, then give yourself one point. (If you didn't notice the leg, then go back and try again. As Thall commented at PsD, "That women could birth a horse or two with those hips!") Other oddities include:

• Her waist is out of proportion.
  
• There is a black triangle between her torso and floating leg. The artist forgot to cut out this area.
  
• Her left forearm (photo right) is significantly longer than her right forearm.
  
• She has two thumbs on her lower hand. (One thumb could be her foot showing through a strap in the shoe, but it is actually blended into the hand.) Oh, and don't mind the "S" on her finger; other photos show it as a tattoo or something.
  
• Her neck is showing a tendon, indicating that her head is turned. However, her head is looking straight and is not centered on the neck. Yes, they cut off her head and pasted on a different head.
  
• She is missing a clavicle (shoulder bone) -- one is there, but the other was erased.
  
• Her boobs are different sizes, and not in a natural way (unless the left one is half as long and deflated...).
  
• The shadow under her head indicates a bright light to the upper right. But the floating leg isn't casting the same shadow onto the other leg. And the shadow from the sleeve onto her straight arm has a shadow going the other direction. Inconsistent lighting means splices.
  
• Of course, all of these are issues with the woman. Over at PsD, ZaphodQB noticed that the reflection of the black polish does not meet the bottom of the black bottle.

This isn't the full list. What else do you see? No wonder their product is called "Oops!"

The Perfect Model

I'm always looking for good sample images that demonstrate specific points. Ideally, I want one picture that only demonstrates one thing, then another that demonstrates the same thing with more complexity, and finally an example that brings everything together.

From the Oops! example, we know to look for different classes of manipulation. These attributes become our checklist:

• Limbs: Are all of them accounted for? Are all connected? Are they the right proportions?
  
• Reflections: Do items line up properly?
  
• Shadows and lighting: Are they consistent?

Now we can apply this to a new set of pictures.

At Photoshop Disasters, they featured a picture from the French fashion house, Louis Vuitton. However, the web page at Fashion Gone Rogue contains many pictures from the "Louis Vuitton Fall 2010 Campaign" (also available at Fashionologie). It is an homage to digital distortions.

Starting at the top is the banner for Fashion Gone Rogue. Her upper arms are very different lengths. It is also faint (better seen with luminance gradient), but it looks like there is a strap or something going across her shoulder and down her cleavage. (This could be where the artist stopped altering the skin.)

Mirror Mirror On The Wall

The various photos from Louis Vuitton have been equally mangled. Let's use our new checklist...

The picture claims to show three women in a dressing room. Each has different color hair: red, blonde, and brunette.

Limbs
Every person has two arms? Check! Extra fingers? Nope. Legs and feet? Uh... the brunette on the right has an ankle but is missing toes.

Reflections
The right-most mirror (behind the toe-less brunette) is not reflecting anyone in the room. The blonde has her hand up in the room but her hand is down in the mirror. That same mirror also shows a light bulb in the reflection, but the bulb does not exist in the room.

The second mirror from the right shows bulbs but they don't align with the bulbs in the room.

The mirror on the far left shows red's head from the back. However, red's head is not turned to show her back to that mirror. And the mirror's reflection shows the lamp on the wrong side. The reflection does not match the room.

Lights and Shadows
When an item sits next to a illuminated light, it is made brighter. And when items are facing away from the light, they are in shadow. Complex lighting, such as floods, reflectors, and bright ambient lighting, can mitigate shadows.

However, those mirrors have a lot of bright lights. The women should have brightly lit backs. But this isn't what we're seeing. The brunette has bright reflections off her chest but not her back. The blonde has a bright clavicle but an under-lit neck. The pile of junk in the back has a brown fabric thing above the handbag; it is lying next to a light bulb and not lit up.

This isn't a comprehensive list and there are other oddities that are not in our checklist. For example, the blonde's dress seems to have a layering issue with red's chair. The dress fabric suddenly becomes semi-transparent and you can see the chair through it.

Frankly, I kind of doubt that these three women even posed together for this picture.

Some of the pictures in this series are much worse than others...


Dear Louis: While fabrics may be diaphanous, people are not. And while models may be vamps, they are not vampires. Please fix the left mirrors. You know, the ones with the time-delay reflections that show the brunette in two alternate positions and don't reflect the blonde.

Dress For Success

While I can criticize these ads for pasting in people, changing reflections, and digitally altering lighting, I have to give Vuitton one piece of credit:

Beyond expected color enhancements (applied to the entire picture) and spicing blends (expected from a composite image), I have not detected any modifications to the clothing. Well done. Unlike Ralph Lauren and Victoria's Secret, Vuitton's pictures do not appear to be a product bait-and-switch.

A little over a week ago a US intelligence analyst was arrested for submitting classified documents to Wikileaks. I have some serious issues about this arrest. While the analyst may have thought he was doing something ethically right, he went about it by doing something legally wrong. For example, while some of his wikileaked materials probably did need to be exposed (like the mistaken killing of two journalists and the subsequent cover up), how many operations and soldiers lives were put in danger by the leak?

I can hear some people right now saying "Huh? What?" Think about it. With the exception of leaked videos, the general public do not know our full, technical capabilities. As I recently heard on an NCIS repeat: the schematics for Air Force One are a secret. Hollywood just guesses at the layout. But here is SPC Bradley Manning, showing how things are really done. This is information that the enemy can use against us. By leaking an uncensored video with audio, Manning may have done far more harm than good; he exposed a cover up, as well as processes, procedures, and technologies that the United States and its allies use against real terrorists and threats to our nation.

There were also better ways to expose a cover-up. For example, he could have anonymously contacted a congressman. This would make the information public without releasing the video. Any anti-war congressman would have been a good choice.

While Manning may have thought that he was ethically correct in releasing the video, I cannot think of anything that would make leaking "an entire repository of classified foreign policy" documents, "260,000 classified U.S. diplomatic cables", or "a classified Army document evaluating Wikileaks as a security threat" ethically correct. Manning's actions look like treason to me.

From Bad to Worse

Wikileaks is intended as a forum for anonymous whistle blowers. If you are going to do something anonymously, then do it anonymously. Don't go around telling people that you were actually behind it. And if you're going to tell someone it was you, then don't tell it to a reporter. And of all the reporters you could talk to, don't choose one who has a history of unethical behavior!

That's right: Manning chatted with Wired's Adrian Lamo. When people create lists of hackers, they always include the notorious ones: Kevin Mitnick, Jonathan James (aka c0mrade), Max Ray Butler (aka Max Vision), Kevin Poulsen (aka Dark Dante), and others -- including Adrian Lamo (aka The Homeless Hacker). Even lists that don't list the "most notorious" include Lamo. (Thanks Adam for the link.)

Is there any reason to think that Lamo would not turn in Manning? I think not. Frankly, there are few reporters that I trust (very few). Most are more interested in sensationalism than accuracy. That, along with Lamo's established ethical lapses makes me distrust him more than most reporters. Manning put his trust in a reporter with a criminal record, and the reporter exposed his source for notoriety.

Looking for the Good

Every list of "hackers" that I found online mentioned the evil ones. The lawbreakers, criminals, and socially deviant ones. However, not all hackers are evil. I've recently had conversations about identifying good hackers. (Thanks to Mike, Bill, R., and the Internet Storm Center's handlers for the great insight.)

When it comes to naming hackers, people immediately recall the bad guys. I mean, everyone has heard of Kevin Mitnick, but who can remember the name of the guy who caught him -- without consulting Wikipedia or Google? (answer: Tsutomu Shimomura; half credit if you remembered John Markoff.)

Perhaps one reason is the postage stamp mentality. The US Post Office won't put someone on a stamp until they are dead. The reason: Bad people may continue to do bad things without harming their reputation. However, a good person may screw up at the end and tarnish everything they have previously done. So someone who is an awesome, positive role model and hacker today could be tomorrow's villain.

The other problem comes from the large number of good hackers who are better known by their software than their own actions. For example, Snort is an awesome piece of software, but who can remember that Martin Roesch created it? Roesch is a good guy hacker, but his software is better known than him. The same goes for Tatu Ylonen and Bjorn Gronvall (SSHv1 and SSHv2), Giorgio Maone (NoScript), and many other people.

The real question is: What sets a notable good guy apart from the rest? If writing good code is good enough, then certainly Flash, HTML, and Photoshop could also be included. (Their developers were not intentionally evil...) But can you actually say that someone changed how we act (or react) in a positive way?

I guess what I'm really wondering...
If you had one team of evil villains (Mitnick, Lamo, Poulsen, etc.) on one side, who would you stack against them as memorable good guys on the other side? (Mitnick vs Frank Abagnale Jr. -- after Frank turned good; Poulsen vs Mudge? Lamo vs ?)

Here's my short list of good guy hackers who's influence is far more than just code.

• Jim Christy and Clifford Stoll. They cracked the Hannover Hackers and brought international awareness to hacking as cyber espionage. Before that point, nobody realized the threat and all discussions were theoretical. Jim went on to found the Pentagon's first digital forensic lab and was the director of the Defense Cyber Crime Center (DC3) -- the first and largest computer forensics lab. (It just goes to show that a $0.75 accounting error can lead to more than a few pennies.)
  
  
• Mark Rasch. He spent nine years as the head of the United States Department of Justice computer crime unit. During that time, he was responsible for investigating the Hannover Hackers, Kevin Mitnick, and Robert T. Morris. He also helped the FBI and Treasury Department develop their procedures on handling electronic evidence.
  
  
• Phil Zimmermann. Without Phil, public crypto would probably still be nothing but underground software and munitions. Phil is more than just PGP -- today's PGP was created by a slew of developers. The remarkable element is how Phil paved the way for the world to use cryptography.
  
  
• Marcus Sachs. A voice of reason, advising Presidents and helping set national policies on cyber threats. He's a hacker who's influence is more than just a piece of code. Marcus also heads the Internet Storm Center -- the ISC handlers are like the Super Friends or Justice League. (Is Swa Frantzen the Belgium equivalent of Gleek?)
  
  
• Jeff Moss (aka Dark Tangent). While not known for code or exploits, this geek has put together a world-renowned set of hacker conferences: Defcon and Black Hat. More earth shattering updates come from one week of these conferences than an entire year of Patch Tuesdays.
  
  
• Bruce Schneier. This cryptography guru continually exposes security theater, where peddlers use snake-oil and provably inaccurate beliefs to influence and set policies.

A couple of people mentioned Dan Kaminsky. Dan's a nice guy and has done oodles of good things by making vulnerabilities public -- and I am still in awe of how he handled that world-wide DNS update. However, he likes to get drunk while giving presentations at Defcon and other conferences... While Dan is fun to watch, public drunkenness doesn't exactly scream "role model".

There are plenty of other people I could add to this list. I'm curious who other people think should be listed here. Remember the requirements: good guy, computer security or computer forensics, hackers, and most of all, influence beyond their immediate field or software.

My friend Bob G. recently told me about a column on page 12 of National Geographic Magazine's June 2010 issue. The column titled "Getting Real" is a rant about how a photographer supplied the magazine with a digitally altered photo. (A shorter version of the article is available online under the title "Your Shot Digital Manipulation".)

Basically, "photographer" William Lascelles submitted a photo to National Geographic and claimed that it was real. The magazine asked Lascelles to verify the photo, and he submitted a second fake photo. National Geographic then printed the image, only to learn that Lascelles lied to them twice; they were duped into printing a fake photo.

Whenever I post blog entries about photo manipulation, there is always someone who asks "what's the harm?" and "what about artistic freedom?" In this case, the harm directly impacts National Geographic's reputation. The photo contest is restricted to real photos for a reason: the magazine strives to use photos based on reality and not fake or doctored or modified situations that are fictitious.

In response to being duped, National Geographic published a full column identifying the fraud by William Lascelles and repeatedly naming him. The printed column also names another fake, by Dobrev and published in December 2009. This is more than just posting a small correction. This is a full outing.

Told You So

Like National Geographic, Smithsonian Magazine holds an annual photo contest. In March 2009, I wrote to the Smithsonian and informed them that some of their finalists used digital modifications that appeared to be outside the permitted amount of allowed manipulations. The Smithsonian responded a few days later. They had investigated the images and queried the photographers. In the end, three of the five modified photos were disqualified from the contest. The remaining two were determined to have an acceptable amount of digital modification.

I was thrilled with the Smithsonian's reply. They listened to the concern, investigated the situation, and took appropriate action.

I sent a similar letter to National Geographic on Tue, 24 Nov 2009 21:36:10 -0700. I had noticed that at least one of their contest submissions was digitally modified. Here's the reply they sent me:

Subject: Re: Digital Manipulation in the Photo Contest
To: "Dr. Neal Krawetz"
Sender: [redacted]@ngs.org
Date: Tue, 8 Dec 2009 13:48:19 -0500

Dear Dr. Neal Krawetz:

Thank you for contacting the National Geographic Society.

Your comments regarding photos submitted to the 2009 International Photo Contest are very much appreciated. While we provide information on photo manipulation and what is and is not accepted in the contest rules <http://ngm.nationalgeographic.com/photo-contest/manipulation>, all the photos in the contest are submitted by the individual photographers. We do not go through each one of them and remove them, even if we feel they have been manipulated. We simply do not have the time or staff to do that. You can view the winning photos on our website at http://ngm.nationalgeographic.com/photo-contest/past-winners

Best wishes,

[Name redacted]
National Geographic Society

That's right: National Geographic does not review each of the images that they accept for publication on their web pages, and they do not remove images that they know or believe are digitally modified. As seen with William Lascelles, their "verification" consists of asking the photographer -- who has already lied and won't mind lying again. I can understand that they don't have a big budget or staff for image analysis. However, this is their reputation. One would think that they would be interested in protecting it.

In the fraud exposure column, National Geographic wrote that they have changed their policy: "Now we're looking more closely at all Your Shot pictures." It's about time!

Frankly, if you are going to hold a photo contest and require original photos (not digitally altered), then you should take the time to verify every finalist. I'm not saying that you should police every submission. Rather, attempt to evaluate every finalist or semifinalist -- probably a dozen or so pictures. You usually don't even need special tools; a critical eye is usually good enough. If you don't have the time or resources to validate potential winners, then perhaps you shouldn't hold the contest.

And I'm sure that someone is going to ask "Where can they get my tools?" It isn't so much the tools as the training. If you are not trained to spot photo manipulation, then the best tools in the world won't help you. And as my friend Cynthia Baron has repeatedly demonstrated to me: people with the right training and experience can do this without any specialized tools.

Going Deeper

The dog picture by Lascelles is available online at http://s.ngm.com/your-shot/img/faked-blue-dog-615.jpg. Let's see what can be found with real image analysis...



Disclaimer
Please keep in mind: I'm not analyzing the original submission that National Geographic received. I'm analyzing a resave that National Geographic likely scaled for use on their web page. I seriously doubt that this is the original submission. In effect, I'm looking at a bad photocopy of the original submission. As with any bad photocopies, some results may be inaccurate due to artifacts introduced during the reproduction process and some evidence of modification may be completely wiped out. And more importantly: the original submission was fake, so it also includes modifications and artificial artifacts.

Photo Ballistics
Lascelles' file was saved as a JPEG and includes a JPEG APP12 section labeled "Ducky". (If you search for strings in the image, you will see "Ducky".) What is Ducky? It is a section added by Photoshop's "Save For Web" and includes the saved quality level. In this case, someone used Photoshop's Save For Web and selected 83% quality. (The "someone" was probably National Geographic.)

However, the quantization tables do not match the stated quality level. Instead, the Quantization tables match 94% compression. The discrepancy is due to the saved settings. Specifically, the last save used Save For Web with "JPEG", "Very High" and 83% quality. (The "Very High" setting selected the 94% quantization tables. Photoshop's quality level does not represent the quantization tables.)

Principal Component Analysis
PCA is great at identifying JPEG artifacts from resaves. These appear as rectangular blocks that are either 16x16, 16x8, 8x16, or 8x8. The more extreme the blocks, the lower the JPEG quality. High quality or original images should have very few visible artifacts.



In this case, the PCA shows severe blocking in the sky -- this is a low quality image from multiple resaves. But there is a problem... The blocks are not 8x8; they are smaller. In this case, the big squares appear to be 7x7 and small squares appear to be about 3x3. This means that the image was low quality and then scaled smaller. (I wouldn't be surprised if National Geographic scaled the final image for presenting on their web page.) The final image is probably about 40% smaller than the previous version. Since this image is 201x134, the previous image was somewhere around 500x335 (or larger with multiple resaves that scaled it smaller).

The other thing to notice is the block quality. The sky has big chunks indicating a low quality image. The front of the house has small blotches with no visible grid. The dog shows grid-like blocks on his ear and face that match the sky but not the house. And the jets have no visible blocky artifacts. So while the dog may go with the sky, it does not match the house or the jets. The mottled pattern on the front of the house actually matches what I would expect from a picture of this quality. (I even ran a few tests with other pictures using Save For Web and "Very High" and 83% -- the tests generated the same blotchy pattern seen on the house.) This means that the dog, jets, and sky are wrong for this picture.

Error Level Analysis
Taken without any context, the ELA for this image identifies the dog and jets as being at a higher error level potential (newer) than the rest of the image. However, this difference could be explained due to a combination of Photoshop and scaling. Photoshop attempts to counteract the JPEG losslessness by over-emphasizing high frequency areas. (See my Alyson Hannigan write-up.) This image does have a large amount of rainbowing (the red/blue/purple coloring), so this certainly matches the meta data that identified Photoshop. With scaling, every pixel is modified and high frequency areas (like the dog's fur) can have pixel values altered more than the rest of the image.



However, there are two issues that really stand out. First, the jet planes appear to be uniform in color (low frequency) yet have a high error level. So this is a modification.

Second, ELA should identify similar 8x8, 16x16, etc. blocks as PCA; for low quality images, ELA identifies the chrominance subsampling. And this is a problem. The subsampling should be the same across the entire image. However, the sky clearly shows 16x8 subsampling (scaled to fit 7x3 grids). However, the house only has square subsampling (either 8x8 or 16x16 scaled to 7x7; you can easily see them on the roof). With the dog and the jets, I don't see the subsampling grid. This means that the image must be made from four separate components: sky, house, jets, and dog.

Blur Detection
Blur detection identifies subtle, high frequency edges created from artificial blurs. Ideally, each edge should either consist on one thin line, or two parallel lines that are one pixel apart (1 pixel wide double line). Anything else indicates an artificial blur.



In this picture, the dog has 1-pixel wide double lined edges -- it is real. However, the house and jets both have wide double edges; the jets and house have artificial blurs.

Color Distance
A new algorithm that I've been working on is based on color distances. Basically, real pictures blend colors along edges. When pictures are spliced (one pasted onto another), there is no blending. This algorithm measures the amount of blend. If you see a thin black line outlining anything, then it was spliced.



In this case, the dog has a thin, black outline against the sky. He was spliced into the picture. It is a little more difficult to see, but the upper 4 jets also have thin black lines. They were pasted into the picture.

Observation

Alright, so it really looks like the dog and jets were spliced into the image. How many jets were there originally?



In this example, I have shifted the picture down and to the left a little. This allows me to overlay the top three jets onto the bottom three jets. Guess what? Two of them are perfect matches. Let's number the jets for clarity:

4

 2

5 1

 3

6

From what I can tell jets 2, 3, 4, and 5 are all the same plane and all uniformly spaced. Similarly, jets 1, 3, and 6 are the same plane. That is a total of two unique planes.

In real life, jets flying in formation will not be perfectly at the same angle to the viewer and they will not be perfectly spaced. Instead, they will all be ever so slightly different.


DoD photo by Airman 1st Class Gul Crockett, U.S. Air Force/Released


Same image, shifted to align the top plane on the middle plane, and overlaid to show differences.

This uniqueness factor even holds when the image is scaled smaller, like when the wingspan is only 20 pixels across. They may be small, but they should all be different. I think Lascelles cloned some jets.

Also, notice how none of Lascelles' vapor trails look the same. If they are about the same thickness and same color and the sun is in the same place, then they should all look similar. Jet #3 has a much whiter vapor trail -- likely two trails pasted next to each other. Jet #5 has a dark edge, probably from blending an overlapping paste.

National Disaster

Even without specialized tools, National Geographic should have noticed the cloning of the jets, varying vapor trails, and the artificial blur. Since they have the original submission, they could have checked the meta data and quantization tables to see if it matched the digital camera. This alone would have identified the fraud. And most importantly: don't just accept the word of a photographer; it may be significantly altered even if they claim that it is "100% real".