FotoForensics just received its 5 millionth unique picture upload! (Woo hoo!) As a research platform, we index the uploaded pictures and tag them for specific research uses. Often, we come across patterns among the uploads. Some patterns are associated with content, while others are associated with tools or techniques. To celebrate the "5 Million!" milestone, I thought I should share one of the findings with the community.

There is a specific alteration technique that I've been watching since 2015. The pictures stand out from the typical UFO and cat photos because of the content and alteration approach. These images typically show a room full of people watching some kind of PowerPoint presentation. However, the PowerPoint slide in the photos has been digitally altered. Sometimes it has been replaced with a sharper version of the same content, while other times it shows a completely different slide. The alteration stands out because the technique is not just a simple cut-n-paste. (I'm not going to detail the technique's signature here, but it is very distinct and unusual.) The content is also very specific:

• The people appear to be medical or pharmaceutical professionals.


• The slides are usually about some kind of pharmaceuticals and directly associated with pharmaceutical companies.


• All of the pictures are from Asia. Most come from China, but there are also some from Vietnam, Malaysia, Indonesia, Philippines, Singapore, Taiwan, Hong Kong, and other regions around China. (Yes, I know I listed Taiwan and Hong Kong independently of China. Cue the Chinese government's temper tantrum.) I do want to point out that I have not seen these alterations uploaded from Japan.


• When there is text on the slides, it is usually in Chinese or Vietnamese. Only two pictures (both uploaded from South Korea) used English.

The slide in the photo is almost always the thing that was altered. However, sometimes they also add people to the room. A few times, they have added a person as a presenter. Here are a few examples (click to view it at FotoForensics):

		The basic edits change the presented slide's content.
		Again, the presented slide is changed.
		Sometimes they change more than the presented slide. In this case, they added the presenter and the timestamp.
		Two versions of the same picture. The only difference is the presented slide. (Both are altered.)
		Sometimes they upload a picture where the audience was added or altered. In this case, they also uploaded the pre-edit image that shows fewer people.

In each of these examples, the error level analysis (ELA) shows a significantly different compression rate between the alteration and the rest of the image. The alterations also show up with other analysis tools.

So far, FotoForensics has received over 1,400 of these pictures.

A word of warning

I want to make a few things really clear before I dive any deeper:

1. None of these pictures are related to COVID. After over 1,400 pictures, I have seen no fakes that use this technique that are associated with COVID. The featured drugs are usually associated with common ailments like asthma, cholesterol (statins), heart disease, and blood pressure.


2. All of the altered pictures are from Asia. I have not seen the alteration method spread to Europe, Africa, or the Americas.


3. None of these pictures are "deep fakes". These are regular fakes that most people don't notice.

Bayer's Blue Binders

Initially, all of the pictures were uploaded by the same company: Bayer. Bayer is a huge pharmaceutical company. (If you've heard of aspirin, then you've heard of Bayer.) They are a world-wide multinational corporation. However, different regions have different headquarters. These pictures were specifically coming from Bayer Asia, and not North America, South America, or Europe.

I know the company's name because (1) some of the pictures featured Bayer products, and (2) Bayer has their own subnets on the internet. Many of the pictures came from their corporate networks, like AS140610 (Bayer South East Asia Pte Ltd). Even the pictures that came from Bayer in the United States (AS23167) had content in Chinese, featured Asian people, and browsers configured for Chinese, Vietnamese, and other Asian languages. The US network addresses appear to be part of a corporate proxy configuration and not where the pictures were altered.

Initially there were dozens of pictures being uploaded. However, they appeared to be created and uploaded by multiple people. IP addresses changed, browsers changed, and while it was the same basic alteration approach, there were subtle variations. When people are developing an alteration technique, you typically see a progression as they develop and adopt new approaches. In this case, there were small variations but no consistent adoption. This looked like a group of people following a step-by-step instruction manual and making very minor changes that differed from the instructions on a per-person level. (In the days of Morse Code, this was called the radio operator's "fist". Everyone used the same Morse Code, but minor personal differences in speed and cadence became distinctive per-person attributes.)

I knew a few people who worked at Bayer in the United States, so I asked what they thought was going on. While none of them knew about the pictures, none were surprised by the step-by-step instructions. At Bayer, everything is written down as step-by-step processes and placed in blue binders. It doesn't matter if it's how to run a chemical test or how to wash your hands, everything is written down and stored in a blue binder. (One of my contacts speculated that the custodial staff probably had a blue binder with instructions for changing a toilet paper roll.)

According to my sources, the company is hyper-focused on step-by-step instructions, and variations are discouraged. It does not matter if there is a faster, easier, or cheaper way to do something -- if it differs from the blue binder's instructions, then don't do it. Most likely, someone documented how to alter pictures and the steps are in a blue binder. Any variations that I see are because some element was not written down. They are probably using FotoForensics because someone wrote that down.

As soon as they mentioned this, I went looking through the collection to see if there were any photos with blue binders. It turns out, shelves with blue binders were pretty easy to find. Here are a few examples (click to see larger):

(If you look closely, you might notice that the slide in the 3rd picture doesn't match the reflection in the table.)

Why alter photos?

At this point, I knew "who", "what", and "how". But I didn't know "why". My friends had never seen anything like this, so they could only speculate. We brainstormed and came up with a few possible options (but these are not the only options):

• Deception: When there is a new process or new drug, it is supposed to be presented to peers. How do you prove peer review? You take a photo. If you don't want a real peer review, then you can supply a fake photo.


• Enhancement: Photos can be blurry or hard to read. If the photo doesn't clearly show the presentation, then they could alter the photo to make it easier to see. (This would be edited for readability, and not intended to be misleading.)


• Redaction: If they wanted to sanitize a photo for distribution, then they could alter it in a specific way. If the slides show confidential information, then they could replace it with something that is safe to distribute.


• Validation: Perhaps they have an internal problem with altered images. Maybe I'm not seeing uploads from people who make the alterations. Rather, maybe I'm seeing their review staff upload images to check for alterations. The only reason I doubt this option is that I didn't see many unaltered images. If it was a forgery-detection review process, I would expected to see more real pictures.

That third option, redaction for internal distribution, was interesting because it's easy to check. I looked all over Bayer's web sites and literature, but didn't see anything that looked like these pictures. If they were being distributed, they appeared to be internal-only.

A few years later, I received some pictures from non-Bayer pharmaceutical companies in the United States and Europe. Their picture featured Asian people with text written in Chinese, Vietnamese, and other Asian languages. I'm not naming the companies because I don't think they created the pictures. Instead, I think they received the pictures and uploaded them for testing. Unlike Bayer Asia, which uploaded hundreds of pictures and still uploads images, these companies only uploaded a few pictures in bulk and then stopped. Here are a few examples of pictures that came from competing pharmaceutical companies:

With permission

I want to point out that this blog entry is explicitly naming Bayer in Asia. Back in 2016, we knew who was uploading these pictures and we contacted them. (It was a very polite phone call. Late night here was work-time in Asia.)

• They acknowledged that they were uploading altered pictures.


• We pointed out that this was a public site, other people would see the pictures, and the pictures would be used for research purposes. They explicitly said that they did not mind. (Part of 'research' includes the disclosure of findings, like in this blog entry.)


• We mentioned that this appears to be commercial use and if so, they should use the commercial Lab service. They were not interested.


• For fun, we also asked if we could assist them -- either through training or peer review -- since there were ways to improve their alteration techniques. Again, they were not interested.

Usually I ban people for commercial use. But in this case, I permitted it because I was learning from them.

Community memetic spread

The technique did not stay internal to Bayer for more than a year. I first saw it spread to the Chinese government-controlled media. It was just a few pictures, but they definitely used the same technique. For example, this picture came from sina.com.cn (2016). I don't know what was originally on the TV, but it wasn't that picture.


The technique briefly surfaced in photos from a few other countries adjacent to Asia, including Turkey and India. It also appeared within a few other industries, but it never really took off.

However, over the last two years, it has spread to other Asia-branches of large pharmaceutical companies. I assume these are the companies because of the products that are featured, text on papers, and (when present) logos on employee badges. For example:

	Amiparen is a drug from Zuellig Pharma. They are headquartered in Hong Kong, but have branches all over Asia. This specific picture came from Vietnam and was titled "hinh.jpg" (hinh in Vietnamese translates as "picture").
	Espumisan is a drug from Berlin-Chemie AG, part of the Menarini Group. They are a world-wide pharmaceutical company. I don't know if this picture is from Berlin-Chemie, or being displayed as research from some other group. This picture was also uploaded from Vietnam and the standing man was added to the picture.
	This was a difficult picture to identify. Fortunately, my friend Mek has been learning to read Chinese. He found the drug's name and logo. It comes from Chia Tai Tianqing Pharmaceutical Group in China.
Many of the current pictures from China feature products from a pharmaceutical company named Roche.

Although each of these pictures came from different companies, they each used the same "Bayer Method" for altering the images. (My friend Shawn described this as a "community memetic spread.")

Continuing Trends

The uploads using this method used to come in bursts. When they uploaded, there were a few days with 1-2 sightings followed by a burst of 12-40 uploads. There were plenty of days, weeks, and even months with no uploads. However, the last two years has seen a steady increase as other pharmaceutical companies began using this method. Today, I see 1-6 uploads per weekday. I've tagged over 1,400 pictures so far. Most show presentations, but some include annotations and indicate some type of review process. For example, this altered picture is annotated to show an editing issue. The red text translates as "The inverted PPT content is inconsistent" -- the slide in the reflection does not match the slide being presented:


Usually I would use words like "systemic" or "endemic" to describe a widespread forgery approach. But in this case, it appears to be the conscious adoption of a forgery method within a specific industry. The method appears to have started with Bayer in Asia, and then spread to other Asian pharmaceutical companies. Today it appears to be a widespread practice in a specific geographical region. The combination of the distinct alteration approach, content associated with a niche market, and slow-but-steady intentional spread makes the Bayer Method stand out. We will continue to follow this technique at FotoForensics.

Computer forensics courses cover a wide range of tools and methodologies. Some are focused on specific tools, while others are focused on specific techniques. However, there is one consistent tool that they all cover: strings.

In technical terminology, a string is any sequence of text characters and spaces. (It doesn't have to look like a real word.) Binary or non-printable characters are not displayed when strings are extracted from a file.

In almost all areas of computer forensics, you will encounter binary files. These could be executables, packet captures, videos, pictures, documents, or large blobs of unknown content. If you just need a quick hint about the file's contents, then strings are a great way to take a peek.

Basic Strings

On every Linux and MacOS system, there is a program called 'strings'. (Windows users need to download it separately. Note: I'm not a Windows user so use at your own risk.) The strings program takes a target file on the command-line, opens the file, and displays every sequence of four or more bytes that looks like it could be text. While it will display every plain-text word, sentence, or paragraph, it will also display lots of random sequences. These random lines are where a short series of binary bytes coincidentally use the same character set as plain text.

Here's an example of the first few lines of raw strings from a JPEG file:

JFIF
;CREATOR: gd-jpeg v1.0 (using IJG JPEG v80), quality = 85
3CFl
bfFI3RH
Adobe
`%d3!@
2@"b
MhK4H
fb\V
&I!d
.=~/
XPTP
FRKH
ICfK
2Jd3
2jss2
VI@#&
bK!m

At the beginning of this listing are things that look like text. Then it quickly displays a bunch random characters. Those are just things that look like they may be text but that come from some binary sequences.

In general forensic use, 'strings' can help identify an unknown file. For example:

• If you don't know a program's usage, then strings might provide some assistance. Often 'usage' messages are stored in the file as plain text.


• Programs often use common system library calls. Those function names are usually written in text. Using strings on a program can sometimes help identify the program's purpose -- without running the executable.


• With Windows programs, I often use 'strings' to identify dependencies and other oddities. For example, why does the "strings.exe" from Microsoft's sysinternals include an X.509 certificate and links to Microsoft's Root Certificate Authority? (Strings is such as simple program that it shouldn't need any cryptography. Update: It was pointed out in the comments that this may be Microsoft signing every program for authenticity.) If a program modifies the registry or accesses the network, then you'll probably see the function names written in text.


• If you have a proprietary binary file, such as a custom database, then strings might help you identify some of the contents -- without needing to disassemble the file's format.


• If you have an unknown binary file, then strings might be able to help you identify the parent application or extract some of the content.

Because of these uses, the strings program is a powerful forensics tool. However, while strings give a little insight into the file's contents, it has some serious downsides. One of the biggest problems with strings is that you see the values without any sense of purpose or meaning. In the above strings example, I see the word "Adobe". However, I can't tell from the text extraction whether that is a comment, type of data structure, coincidental bytes that happen to look like a word, or an indication of the tool that was used to create the file. Even if I guess that it represents a data structure, I don't know if it's part of the main image or some internal thumbnail image. It is literally "text without context".

Strings at FotoForensics

At FotoForensics, I don't have any unknown file types because I already know the format: JPEG, PNG, WebP, HEIC, AVIF, etc. These are all well-known binary data formats.

Instead of using the standard 'strings' program, I've been using my own variation for a few years. In my case, I already know how to parse the formats, allowing me to put my own twist on string extraction. For my strings extractor, I identify the top-level data structures as headings and then list every string. The headings give a minimum amount of context to the file structure. Here are the first few strings from the same sample JPEG file:

JPEG SOI

JPEG APP0: JFIF

0x00000006: JFIF

JPEG Comment

0x00000017: ;CREATOR: gd-jpeg v1.0 (using IJG JPEG v80), quality = 85

JPEG DQT

JPEG DQT

JPEG SOF: Progressive

JPEG DHT

0x0000014c: 3CFl

JPEG DHT

0x00000184: bfFI3RH

JPEG APP14: Adobe

0x000001de: Adobe

JPEG SOS

JPEG Image Stream

0x000001f7: ` %d3!@
0x00000239: 2@"b
0x00000242: MhK4H
0x00000253: fb\V
0x0000025f: &I!d
0x00000282: .=~/
0x00000296: XPTP
0x000002c8: FRKH
0x0000031e: ICfK
0x00000406: 2Jd3
0x0000047f: 2jss2
0x000004bd: VI@#&
0x0000052d: bK!m

The output shows you the structural segment that precedes the data, the byte offset (in hexadecimal) showing where the string is located in the file, and the text string. (This is much more informative information than you get from the command-line 'strings' program.)

With this output, I can immediately determine that most of the random character sequences come from the binary data stream. Additionally, the text string "Adobe" is located at byte offset 0x1de (478 bytes from the beginning of the file) and I can identify the purpose: it specifies a well-known metadata application block that always starts with the letters "Adobe". These letters are not an indication of a file created by an Adobe product.

Because files are already parsed by the metadata analyzer, digest system, and other tools, my strings extractor is really only needed in a few corner-case situations. For example, if the metadata extractor or parasite detector identifies unknown content, then strings might help with the analysis.

Similarly, many vendors include proprietary MakerNotes data structures inside JPEG files. Most of what we know about the MakerNotes contents comes from reverse-engineering these structures. However, this also means that there is some data inside the MakerNotes that may not be decoded. I recently came across a photo from a Nikon camera that contained a partially-decoded MakerNotes. The undecoded section included the camera's serial number in plain text. Strings readily displayed this information.

Warnings about Strings

I've been holding off releasing this strings extractor because it has a lot of potential for causing confusion and misuse. People who don't read the tutorial are very likely to jump to the wrong conclusions.

Among other things, I expect people to confuse strings from well-known metadata types with the names of applications. For example, Adobe (company) has a graphics program called "Photoshop". However, this same company also created some common data structures that are found in JPEG files:

• The company called Adobe created a data structure for defining JPEG color transformations and named it "Adobe".


• This same company also created a data structure for storing IPTC information that they named "Photoshop".


• Developers at Adobe created the Extensible Metadata Platform (XMP) for storing metadata. XMP is an XML format that begins by defining applicable namespaces. The namespaces look like URLs are often include "adobe.com".

Each of these data structures begin with a text string that says the name. However, the data structure called 'Adobe' is not the same as the company 'Adobe', the data structure named 'Photoshop' is not the same as the application 'Photoshop', and a URL that says "adobe.com" may not actually be used as a link to that web site. Lots of non-Adobe and non-Photoshop systems create pictures that include these data fields; these fields are not indicators of specific applications. (As an aside, since these metadata fields have become widely used standards, does that mean the names 'Adobe' and 'Photoshop' in relation to JPEG images are in the public domain?)

Timestamps will certainly cause another problem. Strings will list timestamps that are stored in plain text. However, without any context, you won't know if the text represents the creation time, modification time, an unrelated comment, or something else. By the same means, many timestamps are stored as encoded values. Strings will neither decompress nor decode any non-textual elements, so those values will be missed by the text listing.

These kinds of mistakes are, sadly, not uncommon among professional analysts. A few years ago, I was retained as an expert consultant on a legal case. Part of my job included reviewing the report written by the opposing counsel's expert witness. The report contained nearly 10 pages evaluating the output from 'strings' on a JPEG file, while never once mentioning any metadata analysis tools. As a result, their expert reached a bunch of unsupported conclusions. (E.g., "Photoshop" did not indicate tampering, and his conclusion about a timestamp's purpose was speculative.) At one point, the attorney (with my assistance) asked the other attorney to ask the expert witness why he didn't use a metadata analyzer. The response? He claimed that he had. When pressed about which one and why the results were not in his report, he didn't have any good answers. To me, this looked like an inexperienced 'expert' who ignored metadata and tried to manually interpret a list of strings.

In contrast to string extraction, the metadata analyzer decodes known values and displays them in context. When evaluating a picture, you should always start with the metadata analyzer before considering any strings extraction. Strings are usually not needed and definitely should not be the go-to solution for most problems.

Using Strings

If you compare the outputs from the metadata analyzer with the strings extractor, you may see some of the same values. However, there are also plenty of differences. For example:

• The strings extractor lists the top-level structural components found in the file. This doesn't always match the segments identifies by the metadata analyzer. The structural components often have cryptic names that do not represent the purpose. (E.g., HEIC and AVIF have a structure called 'meta', but 'meta' usually doesn't include any textual metadata. Any actual text is usually stored in the 'mdat' data structure.)


• The metadata analyzer parses the file's structures and extracts the values in context. The strings extractor only lists segments that could potentially be text, without any context. Since strings skips any encoded data, the metadata analyzer will always display more useful information than the strings extractor.


• The metadata analyzer shows fields and values, providing the information in context to how it is used. In contrast, the strings extractor displays text with little context. Unless you really know what you're doing, do not assume the purpose of any extracted string.


• The metadata analyzer lists fields in the order they are found and the values with each field. The strings extractor lists text in the order it is found. With many data structures, the fields include pointers to values. The values may not be listed in the order they are used. In general, you should not assume any relevance based on the ordering of strings.


• The strings extractor displays all text, regardless of the purpose. The metadata analyzer evaluates the purpose of the text, reformats some textual values for readability, and omits some strings if they only serve to define a data structure. (Knowing the internals of the data structures usually does not provide additional information when evaluating metadata.) For example XMP metadata is often stored as a convoluted XML structure. The metadata view parses it into something that is more human-friendly and removes unnecessary cruft, such as namespace definitions.

While the strings extractor should not be used all of the time, it does have its occasional uses. For example:

• If you're curious about the file's internal format, then strings shows the top-level data structures. The ordering of these structures is often application-dependent.


• If the metadata analyzer identifies MakerNotes or any unknown data blocks (e.g., a JPEG with "Unknown APP4 Segment"), then strings can reveal any text-based contents that were skipped by the metadata analyzer.


• On Lab, the parasite detector flags unknown data blocks. Strings can help identify the type of parasite.

With that preface, I am enabling my strings analyzer for use at FotoForensics, on both the public and commercial Lab services.

I've been holding off writing this blog entry for 8 years. It isn't that I'm a fan of PhotoDNA; in fact, NCMEC never granted me access to it. Rather, even with it being treated by NCMEC and Microsoft as a big secret, I had been told that it was the only real automated solution that NCMEC had in their toolbox. Since it was helping a few organizations identify child pornography (CP) and child sexual abuse material (CSAM), I didn't want to hinder their efforts.

But like I said, that was years ago (2012-2015). The recent news coverage of Apple's CSAM announcement and updates has turned attention back to PhotoDNA. (Apple's new solution isn't using PhotoDNA, but that hasn't stopped the revised interest.) In the last two weeks, three different groups have contacted me about helping them reverse PhotoDNA. Each group made it very clear that they want to identify and publicize the algorithm's limitations. Why ask me? Well, as far as I know, I'm the only person who figured out how it works without being under an NDA. Back in 2014, I wrote a confidential whitepaper describing how it works and identifying specific limitations.

Last week, one of the groups learned where they could get "photodna.dll" -- the compiled library. As I understand it, this is supposed to be protected by an NDA or license agreement. However, a forensic vendor leaked it by inadvertently permitting access to their commercial software without a password. (I reported this to the vendor and then the github project found an alternate source.) On Tuesday (Aug 24), another research group went hunting for the same DLL. While I believe it is legal to download the available software and extract the DLL for reverse-engineering, the ethics seem questionable to me. In any case, I expect them to figure out how PhotoDNA works and identify the problems within weeks or months, not years. (One way or another, all of the secrets around PhotoDNA are about to come out.)

I suggested to some of the research groups that they contact NCMEC. (This would clear up any ethical issues.) However, each group made it clear that they were not interested in this approach. So I contacted NCMEC. I was informed that, in the last few years, NCMEC has added additional solutions beyond PhotoDNA. This includes Google's CSAI and Facebook's open-source video/image matching tools. What this means: my excuse of not wanting to take away NCMEC's only tool is no longer an issue. NCMEC now has other options. Moreover, NCMEC did not voice any concerns about this information being made public. (Poof! Ethical dilemma is gone!)

Rather than giving personal assistance to a few research groups, I'm making it all public now. In this blog entry, I'm going to cover how PhotoDNA works and the algorithm's limitations. (To the vendors who are still using PhotoDNA: I strongly suggest moving to something else.)

References and Disclaimer

A few years ago, getting the PhotoDNA code from NCMEC requires signing an NDA and then going through multiple layers of approvals. While I signed the NDA twice, NCMEC never countersigned it, never provided me with code, and stopped responding to my requests for status updates. Since they were treating it as a secret, I decided to figure it out for myself. (Update 2021-08-30: Today NCMEC informed me that Microsoft ended NCMEC's ability to sublicense the code a few years ago. Now it must come directly from Microsoft.) Fortunately, Microsoft had released a few documents that contained enough details for me to piece it together. (It's been 9 years; most of the docs are no longer online and the Internet Archive doesn't have copies of everything.) Here are some of the references I used (hyperlinks go to documents that still exist online):

• http://www.microsoft.com/en-us/news/download/presskits/photodna/docs/photodnafs.pdf (hyperlink goes to the Internet Archive)
• http://www.itu.int/en/cop/case-studies/Documents/ICMEC_PhotoDNA.PDF
• flowchart_photodna_Web.jpg (I originally found the picture at Microsoft, but it's no longer there. SmartPrix still has a copy of it.)
• http://www.coe.int/t/dghl/cooperation/economiccrime/cybercrime/Documents/Reports-Presentations/Octopus2011/WS4_PhotoDNA.pdf (link goes to the Internet Archive)
• http://conferencescco.files.wordpress.com/2012/11/1-ms-valerie-tan.pptx

The two biggest clues came from Valerie Tan's PowerPoint presentation and Microsoft's WS4_PhotoDNA.pdf file. Both documents show that PhotoDNA divides the image into 6x6 grids and then computes a set of four gradients (two horizontal and two vertical). Both documents also include a sample picture and an actual PhotoDNA hash. The PhotoDNA hash contains 144 numbers: 144=4×6×6. Now I knew that each of the 6x6 grids contains four separate gradients. Moreover, the documents include sample values to test against!

After re-creating the algorithm with values that were compatible (not identical, but close enough), I identified the algorithm's limitations. I wrote my confidential whitepaper in 2014 and quietly distributed it at non-public conferences and to people who had a need to know (NCMEC, Microsoft, some ICACs, and specific vendors who used the algorithm). Through these contacts, I began to receive hints on how to improve my implementation's results. Today I have an algorithm that generates the same values as the real PhotoDNA software, but it includes so many hints that I don't feel comfortable calling it "my code". For this reason, I will not be distributing code or detailing the whispered hints. None of these hints changed the limitations identified in the 2014 whitepaper; they only improved the compatibility of the generated hashes.

This blog entry describes how the algorithm works and the limitations, but not my thought process for reverse-engineering it. Much of this blog includes text and figures as-is from my 2014 whitepaper.

The Basic Algorithm

Every perceptual hash (PhotoDNA, pHash, aHash, etc.) has a few common steps. First, the image is normalized. This usually means reducing the size and converting to grayscale. Then the normalized image is converted to a hash. The specific conversion is dependent on the hash algorithm.

The Microsoft WS4_PhotoDNA.pdf presentation includes a high-level how-to for creating the hash. First, they convert it to a grayscale image. Next, they scale it to a normalized size. Finally, they compute the hash by segmenting the small image into 6x6 grids and compute the sum of Sobel gradients.

This image shows the scaled down 36x36 "pixels" divided into 6x6 grids. Each grid has a color. The diagonal 6 grids are highlighted with a thick border.

The problem with the 36x36 pixel solution is that it really isn't resistant to minor cropping. Removing as little as 1% off any edge is equivalent to half a pixel when normalized to 36x36. That causes the gradients to shift and the hash values to change.

To solve this problem, PhotoDNA uses overlapping grids. Each 6x6 pixel grid overlaps the next 6x6 pixel grid by 2 pixels. The normalizing step initially reduces the image to 26x26 pixels that are divided into 6x6 grids that overlap by 2 pixels. Now you can crop about 2% off any side and still have nearly the same hash. (A half-pixel shift is 1÷(26×2) = 1.9%; due to the overlap, this is not enough to shift any gradients by one pixel.)

This image shows 26x26 "pixels" divided into 6x6 overlapping grids. The overlap regions are shaded. The diagonal 6 grids are still highlighted with a thick border.

The pixels within each grid are used to compute a gradient. A gradient is simply the difference between pixel values. For example, if two adjacent pixels have the values 3 and 5 then the gradient is 2 since the value differs by two. The PhotoDNA hash consists of four summations: the total number of horizontally increasing values, total sum of horizontally decreasing values, the sum of the vertically increasing values, and the sum of the vertically decreasing values. A single grid with the hash value “1,12,8,2” (left, right, up, down) means that the total sum of horizontally decreasing values (left) is 1, the sum of horizontally increasing values (right) is 12, the sum of the up values is 8, and the down values is 2.

Using this approach (26x26 normalized image, 6x6 overlapping grids, sum of Sobel gradients per grid) won't generate the exact same values. However, your values will be in the ballpark and have the correct relative magnitudes. At this point, the only difference is the scaling algorithm, blurring, and an equalization to convert extreme values into something in the range of 0 to 255.

Comparing Hashes

Each PhotoDNA hash contains 144 values. The values are in the range 0 to 255 and represent the four gradients from each of the 6x6 grids.

I don't know the comparison algorithm or threshold that NCMEC uses. However, there are only a few ways to compare a vector of hashes:

• MSE. The mean square error is the sum of the differences between values. In pseudo code:
  

  function MSE(hash1[144],hash2[144]) {
  
    sum=0;
  
    for(n=0; n < 144; n++) {
  
      sum += (hash1[n]-hash2[n])*(hash1[n]-hash2[n]);
  
      }
  
    return(sum/144);
  
  }



• Linear Distance. This is almost the same as the MSE. Rather than returning sum/144, you return sqrt(sum).


• Raw squared difference. If you care about speed, then drop off the division or square root and just return the raw squared difference.

Microsoft's documents use a windmill as an example photo.

Image	PhotoDNA
Microsoft's windmill	2,9,2,13, 4,10,6,12, 13,14,60,10, 16,6,29,11, 6,0,3,11, 5,0,0,12, 7,5,133,16, 42,3,172,14, 182,63,255,99, 35,180,255,27, 14,3,209,20, 20,0,207,13, 14,20,255,1, 26,9,255,6, 131,29,255,36, 68,155,255,11, 31,5,255,4, 37,2,255,5, 0,43,45,47, 7,32,54,57, 115,23,49,36, 58,122,59,30, 18,4,57,34, 24,1,48,30, 3,16,133,14, 9,14,164,8, 100,16,148,10, 47,105,127,19, 20,14,137,26, 31,5,132,45, 6,13,36,9, 6,9,32,10, 28,8,25,20, 17,29,34,24, 14,7,63,10, 20,4,71,10 You might notice that my PhotoDNA hash is a little different from Microsoft's published hash. This is because I'm working off of the scaled and recolored image that I extracted from their PDF; this is not the original source that they used. For clarity: I am generating the correct PhotoDNA hash for this specific version of this file.
Slight cropping:	A slight crop changes the hash values a little. The numbers in bold are different from the uncropped source and red denotes values that changed by more than 5. The differences appear as mostly small numerical changes. 2,9,2,12, 4,10,6,11, 13,14,63,9, 16,7,32,10, 6,0,3,11, 5,0,0,12, 7,5,131,16, 41,3,169,14, 183,60,255,106, 39,186,255,30, 13,4,206,20, 20,0,204,12, 14,20,255,1, 25,8,255,5, 128,27,255,36, 73,157,255,12, 30,6,255,4, 39,1,255,5, 0,43,42,46, 6,31,48,56, 111,21,44,37, 63,126,54,29, 17,5,52,34, 22,1,41,29, 3,17,134,16, 9,14,165,10, 99,15,150,9, 54,112,125,16, 20,13,133,23, 29,6,128,40, 7,12,32,9, 6,10,32,10, 29,7,26,23, 17,33,34,27, 14,9,66,12, 20,4,74,13 One of the hints that I received applied an equalization across the sum-of-gradients in order to scale the values to the range 0 to 255. This has the unfortunate impact of changing lots of values when only a few sum-of-gradients changed. This equalization dramatically increases the number of false-negative matches. (This is the only additional limitation that I've found since writing the original 2014 paper.)
Image from 'thispersondoesnotexist.com':	113,92,33,67, 158,14,72,33, 14,35,1,126, 2,40,0,145, 61,6,1,103, 111,1,11,60, 163,78,10,25, 131,31,28,21, 0,53,21,14, 3,39,20,15, 80,1,10,30, 143,0,19,32, 132,48,115,9, 174,18,79,43, 16,97,86,47, 48,55,100,51, 42,65,61,44, 141,2,4,66, 113,55,246,0, 47,79,126,38, 19,88,60,55, 41,55,49,63, 19,54,29,42, 146,13,35,44, 130,18,69,20, 15,162,138,17, 17,113,78,31, 28,70,56,46, 62,48,135,13, 208,36,222,14, 93,12,27,35, 50,99,86,59, 20,161,210,21, 87,45,255,21, 142,8,255,7, 46,139,92,78 Nearly every value is significantly different. This (artifically generated) child is not a windmill.

Visually, the first two pictures may appear the same. However, the second one is cropped. (Original is 622x414, cropped is 615x406.) The PhotoDNA hashes are only a little different. The MSE is 5.3 and the linear difference is 27.7. These low numbers denote similar pictures.

The source windmill and child's picture are visually different. The hashes have a MSE of 9189.9 and a linear distance of 1086 -- these are much larger numbers. This difference allows tuning a threshold for an acceptable match.

Now that we know how the algorithm works, we can start addressing the limitations.

Limitation: Reversible Gradients

Because PhotoDNA stores the sum of opposite directions, the general texture of each grid can be identified. For example, a grid with the values "1,1,1,1" indicates a smooth grid (virtually no texture). The gradient to the left of the grid changes just as much as the gradient to the right. This can only happen if there is a single 1-value impulse near the middle of the 6x6 pixels grid. If the pixel change were along the edge of the 6x6 pixel grid, then one of the directions would have a value of zero (0).

A grid with the values "1,1,8,1" indicates a smooth upward gradient with a single impulse. The smooth gradient may be a horizontal line that spans the full six-pixel width with a gradient of 7 or multiple lines full-width lines that have a cumulative sum of 7. The single impulse is not located on or adjacent to any of the horizontal gradient lines, nor along the left, right, or bottom grid edges. There are a few dozen combinations that define the "1,1,8,1" pattern (not millions of combinations). Moreover, all of the combinations are visually similar - the strong upward gradient dominates the grid coloring.

In the windmill example, there is a grid with the values "5,0,0,12". This indicates no right or upward gradients, and only a slight left and downward gradient. This suggests a near-uniformly colored grid. This grid comes from the top-right corner of the photo (position [1,0] within the 6x6 overlapping grids) - where the sky is a near-uniformly color.

The windmill example also has a grid with hash values "20,0,207,13". These values define a rough slope that brightens upward sharply. The grid contains dirt and weeds from the bottom of the picture (position [1,5] within the 6x6 overlapping grid).

In general, lower values are the easiest to reverse into a pattern. Higher values have more potential combinations, but usually define a bumpier surface.

Limitation: Capped Values and Ambiguous Patterns

The only complex situation when reversing a PhotoDNA hash comes from the value 255. If the sum of the gradients in a direction is greater than 255 (e.g., 300), then the values are capped at 255. In effect, the value of 255 means "255 or more".

The capped 255 value leads to an ambiguity: completely different looking patterns can generate grids with multiple 255 values. This also means that non-linear color adjustments, such as a gamma correction, histogram equalization, or white balancing are likely to generate more 255 values since these color operations increase the gradient slopes.

However, there is also the final equalization that tries to scale the values into the range 0 to 255. This results in significant changes to all other hash values as the algorithm tries to reduce the number of 255 values.

This ambiguity also increases the amount of error when comparing hashes. The false-positive rate for incorrectly matching a picture to a known PhotoDNA hash increases with the number of 255 values in the known hash. Unfortunately, 255 values will usually appear wherever there is an extreme bumpy texture or color-equalized image. A grid that contains a bumpy rock will look like a grid that is filled with really curly hair (high-contrast texture). But at 6x6 pixels, it all looks similarly "bumpy".

Limitation: Constrained Values by Pixel Alignment

By strictly evaluating the four gradients associated with each grid, the texture and relative layout of the grid can be determined. Reversing the image with just the per-grid values will result in a blocky/blurry 36x36 image.

Unfortunately, PhotoDNA's 6x6 grids do not use a 36x36 pixel image. Instead, the algorithm uses a 26x26 pixel square and each grid overlaps with its neighboring grids by two pixels.

The two-pixel overlap between adjacent grids enforces a strong dependency. If one grid's gradients are mostly flat (e.g., [1,1,1,1]) and its adjacent neighbor is bumpy (e.g., [100,100,100,100]), then the bumpiness must be in the non-overlapping pixels and away from the flat neighbor.

Without the overlap, reversing a single grid's pattern from the four PhotoDNA hash values may yield a few thousand possible combinations. With the overlap enforcing constraints, the number of possible combinations is greatly reduced.

The middle four grids in the 6x6 grid segmentation are generally more important with pictures because photographers typically try to center subjects in the photos. The four middle grids within the 6x6 grid segmentation have overlapping regions with all of their neighbors. Even if the middle grid and its neighbors are bumpy, the constraints ensure that there are likely only a few dozen combinations that will generate the same gradients - and all of the combinations will appear visually similar.

Limitation: Constrained Values by Gradients

There are a few different approaches for computing a gradient. The direct approach is to simply subtract adjacent pixel values; a value of 2 next to a value of 5 will have a gradient of 3. However, another method uses a kernel, a small vector or matrix for computing a smoother gradient.

For example, the 1x3 Sobel gradient [-1 0 +1] ignores the value of the middle pixel. Instead, it subtracts the pixel on the left from the pixel on the right. A single impulse, such as [5 12 5] has a gradient of zero.

The sample workflow provided by Microsoft (WS4_PhotoDNA.pdf, slide 5) includes a Sobel gradient. With this gradient, the outer edge of each grid cannot be included in the gradient computation because one of the 3 positions in the 1x3 Sobel gradient will be outside of the grid. For example, each grid is 6 pixels wide, but the sum of horizontal gradients only uses the middle 4 pixel gradients per column.

The use of the Sobel gradient adds a constraint between the adjacent grids. For the middle four grids, the center 2x2 pixels are the only portion not in any direct overlap with neighboring grids. However, every gradient that uses these non-overlapping 2x2 pixels must include at least one pixel value from a constrained region. The 16 hash values from the middle four grids are all directly dependent on the texture and pixel values seen in the surrounding 12 grids.

This additional constraint further limits the possible values when reversing a PhotoDNA hash back into an image. This permits a significantly sharpened image from the hash projection.

Limitation: Weighted Regions

The two-pixel overlap permits PhotoDNA to identify similar pictures that differ by a minor cropping or image shift. A picture will need to have more than 4% of the image cropped from one side; cropping both sides of an image can usually match hashes with a nearly 8% size reduction across one dimension (horizontal or vertical).

However, the overlap regions enforce a weighted importance. Pixel values in the overlapped areas are more important for hash generation than pixels in the non-overlapped areas. They are more important because they influence hash values in multiple grids.

This image shows the 26x26 normalized image with colorized 6x6 grids; overlapped regions are in shades. The thick borders enclose the regions of the image that are not involved in any overlap. These are regions that carry the least amount of influence on the overall PhotoDNA hash since they only impact one set of grid gradients.

Within the 6x6 grid are a total of twenty-five 2x2 pixel regions that provide the most influence on the PhotoDNA hash values. Each of the 2x2 regions represent 0.6% of the total image (4 pixels out of 26x26 pixels), yet each 2x2 grid influences 11% of the picture's hash values. The four overlapping grid regions that share one 2x2 max-overlap area account for 10x10 of the 26x26 normalized pixels (15% of the image).

Modifying just one of these 2x2 maximum-overlap regions will alter less than 1% (0.6%) of the total image but can result in a significant change to 16 of the 144 PhotoDNA hash values, or 11% of the total hash. A minor change to the picture in one of the 25 specially selected areas can result in a significant change to the PhotoDNA hash value.

Modifying two of the 2x2 maximum-overlap regions, where the two regions share no common grids (such as the top-left and bottom-right maximum overlap regions) requires changing less than 1.2% of the picture but will result in a change to 8 of the 36 grids, or 22% of the PhotoDNA hash values. This is enough to generate a PhotoDNA hash that appears significantly different to the unaltered photo's hash, without significantly altering the visual content.

A hostile attacker who wishes to defeat PhotoDNA detection only needs to modify two of these 2x2 maximum-overlap regions. The attacker can either force a visually similar image to have a significantly different PhotoDNA hash value (forcing a false-negative result) or can make a visually dissimilar image appear to match a known PhotoDNA hash value. For example, the following two pictures have inverted two sets of 2x2 from the 26x26 positions. In both pictures, the edits account for 1.18% of the total image.

Edits to two minimum-weighted regions

1,14,2,18, 4,10,6,12, 13,14,60,10, 16,6,29,11, 6,0,3,11, 5,0,0,12, 7,5,133,16, 42,3,172,14, 182,63,255,99, 35,180,255,27, 14,3,208,20, 20,0,207,13, 14,20,255,1, 26,9,255,6, 131,29,255,36, 68,155,255,11, 31,5,255,4, 37,2,255,5, 0,43,45,47, 7,32,54,57, 115,23,49,36, 58,121,59,30, 18,4,57,34, 24,1,48,30, 3,16,133,14, 9,14,164,8, 100,16,148,10, 47,105,127,19, 20,14,137,26, 31,5,132,45, 6,13,36,9, 6,9,32,10, 28,8,25,20, 17,29,34,24, 14,7,63,10, 19,7,69,12 The top-left and bottom-right corners only impact the first and last gradient sets (in gray). (The other two spurious changes are due to the equalization. Both values differ by 1.)

Edits to two maximum-weighted regions

197,84,201,95, 20,155,112,55, 10,11,48,8, 13,5,23,8, 5,0,2,9, 4,0,0,9, 104,42,123,156, 42,76,146,88, 145,50,255,79, 28,143,220,21, 11,3,166,16, 16,0,165,10, 11,16,255,1, 20,7,255,4, 104,23,255,29, 54,123,255,8, 25,4,255,3, 30,1,255,4, 0,34,36,37, 5,26,43,45, 91,18,39,29, 46,97,47,24, 14,3,46,27, 19,1,38,24, 2,12,105,11, 7,11,130,6, 79,13,118,8, 37,84,101,15, 18,27,108,36, 46,12,103,65, 5,11,28,7, 5,7,26,8, 22,7,19,16, 13,23,27,19, 14,43,73,16, 67,23,101,24 Inverting two of the maximum overlap regions directly impacts four gradient sets (in gray) and changes nearly all hash values. Why didn't it only change 8 of the 36 grids or 32 of the 144 values? The algorithm applies an equalization after calculating the gradient sums. This causes big changes to propagate across all of the hash values. Most of the red values are clustered around the 8 grids that are directly impacted by the edits.

Both of these altered images contain the same amount of edits; the only difference are the locations. These images can be compared against the source windmill picture.

• In the picture with edits to the minimum-weighted regions, the MSE is 0.49 and the linear distance is 8.42. Both of these metrics result in very low numbers, indicating very similar pictures.


• The picture with edits to the maximum-weighted regions has an MSE of 1335.1 and a linear distance of 438.5. Those are large differences and imply that these should be different pictures. I don't know what thresholds NCMEC uses, but I was told (by an organization that has access to PhotoDNA) that these differences are large enough to be a "miss".

Update: On Twitter, Jan Kaiser noted that the tools he has looked at uses the raw difference. Different tools use different similarity cutoff values, but all seem to be between 40,000 and 50,000. (Below the cutoff is considered "same" and above is "different".) The minimum overlap alteration has a raw difference of 71 -- far below any threshold and denotes a strong match. The maximum overlap alteration has a raw distance of 192,262, strongly denoting a "miss" regardless of the threshold.

PhotoDNA was designed to match similar pictures with minor cropping. It can correctly match visually similar pictures that have edits on the non-overlapping regions. However, it can have significant problems when there are even minor edits in the maximum overlap regions.

Summarizing PhotoDNA Technical Limitations

PhotoDNA has some significant design weaknesses:

• The four sum-of-gradient values from each grid define each grid's texture. By providing a matched set of opposite directions (up and down, left and right), the surface within the grid can be reversed to a set of a few hundred possible values.


• The overlapping two-pixel region between grids reduces the set of possible values in each grid to a few dozen possibilities that are all visually similar.


• The multi-pixel Sobel gradient further reduces the possible set of values and permits sharpening any hash projection.


• The use of an equalization for scaling the sum-of-gradients increases the likelihood of a false-negative for any minor edit.

Based on these constraints, the PhotoDNA perceptual hash should be reversible to a recognizable image. Although multiple viable results are likely, all should be visually similar.

PhotoDNA does not detect flips, mirroring, 90-degree rotations, or inverting. However, it is supposed to detect visually similar pictures. Digitally alter less than 2% of the picture in very specific locations can effectively avoid detection. Moreover, these edits can be applied to non-salient regions of the picture.

Someone who wants to generate false-positive results only needs to modify a few selective portions of the picture. Forcing false-positive results can be used to justify plausible deniability in a court of law. (If you're involved in a CP/CSAM case, make sure your attorney receives the picture and not just a claim that the hash matched. If the evidence doesn't have the same SHA1, SHA256, or other strong cryptographic checksum, or isn't visually similar as identified by a human, then have the evidence excluded. It's not that I'm pro-CP, it's that I've heard of cases where people were accused based only on hash matches.)

Pseudo-Code

Back in 2014, I wrote some pseudo-code and test cases for reversing a PhotoDNA hash. (I'm sure my approach is inefficient and crypto gurus can do a better job.) My basic approach treated it like a 26x26 Sudoku puzzle. The approach brute forced every combination while obeying the constraints imposed by the gradients and overlapped grid positions. A brute-force approach should not take more than a few days to reverse one hash. I confirmed this approach by altering the prior normalized 26x26 image and observing the changes to the hash values. (That was in 2014. Today I would use a neural network since AI should be able to easily learn the mapping from hash to image.)

I have made no attempt to automate this reverse-PhotoDNA solution. It is my belief that, upon creating an automated solution, everyone who distributes NCMEC's PhotoDNA hashes will be distributing child pornography, and everyone in possession of NCMEC's PhotoDNA hashes will be in possession of child pornography.

Reversing a PhotoDNA hash should result in a 26x26 version of the picture that is grayscaled and slightly blurry. (A 26x26 pixel image is about the same resolution as a small desktop icon.) The final result is unlikely to be identical to the source image, but should be visually similar and permit identifying specific people (when the face is the main picture's focus), the number of people in the photo, relative positioning, and other large elements (doors, beds, etc.).

I hand-delivered my whitepaper to NCMEC and Microsoft representatives in 2014. Additional copies were delivered to them between 2014 and 2016. There has been no feedback from them. Meanwhile, in the statements repeatedly made by Microsoft and NCMEC, they have been keen to point out that "the PhotoDNA hash is not reversible, and therefore cannot be used to recreate an image." I believe they are wrong.

My in-box has been flooded over the last few days about Apple's CSAM announcement. Everyone seems to want my opinion since I've been deep into photo analysis technologies and the reporting of child exploitation materials. In this blog entry, I'm going to go over what Apple announced, existing technologies, and the impact to end users. Moreover, I'm going to call out some of Apple's questionable claims.

Disclaimer: I'm not an attorney and this is not legal advice. This blog entry includes my non-attorney understanding of these laws.

The Announcement

In an announcement titled "Expanded Protections for Children", Apple explains their focus on preventing child exploitation.

The article starts with Apple pointing out that the spread of Child Sexual Abuse Material (CSAM) is a problem. I agree, it is a problem. At my FotoForensics service, I typically submit a few CSAM reports (or "CP" -- photo of child pornography) per day to the National Center for Missing and Exploited Children (NCMEC). (It's actually written into Federal law: 18 U.S.C. § 2258A. Only NMCEC can receive CP reports, and 18 USC § 2258A(e) makes it a felony for a service provider to fail to report CP.) I don't permit porn or nudity on my site because sites that permit that kind of content attract CP. By banning users and blocking content, I currently keep porn to about 2-3% of the uploaded content, and CP at less than 0.06%.

According to NCMEC, I submitted 608 reports to NCMEC in 2019, and 523 reports in 2020. In those same years, Apple submitted 205 and 265 reports (respectively). It isn't that Apple doesn't receive more picture than my service, or that they don't have more CP than I receive. Rather, it's that they don't seem to notice and therefore, don't report.

Apple's devices rename pictures in a way that is very distinct. (Filename ballistics spots it really well.) Based on the number of reports that I've submitted to NCMEC, where the image appears to have touched Apple's devices or services, I think that Apple has a very large CP/CSAM problem.

[Revised; thanks CW!] Apple's iCloud service encrypts all data, but Apple has the decryption keys and can use them if there is a warrant. However, nothing in the iCloud terms of service grants Apple access to your pictures for use in research projects, such as developing a CSAM scanner. (Apple can deploy new beta features, but Apple cannot arbitrarily use your data.) In effect, they don't have access to your content for testing their CSAM system.

If Apple wants to crack down on CSAM, then they have to do it on your Apple device. This is what Apple announced: Beginning with iOS 15, Apple will be deploying a CSAM scanner that will run on your device. If it encounters any CSAM content, it will send the file to Apple for confirmation and then they will report it to NCMEC. (Apple wrote in their announcement that their staff "manually reviews each report to confirm there is a match". They cannot manually review it unless they have a copy.)

While I understand the reason for Apple's proposed CSAM solution, there are some serious problems with their implementation.

Problem #1: Detection

There are different ways to detect CP: cryptographic, algorithmic/perceptual, AI/perceptual, and AI/interpretation. Even though there are lots of papers about how good these solutions are, none of these methods are foolproof.

The cryptographic hash solution

The cryptographic solution uses a checksum, like MD5 or SHA1, that matches a known image. If a new file has the exact same cryptographic checksum as a known file, then it is very likely byte-per-byte identical. If the known checksum is for known CP, then a match identifies CP without a human needing to review the match. (Anything that reduces the amount of these disturbing pictures that a human sees is a good thing.)

In 2014 and 2015, NCMEC stated that they would give MD5 hashes of known CP to service providers for detecting known-bad files. I repeatedly begged NCMEC for a hash set so I could try to automate detection. Eventually (about a year later) they provided me with about 20,000 MD5 hashes that match known CP. In addition, I had about 3 million SHA1 and MD5 hashes from other law enforcement sources. This might sound like a lot, but it really isn't. A single bit change to a file will prevent a CP file from matching a known hash. If a picture is simple re-encoded, it will likely have a different checksum -- even if the content is visually the same.

In the six years that I've been using these hashes at FotoForensics, I've only matched 5 of these 3 million MD5 hashes. (They really are not that useful.) In addition, one of them was definitely a false-positive. (The false-positive was a fully clothed man holding a monkey -- I think it's a rhesus macaque. No children, no nudity.) Based just on the 5 matches, I am able to theorize that 20% of the cryptographic hashes were likely incorrectly classified as CP. (If I ever give a talk at Defcon, I will make sure to include this picture in the media -- just so CP scanners will incorrectly flag the Defcon DVD as a source for CP. [Sorry, Jeff!])

The perceptual hash solution

Perceptual hashes look for similar picture attributes. If two pictures have similar blobs in similar areas, then the pictures are similar. I have a few blog entries that detail how these algorithms work.

NCMEC uses a perceptual hash algorithm provided by Microsoft called PhotoDNA. NMCEC claims that they share this technology with service providers. However, the acquisition process is complicated:

1. Make a request to NCMEC for PhotoDNA.
2. If NCMEC approves the initial request, then they send you an NDA.
3. You fill out the NDA and return it to NCMEC.
4. NCMEC reviews it again, signs, and revert the fully-executed NDA to you.
5. NCMEC reviews your use model and process.
6. After the review is completed, you get the code and hashes.

(Update 2021-08-30: Today NCMEC informed me that Microsoft ended NCMEC's ability to sublicense the code a few years ago. Now it must come directly from Microsoft.)

Because of FotoForensics, I have a legitimate use for this code. I want to detect CP during the upload process, immediately block the user, and automatically report them to NCMEC. However, after multiple requests (spanning years), I never got past the NDA step. Twice I was sent the NDA and signed it, but NCMEC never counter-signed it and stopped responding to my status requests. (It's not like I'm a little nobody. If you sort NCMEC's list of reporting providers by the number of submissions in 2020, then I come in at #40 out of 168. For 2019, I'm #31 out of 148.)

Since NCMEC was treating PhotoDNA as a trade secret, I decided to reverse engineer the algorithm using some papers published by Microsoft. (No single paper says how it works, but I cobbled together how it works from a bunch of their marketing blurbs and high-level slides.) I know that I have implemented it correctly because other providers who have the code were able to use my hashes to correctly match pictures.

Perhaps there is a reason that they don't want really technical people looking at PhotoDNA. Microsoft says that the "PhotoDNA hash is not reversible". That's not true. PhotoDNA hashes can be projected into a 26x26 grayscale image that is only a little blurry. 26x26 is larger than most desktop icons; it's enough detail to recognize people and objects. Reversing a PhotoDNA hash is no more complicated than solving a 26x26 Sudoku puzzle; a task well-suited for computers.

I have a whitepaper about PhotoDNA that I have privately circulated to NCMEC, ICMEC (NCMEC's international counterpart), a few ICACs, a few tech vendors, and Microsoft. The few who provided feedback were very concerned about PhotoDNA's limitations that the paper calls out. I have not made my whitepaper public because it describes how to reverse the algorithm (including pseudocode). If someone were to release code that reverses NCMEC hashes into pictures, then everyone in possession of NCMEC's PhotoDNA hashes would be in possession of child pornography.

The AI perceptual hash solution

With perceptual hashes, the algorithm identifies known image attributes. The AI solution is similar, but rather than knowing the attributes a priori, an AI system is used to "learn" the attributes. For example, many years ago there was a Chinese researcher who was using AI to identify poses. (There are some poses that are common in porn, but uncommon in non-porn.) These poses became the attributes. (I never did hear whether his system worked.)

The problem with AI is that you don't know what attributes it finds important. Back in college, some of my friends were trying to teach an AI system to identify male or female from face photos. The main thing it learned? Men have facial hair and women have long hair. It determined that a woman with a fuzzy lip must be "male" and a guy with long hair is female.

Apple says that their CSAM solution uses an AI perceptual hash called a NeuralHash. They include a technical paper and some technical reviews that claim that the software works as advertised. However, I have some serious concerns here:

1. The reviewers include cryptography experts (I have no concerns about the cryptography) and a little bit of image analysis. However, none of the reviewers have backgrounds in privacy. Also, although they made statements about the legality, they are not legal experts (and they missed some glaring legal issues; see my next section).


2. Apple's technical whitepaper is overly technical -- and yet doesn't give enough information for someone to confirm the implementation. (I cover this type of paper in my blog entry, "Oh Baby, Talk Technical To Me" under "Over-Talk".) In effect, it is a proof by cumbersome notation. This plays to a common fallacy: if it looks really technical, then it must be really good. Similarly, one of Apple's reviewers wrote an entire paper full of mathematical symbols and complex variables. (But the paper looks impressive. Remember kids: a mathematical proof is not the same as a code review.)


3. Apple claims that there is a "one in one trillion chance per year of incorrectly flagging a given account". I'm calling bullshit on this.

Facebook is one of the biggest social media services. Back in 2013, they were receiving 350 million pictures per day. However, Facebook hasn't released any more recent numbers, so I can only try to estimate. In 2020, FotoForensics received 931,466 pictures and submitted 523 reports to NCMEC; that's 0.056%. During the same year, Facebook submitted 20,307,216 reports to NCMEC. If we assume that Facebook is reporting at the same rate as me, then that means Facebook received about 36 billion pictures in 2020. At that rate, it would take them about 30 years to receive 1 trillion pictures.

According to all of the reports I've seen, Facebook has more accessible photos than Apple. Remember: Apple says that they do not have access to users' photos on iCloud, so I do not believe that they have access to 1 trillion pictures for testing. So where else could they get 1 trillion pictures?

• Randomly generated: Testing against randomly generated pictures is not realistic compared to photos by people.


• Videos: Testing against frames from videos means lots of bias from visual similarity.


• Web crawling: Scraping the web would work, but my web logs rarely show Apple's bots doing scrapes. If they are doing this, then they are not harvesting at a fast enough rate to account for a trillion pictures.


• Partnership: They could have some kind of partnership that provides the pictures. However, I haven't seen any such announcements. And the cost for such a large license would probably show up in their annual shareholder's report. (But I haven't seen any disclosure like this.)


• NCMEC: In NCMEC's 2020 summary report, they state that they received 65.4 million files in 2020. NCMEC was founded in 1984. If we assume that they received the same number of files every year (a gross over-estimate), then that means they have around 2.5 billion files. I do not think that NCMEC has 1 trillion examples to share with Apple.

Perhaps Apple is basing their "1 in 1 trillion" estimate on the number of bits in their hash?

• With cryptographic hashes (MD5, SHA1, etc.), we can use the number of bits to identify the likelihood of a collision. If the odds are "1 in 1 trillion", then it means the algorithm has about 40 bits for the hash. However, counting the bit size for a hash does not work with perceptual hashes.


• With perceptual hashes, the real question is how often do those specific attributes appear in a photo. This isn't the same as looking at the number of bits in the hash. (Two different pictures of cars will have different perceptual hashes. Two different pictures of similar dogs taken at similar angles will have similar hashes. And two different pictures of white walls will be almost identical.)


• With AI-driven perceptual hashes, including algorithms like Apple's NeuralHash, you don't even know the attributes so you cannot directly test the likelihood. The only real solution is to test by passing through a large number of visually different images. But as I mentioned, I don't think Apple has access to 1 trillion pictures.

What is the real error rate? We don't know. Apple doesn't seem to know. And since they don't know, they appear to have just thrown out a really big number. As far as I can tell, Apple's claim of "1 in 1 trillion" is a baseless estimate. In this regard, Apple has provided misleading support for their algorithm and misleading accuracy rates.

The AI interpretation solution

An AI-driven interpretation solution tries to use AI to learn contextual elements. Person, dog, adult, child, clothing, etc. While AI systems have come a long way with identification, the technology is nowhere near good enough to identify pictures of CSAM. There are also the extreme resource requirements. If a contextual interpretative CSAM scanner ran on your iPhone, then the battery life would dramatically drop. I suspect that a charged battery would only last a few hours.

Luckily, Apple isn't doing this type of solution. Apple is focusing on the AI-driven perceptual hash solution.

Problem #2: Legal

Since Apple's initial CSAM announcement, I've seen lots of articles that focus on Apple scanning your files or accessing content on your encrypted device. Personally, this doesn't bother me. You have anti-virus (AV) tools that scan your device when your drive is unlocked, and you have file index systems that inventory all of your content. When you search for a file on your device, it accesses the pre-computed file index. (See Apple's Spotlight and Microsoft's Cortana.)

You could argue that you, as the user, have a choice about which AV to use, while Apple isn't giving you a choice. However, Microsoft ships with Defender. (Good luck trying to disable it; it turns on after each update.) Similarly, my Android ships with McAfee. (I can't figure out how to turn it off!)

The thing that I find bothersome about Apple's solution is what they do after they find suspicious content. With indexing services, the index stays on the device. With AV systems, potential malware is isolated -- but stays on the device. But with CSAM? Apple says:

Only when the threshold is exceeded does the cryptographic technology allow Apple to interpret the contents of the safety vouchers associated with the matching CSAM images. Apple then manually reviews each report to confirm there is a match, disables the user’s account, and sends a report to NCMEC.

In order to manually review the match, they must have access to the content. This means that the content must be transferred to Apple. Moreover, as one of Apple's tech reviewers wrote, "Users get no direct feedback from the system and therefore cannot directly learn if any of their photos match the CSAM database." This leads to two big problems: illegal searches and illegal collection of child exploitation material.

Illegal Searches

As noted, Apple says that they will scan your Apple device for CSAM material. If they find something that they think matches, then they will send it to Apple. The problem is that you don't know which pictures will be sent to Apple. You could have corporate confidential information and Apple may quietly take a copy of it. You could be working with the legal authority to investigate a child exploitation case, and Apple will quietly take a copy of the evidence.

To reiterate: scanning your device is not a privacy risk, but copying files from your device without any notice is definitely a privacy issue.

Think of it this way: Your landlord owns your property, but in the United States, he cannot enter any time he wants. In order to enter, the landlord must have permission, give prior notice, or have cause. Any other reason is trespassing. Moreover, if the landlord takes anything, then it's theft. Apple's license agreement says that they own the operating system, but that doesn't give them permission to search whenever they want or to take content.

Illegal Data Collection

The laws related to CSAM are very explicit. 18 U.S. Code § 2252 states that knowingly transferring CSAM material is a felony. (The only exception, in 2258A, is when it is reported to NCMEC.) In this case, Apple has a very strong reason to believe they are transferring CSAM material, and they are sending it to Apple -- not NCMEC.

It does not matter that Apple will then check it and forward it to NCMEC. 18 U.S.C. § 2258A is specific: the data can only be sent to NCMEC. (With 2258A, it is illegal for a service provider to turn over CP photos to the police or the FBI; you can only send it to NCMEC. Then NCMEC will contact the police or FBI.) What Apple has detailed is the intentional distribution (to Apple), collection (at Apple), and access (viewing at Apple) of material that they strongly have reason to believe is CSAM. As it was explained to me by my attorney, that is a felony.

At FotoForensics, we have a simple process:

1. People choose to upload pictures. We don't harvest pictures from your device.


2. When my admins review the uploaded content, we do not expect to see CP or CSAM. We are not "knowingly" seeing it since it makes up less than 0.06% of the uploads. Moreover, our review catalogs lots of types of pictures for various research projects. CP is not one of the research projects. We do not intentionally look for CP.


3. When we see CP/CSAM, we immediately report it to NCMEC, and only to NCMEC.

We follow the law. What Apple is proposing does not follow the law.

The Backlash

In the hours and days since Apple made its announcement, there has been a lot of media coverage and feedback from the tech community -- and much of it is negative. A few examples:

• BBC: "Apple criticised for system that detects child abuse"


• Ars Technica: "Apple explains how iPhones will scan photos for child-sexual-abuse images"


• EFF: "Apple's Plan to 'Think Different' About Encryption Opens a Backdoor to Your Private Life"


• The Verge: "WhatsApp lead and other tech experts fire back at Apple's Child Safety plan"

This was followed by a memo leak, allegedly from NCMEC to Apple:


I understand the problems related to CSAM, CP, and child exploitation. I've spoken at conferences on this topic. I am a mandatory reporter; I've submitted more reports to NCMEC than Apple, Digital Ocean, Ebay, Grindr, and the Internet Archive. (It isn't that my service receives more of it; it's that we're more vigilant at detecting and reporting it.) I'm no fan of CP. While I would welcome a better solution, I believe that Apple's solution is too invasive and violates both the letter and the intent of the law. If Apple and NCMEC view me as one of the "screeching voices of the minority", then they are not listening.

Update 2021-08-09: In response to widespread criticism, Apple quickly released an FAQ. This FAQ contradicts their original announcement, contradicts itself, contains doublespeak, and omits important details. For example:

• The FAQ says that they don't access Messages, but also says that they filter Messages and blur images. (How can they know what to filter without accessing the content?)


• The FAQ says that they won't scan all photos for CSAM; only the photos for iCloud. However, Apple does not mention that the default configuration uses iCloud for all photo backups.


• The FAQ say that there will be no falsely identified reports to NCMEC because Apple will have people conduct manual reviews. As if people never make mistakes.

This is far from the complete list of issues with their FAQ. It does not resolve any of the concerns raised in this blog entry.