~A proof of concept~
From chatGPT to Stable Diffusion, Artificial Intelligence (AI) is having a summer the likes of which rival only the AI heydays of the 1970s. This jubilation, however, has not been met without resistance. From Hollywood to the Louvre, AI seems to have awoken a sleeping giant — a giant keen to protect a world that once seemed exclusively human: creativity.
For those desiring to protect creativity, AI appears to have an Achilles heel: training data. Indeed, all of the best models today necessitate a high-quality, world-encompassing data diet — but what does that mean?
First, high-quality means human created. Although not-human-created data has made many strides since the idea of a computer playing itself was popularized by War Games, computer science literature has shown that model quality degrades over time if humanness is completely taken out of the loop (i.e., model rot or model collapse). In simple terms: human data is the lifeblood of these models.
Second, world-encompassing means world-encompassing. If you put it online, you should assume the model has used it in training: that Myspace post you were hoping only you and Tom remembered (ingested), that picture-encased-memory you gladly forgot about until PimEyes forced you to remember it (ingested), and those late-night Reddit tirades you hoped were just a dream (ingested).
Models like LLaMa, BERT, Stable Diffusion, Claude, and chatGPT were all trained on massive amounts of human-created data. And what’s unique about some, many, or most human-created expressions — especially those that happen to be fixed in a tangible medium a computer can access and learn from — is that they qualify for copyright protection.
Anderson v. Stability AI; Concord Music Group, Inc. v. Anthropic PBC; Doe v. GitHub, Inc.; Getty Images v. Stability AI; {Tremblay, Silverman, Chabon} v. OpenAI; New York Times v. Microsoft
Fortuitous as it may be, the data these models cannot survive without is the same data most protected by copyright. And this gives rise to the titanic copyright battles we are seeing today.
Of the many questions arising in these lawsuits, one of the most pressing is whether models themselves store protected content. This question seems rather obvious, because how can we say that models — merely collections of numbers (i.e., weights) with an architecture — “store” anything? As Professor Murray states:
Many of the participants in the current debate on visual generative AI systems have latched onto the idea that generative AI systems have been trained on datasets and foundation models that contained actual copyrighted image files, .jpgs, .gifs, .png files and the like, scraped from the internet, that somehow the dataset or foundation model must have made and stored copies of these works, and somehow the generative AI system further selected and copied individual images out of that dataset, and somehow the system copied and incorporated significant copyrightable parts of individual images into the final generated images that are offered to the end-user. This is magical thinking.Michael D. Murray, 26 SMU Science and Technology Law Review 259, 281 (2023)
And yet, models themselves do seem, in some circumstances, to memorize training data.
The following toy example is from a Gradio Space on HuggingFace which allows users to pick a model, see an output, and check — from that model’s training data — how similar the generated image is to any image in its training data. MNIST digits were used to generate because they are easy for the machine to parse, easy for humans to interpret in terms of similarity, and have the nice property of being easily classified — allowing a hunt of similarity to only consider images that are of the same number (efficiency gains).
Let’s see how it works!
The following image has a similarity score of .00039. RMSE stands for Root Mean Squared Error and is a way of assessing the similarity between two images. True enough, many other methods for similarity assessment exist, but RMSE gives you a pretty good idea of whether an image is a duplicate or not (i.e., we are not hunting for a legal definition of similarity here). As an example, an RMSE of <.006 gets you into the nearly “copy” range, and an RMSE of <.0009 is entering perfect copy territory (indistinguishable to the naked eye).
🤗 A model that generates a nearly exact copy of training data (RMSE at .0003) 🤗
To use the Gradio space, follow these three steps (optionally build the space if it’s sleeping):
STEP 1: Select the type of pre-trained model to useSTEP 2: Hit “submit” and the model will generate an image for you (a 28×28 grayscale image)STEP 3: The Gradio app searches through that model’s training data to identify the most similar image to the generated image (out of 60K examples)
As is plain to see, the image generated on the left (AI creation) is nearly an exact copy of the training data on the right when the “FASHION-diffusion-oneImage” model is used. And this makes sense. This model was trained on only a single image from the FASHION dataset. The same is true for the “MNIST-diffusion-oneImage” model.
That said, even models trained on more images (e.g., 300, 3K, or 60K images) can produce eerily similar output. This example comes from a Generative Adversarial Network (GAN) trained on the full 60K image dataset (training only) of MNIST hand-drawn digits. As background, GANs are known to produce less-memorized generations than diffusion models:
RMSE at .008
Here’s another with a diffusion model trained on the 60K MNIST dataset (i.e., the type of model powering Stable Diffusion):
RMSE at .004
Feel free to play around with the Gradio space yourself, investigate the models, or reach out to me with questions!
Summary: The point of this small, toy example is that there is nothing mystical or absolute-copyright-nullifying about machine-learning models. Machine learning models can and do produce images that are copies of their training data — in other words, models can and do store protected content, and may therefore run into copyright problems. True enough, there are many counterarguments to be made here (my work in progress!); this demo should only be taken as anecdotal evidence of storage, and possibly a canary for developers working in this space.
What goes into a model is just as important as what comes out, and this is especially true for certain models performing certain tasks. We need to be careful and mindful of our “back boxes” because this analogy often turns out not to be true. That you cannot interpret for yourself the set of weights held by a model does not mean you escape all forms of liability or scrutiny.
— @nathanReitinger stay tuned for further work in this space!
Unless otherwise noted, all images are by the author
Do Machine Learning Models Store Protected Content? was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.
