How to create new memes using images from Wikimedia Commons and OpenImages with captions generated automatically by GPT-3 and GPT-Neo
In this article, I’ll show you how I built a system called AI-Memer that generates memes using the latest AI models. I start with a high-level description of the system components before getting into the background of memes and details of the components. I’ll then show you how to generate your own memes using the Google Colab, here. After a brief discussion of results and next steps, you can see some sample memes in the appendix. Oh, and I’ll show a newly generated meme at the head of each section 😄.
Hands-on Tutorials
How I used CLIP, SWAGAN, and a custom genetic algorithm to create modern paintings from text descriptions
My latest project in using AI for creative endeavors is called MAGnet. I built a custom Genetic Algorithm (GA) to drive the creation of modern paintings using a Generative Adversarial Network (GAN) from a textual description over several generations. MAGnet uses the CLIP model from OpenAI [1] and a variant of StyleGAN2 ADA [2] from Nvidia called SWAGAN [3], which uses wavelets to create images.
All of the source code for this project is available here. A Google Colab that you can use to create your own paintings is available here.
Prior Work
How I trained a GAN using public domain paintings
For the past three months, I have been exploring the latest techniques in Artificial Intelligence (AI) and Machine Learning (ML) to create abstract art. During my investigation, I learned that three things are needed to create abstract paintings: (A) source images, (B) an ML model, and (C) a lot of time to train the model on a high-end GPU. Before I discuss my work, let’s take a look at some prior research.
This is the first part of my series of articles on how AI can be used for creative endeavors. …
What Latif Nasser didn’t tell us about the “First-Digit Law” on his Netflix show, Connected.
If you haven’t seen it yet, check out the Netflix series, Connected. It’s a good show. The host, Latif Nasser, discusses various subjects in popular science. Netflix touts it as a series “that investigates the surprising and intricate ways in which we are connected to each other, the world and the universe.” [1]
In Numbers, the fourth episode, Latif explores Benford’s Law (BL), also known as the First Digit Law. It’s an observation that many datasets, both man-made and from nature, contain more digits that start with the number 1 than any other digit, about 30% of all numbers. And…
How I used Adaptive Discriminator Augmentation and Learning Transfer to generate improved abstract paintings with AI.
Back in August 2020, I created a project called MachineRay that uses Nvidia’s StyleGAN2 to create new abstract artwork based on early 20th century paintings that are in the public domain. Since then, Nvidia has released a new version of their AI model, StyleGAN2 ADA, that is designed to yield better results when generating images from a limited dataset [1]. (I’m not sure why they didn't call it StyleGAN3, but I’ll refer to the new model as SG2A to save a few characters). In this article, I’ll show you how I used SG2A to create better looking abstract paintings.
MachineRay 2
Overview
Similar…
How I used GPT-3 to reimagine classic songs in the style of Bob Marley, Radiohead, and Megan Thee Stallion
OpenAI’s third-generation transformer, GPT-3, is currently the state-of-the-art model for generating text [1]. It can create high-quality text given only a few examples. I wanted to see if I could prompt the system to adopt the writing style from one writer and transfer it to another writer’s work.
For this experiment, which I call VoxMorphia, I chose to transform songs that are in the public domain, including classic tunes like “Yes Sir, That’s My Baby.” …
Hands-on Tutorials
How I trained StyleGAN2 ADA with 5,000 Impressionist landscape paintings in the public domain
This is my third article on experimenting with Generative Adversarial Networks (GANs) to create fine art. The first two articles focused on creating abstract art by using image augmentation, but this one focuses on creating Impressionist landscape paintings.
I posted all of the source code for GANscapes on GitHub and posted the original paintings on Kaggle. You can create new landscape paintings using the Google Colab here.
Prior Work
There have been several projects and papers that show how to use GANs to create landscape paintings. There’s Drew Flaherty’s master’s thesis from the Queensland University of Technology entitled “Artistic approaches to machine…
Using natural language to search and find over sixty thousand industrial designs that have entered the public domain since 2002
Everybody knows about using patents as a way to protect new inventions. Unless you are a patent attorney, you probably only know about one of the three major types, utility patents. However, there are two other types, design and plant patents. Currently, there are over 10 million granted utility patents, 910 thousand design patents, and almost 32 thousand plant patents.
If you guessed that plant patents are for new varieties of plants, you would be correct! But what are design patents? Here’s the definition from the US Patent and Trademark Office:
“The patent laws provide for the granting of design…
How a neural network trained on the US Patent Database can produce new and possibly useful inventions.
In July 2019, an inventor named Stephen Thaler filed for a patent for a warning beacon that flashes in a special way to get people’s attention. On the patent application form, Thaler listed himself as the applicant and the assignee but not the inventor. Who invented this improved beacon? A better question is what invented this improved beacon?
The answer is DABUS, which stands for “Device Autonomously Bootstrapping Uniform Sensibility” [1]. This is a neural network that Thaler created and trained to invent things. (Brief side note: the improved beacon, in turn, uses a neural network to attract the attention…
How Robert Frost’s words can breathe life into computer-generated music
Since January 1, 2019, many of Robert Frost’s poems have entered the public domain [1], including works like “Mending Wall” and “The Road Not Taken”. In this article, I’ll show how his poems can be used to improve the structure, phrasing, and emotional content of computer-generated music. For example, here is the first verse from one of Frost’s more famous poems from his book, New Hampshire [2].
Stopping by Woods on a Snowy Evening (excerpted)
Whose woods these are I think I know.
His house is in the village, though;
He will not see me stopping here
To watch his woods fill up with…
Robert A. Gonsalves
Robert A. Gonsalves is an artist, inventor, and engineer in the Boston area.
.jpg){kind=link}
.jpg){kind=link}
