AI‑Generated Artwork with CLIP‑CLOP
Research Paper by Piotr Mirowski, Dylam Banarse, Mateusz Malinowski, Simon Osindero, Chrisantha Fernando (Deep Mind London).
All of the collages you see were generated with CLIP‑CLOP (Mirowski et al.), run in an open‑source Colab notebook on a TPU runtime for rapid iteration.
The code is avibale here:
https://colab.research.google.com/github/openai/clip/blob/master/notebooks/Interacting_with_CLIP.ipynb
I honed each composition by crafting prompts that blended philosophical keywords (“shadow,” “enlightenment,” “atonal texture”) with visual descriptors, then watched CLIP loss metrics and intermediate renders to tweak learning rates and patch‑placement randomness over 500–1,000 optimization steps per artwork.
Image Generation Process Image Generation Process
Behind each collage lies a custom library of roughly 300 PNG “patches”—archival cave photos I created using scanned artifacts, urban textures, and organic elements—standardized to 512×512 px with transparent backgrounds. I fine‑tuned CLIP‑CLOP’s patch‑placement network for ten epochs on a representative subset of these patches, saving and evaluating multiple checkpoints until I found the sweet spot between coherence and creative abstraction.
Live Installation & Audience Interaction
The gallery transforms into an VR installation: I arrive early to dim the lights, arrange flickering LED candles along the floor, and position the large display screen with its soft‑panel uplights.
Controllers rest on a pedestal beside a headset, inviting each visitor to step in and guide their own journey. As they don the headset and grip the controllers, the virtual world springs to life—first revealing the cave, then seamlessly transporting them into the bright, modern hall of AI‑generated art.
Ambient audio—from dripping water to atonal musical motifs—surrounds them via hidden speakers, heightening the sense of presence. When they emerge, a row of printed miniatures on a nearby shelf offers a tactile recap of the AI collages they’ve just explored.
All of the collages you see were generated with CLIP‑CLOP (Mirowski et al.), run in an open‑source Colab notebook on a TPU runtime for rapid iteration.
The code is avibale here:
https://colab.research.google.com/github/openai/clip/blob/master/notebooks/Interacting_with_CLIP.ipynb
I honed each composition by crafting prompts that blended philosophical keywords (“shadow,” “enlightenment,” “atonal texture”) with visual descriptors, then watched CLIP loss metrics and intermediate renders to tweak learning rates and patch‑placement randomness over 500–1,000 optimization steps per artwork.
Behind each collage lies a custom library of roughly 300 PNG “patches”—archival cave photos I created using scanned artifacts, urban textures, and organic elements—standardized to 512×512 px with transparent backgrounds. I fine‑tuned CLIP‑CLOP’s patch‑placement network for ten epochs on a representative subset of these patches, saving and evaluating multiple checkpoints until I found the sweet spot between coherence and creative abstraction.
Live Installation & Audience Interaction
The gallery transforms into an VR installation: I arrive early to dim the lights, arrange flickering LED candles along the floor, and position the large display screen with its soft‑panel uplights.
Controllers rest on a pedestal beside a headset, inviting each visitor to step in and guide their own journey. As they don the headset and grip the controllers, the virtual world springs to life—first revealing the cave, then seamlessly transporting them into the bright, modern hall of AI‑generated art.
Ambient audio—from dripping water to atonal musical motifs—surrounds them via hidden speakers, heightening the sense of presence. When they emerge, a row of printed miniatures on a nearby shelf offers a tactile recap of the AI collages they’ve just explored.
.