Using VQGAN+CLIP For Image/Video Generation

by Cade Brown <me@cade.site>


A very cool technology

If you’re in the AI space, specifically generative models, you owe a great deal of respect to VQGAN and CLIP. When combined, they can be used to iteratively refine images to more closely match a prompt.

These are some results from recent work on generating art using machine learning and artificial intelligence. Most of the following are using VQGAN+CLIP, or just CLIP alone.

I gave a presentation at UTK’s Innovative Computing Laboratory (ICL) that describes some AI art processes:

Short Square Morph Videos

mush2dna

Fungus morphing into DNA

CEB 0

My face morphing into polygons

AZH 0

My boss undergoing a revelation

Music Videos

(demo) ghoti - Towards Holier Places