“Ever since OpenAI released the weights and code for their CLIP model, various hackers, artists, researchers, and deep learning enthusiasts have figured out how to utilize CLIP as a an effective “natural language steering wheel” for various generative models, allowing artists to create all sorts of interesting visual art merely by inputting some text — a caption, a poem, a lyric, a word — to one of these models.”
A few nights ago, I was scrolling through Kevin Kelly’s Twitter page and noticed that he was sharing a bunch of art and art-related posts. This one, in particular, caught my eye given a newly discovered passion for AI/generative art:
Although I was aware of CLIP & NCA models from Magnus Petersen’s (@Omorfiamorphism) work, I had focused most of my efforts in the AI+Art space on generating static images using Neural Style Transfer (models that combine content + style to create images like the one below).
I like Kelly’s coinage of describing how a human interacts with these AI as “whispering.” Does that make those of us working with these models ‘AI Whisperers’?
I suppose time will tell.
My first attempts at “whispering” or “steering” a CLIP model didn’t pan out so well, but I decided to try again as I suspected there was a lot still to learn.
I’m glad I did, because this time I had much better results:
As @sea_snell points out in his excellent overview of the emerging art scene from June 2021, Generative Art might represent the early days of a whole new art movement with an almost alien, and yet, still resonant aesthetic.
While the ‘black-box’ nature of AI presents legitimate risks in many areas of human endeavor, within the arts, it may prove to be a powerful tool for serendipitous, co-creation.
By “jamming” with these models, it’s possible to create truly incredible images.
The possibilities are truly mind-blowing
