Where the timeless art of pointillism meets cutting-edge AI technology
Experience an innovative fusion of art and AI, where AI meticulously optimizes the placement, size, color, and opacity of 3D Gaussian points to harmonize with multiple 2D perspectives. These volumetric, multi-directional art pieces bring depth, vibrancy, and movement to traditional art, transforming it into a truly futuristic and immersive experience.
Generative AI has demonstrated significant potential in creating images that mimic traditional art. With this project, I wanted to explore how AI could transform traditional artworks into an entirely new experience.
The technique I employed is a modified version of 3D Gaussian Splatting, a cutting-edge AI-based method for reconstructing 3D scenes from multiple images, replacing anisotropic splats with isotropic points. Instead of using real-world photographs, I curated a set of related artworks, each serving as a constraint in a given viewing direction on a volume of randomly distributed 3D Gaussian points. Through iterative optimization, the AI refines the position, size, color, and opacity of these points, aligning them with the input artworks from their respective perspectives.
Unlike traditional pointillism, which relies on the optical blending of static dots on a 2D surface, this method constructs a dynamic 3D composition, where points shift and adjust based on perspective, introducing depth and movement that conventional art lacks.My role in the creative process involves tuning hyperparameters and experimenting with different loss functions to guide the AI toward generating the desired outcome. Additionally, I instructed the AI to minimize the number of points, encouraging it to discover patterns and connections among the artworks. This also raises intriguing questions about the role of AI in creative expression, as the system autonomously refines itself toward artistic cohesion while being guided by human input.
I deliberately chose artworks with similarities in color, style, and subject matter—not because the AI requires them, but because they add meaning for human perception. It’s fascinating to observe how the AI can transform even non-impressionist pieces into impressionistic renditions.
I also experimented with generative AI to create the input artworks. Impressionistic pieces generated by Midjourney were incorporated into the 3D Gaussian optimization process, allowing one AI model to feed its output into another, effectively completing the creative cycle through generative methods.
To enable interactive exploration in a web browser, I adapted a Three.js-based Gaussian splat renderer. This renderer automatically cycles through predefined viewpoints while allowing users to navigate freely and explore the 3D artwork from multiple perspectives.
Intermediate images from an optimization session with input artworks acting as directional constraints on a volume of 3D Gaussian points.
An AI-generated artwork is used as input for the optimization—generated result can be found in the gallery. Image credit: PromptDervish Pastiches
"Very interesting and and exciting work ! This is a very original way of combining the 3D volumetric nature of 3DGS and artistic images. I really like the 'volumetric mix' of different paintings of the same artist."
— George Drettakis, principal investigator of the original 3DGS paper.
Beyond online viewing, I am exploring the potential of 3D laser engraving to bring these AI-generated volumetric artworks into the physical world. The input can be photos or drawings, making them ideal for souvenirs, special occasions, corporate gifts, and product promotions. If you own a 3D laser engraving business, I'd love to connect and explore potential collaborations!
Can you uncover all four hidden images cleverly concealed within the point cloud?
Explore from different angles and see what secrets emerge!