No, we don’t need ‘Text To 3D’ yet… but there definitely are AI features that could upgrade your 3D modeling and rendering workflow.
I hate to phrase it this way, but it does seem like AI has somewhat upended photography. If you remember, last year’s Sony World Photography Award witnessed a unique scandal – the winning image was generated using AI. While that event shook the photography world to the core, everyone and their grandfather knew the 3D industry was the next to face an AI wave. GenAI tools have been very good at disrupting the 2D image world – the internet is overwhelmingly 2D, and is filled with hundreds of trillions of images that serve as a nifty database for these AI tools – for 3D, not so much. Sure, companies are working on developing 3D-specific AI tools, but we’ve got years before an AI tool can accurately create stunning 3D models from scratch.
However, fearing the potential of AI disruption in the future unfairly undercuts its usefulness in the present. For 3D modelers and renderers, AI holds a LOT of potential in upgrading your workflow. The only problem – not a single major rendering software has embraced any AI feature yet (barring just a tiny handful). However, here are a handful of powerful AI features that rendering software like Blender, Octane, V-ray, SketchUp, and KeyShot should absolutely add to their arsenal. Until then, I’ve also made the liberty of providing alternatives.
AI Environment Generation (Alternative – SkyBox.ai)
What most first-time designers don’t realize with product rendering, is that the product itself is just a small part of the picture. You still need to design or compose a background or scenario in which to place your product. A car needs a nice empty road or lighting studio, a vase needs a mantelpiece, a chair needs a living room, etc. You could spend hours and quite a bit of money finding 3D assets for your scene… but what if you could just ask AI to create a nice HDRI environment for you instead? Need an empty studio, ask the AI. What about a cyberpunk neighborhood? Simply ask the AI! GenAI could make rapid environment generation incredibly easy, allowing you to compose and render in minutes instead of hours or days. Although rendering software haven’t implemented such an idea, a website called SkyBox by Blockade Labs lets you create and even edit 360° environments using AI, and then download and apply them in your rendering scene.
AI Background/Backplate Generation (Alternative – Midjourney)
A 360° environment is one thing – what if you need something less complicated? Like a nice gradient for your product’s background, or some wacky splashes of water to make your rendering of a juice box look fresh, or perhaps just a nice model hand that you can then edit to make it look like it’s holding the product. This is perhaps the easiest application of GenAI in 3D product visualization. I find myself gravitating to Midjourney for its sheer realism, but there are a WHOLE bunch of free alternatives like Stable Diffusion, Leonardo.ai, even DALL-E inside Microsoft Bing. If you’ve got some money to spare, Midjourney is a good option, but Canva’s AI tools are pretty neat too.
AI Render Upscaler (Alternative – Magnific.ai)
This one is an absolute no-brainer. Imagine this – you’ve got just half an hour before a meeting or pitch, and the client’s asked for a new round of renders. Each render at maximum resolution takes a good 5-10 minutes, leaving you with hardly any time on hand. What if there was a way to just quickly upscale low-res renders? AI could absolutely help, and could do a phenomenal job too. Take your low-res 800px-wide render and feed it into an AI upscaler; just one minute later, you’re presented with a render that’s 2400px wide, enough to present to a client on a large screen, or even have printed! There are a few upscalers out there – Topaz Gigapixel (for Windows and Mac) and Pixelmator (for Mac) come to mind, although if you have some extra cash to spare, Magnific.ai is the gold standard. Not only does it upscale your images, it lets you upscale them with text commands. You can turn a blurry render into a steampunk image, a realistic visualization, or even a watercolor painting. Pretty cool, eh?
AI Material Generation (Alternative – WithPoly)
Most rendering software come with a preset material bank and most designers make the mistake of only working within the parameters of those presets. To truly unlock great rendering and visualization, you HAVE to step out of your comfort zone and build your own materials from scratch. Whether it’s making realistic wood-grain, fabric, or even woven carbon fiber – never use the default materials that come free with your rendering software. You could go to websites that offer material assets (PolyHaven and Quixel Megascans are great for free stuff), but you’ll end up stuck if you can’t find the exact material you need. So, what if AI could make materials FOR you? Imagine asking the AI to make pink carbon fiber, or glowing hot metal, and having all the image and texture assets ready for you. Sure, it would be nice to have rendering software with this built-in feature… but until then, WithPoly is a great alternative. With a freemium tier, you should be able to generate pretty decent 2K-quality textures and materials simply through text. The AI creates diffuse, bump, displacement, specular, AO, and a host of other perfectly matching and seamless texture files, all for free. If you want to upscale your textures or download them in different formats, you’ll have to buy credits, but for beginners, the free version is more than capable.
AI Render Style-Matching (Alternative – PhotoShop AI Suite)
Picture this – you’re scrolling through Behance or Pinterest (or even Instagram) and you see a wonderful product photo or render. The lighting is perfect, the background seamless, and you love the composition. You wish you could create such an image too, but it isn’t so easy. What if you could render a basic image and have the AI match it to the photo you liked? Well, such a tool doesn’t exist yet, but Photoshop‘s AI features allow you to achieve similar results. The software’s new AI tools include a generative fill feature, which lets you fill in selections with AI-generated objects. You could load a photo of a face, select your eyes, and have the AI generate sunglasses. Or change your clothes to a tuxedo… but more importantly, you can now even add reference images to tell the AI what to generate, or use Photoshop’s newly introduced Neural Filters to directly perform style-transfers. Imagine seeing a gorgeous photo of a perfume bottle and wanting to recreate the scene with your own 3D model – all you do is render your 3D model with flashy lighting, and add the background to your image using the reference feature. It’s a lot like using Midjourney to generate backplates, but what Photoshop does so well is integrate subject and background seamlessly. Photoshop will match your product’s shadows on the floor, create the right glare or caustics, and make a fairly believable-looking render in probably a minute or two!