Lightly edited from the recording for readability. Wording is the speakers’ own.
Today we'll be walking through the 3D node inside of FLORA with DJ. - Hey everyone, DJ, creative workflow specialist at FLORA. So today's mainly gonna be a quick walkthrough of 3D nodes, how to use them, where they are at and what they're capable of. And I prepared like a short walkthrough project for this today. This is gonna be sent out after the webinar.
So you guys can play around with it. I know this space is advancing in such a fast pace. And we also had 3D node out for a while. You might see a lot of crazy stuff on like LinkedIn and X. Yeah, eventually.
So my goal is to like after this, you can also have the capability to create those kind of cool stuff inside FLORA. We had 3D node actually out for like a couple weeks. And I've been experimenting with it. I found it pretty useful for a lot of different kinds of use cases.
But yeah, just in short, today's focus is gonna be visualizing products using the 3D node inside FLORA. First concepting on the image level and then visualizing them in 3D with the 3D node here. Yeah, first of all, you can generate 3D assets and import 3D assets if you have onto Canvas. And then eventually I'm gonna show something like this, like in-store product visualization video to video.
So yeah, using 3D space of the store and the product composite together in order to make these imaginary products that displayed inside of FLORA. We're also gonna touch a little bit on 360 video. I know this was a big issue with making with image models and video models, maintaining consistency of like the side and the back view of the same product was a big issue. 3D can solve it, which is really big. So the big chapters today are gonna be something like this, kind of just roughly wrote it down. First of all, I'm gonna touch on the basics of 3D node. What I mean basics is like model comparison. We have a couple, so it might be a little confusing at first, but yeah.
So seeing how each model does and the pricing and all that. Second chapter would be dealing with materials. For those of you who have a little expertise in 3D, I know PBR materials. There's such a big element inside working with 3D. We also support generating PBR textures, which is great news. The third chapter will be product visualization.
So actually getting into visualizing these products. Yeah, to break down the product visualization step, roughly it's gonna be something like working with the front side back images, visualizing them into 3D models. And then I included the step of Blender here, but it's not required. Because like agents and MCP is such a big wave right now, I did some experience with it, but yeah, it just hits off. It's not required. You can do it inside FLORA, but in order to do more precise control, I also wanted to show some steps inside Blender. The fourth step is gonna be store visualization.
So yeah, separately creating concept images of the store, visualizing those into a 3D space. And then using the FLORA MCP and the Blender MCP to render camera movements in quick white box, 3D environments, and then eventually placing the products and rendering the whole video with video models, video to video models. The 3D node is, you can find them inside the collections, the collection of our core nodes, text, image, video, audio, and other audio is gonna be 3D. So you can find the node here. Of course, if you spawn the node at first, it's gonna be empty, but you can choose the models from five groups here. We're constantly adding more, but yeah, this is the 3D node. The hot key is three.
So if you hit the number pad key three, there you go is the 3D node. You can plug in either a prompt or an image. I recommend you plug in an image. Yeah, just the same as, like making videos, it's better to do image to video instead of text to video, because it has a stronger visual reference.
So yeah, before we talk about the models, 3D, actually, I can show a ready made group of 3D nodes here. So I tried five different models using two images here. So yeah, just like creating videos and images, there are different kinds of modalities. You can start from one image and make it into 3D or two images or multiple images. The advantage of using multiple images, I would say is visual consistency and taking a close look closer to the product you actually imagine, because in this case, it's a coin. I wanted to make it somewhat like a token.
But, like 3D is literally three dimensions and this image can be like a can from the top or, there can be a lot of different ways to visualize this. Generating with 3D, it's kind of crucial to have multiple assets of the product. So I tested that capability with us by feeding in these two images into five different models. So we got meshy, we got rodent, we got high 3D, we got trellis, we got people. They are all accessible inside the menu here. Yeah. And inside the group, you'll see multiple different models.
But the only difference between these are what's labeled as multiple multi view takes multiple images as multiple different views. You can see the label multi view and other models as well. And all the models that are labeled multi or multi view, they can take two images and three images as different sides of the product. So, yeah, I'll show this one, for example, just to go over the very basic controls of 3D. Node, I'll actually show this one because the generation is better here.
So 3D node, as soon as you generate an asset, it might look a little different because we wanted to emphasize the full screen navigation control. It's default kind of hit. So the prompting bar and the reference image bar is kind of hidden. Yeah, if you hover over, you can always find them again. You'll see a lot of control inside this 3D node. First of all, there's these three buttons.
So you can switch between orbit, move, rotate, scale, just like in any other DCC tool. But if you're on a Mac and you have a tripod, I think this is pretty convenient. Instead of choosing these, you can like left click, drag around, and right click drag around to pan. And if you scroll, it's actually zooming in and zooming out.
So the built in controls are pretty convenient. If you're not used to that, you can also just yeah, if you want to actually move this product around, you can change the controls like this, rotate, yeah, these are also all resettable. You can reset the view or reset the transformation to go back to where it was in the very beginning. And this button right here, it's going to change the shading mode.
So yeah, if it's shaded, you'll see it is the best way to see texture. I like to put it to unload just to see just to check the details. You can also set it to wireframe and to check how detailed the mesh and polygons are. You can also import your HDR or EXR.
But yeah, that's a little bit about the settings. I would like to more go over these things. We have some buttons, taking a photo, taking a video and taking a four way, eight way, 16 way sheet. First of all, taking a photo, it literally means taking a screenshot.
So if you take a screenshot and the node is popping up somewhere else, but yeah, it will give you the screenshot in a separate node like this. So yeah, it's for taking snapshots of the product in different angles. You can do the same thing by clicking video. And yeah, this video button gives you a recording button. And you can for example, spin or like orbit around the product. And when you stop recording, it will give you the recording video in another node.
So yeah, those are some controls this four way, eight way. It's taking, it's like a macro of taking screenshots of a certain set of images. So it gives you four shots of front, side, back, and the other side. Sometimes that can be pretty helpful. Yeah, so it's a little bit about the basic controls of 3D node. The next thing I want to go over is a model comparison.
So I think that's the question. You guys might actually have. I did test it with these product images. I think it was a good thing to test because it has a shape, it has a dimension, and it has some details that it should follow. As you see here already, there are some differences between the models. When it comes to text or labels, it's a good way to stress test these specific models. Yeah, as you see here, the only thing that got it right was high 3D, which is actually apparently the most expensive model.
But yeah, I think it's a good knowledge to take in. Whenever you're working with detailed text or logos, high 3D tends to keep it the best. I left a note here, but that doesn't mean every other models are bad. When it comes to text, high 3D has its strength, but all the other models also are pretty good at preserving shapes. I would say the most cost efficient one would be meshy or trellis was also pretty good based on my experience. There might be some other differences if you test the materials and look into the details and all that. There's also some options.
Of pose, which is apparently for a character, but in this case is an object. I didn't really take that into consideration. There's a lot of things to test with 3D notes. Let me know what you guys think. That was a short comparison of each models. I think we can go over to the next chapter, materials. I know PBR materials are a big part of 3D.
And actually an image model called patina. If you search patina inside the image models, you'll get three options. All these models have to deal with materials either generating a new material, generating a new material with tiling options and extracting a material from an existing photo. Those are the three options here. The one on the bottom patina is just generating PBR materials. Patina material, it says it in the description here.
This one is tileable. I'll show you in a minute what that means. Patina material extract is a very specific image model for extracting a material from an image. This is the example that I want to show. I have this Tokyo Hermes store and I worked on a project on a mock project on this a couple of days ago. I just brought this image. The good thing about patina material extract is it's an image node and you can literally just connect it and tell what to extract from it.
I just said glass wall. I wasn't being really specific, but it really did a good job on focusing on only the glass wall here. It generates the normal map and roughness, metalness and albedo and everything. That was pretty interesting to play with. You can tweak the tile size. There's some other upscaling options. Right now, I just didn't put it to upscale, but if you want high quality material, you can always put it to two times, four times.
The thing I wanted to show was the tiling option where you can see when you full screen material. Once you full screen, there's going to be a button that looks like a tile next to the zoom buttons. Once you enable a tile, you can see the material in actually like a tiled way. This is pretty cool.
This might be a good preview of what this would look like if it's expanded and applied on an object. Another view I want to show is 3D preview. This is pretty cool. Just like you see in other material libraries on the internet, you can put it on a sphere to actually see how it looks like on an object. You can see the actual texture and reflections and how bumpy it is and everything. Also, in the non-full screen mode as well, you can navigate on a node as well. Same with the wooden form of material.
This time, I provided this image. Once I can full screen, I can see all the maps. Another thing I wanted to talk about-- sorry about that. Sometimes that happens when the canvas is overloaded. Another thing I want to talk about materials is we have this material. You can re-texture something.
This comes from the token 3D model that I already created. You can see that this is referencing the token, the good generation that I had previously. I plugged this into another 3D node and I plugged in an image as well. I set the model to re-texture, which is changing the texture of the existing model. Re-texture models are also in Meishi as well. Currently, these two are the re-texturing models from Microsoft and Meishi. If you provide the material, you can change the material.
You don't even have to prompt. Previously, it was the flora metallic coin. Now, it changed into this mineral type material. This is pretty cool. I also did a model comparison here for you guys. In case you're curious between difference of Meishi and trellis, they're both re-texture models. Apparently, trellis does a better job.
I provided the same material, but Meishi-- I don't know what's happening here. So far, my findings were for re-texturing, trellis is the best. And for generating multiple views, 3D is the best. Another thing I want to go over materials is I tried generating 20 different materials using the batch node. We have different kinds of mineral style materials. I plugged everything into 3D node and ran everything 20 generations inside the same 3D node, which was pretty cool.
So you can see how each texture shows on the flora token. You can set it to unload it to check the details. Yeah, so I think that's enough for materials. The next chapter I want to go over is 3D product visualization.
So yeah, 3D product visualizing part is here. I first of all started with these five images. The structure here is you can just look at one group. I just did the same thing five times for five different products.
But yeah, they're all the same workflow. And you can just focus on this one if it looks kind of confusing. But yeah, I started with a single product. I plugged it into this technique that I made. I just named the product multi-view. What it actually does is it takes an original product and runs through these four nodes and makes a back view and a top view. I didn't include a side view because most of the times when I try generating a side view, it gets it wrong.
Or there might be a way to use the generative models together a little bit in a better way. But I find it still pretty useful if I only have front, back, and top. And that's enough for generating our product. So yeah, I put it into this technique. I did the same thing for the other products as well.
So yeah, it generates a back view and a top view, front view, back view, top view, front view, back view, and top view, front, back, and top. So yeah, I found it a pretty useful technique, not only for 3D, but yeah, these were the images that I generated I used the same multi-view 3D model, but this time I used meshy because it didn't really have text. So yeah, this can tell that you don't always need to use the best possible model, which was apparently high 3D, but this is expensive and it takes a lot of time. So for this one, because it didn't have any logos or text on it, I just chose meshy for this, which was like $1.4. I can, of course, play around with the settings here to make it more detailed.
But yeah, that would take more time and the cost will go up. I think this was enough because eventually it depends on what you do. But what I'm trying to do here is not-- the final output is not going to be a 3D. It's going to be in video, so it's going to be 2D.
So these were just for reference. So I made the decision depending on the final output, I guess. So yeah, these were the 3D visualizations of the products. I think it did a pretty good job because the base image was a single front view image. And using three images, it got to a pretty detailed level. And of course, if the thing I most care about was materials, I should have run this through a re-texture model to make it more glossy and make it more reflective and mineral-ish.
But as I said, these were just for shaping references and product placement. So I just stopped the process here. And yeah, shape-wise, it pretty much resembled the original product. So I was pretty satisfied with all five of them. Yeah, for this one, for some reason, it came out pretty dark.
So I had to change it to unlit to see the actual details. And yeah, I think it did a pretty good job. For this one, the only thing I would care about is the sphere inside here. The materials are pretty delicate. And yeah, I think it kind of captured that. Oh, one thing I forgot to mention about the basic controls is the cool thing about 3D, if you full screen it, you can control the lighting. It's actually an unlit, so you can't-- So yeah, you can control where the light is at.
So yeah, there's a directional light baked inside, just like control the light in Unreal Engine or Blender. Sorry, I didn't see some cool lighting shades. Yeah, so this was the last product. Manishii kind of did a good job on replicating like very micro detailed carvings, engravings. I'm not sure what to call these.
But yeah, so now I got my five products visualized into 3D products. I then, first of all, took these to generate 360 videos. So first of all, turntable videos. How people did turntable videos, I think, is either just provide a single image to I don't know, like SeaDance or Cling or Google Omni and prompted to make like a 360 orbiting rotation camera movement or something like that, which is very not precise. Because technically, it only has one image of the product. And the video model is imagining what it looks like in the backside. And the side view.
Yeah, I mean, sometimes it can work. But in terms of precision, it's like the worst way to do it. A better way to do it actually is creating multiple sides, like front view, side view, and back view, like 90 degrees, 180 degrees. And then using those as first frame, last frame, and like stitching those videos together, that might be a little more accurate.
But it's really hard to control the speed of the camera and the timings of when to turn and stuff like that. Another way can be like plugging into an object that's already rotating and using that as a motion reference and plugging in the actual images of the product. But yeah, that can also be like that can also end up of the video model hallucinating. So I would say the most accurate way is to make a 3D model that somewhat resembles the original product and plug that as a reference video.
So that now I'm not even using SeaDance 2.5 and SeaDance 2.0, which are the most expensive video models out there. Minimax H3 is a pretty decent model with like almost 10%, 20% of the price. SeaDance 2.5 can get crazy expensive. Sometimes you can get like 20 something dollars, but yeah, Minimax is a pretty good option. The way I generate these is this is where the blender step comes in. I know some of you guys might have been playing around with MCU and agents. We have an open and agent button on the top of the canvas.
So this lets you open FLORA inside Claude, ChatGPT, cursor, whatever agent app you're using. The benefits of that is you can let your agent drive FLORA, but not only drive FLORA because like ChatGPT Claude is a third party harness it can also drive other tools as well, like just like blender or other creative tools if they have the MCP enabled. So that means that you can use like ChatGPT or Claude to orchestrate these different kinds of tools. And we're going to have a separate webinar on this later, but yeah, just to give you like a quick preview, it's using ChatGPT or Claude to for example, make 3D videos inside Blender, take that and import into FLORA. Use that as a reference and do all the generations inside FLORA.
So these kind of things can be possible. The benefits of that, it's pretty obvious. Like Blender obviously is a better 3D tool than whatever FLORA can be. And just, yeah, like utilizing that benefit from Blender, utilizing some other benefit from another creative tool. And combining all those good parts. That's what MCP is, an agent is.
So yeah, that's what I did here. By using Blender MCP, I created these very stable, very precise 360 animations. This is what's happening in ChatGPT. The way I asked is, first of all, I copied and pasted the FLORA project link so that ChatGPT also has context to it.
And then I, first of all, downloaded the relevant GLB files of the products and placed it inside my local folder, just so that ChatGPT can look at it. And I just requested make a 360 turntable animation in Blender. And once you're done, place them back inside FLORA. As you see here, it worked for only nine minutes and it finished all the 360 videos in 1080p, 24 frames per second, in each eight seconds. You can change these settings. You can also make this into a skill to like reuse it later.
This was the first question I asked ChatGPT. And then it created the five videos. I'll go back to the canvas. So these were the five videos that Blender created. The exact process I made, the turntable renderings are, first of all, I plugged this original video from Blender into Minimax H3. And you can see the references here.
This was the 3D video. I plugged in the actual product image and a background reference. I wanted the product to be standing inside this background. And I took the product image from here and if I combine all three of them with, this text note is actually just holding the prompt. It's saying camera orbiting around the object, place it on the background. I put in shallow DOF, which means shallow depth of field, just to get some cinematic effects. And I added a guideline of keep these architecture and visual details of the object, not to like change anything from the original object.
So yeah, this way I can achieve precise control of camera movements and object placement. And I only tried these multiple tries were actually trying different models. But yeah, if you get the settings right, I can say that I can guarantee that you can get in like a couple of tries of one or two tries. And yeah, just for like comparison, I also did like CS2.5 and other models as well. Minimax H3, it was pretty decent with a fraction of price. I use this text note. I mean, this static text note on the holding the prompt to provide this to all five videos.
And yeah, you can see I did the same thing for all five videos and created the same effect, camera orbiting 360. So these are the kind of things that without using a 3D reference video would be very tricky, because, it's hard to get the actual shape of the product, like this angle, like this part of the detail on the top of the product on the bottom of the product. Imagine you being prompting this video, it will take like 10s and 20s generations. But yeah, because I have, because I already have a 3D reference here that already resembles the product, I can just do like one generation. It was actually just one Minimax generation, I compared it with CS2.0 and CS2.5. Yeah, CS2.0 did a pretty good job. Surprisingly, I liked H3 better than CS2.5.
So yeah, that was about product 360 videos. The last part I want to show today, I first of all created the showroom concept images. So because I eventually wanted to place this, place these products inside a showroom, I first of all started with some concept images. So I have four showrooms here. They're all showing like just one aspect of the room.
But same as the objects, I needed multiple angles. So yeah, I used this technique. It's actually kind of the same technique that I used over here, but just named it differently. And the prompts are a little different. The structure is the same.
So it takes an initial image, splits it into two different angles. Actually, sorry, three different angles. And it just generates alternative views, shot sizes, and angles of the same scene. So yeah, I use this technique to generate different angles of the same scene here.
So yeah, because I wanted to eventually make these into 3D scenes, I needed multiple assets. So yeah, we have a shot from different angles, a shot from close up. And eventually, what I did is before I go on to the next step-- oh, I shall show the outputs first. So these were the actual outputs. And I'll show the process as well.
So what I wanted was close up footages of each products. And this was interesting because it's taking two products in one video. So these were the close ups. I also wanted a dolly in footage of each showroom to show all the products at once. And the interesting thing here is I actually color coded them because it's very tricky to get three different products in one shot and make them look all accurate.
So I was having a color coded video. And the prompt here-- I'll show this later. So first of all, what I did is I took this to tragedy first. And using the MCP, the first question I asked was, make the set of images called showroom constant images into four different blender previous scenes. Yeah, it worked for six minutes. It created a blender scene with all four locations because I had four sets of images. And I asked for 720p preview videos.
Yeah, just in case if you're curious, I just set it to 720p in 24 frames per second because it's fast and it's not heavy at all. It's no good to generate 4k videos if it's just a preview, if it's just a white box previous. You don't need that much. 720p is enough. The only point here that I want to refer to is camera movement.
So it can be OK even if the video is low quality. The next question I asked was product placement for each background. Yeah, I specifically requested for scene one, scatter all five of them because I have five products. Scatter all five of them evenly everywhere.
But for scene two, scene three, scene four, it has limited space. So I can't put all five products. So I had to choose one to three, two to five, and two to four on each pedestal. So that was the product placement. I asked for two types of footage. The first one was dialing in. Second one was close up.
So that's what I showed you as the results. And here I asked for five seconds, 720p. And it worked for seven minutes. Again, it rendered out all the videos. You can also preview the videos inside China GPT, which is pretty amazing. I ended up not using this video because it had too many products. I just wanted to show these that has maximum four products inside.
They turned it out pretty interesting. So you see it has a low-poly preview of what the scene looks like. And it has the actual products composite inside. I thought this was good enough.
But as I said, the problem was-- in order to place the actual products accurately, if it has more than two products inside the same scene, it's really hard because you have to place product one on the left side of the image, place product two on the center, place product three on the top shelf of room number three or something like that. It's going to be like a whole paragraph. So I didn't want to do that. I did another iteration. I was let's change the plan. I actually want to color code these and make the products from one to five respectively in the colors of red, green, blue, purple, and yellow. These were just arbitrary colors, but it's good to have these color coded and be very different colors in order to-- let the video model distinguish each products.
And I just asked for rendering the Dalian videos-- only the Dalian videos again. Because the close-ups were fine. They're only just one part of each. The only problem was happening in Dalian.
So that's what it rendered on the second time. Now it actually has the colors applied to each product. That was what I did inside ChatGPT. And because I'm using MCP, I can let ChatGPT to put the videos on the canvas, which was pretty inconvenient. I didn't have to search for all the videos and drag them onto the canvas myself.
So these were the color coded videos that I made in Blender. And the way I prompted them is I, first of all, added all the relevant products here. And I mentioned each product. And I was place number two in green, number three in blue, number four in purple, and number five-- oh, sorry, number one in yellow. And I had the same guidelines-- exact placement, camera movement, follow, do not distort. Yeah, that's how I achieved these videos. To be honest, it was pretty impressive.
I think it was easy enough for Minimax to follow because they were already in very different colors. Same for the close-ups. The close-ups were actually easier because-- they don't have to deal with product placements. And it already shows a very accurate preview of what the product looks like.
This one got it quite wrong, I just realized. But yeah, you can always change the prompt a little bit or regenerate it. And by the way, these are all Minimax H3. I didn't really end up using the CDENCE 2.5 that much.
So yeah, I guess that's it for what I prepared for the webinar today.
