Create A Better Reality Scan by Mapping High Resolution Color Images To Any 3D Model
By Polyga Support — November 17th, 2022.
Create A Better Reality Scan by Mapping High Resolution Color Images To Any 3D Model with Free Tools
Today we’re going create a better reality scan be adding seamless full-color mapping to a Monochrome 3D model scanned using a 3D scanner.
All we’ll need is the open source program Meshlab and this series of photographs of our model. I’ve been using this process in visual effects, and I find the control Meshlab gives you over every step to be worthwhile.
Here’s the end product. Look how well it’s blended the projections of each photo, and I can’t see any seams which makes the detail fantastic. We even picked a tricky model to really give Meshlab a workout, and you can see how well its weighting algorithms preserved the color details.
Let’s get Started on this Reality Scan!
With Meshlab open the first thing we need to do is bring in our 3d mesh. From the file menu select import mesh.
I’ll be bringing in this full res mesh to start. As you can see in the drop-down Meshlab supports many different 3D formats. Now depending on the file size of your mesh it may take a moment to load. Let’s try and rotate this model in the viewport with left mouse button drag. I can see that we’re only getting a few frames per second.
The layer panel we see on the right side of the screen can be toggled on and off using its icon on the tool shelf. Visibility of each mesh can be controlled by clicking on this eye icon. Let’s hide our full res mesh for now.
We’ll just use ctrl-h to reset the camera and we get our center back. Instead let’s use shift drag to zoom. This mode will not change the center of rotation. To change the field of view we hold down shift while using the mouse wheel. Let’s try that now. This will be essential for aligning our camera projections with our photographs. The current field of view is shown as FOV in the bottom left corner. I’m going to turn off the quick help from the help menu for now. It’s a good time to save our project file. Don’t forget to save often and feel free to version up, Meshlab scene files are very compact. Next I’m going to import all of the photos of the model you saw at the start of this video.
If things don’t seem to be lining up try changing the field of view by holding shift while spinning the mouse wheel. The lineup doesn’t have to be perfect as Meshlab will try to snap the view into place.
If you are unsure of the field of view in your lineup, try enabling optimized focal. If you’d like Meshlab to try even harder to snap the projection, increase the number of iterations above the default of 200.
Your alignment is now much closer with the models right arm and leg.
Before continuing let’s disable both raster alignment mode and show current raster mode by clicking their icons on the toolbar. Now when we rotate our camera view we won’t accidentally break our first photo’s projection. This is important to understand.
Make sure the second photo is selected from the list and repeat the process we did for the first photo. Remember the show current raster mode and raster alignment buttons on the toolbar.
As a hint, pay attention to the negative space in your silhouette. I found this really helped me with quickly lining things up. Also if all of your photographs were taken using the same lens, keep that in mind when you find a field of view that works for line up and start from that for each photograph. When you’re happy with the lineup don’t forget to snap the projection using the raster alignment on the toolbar. With our second photo aligned make sure to deactivate show current raster mode from the toolbar.
In the options panel we can set our texture resolution in pixels and name our texture file. The texture file will be saved in the same location as your project file.
To see the UV projected texturing we need to set our shading to none and our color to user-defined. Then make sure texture coordinates on the layer panel is set to on.
Up until this point we’ve been building color maps onto the proxy model. This keeps things fast until we’re happy with the results. When you’re ready to build color maps to the full res model, hide the proxy model in the layer editor and unhide the full res model.
When you’re happy with your results it’s time to export your color map to mesh from Meshlab.
Here’s the final result of our color mapped model.