Project 1: Compositing with Light and Shadow
To start off the quarter, our first project entails the integration of 3D CG elements with a live action plate, managing both light and shadow through compositing. Using Maya, we will be developing a CG scene to then be composited in Nuke, giving the illusion of an object rolling and interacting with live shadows.
Camera Match
The first step for the project is matching a CG camera to the live action plate. Using a reference plate that includes a cube, and metadata information from the camera like focal length, we can eye the alignment of the CG camera and cube to the reference.
Basic Lighting Setup
Our next step in building this scene is to begin developing the CG lighting. The live plate of course has a very intricate shadow that will need to effect our objects, but for now, we will begin with the basics: the keylight, or in this case the sun; the bounce light, or the ground; the HDRI, for accurate reflections and depth in light; and last the object's shadow, which includes ambient occlusion and the actual cast shadow.
​
To start, I used a spotlight to act as the sun. Using the grease pencil tool in Maya, I can draw a quick outline of where the bounds of the shadow are in the live plate. Manually adjusting the position of the keylight, I can then begin to eye the position of the light based on the shape of the shadow.
Now, I can begin tackling the next aspect of our lighting, which will be the bounce light. By creating a ground plane with a surface shader, then setting the color to the live plate projected from the camera, the color of the ground will now bounce from the CG ground plane onto the objects. As you can see from the images below, the bounce light gives a subtle yellow tint to the base of the spheres, as compared to a plane with a basic grey shader.
Default AiStandardSurface
Surface Shader with clean plate projection
With the bounce light added, we now need to bring in the HDRI. To create the HDRI, we would first need to photograph either a 360 degree view of the location with a special camera, or we can use an image of a chrome ball taken on site. In the future I would be taking these photographs myself, however for this project, the plates have been conveniently provided. In the Maya file, I then add in an aiSkyDome light and assign its color to the image of the chrome ball. The image will at first display incorrectly or flipped, so I change the format setting from the default, latlong, to mirrored_ball. By comparing the original image of the chrome ball to our CG chrome ball, we can eye the positioning to make sure the reflections in CG are accurate to the live scene.
CG Chrome Ball
Live Chrome Ball
After we've added the first iterations of the lighting setup, we can now begin setting up our compositing in Nuke. In order to make sure we have enough information for the comp, in Maya, we will first set up three render layers: Ambient Occlusion, Shadow Catcher, and Beauty. For the Occlusion layer, we will assign the objects an aiAmbientOcclusion shader. For the Shadow layer, after removing the primary visibility of the objects, we will assign the ground plane an aiShadowMatte shader.
Beauty Layer Render
Shadow Layer Render
Next, we will conduct some preliminary compositing in Nuke to make sure our integration is on the right track. With our clean plates and basic renders, we can merge in the shadow layer as well as the beauty, shown in the node graph below.
Live Sphere Shader Match
One part of our assignment is to create a match or replica of the sphere used in the live scene. In order to achieve this, we must first observe the live sphere and note what aspects will define the shader we create. Then, we can start to build up the shader layer by layer.
These are a few things I notice when looking at the sphere:
​
Base Color:
- Slightly warm toned
- Small scratches or markings
​
Reflectivity:
- Highlight somewhat dim on surface
- Subtle reflections
​
Bump:
- Two layers of brush strokes
- Circular pattern of brush
- Small indents and imperfections
​
Bump + Specular
Base Color - Scratches + Imperfections
Combined
Updated Render - Beauty
Updated Render - Shadow
Environment Shadow
After we have set up the basic lighting, we now need to tackle integrating the cast shadow seen on the ground, allowing the CG objects to interact with the environment more realistically.
Original Shadow
First, I brought the original plate into Photoshop and created a basic mask of where the shadow is, outlined in red. Next, I apply this image as a surface shader, projected from the view of the Render Camera.
Shadow Projected from Render Camera
I then go to the keylight, select its camera, then render out the view of the projected image from the angle of the light.
View of projection from Keylight Camera
Last, I can go into Photoshop and extend / clean up the image, extending the shadows across the plane. This ensures that when our CG objects interact with this element, and potentially leave the bounds of the previous version, their shadows will remain accurate and consistent.
Cleaned view of projection from Keylight Camera
After applying the same shader used for the ground to the 3D objects, we can observe how the shadow now interacts directly with the objects, accurate to the angle of the light. The red areas can be used as a mask to create the intricate shadows.
Rendered
AOVs for Environment Shadow Composite
Once we have a basic mask for the environment shadow that correlates to the CG objects, we now need to set up AOV layers for the render in order to use the mask properly. I noticed how the shadow mask wraps symmetrically around the object, as the shader projection is essentially guessing what is behind its view.
​
In order to have the environment shadow only impact the areas currently in the light, we can set up AOVs to separate this. Within Maya lights, for our case the aiSkyDome and spotlight, you can find a section under the Visibility tab called AOV Light Group*. By assigning a name to this setting, here being Sun and Sky, it effectively tells Maya what light we want to set up an AOV for. Typically, we can create diffuse direct and indirect AOVs within the render settings, but now, we have control over what light group is appearing in the AOV. By naming the AOVs diffuse_direct_Sun, for instance, we can now observe the direct diffusion isolated with the sun object.
Diffuse Direct Sun
Diffuse Direct Sky
Diffuse Direct
With our updated AOVs, we will now make adjustments to the Nuke script. Using a Shuffle node, you can separate the render into its separate AOVs, merging them back together after making appropriate edits to the layers. Which, in this case, we will need to use the previous environment shadow mask in order to grade the diffuse_direct_Sun AOV to produce the proper shadow. Then, using a "plus" merge, we can add together the diffuse direct, indirect, and specular AOVs. Here is what our current render is looking like:
As a test, I repeated the same steps as displayed before, this time with a different asset. In order to save a bit of time, I reused the model and texture map of the Sprite can from the SCAD x MILL collaboration. Here is the updated render, with small tweaks to the grade used to replicate the environment shadow.
To be continued!