Masking and Rendering

Week: 6 – 9 December

During the final week of my project, there were still several tasks need to be done which are rotoscope masking and rendering. I have finished the character animation in all tracked footages last week. So based on the animation, I’m able to identified which shots and areas in the footages that need the masking treatment to make the objects that supposed to be in front to appear on top of the 3d characters.

There are 4 shots that required masking; Shot 01, 02, 03 and 10.

Shot 01: Requires masking for the traffic light pole & several pedestrians

Shot 02: Requires masking for the board, poles and straps

Shot 03: Requires masking for the trash bins, benches and people

Shot 10: Requires masking for the pole only. The easiest one! (I think)

Tutorials

I’ve never done any rotoscope masking using Blender before, so I watched several tutorials from YouTube on how to do a proper masking. I think the tools and method are almost similar to After Effects which I’m quite familiar with.

Shot 03

Shot 03 probably needed the most number of masks when compared to the other three, but nevertheless I started with this shot first as I was eager to see how to result will turn out. I began drawing the mask for the bench and because it has holes at several areas, I just made and combined multiple masks as a quick solution without using another mask to cut the hole similar to the ‘pathfinder’ tools in Illustrator.

I then continued drawing several new masks for the other bench, tree, trash bins, and people.

All masks needed for this shot has completed for frame 1, but the camera is actually moving and panning throughout this shot. The problem was the masks were not moving together and only stay at the same spot. I could animate the mask shapes frame by frame but it will take too much time. Good thing is this footage was already tracked and has multiple tracking markers in it. So I used the advantage by parenting the masks with one of the markers. And voila!, the masks are now moving together with the objects and view.

However, there were still some slight miss-alignment to the mask with the objects. So I made a quick adjustments to the mask’s shapes and positions at several frames to realign them.

Compositing Nodes

The masks need to be composed using nodes in order to make it appear correctly with the 3D characters and original footage layers. In Blender, I discovered after working with my personal project in Term 3, that when compositing 2 layers using the ‘Alpha Over’ node, the layer attached at the bottom will appears on top of the other layer in the render.

Below is part of the first iteration of nodes I setup to compose the masks with the background and 3D characters.

Composting nodes: First iteration with Mask node included

I noticed that the mask has jagged edges in render. I’ve tried changing multiple settings in the mask editor to solve the problem but the jagged edges were still visible and didn’t changed a bit.

I then thinking the problem probably came from the Mask node, but the settings are very limited. The ‘Feather’ and ‘Motion Blur’ in the node have nothing to do with the jagged edges. So I tried adding and testing several nodes in between the Mask and Alpha nodes.

With the Blur node applied, the jagged edges were still very visible, but blurred. So this is not the real solution.

I rewatched the tutorials and noticed that they added the Dilate/Erode node. But this was still not fixing the problem as the function of the node is to quickly shrink and enlarge the overall shapes of the mask and not smoothen the edges.

Trying to add Dilate/Erode node from the tutorial

Nope! The node is to make the mask bigger or smaller only, not fixing the jagged edge

I then looked and searched for nodes that has anything to do with ‘smooth’ or ‘anti-alias’. After multiple tries and errors, I found the ID Mask node (unexpected name for its function). It don’t have any effect at first with the ‘Anti-Aliasing’ option turned on, but increasing the index to ‘1’ fixed the problem!

Found a solution! Anti-Aliasing by adding ID Mask node

I used the ID Mask in conjunction with the Blur node to make the smooth edge blur a bit, so it will match with not-so-sharp footage.

After the Shot 03 finished, I proceed with the other three shots. The process was almost similar to the Shot 03. It took me about a day to finish the masking in all four shots.

Shot 01