Conclusion & Reflection

I’ve learned many good things throughout the Advanced and Experimental Unit for these two terms. From learning different new software, tools and techniques, making more involvement with friends and team, and to finally discover what I really want to pursue. I’ve discussed about my thinking on future path that was still unclear at that time in my old entry (https://kamil.myblog.arts.ac.uk/2021/02/01/self-reflection-and-future-path/).

By the end of this term, it has become more clear that I want to focus my career on animation especially character animation. I also discovered that I somehow able to perform camera tracking works quite nicely, although not at expert level. I managed to track footages from my personal project as well as 6 out of 8 challenging footages from the Indie Film project, which I originally didn’t signed up as matchmover but as animator only. Before this, I always assumed that I’m very weak with tracking and don’t have much interest in it. But thanks to Luke, he was the one who encouraged me to keep the idea of doing camera tracking in my personal project, which lead me to try doing the same with the Indie Film project.

It’s completely different now that I even proposed to include camera tracking as part of my final major project (FMP) with a combination of 3D character animation.

During this term, I’m more active and engaged in conversations with others when compared to the previous terms. I also tried my best to assist friends who asking for help in the class and group Discord server when they have problems with technical and software. Probably because of this, several classmates started messaging me personally when they need help. I believe by helping others, I can also learn something new when I try to solve the problems.

Speaking of learning new things, in terms of techniques, I’m now more knowledgeable with tracking tools in Blender and 3DEqualizer, discovered few tricks of how to solve a challenging camera track, learned and understand more tools in several 3d software that I used in this term. I also learned how to use Houdini and quite amazed with the software capability, although I didn’t plan to use it as my primary animation tool at this moment.

With all that I have learned now and in the future, I hope that they are not only for my own benefit but to keep developing it further to contribute and give benefits to other people as well. This is something that I want to achieve with my future works and thesis. To me, that is the most important.

Week: 31 May – 6 Jun

During the weekend, I re-watched quickly the Dom’s tutorials to recall some of the important technical aspects. I also watched several other tutorials on YouTube to learn more about the software.

This week, I also doing camera tracking for all the footages of my personal project. So the timing was quite right as my mind will only about camera tracking this week although both project are using 2 different tracking software which are 3DEqualizer and Blender.

I have 8 shots for the indie film project to track and 12 shots for personal project and. So I called this week as ‘CAMERA TRACKING MAYHEM WEEK!’

https://youtu.be/vI9_s8oRB_w
The above tutorial won’t let me embed the video here, but it helped me a lot to understand more about tracking data to import to Maya, the use of camera group and scaling.

Above was of the earliest tutorials I watched for 3DEqualizer on YouTube. This one shows how to deal with blurry footage and rolling shutter which can be helpful as the footages that I’m going to work on has the same problems.

I watched the above tutorial just to learn to how to import 3DEqualizer tracking to Blender since I used Blender for my personal project. In case Blender failed to track my footages, I will have a backup software for it. And after watching this, I learned that we cannot simply import 3DE tracking to Blender but to use and run some scripts for it to work properly. I didn’t try this but something I have taken notes of.

This one was a quite simple tutorial but it teaches how to use lineup controls constraint markers to plane to align the camera and track.

I watched many others 3D Equalizer tutorials that I bookmarked them for future references.

Tracking with 3DEqualizer

I then proceeded with the tracking process and I managed to track 6 out of the 8 footages for this week. It took me about 5 days to track all the shots while also doing camera tracking with my personal project’s footage.

Along the way, I experienced many challenges, bad tracking, bad camera alignment and software crashes. At the same time, I also learned a lot of things and discover several tricks that helped me solve the camera movements. My final tracking was not really 100% perfect as there are several small vibrations in certain frames.

It is probably too long to blog everything for each shots here, so I just make a summary for shots that I think have unique solving methods than the normal and standard ones.

The shots that I’m able to track:
CC_0100, CC_0200, CC_0500, CP_0100, CP_0200_0300 & CP_0400

Remaining two shots:
CC_0300 & CC_0400.

The CC_0300 & CC_0400 were the hardest to track for me with their camera shakes, rotation, and motion blurs. I’m really struggled to get good track with it. I felt almost like those 2 shots don’t want to be defeated by me at this point. I’m a bit disappointed but at the same time also happy that I’m able to solve 6 of the footages. I might return to those 2 shots in another week.

Among the shots that I’m able to track, the hardest were CP_0200_0300 and CC_0100. Which is really really weird!, because in Blender, CC_0100 was the easiest and the fastest to solve. Blender was able to quickly recognised the camera zooming motion with changing focal length without problem. It took me about 1 day just to solve this shot in 3DE compared to approximately 30 minutes in Blender.

I tried everything; from placing markers near each corner of the shot to make 3D Equalizer recognise the zoom, to changing camera constraint and enabled dynamic focal length. None of that giving satisfactory results. It can track the camera movement, but not the zoom. It think the camera is moving forward, instead of zoom even after I set the positional camera constraint to fixed.

So, as a special method for shot CC_0100, I finally decided to animate the changes of focal length (zoom) value manually. To get the correct estimation for focal length, I put a 3d plane in the shot as a measurement object. I aligned the 3d plane with the road at the beginning of the shot and keyframed the camera’s focal length value right before the zoom started.

I then go frame by frame to find where the camera stop zooming. I then changed the focal length value carefully until the 3d plane shape and perspective are matching with the road again. With full suspense, I clicked the ‘Calc All From Scratch’ button and….. it works!!! It solved the camera movements perfectly! I can’t describe how happy I am at that moment.

The final solve error aren’t pretty because 3DE think the markers deviated so much, but since the movements were correct, I think I can’t rely on the number.

The same thing can be said for CP_0200_0300. The deviation was very high but this was the best I could get and cannot get any lower than this. And since the camera movement was correct, I ignored the number. The camera movement was quite challenging to track with the camera rotation. In the first few tries, I cannot solve it and then I changed the ‘Positional Camera Constraint’ to ‘Fixed Camera Position Constraint’ able to help with the tracking.

Below are the rest of the footages that I managed to track without much problems.

I didn’t have much problem with CC_0200 but the I cannot get any lower solve error before the camera movement become broken when I delete any markers that have high deviation value. There were a lot manual tracking and re-position of markers frame by frame by hand. I’m very careful to predict their location when the shot became blur.

Videos

I exported the tracking data for 3 of the random shots that I managed to track into Maya first just to test how they look with the car model.

I uploaded those 3 videos first to our Discord server. I didn’t have time yet to finish the road geo, realigning and rescaling all the 6 shots in Maya as I’m also finishing the camera tracking for the footages of my personal project at the same time. So I will update the team again after I finish with the Maya files.

To be honest, my mind was very tired this week after doing all the tracking works in both the indie film and personal project. Nevertheless, I’m very happy I that managed to track 6 out of 8 footages from this project.

Week: 24 – 30 May

Early this week, I tried to track several shots from the indie film project as personal practice since I’m not signed up as a matchmover in this project. Last week, I noticed that the videos have a lot of motion blurs which Emma and Marianna experienced difficulties to solve the tracking in 3D Equalizer. So I wanted to try and see if I’m able to do it in Blender using the knowledges that I learned several weeks ago. After looking at all the videos again, I decided to try on the CC_0100 and CC_0200 first.

CC_0100

CC_0100 has a fast camera zoom at the beginning which caused very high blur. The first method I tried was to use the ‘Detect Features’ to automatically place markers on areas where Blender think it can track for the whole duration of the videos. Unfortunately, it failed to recognise the ground area probably because of the blur during the zoom. I proceeded with the ‘solve’ function just to see if it can actually solve the camera movement with the automatic markers but the result was very bad.

I had no other option but to manually place markers and probably tracked it frame by frame. I analysed the video again and noticed the image was sharp towards the end of the footage. So I decided to do a backward tracking. There were actually many contract areas which I can place the markers, but most of them were small which completely disappear during motion blur.

When I skimming through the video, the only contrast areas that still has good visibility during motion blur was the road lines. So I placed several markers at the sharp corners of the road lines.

I then tracked all the markers backward frame by frame carefully and stop when the markers deviated from its original place because of the blurry frames. I manually placed the markers by guessing the correct positions. The road lines make it quite easy to predict the locations even when its blur because I can still see the line’s corners.

After several tries, correcting positions and removing bad markers, I managed to solve the camera movement with solve error of 0.89. It’s not great but that was lowest I can get before the camera alignment broke if I do a bit more clean up.

Video

CC_0200

Next I proceeded with the next video, CC_0200. This footage has a lot of camera movements with rotation and position. The same problems happened with this shot when using the auto detect features. I then used the same tricks as the CC_0100 to manually place and track the marker.

This shot was multiple times harder to track compared to CC_0100.

It took me about 5 hours on this shot and I finally managed to solve the camera with accurate camera movement, but with a very high solve error value of 15.30 px. I could get lower value than 1.0 by deleting high deviation markers, but the camera movements would not aligned properly anymore. I guessed I cannot rely on the numbers only and probably because of the high vibration of the original camera, Blender think the markers have bad deviation.

Video

I then sent both videos to our Discord server during today’s class (26th May). Luke said the tracked were good and asked if I’d like to pick other shot and help the team with the camera tracking.

I’m not really sure because the tracking I did before was done using Blender and not 3DEqualizer. I don’t remember much about 3DEqualizer since I only used it several time only during Term 1. I may need to re-watch the Dom’s tutorials to recall the important technical aspects of the software. I think this as a good opportunity for me to practice the software again as 3DEqualizer is one of the industry’s standard for matchmove and tracking.

So I said to Luke that I will try to track on the same footages that the other matchmovers were doing at this moment as a backup and see if I can good results.

Week: 17 – 23 May

Today (19th May), we finally, we received the actual video footages. Luke made a briefing video and spreadsheet about our tasks for each shot.

There are total of 8 videos that divided into 2 categories, which are CC (car crash) and CP (car pass). CC shots would need additional VFX works for the car crash simulation and effects.

The flow for each shot will be basically like this: ‘Matchmove > Animation > VFX > Lighting & Rendering > Compositing‘. For the animation tasks, me and the other animators (Layne & Tina) would need to wait for the footages to be tracked by the matchmovers first (Emma, Marianna & Antoni) so that we can animate the car accordingly to the camera angle, and ground surface of the scene. We can actually start animating the car while the footages being tracked, but the animations might need to be changed again when we receive the camera track to match the path and direction of the road in the shots.

I checked the footages in the collaboration folder and noticed that all of the videos were blurry or have motion blur when the camera moves. I’m not an expert in camera tracking, but I’m a bit concerned that the videos could be very challenging to track.

About 2 days later, my concern was true as Emma and Marianna were discussing for several days that they were having trouble to work with the blurry videos.

Although I’m not a matchmover in this project, I was thinking to try to track some of the shots in Blender to see if I’m able to solve it. This can also a good practice for me since I’m also doing camera tracking in my personal project.

Week: 3 – 9 May

This week, the actual footages was still unavailable and might still being filming. Everyone in the team were doing practices and test since last week as preparation for the actual work later.

The Ftrack page that Luke created was being populated with everyone test videos from all departments. I saw some of them and very impressed with their works. Watching other people works always inspire me to keep improving myself.

I also uploaded my car animation test I did last week to Ftrack and below is Luke’s feedback on my animation.

Week: 26 Apr – 2 May

Last week, Luke created a new collaborative folder for the indie film project on his shared OneDrive for us to share and upload our working files. He also opened a new review page on Ftrack for anyone wanting to share videos for feedback.

Since we were still waiting for the actual footages to work on, Luke provided several materials for all departments to practice first. For the animation team, Luke made a tutorial video on how to animate a car with realistic motion and rumble. We can practice using the temporary car model from the collaborative folder.

I downloaded the car and road model including their texture files. The car was rigged very nicely with necessary controllers to animate the car complete with automatic tire rotation when the car moves. I setup a new project folder and put everything in the respective directories.

Even this is just a practice, I used reference system when importing both models into a new empty Maya file, which will become an animation file. The animation will be done on a model that was linked to the external file. So any update to the original car model will be applied back to the animation file automatically.

I repositioned and placed the car properly in a suitable location on the road. I then made a ‘camera and aim‘ and parent it to the car’s root controller so the camera will move together with the car.

I used animation layers as suggested by Luke in his tutorial. I created 2 additional animation layers for this. BaseAnimation layer is for the main car movement while carbody_02 and carbody_03 are for the car’s body animations.

I started with the rumble animation first on the carbody_02 layer. I made a short 60 frames sequence with a slight bumping motion. I then loop the sequence in the Graph Editor using Curves > Post Infinity > Cycle with Offset.

I then made another animation in carbody_03 layer with much longer sequence with subtle and smoother wobble animation. The animation in both layers will blend together to create more organic motion and not look too repetitive.

I tweaked both animations several times by adjusting the curves and keyframes in the Graph Editor until I satisfied with the result. The objectives were to make the rumble to look very subtle but still noticeable and realistic.

After the rumble motion finished, I then animated the car’s root in the BaseAnimation layer to make the car moving down the road. I only animated a small portion of the animation and loop it again using the same function as before to create a continuous motion.

For the last part, I animated the camera using it’s target (aim). I made a subtle camera shake motion manually to the camera’s aim to make the camera look like it is being held by hand.

Below is the final animation for this practice.

Final Animation

Week: 19 – 25 April

During this week class (21st April), Luke gave a bit overview of what tasks will need to be done by each department in the coming weeks. I took the animator role, so I my task would be animating a car. Perhaps to make it look realistic since this 3d car will be combined into a real footages. Luke said that the videos are still being filming at the moment, so we will need to wait a little bit until we receive the videos.

Few days later Abbie setup a new Discord server for the project, and everyone involved quickly joined the server. There are several team members from the VFX course as well, and I’m glad to see Antoni & Ghera joined this project. I’ve teamed up with them during the Term 2 collaborative project, and they are very good and cooperative.

Week: 19 – 25 April

After thinking for several days and watching quite a lot of short animations since the last week post, I decided to do 3d character animation in a real video footage for my personal project. I really love character rigging and animation, and quite weak with camera tracking. So with this project, I can practice character animation while at the same time improve my camera tracking skill. For the collaborative project, I decided to join the indie film project as animator. I then emailed my project proposal to Luke.

I then booked a session with Luke on 6^th May for his further advices and also to show my initial progress of the personal project.

Week: 12 – 18 April

I’m now in Term 3. In this week class, Luke briefed us on what we have to do throughout this term. There should be 2 minimum projects which are another collaborative project and personal project. The collaborative should be a teamwork project with students from other departments, while the personal project is solely our own ideas which can be something that we interested to explore and showcase our preferred style and aim for the skillset to jobs that we are looking. Luke mentioned that there is a collaborative indie film project for anyone interested to involve as animator, matchmover, VFX and render artist.

We will do a lot of self directed learning and research for the projects, but at the same time we can book a session with several mentors for their advice and guidance. During this term also, we will need to be planning for our thesis and final major project and pithing them in week 9.

Our task for this week is to think about what projects we want to do for this term and why, then send the email proposal to Luke before class next week.

Personally, I’m not really sure yet what I want to do for my personal project, but I really like character animation. For the collaborative project, I probably decide to get involve in the indie film project. So I have until Monday to think about this. I’m going to watch short animations on YouTube to get some ideas.