Creating flat matte videos for chroma keying in Unity with Adobe After Effects

The time seems to be whizzing by lately - it only seems like yesterday we were filming our greenscreen zombie footage and it's already been nearly two weeks! Now we've a load of video footage, and plenty of work to do, to make it "game ready" (i.e. dropping out all the unusable background). We've loads of little clips, so we're starting small - but this is the main technique we used for pretty much all of them (and will continue with for future footage):

Here's a quick clip of Andrea wandering around, getting ready to shoot some zombies with his puny pistol.

As you can see, the background is hardly a perfect flat green (although it didn't quite look that bad when we were actually shooting the footage - the higher contrast of the output video makes the wrinkles in the screen really stand out!)

The footage is also "too big" - it was shot at 1920 x 1088 and included far too much footage, so we've had to crop the video and rescale to a more manageable 720 x 405, as well as trim out the unwanted footage both before and after the action we want to capture

After importing the video into After Effects, the final output render size was massive and there was a lot of unwanted extra footage.

We used the keyboard shortcut to "splice" the video footage at the current keyframe, using Ctrl + Shift + D. This not only creates a razor cut in the video, but splits it out onto two separate frames. Before cutting any footage, we rescaled the actual video and moved it around inside our composition window, so that only the actual footage we wanted was captured.

We made a cut at the start and at the end of the section of footage we wanted to keep. This split the video onto three layers, making it easy to select the bits before and after to remove them completely.

Now, we've had mixed success with the following technique, using the entire footage, so at this point we exported our video selection to a new, separate file, and imported it into a new project. As we've still plenty of work still to do with this footage, we exported it as uncompressed video without using any compression or using a video codec. This makes massive files (100Mb for 5 seconds) but does mean fewer compression artifacts and no blockiness in the new video.

Next we imported the newly rendered file and made sure that it filled our composition size of 720 x 405 (the earlier footage was much larger than this and needed to be cropped). Then, following the instructions at creativecow website (https://library.creativecow.net/articles/rabinowitz_aharon/junk_mattes/video-tutorial) we made the entire background transparent. The basic steps are:

With the video footage selected in After Effects, apply the KeyLight (v1.2) key chroma effect and select a green close to the actor (or one nearest the "average" green of the background).

After applying KeyLight chroma key, we can see that there are lots of imperfections in the footage that didn't get captured by the key colour.

This takes away most of the background, but still leaves semi-transparent shadows in the background. So now, we need to "trace" our actor to completely isolate them from the background. We do this by selecting the video footage and then Layer - AutoTrace and tracing based on the alpha of the channel.

Maybe 70% is a bit much, but it worked for this particular bit of footage, so we're sticking with it - expect to use values 50%-70%.

A short while after applying the trace (it takes a few seconds to go through every frame in the video) a new layer appears with an outline around the main character. We can scrub through the footage to make sure that it captures just our main character on each frame of the video and that nothing is either truncated, or left in when it should be removed.

Next comes the clever bit - we need to expand the outline a few pixels away from the main character, to create a green "halo" around the character. So we select the traced layer and then Effect - Matte - Simple Choker. Setting the choke value to a negative value expands the matte away from the main character. We went with something in the region of -25

Depending on your version of After Effects, you may need to locate and click the "Toggle Switches/Modes" button to enable the "track matte" feature on the original footage. Set this track matte setting to the traced layer (immediately above it).

What's the point of doing all this? Well, what it means is we're only using a traced area of the original footage now, for the chroma key - we've got rid of most of the (wrinkly, unevenly lit) green background, and we're only actually keying the "green halo" around our main character. It basically means we need a much less "harsh" keying setting to completely remove all of the remaining green background (since there is so much less of it to begin with).

What we've done so far is create a "junk mask" around our main character, so remove most of the background, but there are still a few little areas around him that are not quite right. Now we just need to play with the KeyLight Chroma Key settings to get rid of any extra artifacts around the outside of our main actor.

We set the screen gain to 110 and the Matte Black setting to about 15 and all of the artifacts were gone.

One thing we found was that with a slightly more "aggressive" setting to remove the green, we started to lose opacity on our main character. In particular, the body started to disappear, a bit like Marty McFly at the Enchantment Under The Sea Ball. It was quickly corrected by adjusting the Matte White setting down from 100 to about 80.

The final step is to simply export the video footage to a format that supports RGB+ (or RGB+Alpha or 32-bit RGBA colour). For this project, we went with Quicktime as it created the smallest files (about 2.5Mb for our 5 second video). To ensure that the video plays back ok in media players that may not support alpha transparency, we double-checked that our composition settings had a bright green (#00FF00) background. Where alpha channels are supported, the video should play back with a transparent background. Where they are not, a flat, matte, solid green background should appear.

Gone are those nasty greenscreen wrinkles and badly-lit backgrounds! Our video footage is now "game ready".

With our footage now "game-ready" all we need to do is drop it into a game engine that supports alpha videos (Flash has been able to do this since about version 7 or 8) and place the appropriate background bitmap behind it.

The absolute worst case scenario would be to use Flash or Unity (or some other gaming engine) to chroma key out the green at runtime, now that we've got it a nice, solid, uniform bright green throughout the entire video...