Roll With A Vengeance (Now With GUI)

It's your lucky day! I've been busy preparing for my alpha review tomorrow, which means I don't have time to write up a long and witty blog post.

First, remember my GUI? No? Fine, here it is again.

Okay, now get ready to experience this video. Things to notice:
- The GUI looks like my original GUI drawing! On the top left is the master track, which shifts when you press 1, 2, or 3 on the keyboard. Next to the master track are views for the three cameras.
- Speaking of which: multiple cameras! Right now I'm thinking that it would be jarring if the cameras appear and disappear as events occur, so it might be better to just have a set number of cameras which are always there—maybe the user can set the maximum number of cameras himself, and if too many events are occurring, the user can set priorities/rankings for different events
- The user can now navigate freely through the environment, with mouse/keyboard commands similar to Maya. This ended up being more of a pain than I was expecting, but it works now.
- You can see the cameras in the environment now, represented as little boxes. They're also color coded by the backgrounds that appear in their views.
- Two events occurring: ball rolling down some slopes, and ball bouncing up and down. Two different cameras cover them, and you can switch between them on the master track.

Ignore the long stretches of video where nothing seems to be happening. I'm using this during my alpha review, when I'll be speaking during it. So while you watch, just imagine me talking (or singing) in the background.

I can't wait for "Ball Rolling Down A Hill Into A Wall: The Movie: 4: Live Free Or Roll Hard"... it will surely be exciting!

After that I'll have to switch to another franchise. Suggestions?

Roll Harder (Now With 100% More Splitscreens)

I've been incredibly busy with other work for the past week, so this post will be uncharacteristically short. Consider yourself lucky! I'm sure next week will see the return of my overly-long and excessively-wordy blog posts with unnecessary illustrations.

Sorry, I'm already getting overly-long with this one.

Anyway, I did manage to figure out windows-within-windows! Now when the smart camera is active, a window pops up to show what it sees. I also wanted to learn GUI text, so I stuck in a status which tells you whether an event is occurring or not. Again, for this test, the event is simply "the ball is moving." Of course, I didn't want the camera to quickly appear and disappear whenever the ball jitters a little bit, so I've made an easily adjustable variable for how long the ball must have been moving for it to be considered an "event." Likewise, you can determine how long the ball must be immobile before it's declared that the event is over.

As I've said, the user should also be able to control the main camera as if they're using Unity normally (or Maya, since it has the same controls). As far as I can tell, there's no included script for this which can be attached to the main camera... I may have to do that myself. For now, you can right click and drag the view around a little.

Enough of my yammering. Let's see some video!

So there you have it. You can see me move the main camera around a bit at the beginning of the video. Pressing 'x' on the keyboard makes the barrier disappear, causing the ball to start rolling, and the smart camera to start tracking it. Watch the camera view at the bottom, but you can also see the actual camera moving in the top. Exciting stuff.

Next week it's time for alpha reviews! I'm going to try out more complex events, to see how the camera responds to them (like when things become occluded)—right now the camera is just using the SmoothFollow script which is included in Unity, so I will start editing that to make it behave more interestingly. I really want the camera to handle single shots well before it starts cutting to multiple angles.

Time to get to work on "Ball Rolling Down A Hill Into A Wall: The Movie: 3: Roll With A Vengeance"

Ball Rolling Down A Hill Into A Wall: The Movie

Well, I've created my first cinematic masterpiece in Unity! I call it "Ball Rolling Down A Hill Into A Wall: The Movie"

Okay fine, it's not quite a must-see, fun-for-the-whole-family, laugh-a-minute thrill-ride. But I have gotten a much better handle on Unity in the past few weeks. I read a lot of tutorials and watched a lot of video lessons, but easily the most helpful were on Unity 3D Student: these videos are short (2-4 minutes) and very to the point, so you know exactly what you're about to learn, and it's very easy to follow along. Plus, the guy has a British accent, which makes him sound smart. If you're learning Unity for your project, I highly recommend these.

Anyway, after learning all about adding components to objects, writing scripts, and switching between cameras, I put together an extremely simple test: a ball on a slope, a barrier preventing the ball from rolling down, and a wall at the bottom of the hill. I've got a video uploading to YouTube right now, but until then, you should be able to follow the plot through these screenshots.

The main camera simply watches the entire scene from the perspective view (in the screenshot below, the top window is the Scene view of the entire environment, and the bottom window is the Game view which is showing the perspective of the main camera).

The Ball is a rigid body affected by gravity, so it is leaning against the barrier. Pressing 'x' on the keyboard makes the barrier disappear, which causes the ball to start rolling down. More importantly, once the ball starts moving (a simple "event"), a new camera is created, set to be the active camera in the scene, and ordered to follow the ball. In the below screenshot, you can see the 2nd camera in the Scene window, and the perspective of the Game window has been changed to this new camera.

The 2nd camera follows the ball as it careens down the hill, slams into the wall, and comes to an explosive halt. I might be making it sound more exciting than it actually is.

Once the ball stops moving, the "event" is over, so the view of the Game window shifts back to the main camera, and the 2nd camera disappears.

So there you have it! Obviously this is all pretty simple, but I think I'm off to a good start. I played around a bit with GUI elements as well, so I'm going to continue with that for next week: remember, the Game view should just be a perspective view (controllable by the user), and clicking on the smart cameras should make little preview windows show up (just like the little Camera Preview in the Scene window above). So hopefully I'll have those windows-within-windows next week.

Coming soon: "Ball Rolling Down A Hill Into A Wall: The Movie: 2: Roll Harder"

Cinematography Basics: Zoom and Enhance

Tomorrow I'll have a post up about Unity, but let's get some other camera techniques out of the way: zoom and focus. Both of these also contribute to framing, which will be one of the main challenges that I'll have to handle with my intelligent cameras.

Zooming involves adjusting the focal length of the camera. It's probably not something that I'll be dealing with a lot, since my cameras can just move forwards and backwards in space if necessary. There is one cool thing that zooms can be used for, though it's not really a high-priority for me to implement: the so-called "Vertigo Shot," which you may know from Hitchcock's "Vertigo" (or maybe "Jaws"). The idea is to move the camera forward while zooming out, or move the camera backward while zooming in. That way, it makes it seem the distance between the foreground and background is shrinking or stretching. Here's a video with some examples of it used in famous movies:

Focus has to do with the camera's depth of field. I think it would be cool to play around with this: establishing shots should probably have a deep focus so everything is clear; once the camera moves in tighter, objects tied to the event can be kept in focus, while the background can be blurrier. There's also a technique called "rack-focus," where the focus shifts between multiple planes. If there's ever a time where a camera is framing one event, and a second event occurs within the frame but in the background, a rack-focus could be used to direct the audience's attention.

So now we've talking about getting things into the frame. But once they're in there, composition is important too! Here are a few rules:

Don't cut off people at the ankles, knees, or neck. It just looks odd. Still, it will be difficult to tell if objects are characters, so it will be tough for the cameras to avoid cutting people off... to avoid this, my cameras will probably always start with establishing shots, and then make sure to to keep a large chunk of the object in frame when doing closer shots. If only a small part of the object is in frame (like in the "bad" example below) things look weird.

Let the actor lead. If a camera is following a character, it should wait for the character to start walking before it begins panning. If the camera moves prematurely, it can be jarring and pull you out of the movie, because it's obvious that the cameraman started moving in anticipation of what he knew would happen next. This shouldn't be a problem for my project, because the cameras don't know what's going to happen next anyway, and are always waiting for their targets to make the first move. I've sometimes heard that if the camera and character are both moving, the camera should stop before the character does… But I don't think this is as glaring as the start of the action. Besides, my cameras wouldn't know to stop early anyway.

Use negative space. Framing things in the dead-center of the composition is boring. But this is especially important with movement: if a character is walking and the camera is tracking next to them, the composition should leave empty space in from of the character. You really don't want the character to look like they're walking off the frame, or for it to seem like the camera isn't keeping up. Even when a character is still, the camera should leave space in the direction that they're looking. If this rule is being broken, there's probably a reason for it…

…Like maybe you're watching a horror movie. Once you know to look out for this, you can ready yourself for lots of cheap jump scares.

Sure, it's usually just a cat jumping out. But at least you know that the cameraman was shooting that way on purpose.

Next time on Cinematography Basics: Editing and cuts!

Time Off for Good Behavior Trees

One of the main issues that I'll be trying to solve is how to make the cameras behave like... well, cameramen. When an event occurs, filming the entire thing in a single wide shot would work, but it wouldn't be very cinematic. Instead, the cameras will move and cut to different angles. There are rules of composition and editing that I'll get to in another post, but let's first talk about choosing shots in the first place. I originally had a vague idea of using some sort of finite-state machine, until Joe pointed me in the direction of behavior trees—specifically, a chapter in Artificial Intelligence for Games, by Ian Millington and John Funge. It seems like behavior trees will work well for my cameras. So thanks, Joe!

Let's get started by looking at this wacky diagram of a basic camera behavior tree:

This tree describes how a single camera might behave. At the top of the tree is a Proceed Loop followed by a Selector—in simpler terms, this means that the camera is in a constant loop of "selecting" which of the two subtrees (left or right) to follow. Conditions, the white rectangles in the tree, test something and then return either true or false. A selector will go through each of its children, and stop once one of them returns "true."

Essentially, the left subtree is behavior for when the camera is not assigned to an event, and the right subtree is behavior for when it is assigned. The Parallel box means that its children execute simultaneously—so in the left subtree, the Assert box will constantly check to make sure that there's no event assignment, and the camera will "wander" as long as that assertion is returning true. The little "wander" triangle represents a Sequence: if you think of Selection as "OR," then think of Sequence as "AND." Whereas a Selector would try children until it finds one that returns true, a Sequencer would go through successful children and stop once it finds one that returns false. In this case, the children of Wander would be different actions that cause the camera to move sort of randomly through the environment.

The right subtree is the interesting one. It starts with another Parallel box, which makes sure that it will execute while the camera is still assigned to an event (once the event ends, we should revert back to wandering around). So what are we executing though? Another Selector, one which decides what kind of situation we're in. Many of the papers that I researched use the term "idiom" to refer to a series of individual shots for a certain purpose; many idioms involve the number of actors in the scene. Thus, a simple way to break this down further would be into behavior idioms for situations with only one actor (when I say "actor," I may be referring to any sort of object, not just a character), situations with two actors, and situations with more than two actors. Obviously there are more idioms than this, but this will be a good start.

This Selector (the one in the right branch of the tree) will be important. I'm going to call it the Cinematographer Node, since it plays the role of cinematographer: deciding what kinds of shots to use based on the current situation.

So let's say there are two actors in the scene, and the Cinematographer goes down the middle branch. As long as the event still involves 2 actors (by the assertion), the camera will enter the Shot / Reverse Shot idiom, which I've expanded at the bottom of the diagram. This idiom, often used in movies during conversations between two people, would involve the following series of shots:

• A wide shot with both characters in frame
• An over-the-shoulder (OTS) shot of character A (meaning a shot taken over character B's shoulder, so we see character A's face)
• The reverse shot (an over-the-shoulder shot of character B)
• A close-up of character A, without character B in the frame
• A close-up of character B, without character A in the frame
• Back to a wide-shot

This cycle will repeat as long as there are still 2 actors involved. If a third actor joins the event, the assertion will fail, and the Cinematographer will go to the Crowd Shot idiom. Similarly, if actors leave until 1 actor remains, the Cinematographer will go to the Follow idiom, which involves filming the lone actor by itself. If each actor is still labeled an event, we may need extra cameras to follow the individual actors, since they can no longer be contained in one frame. Once it's determined that a lone actor is no longer an event (sorry, lone actor, you're not important anymore), the entire right subtree would fail, and we'd go back to wandering. If extra cameras were added, they might just disappear.

One reason that this behavior tree structure will work well is that once it's set up, it will be easy to add more complicated idioms later—the Cinematographer Node will be happy to have more specific situations to test for. Right now I've got a test environment built in Unity, and I'm going to set up some simple, 1-actor events. The first version of the camera will only have a Follow idiom: for testing, this will involve cycling between a wide shot of the object, a moving shot (which would follow the object if it's moving as well), and a close-up of some feature of the object. After that, I can add idioms for dealing with multiple actors in an event.

There's one major thing missing from the behavior tree, however: while this tree will cause the camera to choose the right types of shots for each situation, it doesn't ensure that the shots are composed well. Worse, it doesn't prevent the camera from breaking cinematic rules while cutting between shots. I'll get to these principles of composition and editing in the next post—but basically, the idioms will need to have asserts inside of them, rather than just being a sequence of shots.