I think, as happens often, I underestimated how long some things would take: Specifically, I didn't expect camera placement and tracking to be as complicated. On my Gantt chart, I actually planned for tracking in more complex environments to be completed today, and that did sort of happen with the Pachinko machine; however, this has been at the cost of the cutting-between-different-shots system, which I am only now working on. But I do think that holding off on cutting was the right decision: I want to make sure that individual shots are handled well before I move on to multiple shots.
On the other hand, some components turned out to be simpler to do than I was expecting. This didn't happen as often as I would like, but I'll take what I can get! The most important feature was that having multiple cameras (and multiple viewports) ended up being decently simple to implement—when I first started, and didn't really know Unity well, I was worried that this would be a pain. I am glad that it wasn't. While my original idea was to have cameras being added and removed as events occurred, I'm thinking now that it would make things unnecessarily complex, and the GUI would be really confusing if viewports kept popping in and out. So for now, I'm just working with 3 cameras at all times.
I like making lists. Let's make a couple:
Things I've done:
- A mouse/keyboard camera system that mimics Maya or Unity's Scene view (allowing the user to pan, rotate, and zoom at will)
- A GUI which displays the viewports of multiple cameras within the scene
- Keyboard commands for the user to cut between each camera on the Master Track viewport
- The ability for objects to determine when they are involved in an event
- Visual representations of cameras within the scene
- Cameras that automatically choose a suitable (not in collision with geometry, and with the target unoccluded) initial position when filming an event
- Cameras that can follow an object through complex environments without colliding with the environment
- Cameras that can follow an object and keep it visible, without resorting to locking it in the center of the frame
- A Director which assigns the cameras to cover different events based on whether they are already engaged
- Very basic behavior trees for the cameras and Director
Things to do:
- Implement cutting to different shots (obeying editing laws like the 180-degree rule)
- Behavior trees for different types of situations (crowd scenes, dialogues, etc)
- Priorities for events
- Allow user to set constraints on cameras
- Implement genres for cameras (action movie, romantic comedy, documentary, etc)
- Figure out how to export video footage from the Master Track
- Continue improving camera movement
So let's talk milestones. The most important remaining objective is getting the cameras to cut between different shots. Once that's working, I've pretty much accomplished the most basic idea of my project: cameras that automatically make a "virtual documentary" of the events around them. Obviously it won't be an interesting documentary, since the shot choices won't be that exciting, but it will still be a documentary. For the upcoming Beta Review, I'm hoping to have camera cutting implemented (along with constraints on some cinematography maxims like the 180-degree rule, the 30-degree rule, and so on).
After that, it will be time to actually make things visually interesting: more advanced behavior trees so that the cameras can react to different types of (more complicated) situations. The priority will obviously be events with multiple actors involved, like handling dialogue scenes with a shot/reverse-shot idiom. So by the time of final presentations, the cameras should use behavior trees to create much more stylish movies—users should also be able to select the cameras and set certain options like genres. Finally, a user should also be able to export their Master Track and watch the actual movie that was produced by the combination of the cameras.
Assuming everything else is completely 100% four-stars Oscar-winningly perfect by that point, remaining time will be used for adding more complex idioms, crazier genres (a high-contrast film noir one would be the best), and other fancier things.
So there's my to-do list, my did-do list, and my major upcoming milestones. Fun (kind-of-scary) stuff!
nice posts!!! I was wonderin' what software are you using? Mine, always opencv. got to do some level up.. Im trying to do a research now on kinects... GOOD LUCK on you project!! I will be keeping my eye on your site. I know this would be really fun!!!
ReplyDeletehttp://www.intechopen.com/books/show/title/object-tracking