The VES Handbook of Visual Effects, 3rd Edition

Industry Standard VFX Practices and Procedures
By Susan Zwerman, Jeffrey A. Okun

880 Pages

• Paperback, ISBN 9781138542204
• eBook, ISBN 9781351009409

 
The award-winning VES Handbook of Visual Effects remains the most complete guide to visual effects techniques and best practices available today. This new edition has been updated to include the latest, industry-standard techniques, technologies, and workflows for the ever-evolving fast paced world of visual effects. The Visual Effects Society (VES) tasked the original authors to update their areas of expertise, such as AR/VR moviemaking, color management, Cameras, VFX Editorial, Stereoscopic and the Digital Intermediate, as well as provide detailed chapters on interactive games and full animation. Additionally, 56 contributors share their best methods, tips, tricks, and shortcuts developed through decades of trial and error and real-world, hands-on experience.

This 3rd edition has been expanded to feature lessons on 2.5D/3D Compositing;　3D Scanning; Digital Cinematography; Editorial Workflow in Animated and Visual Effects Features; Gaming updates; General Geometry Instancing; Lens Mapping for VFX;　Native Stereo;　Real-Time VFX & Camera Tracking;　Shot/Element Pulls and Delivery to VFX;　Techvis;　VFX　Elements and Stereo; Virtual Production; and　VR/AR (Virtual Reality / Augmented Reality).

A must-have for anyone working in or aspiring to work in visual effects, The VES Handbook of Visual Effects, Third Edition covers essential techniques and solutions for all VFX artists, producers, and supervisors, from pre-production to digital character creation, compositing of both live-action and CG elements, photorealistic techniques, and much more. With subjects and techniques clearly and definitively presented in beautiful four-color, this handbook is a vital resource for any serious VFX artist.

Table of Contents
CONTENTS
About the VES
Mission Statement
Foreword
Chapter 1 Introduction
Introduction
Chapter 2 Pre-Production/Preparation
Overview
Breaking Down a Script - Budgeting
Ballpark Budget
More Detailed Budgets
Bidding
Plate Photography
Temp Screenings Reviewing Bids Contracts
Rebidding during Shooting
Rebidding in Post
Monitoring the Budget and Schedule
Keeping the Budget Down
Working with the Director and Producer
Demo Reel
The Meeting
Moving Forward
Production Departments
Production Design Camera
Working with the Cinematographer
Special Effects Stunts Wardrobe Makeup Production Visual Effects Editorial Locations
Production Meeting
Designing Visual Effects Shots
Guidelines for Directors Storyboards
Previs
Objective of the Shot
Concept Art
Continuity
Photorealism
Original Concepts Budget
Reality and Magic Camera Angles
Framing
Scale Detail Speed
Scaled Images
Depth of Field
Sequence of Shots Camera Motion
Less Is More
Action Pacing
CG Characters
Creatures and Character Design
Powers of 10 Shots
Visual Effects Techniques
Technique Considerations
Additional Suggestions for Determining Techniques
What is Previs and Other forms of Visualization
What is Previs?
Different Types of Visualization and When to Use Them
Visualization: The New Essential
Development of Previs Techniques
History and Background
The Applications of Previs: Who Benefits and How?
The Benefits of Previs are Twofold:
Projects Types
Possible VR section
Post Visualization
What is Post Visualization?
Why Use Postvis?
Who does Postvis?
Summation
Cautions and Suggestions for Good Practice
The Perils of Previs!
Passing the Work On
The Role of the VFX Supervisor in Previs
Previs: Advanced Techniques
Visualization Usefulness
VR as a Tool
Visualization in Engine
Render in Engine
Visualization in Realtime
AR as a Tool
Conclusion
Camera Angle Projection
Drawing What the Lens Sees
Techvis
What is Techvis?
Who is Techvis for?
Virtual Production
What is Virtual Production?
How is Virtual Production Used?
Chapter 3 Acquisition/Shooting
Working on Set
Common Types of Special Effects
What Are Special Effects?
A Brief History of Special Effects
The Special Effects Supervisor
Working with the Visual Effects
Visual Effects in Service to SFX
Special Effects Design and Planning Storyboards and Previs
The Elements: Rain, Wind, and Snow and Ice
Smoke, Fire, and Pyrotechnics
Mechanical Effects
Flying Wire Rigs and Stunts
Safety
Greenscreen and Bluescreen Photography
Best Practices and Otherwise
Overview
Function of the Backing
Negative Scanning and Digital Conversion
Backing Uniformity and Screen Correction
The Alpha Channel
The Processed Foreground
The Composite
Recommended Specifications and Practices
How to Expose a Green Screen Shot, and Why
Setting Screen Brightness
Choosing the Backing Color
Floor Shots, Virtual Sets Foreground Lighting
Controlling Spill Light
Lighting Virtual Sets
Tracking Markers
On-Set Preview
Cameras for Bluescreen or Greenscreen Photography
Underwater Photography
Compositing Software
On-Set Data Acquisition
Camera Report
Tracking Markers
Props for the Actors
Cyberscanning
Digital Photos Lidar/Laser Scanning
Lens Distortion Charts
HDRI and Chrome Balls
Lidar Scanning and Acquisition
On-Set 3D Scanning Systems
Types of Technology
Lidar
Photogrammetry
Prop Scanners
Vendor Questions
Lighting Data
Gathering Lighting Data
Beware of False Savings!
Using Conventional Still Cameras Reference Shooting Considerations
Clean Plates
Shooting the Clean Plate Locked-Off Camera
Moving Camera
Other Issues Post-Process
Alternates without Clean Plates Other Uses for Clean Plates
Monster Sticks
On-Set Animation Capture: Witness Cam (IBMC)
Wireless Non-Video Motion Capture
Factors Affecting Witness Cameras
Dealing with the Data in Post-Production
Camera Tracking for Real Time Visualization
Camera Tracking Pre-Production
The Camera Department
Prior to Shooting
Current Tracking Systems in use:
Triangulation as a Method of Recording Camera Data
Camera/Subject Positional Information
Basics: The Tool Kit
Basics: Nodal Point
Photographic Reference
How to Proceed
Shooting Video as a Reference
Rules, Setup, and Testing
Why Run Through Example or Test Shots?
Digital Cinematography
Digital Definitions
High Dynamic Range (HDR)
Lens Metadata
Look Management
The Recording System
Lens Mapping for VFX
VFX Photography
The Camera Array
Designing an Array Shot Technicians
Shoot Day
Special Techniques
Post
The Future
Filming Live-Action Plates to be Used in VFX
Camera Position (Station Point)
Angle of View
Lighting Considerations
Camera Tilt
Background Quality
Moving Plates
Scouting the Camera Positions
A Case Study
Camera Cars
Camera Car Safety Issues
Purpose-Built Crane Cars
Vibration and Camera Stabilization Road Speed
Precautions Panoramic Rigs On the Water
Air to Air
Cable Systems
Shooting Elements for Compositing
What Is an Element?
Stock Footage
Types of Elements
Generic versus Shot-Specific Elements Determining Element Needs
Cheating Backgrounds
Black Backgrounds Line-Up
Camera Format Considerations
Assorted Methods for Shooting Elements
High-Speed Photography and Filming Elements
High-Speed Photography Cameras
Technicians
Director of Photography Lighting
Application
Locking Down the Camera
Video Assist
Post
Supervising Motion Control
What Is Motion Control?
Performance Choreography
Multiple-Pass Photography
Scaling
Import and Export of Camera Move Data
The Data
Types of Motion Control Systems
Motion Control Software
Camera Types
Sync and Phase
Dealing with Production
Acquisition of Motion/Still Photographic Textures for Mapping onto CG
Panoramic Backgrounds
Tiled Stills
Motion Tiling and Synchronous Plates
Practical Considerations
Stills for Textures and Lighting
Stop-Motion
Evolution of Stop-Motion Photography
The Time Required to Shoot in Stop-Motion
Preparation before Shooting
Setting Up a Shooting Space for Stop-Motion
Use of Motion Control in Stop-Motion
Useful Caveats
Evolution of a Shot
Use of Stop-Motion in Visual Effects
Chapter 4 Performance and Motion Capture
What is Motion Capture?
Is Motion-Capture Right for a Project?
The Mocap Look
Technical Specifications
Entry Point
Budget
Which Technology is Right for a Project?
Gauging a Project's Needs and Constraints
Passive Retroreflective Optical
Active Optical
Inertial
Structured Light
Dense Stereo Reconstruction
Bend Sensors
Preparing for Preparing for Motion Capture
Actors
Motion Capture Suits
Marker Placement Body
Marker Placement Face
Rigging for Motion Capture
Shot List
Hardware
The Strobe
Markers
Lenses
Filter
Image Sensor
Onboard Processor Inputs/Outputs
Setup
Software
Acquisition Calibration
Post-Processing Reconstruction Labeling
Gap Filling Cleaning
Solving Motion Capture
Facial Motion Capture
Facial Actor Survey
Actor Survey-Hardware Reference Data
Statistical Data
Facial Rigging
Facial Acquisition Audio
Facial Motion Capture Solving
Real Time Motion Capture
Real Time Uses
Real Time Limitations
Alternate Technologies
Motion Capture Resources
Virtual Production
World Building Previsualization Onset Visualization
Virtual Cinematography
Chapter 5 Stereoscopic 3D
How 3D Works
Accommodation and Convergence
Interaxial Separation
Toe-in versus Horizontal Image Translation
Parallax or Depth Budget
Positive and Negative Parallax
Floating Windows
Fix It in Post
Stereoscopic Design
The Emerging Grammar of 3D Creative
Use of Depth Previsualization
Avoiding Painful 3D
The Aesthetic of Scale Cutting for 3D
Designing for Multiple Release Formats
Immersion-Based versus Convergence-Based Stereo
Native Stereo
Pre-Production
On-Set
Stereography in the Visual Effects Process
Stereography for Finishing
HFR as a solution for better 3D movies
Summary
VFX Elements and Stereo
Introduction – How VFX Elements are Used
Native Stereo Content
Mono Capture – Packaged script and Element Deliveries
Mono Capture – Hybrid Approach for Stereo Delivery
Mono Capture – Full CG Approach for Stereo Delivery
Creating Depth – Layout and Stereography
Stereo Camera – Depth Wedges
Stereo Compositing
Requesting a Full CG stereo Render Mid-Production
VFX Production Tasks
Summary
2D to 3D Conversion
Depth Creation Preparation
Visual Analysis of 2D Depth Cues
Pre-Production and Previs for Conversion
Source and Target Perspective
Shared Shots/Shared Workflows
Main Stages of 2D-to-3D Conversion
Major 2D-to-3D Conversion Workflows
Special Cases
Reprojection Mapping Workflow
Pixel Displacement or Pixel Shifting
Other 2D-to-3D Conversion Workflows
Is "Real" Always Right?
2D-to-3D Conversion Management
Stereoscopic Visual Effects
Prepping for the Third Dimension Shooting the Third Dimension
Visual Effects in the Third Dimension Photographed Elements
Accuracy and Attention to Detail Artistic Skill Level
Data Management
Stereoscopic Digital Intermediate Workflow
Stereoscopic 3D Process Milestones Viewing 3D Dailies
Projection Screens for Stereoscopic Viewing
3D Editorial Processes
Data Workflow
Applying the 3D Grade
3D Stereo Deliverables
3D Home Video Deliverables
Stereoscopic Window
The Stereoscopic Window
Placement of the Window in Relation to the 3D Scene
Window Violations
Window Placement Logic
How to Create a Stereoscopic Window
Producing Movies in Three Dimensions
Development—Getting the Green Light
Production—What to Look Out for
　
　
　
Chapter 6 Post-Production/Image Manipulation
Resolution and Image Format Considerations
Formats
Transport
Resolution
Academy Encoding Color System (ACES)
ACES Components
ACES Benefits
ACES Color Space Encoding Viewing ACES
Preparations for Using ACES
Image Compression/File Formats for Post-Production
Image Encoding
Still Image Compression
File Formats
Color Management
The Three Guidelines
Digital Color Image Encodings and Digital Cameras Color Management at the Desktop
Bringing Color Management to Film Workflows
Digital Intermediate
Shot Element Pulls and Delivery to VFX
Introduction
The Lab
"Production" Is To "Lab" As…
The Merge
The Handoff
VFX Editorial
The Select
The Pull
VFX Editorial
Editing within a Shot: The Art of Pre-Compositing (Precomp)
How It Came to Be
Modern Day Tracking and Disseminating of Information
As the Shot Changes Wrapping It Up
Editorial Workflow in Feature Animation
Introduction
Editorial Crew Staffing and Structure
Editorial Involvement with Feature Animation Production Stages
Communication with Artists
Starting
Working with Teams
Working Globally
Reference and Perspective
Shot Production
Communicating with Artists in Other Departments Completion
Compositing of Live-Action Elements
Modern Digital Compositing
Scene Tracking
Rotoscoping and Paint Rotoscoping
Digital Painting and Plate Reconstruction
Matte Paintings/Creative Environments
Matte Paintings: Art of the Digital Realm
What Is a Matte Painting?
Matte Painting Pioneers and History
Visualizing the Matte Painting Shot in Pre-Production
On-Set Supervision for Matte Painting Shots
Basic Skills and Tricks of the Trade
Miniatures and Computer-Generated Sets
Finding the Best Frame
Re-Projected Photo Survey
The Need for Creative Compositing
3D Matte Painting
Chapter 7 Digital Element Creation
Digital Modeling
Overview: The Importance of Modeling
Types of Modeling
Model Data Types
Development of Models
Modeling for a Production Pipeline
Engineering Aspects for Polygons
Engineering Aspects for NURBS
Rigging and Animation Rigging
Rigging: What Is It?
Animation Rigging
Deformation Rigging
Texturing and Surfacing
The Importance of Texture Painting
Hard Surface Models
Creature Models
Types of Geometry: Their Problems and Benefits
Prepping the Model to Be Painted
Texture Creation
Various Other Map-Driven Effects
Texture Painting in Production
Model Editing
Digital Hair/Fur
Hair Generation Process
General Issues and Solutions
Digital Feathers
Morphology of Real Feathers Modeling Digital Feathers
Similarities between Hair and Feathers
Differences between Hair and Feathers
General Geometry Instancing
Asset Creation
World Building
Shot Considerations
Dynamics and Simulation
How Is a Simulation Created?
When Is Simulation Appropriate?
Tricks and Cheats
Important Considerations
Planning and Preparation
Software Solutions: A Broad Overview of Current Options
　
Particles
What Are Particle Systems?
The Next Step
The Birth of Particles
Creating Effects
Rigid-Body Dynamics
How Rigid-Body Dynamics Are Created
Potential Problems
Other Issues
Tricks for Getting It Done
Digital Lighting
Light in Reality and in Computer Graphics
Case Study of Reality Compared with Simple CG Simulation
Visual Sophistication through Texture Mapping
Physically Derived Shading Models
Beneath the Surface
Goals of Lighting in Visual Effects
Work Flow for Successful Creative Digital Lighting
The Technologies of Lights in Computer Graphics
Direct Lighting: Source to Surface to Camera
Reflections
Photographed Reflections
Shadows
Image-Based Lighting
Rendering Occlusion
Ambient Occlusion
Reflection Occlusion
Creating Light Sources from Environment Maps
Physically Based Rendering
Physically Plausible Rendering
Volumetric Lighting Effects
Shader Basics
What Are Shaders?
Shading Models
Bump and Displacement
Map-Based Shaders
Procedural Shaders
Shader Design
Antialiasing Considerations
2D Compositing
2D File Formats
Image Quality: Color Bit Depth and Concatenation
Log vs. Linear
Low Dynamic Range and High Dynamic Range Images
Mattes and Pre-multiplied Alpha
Working with Rendered CG Elements
Integration Techniques
2.5D Compositing
Z-Depth Compositing
Adding Depth of Field
Adding Motion Blur
Relighting
3D Compositing
Working with 3D Data in a Compositor
Pan and Tile
Camera Projections
Set Extensions
Coverage Mapping
Cubical Camera Setup
3D Mattes
3D Retouch and Cleanup
Adjusting Camera Moves
Particles
Deep Compositing
Crowd Generation and Simulation Techniques
Live-Action Replication
Sprites
Computer-Generated Crowds
Modeling for Replication
Variation
Mesh Density
Animation Cycles for Replication
Motion Capture
Keyframe Animation
Dynamic Motion Synthesis
Behaviors and Crowd Control
CG Prosthetics and Actor Enhancements
On-Set Tracking and Capture Considerations
Eye Enhancements
3D Techniques
2D Techniques
2.5D Techniques
Silhouette Changes
Re-Projection
3D Products, Systems, and Software
Digital Element Creation Process
3D Graphics Software
3D Tracking
Special Effects
Rendering
Texturing
Chapter 8 Interactive Games - (Revised)
Overview
How the Gaming Industry and Film/TV Industries are different
Game Engines and Real Time Rendering
Runtime Component
Disciplines and Job Titles
Game Design
Engineering:
Production:
Test
Art
The Art Director:
Concept Art
Environment Artists:
Texture Artist:
Characters:
Hard Modelling:
Props:
Lighting:
Effects
Tech Art
Animation
UI
Real Time Shaders and Materials
Pre-Rendered Cinematics vs. Real Time Visuals
Optimization and Runtime Budgets:
Performance Analysis and Profiling:
CPU vs. GPU bound
Technical Terminology:
User Calibration:
Latency:
"Game Mode" on televisions:
HDR10:
PBR - Physically Based Rendering:
FBX:
Mip Mapping:
Filtering:
Texel:
Screen Space Ambient Occlusion:
Level Of Detail:
Vertex Shaders and Fragment Shaders
AR/VR
Future of Gaming
On-demand Rendering, Cloud Distribution and Raytracing
Chapter 9 Complete Animation
What is an Animation Project?
Full Animation versus Visual Effects
Difference Between Visual Effects and Animation
Production Pipelines
Production
A Survey and History of Animation Techniques
Traditional Animation
Stop-Motion
Computer Graphic Technology
Considerations for a Full CG-Animated Feature Pipeline
CG Feature Animation Pipeline
Managing an Animated Film
Film Management and Personal Style
Building Brain Trusts
Building the Core Creative Team
Writing and Visual Development
Working with a Studio
Facilities and Environment
Managing the Event
The Production Process: An Animator's Perspective
Working on CG-Animated Content in Live-Action Features
Planning the Process
Production
Character and Environment Interaction
Chapter 10 Other Workflow Considerations
Virtual Studio Technology
Analysis of a Production Workflow
From Workflow to Pipeline
Service Bureau versus In-House Requirements
Design of a Production Workflow
From Analysis to Design
Deploying a Production Workflow
From Design to Implementation
　
　
Infrastructure
Tracking Assets
What Is Task and Asset Tracking?
Commercial Task and Asset Tracking Systems
Building Task and Asset Tracking Systems
Scene Assembly
3D Scene Assembly
2D Scene Assembly (Compositing)
Working across Multiple Facilities
Images
Models
Texturing
Animation
Compositing
R&D
Chapter 11 VR/AR (Virtual / Augmented Reality)
Prelude to Virtual Reality & Augmented Reality
Pre-Production for VR/AR
Production for VR/AR
Post Production for VR/AR
Editorial Post Production for VR / AR
How to direct the viewer?
The post-process
Non-Linear Editorial, Timelines & Edits
World Lock and Forced Perspective
Types of Head-Mounted and Handheld Displays
What are the HMD and the "VR Presence"?
Electronical Designs of HMDs and their respective performances
Optical Designs of HMDs and their Respective Usages
Image Quality Factors in VR HMD Displays
Hemispheres & Domes
Game Mechanics are what it is all about
Overcoming doubt and preconceived notions
Dome Projections
The Future of Domes
VR / AR Tracking Displays
Uses of VR
Enterprise
Narrative Storytelling
Future of VR & AR
This is just the beginning
Acknowledgments
Appendix A: Charts and Formulas
Appendix B: Credits/Titles to Be Submitted in Accordance with VES
Guidelines
Appendix C: Glossary
Index

Biography

Jeffrey A. Okun, VES, is an award winning Visual Effects Supervisor, having worked on a large number of feature films, commercials and television shows and started in the industry in 1976. He is a member and a Fellow of the VES, The Academy of Motion Pictures, Arts and sciences as well as ASC, the Television Academy and the Editor’s guild. He was the chair of the VES for 9 years and 1st vice chair for 16 years, and chair of the L.A. Section for 2 years. He created the VES awards along with Tim McGovern and Kim Lavery.

Okun created a visual effects tracking and bidding software in 1992 that is in wide use within the industry today, as well as the revolutionary visual effects techniques dubbed the "PeriWinkle Effect" (an underwater blue screen technique) and the "Pencil Effect" (accurately predicts the final visual effects count and budget).
Susan Zwerman, VES has been a member of the VES since 1998. She is a highly respected Visual Effects Producer who has been producing visual effects for more than 25 years. Zwerman is also a well-known seminar leader and author. As Chair of the DGA’s　UPM/AD　VFX Digital Technology Committee, Susan emphasizes the importance of the visual effects teams to DGA members at visual effects seminars organized under her guidance.