Overview
free

Appendix A - Diagram Templates by Step
Appendix B - Mapping Concepts to Real-World APIs and Engines
Appendix C - Readiness Checklists for Moving Up the Incremental Ladder
Appendix D - Glossary of Canonical Terms

Course Setup and the Incremental Ladder
Why "Pixels to Photons": Rendering as an Integration Problem
How to Use This Course: Modules as Rendering Slices
The Incremental Ladder (Step 0 -> Step 7): Building Rendering Capability Without Losing Boundaries
The Course Lenses: Five Ways to Look at the Same Frame
Diagram Legend and Notation Types: Making Rendering Boundaries Visible

What Is a Rendering Engine?
Rendering Engines vs Game Engines vs UI/Compositing: Responsibilities and Boundaries
Core Responsibilities: Transforms -> Shading -> Composition -> Output as the Minimal Engine Contract
Real-Time vs Offline at a High Level: Budgets, Quality Targets, and Architectural Implications

Pixels, Color, and Raster Images
Pixels, Resolution, Aspect: Sampling the World into a Grid and the Consequences for Aliasing
Color Channels and Color Spaces: Linear vs Gamma Concepts, and Why Wrong Space Breaks Lighting
Framebuffers and Presentation: Render Targets and Swap Chains as the Display Boundary (Conceptual)

Coordinate Spaces
Local, World, View, Clip, Screen: The Canonical Space Ladder and What Each Space Is For
2D vs 3D Coordinate Systems: Why the Same Math Underlies Both but Different Conventions Matter
Transform Stacks Conceptually: Translation/Rotation/Scale Composition as the Engine's Spatial Contract

The Modern GPU Pipeline
Fixed-Function vs Programmable Stages: What Is Baked Into Hardware and What You Control
Vertex to Fragment to Blend: The End-to-End Dataflow of a Draw Call
API-Agnostic Pipeline Thinking: Designing Around Concepts Rather Than a Specific Graphics API

Diagramming Rendering Systems
Draw-Call Flow Diagrams: App -> Command Recording -> GPU Submission as the Execution Boundary
Transform Flow Along the Pipeline: How Coordinates Change at Each Stage and How to Debug Mistakes
Layered Renderer Diagrams: Scene Representation, Culling, Passes, Post-Processing as Separable Subsystems

Step 0 Drawing 2D Geometry
Lines, Triangles, Quads: The Primitive Vocabulary of Raster Engines
Screen-Space vs World-Space Primitives: Choosing Coordinate Conventions for 2D Scenes
Textured Quads: Mapping Images to Geometry as the First Bridge to Materials

Step 0 Transforming Geometry
Object -> World -> View -> Projection: Composing Transforms as the Canonical Render Path
Orthographic vs Perspective Projections: What Projection Choice Implies for Depth and Scale
The Camera as "Moving the World": Debugging Camera Transforms and Common Sign Errors

Step 0 Rasterization and Sampling
Triangles to Fragments: Rasterization as a Sampling Process
Depth and Visibility: Z-Buffer Intuition and Why Ordering Matters Less Than You Expect
Texture Sampling Basics: How Fragment Shaders Look Up Texels and Where Aliasing Begins

Step 0 Simple 2D Rendering Pipeline
Sprites and UI Elements: Building a Minimal Retained Model or Immediate Draw Layer
Layering and Z-Order: Compositing Discipline for Predictable 2D Scenes
Alpha Blending: Transparency Basics and the First Encounter with Ordering Problems

Step 0 Minimal 3D Rendering Pipeline
Upload Mesh, Use Camera, Draw: The Smallest 3D Loop That Demonstrates the Full Transform Chain
Frame Lifecycle: Update -> Record -> Submit -> Present as the Core Engine Heartbeat
OS/Window Integration Boundary: The Platform Edge Where Rendering Meets Presentation

Step 1 Shaders as Programs on the GPU
Vertex and Fragment Shaders: Where Geometry Becomes Pixels
Shader Inputs: Attributes, Uniforms, Push Constants, Textures as Explicit Interface Contracts
Compilation and Linking: Managing Shader Build Artifacts as Part of the Engine Toolchain

Step 1 Materials and Parameterization
Materials as Shader + Parameters: Separating "What Code Runs" from "What Values Drive It"
Surface Parameters: Color, Roughness, Metalness-Like Concepts and Why They Reduce Content Ambiguity
Parameter Maps: Albedo/Normal/Roughness and How Texture Conventions Become Engine Contracts

Step 1 Texturing and Sampling
UVs and Texture Mapping: The Address System for Surface Detail
Filtering Concepts: Nearest vs Linear and Why Filtering Changes Perceived Material Properties
Mipmaps and Aliasing: Reducing Shimmering by Trading Detail for Stability

Step 1 The Rendering Equation Without the Math Pain
Incoming Light to Outgoing Light: The Conceptual Inputs and Outputs of Shading
BRDF Intuition: How the Surface Responds as a Programmable Model
Integration as an Idea: Why Light Comes From Many Directions Even if You Approximate It

Step 1 Material Systems in Engines
Material Graphs vs Parameter Blocks: Flexibility Versus Predictability and Performance
Material Instances: Reusing a Base Material with Different Parameters as a Content-Scaling Strategy
Multiple Shading Models: Supporting Variety Without Exploding Engine Complexity

Step 1 Shader and Material Abstractions
Decoupling Engine Logic from Shader Source: Maintaining Stable Interfaces Across Refactors
Cross-Platform Shader Handling: API-Agnostic Strategies That Avoid Leaking Backend Details
Permutations and Variants: Controlling Combinatorial Growth in Features and Quality Tiers

Step 2 Local Illumination Models
Diffuse and Specular: Splitting Response into Conceptually Distinct Lobes
Classic Models: Lambert and Simple Specular as the Baseline Approximations
Multiple Light Types: Directional, Point, Spot and How Each Changes Attenuation and Cost

Step 2 Modern Shading Models (High-Level)
Energy Conservation and Roughness: Why "Physically Inspired" Reduces Content Tuning Chaos
Environment Lighting: Image-Based Lighting Concepts and the Content Dependencies It Introduces
Consistency Across Scenes: Shading as an Engine-Wide Contract, Not Per-Asset Artistry

Step 2 Shadows
Shadow Mapping Flow: Rendering Depth from Light and Using It at Shading Time (Conceptual)
Partitioned Approaches: Cascaded and Partitioned Strategies as a Response to Scale and Resolution Limits
Soft Shadows Conceptually: Penumbra, Filtering, and the Cost of Plausibility

Step 2 Ambient and Indirect Lighting (Conceptual)
Ambient Terms: Why They Exist and Why They Are Often Visually Wrong but Operationally Cheap
Indirect Lighting at a High Level: What Global Illumination Tries to Capture Conceptually
Approximations and Baking: When You Precompute Light to Buy Back Frame Time

Step 2 Normal Mapping and Detail
Normals as Shading Inputs: Why Surface Orientation Is the Core of Lighting
Tangent-Space Normal Mapping: Adding Detail Without Adding Triangles
Combining Geometry and Normal Detail: Avoiding Artifacts from Mismatched Scale and Filtering

Step 2 Lighting Architectures in Real-Time Engines
Forward vs Deferred (Conceptual): Moving Cost Between Shading and Composition
Many Lights at Scale: Managing Light Lists, Culling, and Shading Cost Growth
Simplicity vs Flexibility vs Performance: Why Lighting Architecture Is an Engine Identity Decision

Step 3 2D Rendering Models
Immediate Mode: Draw-Per-Frame Simplicity and Why CPU Overhead Becomes the Limiter
Retained Mode: Scene Graphs and Retained Objects as a Batching and Tooling Enabler
Composition for UI/HUD: Layering Policies and the Interaction Between 2D and Post-Processing

Step 3 Sprites, Tilemaps, and Vector Graphics
Sprite Batching and Atlases: Reducing Draw Calls by Changing Content Representation
Tilemaps and Parallax: Organizing Large 2D Worlds as Repeated Structure
Vector Primitives Conceptually: Paths and Strokes and Why They Imply Different Tessellation Strategies

Step 3 Text Rendering
Glyphs and Atlases: Caching as the Difference Between Usable and Unusable Text Systems
Hinting and Scaling Concepts: Why Text Quality Is a Sampling and Layout Problem
Text in the Render Pipeline: Ordering, Blending, and Performance Interactions with UI

Step 3 3D Scene Representation
Scene Graphs vs Flat ECS-Style Lists: Hierarchy Benefits Versus Update and Traversal Cost
Cameras, Lights, Meshes, Bounds: The Minimal 3D Scene Vocabulary
Spatial Partitioning Concepts: Grids and Trees as Visibility and Query Accelerators

Step 3 Visibility and Culling
Frustum Culling: Removing Invisible Work Before It Becomes GPU Cost
Backface and Occlusion Concepts: Visibility Pruning Trades Compute for Bandwidth
Culling Complexity vs Wins: When the Cure Becomes Worse Than the Disease

Step 3 Combining 2D and 3D in One Engine
UI Overlays and HUDs: Compositing Boundaries and Render Ordering
2D in 3D: Billboards and Screen-Space Effects as Hybrid Representations
Render Order and Post: Managing Transparency, Post-Processing, and Composition Consistency

Step 4 Real-Time Rendering Constraints
Frame Budgets: Stable Frame Time as a User-Facing Contract
Spikes vs Throughput: Why Occasional Stalls Feel Worse Than Slightly Lower Average Quality
Interactivity vs Quality: Making Approximation Choices Explicit Per Feature

Step 4 Offline Rendering Characteristics
Quality Without Hard Budgets: Trading Time for Accuracy and the Operational Implications
Samples and GI Conceptually: Why Noise and Convergence Dominate Offline Pipelines
Render Farms and Batch Pipelines: Distributed Rendering as a Compute Orchestration Problem (Conceptual)

Step 4 Baking and Precomputation
Lightmaps and Probes: Precomputing Expensive Lighting into Textures and Volumes
Offloading Runtime Work: Shifting Computation from Frame Time to Build Time
Dynamic vs Static Lighting: Splitting Content by Change Frequency as an Engine Constraint

Step 4 Hybrid Pipelines
Offline Precompute in Real-Time: Integrating Baked Data Without Losing Flexibility
Real-Time Compositing of Offline Content: Layering Cinematics and High-Quality Passes
Cinematics vs Gameplay Rendering: Differing Quality Targets Inside One Product

Step 4 Choosing Rendering Approaches Per Feature
When to Accept Approximations: Cost-Benefit Decisions Under a Frame Budget
Dynamic vs Static Splitting: Isolating Expensive Features to Avoid Global Budget Collapse
Feature-Level Decisions: Hair, Particles, and Water as Domain-Specific Trade Spaces

Step 5 Where Time Goes in Rendering
CPU vs GPU Responsibilities: Submission, Culling, State Setup Versus Shading and Bandwidth
Draw Calls and State Changes: Why "Too Many Small Things" Is a Common Failure Mode
GPU Bottleneck Taxonomy: Vertex, Fragment, Bandwidth and How Symptoms Differ

Step 5 Overdraw, Fill Rate, and Complexity
Overdraw Costs: Why Transparency and Layering Can Dominate Frame Time
Depth Pre-Pass Philosophies: Reducing Wasted Shading by Paying Extra Passes (Conceptual)
Resolution and MSAA-Like Costs: Trading Clarity for Bandwidth and Shading Time

Step 5 Geometry Optimization
LOD Strategies: Reducing Geometry Cost Without Visible Popping
Instancing: Amortizing Submission and State Across Repeated Objects
Mesh Optimization Concepts: Vertex Cache Behavior and Index Ordering as Hidden Performance Levers

Step 5 Texture and Memory Management
Texture Budgets and Streaming: Managing VRAM as a Constrained, Shared Resource
Mip Control and Streaming Policies: Choosing What Stays Resident to Avoid Thrash
Texture Format Concepts: Block Compression Versus Uncompressed and the Quality/Performance Trade

Step 5 Resource Lifetime and Pools
Managing Buffers, Textures, Targets: Lifetime Discipline as the Antidote to Leaks and Spikes
Pools and Transient Allocations: Reducing Allocation Overhead and Fragmentation
Fragmentation and Leaks: How Memory Failures Show Up as Stutters and Instability

Step 5 Profiling and Debugging Performance
Profilers and Counters: Turning Frame Time Into Evidence
CPU/GPU Sync and Pipeline Bubbles: Diagnosing Stalls as Boundary Problems
Building Internal Visualization Tools: Making Performance Explainable

Step 6 Assets and Content as First-Class Citizens
Asset Types and Dependencies: Thinking in Graphs
Source vs Engine-Ready Formats: Conversion as a Boundary
Metadata and Versioning: Making Content Reproducible

Step 6 Importers, Converters, and Build Pipelines
Offline Conversion and Baking: Build-Time Work That Buys Runtime Determinism
Re-Import and Change Detection: Keeping Iteration Fast Without Corrupting Outputs
Engine-Specific Formats: Optimized Representations Rather Than Raw Content

Step 6 Material and Shader Authoring Tools
Node-Based Material Editors: Enabling Artist Iteration While Preserving Engine Constraints
Shader Graph vs Hand-Written Code: Productivity Versus Control and Debugging Burden
Live Preview and Iteration Loops: Shortening the Feedback Cycle Without Making Builds Non-Reproducible

Step 6 Scene and Level Editors
Editing Scenes In Context: Object Placement, Lights, Cameras, and the Need for Stable Serialization
Editing Materials and Post Effects: Authoring with Immediate Visual Feedback and Correct Defaults
Saving, Loading, and Versioning Scenes: Avoiding Content Corruption in Collaborative Workflows

Step 6 Debugging and Visualization Tools
Diagnostic Views: Wireframe, Overdraw, Normals and What Each Reveals
Buffer Inspection Flows: Frame Capture and Pass Inspection (Conceptual)
Visualizing Culling, LOD, and Light Complexity: Making Invisible Performance Costs Visible

Step 6 Integrating Tools into the Engine Workflow
In-Editor vs External Tools: Choosing Integration Depth and Managing Complexity
Asset CI and Build Pipelines: Treating Content as Build Artifacts with Tests and Reproducibility
Collaboration Workflows: Artists, Tech Artists, Engineers and the Contracts that Keep Teams Unblocked

Step 7 Engine Architecture and Subsystems
Core Subsystems: Resource Manager, Renderer, Scene System, Tools and Why Boundaries Matter
Platform Layer and API Abstraction: Isolating Backend Variability Without Hiding Essential Constraints
Game/App vs Renderer Boundary: Defining the Interface that Prevents Unbounded Coupling

Step 7 Render Graphs and Multi-Pass Rendering
Render Pass Dependencies: Expressing Ordering and Resource Usage as an Explicit Graph
Render Graph as Description: Separating "What to Render" from "How to Schedule" (Conceptual)
Post, G-Buffers, Offscreen: Composing Complex Pipelines Without Losing Debuggability

Step 7 Multi-Platform and Multi-API Rendering
Abstracting APIs Without Leaks: Deciding What Must Remain Backend-Specific
Capability Differences: Designing Around Tiers Rather Than Assuming Uniform Hardware
Feature Detection and Quality Levels: Mapping Capabilities to Content and Shader Permutations

Step 7 Extensibility and Plugin Architectures
Adding New Passes and Effects: Extension Points That Preserve Engine Stability
Modularizing the Renderer: Isolating Experiments From the Stable Core
Supporting Custom Pipelines: Enabling Variation Without Fragmenting the Codebase

Step 7 Testing, Validation, and Stability
Visual Regression Testing: Golden Images and How to Manage Acceptable Variation
Performance Regression Testing: Preventing "Small Changes" From Breaking Frame Budgets
Maintenance Over Time: Refactors, Deprecation, and Compatibility Across Hardware Generations

Step 7 Reference Architectures and Maturity
Minimal Engine - A Stable Foundation You Can Ship
Intermediate Engine - Managing Growth Without Losing Boundaries
Advanced Engine - Explicit Policy, Capability Tiers, and Long-Term Evolution

/pixels-to-photons/pixels-to-photons

The Modern GPU Pipeline

Sign in to access this lesson.

Sign in Create account