GAMES101现代计算机图形学入门

2024-06-05

🏷️ 图形学

以下为笔者学习GAMES101过程中的笔记，包含了从几何变换到光栅化的内容。

Overview

课程主页

Computer graphics:The use of computers to synthesize and manipulate visual information.

Course Topics (mainly 4 parts)

  • Rasterization
  • Curves and Meshes
  • Ray Tracing
  • Animation/Simulation

即光栅化成像，几何表示，光的传播理论，动画与模拟

光栅化(Rasterization)指的是将三维图形实时显示在二维平面上。在图形学中，30 FPS 就可以认为是实时的，低于这个帧率的则认为是离线渲染。

光线追踪(Ray tracing)指的是像每一个像素发射光，令光线不断反射。这样的技术能够让渲染更加真实，但是效率会更低。使用光线追踪是效果和效率的 Trade-off。

从二维图像和三维模型的角度出发，我们认为一切以三维模型为起点的都是计算机图形学(Computer graphics)的范畴，而所有从二维图像为起点的都是计算机视觉(Computer vision)。特别的，从三维模型出发获得二维图像的称为渲染(Rendering），从三维模型出发获得新的三维模型的可以是模拟(Simulation)。但是上述的分类也是粗糙而模糊的。

Transformation

几何变换(translate——位移，transform——转换)

基础知识: 线性代数，向量和矩阵的基本运算 (向量的叉乘可通过矩阵形式表示，如下图)

2D Transformations

Representing transformations using matrices

  • Scale Matrix:[x′y′]=[Sx00Sy][xy]\begin{bmatrix}x'\\y'\end{bmatrix}=\begin{bmatrix}S_x&0\\0&S_y\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix}

  • Reflection Matrix:[x′y′]=[−1001][xy]\begin{bmatrix}x'\\y'\end{bmatrix}=\begin{bmatrix}-1&0\\0&1\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix}

  • Shear Matrix:[x′y′]=[1a01][xy]\begin{bmatrix}x'\\y'\end{bmatrix}=\begin{bmatrix}1&a\\0&1\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix}

  • Rotation Matrix:[x′y′]=[cos⁡θ−sin⁡θsin⁡θcos⁡θ][xy]\begin{bmatrix}x'\\y'\end{bmatrix}=\begin{bmatrix}\cos\theta&-\sin\theta\\\sin\theta&\cos\theta\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix}

旋转矩阵是正交矩阵，想求逆直接求其转置

  • Linear Transforms = Matrices(of the same dimension)
x′=ax+byy′=cx+dy[x′y′]=[abcd][xy]x′⃗=Mx⃗\begin{gather} x'=ax+by \\ y'=cx+dy \\ \begin{bmatrix}x'\\y'\end{bmatrix}=\begin{bmatrix}a&b\\c&d\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix} \\ \vec{x'}=M\vec{x} \end{gather}
  • Translation cannot be represented in matrix form(So, translation is Not linear transform!)
[x′y′]=[abcd][xy]+[txty]\begin{gather} \begin{bmatrix}x'\\y'\end{bmatrix}=\begin{bmatrix}a&b\\c&d\end{bmatrix}\begin{bmatrix}x\\y\end{bmatrix}+\begin{bmatrix}t_x\\t_y\end{bmatrix} \end{gather}

Affine map = linear map + translation (x′y′)=(abcd)⋅(xy)+(txty)\begin{pmatrix}x'\\y'\end{pmatrix}=\begin{pmatrix}a&b\\c&d\end{pmatrix}\cdot \begin{pmatrix}x\\y\end{pmatrix}+\begin{pmatrix}t_x\\t_y\end{pmatrix} Using homogenous coordinates: (x′y′1)=(abtxcdty001)⋅(xy1)\begin{pmatrix}x'\\y'\\1\end{pmatrix}=\begin{pmatrix}a&b&t_x\\c&d&t_y\\0&0&1\end{pmatrix}\cdot \begin{pmatrix}x\\y\\1\end{pmatrix}

3D Transformations

Use 4×44\times4 matrices for affine transformations (x′y′z′1)=(abctxdeftyghitz0001)⋅(xyz1)\begin{pmatrix}x'\\y'\\z'\\1\end{pmatrix}=\begin{pmatrix}a&b&c&t_x\\d&e&f&t_y\\g&h&i&t_z\\0&0&0&1\end{pmatrix} \cdot \begin{pmatrix}x\\y\\z\\{1}\end{pmatrix}

(注意绕y轴旋转的矩阵较为特殊，原因是：x⃗×z⃗=−y⃗\vec{x}\times\vec{z}=-\vec{y}) Rotation by angle α\alpha around axis nn R(n,α)=cos⁡(α)I+(1−cos(α))nnT+sin⁡α(0−nznynz0−nx−nynx0)R(n,\alpha)=\cos(\alpha)I+(1-cos(\alpha))nn^T+\sin\alpha\begin{pmatrix}0&-n_z&n_y\\n_z&0&-n_x\\-n_y&n_x&0\end{pmatrix} (默认nn为过原点的单位向量) 推导过程如下: (其中S⃗绕a⃗转θ∘\vec{S}绕\vec{a}转\theta^{\circ})

Viewing Transformation

View/Cemera Transformation

  • What is view transformation?

  • Think about how to take a photo

    • Find a good place and arrange people (model transformation)
    • Find a good “angle” to put the camera (view transformation)
    • Cheese! (projection transformation) (三个变换过程) (为了计算方便，把观测点移到原点并通过旋转和坐标轴对齐，再对物体进行同样的变换，保证相对位置不变； 求旋转矩阵的方法：先求出逆变换，再转置)

  • Projection in Computer Graphics

    • 3D to 2D
    • Orthographic projection
      • In general, we want to map a cuboid [l,r]×[b,t]×[f,n][l, r] \times [b, t] \times [f, n] to the “canonical (正则、规范、标准)” cube [−1,1]3[-1, 1]^3
    • Perspective projection

如何推导? 变换满足条件: 所有点的x和y坐标上原来的nz\cfrac{n}{z}倍，同时近平面上坐标全部不变，远平面上z坐标不变，且中心点坐标不变

Mpersp→ortho=(n0000n0000n+f−nf0010)Mpersp=MorthoMpersp→ortho\begin{gather} M_{persp\to ortho}=\begin{pmatrix}n&0&0&0\\0&n&0&0\\0&0&n+f&-nf\\0&0&1&0\end{pmatrix}\\ M_{persp}=M_{ortho}M_{persp\to ortho} \end{gather}

Rasterization

  • Rasterizing one triangle
  • Sampling theory
  • Antialiasing

What’s after MVP ? 将所得的模型输出到屏幕上

  • Irrelevant to z
  • Transform in xy plane: [−1,1]2[-1, 1]^2 to [0,width]×[0,height][0, width] \times [0, height]
  • Viewport transform matrix:
Mviewport=(width200width20heigtht20height200100001)\begin{gather} M_{viewport}=\begin{pmatrix}\cfrac{width}{2}&0&0&\cfrac{width}{2}\\0&\cfrac{heigtht}{2}&0&\cfrac{height}{2}\\0&0&1&0\\0&0&0&1\end{pmatrix} \end{gather}

Triangles—Fundamental Shape Primitives

Rasterizing Triangles into Pixels 判断点在三角形内的方法：向量叉乘(对边界情况不做处理) 简化计算: Use a Bounding Box! http://www.sunshine2k.de/coding/java/TriangleRasterization/TriangleRasterization.html#algo1

Sampling Artifacts(Errors/ Mistakes/ Inaccuracies) in Computer Graphics Signals are changing too fast (high frequency), but sampled too slowly

Antialiasing Idea: Blurring (Pre-Filtering) Before Sampling 采样之前进行模糊处理 (出现锯齿的本质原因是信号变换太快而采样速度跟不上，通过模糊的方法降低频率，从而保证采样率能跟上) Filtering = Getting rid of certain frequency contents = Convolution(= Averaging) Convolution in the spatial domain is equal to multiplication in the frequency domain, and vice versa

How Can We Reduce Aliasing Error? Option 1: Increase sampling rate

  • Essentially increasing the distance between replicas in the Fourier domain
  • Higher resolution displays, sensors, framebuffers…
  • But: costly & may need very high resolution Option 2: Antialiasing
  • Making Fourier contents “narrower” before repeating
  • i.e. Filtering out high frequencies before sampling

Antialiasing by Computing Average Pixel Value Antialiasing By Supersampling (MSAA) 对一个像素点采样更多次，取平均值

Visibility / occlusion

  • Z-buffering Idea:
  • Store current min. z-value for each sample (pixel)
  • Needs an additional buffer for depth values
    • frame buffer stores color values
    • depth buffer (z-buffer) stores depth For simplicity we suppose z is always positve(smaller z -> closer, larger z -> further)

Shading

The process of applying a material to an object.

  • Illumination & Shading
  • Graphics Pipeline

  • Blinn-Phong reflflectance model

    • Diffuse
    • Specular
    • Ambient

  • At a specifific shading point

  • Shading frequencies

  • Graphics pipeline

  • Texture mapping

  • Barycentric coordinates

（实际上允许使用视角向量和镜面反射向量的接近程度来计算高光，而这种计算方式就称为 Phong 模型，Phong模型与Bling-Phong模型产生的高光效果不一样，Bling-Phong产生的高光更加柔和。）

Shading Frequencies: Shade each triangle (flat shading)

  • Triangle face is flat — one normal vector
  • Not good for smooth surfaces Shade each vertex (Gouraud shading)
  • Interpolate colors from vertices across triangle
  • Each vertex has a normal vector (how?) Shade each pixel (Phong shading)
  • Interpolate normal vectors across each triangle
  • Compute full shading model at each pixel
  • Not the Blinn-Phong Reflectance Model

Graphics (Real-time Rendering) Pipeline

Shader Programs

  • Program vertex and fragment processing stages
  • Describe operation on a single vertex (or fragment)

Texture Mapping Visualization of Texture Coordinates Each triangle vertex is assigned a texture coordinate (u,v)

Interpolation Across Triangles: Barycentric Coordinates Interpolation Across Triangles Why do we want to interpolate?

  • Specify values at vertices
  • Obtain smoothly varying values across triangles What do we want to interpolate?
  • Texture coordinates, colors, normal vectors, … How do we interpolate?
  • Barycentric coordinates
α=−(x−xB)(yC−yB)+(y−yB)(xC−xB)−(xA−xB)(yC−yB)+(yA−yB)(xC−xB)β=−(x−xC)(yA−yC)+(y−yC)(xA−xC)−(xB−xC)(yA−yC)+(yB−yC)(xA−xC)γ=1−α−β\begin{aligned} \alpha = \cfrac{-(x-x_B)(y_C-y_B)+(y-y_B)(x_C-x_B)}{-(x_A-x_B)(y_C-y_B)+(y_A-y_B)(x_C-x_B)}\\ \beta = \cfrac{-(x-x_C)(y_A-y_C)+(y-y_C)(x_A-x_C)}{-(x_B-x_C)(y_A-y_C)+(y_B-y_C)(x_A-x_C)}\\ \gamma=1-\alpha-\beta \end{aligned}

Applying Textures 一个像素，在近处覆盖的区域小，在远处覆盖的区域大 Texture Magnification (What if the texture is too small?) Bilinear Interpolation

Super-sampling: costly, signal frequency too large in a pixel 大量采样取平均值->不采样，直接范围查询 Mipmap

  • don’t sample, get the average value within a range Allowing (fast, approx., square) range queries 只能做近似的正方形范围查询 Overblur 一个像素映射到纹理上的形状可能不是正方形 使用各向异性过滤解决矩形插值