VGL

I was introduced to programming through Python, building simple games with pygame. Eventually I tried to simulate physics with it and ran straight into pygame's performance ceiling. Everything it draws is blitted on the CPU, one pixel operation at a time, and that stops scaling the moment you want more than a few hundred moving things on screen. I'd also always been fascinated by how computers actually work, the interaction between CPU and GPU, and how a frame of graphics really gets built. So I wanted real experience with a low-level graphics API, OpenGL, writing shaders from scratch, but I wanted to end up with something as easy to reach for as pygame was. That's what became VGL.

How a GPU actually draws something

Before getting into VGL specifically, it's worth being precise about what a GPU is doing when you ask it to draw a triangle, because the whole reason "shaders" exist falls out of this. The GPU rendering pipeline is a fixed sequence of stages, some of them programmable.

1. Vertex processing. Every vertex you submit runs through a small program (the vertex shader) that transforms it, typically from model space into clip space via a chain of matrices.
2. Primitive assembly and rasterization. The GPU groups transformed vertices back into triangles (or lines, or points) and figures out which pixels each triangle covers. Fixed function, not programmable.
3. Fragment processing. Every covered pixel ("fragment") runs through another small program (the fragment shader) that decides its final color, using whatever was interpolated across the triangle from the vertex stage.
4. Output merging. Depth testing, blending, and finally writing to the framebuffer. Mostly fixed function again.

The two stages you actually write code for are the vertex and fragment shaders, and that's the entire reason "writing shaders" is synonymous with "graphics programming" at this level. Everything else in the pipeline is the GPU doing fixed, hardware-accelerated work you don't control directly.

Shaders, concretely

A shader is a small program, written in GLSL (OpenGL Shading Language, C-like syntax), that runs once per vertex or once per fragment, but runs on thousands of vertices or millions of fragments simultaneously, across the GPU's parallel cores. That massive parallelism, not raw clock speed, is what makes GPU rendering fast. pygame's software rasterizer processes pixels on one CPU core in sequence. A GPU fragment shader processes an entire screen's worth of pixels essentially at once.

Here's VGL's actual default vertex shader, trimmed slightly:

#version 330 core
layout(location = 0) in vec3 aPos;
layout(location = 1) in vec3 aNormal;

uniform mat4 model;
uniform mat4 view;
uniform mat4 projection;

out vec3 FragPos;
out vec3 Normal;

void main() {
  vec4 worldPos = model * vec4(aPos, 1.0);
  FragPos = worldPos.xyz;
  Normal  = mat3(transpose(inverse(model))) * aNormal;
  gl_Position = projection * view * worldPos;
}

Every vertex gets carried through three matrices: model (object space to world space), view (world space to camera space), projection (camera space to clip space). The result, gl_Position, is what rasterization actually uses. The normal gets its own transform, the inverse-transpose of the model matrix rather than the model matrix itself, because a non-uniform scale would otherwise skew normals off-perpendicular from the surface they belong to, a classic graphics gotcha that's easy to get wrong once and then never think about again.

The fragment shader is where per-pixel shading actually happens. VGL's default lighting model is Blinn-Phong: diffuse light proportional to the angle between the surface normal and the light direction, plus a specular highlight from the half vector between light and view direction.

$$I = k_a + (\hat{n}\cdot\hat{l})\, + \left(\hat{n}\cdot\hat{h}\right)^{s}$$

Here $\hat{h} = \widehat{\hat{l} + \hat{v}}$ is the halfway vector between the light direction $\hat{l}$ and the view direction $\hat{v}$, and $s$ is a shininess exponent controlling how tight the highlight is. It's an approximation of real light transport, not a physically based renderer, but it's cheap (a couple of dot products per pixel) and looks convincing enough for real-time work, which is exactly the tradeoff every real-time shading model is making somewhere.

What OpenGL actually is

It's worth clarifying something that isn't obvious walking in. OpenGL isn't a library you link against the way you'd link, say, a math library. It's a specification, a defined API surface and a state machine, that your GPU vendor implements as a driver. When you call an OpenGL function, you're not calling into portable code that runs the same everywhere. You're calling into NVIDIA's, AMD's, Intel's, or Apple's own driver implementation of that spec, and behavior at the margins (extension support, exact precision, even some bugs) genuinely differs between them.

That also means OpenGL by itself doesn't create a window, read keyboard input, or load its own function pointers. That's explicitly out of scope for the spec, and it's why every real OpenGL project reaches for a couple of companion libraries. VGL uses GLFW for windowing, input, and OpenGL context creation (the thing that actually hands you a working, current OpenGL context to issue draw calls into), and GLEW to load the actual function pointers for anything beyond OpenGL's oldest fixed baseline. Modern OpenGL functions aren't guaranteed to be linkable symbols at compile time. They're resolved at runtime against whatever the driver actually exposes. None of that is specific to VGL. It's the standard shape every raw-OpenGL project takes, and understanding why each piece exists (spec versus driver versus windowing versus function loading) makes the whole ecosystem make a lot more sense than just copying a boilerplate setup.

How different OSes actually handle graphics

This part surprised me the most going in, because it's easy to assume "OpenGL" means the same thing everywhere. It doesn't, and the differences are almost entirely a story about who owns the driver.

• Linux. Mostly open. Mesa provides open-source OpenGL drivers for AMD and Intel GPUs, communicating with the kernel's DRM/KMS graphics stack. NVIDIA instead ships its own proprietary driver. Either way, GLFW talks to the windowing system (X11 or Wayland) underneath, and OpenGL support is generally current and well maintained.
• Windows. Vendor owned. NVIDIA, AMD, and Intel each ship their own OpenGL implementation as part of their standard graphics driver (an "ICD," installable client driver), sitting alongside, not replaced by, Direct3D, Microsoft's own competing graphics API that most AAA Windows games actually target instead.
• macOS, and specifically Apple Silicon. This is the interesting one. Apple deprecated OpenGL back in 2018 (macOS Mojave) in favor of their own API, Metal, and hasn't shipped a new OpenGL version since. macOS is permanently capped at OpenGL 4.1, itself already over a decade old. On Apple Silicon there's no native hardware OpenGL driver at all anymore. Apple's own OpenGL framework is a translation layer sitting on top of Metal, not a first-class driver path. It still works, and VGL runs on it fine, but it's genuinely the least native path on any of the three platforms. Every OpenGL call is ultimately being reinterpreted into Metal calls, one abstraction removed from the actual GPU driver, unlike Linux and Windows, where OpenGL drivers talk to the hardware directly.

Practically, this means a raw-OpenGL renderer like VGL is guaranteed a working baseline everywhere via GLFW, but on Apple Silicon it's running on borrowed time architecturally, not just older-spec-version time. If I ever wanted VGL's newest features or best performance on a Mac, the honest long-term answer is a Metal backend, not a newer OpenGL version that Apple isn't going to ship.

Designing the API: pygame's feel, the GPU's throughput

The actual design goal was recreating pygame's drawLine/drawCircle immediate-mode feel without recreating its CPU bottleneck. The trick is that "immediate mode" only has to be true from the caller's perspective. VGL's LineBatch2D/PointBatch2D accumulate every draw call made during a frame into a CPU-side buffer, then upload once and issue a single GPU draw call for the whole batch, using glVertexAttribDivisor to instance a shared unit-circle mesh once per point rather than generating unique geometry per primitive. The caller writes drawLine(...) a thousand times per frame and it feels immediate mode. The GPU sees one draw call.

The other half of the 2D side is built for CFD-style scalar-field visualization, velocity, pressure, vorticity grids, which is a genuinely different problem from drawing shapes. DataTexture uploads the raw scalar grid straight into a floating-point GPU texture, and a fragment shader maps every pixel through a colormap (Viridis, Jet, CoolWarm) entirely on the GPU. No per-pixel CPU work at all, which is the only way that kind of visualization stays real-time once the grid gets large. Reading the field back to the CPU to color it by hand would put you right back into pygame's bottleneck, just with extra steps.

Performance lessons that don't show up in a correctness check

Two bugs taught me that "it draws the right thing" and "it draws the right thing fast" are genuinely separate bars to clear, and the gap between them doesn't show up until a scene gets busy.

Uniform lookups on every draw. Shader::setMat4/setFloat were calling glGetUniformLocation(id, name), a driver-side string lookup by name, on every single call, every frame, for every draw. It's correct, and invisible in a scene with a handful of objects. It becomes a real cost once a frame is issuing hundreds of these lookups per draw call. The fix was a name to GLint cache on Shader, invalidated only when the program relinks, so each uniform name resolves once per program's lifetime instead of once per frame.

GPU object churn in immediate-mode line drawing. GUI::drawLine/drawArrow were destroying and recreating the underlying VAO/VBO from scratch on every single call. Fine for one debug line, expensive for a frame full of vector-field arrows. I rewrote it to match the pattern the batch renderers already used: allocate once, grow the buffer with headroom only when a frame actually needs more capacity, and glBufferSubData into the existing allocation otherwise. Both bugs share a shape worth naming. The code was correct and the API looked identical from the outside. The cost was purely in how often the GPU boundary got crossed, which is exactly the kind of thing a quick correctness check will never catch.

The bug that taught me the most: it wasn't a crash

The most instructive problem I hit wasn't a crash. It was "why does every 3D scene look almost black except directly facing the light." The fragment shader had a single hardcoded ambient = 0.15 constant and one directional light. Any face angled away from that light dropped to 15% brightness with nothing to fill it in, and there was no way to configure it beyond rotating the light itself. What made it take real investigation: OBJMesh was already parsing Ka/Ks/Ns (ambient, specular, shininess) out of .mtl material files into a Material struct. The data was there. The shader just never read it. Every object got the same fixed shininess and specular regardless of what its actual material specified, so my first instinct ("the light direction must be wrong") was chasing the wrong layer entirely.

The real fix had two parts: replace the flat ambient constant with a hemisphere ambient term (a sky color and a ground color blended by how much a surface normal faces up) so backfaces pick up a believable fill instead of going flat dark, and actually thread the already-parsed Material struct into the shader as real per-object uniforms, plus a lit flag so an object can opt out of lighting entirely for a flat-shaded look. Getting from "the light must be positioned wrong" to "the ambient model has no fill term at all, and unrelated to that, the material data is being silently discarded" was the part that actually took time. Once correctly diagnosed, the shader and API changes themselves were mechanical.

Lessons learned

Batching and buffer-reuse patterns are easy to get right once, in one place, and easy to forget to apply everywhere else that looks similar. LineBatch2D had the right growable-buffer pattern from the start. A separate, older immediate-mode line path in GUI didn't, purely because it predated the batch renderers and nobody went back to reconcile the two. Configurability rots quietly the same way. OBJMesh gained real Material parsing before the shader had any uniforms to actually receive it, so the data sat parsed but unused for a while with no error or warning, the kind of gap that only surfaces when someone reads both sides of an interface at once, instead of just the side they're currently changing. Both are the same underlying lesson from two different angles. A change that's locally correct can still leave the system inconsistent, and finding that requires deliberately going looking, not waiting for a crash to point at it.