It’s past time I wrote a bit about the work I’ve been doing for the VICE project in detail.
Most of it is drudgework, really; there a large number of events that the emulation core can send to the UI, and the UI has to make them visible to the end user. Likewise, the end user will interact with the system and the UI has to translate those actions into calls that the emulation core can understand. Most of that is finicky but also very ad-hoc and not terribly generalizable.
That said, there are still a few techniques that have come up that are simple enough to fit in a blog post, complex enough that they aren’t totally free, and niche enough that there aren’t a billion writeups of the techniques involved already.
Here’s one of them.
Here’s what VICE looks like while it’s running:
We are interested in the primary portion of the display; the large widget in the middle of the application that actually displays the Commodore 64’s “screen”. The actual data is constructed, pixel by pixel, by the emulator core and put into a memory buffer. Our job is to make use of it. The basic requirements are pretty simple:
- All of the pixel data must actually go to the screen.
- The pixel data should thus imply a minimum window size, so that everything generated fits.
The rest of the requirements make things a bit more exciting, though:
- The window should be able to be resized arbitrarily as long as the minimum window size is respected.
- The emulated display should be as large as it can be while still fitting within the current window size.
- The emulated display should preserve its aspect ratio within the application window as a whole, letterboxing or pillarboxing as necessary but but without constraining the shape of the application window itself. (This is a relatively modern requirement, but is necessary to correctly handle “fullscreen” applications. In earlier years, one went fullscreen by actually altering the display’s resolution. This was easy for applications to code for but produced a lot of havoc on desktop icons, any other applications that might have been running, and any kind of multi-monitor system. The rise of ubiquitous GPUs—and the development of techniques like the one in this article—altered expectations so that a fullscreen application is equivalent to a single window on the desktop that covers the entire desktop.)
- The displayed image should preserve the fact that the 8-bit computer’s pixels weren’t square.
- The minimum window size should adjust itself so that it’s just large enough to contain the screen of non-square pixels, despite the fact that the window’s own pixels are still square.
That’s a much longer list, but an awful lot of it boils down to making sure that the machinery of the UI interacts sanely with the rendering context. From the point of view of the rendering context, it only sees three tasks:
- We have a rectangle to draw the display into. Start by clearing it to some nice neutral color like black or dark grey.
- Draw another rectangle inside of it.
- Have that rectangle take its colors from some rectangular array of pixel data.
Everything in our list of requirements boils down to determining the size of that second rectangle.
The Old School
When I was working on The Ur-Quan Masters we faced the same issue, but in 2001 most people’s graphics cards didn’t work the way modern ones do—they were more like specially-designed circuits built to render certain kinds of scenes, and they needed to be tricked into actually doing the work we want.
The general technique was to set up the camera and the projection transforms so that they represented a space the size of the screen to display, without any real notion of distance (that is, it was an “isometric” instead of “perspective” transform). One would then disable lighting computations and render two flat matte-white triangles that created that internal rectangle. The older graphics pipelines had a notion that these triangles could be “textured” with a repeating, small pattern that would look like grass or brick or similar. We would load the screen we wanted to display into one enormous texture and then map it to the rectangle such that it does not repeat at all.
The Modern World
Even in 2001, though, change was in the wind. Graphics cards were moving from, essentially, ASICs designed to do perspective transforms of textured triangles with hardcoded lighting and fog equations into massively parallel vector processors that graphics developers would program directly. It would take another five years or so before this became ubiquitous, but once it did it became very ubiquitous. Despite being more generic, it also ended up ultimately becoming more consistent across both the low and high end devices.
The best treatment of how modern graphics programming work is Learning Modern 3D Graphics Programming by Jason L. McKesson. It makes a deliberate decision to ignore the history of the field, treating the entire fixed-function rendering pipeline of the late 1990s and early 2000s as a giant misstep. That’s probably the best way to learn it, honestly, but I spend a lot of time on Bumbershoot Software looking at the how the past relates to the present, so…
The earlier rendering techniques used two transform matrices to represent the world and the camera, and optionally included additional information about color and texture information that was associated with each vertex. All of these operations are collapsed into a single program called the vertex shader, which consumes arbitrary arrays of information and outputs new arrays, one of which is the actual final location of some vertex.
Once the vertices are computed they’re then formed into polygons and filled in with the actual pixels to display. (OpenGL calls them fragments because they technically might not actually be pixels, but in practice people seem to be pretty casual about the distinction.) The results from the vertex shader is thus fed to the fragment shader, which takes the results computed from the previous phase and uses them to determine what color should be output at that point. (There are many pixels for each polygon, of course. Shaders can indicate which values should be treated as constant for a polygon and which will be interpolated across the face of it.) Fragment shaders can be used to directly create a bunch of effects that, in the early 2000s, were usually acheived by precomputing a bunch of textures and relying on texture blending operations to produce the desired result.
This is a major oversimplification of the full capabilities of our modern graphics APIs—in particular, I’m completely ignoring the ability of shaders to generate new geometry on the fly, and the way GPUs may be used to perform arbitrary computation on their own—but these are the bits we need.
Our general strategy is also quite similar to the one using the older hardware was, but without nearly as much tapdancing:
- The geometry we submit will be two triangles that form a rectangle, as before, but they will be implied to cover the entire rendering area.
- The vertex shader will scale this rectangle to the proper shape while leaving it centered.
- The fragment shader will simply perform the texture lookup and return that as the color for that point.
The hardware’s more powerful, and part of that additional power is that we get to ask it to do less work. I’m completely on board with that.
Below the fold, we’ll delve into the details.