▶ YouTube Summaries

← All summaries
tsodingdaily#▶ Watch

Learning XRender to scale game frames on the X11 server

2026-10-22 Thu ⏱ 2 hr 14 min tsodingdaily

Tsoding's software-rendered game draws at a low resolution and upscales every frame on the client with a hand-written nearest-neighbor loop, then sends the large image to the X server with XPutImage. Shared memory (XShm) did not help, which pointed to the scaling, not the transfer, as the bottleneck. He studies the XRender extension, which can composite and transform images on the server side, with the goal of delegating the upscaling to X11 the way SDL delegates it to the GPU through OpenGL. He builds a standalone example that loads a PNG, uploads it as a pixmap, wraps it in a picture and composites it onto the window with a 2x scale transform. He then ports the technique into the game, removing the client-side scaling and resize handling. The result is roughly 40 FPS more than before, about on par with the SDL2 backend.

Motivation

  • The game is developed in the background with as few dependencies as possible. The current backend is SDL2, but an X11 backend exists too, selected by platform name (X11 or SDL2).
  • The X11 backend renders into an XImage and puts it on the window with XPutImage, which is slow.
  • The game renders at a smaller resolution and stretches the image manually. The stretching is a plain nearest-neighbor algorithm that could be sped up with SIMD, but a better option is to make the platform do it.
  • Switching to shared memory with the X server avoided sending pixels over the wire but did not speed things up much. That means the cost was in the scaling.
  • With SDL2 the low-resolution frame becomes a texture stretched on the GPU through OpenGL. The aim is the same offloading for X11, and XRender was suggested by viewers.

Bare X11 window baseline

  • A window with Xlib takes surprisingly little code. He compares it with Win32 and judges it about on par.
  • The event loop only needs to handle the configure notify event (resizing) and the WM_DELETE_WINDOW client message, which is the request to close the window. Selecting input is unnecessary if the application handles no user input.
  • Including the XRender header from the X11 extensions directory compiles and links without trouble. He notes how instantly an empty Xlib application starts and complains about how slow and laggy modern graphical applications are in comparison.

Reading the XRender documentation

  • The documentation is the official spec and a Wikipedia page. It has no examples, and he finds several parts bureaucratic and unclear.
  • XRenderQueryExtension takes pointers for the event and error base, which the library fills in later. He finds the design bizarre and replaces it with a plain XRenderQueryVersion call, failing immediately if the extension is missing.
  • XRenderQueryFormats, sub-pixel order and the picture format matching functions are not clear about what they are used for, so he skips them.
  • A visual in X11 is effectively a pixel format description. Picture formats are the XRender counterpart.
  • The functions that matter are picture creation, XRenderFillRectangle, XRenderComposite and the transformation and filter functions. The operation argument (the op parameter) is barely explained. PictOpSrc is the value everybody uses, and he adopts it without understanding the alternatives.

Finding usage examples in Tiger VNC

  • Since there are no examples, he searches GitHub for real usage and finds Tiger VNC, a VNC client.
  • Tiger VNC shows how to turn a window's visual (from the window attributes) into an XRender picture format with XRenderFindVisualFormat. It also shows the XRenderComposite call with a null mask and PictOpSrc, and how pictures are created from pixmaps.
  • He praises the readability of the codebase despite it being C++, and stars the repository.
  • A tangent complains about projects advertising LLM tooling in their repositories. He considers an unwillingness to avoid it a sign of weak technical skill.

Standalone example

  • The first picture is created for the window, using the format from the window visual. Filling a 100x100 red rectangle through XRenderFillRectangle works on the first try.
  • A picture needs a drawable as its source, and an XImage is not a drawable. The workflow is therefore:
  1. Load the PNG with stb_image into RGBA pixels.
  2. Wrap the pixels in an XImage.
  3. Create a pixmap of the same size.
  4. Copy the image onto the pixmap with XPutImage, using a graphics context.
  5. Create an XRender picture from the pixmap.
  6. Composite that picture onto the window picture.
  • He comments on X11 having separate image, pixmap and picture concepts that all differ semantically.
  • Debugging with GDB shows the BadMatch-style failure of XPutImage comes from a depth mismatch. The pixmap must use depth 24 to match the visual. Colors come out wrong afterward because the pixel order on this machine is BGR, which is fixed by swapping the red and blue bytes of the loaded pixels.

Scaling through a transform

  • Picture transforms are set with XRenderSetPictureTransform using an XTransform, a 3x3 matrix.
  • A search turns up an example with a scale matrix. Applying a 2x scale to the source picture scales the composited image on the server side.
  • The key benefit is that a small low-resolution image is uploaded and the X server scales it up, so far fewer pixels are transferred than before, when the client scaled the image and sent the large result.
  • Details such as the filter used for scaling are left unexplored for now. Further exploration of the transformation abilities is planned, since XRender pictures resemble SDL textures.
  • The example is published as a public repository (MIT licensed) to serve as a reference later.

Learning without an LLM

  • He could have asked an LLM for the example but chose to piece the API together by hand, because that is how he learns.
  • Argument: people who rely less on LLMs prompt them better and extract more value. Heavy use erodes skills, which lowers the value extracted, forming a doom loop. The recommended strategy is to avoid LLMs when there is no deadline, use them when speed is required while leaning on knowledge built without them, and do recreational coding sessions regularly to stay sharp.
  • He compares LLMs to drugs and notes that no earlier tool got worse the more it was used.

Integrating into the game

  • The game already uses XShm, so XRender combined with XShm should have the highest performance. Pixels already sit in shared memory, so XShmPutImage on the pixmap transfers nothing over the wire.
  • Changes to the game:
  • Query the XRender version and link against libXrender.
  • Get the picture format from the window visual.
  • Remove the second client-side screen buffer and all scaling and centering logic. Previously every window resize required reallocating shared memory, which is no longer needed.
  • Pre-allocate the frame buffer at the fixed game display size and replace the scaling copy with a plain memcpy.
  • Create a pixmap and a picture for the frame, and put the image onto the pixmap.
  • Composite the frame picture onto a canvas picture of the window.
  • Centering is dropped for now, so the output is not centered, and mouse capture offsets are set to zero.
  • The first run fails with a bad request for an invalid pixmap or window. Cause is passing the screen picture variable to XPutImage where the drawable (pixmap) was expected. The C API accepts the wrong type without a compile error, which he blames on the API design and not only on C.

Results

  • Without scaling the frame rate stays around 100 FPS. With a 2x transform set on the frame picture and the composite size doubled, the X11 backend without OpenGL runs at roughly 120 FPS.
  • With the previous client-side scaling (stashed changes) the frame rate is below 100, so XRender gives about a 40 FPS improvement.
  • The SDL2 backend is slightly slower or on par, and the numbers fluctuate. The memcpy into shared memory could be removed by rendering directly into the shared buffer.
  • Chat discusses whether XRender uses the GPU. He suspects it sits on the same DRM layer as OpenGL on Linux but is not sure, and notes that avoiding OpenGL is intended to avoid accusations of cheating. A chat comment points out that XRender was how X11 compositors were built before OpenGL-based ones.
  • Planned follow-ups are a stream on the Tracy profiler and one on DRM.