Q2WF Map Extraction for a Modern Game Environment

Starting with a strong foundation

When we explore the intricacies of the vision and what WFR is aimed to be, we need to think about the depths of the base game and the mod we are rebuilding. With quake 2 comes a history of many mods and maps over the past 25 years. Specifically with maps and models created for usage in quake 2 weapons factory and the vision being an authentic remake in Roblox means a strong foundation of maps and assets to either utilize to my advantage or remake from scratch.

With that said, I've kept my eyes open for engineering possibilities where necessary as early as possible to save overall game developer time.

Back in Q1 2023 after some engineering exploration, the decision was made to not spend months remaking maps and assets that could possibly be extracted, rebuilt, modernized, uprezzed and reworked for WFR. This decision presented different kind of engineering challenges that I wanted to pursue and become more comfortable with.

These learn and grow opportunities included:

• Learning the ins and outs of the BSP model format

• Unpacking structs using Python

• Reverse Engineering Entity descriptions in a BSP

• Improved familiarity with PIL libraries

• Improved familiarity with Python FBX SDK

• Improved familiarity with developing a CLI

• A reason to use Python3

It has been more exciting discovering solutions to problems that come out of exporting extracted data to be useable in a more modern format that would support a game development process today. In the end, I believe it has been faster to have something to begin with as a solid starting point then it would be to develop everything completely from scratch. Additionally, we get the bonus of legit authenticity in Weapons Factory Roblox, with a bit of re-imagination and that's close enough to what the original vision is described as.

BSP Export to Modern File Formats

What is contained in a BSP file format?

In the context of what we need out of the BSP to rebuild it in a modern game development environment,

the BSP file contains a collection of vertices, edges, faces, and relative textures paths that define the shape and face texture assignments of all static 3D objects in the map. It also can contain data or "entities" that define lights, color, position, and item c variables that refer to placed models, like teleports, spawn points, item pickups, and flag positions. The edge case is that BSP also supports a level of animation for elevators or rotation for warehouse fans, but we won't use this data.

A Custom Export Solution for BSP

Q2BspExport extracts the maps models with material assignments including entities and exports as a single FBX. It also includes full entity support with placed md2 models and lights with proper XYZ coord space position and color as intended in q2wf. More details below.

The requirement is that all md2's are converted to FBX in a relative location to the q2 namespace using Q2Md2Export.

We'll talk about that in next week dev blog.

How do we get data from BSP?

We use a heavily modified version of vgio because it only brought us 50% of the way there by default for quake 2’s model formats, BUT it gave us a great start to get moving into this space of automating the output of textures and mesh from a BSP38. Luckily the author Joshua Skeleton was open to me forking and making modifications, with the hope to merge into his repo later on. I'll do that one day.

How do we learn about BSP?

Here is the Quake 2 Source Code to directly reference how the BSP is structured and loaded. I've also found much usage of the explanations on Flipcode - Quake 2 BSP File Format document on multiple occasions to understand how the format works in order to get UV’s, Normals and Lightmaps out properly. I didn't see much info on Entities though.

How do we convert to FBX?

Wrote a custom conversion of the mesh data outputting to FBX with materials, using the Python FBX SDK where mesh is split by material assignment. With the re-use of textures across the board and method of how texture assignments were just set onto faces, this was the fastest way to get this into a modern game engine. Modern game engines do not support material assignments on faces.

How does it get textures from a BSP?

The relative texture path is simply referenced in the BSP. All textures must be unpacked from all PAK files to find the texture that matches, and in WF case, it could also be loaded from an external location relative to the q2 root folder in “wf\textures”. Once we find a match in the list of files in the pak file with “.wal” at the end, we use the PAK object unpack the file by relative texture name. Once loaded as a PakFile object, we can then pass that file object to the WAL object to load the texture, which gives us Name, Width, Height, Offset, etc.

How does it get entities from a BSP?

Entities are mostly models referenced outside of the BSP, with data about position and sound, like item and flag pickups. It is also a way for the BSP to know where lights are, and which models in the BSP are sometimes animated. We refer to the .entities property to convert a string buffer to a json file, where we can read model number, position, angles, and light color with proper data formats like a list of floats instead of spaced out strings. We also flip position values to be usable in modern XYZ coord space, which replaces Z-Down with Y-Up. This means XZY → XY-Z to flip it as expected. We get the model path from the PAK files by matching the classname with a variable in various hard coded c files. In fact some of them were actually explicit functions. Instead of parsing C file matching variables from classname, I fed the c files to a custom GPT Bot, and asked to parse out the data for me in a json format. After some back and forth, and realizing they were spread across 3 different c files near “game\g_items.c” GPT was able to save me hours of solving that problem by providing a json file called “q2\resources\items_in_bsp.json”. Here is a small small section of what that looks like.

As we can see, if the entity string buffer referred to a c variable, we can now refer to them in a python dictionary like format for simple extraction of the useful hardcoded data within the quake 2 source code that we need to determine. For example "item_armor_body" c variable refers to the "models/items/armor/body/tris.md2" placed model with the sound "misc/ar1_pkup.wav" so lets string munge the .md2 to .fbx and read that file to merge and position/rotate as intended into the main fbx file we are converting from BSP.

What does a final result look like in a 3d App?

This one is called Stronghold WF by Lunitari[RAV]

Hopefully some of these old map creators see their Q2WF maps as full glow ups in WFR one day, as I plan to ensure I provide credit where it is due. And if all i have is there Gamer Alias from 1999, then that'll have to do.

What's Next in Quake 2 Weapons Factory Roblox?

Next week I'll provide a more detailed post about the Q2Md2Export process because it gets a bit more involved with Vertex Animation once in the 3d App.

Later in the month, we look forward to more artwork, more extracted map reworks and details about how we'll close out the work for the year 2023 in WFR to release a version of the game in January 2024, along with what the 2024 roadmap release cadence looks like.