GSoC 2015: The Proposal
Now that finals are over, it’s time to get down to business with GSoC 2015.
For my proposal, I decided to implement gamecasting/game recording within the PySoy video game engine. For those unfamiliar, recording videogame gameplay is a nauseating technical problem. The option taken by most programs is to capture the pixel data on the screen for every frame that is displayed to the gamer, then save that data to a file on the hard drive. This leads to recordings that are multiple gigabytes in size for short videos, stalls on older graphics cards, and complete saturation of the hard drive’s scheduler - leading to long load times and even more stalling.
The other, less common option, is to record certain kinds of data about the game world - player position, look direction, movement of enemies, light sources, animation keyframes, opengl buffers, and use this data to reconstruct the scene displayed to the player after the game session ends. This method leads to very low overhead during recording, but requires incredible amounts planning to implement, and results in code that will touch almost every part of the game’s codebase.
In a past game I worked on, I had implemented video recording using the second option, where I recorded all of the player’s inputs and the state of the RNG during gameplay, then played that data back into the engine and rendered the display buffer to a video file after the session ended. This allowed for super low-overhead recording while the game was being played, and as a major added perk, allowed the developer to debug bugs that were captured in a recording.
Since I had experience with the input-capture method of recording, I decided to write up my original proposal using that method to record gameplay. This would involve capturing player input, RNG seeds, timer/stopwatch calls, and a variety of other stateful information used to modify the state of the game engine.
During an interview in the selection phase, one of the mentors, Anthus Williams, pointed out that my scheme would get much more complicated when you take networked games into consideration. Not only would you have to capture all that stateful client data, you would also have to capture networking information to get the position of enemies and any game object modifications made by the server/other players. Needless to say, this would have made the proposal much more complicated, and would have opened doors for hackers to get information about the game state that they shouldn’t have.
We eventually came to the conclusion that the input-capturing method of video recording wouldn’t be economical to implement. The alternative we came up with is to capture calls made to opengl, and to record buffer uploads, modifications, and deletions. This way, we can play back the opengl calls without having to worry about game state, determinism, networking, or any of that other complicated stuff. The two downsides to using this approach are that save files will be MUCH bigger, and the debugging functionality you would have gotten from being able to play back the game state are gone.
Even with these setbacks, the issues with recording opengl calls are still an order of magnitude less significant than the issues with directly recording the screen directly, so I’m confident there won’t be too many snags during implementation.