Showing posts with label Tools Programming. Show all posts
Showing posts with label Tools Programming. Show all posts

Feb 5, 2025

Sync Engine

Team of 4
September 2024 - Current
Skills Utilized: C++, Multithreading, Reflection, Serialization, Profiling & Optimization, Game Engine Programming

Introduction

For my senior year project, I set out to build a fully-fledged game engine. While I had prior experience with game engine development, I had never created a complete engine from the ground up and wanted to take on the challenge.

The engine was developed in C++20, utilizing CRTP to elegantly collect type information. This allowed the engine to automatically gather information on all subclasses of Actors and Components. I designed a highly intuitive serialization and reflection system, requiring only a single-line declaration per class while ensuring compatibility with field reordering, addition, removal, and even virtual inheritance. To support fast iteration, I implemented DLL hot reloading for game code. Additionally, I developed an intrusive profiler module using assembly to automatically track execution times for all functions, which helped optimize serialization performance by up to 1300%.

I handled all aspects of the engine except for graphics backend, which were delegated to two graphics programmers.

Showcase

↑ Overall Demonstration

↑ DLL Hot Reloading Demonstration

Technical Highlights

Separate Graphics Thread

To improve both game logic and rendering performance, I separated the render thread from the game thread and built a dedicated communication layer between them. When registering graphics objects, I implemented a double-buffered approach: two lists were maintained, with the render thread processing one while the game thread added new objects to the other. Once the render thread completed its work, the lists would swap, ensuring that neither thread had to wait on a lock.

Additionally, the editor, built using ImGui, required updates to be processed on the render thread due to graphics constraints. However, modifying actor or component fields directly from ImGui led to data races. To resolve this, I redesigned all value modifications as commands that were stored in a buffer, allowing synchronized access between the game and render threads. This solution later proved invaluable also when implementing real-time collaborative editing.

Reflection

I designed an intuitive reflection system for actors and components using template metaprogramming and macros, making field reflection and serialization effortless. For example, the reflection code for a Transform Component looks like this:


Fields that need to be reflected or serialized are listed once inside the class and once externally. Additionally, editor-specific configurations like specifying Getters/Setters can be done in a single line, keeping the process simple and intuitive.

Serialization

Level data is stored in a binary format while ensuring that field reordering, addition, or removal does not break compatibility with previously saved data. This is achieved by maintaining type metadata for each class. The system stores a hashed name and type ID for each field, allowing it to compare stored field signatures with the current structure at load time to ensure proper deserialization.

↑ Type information structures

Additionally, pointer fields for Actors/Components and Assets are handled seamlessly. Actor and Component pointers store memory offsets, ensuring that objects are reconstructed at the correct locations during loading. Asset pointers, on the other hand, store asset references (paths) instead of raw pointers, preserving data integrity across sessions.

DLL Hot Reload

To enable hot reloading for game code, the engine compiles the game logic into a DLL, which the engine executable loads dynamically. The engine build process produces both an EXE and a LIB, and the game DLL links against this LIB.

A major challenge in this setup was handling global variable ownership between the EXE and DLL. For instance, if the Game class has a global variable and it is defined in a header, both the EXE and DLL end up with their own separate instances. Even using extern and defining it inside the engine’s source files doesn’t solve the problem—since the engine’s LIB is linked to the game DLL, the DLL will still create its own instance.

The solution I devised was to use preprocessor directives to control the definition depending on whether the engine or game is being built:
During the engine build, the header file defines an inline instance. During the game build, do not declare the instance. Then, both for the engine code and the game code, use a pointer to that variable exclusively. This pointer is declared as such using the preprocessing directives:
Engine: extern "C" __declspec(dllexport) inline Game* GameInstance = nullptr;
Game: extern "C" Game* GameInstance;


Using a pointer instead of a direct instance prevents link-time errors in the game build due to missing definitions.

At runtime, the appropriate instance is assigned based on whether the ownership should reside in the engine or the game, and I implemented a helper function to automate this assignment, ensuring clarity and preventing misuse.

↑ List up the global variable to be shared,

↑ And this function will automatically synchronize the pointers.

Intrusive Profiler & Optimization

I developed a profiler that automatically records the execution time of all functions without requiring any additional code from the user. This was achieved by writing _penter and _pexit in assembly and enabling the /GH and /Gh compiler options to ensure these functions are executed at every function entry and exit.

One key challenge was preventing the profiler from recursively profiling itself. To avoid this, the profiler was built as a separate DLL, ensuring its code wouldn’t be instrumented. This required adding EXPORT after the _penter PROC declaration.

↑ Part of the profiler assembly

↑ Example profiling output

Using this profiler, I discovered that directly writing to fstream during serialization was creating a major bottleneck. To address this, I leveraged a custom memory manager to accumulate all data first and write to fstream only once at the end. This optimization resulted in a 1300% performance improvement.

Conclusion

This project allowed me to tackle complex engine systems and refine my problem-solving approach. Designing a reflection-based serialization system that remained compatible across versions, implementing a multi-threaded rendering pipeline, and handling global state in DLL hot reloading were all challenges that required careful consideration.

Beyond technical implementation, the experience reinforced the importance of profiling and optimization, as seen in the significant performance gains achieved in serialization. It also highlighted the necessity of maintainable and scalable design choices.

The lessons learned from this project will inform my future work in engine development, particularly in balancing flexibility, performance, and ease of use.

Feb 4, 2025

Mercy for Machines

Team of 13
September 2023 - April 2024
Skills Utilized: C++, Unreal Engine, Tools Programming

Introduction

Our team, consisting of programmers, designers, artists, and sound designers, collaborated to develop a game using Unreal Engine 5.2. Mercy for Machines is a 3rd person, narrative driven sci-fi horror game. My primary role was gameplay programming, and I also developed editor tools for designers and implemented destruction mechanics using Chaos physics.

Showcase

↑ Actor Collector

↑ Chaos Destruction

Technical Highlights

Actor Collector

While observing the map creation process, I noticed that manually maintaining lists of various actors was a frequent and inefficient task. To streamline this, I developed a tool that automatically collects and visualizes actors. Users can intuitively select actors using box and sphere volumes, with additional filtering options based on actor class, interface, or component class. To enhance visibility, I implemented an outline material for collected actors and a material effect that desaturates everything else in the scene. To minimize runtime overhead, the tool only retains the collected actor list in the final build without performing any additional operations.

Chaos Destruction

One gameplay sequence required a staircase to collapse, causing the player to fall. I immediately thought of Unreal Engine 5’s Chaos physics engine and its destruction system. While fracturing the mesh was straightforward, controlling how the pieces scattered proved challenging. I achieved the desired effect by carefully utilizing Anchor Fields and Master Fields to direct the fragments’ movement.

Conclusion

Mercy for Machines was a valuable experience that expanded my understanding of Unreal Engine 5. Developing the Actor Collector tool improved level design efficiency, while implementing Chaos Destruction deepened my knowledge of the engine’s physics system. Through this project, I gained hands-on experience with Unreal’s editor scripting, rendering, and physics features, further refining my ability to develop both gameplay mechanics and engine tools.

Aug 26, 2023

Typoon

Solo Project
April 2023 - December 2023
Skills Utilized: C++, Multithreading, Windows Programming, Tools Programming, Unit Test

Introduction

Typoon is a unique text expander designed specifically for Hangeul, the Korean script. Most of the text expander out there doesn't really work well with Korean typing system, so I ended up making one myself. It supports the alphabets as well. It was heavily inspired by espanso.
In this project, I designed and implemented efficient data structures and algorithms to store and process text data, accurately detect letter composition, and correctly replace input text. Leveraging Windows programming, I enabled automatic file change detection and user notifications for convenience. To ensure reliability, I implemented comprehensive unit testing, overcoming the testing framework’s lack of wide character support.


What is a text expander?

"A text expander is a program that detects when you type a specific keyword and replaces it with something else. This is useful in many ways:
- Save a lot of typing, expanding common sentences.
- Create system-wide code snippets."

 -- From the espanso's readme file

Showcase



↑ Typoon comes with a convenient system tray icon

↑ You can write your own matches intuitively

Technical Highlights

Managing the trigger strings

In a nutshell, Typoon builds a "trigger tree" with 2 iterations to manage all the match data.
Let's say there are 4 matches:

Within the first iteration, Typoon actually builds a tree with the given data. While building, there can be 'stale' nodes that cannot be reached. Those nodes will be discarded in the second iteration.

In the second iteration, the tree gets flattened in level order to maximize the locality of reference and minimize the memory usage. This is the final form of the data that is used in the runtime.

Detection of Korean letter composition & IME simulation

Understanding the computer-based composition of Korean letters is essential before diving into this topic. A Korean letter comprises three components: an initial, a medial, and an optional final. Unlike alphabets, where one keyboard press corresponds to a single letter, forming a Korean letter may require up to five keystrokes. Hence, capturing keyboard input alone is insufficient; knowing the actually composed letters is crucial.

Source: https://rcneira.weebly.com/learn/reading-the-korean-alphabet-hangul

You might ask, "Why not decompose all the letters and use that to match the input?" Unfortunately, this approach often leads to an ambiguity. For instance, decomposing "곡" into "ㄱㅗㄱ" might inadvertently match "고가," which decomposes as "ㄱㅗㄱㅏ." This mismatch is usually undesirable. Well, matching the composed letters is the only option it is, then.

Acquiring the composed letters presented two options. First, extracting them from the Input Method Editor (IME), a layer for Korean composition. Second, simulating the IME to compose letters within Typoon based on the keyboard inputs.

Obviously, I went for the former one at first. But after days of researches, I've come to a conclusion that there is no way to get the letters from either the processes or the IMEs attached to them. You simply cannot access to the other processes' IMEs, and there is no unified way to get the composed letters from all kinds of processes.

So, the only option was to simulate the IME myself. It was a pretty deep case analysis, but I managed to figure it out.


File watcher

Typoon incorporates a file watcher feature capable of promptly detecting and implementing changes to both the config and match files upon saving. This functionality leverages Windows-specific functions: WaitForMultipleObjectsEx() and ReadDirectoryChangesW().
In essence, you put a thread into sleep and tell it to wake up if any of the watching events triggers(WaitForMultipleObjectsEx), and you tell the OS to trigger an event when a change is made to a given directory(ReadDirectoryChangesW).



Addressing the lack of wide character support in doctest

Typoon's focus on Hangeul and Windows necessitates the use of wide characters throughout its codebase. However, most unit test libraries often fall short in handling wide characters, resulting in character display issues like '�'.
So I decided to settle on one and devise a workaround. By delving into doctest's result printing functions, I identified a solution. I introduced a template specialization of a class responsible for converting objects to strings. In addition to that, given that I employed diverse string types for wide characters, such as std::wstring, std::wstring_view, wchar_t[ ], and others, I ensured that the specialization accommodate all these variations by utilizing the std::convertible_to concept.

Conclusion

Typoon has been a particularly enjoyable project for me, offering numerous valuable learning experiences. I gained expertise in Windows programming, realized the effectiveness and the importance of the unit testing, learned how to build an installer for Windows, delved into original data structures and algorithms, got familiarized with multithreading, and the list goes on.
Even though I'm having a temporary break right now, I will definitely pick it up again soon in the future.

Nov 9, 2020

U.R.U.K

Team of 4
February 2020 - July 2020
Skills Utilized: C++, Game Engine Programming, Tools Programming, Memory Management

Introduction

In this semester, the goal of the GAM class was to form a group and make a complete game. The main mechanic of this game was a gate that opens when the count of creatures matches a certain number. Given the substantial scope of the game and the restriction on using pre-existing game engines, I took the lead to develop a custom game engine.
Designed a custom game engine with an Actor-Component architecture and a memory manager that stores components contiguously for better cache efficiency. Used CRTP-based static polymorphism to implement deserialization for each component. Developed a state machine editor and map editor, implementing an undo/redo system using a command-based action structure.

Showcase




↑ State machine editor showcase

↑ Controls of the state machine editor




↑ Automatically generated state machine codes

↑ The state machine of the main character used in production


↑ A screenshot of the map editor

Controls
- How to Play will teach you the basic controls.
- ` : (Cheat) Press ` while on a Fish Gate to skip to the next floor.

Responsibilities

  • Technical Director
  • Core of the game engine
  • Entity-component based object management
  • Custom memory manager
  • State machine editor
  • Map editor
  • Minimap editor
  • UI-related components
  • Camera
  • Many other gameplay programming

Technical Highlights

Entity-component structure custom engine

In this project, I was in charge of designing and implementing the game engine. We built our own since the use of existing ones were restricted.
A game is a singleton, it has multiple levels in it. Levels contain objects, and objects conceptually contain components. I used the word conceptually because they are actually stored separately.

Engine Reference Document


↑ The technical structure of the game (better resolution)

It's the components where the most of the logic resides in. The objects are there just to bundle them up abstractly.

↑ A portion of the interface of the component class

↑ A part of the function that adds a component to a level

Custom memory manager

I implemented a custom memory manager for this project. I know it would be redundant for a student-size project, but I just wanted to give it a try. We used it just for storing the components.
The structure of the memory manager is as follows: allocate a huge chunk of initial memory, and divide it on demand. If a new type of component requests a memory, assign a large, contiguous part of it for that type of components. Later, when the same type of a component requests a memory, it'll give one from the preassigned part. So after some uses, it'll look like this:


A notable challenge was handling component removal. To optimize reference locality by ensuring contiguous components, I efficiently filled gaps after a component was removed. This was achieved by swapping the last element with the removed one, while considering component's memcpy-ability.


The reason I couldn't use std::swap was the fact that swapping with the "deleted" ones whose destructor was called caused problems. Now I come to think of it, it would have been sufficient to swap the elements beforehand and then to call the destructor of the to-be-removed one. 

Aside from that, moving the components within the memory implies one another problem: we cannot use the normal pointers to access to the components. To address this issue, I chose handles to give access to the components to the outside, which is in this case, an id.


Virtual static functions for deserialization

During the late stages of development, a demand for the serialization support emerged. Implementing the serialization was straightforward – just add a virtual function that gets called when serialization happens. However, deserialization posed a challenge. Many components relied on non-default constructors, so forcing them to have a default constructors was not viable. How could we call a virtual function when the object had not yet been instantiated?
There is a technique called "static polymorphism". In short, you call a static function of the child class in parent class using CRTP. Derived from this idea, I devised an approach: Components with non-default constructors were to declare a static function with a specific signature that could be invoked during deserialization, returning an instance of themselves. These functions were stored in a std::map. The store happens automatically if the component has a static variable of a special type designed for this purpose.
So yes, it's not "virtual static", it's rather a dispatcher. But the users felt like as if it were virtual :)



Text alignment

The graphics library provided by the school didn't have any of the text alignment functionalities. So I took the charge and implmented horizontal and vertical text alignment, in addition to line wrapping.
To achieve this, I analyzed the file format used by the library reading the document. Based on the information, I wrote the code to simulate the library's calculation of the width and the height of a given text. However, the calculation was a bit off even though I strictly followed the documentation. After some serious investigation, I reached out to the professor who wrote the library and got the actual calculation used in the library. It turns out it was actually the library that didn't follow the documentation. To accommodate these differences, I implemented appropriate workarounds and successfully finalized the implementation.



Command-based architecture of the state machine editor

The structure of the state machine editor mirrored that of existing engines, functioning as a node-link editor. To accommodate undo and redo functionality for enhanced user experience, I implemented a command-based architecture. Every operations were encapsulated as distinct commands, each comprising OnExecute and OnUndo functions. The program maintained a series of commands, executed the appropriate functions when necessary.


Conclusion

This project represents a significant investment of effort on my part. With 382 commits, I added 34,682 lines and deleted 13,419 lines, excluding library and resource changes. The experience gained was extensive, highlighting the rigorous process of building an engine. Despite the challenges, the project was enjoyable, I had the chance to apply programming patterns and patterns, realized the importance of patience and determination in implementing complex functionality. It also provided an opportunity to devise solutions for intricate problems.
In summary, this project has been instrumental in my growth as a software engineer, and the skills developed will undoubtedly shape my future work.