x64dbg Message/Memory Breakpoint

type

status

date

slug

summary

Case Study

The image below shows a simple Win32 dialog box, which contains two static texts, two text input fields (edit controls), and two buttons.

When the OK button is pressed, the program checks if the combination of username and password is correct and pops up a corresponding dialog box. In this blog, our task is to find the message handling function for this dialog box and see what it does when the OK button is pressed.

Program Analysis

To execute the program, simply drag it into x64DBG, which will automatically pause the program at the original entry point defined in the PE header file.

We skip this breakpoint to show the dialog box and let the program run normally. At this point, we switch to the Handles view and refresh it by right-clicking.

The Handles view in x64DBG displays the handles and message handling functions of all window elements in the current program. Here, we only care about the message handling function of the OK button. But we need to understand one concept: in Win32, components like buttons and text input boxes do not have their message handling functions. When user interactions occur (button click, text input), Win32 first calls a system function, which in turn calls the message callback function of the parent window of these components. The address we see in the Handles view is the address of this system function.

We want to pause the program at the entry point of this system function only when the OK button is clicked, but the program is generating a large number of events every frame. If we set a breakpoint directly at the function entrance, it will be triggered all the time. Alternatively, we can use the conditional breakpoint feature in the breakpoints view, but that means we need to write a lot of complex condition code. So, is there a simpler way?

Message Breakpoint

Short answer is yes - In the Handles view, right-click the address of the system function and select Message Breakpoint. Now we see a window like this:

By setting a message breakpoint, we can trigger the breakpoint only when a specific event occurs. It's important to note that the events received by the system callback function are raw events that have not been encapsulated. Therefore, when setting a message breakpoint, we need to change the message type from the encapsulated WM_COMMAND to the WM_LBUTTONUP type.

After setting the message breakpoint, we can successfully stop the program at the entrance of the system function only when the user clicks the OK button!

Memory Breakpoint

The second step is to find the dialog box processing function defined by the application program from this system function. We should note that the code for the callback function provided by Win32 is included in a DLL, while the code for the application callback function generally resides in the code segment of the EXE file. After the system function encapsulates the button-click event, it must call the event callback function of the application program at some juncture, enabling the application program to handle this button-click event as per the demands of the business logic. In other words, the program must eventually return to the code segment in the EXE file. So the question is, can we make the program stop at the moment it jumps back to the EXE code segment?

Indeed, we can. The relative offset address of the EXE file code segment can be located in the Memory Map view.

We use the memory breakpoint feature provided by x64DBG to pause the program the next time it accesses any memory address in the EXE code segment. And with that, the second step is completed!

Program Analysis - part 2

At this point, we have successfully paused the program at the message box processing function. It's important to remember that the system function might call this processing function multiple times when we click the OK button. Therefore, when the program pauses, we need to ensure the message type at that time is indeed the encapsulated button-click event type (WM_COMMAND | 0x111). According to the function signature of the message box processing function, the second parameter of the function is the message type.

With the help of x64DBG's stack view, we can see that the current value of the second parameter is 0x135, which is not what we want. Therefore, we repeat the above steps until the value of the second parameter becomes 0x111.

Finally, we just need to step through the function's execution to see what happens when the OK button is pressed. The image below demonstrates how the dialog box message handling function handles the OK button click event. In short, it calls the Win32 API to get the user-entered username and password, compares it with "zack | 123456", and if they match, a successful login message box appears.