Introduction
Writing a custom debugger can be very useful for many program analysis tasks. MSDN provides a useful template which I ended up using many times including in my TriageTool crash triaging tool which I mentionned in my previous post. However I got tired of copying this boilerplate code and have since ported all my tools to Rust which is why I decided to write rdbg.
rdbg is a small Rust library with no dependencies. It provides Rust-friendly wrappers to the Windows debugging API and implements much of the boilerplate debugger code. It exposes APIs to spawn a process, read and write its memory, place breakpoints which execute a closure when reached, single step threads and provide callbacks for many debugging events.
While using one of the crates providing Rust bindings to the Windows API was a possibility, this usually results in non-idiomatic Rust code (e.g checking tags for enums instead of pattern matching) and requires extensive use of unsafe code.
In the next section I’ll present rdbg’s API. We’ll then go over two simple examples of using it, first to solve a reverse-engineering ctf challenge and then to triage crashes.
rdbg
rdbg is built around two components:
- The
Debuggerstruct which allows spawning a process, running it, placing breakpoints, single stepping etc. - The
DbgCallbackstrait which allows the user to define functions which will be called on various debugging events (thread creation, dll loading, access violation, etc.)
We will now present them briefly but I recommend taking a look at the documentation (which can be generated with cargo doc --open from the root of the repository) for more details.
Debugger
The Debugger is used to spawn and run a process. Note that spawn accepts an Option<String> for stdin which will be fed to the target process’ stdin if provided.
1
2
3
4
5
6
/// Spawn a new process under the debugger.
/// e.g cmdline = ["program_path", "arg1", "arg2", "arg3"]
pub fn spawn(cmdline: &[&str], stdin: Option<String>) -> Self
/// Run the process returning its exit code
pub fn run(&mut self, cbs: &mut impl DbgCallbacks) -> u32
The Debugger can then be used to read and write the debuggee’s memory:
1
2
3
4
5
6
7
/// Attempts to read `buf.len()` bytes of memory at `addr` in the debugged
/// process. Returns the number of bytes read.
pub fn read_mem(&self, addr: *const u8, buf: &mut [u8]) -> usize
/// Attempts to write `buf.len()` bytes of memory to `addr` in the debugged
/// process. Returns the number of bytes written.
pub fn write_mem(&self, addr: *mut u8, buf: &[u8]) -> usize
The Debugger can place breakpoints and provide a closure to be executed when the breakpoint is hit. If the module hasn’t been loaded yet, the breakpoint is deferred and will be registered when the module gets loaded.
1
2
3
4
5
6
7
8
9
10
11
/// Registers a breakpoint at address <module>+off
/// Upon reaching the address, the closure `cb` will be invoked.
/// If `permanent` is false, the breakpoint is deleted after it's hit once
/// If `permanent` is true, the breakpoint triggers every time the address
/// is reached.
pub fn register_breakpoint(&mut self, module: &str, off: usize,
cb: Box<BreakpointCallback>, permanent: bool)
/// Type of the closure called when a breakpoint is hit.
/// Arguments are: (dbg: Debugger, pid: u32, tid: u32, exception: EXCEPTION_RECORD)
pub type BreakpointCallback = dyn FnMut(&mut Debugger, u32, u32, &EXCEPTION_RECORD);
Finally, we can single-step threads:
1
2
3
4
5
6
/// Enable single stepping for thread `tid`
pub fn enable_single_stepping(&mut self, tid: u32)
/// Disable single stepping for thread `tid`. Has no effect if it was not
/// enabled
pub fn disable_single_stepping(&mut self, tid: u32)
DbgCallbacks
This trait allows the user to provide functions that will get called when a specific event occurs.
Here is the list of events currently available:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
/// Called when the debugged process causes an exception
fn exception_cb
/// Called on thread creation
fn create_thread_cb
/// Called on process creation
fn create_process_cb
/// Called on thread exit
fn exit_thread_cb
/// Called on process exit
fn exit_process_cb
/// Called on DLL load
fn dll_load_cb
/// Called on DLL unload
fn dll_unload_cb
/// Called when the debugged process causes an access violation
fn access_violation_cb
/// Called on every single step
fn single_step_cb
/// Called when the debugged process attempts to execute an illegal instruction
fn illegal_inst_cb
/// Called when the debugged process divides by zero
fn div_by_zero_cb
/// Called when the debugged process uses up its stack
fn stack_overflow_cb
The user should implement this trait for a struct and pass it to the Debugger’s run method.
1
2
3
4
5
struct MyAnalysis;
impl DbgCallbacks for MyAnalysis { }
...
let mut dbg = Debugger::spawn(&["program.exe"], None);
dbg.run(&mut MyAnalysis);
Examples
Let’s now go over two examples of using rdbg.
Solving a reverse-engineering CTF challenge
This CTF challenge is a 32 bit executable. It accepts a password on argv, performs some checks in check_password and either prints "Wrong Password :(" or "Congrats! :)":
1
2
3
4
5
6
7
8
9
10
11
12
.text:004010DB call check_password
.text:004010E0 add esp, 4
.text:004010E3 test eax, eax
.text:004010E5 jz short loc_4010FA
.text:004010E7 push offset aCongrats ; "Congrats! :)\n"
.text:004010EC call puts
.text:004010F1 add esp, 4
.text:004010F4 xor eax, eax
.text:004010F6 jmp short loc_40110C
.text:004010F8 jmp short loc_40110C
.text:004010FA push offset aWrongPassword ; "Wrong password :(\n"
.text:004010FF call puts
Often, simple challenges rely on comparisons that stop as soon as an incorrect character is encountered. This means that if none of the characters are correct, the check fails immediately, if only the first character is correct, one check succeeds but the next one fails and so on. In other words, the more valid characters we provide, the further we will go in check_password before failing.
Therefore we can use the number of instructions executed during the password check as a side channel.
We can use rdbg to count the number of instructions executed in check_password. To do so, we will spawn the target binary under the debugger and single step the check_password function:
1
2
3
4
5
6
7
8
9
10
11
12
13
// Spawn chall.exe under the debugger without providing any stdin (this
// challenge accepts input on argv).
let mut dbg = Debugger::spawn(&["examples\\crackme_example\\chall.exe", pass], None);
// Start single stepping when we reach the `call check_password` instruction
dbg.register_breakpoint("chall.exe", 0x10DB, Box::new(|dbg, _pid, tid, _exception| {
dbg.enable_single_stepping(tid);
}), false);
// Stop single stepping when we return from `check_password`
dbg.register_breakpoint("chall.exe", 0x10E0, Box::new(|dbg, _pid, tid, _exception| {
dbg.disable_single_stepping(tid);
}), false);
We now need to count the number of single steps performed during the execution:
1
2
3
4
5
6
7
8
9
10
/// An analysis which counts the number of single steps during the debuggee's
/// execution.
struct SingleStepCounter(usize);
impl DbgCallbacks for SingleStepCounter {
// This function gets called on each single step
fn single_step_cb(&mut self, _: &mut Debugger, _: u32, _: u32) {
self.0 += 1;
}
}
We can then run the target program and retrieve the result:
1
2
3
let mut single_step_counter = SingleStepCounter(0);
dbg.run(&mut single_step_counter);
let n_inst_executed = single_step_counter.0;
Finally we need to write some brute force logic that tries every character for each position and always picks the one that leads to executing the most instructions. I’ll spare you the details but you can find the code here.
Output:
1
2
3
4
5
6
7
C:\tools\rdbg>cargo run -p crackme_example
S__________
S3_________
S3c________
S3cR_______
S3cR3______
S3cR3t_____
Writing a crash analysis tool
Custom debuggers can also be useful to automate crash analysis. In this context, many callbacks in dbg_callbacks.rs can be interesting such as access_violation_cb, illegal_inst_cb, div_by_zero_cb or stack_overflow_cb but to keep this example short, we will only report access violations.
Let’s define a function to be called when an access violation is triggered. It will parse the exception record and display information about the crash:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
/// An analysis which prints information about a crashing program
struct CrashAnalysis;
impl DbgCallbacks for CrashAnalysis {
fn access_violation_cb(&mut self, dbg: &mut Debugger, _pid: u32, _tid: u32,
exception: &EXCEPTION_RECORD, first_chance: bool) {
let addr = exception.ExceptionAddress;
// Resolve the address to module + offset
let (module, offset) = dbg.resolve_addr(addr);
let details = if exception.NumberParameters > 1 {
let a = exception.ExceptionInformation[1];
let av_type = match exception.ExceptionInformation[0] {
0 => "read",
1 => "write",
8 => "exec",
_ => unreachable!(),
};
format!("Invalid {av_type} to {a:#x}")
} else { String::new() };
println!("Access Violation @ {module}+{offset:#x} ({} chance). {}",
if first_chance { "first" } else { "second" }, details);
}
}
We can then execute our crashing program under the debugger:
1
2
let mut dbg = Debugger::spawn(&["examples\\triage_example\\crash.exe"], None);
dbg.run(&mut CrashAnalysis);
Once again, the code for this example is available here
Output:
1
2
3
C:\tools\rdbg>cargo run -p triage_example
Access Violation @ crash.exe+0x1019 (first chance). Invalid write to 0xc
Access Violation @ crash.exe+0x1019 (second chance). Invalid write to 0xc
Conclusion
In this post, we introduced rdbg, a library for writing custom Windows debuggers in Rust. We then went over two use cases to show how one can use it to quickly automate program analysis tasks.
Don’t forget to check out the code of rdbg on Github. I hope you’ll find it useful and welcome any PR or feedback.