instr_trace

A utility for collecting instruction traces as a target binary executes.

Implemented as DynamoRIO plugin.

About

instr_trace monitors a target binary's exection and collects instruction traces. instr_trace is implemented as a DynamoRIO plugin. DynamoRIO (specifically drrun) starts the target binary and injects itself into the target process. DynamoRIO monitors and controls the target process's execution. instr_trace, as a DynamoRIO plugin, registers callbacks with DynamoRIO and DynamoRIO calls these functions with information about observed code blocks and execution flow. instr_trace sends block and control flow data across a named pipe to a client, e.g. tbdisasm.

instr_trace defines an event handler interface in:

• instr_trace/include/instr_trace/instr_trace.hpp
• instr_trace/include/instr_trace/instr_trace_event.hpp

Clients of instr_trace, e.g. tbdisasm, can implement this interface and receive tracing data from instr_trace during execution.

Build

instr_trace uses cmake for building.

Linux

instr_trace has a Docker image that defines its build and runtime environment for Linux. See instr_trace/.ci/Dockerfile.

To build instr_trace in Linux:

git clone https://git.grammatech.com/reverse-engineering/common/instr_trace.git
cd instr_trace
cmake .
cmake --build .

Windows

Requirements for building instr_trace on Windows include:

• Microsoft Visual Studio 16 2019 (sync with ddisasm/gtirb)
• DynamoRIO 9.0.1 (add DynamoRIO-Windows-9.0.1/bin32 to path)
• cmake 3.22.1 (add cmake to path)
• boost 1.69.0 (add Boost/bin to path)

For an example of how to download, build, and install instr_trace's build and runtime requirements for Windows, see instr_trace/.gitlab-ci.yml (the create_package_windows job).

To build instr_trace in Windows:

git clone https://git.grammatech.com/reverse-engineering/common/instr_trace.git
cd instr_trace
cmake -G "Visual Studio 16 2019" -A Win32 .
cmake --build . --config Release

Run

instr_trace has the following core binaries:

• instrumenter.so (Linux) / instrumenter.dll (Windows)
• instr_trace_driver (Linux) / instr_trace_driver.exe (Windows)

instrumenter.so/.dll is a DynamoRIO plugin.

instr_trace has the following runtime requirements:

• DynamoRIO 9.0.1 is installed
• DynamoRIO-{Linux | Windows}-9.0.1/bin32 is on the path
• instrumenter.so/.dll is on the path

instr_trace is intended to be used with a client, e.g. tbdisasm. The client is responsible for invoking drrun and the instrumenter library.

To run instr_trace in standalone (useful for development and debugging), use instr_trace_driver. For details run instr_trace_driver -h.

Development and Debugging

DynamoRIO and Antivirus

DynamoRIO uses techniques that are similar to malware. Antivirus may alert, quarentine, or delete DynamoRIO. Contact IT for help. Generally, IT can add an exception for DynamoRIO so that the antivirus endpoint will allow it to run on a development machine.

Windows Debug Builds

For Windows builds, enable debug builds by setting ENABLE_WINDOWS_DEBUG in instr_trace/CMakeLists.txt:

  set(ENABLE_WINDOWS_DEBUG ON) < --- LOOK HERE!  ADD THIS LINE!
  if(ENABLE_WINDOWS_DEBUG)
    add_compile_options(-DDRRUN_DEBUG) # Enable debug options in drrun
    add_compile_options(-Zi) # Produce PDB files
    add_compile_options(-Od) # No optimizations - easier debugging
    add_link_options(-DEBUG:FULL) # All debug info is in the PDB
  endif()

This option disables optimization and produces PDB files, which can be used to debug instr_trace in WinDbg.

Debugging instr_trace and DynamoRIO

Debugging instr_trace is tricky. DynamoRIO injects it into the target process's space, loads it via unorthodox means, and hides it from the operating system. See DynamoRIO's documentation for help with debugging DynamoRIO and plugins: https://dynamorio.org/page_debugging.html.

To enable DynamoRIO debug options in invocation of drrun/instr_trace, see DRRUN_DEBUG in instr_trace/src/interface/instr_trace_impl.cpp:

// Uncomment DRRUN_DEBUG define to enable debug mode in DyanamoRIO drrun
#define DRRUN_DEBUG 1 < --- LOOK HERE!
#ifdef DRRUN_DEBUG
  // drrun options - useful for debugging DynamoRIO and/or client instrumenter
  procArgs.insert(procArgs.end(),
                  {"-debug", "-no_hide", "-msgbox_mask", "15",
                   "-ignore_assert_list", "*", "-loglevel", "1"});
#endif

DynamoRIO logs can be challenging to wade through, but sometimes there is useful insight into what is happening in DynamoRIO.

Debugging instr_trace

instr_trace_driver has helpful logging capabilities:

The -l 3 CLI option outputs all instr_trace packets to a log file. This is useful for seeing events as instr_trace writes them to the named pipe. These events include module loads (executable file path and load address), memory dumps (address and contents), and basic blocks (address and contents). This is a useful view into what instr_trace thinks is happening in the target binary during execution.

The -l 4 CLI option logs instr_trace packets and DynamoRIO disassembly. This provides a view into what DynamoRIO thinks is happening in the target binary during execution.