A tool for performing dynamic disassembly of software.
tbdisasm has a Technical Readiness Level (TRL) of 4.
tbdisasm is dynamic disassembler that operates by monitoring execution of subject binary software in order to recover a GTIRB representation of that software. By observing dynamic execution, tbdisasm produces a precise under-approximation of the control-flow graph of the binary software. It is an under-approximation because it can only produce IR that corresponds to the parts of the software that are actually executed. Parts not exercised by the tracing are not included in the generated GTIRB representation. On the other hand, tbdisasm can capture parts of the software that a static disassembler may miss - in particular indirect control-flow edges (calls, jumps, returns). Further, tbdisasm can take advantage of observing program state to help recover the layout of variables in memory.
Because dynamic disassembly only operates on code that actually executes, tbdisasm can be used to limit the amount of analysis needed when studying large pieces of software. This may be useful in contexts where large libraries are present. Focussing on just the code actually exercised by a test suite will allow the analysis to identify and ignore unused portions of the software.
Static disassembly often makes heuristic assumptions about the structure of a binary. For example, a static disassembler may use an imprecise symbolic analysis to attempt to resolve indirect jumps. Or it may not even attempt to resolve indirect function calls. If the target code is not directly invoked anywhere else, disassembling it generally requires finding it using heuristic pattern matching, which has both false positives and false negatives.
Combining a static disassembler with a dynamic disassembler enables both techniques to shore up the weaknesses of the other. Since tbdisasm operates on GTIRB files and can take a pre-existing GTIRB file as a starting point, it is simple to pass it an existing GTIRB file generated by a static disassembler and have tbdisasm augment the IR with additional information.
There are a number of obfuscation techniques that target static disassemblers. When such techniques are employed, a static disassembler may be tricked into incorrectly deciding that a chunk of data is code or vice versa. In variable-length instruction sets, obfuscation can cause a static disassembler to desynchronize with the actual instruction stream and decode the instruction stream incorrectly. Some obfuscations operate by saturating the binary software with excess decoy code that is dead, causing the static disassembler (and subsequent analyses) to waste a lot of time processing that uninteresting code.
Dynamic disassembly avoids these issues by observing only the actual instructions that execute, thus rendering these obfuscation techniques harmless.
tbdisasm uses cmake for building.
tbdisasm has a Docker image that defines its build and runtime environment
for Linux. See tbdisasm/.ci/Dockerfile.
This Docker image is available on GT Gitlab:
docker pull docker.grammatech.com/reverse-engineering/common/tbdisasm/tbdisasm-internal:latest
To build and install tbdisasm on Linux:
https://git.grammatech.com/reverse-engineering/common/tbdisasm.git
cd tbdisasm
cmake .
cmake --build .
cmake --install . --prefix /opt/tbdisasm
export PATH=$PATH:/opt/tbdisasm/bin/:/opt/tbdisasm/lib/
export LD_LIBRARY_PATH=/opt/tbdisasm/lib/:/opt/instr_trace/lib/:/opt/gtirb/lib/:/opt/capstone/lib
Requirements for building instr_trace on Windows include:
- Microsoft Visual Studio 16 2019 (sync with ddisasm/gtirb)
- DynamoRIO 9.0.1 (add DynamoRIO-Windows-9.0.1/bin32 to path)
- cmake 3.22.1 (add cmake to path)
- boost 1.69.0 (add Boost/bin to path)
- capstone 5.0.7 (add capstone/bin to path)
- protobuf 3.6.1 (add protobuf/bin to path)
- GTIRB dev/latest (add gtirb/bin to path)
- LIEF 0.13.0 (ad lief/lib to path)
- instr_trace dev/latest (add instr_trace/bin to path)
For an example of how to download, build, and install tbdisasm's build and
runtime requirements for Windows, see tbdisasm/.gitlab-ci.yml (the
create_package_windows job).
To build and install tbdisasm on Windows:
git clone https://git.grammatech.com/reverse-engineering/common/tbdisasm.git
cd tbdisasm
cmake -G "Visual Studio 16 2019" -A Win32 -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded .
cmake --build . --config Release
cmake --install . --prefix C:/tbdisasm-dev
set PATH=%PATH%;C:\tbdisasm-dev\bin
Also, there is a Windows release package, tbdisasm_package.zip, which is
generated by the create_package_windows CI job and contains tbdisasm and all
essential files, including the DynamoRIO 9.0.1 release package for Windows. To
install tbdisasm from it, extract this zip file and add its root directory to the
path. Then extract the DynamoRIO zip file and add the bin32/ subdirectory to
the path.
tbdisasm has a full usage/help message. To view:
tbdisasm --help
An example usage of tbdisasm is:
tbdisasm --ir hello_world.gtirb -c hello_world
An example of how to work with the GTIRB emitted by tbdisasm is:
pip install gtirb
python
>>> import gtirb
>>> ir = gtirb.ir.IR.load_protobuf("hello_world.gtirb")
>>> module_names = [module.name for module in ir.modules]
tbdisasm uses DynamoRIO to monitor the target binary's execution. DynamoRIO uses techniques that are similar to malware. Antivirus may alert, quarentine, or delete DynamoRIO. Contact IT for help. Generally, IT can add an exception for DynamoRIO so that the antivirus endpoint will allow it to run on a development machine.
For Windows builds, enable debug builds by setting ENABLE_WINDOWS_DEBUG in
tbdisasm/CMakeLists.txt:
set(ENABLE_WINDOWS_DEBUG ON) < --- LOOK HERE! ADD THIS LINE!
if(ENABLE_WINDOWS_DEBUG)
add_compile_options(-DDRRUN_DEBUG) # Enable debug options in drrun
add_compile_options(-Zi) # Produce PDB files
add_compile_options(-Od) # No optimizations - easier debugging
add_link_options(-DEBUG:FULL) # All debug info is in the PDB
endif()
This option disables optimization and produces PDB files, which can be used to debug tbdisasm in WinDbg.
tbdisasm interface with instr_trace for execution tracing. To enable more
debug logging in instr_trace while running it under tbdisasm, see
tbdisasm/src/InstrTraceHandler.cpp:
void runInstrTraceTracer(instr_trace::ETM TraceMode, Disassembler& Disasm,
const std::vector<std::string>& cmdline,
const std::string& libPath,
const std::string& drrunPath) {
using namespace instr_trace;
// Create TraceCollection object - use defaults (see instr_trace.hpp),
// except let trace mode be configurable
std::unique_ptr<TraceCollection> TC = std::make_unique<TraceCollection>(
0, < --- LOOK HERE! Logging levels: 0, 1, 2, 3, 4
instr_trace::TCM::DEFAULT,
TraceMode,
0,
0,
1000,
PEH::DISABLE);
instr_trace has multiple logging levels. The most useful levels are:
- 0 - no logging (default)
- 3 - log all instr_trace data
- 4 - log all instr_trace and DynamoRIO block disassembly
tbdisasm has two modes, block (--mode block) and transition
(--mode trans). Block mode reports a sequence of block executions: every
time a basic block is executed, its ID is reported. Block mode is slower but
tends to be less likely to crash. Transition mode reports every unique pair of
basic block executions: when two basic blocks execute in sequence, their IDs
are reported once. Transition mode is faster but tends to have more stability
issues than block mode.
Transition mode tends to have better performance. However, sometimes one mode or the other may fail (possibly a bug in tbdisasm, instr_trace, or DynamoRIO) but the other one will suceed so if one mode crashes then try the other.