How can I break on UBSan reports in gdb and continue?

Question

Recent versions of GCC and Clang feature Undefined Behavior Sanitizer (UBSan) which is a compile flag (-fsanitize=undefined) that adds runtime instrumentation code. On errors, a warning such as this one is shown:

packet-ber.c:1917:23: runtime error: left shift of 54645397829836991 by 8 places cannot be represented in type 'long int'

Now I would like to debug this and get a debug break on said line. For Address Sanitizer (ASAN) there is ASAN_OPTIONS=abort_on_error=1 which results in a fatal error that is catchable. The only UBSan option that seems usable is UBSAN_OPTIONS=print_stacktrace=1 which results in a call trace dump for reports. This however does not allow me to inspect the local variables and then continue the program. Use of -fsanitize-undefined-trap-on-error therefore not possible.

How should I break in gdb on UBSan reports? While break __sanitizer::SharedPrintfCode seems to work, the name looks quite internal.

Answer 1

While breaking on the detection functions (as described by @Mark Plotnick and @Iwillnotexist Idonotexist) is one option, a better approach is breaking on the functions that report these issues after detection. This approach is also used for ASAN where one would break on __asan_report_error.

Summary: You can stop on an ubsan report via a breakpoint on __ubsan::ScopedReport::~ScopedReport or __ubsan::Diag::~Diag. These are private implementation details which might change in the future though. Tested with GCC 4.9, 5.1.0, 5.2.0 and Clang 3.3, 3.4, 3.6.2.

For GCC 4.9.2 from ppa:ubuntu-toolchain-r/test, you need libubsan0-dbg to make the above breakpoints available. Ubuntu 14.04 with Clang 3.3 and 3.4 do not support the __ubsan::ScopedReport::~ScopedReport breakpoints, so you can only break before printing the message using __ubsan::Diag::~Diag.

Example buggy source code and a gdb session:

$ cat undef.c
int main(void) { return 1 << 1000; }
$ clang --version
clang version 3.6.2 (tags/RELEASE_362/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
$ clang -w -fsanitize=undefined undef.c -g
$ gdb -q -ex break\ __ubsan::ScopedReport::~ScopedReport -ex r ./a.out 
Reading symbols from ./a.out...done.
Breakpoint 1 at 0x428fb0
Starting program: ./a.out 
undef.c:1:27: runtime error: shift exponent 1000 is too large for 32-bit type 'int'

Breakpoint 1, 0x0000000000428fb0 in __ubsan::ScopedReport::~ScopedReport() ()
(gdb) bt
#0  0x0000000000428fb0 in __ubsan::ScopedReport::~ScopedReport() ()
#1  0x000000000042affb in handleShiftOutOfBoundsImpl(__ubsan::ShiftOutOfBoundsData*, unsigned long, unsigned long, __ubsan::ReportOptions) ()
#2  0x000000000042a952 in __ubsan_handle_shift_out_of_bounds ()
#3  0x000000000042d057 in main () at undef.c:1

---

Detailled analysis follows. Note that both ASAN and ubsan both originate from a LLVM project, compiler-rt. This is used by Clang and ends up in GCC as well. Links in the following sections point to the compiler-rt project code, release 3.6.

ASAN has made its internal __asan_report_error part of the documented public interface. This function gets called whenever a violation is detected, its flow continues in lib/asan/asan_report.c:938:

void __asan_report_error(uptr pc, uptr bp, uptr sp, uptr addr, int is_write,
                         uptr access_size) {
  // Determine the error type.
  const char *bug_descr = "unknown-crash";
  ...

  ReportData report = { pc, sp, bp, addr, (bool)is_write, access_size,
                        bug_descr };
  ScopedInErrorReport in_report(&report);

  Decorator d;
  Printf("%s", d.Warning());
  Report("ERROR: AddressSanitizer: %s on address "
             "%p at pc %p bp %p sp %p\n",
             bug_descr, (void*)addr, pc, bp, sp);
  Printf("%s", d.EndWarning());

  u32 curr_tid = GetCurrentTidOrInvalid();
  char tname[128];
  Printf("%s%s of size %zu at %p thread T%d%s%s\n",
         d.Access(),
         access_size ? (is_write ? "WRITE" : "READ") : "ACCESS",
         access_size, (void*)addr, curr_tid,
         ThreadNameWithParenthesis(curr_tid, tname, sizeof(tname)),
         d.EndAccess());

  GET_STACK_TRACE_FATAL(pc, bp);
  stack.Print();

  DescribeAddress(addr, access_size);
  ReportErrorSummary(bug_descr, &stack);
  PrintShadowMemoryForAddress(addr);
}

ubsan on the other hand has no public interface, but its current implementation is also much simpler and limited (less options). On errors, a stacktrace can be printed when the UBSAN_OPTIONS=print_stacktrace=1 environment variable is set. Thus, by searching the source code for print_stacktrace, one finds function MaybePrintStackTrace which is called though the ScopedReport destructor:

ScopedReport::~ScopedReport() {
  MaybePrintStackTrace(Opts.pc, Opts.bp);
  MaybeReportErrorSummary(SummaryLoc);
  CommonSanitizerReportMutex.Unlock();
  if (Opts.DieAfterReport || flags()->halt_on_error)
    Die();
}

As you can see, there is a method to kill the program on errors, but unfortunately there is no builtin mechanism to trigger a debugger trap. Let's find a suitable breakpoint then.

The GDB command info functions <function name> made it possible to identify MaybePrintStackTrace as function on which a breakpoint can be set. Execution of info functions ScopedReport::~ScopedReport gave another function: __ubsan::ScopedReport::~ScopedReport. If none of these functions seem available (even with debugging symbols installed), you can try info functions ubsan or info functions sanitizer to get all (UndefinedBehavior)Sanitizer-related functions.