257 lines
9.1 KiB
Markdown
257 lines
9.1 KiB
Markdown
+++
|
|
title = "Automate debugging using GDB scripting"
|
|
date = 2021-11-13
|
|
+++
|
|
|
|
For a while, have had the pleasure of working on a GStreamer
|
|
[plugin](https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/572)
|
|
in Rust at work. The plugin basically rounds the corners of an incoming video,
|
|
something akin to the `border-radius` property in CSS. Below is how it looks
|
|
like when running on a video.
|
|
|
|
|
|
|
|
The GStreamer pipeline for the same.
|
|
|
|
```bash
|
|
gst-launch-1.0 filesrc location=~/Downloads/bunny.mp4 ! decodebin ! videoconvert ! roundedcorners border-radius-px=100 ! videoconvert ! gtksink
|
|
```
|
|
|
|
This was the first time working on a video plugin in GStreamer. Had a lot to
|
|
learn on how to use the `BaseTransform` class from GStreamer, among other
|
|
things. Without getting into the GStreamer specific details here, basically ran
|
|
into a problem for which needed some debugging for figuring out what was going
|
|
on in the internals of GStreamer.
|
|
|
|
Now, while using GDB from the command line has never been a problem, but, the
|
|
straight forward regular approach is time-consuming. Start the pipeline, then
|
|
attach gdb to a running process, place breakpoints by manually typing out the
|
|
whole thing and then start. For one off debugging sessions, where perhaps you
|
|
just want to inspect the backtrace from a crash or may be look into a
|
|
deadlock condition where your code hung, this could be fine. However, when you
|
|
have to repeat this multiple times do a source code change compile and then
|
|
select again it becomes frustrating.
|
|
|
|
## GDB Dashboard
|
|
|
|
Looking for a better way, [gdb-dashboard](https://github.com/cyrus-and/gdb-dashboard)
|
|
is what first came up as an option. This is quite useful since it can give the needed
|
|
information without having to type anything. Using gdb
|
|
[hooks](https://git.sr.ht/~sanchayanmaity/dotfiles/tree/master/item/gdb/.gdbinit.d/hooks),
|
|
the dashboard can be triggered when appropriate. See the rest of the [gdb
|
|
configuration](https://git.sr.ht/~sanchayanmaity/dotfiles/tree/master/item/gdb/.gdbinit.d)
|
|
to get an idea. This is useful in scenarios like where code is stuck due to a
|
|
deadlock and one needs to look at the backtrace of a crash or any such one off
|
|
simple investigation.
|
|
|
|
## Construct breakpoint command in neovim & copy to clipboard
|
|
|
|
The next small improvement is more specific to neovim. Navigating source code
|
|
with neovim opened in one kitty tab and gdb running in terminal in next tab or
|
|
a split is a preferred workflow personally. Being able to place a breakpoint
|
|
without having to type anything out on the gdb prompt would be convenient. The
|
|
vimscript code below generates the gdb command, considering the current line
|
|
and file on which the cursor is at in the source when opened in neovim.
|
|
|
|
```vimscript
|
|
function! CopyBpLocToClipboard() abort
|
|
let linenumber = line(".")
|
|
let filepath = expand("%")
|
|
let breakpoint = "break " . filepath . ":" . linenumber
|
|
silent execute "!wl-copy " . breakpoint
|
|
endfunction
|
|
|
|
nnoremap <silent> <Leader>yb :<C-U>call CopyBpLocToClipboard()<CR>
|
|
```
|
|
|
|
By using the preceding key binding, a command like below gets copied to the
|
|
clipboard which can be just pasted on gdb prompt.
|
|
|
|
```bash
|
|
break subprojects/gst-plugins-base/gst-libs/gst/video/video-frame.c:104
|
|
```
|
|
|
|
Nifty!!!
|
|
|
|
## GDB scripting
|
|
|
|
Now imagine a scenario where perhaps one wants to look at multiple places in the
|
|
source code and when the program is running, inspect certain variables or just
|
|
print out a back trace each time a specific code point is reached.
|
|
|
|
The manual way to do this is to load the executable in gdb or attach to a
|
|
running process, place a break point, run, inspect the local variables or print
|
|
stack trace, place the next break point and repeat this whole process. Just
|
|
time-consuming.
|
|
|
|
GDB can completely automate the preceding process like below.
|
|
|
|
Below is the `.gdbinit` file applicable for the problem facing encountered at
|
|
work. This is what's called a command file by gdb.
|
|
|
|
```bash
|
|
set confirm off
|
|
set breakpoint pending on
|
|
set logging on
|
|
set logging overwrite on
|
|
set print pretty on
|
|
set pagination off
|
|
break subprojects/gst-plugins-base/gst-libs/gst/video/video-frame.c:104 if meta->n_planes == 4
|
|
break subprojects/gst-plugins-base/gst-libs/gst/video/gstvideometa.c:228
|
|
break subprojects/gstreamer/gst/gstbuffer.c:1410
|
|
break subprojects/gst-plugins-base/gst-libs/gst/video/gstvideometa.c:231
|
|
break subprojects/gst-plugins-base/gst-libs/gst/video/gstvideometa.c:237
|
|
break subprojects/gst-plugins-base/gst-libs/gst/video/video-frame.c:136
|
|
|
|
commands 1
|
|
print i
|
|
print *frame
|
|
enable 2
|
|
enable 3
|
|
enable 4
|
|
enable 5
|
|
enable 6
|
|
continue
|
|
end
|
|
commands 2
|
|
print offset
|
|
continue
|
|
end
|
|
commands 3
|
|
print offset
|
|
print size
|
|
continue
|
|
end
|
|
commands 4
|
|
print *(GstBufferImpl *)buffer
|
|
print idx
|
|
print length
|
|
print skip
|
|
continue
|
|
end
|
|
commands 5
|
|
disable 2
|
|
disable 3
|
|
disable 4
|
|
print *(GstBufferImpl *)buffer
|
|
print info->data
|
|
print skip
|
|
print *data
|
|
continue
|
|
end
|
|
commands 6
|
|
print *frame
|
|
quit
|
|
end
|
|
|
|
disable 2
|
|
disable 3
|
|
disable 4
|
|
disable 5
|
|
disable 6
|
|
run
|
|
```
|
|
|
|
Below is the command to debug the GStreamer plugin in this pipeline with gdb.
|
|
|
|
```bash
|
|
gdb --nx -x .gdbinit --args env RUST_BACKTRACE=1 GST_DEBUG=3,basetransform:6 GST_PLUGIN_PATH=$GST_PLUGIN_PATH:~/GitSources/gst-plugins-rs/target/debug gst-launch-1.0 filesrc location=~/Downloads/bunny.mp4 ! decodebin ! videoconvert ! video/x-raw,format=I420 ! roundedcorners border-radius-px=100 ! video/x-raw,format=A420 ! videoconvert ! gtksink
|
|
```
|
|
|
|
In the preceding command, the `-x` parameter tells gdb to use the command file.
|
|
The `--nx` flag tells gdb to not read any `.gdbinit` files in any directory, as
|
|
`gdb-dashboard` isn't intended to be used for this. `--args` is how one tells
|
|
gdb what to run, which is the GStreamer pipeline in this case. See `gdb --help`
|
|
for details on the flags.
|
|
|
|
Now, consider what the command file does. The ones below are just some
|
|
settings for gdb to use. Note that logging and pretty printing are enabled.
|
|
|
|
```bash
|
|
set confirm off
|
|
set breakpoint pending on
|
|
set logging on
|
|
set logging overwrite on
|
|
set print pretty on
|
|
set pagination off
|
|
```
|
|
|
|
Next, specify the breakpoints. There are six breakpoints. These are the source
|
|
code locations of interest.
|
|
|
|
```bash
|
|
break subprojects/gst-plugins-base/gst-libs/gst/video/video-frame.c:104 if meta->n_planes == 4
|
|
break subprojects/gst-plugins-base/gst-libs/gst/video/gstvideometa.c:228
|
|
break subprojects/gstreamer/gst/gstbuffer.c:1410
|
|
break subprojects/gst-plugins-base/gst-libs/gst/video/gstvideometa.c:231
|
|
break subprojects/gst-plugins-base/gst-libs/gst/video/gstvideometa.c:237
|
|
break subprojects/gst-plugins-base/gst-libs/gst/video/video-frame.c:136
|
|
```
|
|
|
|
Breakpoints can be enabled conditionally. The `if meta->n_planes == 4` implies
|
|
to consider this breakpoint only when a video frame with 4 planes is received.
|
|
|
|
Next gdb has to be told what should be done when each of the preceding
|
|
breakpoints is hit.
|
|
|
|
```bash
|
|
commands 1
|
|
print i
|
|
print *frame
|
|
enable 2
|
|
enable 3
|
|
enable 4
|
|
enable 5
|
|
enable 6
|
|
continue
|
|
end
|
|
```
|
|
|
|
`commands 1` implies these are the commands for gdb to execute when breakpoint
|
|
1 is hit. When breakpoint 1 is hit, the value of `i` and `frame` gets printed.
|
|
The other breakpoints get enabled only after the first one is hit. This is
|
|
because at the end of command file, the following commands
|
|
|
|
```bash
|
|
disable 2
|
|
disable 3
|
|
disable 4
|
|
disable 5
|
|
disable 6
|
|
```
|
|
|
|
instruct gdb to start with these breakpoints off. These get enabled only
|
|
when breakpoint 1 is hit. The `continue` just tells gdb to continue, as gdb
|
|
shouldn't stop on hitting a breakpoint and logs can be inspected in the
|
|
end using gdb log.
|
|
|
|
Other breakpoints are specified similarly.
|
|
|
|
The `run` at the end tells gdb to start executing immediately. In normal usage
|
|
one would have to explicitly type `run` on the gdb prompt to make gdb start
|
|
debugging.
|
|
|
|
If it's not clear so far, basically whatever gdb commands would have been used
|
|
for debugging at the gdb prompt, is what gets specified in the command file as
|
|
well.
|
|
|
|
After running the below on the terminal
|
|
|
|
```bash
|
|
gdb --nx -x .gdbinit --args env RUST_BACKTRACE=1 GST_DEBUG=3,basetransform:6 GST_PLUGIN_PATH=$GST_PLUGIN_PATH:~/GitSources/gst-plugins-rs/target/debug gst-launch-1.0 filesrc location=~/Downloads/bunny.mp4 ! decodebin ! videoconvert ! video/x-raw,format=I420 ! roundedcorners border-radius-px=100 ! video/x-raw,format=A420 ! videoconvert ! gtksink
|
|
```
|
|
|
|
The pipeline gets executed by gdb, considering the command file it was passed
|
|
and log whatever it was asked to log when each breakpoint is encountered. Since
|
|
logging and pretty printing were enabled earlier, gdb logs everything in
|
|
default `gdb.txt` file. The exact log text file can be seen
|
|
[here](https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/572#note_1107146),
|
|
with `gdbinit` and the other two log files attached.
|
|
|
|
Now, one can comfortably look at this log and see what's going on. Once the
|
|
command file is written, the whole debugging process is completely automated.
|
|
Run, sit back and then look at the logs.
|
|
|
|
Using gdb is now a breeze and hassle-free experience. Being able to automate
|
|
and log the debugging process like this, also means you could share your
|
|
command file and someone else can replicate this.
|