#StackBounty: #linux #networking #gdb #remote-debugging GDB hangs during remote debugging, library version mismatches

Bounty: 100

I’m using linux and am trying to remote debug a program.

I launch gdbserver on the target, from .xinitrc with

gdbserver localhost:9134 /root/game/game

On my local pc, which I’m running inside a virtualbox vm, I connect to the target from gdb with

target remote 192.168.1.20:9134

and it connects fine.
I can set a breakpoint at main with

b main

and then I can continue and it will break there. I can single step for a ways until it gets to the call SDL_Init(), from which it will never return back to gdb.
If I don’t single step to SDL_Init but instead set a breakpoint further on in the program, the program will start up and run normally (so it gets past SDL_Init). But when it reaches the breakpoint, it freezes up on the target machine and gdb on my local machine never shows a prompt. The entire thing hangs and must be restarted. It’s not completely frozen, however, as the mouse pointer still moves on the target and you can ping it, but the gdb connection no longer works. So it seems that the graphics systems somehow interferes with this since breakpoints before the graphics system init do work, but not after.

I’ve tried setting the remotetimeout setting to 500 seconds and it exhibits the same behaviour. When I ping the remote target from my local pc the reported time is around 0.3 to 0.4 ms. So that doesn’t seem out of the ordinary, but I wouldn’t rule out any other misconfigured network settings on my part.

It’s on a legacy system (but hey, it still makes money) with gdbserver version 6.8-19.fc10 and gdb version 6.8-29.fc10. Upgrading versions, while a very large headache, could be possible but probably should not be necessary (any upgrades I make to my pc have to also be done to a state regulator’s system, as they use the gdb setup for their testing purposes, but it’s not impossible). Remote debugging was working in the past before I took over the project, and no one who worked on it before is still around to ask. The gdbserver version definitely worked, as I’m using the exact program used previously.

Update 1:
I updated the gdb version on the host machine to version 7.0.1 and it still exhibits the same behavior. I couldn’t do version 8 as it needs a C++11 compiler and the legacy system is before that time.

Update 2:
I’ve tried this in another virtual machine and I even built a fresh dedicated linux install (so no vm), rebuilt the software, and I get the same behavior. So it appears the issue is probably on the target machine’s configuration.

Update 3:
I dug out a serial cable and finally got the remote debugging setup via serial. It still doesn’t work, but it gives me more error messages. I get the error

gdbserver: error initializing thread_db library: version mismatch between libthread_db and libpthread

which I think makes sense since my breakpoints quit working after the graphics system is initialized which involves creating some threads. After googling that error, I’ve tried using set solib-absolute-prefix, set solib-search-path, and set sysroot to the root directory on the host machine of a copy of the filesystem on the target machine (on the host, that is /fw_dev/fgs/cf/initrd/expand, which contains the filesystem that the initrd is made from)

But then when I try to set breakpoints, I get Error accessing memory address 0xb5eb60: Input/output error. I’ve also tried setting those variables to the lib subdirectory, which doesn’t work either. I also tried just copying the local thread libraries from the host’s /lib directory to the /lib on the target, but then x windows won’t even start.


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.