cortex_m: Add debug symbols for OpenOCD/GDB thread support #4058

daniel-k · 2015-10-06T09:55:31Z

GDB has the ability to inspect threads, but to use this feature with RIOT the GDB-server, in this case OpenOCD, needs to know about the thread structures and the stacking. This PR introduces 2 new symbols in order to provide support for GDB threads.

To use this feature you need my patched OpenOCD version found here. I'm working on getting these patches upstream to OpenOCD so that one day this will be a default feature.

I added the needed -rtos auto line to every board that has a openocd.cfg file, but only tested stm32f4discovery and samr21-xpro yet.

OpenOCD can only access symbols, no struct offsets or define macros, so I introduced max_threads to iterate over sched_threads and _tcb_name_offset that will only be available when compiled with DEVELHELP to access the thread name. Although both symbols are declared as volatile they will be optimized away, so that's why I added the hack in sched_task_exit(). I'm sure there's a better way, so please tell me :)

daniel-k · 2015-10-06T10:06:43Z

I just realized that cortex_m might not be a good title as this will introduce these symbols for all platforms, although it is only used by OpenOCD, i.e. Cortex-M-

OlegHahm · 2015-10-06T10:46:56Z

very nice!

OlegHahm · 2015-10-06T10:48:33Z

Works like charm on the iotlab-m3 with gnrc_networking:

(gdb) info threads
  Id   Target Id         Frame 
  7    Thread 7 (at86rfxx :  : Blocked receive) 0x08001716 in _msg_receive (m=0x20000ed0 <_nomac_stacks+868>, block=1) at /home/oleg/git/RIOT/core/msg.c:329
  6    Thread 6 (udp :  : Blocked receive) 0x08001716 in _msg_receive (m=0x20004714 <_stack+850>, block=1) at /home/oleg/git/RIOT/core/msg.c:329
  5    Thread 5 (ipv6 :  : Blocked receive) 0x08001716 in _msg_receive (m=0x20001304 <_stack+858>, block=1) at /home/oleg/git/RIOT/core/msg.c:329
  4    Thread 4 (6lo :  : Blocked receive) 0x08001716 in _msg_receive (m=0x20003f64 <_stack+842>, block=1) at /home/oleg/git/RIOT/core/msg.c:329
  3    Thread 3 (pktdump :  : Blocked receive) 0x08001716 in _msg_receive (m=0x20003b80 <_stack+1384>, block=1) at /home/oleg/git/RIOT/core/msg.c:329
  2    Thread 2 (main :  : Blocked mutex) mutex_lock (mutex=0x20000390 <_rx_mutex>) at /home/oleg/git/RIOT/core/mutex.c:53
* 1    Thread 1 (idle :  : Running) idle_thread (arg=<optimized out>) at /home/oleg/git/RIOT/core/kernel_init.c:74

daniel-k · 2015-10-06T10:49:32Z

@OlegHahm
If you experience problems with backtrace when switching thread, cherry-pick daniel-k/openocd#1
EDIT: Updated PR message to point to already cherry-picked branch.

jnohlgard · 2015-10-07T15:32:09Z

Wow!
Do you have a link to an upstream ticket for tracking the OpenOCD integration?

daniel-k · 2015-10-07T15:37:09Z

Do you have a link to an upstream ticket for tracking the OpenOCD integration?

http://openocd.zylin.com/#/c/3000/, but seems they take more time to review contributions than we do :P

daniel-k · 2015-10-13T15:16:16Z

So what's your opinion on this? Waiting for the OpenOCD change will not make sense I would say. I can't estimate how long they'll take for reviewing my commit and in the end it has to match the RIOT "interface". So it would make sense to agree on this first and then maybe accelerate the OpenOCD change.

jnohlgard · 2015-10-13T16:00:41Z

will $_TARGETNAME configure -rtos RIOT cause any issues when using a non-patched openocd?

jnohlgard · 2015-10-13T16:02:56Z

core/sched.c

@@ -181,6 +190,12 @@ void sched_switch(uint16_t other_prio)

 NORETURN void sched_task_exit(void)
 {
+    /* Prevent optimization of these symbols */
+    max_threads = max_threads;


This will cause warnings from various static analyzers. Does adding __attribute__((used)) help?

@gebart

Doesn't work. It is caused by the linker garbage collection:

export CFLAGS_LINK = -ffunction-sections -fdata-sections export LINKFLAGS += -Wl,--gc-sections

So there doesn't seem to be a way to prevent optimization from within the compiler other than removing -fdata-sections. But that's no option.

Adding

export LINKFLAGS += -Wl,-u,max_threads

to boards/Makefile.include.cortexm_common keeps the variable from being optimized away, but that's not a nice solution either.

Another possible solution would be to define a new section in RAM (linker script) with KEEP(this-section) and moving the variable into that section.

I would rather like to have a solution where it's only marked next to variable in source, but there doesn't seem to be a way. The latter solution is not far away from that at least.

@gebart
See e3a2dcf for how this could look like. This way these symbols should be optimizied away if a platform doesn't have the KEEP directive in the linkerscript, so no overhead for other platforms. Haven't tested though, I'm lacking the toolchains for msp430 and avr.

jnohlgard · 2015-10-14T11:21:04Z

I think still attribute((used)) will be necessary when building with LTO.
After a quick googling it seems like the only solution without actually using the symbols is to put them in a special section, like you did.

daniel-k · 2015-10-14T15:45:00Z

will $_TARGETNAME configure -rtos RIOT cause any issues when using a non-patched openocd?

In general this does no harm. I switched from -rtos auto to -rtos RIOT because the former didn't work out most of the time. Now I found out why: there was a bug in another RTOS that prevented RIOT from being detected (see bugfix. In the end it makes no difference for us which to use now, I also updated my fork on Github, so if you pull and rebuild you'll have that, too.

daniel-k · 2015-10-14T15:53:38Z

To get the OpenOCD patches reviewed I was told to send some RIOT guys over to them to review my patches, as you are really using RIOT. So please head over to OpenOCD RIOT support ~~and maybe OpenOCD mqx bugfix~~ to review my patches, i.e. try out and say it works.

You need to register at their Gerrit but you can use your Github account via OAuth.

jnohlgard · 2016-03-10T10:16:29Z

This PR needs a rebase after tcb_t -> thread_t rename #4578

OpenOCD side needs an update because it expects thread status to be of uint16_t type, but it is now uint8_t since #4924, all threads are otherwise listed as "Unknown Status"

daniel-k · 2016-03-13T15:50:31Z

@gebart
Thanks for the update. I don't have a board at hand so I can't test. I also updated my openocd fork on Github (not pushed upstream though): daniel-k/openocd@01dc813

OlegHahm · 2017-01-15T10:09:28Z

The question is: should we merge it here or wait for the review on the OpenOCD site?

Anyone with an opinion here? I would vote for merging this PR ASAP. At least one could use this nice feature on RIOT with a locally patched OpenOCD version.

daniel-k · 2017-01-15T12:00:03Z

I'm currently not using RIOT in any project, but when working with it this was a crucial feature for me. Concerning the OpenOCD part: they are preparing the 0.10 release right now so I guess it's to late to get RIOT-support merged into 0.10 anymore. But still, when it's merged here it's certainly possible to get the patch accepted for the next release.

jnohlgard · 2017-01-16T08:29:13Z

I agree, this should be merged. Might need updates to some boards which were added after last rebase though.

OlegHahm · 2017-01-16T16:53:45Z

@daniel-k, you could cherry-pick OlegHahm@6c22e3f to add the missing new boards.

daniel-k · 2017-01-16T19:51:19Z

@OlegHahm cherry-picked it is. Thanks!

miri64 · 2017-01-17T08:20:48Z

If I understood this correctly you need a patched version of openocd to have this feature, but without the patch it is still working (just tested it with Open On-Chip Debugger 0.9.0 (2017-01-10-16:08)). So I wouldn't be opposed to merging this either.

miri64 · 2017-01-17T08:21:28Z

(But I leave the actual ACK to a person more knowledgeable with OpenOCD)

OlegHahm · 2017-01-17T10:54:44Z

I'm without hardware right now. Can anyone just briefly check with any board using OpenOCD for flashing and an unpatched version, in order to assure that it is indeed not breaking anything? If the result is positive, I'm willing to ACK.

kYc0o · 2017-01-24T15:21:51Z

I just tested this PR and it didn't break anything, however it seems to succeed always, even if it didn't.

OlegHahm · 2017-01-25T16:03:34Z

however it seems to succeed always, even if it didn't.

What do you mean by "succeed"?

kYc0o · 2017-01-25T16:08:21Z

What do you mean by "succeed"?

When the flashing was successful.

In current master it returns an error which stops make immediately. With this change openocd returns success even if the flashing failed. It's related to what we had before #2897.

OlegHahm · 2017-01-25T16:15:01Z

This sounds odd. Have you locally merged this PR into master or just checked it out?

kYc0o · 2017-01-25T16:24:05Z

Just checked it out locally, with a rebase it works ok.

OlegHahm

ACK

The changeset looks good and @kYc0o verified that it doesn't break anything.

daniel-k · 2017-01-26T00:17:39Z

Finally \o/ Thanks guys :)

OlegHahm · 2017-01-26T08:33:27Z

Well, thank you for the endurance. ;)

biboc · 2020-01-22T15:08:20Z

It seems that openocd PR has never been merged
I'll try to push it to get merged
Debugging thread may be very useful

kfessel · 2020-02-04T16:18:14Z

$_TARGETNAME configure -rtos auto

leads seemingly to detection of a different RTOS with Ubuntu upstream openocd-0.10.0-6

gdb> attach <openocd pid>

provides following backtrace

Thread 1 "openocd" received signal SIGSEGV, Segmentation fault. uCOS_III_update_thread_offsets (rtos=0x564ae6dc72e0) at src/rtos/uCOS-III.c:193 193 src/rtos/uCOS-III.c: No such file or directory.
after that openocd dies

thomaseichinger assigned kaspar030 Oct 6, 2015

daniel-k force-pushed the pr/gdb_threads_openocd branch from 9ae526b to 8593160 Compare October 6, 2015 14:54

daniel-k mentioned this pull request Oct 7, 2015

Bug: code to reproduce xtimer hang with concurrent xtimer_usleep() #4060

Closed

daniel-k mentioned this pull request Oct 12, 2015

Simple duty cycling 802.15.4 MAC protocol #3730

Closed

jnohlgard reviewed Oct 13, 2015
View reviewed changes

daniel-k force-pushed the pr/gdb_threads_openocd branch from e3a2dcf to 0d25acd Compare October 14, 2015 15:48

OlegHahm modified the milestone: Release 2016.03 Dec 8, 2015

OlegHahm assigned OlegHahm and unassigned kaspar030 Feb 26, 2016

daniel-k force-pushed the pr/gdb_threads_openocd branch from 7ae1937 to 6abdde4 Compare March 13, 2016 15:43

OlegHahm added this to the Release 2016.07 milestone Mar 28, 2016

miri64 modified the milestones: Release 2017.01, Release 2016.10 Oct 31, 2016

OlegHahm added the CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR label Jan 15, 2017

cortex_m: Add debug symbols for OpenOCD/GDB thread support

9d403d9

daniel-k force-pushed the pr/gdb_threads_openocd branch from 0980fcc to 9d403d9 Compare January 16, 2017 19:51

OlegHahm added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR and removed CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Jan 17, 2017

OlegHahm approved these changes Jan 25, 2017

View reviewed changes

OlegHahm merged commit c5abb08 into RIOT-OS:master Jan 25, 2017

This was referenced Jan 31, 2017

native broken on OS X #6532

Closed

CI Jenkins: activate macOS #6341

Closed

nmeum mentioned this pull request Jun 21, 2018

drivers: Add support for KW41Z builtin transceiver #7107

Closed

kfessel mentioned this pull request Feb 4, 2020

Bug: openocd 0.10.0-6 Ubuntu dies while debugging with -rtos auto #13285

Closed

cortex_m: Add debug symbols for OpenOCD/GDB thread support #4058

cortex_m: Add debug symbols for OpenOCD/GDB thread support #4058

Uh oh!

Conversation

daniel-k commented Oct 6, 2015

Uh oh!

daniel-k commented Oct 6, 2015

Uh oh!

OlegHahm commented Oct 6, 2015

Uh oh!

OlegHahm commented Oct 6, 2015

Uh oh!

daniel-k commented Oct 6, 2015

Uh oh!

jnohlgard commented Oct 7, 2015

Uh oh!

daniel-k commented Oct 7, 2015

Uh oh!

daniel-k commented Oct 13, 2015

Uh oh!

jnohlgard commented Oct 13, 2015

Uh oh!

jnohlgard Oct 13, 2015

Choose a reason for hiding this comment

Uh oh!

daniel-k Oct 14, 2015

Choose a reason for hiding this comment

Uh oh!

daniel-k Oct 14, 2015

Choose a reason for hiding this comment

Uh oh!

jnohlgard commented Oct 14, 2015

Uh oh!

daniel-k commented Oct 14, 2015

Uh oh!

daniel-k commented Oct 14, 2015

Uh oh!

jnohlgard commented Mar 10, 2016

Uh oh!

daniel-k commented Mar 13, 2016

Uh oh!

OlegHahm commented Jan 15, 2017

Uh oh!

daniel-k commented Jan 15, 2017

Uh oh!

jnohlgard commented Jan 16, 2017

Uh oh!

OlegHahm commented Jan 16, 2017

Uh oh!

daniel-k commented Jan 16, 2017

Uh oh!

miri64 commented Jan 17, 2017

Uh oh!

miri64 commented Jan 17, 2017

Uh oh!

OlegHahm commented Jan 17, 2017

Uh oh!

kYc0o commented Jan 24, 2017

Uh oh!

OlegHahm commented Jan 25, 2017

Uh oh!

kYc0o commented Jan 25, 2017

Uh oh!

OlegHahm commented Jan 25, 2017

Uh oh!

kYc0o commented Jan 25, 2017

Uh oh!

OlegHahm left a comment

Choose a reason for hiding this comment

Uh oh!

daniel-k commented Jan 26, 2017

Uh oh!

OlegHahm commented Jan 26, 2017

Uh oh!

biboc commented Jan 22, 2020

Uh oh!

kfessel commented Feb 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

kfessel commented Feb 4, 2020 •

edited

Loading