Heaptrack - Attaching to Running Process

Hello all,

I’m happy to be back so soon with a status update on heaptrack: It is now possible to attach to an already running process!

Thanks to the great help from Celelibi on StackOverflow, I managed to achieve this important goal. Once you know what to do, it is actually extremely simple to patch a running process. I use GDB to attach to the process, then call dlopen to load a special heaptrack library for runtime-injection. Then I call an initialization function which takes the desired output file as a parameter, and then detach GDB. To actually overwrite malloc & friends, one can leverage dl_iterate_phdr and the public ELF API on Linux systems to find dynamic sections that reference one of our target symbols in their global offset table (GOT). This can then be rewritten to point to our custom hooks. Some refactoring later, which stabilized the shutdown sequence to allows multiple heaptrack attach/detach sequences, we can now do this:

    heaptrack -p $(pidof <yourapp>)
    # wait
    ^C
    heaptrack_print heaptrack.<yourapp>.$$.gz | less

This is a great help when you want to investigate why the memory consumption of your application suddenly rises. No need to restart the app, just attach heaptrack and wait for some, then kill it and heaptrack_print the outputfile.

Please try this new feature and send me bug reports and feedback.

Cheers

Comments

Want to comment? Send me an email!

Comment by Anonymous (2018-03-13 07:18:00)

hello Milian, I came across this heaptrack tool yesterday and was trying it on my application. I need to attach this tool to a running process. So, i downloaded the tool, and compiled it to generate all the required libraries and scripts. Procedure mentioned by you helped me to install it without any problem. However, i am facing issue when i attach heaptrack with my process pid with below error,

/usr/local/bin/heaptrack -p 64013

heaptrack output will be written to “/tmp/heaptrack.debuggee.5216.gz” injecting heaptrack into application via GDB, this might take some time… /usr/local/lib/heaptrack/libheaptrack_inject.so.1.0.0 [New LWP 64357] [New LWP 64086] [Thread debugging using libthread_db enabled] Using host libthread_db library “/lib64/libthread_db.so.1”. 0x00007fe89bf8cbf9 in ?? () from /lib64/libc.so.6 No symbol table is loaded. Use the “file” command. injection finished

Note: i have changed —batch-silent to —batch for additional info from gdb.

My question is does heaptrack run on binaries compiled in release mode i.e. no debug options during compilation. And is this above problem has to do with not having compiled the binary with debug compile options like -g flag etc. Please let me know if there is something that i am missing here. Thanks

~Neeraj

Comment by Anonymous (2016-06-14 08:58:00)

hello Milian,

Would you be able to provide the source file (tar.gz) file link to download it ?

Comment by Anonymous (2014-12-11 10:54:00)

Hi Milian.

Thanks for the new development. Just a small remark on the build process, I suppose you should check for CMake being at least 3.0 instead of 2.8 as you are using the PRIVATE keyword in the target_link_libraries command which apparently, if I read the doc correctly, appeared in 3.0 (it was called LINK_PRIVATE in previous versions).

Emmanuel

Comment by Anonymous (2014-12-11 16:04:00)

Hey there,

I just pushed a change to use LINK_PRIVATE - can you try again?

Thanks

Comment by Anonymous (2014-12-11 17:53:00)

Hi.

Yes, it is working fine now with my old cmake 2.8.10.2. Thanks for the fix.

Published on December 11, 2014.

Tags: