Tag heaptrack

Recent Posts

Heaptrack 1.0.0 (February 28, 2017)

Hey all :)

I’ve finally managed to release heaptrack properly! The first stable release, i.e. v1.0.0 is available for download: https://download.kde.org/stable/heaptrack/1.0.0/src/

You can find more information on the official release announcement over on the KDAB page: https://www.kdab.com/heaptrack-v1-0-0-release/

If you want to read more about what heaptrack is, check out the README.md or have a look at the initial announcement of heaptrack, now three years old!

Cheers, happy profiling!

Heaptrack - Attaching to Running Process (December 11, 2014)

Hello all,

I’m happy to be back so soon with a status update on heaptrack: It is now possible to attach to an already running process!

Thanks to the great help from Celelibi on StackOverflow, I managed to achieve this important goal. Once you know what to do, it is actually extremely simple to patch a running process. I use GDB to attach to the process, then call dlopen to load a special heaptrack library for runtime-injection. Then I call an initialization function which takes the desired output file as a parameter, and then detach GDB. To actually overwrite malloc & friends, one can leverage dl_iterate_phdr and the public ELF API on Linux systems to find dynamic sections that reference one of our target symbols in their global offset table (GOT). This can then be rewritten to point to our custom hooks. Some refactoring later, which stabilized the shutdown sequence to allows multiple heaptrack attach/detach sequences, we can now do this:

    heaptrack -p $(pidof <yourapp>)
    # wait
    ^C
    heaptrack_print heaptrack.<yourapp>.$$.gz | less

Heaptrack - A Heap Memory Profiler for Linux (December 02, 2014)

Hello everyone,

with a tingly feeling in my belly, I’m happy to announce heaptrack, a heap memory profiler for Linux. Over the last couple of months I’ve worked on this new tool in my free time. What started as a “what if” experiment quickly became such a promising tool that I couldn’t stop working on it, at the cost of neglecting my physics masters thesis (who needs that anyways, eh?). In the following, I’ll show you how to use this tool, and why you should start using it.

A faster Massif?

Massif, from the Valgrind suite, is an invaluable tool for me. Paired with my Massif-Visualizer, I found and fixed many problems in applications that lead to excessive heap memory consumption. There are some issues with Massif though:

It is relatively slow. Especially on multi-threaded applications the overhead is large, as Valgrind serializes the code execution. In the end, this sometimes prevents one from using Massif altogether, as running an application for hours is unpractical. I know that we at KDAB sometimes had to resort to over-night or even over-weekend Massif sessions in the hope to analyze elusive heap memory consumption issues.
It is not easy to use. Sure, running valgrind --tool=massif <your app> is simple, but most of the time, the resulting data will be too coarse. Frequently, one has to play around to find the correct parameters to pass to --depth, --detailed-freq and --max-snapshots. Paired with the above, this is cumbersome. Oh and don’t forget to pass --smc-check=all-non-file when your application uses a JIT engine internally. Forget that, and your Massif session will abort eventually.
The output is only written at the end. When you try to debug an issue that takes a long time to show up, it would be useful to regularly inspect the current Massif data. Maybe the problem is already apparent and we can stop the debug session? With Massif, this is not an option, as it only writes the output data at the end, when the debugee stops.

An optimization kata - profiling 101 at Akademy 2014 (September 09, 2014)

Yesterday my Profiling 101 workshop took place at this years Akademy in Brno. The room was packed and I got good feedback, so I hope you all learned something new :)

During my workshop, I showed you how to improve the performance of a word-count application which also creates a word histogram and finds the longest word of a file. I tried to put as many performance bottlenecks as possible into the original code base, which you can find here:

    git clone git@git.kde.org:scratch/mwolff/akademy-2014.git

Instead of uploading my useless slides full of meme images, instead I’m now pushing my optimized code branch. I urge everyone to review the commits I did and read the individual commit messages ( Note : read this log from bottom to top). There are many useful tips and tricks in there. I furthermore plan to create a techbase article with the most important notes on how to use profilers for a given job. I’ll write another blog post once I’m done with that.