Massif Visualizer - Ready for Testers and Feedback!
memory consumption overview
Hello everyone!
In my opinion, the massif visualizer is ready for testing. I bet there are still a few rough edges, but the most important features are in. So if you are going to do any memory profiling these days, please take a look at my tool and give me feedback. I’d be especially interested in whether the massif visualizer helps in the work flow to analyze massif data files.
My personal work flow so far is the following:
callgraph of detailed massif snapshot
- generate massif log, one way or the other (unit tests preferred since they give you reproducible test cases)
- open log in massif-visualizer, look at overall consumption chart 1. how does the memory consumption evolve? is there a memleak? 2. are there designated peaks which could be reduced? 3. are there any (significant) contributions to the memory consumption, that needlessly stay over the whole application life?
- to find the actual culprits in code and/or to grasp the composition of a memory peak, use the detailed snapshot analysis
I wonder how I could improve the tool to also help with verifying that a fix helps, e.g. by either overlaying two charts or by only showing the difference. The problem here is of course that it would only work with reproducible test cases and that it needs interpolation since the snapshots are taken at random points in time. Still, it would be nice to open two massif logs and seeing the impact on memory consumption of a patch visually.
Note: To anyone interested: I generate the callgraphs by converting the massif snapshot trees into a graphviz DOT file. That one I than visualize with KGraphViewer KPart. Since KGraphViewer was in no shape to visualize the huge amount of data in my use case, I had to optimize it greatly and push in some more features to make it better suitable for thirdparty users. To integrate it better, I had to write a public interface, which also means that you need KGraphViewer installed from source to be able to compile Massif Visualizer (I’ll make it optional later on). Hence get the sources from here (packages by your distributor won’t be enough): http://websvn.kde.org/trunk/extragear/graphics/kgraphviewer/
Comments
Want to comment? Send me an email!
Comment by Eero (not verified) (2010-10-26 17:51:00)
Visualizing output of the resource tracer could be interesting too: http://maemo.gitorious.org/maemo-tools/sp-rtrace
Its current visualizers create graphs with gnuplot (unfreed allocs, alloc activity and individual resource timelines) and graphviz (callgraphs), but the aren’t interactive and re-running the visualizer for several hundred MB resource allocation trace files, to get a new callgraph, can take several minutes as these are python scripts.
Functracer tool which can attach to already running processes (to trace their resource usage), outputs the same trace format: http://maemo.gitorious.org/maemo-tools/functracer
Both of these tools allow toggling tracing on and off because you don’t want them (collecting of backtraces) to slow down the uninteresting parts of the use-cases.
sp-rtrace outputs data only when the trace ends (process terminates or tracing is toggled off), but functracer outputs it constantly. So it could even be possible to do some of the visualization at real-time while the traced program runs. :-)
Comment by adjam (not verified) (2010-03-31 01:55:00)
I could play with your tool just a bit, but let me say that you are doing a fantastic work, you rock! :)
Comment by seb (not verified) (2010-03-29 17:24:00)
Why don’t you integrate this great tool with KCacheGrind ? It’s very promising and a much needed developer tools for linux users. I wish their is more like this. Thx.
Comment by Milian Wolff (2010-03-29 17:53:00)
I was/am in contact with Joseph, the creator of KCacheGrind. My first intention was also to integrate the tools, but KCacheGrind is deeply connected with callgrind data, making it really complicated to support any different log format, like e.g. massif. Also note that Massif produces bottom-up call trees, while callgrind uses top-down trees.
Comment by Anonymous (not verified) (2010-03-29 13:30:00)
In the first screenshot, the right peak of the cyan-colored series might also be interesting, not only the absolute highest value on the left. Maybe incude all peaks that significantly stray from the average, not just the maximum value of each series?
Also, in order to avoid clutter, it might be better to not show the peak labels by default, but instead only the small black dots, and only show the corresponding label when moving the mouse over one of the dots.