Code SnippetsTracking Memory Consumption Using Pmap Syndicate content

Fri, 03/16/2012 - 14:55

Massif is a really nifty tool which is very powerful, especially paired with my visualizer. The caveat of course is that it slows down the application considerably, I’ve seen anything up to a factor of 100… I see no alternative to Massif when it comes to investigating where your memory problems come from. But if you just want to see whether you have a problem at all, tracking the total memory consumption should suffice.

A few days ago, I came across pmap on Stack Overflow, which makes it easy to track the RSS memory consumption of an application using the -x switch. Of course I had to write some bash magic to automate this process and visualize the data using Gnuplot! Behold:

memory consumption of PhantomJS
memory consumption of a PhantomJS script over ~30min

It’s simple, really: $(pidof myapp).

The default timeout is ~1s between snapshots, you can pass a different timeout as second parameter. Bash’s sleep can also take float numbers such as 0.1 to get more snapshots for fast-running apps.

You can also run mem.log.$(pidof myapp) while you are still tracking the memory. The gnuplot window that appears allows you to update the data intermittently, to zoom in and to create images such as the above.

Note: This kind of memory usage tracking costs nearly nothing, your application continues to work at full speed. Also be aware that this just shows the RSS memory consumption. Massif will always give you better, more detailed and accurate results. Still, I think this should already give you an idea on how your application behaves. If the graph goes up and up, you probably got a memory leak! Then it’s time to run Memcheck and/or Massif to find the issue and fix it!

You can find the most-recent version on GitHub:

  1. #!/bin/bash
  3. #
  4. # track memory of given application, identified by PID,
  5. # using pmap -x, to show RSS and Dirty memory usage.
  6. #
  7. # visualization can later on be done with the
  8. # script.
  9. #
  11. pid=$1
  12. sleep=$2;
  14. if [[ "$sleep" == "" ]]; then
  15. sleep=1
  16. fi
  18. if [[ "$(ps -p $pid | grep $pid)" == "" ]]; then
  19. echo "cannot find program with pid $pid"
  20. echo " PID [SLEEP_TIMEOUT]"
  21. echo
  22. echo "example: \$(pidof someapp) 0.1"
  23. exit
  24. fi
  26. logfile=mem.log.$pid
  28. echo "# $(ps -o command= -p $pid)" > $logfile
  29. echo "# $sleep" >> $logfile
  31. cat $logfile
  33. while [[ "$(ps -p $pid | grep $pid)" != "" ]]; do
  34. echo "snapshot " $pid
  35. pmap -x $pid | tail -n1 >> $logfile
  36. echo "$sleep"
  37. sleep $sleep;
  38. done
  40. echo "done tracking, visualizing"
  41. $(dirname $0)/ "$logfile"

You can find the most-recent version on GitHub:

  1. #!/bin/bash
  3. #
  4. # visualize memory consumption over time
  5. # as recorded by pmap /
  6. # script
  7. #
  9. logfile=$1
  11. if [ ! -f "$logfile" ]; then
  12. echo "cannot find memory logfile: $1"
  13. echo
  14. echo "usage: LOGFILE"
  15. echo
  16. echo "example: mem.log.12345"
  17. exit
  18. fi
  20. title=$(head -n1 "$logfile")
  21. timeout=$(head -n2 "$logfile" | tail -n1)
  23. title=${title/\# /}
  24. timeout=${timeout/\# /}
  26. # total:
  27. # '$logfile' using 3 w lines title 'Kbytes', \
  29. gnuplot -p -e "
  30. set title '$title';
  31. set xlabel 'snapshot ~${timeout}s';
  32. set ylabel 'memory consumption in kB';
  33. set key bottom right;
  34. plot \
  35. '$logfile' using 4 w lines title 'RSS' lt 1, \
  36. '$logfile' using 4 smooth bezier w lines title 'RSS (smooth)' lt 7, \
  37. '$logfile' using 5 w lines title 'Dirty' lt 2, \
  38. '$logfile' using 5 smooth bezier w lines title 'Dirty (smooth)' lt 3;
  39. ";

The above is nice, but I’m wondering on whether one should not add this kind of utility to ksysguard: It already allows you to track the total memory consumption of your system, yet I did not find a way to just track a single application and visualize it’s memory consumption.


Yes, please. These should be Fri, 04/13/2012 - 10:01 — Ritesh Raj Sarraf (not verified)

Yes, please. These should be integrated.

Nice scripts. I totally agree Mon, 03/19/2012 - 14:08 — Anonymity is great (not verified)

Nice scripts. I totally agree with you that this should be part of ksysguard. This would definitely increase the awesomeness of ksysguard.

Post new comment

  • You can use Markdown syntax to format and style the text. Also see Markdown Extra for tables, footnotes, and more.
  • You can enable syntax highlighting of source code with the following tags: <code>, <blockcode>, <pre>.
  • Web page addresses and e-mail addresses turn into links automatically.

More information about formatting options