Hey all :)
I’ve finally managed to release heaptrack properly! The first stable release, i.e. v1.0.0 is available for download: https://download.kde.org/stable/heaptrack/1.0.0/src/
You can find more information on the official release announcement over on the KDAB page: https://www.kdab.com/heaptrack-v1-0-0-release/
If you want to read more about what heaptrack is, check out the README.md or have a look at the initial announcement of heaptrack, now three years old!
Cheers, happy profiling!
continue reading...
Hello all!
I have the pleasure to attend Akademy this year again. From my past experience, I’m really looking forward to have a good time again. Lots of hacking, meeting known and unknown faces, drinking beer and socializing ahead! I also love that it’s in a (to me) new country again, and wonder what I will see of the Czech Republic and Brno!
This year, the conference schedule is a bit different from the past years. Not only do we have the usual two days packed with interesting talks and keynotes. No - this year there will also be workshops on the third day! These are more in-depth talks which hopefully teach the audience some new skills, be it QML, mobile development, testing, or … profiling :) Your’s truly has the honor to hold a one-hour Profiling 101 workshop.
continue reading...
Hello all,
I’m happy to be back so soon with a status update on heaptrack: It is now possible to attach to an already running process!
Thanks to the great help from Celelibi on StackOverflow, I managed to achieve this important goal. Once you know what to do, it is actually extremely simple to patch a running process. I use GDB to attach to the process, then call dlopen to load a special heaptrack library for runtime-injection. Then I call an initialization function which takes the desired output file as a parameter, and then detach GDB. To actually overwrite malloc & friends, one can leverage dl_iterate_phdr and the public ELF API on Linux systems to find dynamic sections that reference one of our target symbols in their global offset table (GOT). This can then be rewritten to point to our custom hooks. Some refactoring later, which stabilized the shutdown sequence to allows multiple heaptrack attach/detach sequences, we can now do this:
heaptrack -p $(pidof <yourapp>)
# wait
^C
heaptrack_print heaptrack.<yourapp>.$$.gz | less
continue reading...
Hello everyone,
with a tingly feeling in my belly, I’m happy to announce heaptrack, a heap memory profiler for Linux. Over the last couple of months I’ve worked on this new tool in my free time. What started as a “what if” experiment quickly became such a promising tool that I couldn’t stop working on it, at the cost of neglecting my physics masters thesis (who needs that anyways, eh?). In the following, I’ll show you how to use this tool, and why you should start using it.
A faster Massif?
Massif, from the Valgrind suite, is an invaluable tool for me. Paired with my Massif-Visualizer, I found and fixed many problems in applications that lead to excessive heap memory consumption. There are some issues with Massif though:
- It is relatively slow. Especially on multi-threaded applications the overhead is large, as Valgrind serializes the code execution. In the end, this sometimes prevents one from using Massif altogether, as running an application for hours is unpractical. I know that we at KDAB sometimes had to resort to over-night or even over-weekend Massif sessions in the hope to analyze elusive heap memory consumption issues.
- It is not easy to use. Sure, running
valgrind --tool=massif <your app> is simple, but most of the time, the resulting data will be too coarse. Frequently, one has to play around to find the correct parameters to pass to --depth, --detailed-freq and --max-snapshots. Paired with the above, this is cumbersome. Oh and don’t forget to pass --smc-check=all-non-file when your application uses a JIT engine internally. Forget that, and your Massif session will abort eventually. - The output is only written at the end. When you try to debug an issue that takes a long time to show up, it would be useful to regularly inspect the current Massif data. Maybe the problem is already apparent and we can stop the debug session? With Massif, this is not an option, as it only writes the output data at the end, when the debugee stops.
continue reading...
Yesterday my Profiling 101 workshop took place at this years Akademy in Brno. The room was packed and I got good feedback, so I hope you all learned something new :)
During my workshop, I showed you how to improve the performance of a word-count application which also creates a word histogram and finds the longest word of a file. I tried to put as many performance bottlenecks as possible into the original code base, which you can find here:
git clone git@git.kde.org:scratch/mwolff/akademy-2014.git
Instead of uploading my useless slides full of meme images, instead I’m now pushing my optimized code branch. I urge everyone to review the commits I did and read the individual commit messages ( Note : read this log from bottom to top). There are many useful tips and tricks in there. I furthermore plan to create a techbase article with the most important notes on how to use profilers for a given job. I’ll write another blog post once I’m done with that.
continue reading...
Hey all,
since some people asked me: The slides to my extended Apps on Speed talk from this year’s Qt DevDays Berlin are available for download. If you are interested, get them here: http://devdays.kdab.com/wp-content/uploads/2013/11/qt-dd-2013-apps-on-sp…
I hope you liked that talk. I certainly had fun presenting it and discussing the contents with various attendees later on. I have now quite some ideas on how to extend the talk even further.
The slides of the other presentations are also available. Stay tuned for the video recordings of DevDays Berlin, I’m sure they will be accessible soonish :)
Edit: The video is now available! Enjoy http://www.youtube.com/watch?v=C5EPt50Kgmc
continue reading...
Wow…
I’ve been gone for eight days and returned just a few hours ago to Berlin. It doesn’t feel like that. The last days went by in a blur of awesomeness! The reason why I didn’t write a single blog post in between is just that I never had a spare minute for that. I arrived on Thursday and instantly enjoyed the warmth of Spain / the Basque country and had a tasty and cheap Menu del Dia at a local Restaurant with fellow KDABians and other KDE friends. Then just a few hours later the first party started, near the old district of the city - amazing! More and more hackers and helpers arrived, the atmosphere was once again so good. The social aspect of this years Akademy was without comparison in my opinion - seriously: Hats off to the local team, you did an amazing job!
While the social events on the following days have been just as awesome or even awesomer to awesomest - I especially enjoyed the day trip and jumping into the ocean! - the technical side of Akademy delivered just as well: My favorite talks this year where Mirko’s about ThreadWeaver, which we heavily use in KDevelop. His roadmap and polished API looks much better than what we have nowadays and should allow for much nicer code which might even perform better - kudos!
continue reading...
Hey all,
been some time since I blogged last time. My TODO list is ever increasing and I took my day job at KDAB up again. Among others, I attended a marketing talk by Edmund Preiss. He actually made that marketing talk interesting, not least by his huge knowledge in the business, thanks to ~20 years of working for Intel. Probably the most important info I got out of it is this:
VTune is available free-of-charge under a non-commercial license
Yes, you heard right. Take these links:
note this entry from the FAQ:
What does noncommercial mean?
Non-commercial means that you are not getting compensated in any form for the products and/or services you develop using these Intel® Software Products.
continue reading...
Hey all,
I’d like to have some feedback from you. Consider this code:
#include <iostream>
#include <memory.h>
using namespace std;
struct List {
List(int size) {
begin = new int[size];
memset(begin, 0, size);
end = begin + size;
}
~List() {
delete[] begin;
}
int at(int i) const {
return begin[i];
}
int size() const {
// std::cout << "size called" << std::endl;
return end - begin;
}
int& operator[](int i) {
return begin[i];
}
private:
int* begin;
int* end;
};
int main() {
const int s = 1000000;
for (int reps = 0; reps < 1000; ++reps) {
List l(s);
List l2(s);
// version 1
for ( int i = 0; i < l.size(); ++i ) {
// version 2
// for ( int i = 0, c = l.size(); i < c; ++i ) {
l2[i] = l.at(i);;
}
}
return 0;
}
continue reading...
Easy question: What is the fastest way to determine if a string in PHP is whitespace-only?
Easy answer: ('[^\s]', $string);
Read on for the explanation:
I applied the codeblocks feature of my profile.class.php, here’s the testcase:
<?php
require 'profile.class.php';
$iterations = 10000;
profile::codeblocks(array(
'trim(long string) == ""' => 'trim($long) == ""',
'trim(long string) === ""' => 'trim($long) === ""',
'rtrim(long string) == ""' => 'rtrim($long) == ""',
'rtrim(long string) === ""' => 'rtrim($long) === ""',
'ltrim(long string) == ""' => 'ltrim($long) == ""',
'ltrim(long string) === ""' => 'ltrim($long) === ""',
'!preg_match("[^\s]", long string)' => '!preg_match("[^\s]", $long)',
'ctype_space(long string)' => 'ctype_space($long)',
'trim(short string) == ""' => 'trim($short) == ""',
'trim(short string) === ""' => 'trim($short) === ""',
'rtrim(short string) == ""' => 'rtrim($short) == ""',
'rtrim(short string) === ""' => 'rtrim($short) === ""',
'ltrim(short string) == ""' => 'ltrim($short) == ""',
'ltrim(short string) === ""' => 'ltrim($short) === ""',
'!preg_match("[^\s]", short string)' => '!preg_match("[^\s]", $short)',
'ctype_space(short string)' => 'ctype_space($short)',
), array(
'long' => str_repeat(" \n\t ", 500) . "a" . str_repeat(" \n\t ", 500),
'short' => str_repeat(" \n\t ", 5) . "a" . str_repeat(" \n\t ", 5),
), $iterations);
profile::print_results(profile::flush());
continue reading...
