Sun, 10/28/2012 - 15:18
Before I continue blogging about actual coding achievements, I must mention something else: Many many thanks to Niko’s company Vivid Planet for sponsoring one awesome dinner here at the KDevelop/Kate sprint in Vieanna. The Schnitzel and beer was much appreciated :)
Static Code Analysis
I spent a considerable amount of time to review some feature branches and working towards integrating them. This means that now Aleix’s master thesis work was merged. This one is about a static analysis framework, which he has implemented for C++. This should allow his other work to be merged as well eventually, showing control flow graphs or writing checkers like finding inquired include statements. I hope he’ll blog about that eventually.
File & Project Templates
Furthermore, I’ve finally merged Miha’s GSOC Code which gives us a much better project template support. Furthermore, you can now create (and share!) file templates. A few good examples are the existing QTestLib template e.g. I’m pretty sure that what we have right now is quite cool already. It needs to be polished though to make it nicer to use. Suggestions and bug reports welcome!
Rename file after renaming a class
Another feature that Miha worked on was unit test integration for KDevelop. The branches for that are now also merged into master, awaiting testing and feedback. Aleix and Niko also worked on improving it already, so I hope they will all blog about it.
Rename File Assistant
Aleix had a pending review request which renames the host file when you used ‘Rename Declaration’ on a class. I’ve included that and fixed the implementation. Furthermore, I’ve integrated it into the existing rename assistant, fixing a few bugs there while at it. This is a quite nice addition I think! Please test it and report bugs or suggestions.
Besides merging other people’s work and improving it, I actually spend some time on new stuff on my own :)
Uses in Tool View
Have you ever tried the “show uses” action from the context browser tooltip? It worked but you had to be really careful in not moving the mouse anywhere such that the tooltip was closed… That just sucked. I now fixed this to always use the tool view instead to show the uses. Looking at that code part though made me realize that some serious cleanup work should be done there eventually… And the UI also could use a serious make over, it looks pretty ugly imo. But while at it, I’ve also fixed the bug where the uses in the uses widget where marked with a yellow background without overwriting the foreground color. This yielded unreadable results (white text on bright yellow) for dark color scheme users.
Cleaned Up Open With
Triggered by a review request I decided to cleanup our Open With plugin. Now, the actions are properly grouped and show whether they open an external app or whether they have an embedded editor. I also took the liberty to rename “Advanced Embedded Text Editor” (i.e. Katepart) to “Default Editor”. Oh and see how it’s bold-fonted? Thats the currently configured option by the user. For .ui files e.g. the designer is usually selected. I think the new UI is so simple that we don’t need any extra configuration dialog for that.
Now… back to hacking and enjoying the last day in Vienna :)
Sun, 10/28/2012 - 13:40
Wow… the last days are a blur. I hoped to blog more but once more failed at doing so. What an awesome sprint this is… Once again, many thanks to Joseph for organizing this! But lets now blog about really noteworthy stuff!
Inline Syntax Errors for QML in KDevelop
QML/JS language support
Aleix was doing quite some QML work-work recently. Sadly KDevelop has no language support for that, so to stay productive you start to use Qt Creator for the QML files sooner or later. This is of course perfectly fine, except for those people that love to use Kate e.g. or for those that prefer our interpretation of the IDE metaphor and C++ language support. So what should we do about that? Right, lets write a QML/JS Plugin.
Of course it is still a long long way until we even get close to the feature set offered by Qt Creator for QML files. I’m quite sure though that we will continue to work on that. Considering more and more people relying on QML/JS for their UI work, or looking at the role of JS in the web, I think this is an essential step towards a better KDevelop.
So what are the next steps? Personally, I think we should try to whip up a basic DUChain integration that gives you some context browsing. Once that is done, try to support
imports such that we can figure out the QML API for Qt built in files. Then, we can think about kick-ass code completion. It’s going to take time but we can do it!
Now I said this is not a fork, and I mean it. We copied the code since there is no reusable library (and that just makes sense, you don’t want to keep a stable API for a language AST). I’ve added a README to the copy though, that reads the following:
This code here is copied from Qt Creator:
This contents of this folder can be found in src/libs/. For license information see LICENSE.LGPL and LGPLG_EXCEPTION.TXT.
NOTE: This is not a fork! Do not touch anything beside the CMake build files. Everything else must stay in sync. If you find a bug or have an improvement, make sure to contribute it back to Qt Creator! This plugin would not be possible without their work, so that is the least we can do.
I mean what I wrote there. Many many thanks to the work done by the diligent Trolls and Nokia for creating a reusable parser for QML/JS. I’ve talked with a few Qt Creator developers over the years, and they are a nice bunch of talented developers. I tip my virtual hat and bow to their achievements.
And last but not least: Thanks Aleix for doing the dirty lib import and CMake build system so I could concentrate on the KDevelop language plugin integration :) Without you I wouldn’t have done that! And once more many thanks to the KDE e.V. and Joseph for making the sprint possible!
Thu, 10/25/2012 - 13:37
Finally I take some time to blog again. I’m currently in Vienna for the joint KDevelop/Kate sprint together with lots of other hackers. Many thanks to Joseph for planning and partially financing this sprint! And of course as usual many thanks to the KDE e.V. and all the donors for bringing in the rest of the money required to pull something like this off!
Anyhow, considering that the sprint is running since Tuesday, I need to catch up quite a bit… Actually, I have to start even before that since I committed something quite noteworthy in KDevelop and KMail last week.
Reducing Memory Consumption
Shared Data References
I attended the recent Akonadi sprint that took place at the KDAB office in Berlin (where I work btw.). I heard that Alex Fiestas would come and show us his memory problems in KMail, which sooner or later was eating multiple GBs of memory for him. That sounded like a fun task to improve, fixing performance issues is what I love to do :) So I investigated it with Valgrind/Massif and my pmap script. After quite some time I came up with a patch to fix the memory increase, which is waiting for Stephen Kelly to review. It should be merged into master very soon™.
Now some technical background: What was the issue here? Why wasn’t it found earlier? Usually developers run e.g. KMail through Valgrind with the leak checker and fix all issues. The same was done lots of times in KMail, there where no problems reported. Why then is the memory still increasing over time? The issue is, that this is technically not a “leak”, i.e. when you close the application all memory is properly released. Instead, there was a logical error that resulted in KMail’s ETM (basically the item model for mails and stuff) push shared data items into a QHash without ever deleting them. If you close the app though, the QHash is cleared automatically and all shared data is properly freed, hence Valgrind won’t report any leaks.
How does one find such an issue then, though? Actually, this is really hard… I ran KMail through Valgrind and looked at the top allocation, which relates to the shared data item. But Massif will only show you where the data is being allocated, not where the shared data items are still referenced and thus prevent a proper deallocation. What now? GDB! Yes, the best way I found was adding a breakpoint in the copy constructor of the shared data class and looking at the surrounding code to see whether it behaves properly… Does anyone have a more efficient way to debug such issues? I could imagine that this can potentially take ages to figure out… In KMail at least I could find the problematic place quite fast.
Now while this apparently fixes the ever increasing memory consumption of KMail somewhat, I thought we could do some more improvements. Take a look at https://git.reviewboard.kde.org/r/106836/ e.g. This patch is similar to what I also did for Massif Visualizer once. By creating a central cache we can leverage Qt’s implicit sharing for common strings (in this case email e.g. adresses, domains, …). This way, if you load a folder containing e.g. a mailing list, you will have the main email address (like
firstname.lastname@example.org) only once in memory. Before, that address would be loaded into memory once for every email in the folder…
Now the above was an interesting detour into a project that I don’t usually contribute to. Since I use KMail all the time though, it is just fair to give back and help out the few KDEPIM people a bit.
Back to my favorite pet project: KDevelop :) In the spirit of the memory fixes above, I took another look at the memory consumption of KDevelop. Turns out, we had a similar issue where we did not reuse implicit sharing properly. This resulted in quite some useless allocations blowing up the memory consumption (in this case, KUrl’s of every file in the projects loaded for a session). The fix is already in master. Not only should that decrease the memory consumption considerably for kdevelop sessions with many files in it. No, it actually should save quite a few instructions and thus be much faster as well. Enjoy!
So… quite a long blog post again - sorry for that :) Expect some more KDevelop news the next days - we have lots of interesting stuff happening here at the sprint! Cheers and many thanks to Joseph and the KDE e.V. again!
Tue, 10/23/2012 - 17:32
A first broadcast from the joint KDevelop/Kate sprint! Many thanks already to Joseph for planning this all. As you can see, we also have net access and thus nothing can prevent a very productive week :)
Anyhow, back to the actual news: I finally announced the release of KDevelop 4.4.0. Many thanks to all involved again.
Mind you though, the change list is a bit sparse this time. Yet I’m really looking forward to the 4.5 release already. I’ll blog about it in the next days during the sprint. Stay tuned!
Tue, 08/21/2012 - 16:57
Two of my friends gave me the Logitech Mini Boombox for my birthday. While it can be used with a plain old jack cable, it also supports Bluetooth. I don’t have a smartphone, but my laptop has Bluetooth support, so lets try it out, shall we?
First, we have of course to install all the required software packages. On Archlinux that was all pulled in by installing the
bluedevilpackage. We could now theoretically start Bluetooth already and connect to the device, but lets fix a problem first: Pulseaudio integration: See this Gentoo wiki page, essentially you have to add a
Enable=Socketline to the
[General]section of your
Now, start Bluetooth via
/etc/rc.d/bluetooth start1 and head over to the Bluetooth section in
systemsettings. There you should be able to connect to the Boombox using “Add device”. Once that’s done, go to the Multimedia section in
systemsettingsand prefer the new boombox audio device over your built-in hardware devices. Then, don’t forget to go to the “Audio Hardware Setup” tab, select the Boombox in the “Sound Card” combobox and finally choose the “High Fidelity Playback (A2DP)” profile. If you don’t do that, the audio quality will be abysmal!
Cheers, it should work now. If the Boombox is not connected, your built-in hardware will be used. If you connect the Boombox, the audio device will automatically be switched over and you can enjoy much louder music compared to cheesy laptop speakers.
On Archlinux, if you want to have the daemon running automatically when you restart your machine, don’t forget to add it to the
Tue, 08/14/2012 - 18:01
After quite some slacking on my side, I’ve finally managed to drop the good news: KDevelop 4.4.0 Beta 1 is released!
Our 4.4 branch already contains some more interesting changes for the next beta, stay tuned. Oh and yeah, we’ll try to release 4.4.0 final sometime in September, I hope.
Thu, 04/19/2012 - 10:58
KDevelop 4.3.1 is out! Go read the announcement and update.
Many thanks to all contributors, you rock :)
Tue, 03/20/2012 - 13:21
Finally I managed to get my job as the release dude done: http://kdevelop.org/kdevelop/kdevelop-430-final-released-basic-c11-support
Thanks to all the developers who sent in patches! The same goes to our loyal users for their continued support and bug reports :)
It’s really fun to work on KDevelop and - I’ve said it many times before - I’m really looking forward to our next releases! Even now our code in the master branches has some neat commits that make the eventual 4.4 release something to look forward to!
Fri, 03/16/2012 - 15:42
As I just wrote in another article, Massif is an invaluable tool. The [Visualizer](https://projects.kde.org/massif-visualizer] I wrote is well appreciated and widely used as far as I can see.
A few days ago though, I did a very long (~16h) Massif run on an application, which resulted in a 204MB
massif.outdata file. This proved to be a very good stress test for my visualizer, which triggered me to spent some time on optimizing it. The results are pretty nice I thing, so look forward to Massif-Visualizer 0.4:
Reduced Memory Consumption
Yeah, meta eh? Just how I like it! I’ve used Massif to improve the memory consumption of Massif-Visualizer, and analyzed the data in the Visualizer of course… :)
The initial version of my visualizer took about ~470MB of memory to load the 204MB data file above. 80% of that was required for
QStringallocations in the callgraph of each detailed snapshot, i.e. the function signatures and location. See fig. 1 for the details.
QString to QByteArray
Thomas McGuire gave me the tip of using
QByteArrayinstead, since the Massif callgraph data is just ASCII data. We can convert the data to QString where required, essentially saving us 50% of the memory consumption. You can see that applied in fig. 2. It was simple to code and already reduced the memory consumption considerably.
I committed the above, thinking this was it. But thanks to the awesome people in the KDE community, this time André Wöbbeking, I was thankfully shown wrong: He commented on my commit, arguing that I should try out to leverage the implicit sharing of Qt containers, such as QByteArray. After all, the strings we have here are function signatures and file locations, which are repeated quite often. Especially when you have recursion in your call tree, or the same functions are encountered again and again in Massif snapshots, you can potentially safe a lot of memory by leveraging implicit sharing.
Personally, I’m suprised to see just how much this gains in this case! See fig 3., where the string allocations are nearly gone completely from the Massif log! Now only the tree node allocations, and the containers saving them, are visible in the memory log - something I do not plan to reduce further.
If you are interested in how this was implemented, take a look at commit 4be5dad13fb.
I think this shows quite nicely how to improve the memory consumption of an application. If you want to verify my results, I’ve uploaded the massif log files. Remember that you can open compressed files seamlessly in Massif-Visualizer. The
massif.out.data.bz2file contains the test-data of the 16h Massif run.
You should probably use the latest Massif-Visualizer code though, since I’ve also optimized the performance of it considerably compared to the last released version 0.3. Furthermore, data files are now loaded in the background, showing a nice progress bar while doing that. If you open the big data file in 0.3 you’ll notice why I decided to optimize the visualizer :)
An interesting thing to note btw. is that the callgrind data format compresses files and function signatures, yielding much smaller data files and reducing the KCacheGrind’s memory consumption, esp. since it will automagically leverage the implicit sharing of Qt’s string classes.
Now it is probably time to stop slacking and start work-work again :) I do have quite a few ideas more for the next Massif-Visualizer though, especially an export functionality for the graphs is high on my TODO list!