kde Syndicate content

» LinuxTag 2010 WrapUp

Sun, 06/13/2010 - 01:49

Hello everyone,

I have the urge to write a quick wrap up over the just finished LinuxTag 2010: All in all, I had a good time. Really, imo it was better than last time (granted though, last LinuxTag sucked pretty badly).

Anyhow, this time was my first shot at being a speaker. Boy was I nervous… I trembled quite badly at the beginning but got confident after a few minutes. Too many “ähm“‘s and “äh“‘s though ;-) Anyhow, I apparently did my job well enough: My talk was about KDevelop 4 - Faster C++ Programming. But yeah, I did it in German (it was my first talk on such a convention, and I was already nervous enough :P), but still - since I did a live presentation I was apparently able to impress even non-German speaking attendees by the sheer amount of visual coolness I could present :)

Really, the C++ features in KDevelop I showed are so nuts and awesome, I have to thank David and friends for implementing them in KDevelop. Great job everyone! It made my life as a speaker quite easy :)

If anyone is interested in the German talk (audio is good, visuals not really…), you can download a 23min part of the talk on rapidshare: http://rapidshare.com/files/397758262/Milian-LT2010.M4V.html (Thanks to Adrian for recording and uploading it!)

What this talk showed me in retrospect, esp. combined with the feedback I got afterwards and the recurring nags in #kdevelop on freenode, is that we are in dire need of writing documentation for KDevelop. Since we devels don’t have much time for that (and it’s boring…) I think/hope to write at least a short blog series about “hidden” features, i.e. the stuff I showed in my talk. Starting with that, I hope that one of you, dear readers, steps up and creates a proper Docbook documentation out of that :)

Oh and yeah, I had quite a few chats with the friends from QtCreator developers over at the Qt/Nokia booth. They do a quite good job at catching up with us apparently :P Well, that’s what they have their payed developers for after all ;-) One real “advantage” KDevelop has though, is that it’s inherently language agnostic. The Qt guys have of course no intention to support e.g. PHP. We otoh want that - and already do so. I had a few chats with (sometimes future) users of our PHP plugin and got quite good feedback, which I want to implement eventually. Even so, I actually started implementing the first features for PHP 5.3 support: Namespaces. If someone wants a good laugh, look at how that is designed for PHP 5.3, it must have been a drunken Windows guy that never used Namespaces in any other language before… sigh.

Anyways, back to LinuxTag in general: It was of course nice again to see so many fellow KDE developers and supporters. Also seeing the good old Ubuntu Berlin and LinuxTag crowd I “know” now after four years here, was a good thing. What I have to say is that attending a conference and actually taking the time to visit a few talks is much nicer than staying at the booth all time. Yes, sorry Eckhart for even doing some hacking on the booth, but we should all take that time for ourselves. We’ve been a large crowd at the booth nevertheless, esp. since we had the combined manpower (and women!) of Amarok, Kubuntu and KDE! And attending conferences and having some quiet time makes booth-work much more bearable. It’s not like I didn’t talk to people though, quite the contrary. But as always (esp. at events like LinuxTag), you get to talk to at least one insane person every day ;-) And after the second you kind of start to loose your faith in humanity… :P

Nice as always were the social events surrounding the fair-hours, most notably of course the “friends of Qt/Nokia” dinner that served us awesome food once again. The social event of the LinuxTag at Thursday was much better than last year (even though / because?) it was sponsored by Microsoft this year. Good job that we == KDE rocked the dance floor of course ;-) Tonight we wrapped it up with the usual Ubuntu BBQ at cbase, a nice end to an imo good event. If they can improve it some more next year by pushing the community aspect more once again, I’m confident that LinuxTag in Berlin could become as good as it was a few years ago!

Anyhow, now I really have to do some more catch-up for Quanta GSOC ;-) Good night!

» First Days of Quanta+ GSOC 2010

Fri, 06/04/2010 - 22:16

Ok, finally I take the time to blog again.

The last two weeks have been a busy time for me and I couldn’t find as much time for GSOC as I would have wanted. I had to finish up some things at KDAB where I spent the last three months doing an internship (maybe I should write another blog about that eventually…). Afterwards I took five days of vacation, visiting a friend in Lisboa, Portugal - a very good decision to clear my head :) Esp. considering that I directly dropped back into my old job at the IT administration here at my university, where I had to quickly finish another project that I delayed until the last days possible ;-)

But… Even before GSOC officially started I already committed a few small things. But yesterday and today I finally started for good. Thanks to the discussion with Andris (my mentor) I actually think to know what I have to do ;-) Porting Quanta+ is not an easy task, even though there is already a Quanta binary that links against KDE4 and “works”. There are tons of files and hundres of lines of code commented out that are left to be ported. And I have to wager: What can I port in the few weeks left for GSOC and what should I drop for now? What is superseded, or should be eventually superseded, by a proper KDevplatform integrated plugin?

But well, I think I know what to do for the next few days:

  • make Quanta specific code specific to Quanta (yeah, cool eh :P)
    currently QuantaCore is a plugin and hence also shows up in KDevelop showing Quanta specific stuff there, not nice
  • spruce up Ruan Strydom’s XML plugin for KDevplatform and purge the DTEP stuff in Quanta while merging as much as possible
  • make the ‘insert tag’ etc. pp. related actions, toolbars from Quanta3 work again
  • polish Niko Sams upload, css, xdebug, execute script, execute browser plugins

If I manage most of these, I’m confident to say that I reached an important step towards a once again useful Quanta. Lets see how it works out.

» KDevelop 4 - Looking at the feedback

Mon, 05/03/2010 - 23:46

Hey everyone,

I spent some time today browsing teh interwebz to look at the responses our first KDevelop 4.0 release triggered so far. Quite fun I have to admit, given this is the first release of something I actively helped to develop that actually gets a response on the net ;-) What I noticed among the ‘I use VS’, ‘I use vi’, ‘I use ed’ comments (besides ‘I use emacs’):

  • sadly we didn’t update the screenshots everywhere yet, making some people believe we actually look like kdev 3 still ;-) this is not true, take a look here: http://apaku.wordpress.com/2010/04/25/kdevelop-4-0-screenshots/
  • yes we have support for PHP and everyone knows PHP sucks but still everyone uses it ;-) But few seem to notice that C++ is actually “just another plugin”. And we already have support for Ruby, Java and Css somewhat working in the pipelines. And very experimental stuff for C#, python and XML is also there. Imo what we said in the release announcement is true: KDev 4 is much more open for new languages than anything before. It does take some effort, true, but the result is much more pleasing.
  • we’d really welcome new blood in our dev team, esp. for new language support plugins or things like automake, qmake and qt-designer support. there are outdated plugins available, someone just has to polish them…

But all in all I’m amazed by the trolling/feedback ratio. It’s a really good feeling to see so many positive comments on various websites and $random-stranger defending KDevelop against the forces of the trolls :)

I’ll definitely continue working on KDevelop and make sure we’ll continue to improve over the time. This is just the beginning evil laughter :)

Btw. funniest comment on heise.de (German): KDevelop 4.0.0 is a released intended for developers. Of course, it’s an IDE, duh :P

» GSOC: Revive Quanta+ Brand for KDE 4

Wed, 04/28/2010 - 19:19

Yay I got a GSOC slot :)

So I hope I don’t have to introduce myself anymore to you guys. Instead I’ll show you what I’ve planned to do over the summer:

Motivation for Proposal / Goal:

Back in KDE 3 times, Quanta+ was one of the reasons for me to use KDE. In my eyes it was the IDE for web development out there, and I loved to use it. Sadly it’s bitrotting nowadays without a finished KDE 4 port. That, combined with the fact that more and more distributions drop all KDE 3 packages, makes the need for a port more urgent than ever.

Implementation Details:

Thankfully, KDevelop 4 is nearing it’s first release and the KDevplatform is mature enough nowadays. This means that during summer I shall finish the port of Quanta+ to KDevplatform and supply it with all the plugins required for a proper webdevelopment IDE. My goal is it to provide a proper IDE for PHP webdevelopment. In more detail:

  • make Quanta+ 4 compile
  • remove obsolete plugins or code parts in Quanta+
  • port required plugins to KDevplatform structure
  • polish PHP plugin, including XDebug support
  • polish Script Execute plugin
  • polish CSS plugin
  • get a first working version of a XHTML/XML plugin, if time allows even with HTML (SGML) support
    • support autocompletion
    • support inline validation
    • support documents that use multiple languages (XML, PHP, CSS, JavaScript) at the same time
  • polish the UI/Workflow for Webdevelopment
    • hide KDevelop/C++ specific actions
    • add templates for common PHP frameworks
  • if time allows, get a rough support for JavaScript (at least Outline for functions)

Put these all together with the existing features in KDevplatform we can reuse, we’ll end up with a hopefully useable IDE for webdevelopment. Hence my final goal is it to release a first Beta version of Quanta+ for KDE4.

Tentative Timeline:
  1. getting rough first shell of Quanta+ 4 up and running, removing old cruft, cleaning up old code and porting required things
    ~ 3 weeks
  2. polish existing plugins (PHP, XDebug, Execute Script, CSS, Upload)
    ~ 2 weeks
  3. create XHTML/XML plugin
    ~ 3 weeks
  4. polish UI/workflow
    ~ 2 weeks
  5. bug hunting etc., ending in a first beta release of Quanta+ for KDE 4:
    ~ 2 weeks

Lets see whether it works out as planned. But I think this commit shows you that I’m on the right track:

http://websvn.kde.org/trunk/extragear/sdk/quanta/data/pics/quanta-splash…

» You gotta watch this: RIP: A remix Manifesto

Sun, 04/25/2010 - 18:15

Hey all!

I’m now abusing the fact that my blog is aggregated on the planet to bring this diamond of a documentary some more coverage it deserves so greatly. I’m speaking about Rip: A remix Manifesto. Go and watch it. Now!

I bet every single FOSS user, developer, advocate thrives in watching it. I’m totally blown away and hope that as many people as possible watch it.

And gosh - open source cinema, how cool is that :)

» Where Profiling Sucks

Mon, 04/05/2010 - 03:12

Ok, you should know by now that I love profiling and making things faster. Yet there’s always a “but”. For me it’s blocking syscalls, or anything that makes the app “slow” for the user but doesn’t show up in Callgrind as the Instruction Fetch cost doesn’t go up.

The usual suspect is of course locks (which we have quite a lot in KDevelop) or QProcesses with waitForFinished() or similar… You won’t see them in any Callgrind profile. Does anyone know a way to achieve that? Something that makes Callgrind increase the Ir cost for blocking func calls depending on the time it blocks? Or some other tool that would show me these?

And if you are interested: I was still able to find the cause for slow parsing of Custom Make Manager projects (Qt, Linux Kernel, …) in KDevelop: The cache in the IncludePathResolver never hit, since a operator== was improperly implemented ;-) I really wonder how we could have missed that for so long! I’ve also added some more changes that should make it much faster to parse projects that rely on the IncludePathResolver. I was personally now able to parse 10.000 files of the Linux Kernel in about 9.5 minutes. This is roughly a third of the Kernel, so I’d get to a total of approx 30min. Compare that to the 2.5h for 5% that one of our users reported ;-)

» Profiling Rocks - KDevelop CMake Support now 20x faster

Wed, 03/31/2010 - 01:24

I just need to get this out quickly:

We were aware that KDevelop’s CMake support was slow. Too slow actually. It was profiled months ago and after a quick look that turned up QRegExp, it was discarded in fear of having to rewrite the whole parser properly, without using QRegExp. Which btw. is still a good idea of course.

But well, today I felt like I should do some more tinkering. I mean I managed to optimize KDevelop’s Cpp support recently (parsing Boost’s huge generated template headers, like e.g. vector200.hpp is now 30% faster). I managed to make KGraphViewer usable for huge callgraphs I produce in Massif Visualizer. So how hard could it be to make KDevelop’s CMake at least /a bit/ faster, he?

Yeah well an hour later and two commits later, I managed to find and fix two bottlenecks. Both where related to QRegExp. Neither was the actual parser, instead it was the part that evaluated CMake files, esp. the STRING(...) function. So even if we’d used a proper parser generator, this would still been slow.

The first one was the typical “don’t reinvent the wheel” kinda commit which already made the CMake support two times faster for projects that used FindQt4.cmake, i.e. any Qt or KDE project. Not bad, right? Well, while I fixed that I saw that KDevelop tried to do some Regular expression replacement on the output of qmake --help, this could not been right, could it? With help of Andreas and Aleix we found the bug in the parser and that made the CMake support 10 times faster.

So yeah, CMake projects using Qt or KDE should now get opened a whopping 20 times faster in KDevelop :)

I really love KCacheGrind and Valgrind’s callgrind - again it proved to be the most awesome tool one can imagine! If you are interested in the callgrind files:

  1. without optimization
  2. first optimization
  3. second optimization

Note: with KCacheGrind from trunk you can open these compressed files transparently :)

» Massif Visualizer - Ready for Testers and Feedback!

Mon, 03/29/2010 - 12:47


memory consumption overview

Hello everyone!

In my opinion, the massif visualizer is ready for testing. I bet there are still a few rough edges, but the most important features are in. So if you are going to do any memory profiling these days, please take a look at my tool and give me feedback. I’d be especially interested in whether the massif visualizer helps in the work flow to analyze massif data files.

My personal work flow so far is the following:


callgraph of detailed massif snapshot
  1. generate massif log, one way or the other (unit tests preferred since they give you reproducible test cases)
  2. open log in massif-visualizer, look at overall consumption chart
    1. how does the memory consumption evolve? is there a memleak?
    2. are there designated peaks which could be reduced?
    3. are there any (significant) contributions to the memory consumption, that needlessly stay over the whole application life?
  3. to find the actual culprits in code and/or to grasp the composition of a memory peak, use the detailed snapshot analysis

I wonder how I could improve the tool to also help with verifying that a fix helps, e.g. by either overlaying two charts or by only showing the difference. The problem here is of course that it would only work with reproducible test cases and that it needs interpolation since the snapshots are taken at random points in time. Still, it would be nice to open two massif logs and seeing the impact on memory consumption of a patch visually.

Note: To anyone interested: I generate the callgraphs by converting the massif snapshot trees into a graphviz DOT file. That one I than visualize with KGraphViewer KPart. Since KGraphViewer was in no shape to visualize the huge amount of data in my use case, I had to optimize it greatly and push in some more features to make it better suitable for thirdparty users. To integrate it better, I had to write a public interface, which also means that you need KGraphViewer installed from source to be able to compile Massif Visualizer (I’ll make it optional later on). Hence get the sources from here (packages by your distributor won’t be enough): http://websvn.kde.org/trunk/extragear/graphics/kgraphviewer/

» Massif Visualizer - now with user interaction

Sat, 03/13/2010 - 16:55

Just a quick status update: Massif Visualizer now reacts on user input. Meaning: You can click on the graph and the corresponding item in the treeview gets selected and vice versa. It’s a bit buggy since KDChart is not reliable on what it reports, but it works quite well already.

Furthermore the colors should be better now, peaks are labeled (better readable on bright color schemes, I’m afraid to say…), legend is shown, …

Now lets see how I can make the treeview more useful!

» Transparent loading of compressed Callgrind files in KCacheGrind

Thu, 03/11/2010 - 23:43

Hey everyone!

I just committed an (imo) insanely useful feature for KCacheGrind: Transparent loading of compressed Callgrind files. Finally one does not have to keep those Callgrind files around uncompressed, hogging up lots of space. And what is even more important: It’s much easier to share these files now, as you can send or upload them as .gz or better yet .bz2 and open them directly. KDE architecture just rocks :) So in KDE 4.5 the best profiling visualizer just got better :D

In related news: I’m spending my time as intern at KDAB currently by creating an application to visualize Massif. If you are interested, check the sources out on gitorious: http://gitorious.org/massif-visualizer

It’s still pretty limited in what it offers, yet is probably already more useful than the plain ASCII graph that ms_print generates:


Visualization of a Massif output file

This is very WIP but the visuals are somewhat working now. I plan to make the whole graph react on user input, i.e. zoomable, click to show details about snapshots, show information about the heap items that make up the stacked part of the diagram, …

Also very high on my wish list is some kind of interaction with the KCacheGrind libraries, to reuse it’s nice features like callgraphs, cost maps, etc. pp. you name it :) All these features that make KCacheGrind such an insanely useful application.

Oh and remember: Never do performance optimizations without checking the facts first ;-)