Fri, 06/04/2010 - 22:16
Ok, finally I take the time to blog again.
The last two weeks have been a busy time for me and I couldn’t find as much time for GSOC as I would have wanted. I had to finish up some things at KDAB where I spent the last three months doing an internship (maybe I should write another blog about that eventually…). Afterwards I took five days of vacation, visiting a friend in Lisboa, Portugal - a very good decision to clear my head :) Esp. considering that I directly dropped back into my old job at the IT administration here at my university, where I had to quickly finish another project that I delayed until the last days possible ;-)
But… Even before GSOC officially started I already committed a few small things. But yesterday and today I finally started for good. Thanks to the discussion with Andris (my mentor) I actually think to know what I have to do ;-) Porting Quanta+ is not an easy task, even though there is already a Quanta binary that links against KDE4 and “works”. There are tons of files and hundres of lines of code commented out that are left to be ported. And I have to wager: What can I port in the few weeks left for GSOC and what should I drop for now? What is superseded, or should be eventually superseded, by a proper KDevplatform integrated plugin?
But well, I think I know what to do for the next few days:
- make Quanta specific code specific to Quanta (yeah, cool eh :P)
currently QuantaCore is a plugin and hence also shows up in KDevelop showing Quanta specific stuff there, not nice
- spruce up Ruan Strydom’s XML plugin for KDevplatform and purge the DTEP stuff in Quanta while merging as much as possible
- make the ‘insert tag’ etc. pp. related actions, toolbars from Quanta3 work again
- polish Niko Sams upload, css, xdebug, execute script, execute browser plugins
If I manage most of these, I’m confident to say that I reached an important step towards a once again useful Quanta. Lets see how it works out.
Mon, 05/03/2010 - 23:46
I spent some time today browsing teh interwebz to look at the responses our first KDevelop 4.0 release triggered so far. Quite fun I have to admit, given this is the first release of something I actively helped to develop that actually gets a response on the net ;-) What I noticed among the ‘I use VS’, ‘I use vi’, ‘I use ed’ comments (besides ‘I use emacs’):
- sadly we didn’t update the screenshots everywhere yet, making some people believe we actually look like kdev 3 still ;-) this is not true, take a look here: http://apaku.wordpress.com/2010/04/25/kdevelop-4-0-screenshots/
- yes we have support for PHP and everyone knows PHP sucks but still everyone uses it ;-) But few seem to notice that C++ is actually “just another plugin”. And we already have support for Ruby, Java and Css somewhat working in the pipelines. And very experimental stuff for C#, python and XML is also there. Imo what we said in the release announcement is true: KDev 4 is much more open for new languages than anything before. It does take some effort, true, but the result is much more pleasing.
- we’d really welcome new blood in our dev team, esp. for new language support plugins or things like automake, qmake and qt-designer support. there are outdated plugins available, someone just has to polish them…
But all in all I’m amazed by the trolling/feedback ratio. It’s a really good feeling to see so many positive comments on various websites and
$random-stranger defending KDevelop against the forces of the trolls :)
I’ll definitely continue working on KDevelop and make sure we’ll continue to improve over the time. This is just the beginning evil laughter :)
Btw. funniest comment on heise.de (German): KDevelop 4.0.0 is a released intended for developers. Of course, it’s an IDE, duh :P
Wed, 04/28/2010 - 19:19
Yay I got a GSOC slot :)
So I hope I don’t have to introduce myself anymore to you guys. Instead I’ll show you what I’ve planned to do over the summer:
Motivation for Proposal / Goal:
Back in KDE 3 times, Quanta+ was one of the reasons for me to use KDE. In my eyes it was the IDE for web development out there, and I loved to use it. Sadly it’s bitrotting nowadays without a finished KDE 4 port. That, combined with the fact that more and more distributions drop all KDE 3 packages, makes the need for a port more urgent than ever.
Thankfully, KDevelop 4 is nearing it’s first release and the KDevplatform is mature enough nowadays. This means that during summer I shall finish the port of Quanta+ to KDevplatform and supply it with all the plugins required for a proper webdevelopment IDE. My goal is it to provide a proper IDE for PHP webdevelopment. In more detail:
- make Quanta+ 4 compile
- remove obsolete plugins or code parts in Quanta+
- port required plugins to KDevplatform structure
- polish PHP plugin, including XDebug support
- polish Script Execute plugin
- polish CSS plugin
- get a first working version of a XHTML/XML plugin, if time allows even with HTML (SGML) support
- support autocompletion
- support inline validation
- polish the UI/Workflow for Webdevelopment
- hide KDevelop/C++ specific actions
- add templates for common PHP frameworks
Put these all together with the existing features in KDevplatform we can reuse, we’ll end up with a hopefully useable IDE for webdevelopment. Hence my final goal is it to release a first Beta version of Quanta+ for KDE4.
- getting rough first shell of Quanta+ 4 up and running, removing old cruft, cleaning up old code and porting required things
~ 3 weeks
- polish existing plugins (PHP, XDebug, Execute Script, CSS, Upload)
~ 2 weeks
- create XHTML/XML plugin
~ 3 weeks
- polish UI/workflow
~ 2 weeks
- bug hunting etc., ending in a first beta release of Quanta+ for KDE 4:
~ 2 weeks
Lets see whether it works out as planned. But I think this commit shows you that I’m on the right track:
Sun, 04/25/2010 - 18:15
I’m now abusing the fact that my blog is aggregated on the planet to bring this diamond of a documentary some more coverage it deserves so greatly. I’m speaking about Rip: A remix Manifesto. Go and watch it. Now!
I bet every single FOSS user, developer, advocate thrives in watching it. I’m totally blown away and hope that as many people as possible watch it.
And gosh - open source cinema, how cool is that :)
Mon, 04/05/2010 - 03:12
Ok, you should know by now that I love profiling and making things faster. Yet there’s always a “but”. For me it’s blocking syscalls, or anything that makes the app “slow” for the user but doesn’t show up in Callgrind as the Instruction Fetch cost doesn’t go up.
The usual suspect is of course locks (which we have quite a lot in KDevelop) or QProcesses with
waitForFinished() or similar… You won’t see them in any Callgrind profile. Does anyone know a way to achieve that? Something that makes Callgrind increase the Ir cost for blocking func calls depending on the time it blocks? Or some other tool that would show me these?
And if you are interested: I was still able to find the cause for slow parsing of Custom Make Manager projects (Qt, Linux Kernel, …) in KDevelop: The cache in the IncludePathResolver never hit, since a
operator== was improperly implemented ;-) I really wonder how we could have missed that for so long! I’ve also added some more changes that should make it much faster to parse projects that rely on the IncludePathResolver. I was personally now able to parse 10.000 files of the Linux Kernel in about 9.5 minutes. This is roughly a third of the Kernel, so I’d get to a total of approx 30min. Compare that to the 2.5h for 5% that one of our users reported ;-)
Wed, 03/31/2010 - 01:24
I just need to get this out quickly:
We were aware that KDevelop’s CMake support was slow. Too slow actually. It was profiled months ago and after a quick look that turned up QRegExp, it was discarded in fear of having to rewrite the whole parser properly, without using QRegExp. Which btw. is still a good idea of course.
But well, today I felt like I should do some more tinkering. I mean I managed to optimize KDevelop’s Cpp support recently (parsing Boost’s huge generated template headers, like e.g.
vector200.hpp is now 30% faster). I managed to make KGraphViewer usable for huge callgraphs I produce in Massif Visualizer. So how hard could it be to make KDevelop’s CMake at least /a bit/ faster, he?
Yeah well an hour later and two commits later, I managed to find and fix two bottlenecks. Both where related to QRegExp. Neither was the actual parser, instead it was the part that evaluated CMake files, esp. the
STRING(...) function. So even if we’d used a proper parser generator, this would still been slow.
The first one was the typical “don’t reinvent the wheel” kinda commit which already made the CMake support two times faster for projects that used
FindQt4.cmake, i.e. any Qt or KDE project. Not bad, right? Well, while I fixed that I saw that KDevelop tried to do some Regular expression replacement on the output of
qmake --help, this could not been right, could it? With help of Andreas and Aleix we found the bug in the parser and that made the CMake support 10 times faster.
So yeah, CMake projects using Qt or KDE should now get opened a whopping 20 times faster in KDevelop :)
I really love KCacheGrind and Valgrind’s callgrind - again it proved to be the most awesome tool one can imagine! If you are interested in the callgrind files:
Note: with KCacheGrind from trunk you can open these compressed files transparently :)
Mon, 03/29/2010 - 12:47
memory consumption overview
In my opinion, the massif visualizer is ready for testing. I bet there are still a few rough edges, but the most important features are in. So if you are going to do any memory profiling these days, please take a look at my tool and give me feedback. I’d be especially interested in whether the massif visualizer helps in the work flow to analyze massif data files.
My personal work flow so far is the following:
callgraph of detailed massif snapshot
- generate massif log, one way or the other (unit tests preferred since they give you reproducible test cases)
- open log in massif-visualizer, look at overall consumption chart
- how does the memory consumption evolve? is there a memleak?
- are there designated peaks which could be reduced?
- are there any (significant) contributions to the memory consumption, that needlessly stay over the whole application life?
- to find the actual culprits in code and/or to grasp the composition of a memory peak, use the detailed snapshot analysis
I wonder how I could improve the tool to also help with verifying that a fix helps, e.g. by either overlaying two charts or by only showing the difference. The problem here is of course that it would only work with reproducible test cases and that it needs interpolation since the snapshots are taken at random points in time. Still, it would be nice to open two massif logs and seeing the impact on memory consumption of a patch visually.
Note: To anyone interested: I generate the callgraphs by converting the massif snapshot trees into a graphviz DOT file. That one I than visualize with KGraphViewer KPart. Since KGraphViewer was in no shape to visualize the huge amount of data in my use case, I had to optimize it greatly and push in some more features to make it better suitable for thirdparty users. To integrate it better, I had to write a public interface, which also means that you need KGraphViewer installed from source to be able to compile Massif Visualizer (I’ll make it optional later on). Hence get the sources from here (packages by your distributor won’t be enough): http://websvn.kde.org/trunk/extragear/graphics/kgraphviewer/
Sat, 03/13/2010 - 16:55
Just a quick status update: Massif Visualizer now reacts on user input. Meaning: You can click on the graph and the corresponding item in the treeview gets selected and vice versa. It’s a bit buggy since KDChart is not reliable on what it reports, but it works quite well already.
Furthermore the colors should be better now, peaks are labeled (better readable on bright color schemes, I’m afraid to say…), legend is shown, …
Now lets see how I can make the treeview more useful!
Thu, 03/11/2010 - 23:43
I just committed an (imo) insanely useful feature for KCacheGrind: Transparent loading of compressed Callgrind files. Finally one does not have to keep those Callgrind files around uncompressed, hogging up lots of space. And what is even more important: It’s much easier to share these files now, as you can send or upload them as
.gz or better yet
.bz2 and open them directly. KDE architecture just rocks :) So in KDE 4.5 the best profiling visualizer just got better :D
In related news: I’m spending my time as intern at KDAB currently by creating an application to visualize Massif. If you are interested, check the sources out on gitorious: http://gitorious.org/massif-visualizer
It’s still pretty limited in what it offers, yet is probably already more useful than the plain ASCII graph that
Visualization of a Massif output file
This is very WIP but the visuals are somewhat working now. I plan to make the whole graph react on user input, i.e. zoomable, click to show details about snapshots, show information about the heap items that make up the stacked part of the diagram, …
Also very high on my wish list is some kind of interaction with the KCacheGrind libraries, to reuse it’s nice features like callgraphs, cost maps, etc. pp. you name it :) All these features that make KCacheGrind such an insanely useful application.
Oh and remember: Never do performance optimizations without checking the facts first ;-)
Fri, 02/26/2010 - 16:45
steveire is experimenting with QML so I couldn’t stop but notice that there is no highlighting for it in Kate. Well, there was none ;-) Now you get pretty colors, rejoice!
.js as well. Enjoy!