cmake Syndicate content

» Profiling Rocks - KDevelop CMake Support now 20x faster

Wed, 03/31/2010 - 01:24

I just need to get this out quickly:

We were aware that KDevelop’s CMake support was slow. Too slow actually. It was profiled months ago and after a quick look that turned up QRegExp, it was discarded in fear of having to rewrite the whole parser properly, without using QRegExp. Which btw. is still a good idea of course.

But well, today I felt like I should do some more tinkering. I mean I managed to optimize KDevelop’s Cpp support recently (parsing Boost’s huge generated template headers, like e.g. vector200.hpp is now 30% faster). I managed to make KGraphViewer usable for huge callgraphs I produce in Massif Visualizer. So how hard could it be to make KDevelop’s CMake at least /a bit/ faster, he?

Yeah well an hour later and two commits later, I managed to find and fix two bottlenecks. Both where related to QRegExp. Neither was the actual parser, instead it was the part that evaluated CMake files, esp. the STRING(...) function. So even if we’d used a proper parser generator, this would still been slow.

The first one was the typical “don’t reinvent the wheel” kinda commit which already made the CMake support two times faster for projects that used FindQt4.cmake, i.e. any Qt or KDE project. Not bad, right? Well, while I fixed that I saw that KDevelop tried to do some Regular expression replacement on the output of qmake --help, this could not been right, could it? With help of Andreas and Aleix we found the bug in the parser and that made the CMake support 10 times faster.

So yeah, CMake projects using Qt or KDE should now get opened a whopping 20 times faster in KDevelop :)

I really love KCacheGrind and Valgrind’s callgrind - again it proved to be the most awesome tool one can imagine! If you are interested in the callgrind files:

  1. without optimization
  2. first optimization
  3. second optimization

Note: with KCacheGrind from trunk you can open these compressed files transparently :)

» Shell helper: running KDE unit tests (ctests) the easy way

Thu, 03/26/2009 - 03:09

Unit tests are in my eyes a very important part of programming. KDE uses them, KDevelop does - the PHP plugin I help writing does as well. cmake comes with a ctest program which does quite well to give you a quick glance on which test suite you just broke with your new fance feature :)

But I am very dissatisfied with it. Right now I usually do the following

  1. # lets assume I'm in the source directory
  2. cb && ctest
  3. # look for failed test suites
  4. cd $failed_test_suite_path
  5. ./$failed_test_suite.shell | less
  6. # search for FAIL
  7. cs
  8. cd $to_whereever_I_was_before

That’s pretty much for just running a test. Especially all that cding and lessing became very tedious. Tedious is good, because I eventually fix it:

introducing kdetest

I wrote a bash function (with autocompletion!!!) called kdetest. Calling it without any parameter will run all test suites and gives a nice report of failed functions at the end. Here’s an example (run via cs php && kdetest).

  1. kdetest
  2. # ... lots of test output
  3.  
  4. --- ALL PASSED TESTS ---
  5. ...
  6. PASS : Php::TestCompletion::implementMethods()
  7. PASS : Php::TestCompletion::inArray()
  8. PASS : Php::TestCompletion::cleanupTestCase()
  9.  
  10. 143 passed tests in total
  11.  
  12. --- ALL FAILED TESTS ---
  13. FAIL! : Php::TestCompletion::newExtends() Compared values are not the same
  14. FAIL! : Php::TestCompletion::updateExtends() '! forbiddenIdentifiers.contains(item->declaration()->identifier().toString())' returned FALSE. ()
  15. FAIL! : Php::TestCompletion::updateExtends() '! forbiddenIdentifiers.contains(item->declaration()->identifier().toString())' returned FALSE. ()
  16. FAIL! : Php::TestCompletion::updateExtends() Compared values are not the same
  17. FAIL! : Php::TestCompletion::newImplements() Compared values are not the same
  18. FAIL! : Php::TestCompletion::updateImplements() Compared values are not the same
  19.  
  20. 6 failed tests in total
usage
  • kdetest, i.e. without any arguments runs all tests in this directory and below
  • kdetest path/to/test.shell ... runs that test suite only, ... can by any argument the test suite accepts.
autocompletion

kdetest comes with full support for autocompletion of tests and functions, for example:

  1. milian@odin:~/projects/kde4/php$ kdetest TABTAB
  2. completion/tests/completiontest.shell duchain/tests/expressionparsertest.shell parser/test/lexertest.shell
  3. duchain/tests/duchaintest.shell duchain/tests/usestest.shell
  4. milian@odin:~/projects/kde4/php$ kdetest duchain/tests/usestest.shell TABTAB
  5. classAndConstWithSameName classSelf interfaceExtendsMultiple staticMemberFunctionCall
  6. classAndFunctionWithSameName constAndVariableWithSameName memberFunctionCall staticMemberVariable
  7. classConstant constant memberFunctionInString variable
  8. classExtends constantInClassMember memberVariable variableTwoDeclarations
  9. classImplements functionAndClassWithSameName memberVarInString variableTwoDeclarationsInFunction
  10. classImplementsMultiple functionCall newObject varInString
  11. classParent interfaceExtends objectWithClassName
the code

You can find the code below, or you can obtain the most up-to-date version on github. Just head over to my shell-helpers repo and peek into the bash_setup_kde4_programming file.

» Building KDevplatform, KDevelop and the PHP plugin from Git

Fri, 03/06/2009 - 01:21

In a follow up to my last post about the PHP plugin for KDevelop and Quanta I want to detail how I setup my build environment. Since I’m only working on KDevelop I only want to compile that. For all other applications, I use the packages my distribution makes available. Below you this is possible.

Note: I use Kubuntu and hence some commands like aptitude and the package names etc. will have to be adapted to your specific distribution. But the general idea should be the same and the packages should be called at least somewhat similar. You could always take a look at the techbase article on compilation of KDE 4 to get a more general HowTo.

It is generally a very good idea to read some articles on the techbase. Especially in the Getting Started section.

getting the dependencies

First, you’ll need to install some packages to satisfy dependencies and to get a working compile chain (think cmake, gcc and stuff). The following command only lists very few packages but they have a huge list of dependencies which your package manager will take into account automatically. Don’t be suprised by a possibly huge list of packages ;-) So some disk space will be required (though I doubt it’s more than 250MB in total).

Note: You’ll need the KDE 4.2.x versions of the KDE libraries and development packages. When you use Kubuntu, you can follow this article to get up to date packages for the 8.10 release.

  1. # I hope these are all, it's been a time since I did that the last time
  2. # please report if anything is missing
  3. aptitude install kdelibs5-dev kdebase-workspace-dev \
  4. libqt4-dev build-essential subversion kdesdk-scripts \
  5. bison flex
removing old cruft

To prevent binary compatibility issues that will result in crashes, you have to remove all KDevelop & KDevplatform packages provided by your package maintainer. In karmic it should be enough to remove kdevplatform, and it will remove all dependent packages:

  1. aptitude purge kdevplatform libsublime
setting up your environment

Inspired by the techbase article on improved productivity in KDE4, particularly the bashrc listed there, I’ve come up with the following way to configure my system:

First visit my shell helper git repo on GitHub and download the two files kde4_setup_build_environment.sh and bash_setup_kde4_programming.

set the environment variables

Move the file kde4_setup_build_environment.sh to ~/.kde/env and make it executable. You might want to adapt the paths for the variables KDE_BUILD, KDE_SRC and KDEDIR inside that file to your likings.

This part is required to make sure that compiled programs will act just like normal programs installed globally (e.g. with your package manager) right from the start of your KDE session. I.e. KRunner works fine and custom plugins are found etc.

make your life easier with some bash magic

Now alter your ~/.bashrc and to the bottom of that file add the following line:

  1. . ~/.bash_setup_kde4_programming

NOTE: This assumes that you saved the afore mentioned bash_setup_kde4_programming file to ~/.bash_setup_kde4_programming. You might have to change the path.

NOTE: That file also sources the above script to set the environment variables. I’m not sure it’s required, but it doesn’t hurt. So make sure the path is correct there.

Now to the fun part

Once all that’s done, you have to log out of your bash session to get the environments. Maybe even logout of your KDE session to make it aware of the new paths as well.

checking out KDevplatform, KDevelop and the PHP plugin

Get the latest and greatest directly from Git Master (see also git.kde.org Manual on techbase). But first make sure you setup the Git URL prefixes, by putting the following into your ~/.gitconfig file:

  1. [url "git://anongit.kde.org/"]
  2. insteadOf = kde:
  3. [url "git@git.kde.org:"]
  4. pushInsteadOf = kde:

Now checkout the sources:

  1. cs # cs is not a typo! see above
  2. git clone kde:kdevplatform
  3. git clone kde:kdevelop
  4. # if you only want KDevelop, you can stop here. The rest is for PHP
  5. git clone kde:kdevelop-pg-qt
  6. git clone kde:kdev-php
  7. git clone kde:kdev-php-docs
compiling and installing

Now it’s time to compile all that code you just checked out. Hopefully you got all required dependencies. If you need all bells and whistles, you’ll have to install some more packages I’m sure. If you run the code below, take a good look at the output of the cmakekde command (especially at the beginning). It lists not-found dependencies.

Note: cmakekde is supposed to be run from inside your source folder (e.g. cs $FOLDER; cmakekde).

  1. for p in kdevplatform kdevelop kdevelop-pg-qt php php-docs; do
  2. cs $p
  3. cmakekde
  4. done;
  5. kbuildsycoca4 # make sure new plugins etc. are found

Pretty easy, hum? Let’s hope everything worked fine :)

staying up to date

That above command is only required once. Everytime after that, you can simply do the following which will make sure you are running the latest and greatest:

  1. for p in kdevplatform kdevelop kdevelop-pg-qt php php-docs; do
  2. cs $p
  3. git pull --rebase && make install
  4. done
The End

Hope I have not forgotten anything and that this (rather lengthy…) HowTo is of some help to a few of you out there. Looking forward to feature requests, bug reports etc.

UPDATE 30/03/11: updated to git.kde.org location

UPDATE 18/05/10: updated to include kdevelop-pg-qt, introduced for loops in setup snippets

UPDATE 07/05/10: updated to git locations

UPDATE 08/01/10: updated to extragear location of php & php-docs

UPDATE 01/12/09: added php-docs, updated to extragear location of kdevplatform & kdevelop.

UPDATE 17/12/09: updated to kdereview location of php & php-docs