Milian Wolff

Hey everyone. Been some time since I last blogged… I want to take the chance to give you all a bit of overview over the last changes in the PHP plugin for KDevelop:

Performance Improvements

I spent quite some time profiling various aspects of the PHP plugin, be it parsing and DUChain building over to (just yesterday) code completion. I’m confident to say that I found (and removed) quite a few bottlenecks, making the plugin much more comfortable to use.

I’m really blown away time over time again by valgrind & KCachegrind… What would we do without these tools?

Documentation

An (imo) very strong point for using KDevelop for PHP programming is it’s seamless support for documentation. Be it the “simple” inline documentation in the Declaration Tooltips or the extended integration of the remote PHP.net docs. But these past days I improved our generator for the inline documentation of built-in PHP language constructs considerably:

• add documentation for function/method parameters
• add @since where appropriate
• fix some missing declarations or bugs, esp. some that where part of SPL

These changes increased the size of the generated file to whopping 3.1M. This was too much for me and hence I added support to ship this file zipped (i.e. now it’s 80% smaller) and we decompress it transparently with the use of KZip. Adding such a feature was a breeze, so nice to work with the KDE Api :)

There are still a few things missing and stuff I plan to improve here, so stay tuned for more news in this regard.

Hey everyone! The PHP & PHP-Docs plugins for KDevelop now moved to Extragear! The new locations are:

svn+ssh://svn.kde.org/home/kde/trunk/extragear/sdk/kdevelop-plugins/php
svn+ssh://svn.kde.org/home/kde/trunk/extragear/sdk/kdevelop-plugins/php-docs

Or the anonsvn equivalent:

svn://anonsvn.kde.org/home/kde/trunk/extragear/sdk/kdevelop-plugins/php
svn://anonsvn.kde.org/home/kde/trunk/extragear/sdk/kdevelop-plugins/php-docs

Happy coding!

Ha, what a bit of a bribe can do to me… Someday earlier today a user of the PHP plugin for KDevelop brought up the flickering issue in it again. Well as I told him: I myself find it very annoying and wanted to fix it since quite some time, but never got around to it… Usually that would be it and I’d go watch some more FamilyGuy until I’m in the mood to track this bugger down. But well, thankfully Phlogi wasn’t so easy to dispatch:

<Phlogi> milian: ok… I’ll send you pizza and beer if you fix this!

Hours of gdb sessions later, I finally committed a fix. So Phlogi, if you read this: You owe me ;-)

To all others: If you tried the PHP plugin out and the flickering was too much for you: Give it a try again! I’m personally totally overwhelmed, the difference is huge! I often perceived the PHP plugin to be magnitudes slower than the C++ one. Well, looks like most of this was only due to the flickering. Now things are much smoother.

Happy holidays!

Hey all!

Just a quick note: Niko and me moved PHP & PHP-Docs to kdereview, we hope to move both plugins to extragear/sdk/kdevelop-plugins. So, if I understood things correctly, after a two week period the plugins will get moved there (well, if we pass the review, but I think we can do that).

So for anybody that uses the plugins from SVN, you’ll have to relocate. The new addresses are:

• /trunk/kdereview/kdevelop-php
• /trunk/kdereview/kdevelop-php-docs

See you in two weeks :)

Good news everyone :)

Documentation

After years of pretty much no documentation (except looking at the sources…), The_User aka. Jonathan Schmidt-Dominé started documenting the parser generator that is used for most KDevelop language plugins (java, python, php, …). You can find it here: http://techbase.kde.org/Development/KDevelop-PG-Qt_Introduction

It has to be improved and more examples have to be put in there, but it’s already a huge improvement over the situation before…

Performance

In related news I did some profiling on the parsing of the quite big file that includes all internal PHP declarations (i.e. all functions, classes, definitions,…). It drops in at a whopping 1.9M, with ~80k lines. Well, turns out that this showed a pretty easy to fix bottleneck in KDevelop-PG’s LocationTable, which used to use a linear lookup algorithm. Profiling showed that nearly 75% was spent in that function. But I used the past tense for a reason:

I replaced it with an algorithm that combines a relative search (i.e. relative to the last lookup) with a binary search fallback. That’s comparatively blazingly fast. I added some benchmarks to KDevelop-PG-Qt that proofs that (benchmark below run with release mode build):

********* Start testing of KDevPG::Benchmarks *********
Config: Using QTest library 4.5.2, Qt 4.5.2
PASS   : KDevPG::Benchmarks::initTestCase()
RESULT : KDevPG::Benchmarks::positionAt():"initial, linear":
     1,184.0 msec per iteration (total: 11840, iterations: 10)
RESULT : KDevPG::Benchmarks::positionAt():"initial, random":
     1,008.5 msec per iteration (total: 10085, iterations: 10)
RESULT : KDevPG::Benchmarks::positionAt():"relative, linear":
     1.3 msec per iteration (total: 14, iterations: 10)
RESULT : KDevPG::Benchmarks::positionAt():"relative, random":
     1,185.0 msec per iteration (total: 11850, iterations: 10)
RESULT : KDevPG::Benchmarks::positionAt():"binary, linear":
     31.1 msec per iteration (total: 312, iterations: 10)
RESULT : KDevPG::Benchmarks::positionAt():"binary, random":
     39.2 msec per iteration (total: 392, iterations: 10)
RESULT : KDevPG::Benchmarks::positionAt():"binary & relative, linear":
     0.8 msec per iteration (total: 8, iterations: 10)
RESULT : KDevPG::Benchmarks::positionAt():"binary & relative, random":
     40.2 msec per iteration (total: 402, iterations: 10)
PASS   : KDevPG::Benchmarks::positionAt()
PASS   : KDevPG::Benchmarks::cleanupTestCase()
Totals: 3 passed, 0 failed, 0 skipped
********* Finished testing of KDevPG::Benchmarks *********

In our “realworld” phpfunctions.php example the DUChain building process got twice as fast (in debug mode though, but still).

We should probably get lost of the LocationTable altogether and use an existing container (QList, QVector, QLinkedList,… or any STL variant of them). But this would mean more profiling, and well, lets see when we get to it… But just got a reply on the KDevelop list showing interest on this, so maybe someone does that eventually!

Profiling rocks, and KCacheGrind is such a great application. I love such visualization. Well done! And the QTestLib benchmark utilities are also very solid, nice!

PS: KDevelop beta6 is in the pipelines and we’ll release a beta1 of the PHP plugin together with it. Packagers and users rejoice :)

Hi there again! I’ve been silent again on my blog, but didn’t rest on development. In the one and a half months since the last digest, I started writing a PHP application This finally made me eat my own dog food :). It resulted in lots of polishing and quite a few bug fixes for the PHP plugin in KDevelop. Here’s a list of what I think are the notably changes since the last digest:

(Note: to view screenshots, go to the bottom of this article.)

• refactoring of parts of the Code Completion code, should already result in faster code under certain situations
• properly mark constants as “Kind: Constant” in the declaration tooltips
• offer argument hints for ctors during code completion in class init statements
• greatly improve the generate inline documentation of built-in PHP functions, classes, properties etc. pp.
  • add documentation of public properties
  • support aliased functions (thanks to Victor Grischenko for his patch)
  • show more/all documentation, and not only the first paragraph
  • fix type-lookup
  • don’t get confused when a documentation file documents both, a method and a function (greatly improves e.g. MySQLi documentation)
• don’t offer “jump to declaration” for built-in PHP declarations
• add support for list(...) statement
• cleanup code-completion list, esp. show the return type of functions in the prefix field, and not something a la “function ReturnType ($arg1, $arg2, …)”
• improve the code-completion for include/require statements
• add language constructs to code completion (e.g. class, while, foreach, print, …)
• show declaration tooltip for magic constants, showing their current value
• make functions, methods and classes case-insensitive, just like PHP handles them
• some performance improvements, especially in code completion and parsing of the generated file containing php-internal declarations
• lots of bug fixes, don’t want to iterate them all ;-)

I created a Flickr account and will push screenshots to it for every upcoming digest I do. To see some of the features I talk about above, visit: http://www.flickr.com/photos/milianw/sets/72157622375662249/detail/

EDIT: A note to those that want to try this out , but cannot / don’t want to compile it: I heard that there is / will be an AUR package for Arch users available. And I’d really appreciate it if others could create packages for their favorite distribution as well. You’ll need bleeding-edge kdevplatform + kdevelop though!

Not only KDevelop gets better and better PHP support — the Kate PHP syntax file also got a few new features and fixes over the last weeks. The good thing is of course that all users of KWrite, Kate, Quanta, KDevelop and other editors leveraging the Katepart benefit from these changes.

Improved HereDocs

screenshot of improved highlighting in PHP heredocs

I went over PHP related bugs on bugs.kde.org today and spotted one that was fairly easy to fix:

vim-like syntax highlighting support for heredocs in php.xml

With some magic (IncludeRules just rocks) I got it working fairly easy. You can see the results to the right.

Additionally I added code folding to heredocs, since often these strings include lots of text and hiding it often makes sense.

Better support for overlapping syntax regions

code folding with overlapping syntax regions

Another long standing bug (”accommodate overlapping syntax regions (especially for php)”) got fixed by James Sleeman.

Finally PHP templates with code such as

<?php if ( true ) : ?>
  <!-- some html stuff -->
<?php elseif ( false ) { ?>
  <!-- some other html stuff -->
<?php } ?>

can be folded properly. This kind of spaghetti code is used quite often in simple templates and having the posibility to fold it properly is a huge win in my opinion. Thanks to James Sleeman again!

So, after a period of silence I present you an unsorted list of features and bug fixes I did to the PHP plugin for KDevelop in the last few weeks:

• added support for path autocompletion after require / include statements: This shares code with the Cpp plugin and it’s completion after #include. I plan to use this autocompletion eventually also for functions accepting filenames or paths. Think of fopen, file_get_contents etc. pp. So far only URLs that are covered by an open project can be completed. We will need support for custom include paths so we can support e.g. global PEAR or framework installations.

• worked around a bug where the “schedule all project files for parsing” had no affect. Apaku is currently working on making some internal changes so I can remove the workaround and fix this properly. With this fixed/worked around you should retry the PHP plugin along with frameworks or similar to see how well it works. Just make sure to have the framework inside your project so it gets parsed.

• fixed code completion of static class members for classes that where not defined in the current file

• by sharing code with the cpp plugin I made it possible to show information about includes on hover. You can open the file from that popup or just see what classes, functions, variables etc. are defined there and jump to them quickly.

• report errors when the user tries to include a directory

• totally rewrite a huge part of the code completion: It now utilizes the token stream and can hence work much more reliably. Also the code is much cleaner in my opinion and it should be easier to extend its functionality. If you spot places where you think no code completion should happen or vice versa, please report that to me!

• don’t crash on empty includes (i.e. include ""; )

• ignore xml declarations (i.e. <?xml ... ?>)

• support implicit declarations of referenced arguments, esp. important for e.g. preg_match("#...#", $string, $matches);, before the $matches was said to be undefined. Additionally references should be marked and shown as such now in the popups of e.g. function arguments.

• support implicit array declarations, e.g. $a[0] = 1; // now $a is an array

• fix bug which made the class browser not aware of changes in PHP class structures. Now we can see new classes, methods and members directly after writing them in a PHP file.

• support /// comment syntax for doc-comments, especially useful for type hints of variables.

• report error when someone tries to declare a class constant with an invalid type (i.e. not bool, int, float or string)

I’m very happy to say that we seem to get more and more (happy) users!

Has been a time since I wrote the last update for PHP / webdevelopment related work in KDevelop4…

XDebug

First of all, Niko (nsams) is working diligently on the general debugger framework for KDevelop4. Of course gdb is still the main focus so far yet he always has his XDebug plugin (which is currently unusable) in mind. So expect some great debugging features once he’s done with the framework.

Remote Projects

Another thing I’ve not yet written about is a bunch of commits I did just after the Hackaton: Remote Projects! Yes, finally you can work directly on the server with KDevelop. I still have to make sure that the user does not try to open a remote cmake project (or similar), because that is of course not supported. Only the Generic Project Manager (basically a list of files in a dir) works. There’s for sure some things to polish, yet I’d like to see user feedback for that use-case at least.

Note: There is still some functionality missing in other areas, like creating a new remote project from a template etc.

Out-of-class declarations of member variables

The PHP plugin itself got support for some more code, like declarations of members variables outside the class context which was reported by one of our early adopters (hint hint - do the same! we need more wishes and bug reports!).

Improved DeclarationBuilder

Additionally I changed the logic of our DeclarationBuilder so it does not create a new variable declaration for each assigment expression. I think this makes refactoring, use-highlighting etc. in a PHP environment much better for many cases. Sadly it is not yet perfect since we still need to redeclare variables when their type changes. Fixing that would require potentially big changes to the DUChain and I’m not sure when it will happen.

Error Reporting

Furthermore I’ve added some more inline error reportings, namely for redeclaration of $this in a class context. This is also an area where you might help us a lot: Send us PHP snippets which fail when executed but are not yet highlighted in the editor. I’ll try to add them then. Just use the bugtracker on http://bugs.kde.org.

Performance

Also David (dnolden / zwabel) of Cpp-KDev4 fame made some changes which resulted in a good performance win. Thanks!

Ok, just a quick update on what we are doing / have done the last two days except drinking beer and eating quite a lot of food (though it never seems to be enough somehow….)

• Apol was polishing CMake support, speeding up KDevelop startup time when you enabled the QtHelp plugin, making sure that Kross plugins get loaded properly and some other things
• Adymo started some refactoring of Sublime in an experimental branch (actually they had quite a long discussion right now about things in the UI which should be changed/fixed). Besides that he also fixed some little bugs in the includepath resolver for CMake
• Nsams is fixing PHP support and adding new features here and there. Esp. completion inside a foreach over a Iterator class works now!
• and as always ZWabel did fix quite a lot of bugs, most of them which only were discoverable in a few corner cases, very nice indeed.
• shaforo who sadly already left us again to attend university fixed the sorting in the Project view among other things
• With Apols help I managed to get started with a documentation plugin which (currently) integrates PHP.net. I plan to make it possible to use the downloadable PHP.net documentation as well to speed things up. Actually setting up a documentation plugin is very easy! Maybe I’ll take a look at writing a documentation plugin for Zend, Symfony, etc. pp. as well - should be fairly easy. Maybe I can come up with a generic documentation plugin which makes it possible to include any kind of .html documentation with some configurable mapping… lets see! Or maybe I should take a look at writing Kross plugins (Apol could help me here as well) for that purpose.

Except these changes we also discussed how to support dynamically typed languages like PHP, Ruby, Python and JavaScript better. One result of it that we now plan to fully support the dedicated map types of each language, with autocompletion and such. Of course that won’t be possible for 100% of the use-cases, but we hope to support at least the most common ones (i.e. 90%). You can take a look at http://www.kdevelop.org/mediawiki/index.php/SupportForDynamicMaps for more information.

Additionally Nsams, Apol and Zwabel took a look at the GDB plugin and started working on it a bit. It’s still a work in progress though.

My biggest change is that color highlighting support for dark color schemes now is in trunk. It automatically adapts to changes in the global KDE color scheme, yet you still have to manually change the Kate color scheme. Actually I intend to add an interface to Kate so we can use it’s color schemes instead of the global KDE one - makes much more sense and would fix some bugs (i.e. fix from dark color scheme to a light one). Here’s a screenshot of it (with the PHP.net documentation showing):

dark color scheme and syntax highlighting in KDevelop 4

If you followed the planet kde you might have seen some blog posts about color reception and color generation by Ariya Hidayat. I will take a look at it and implement it in KDevelop. Or I simply pick a given set of colors and use them - for 10 colors or so we need it doesn’t make that much sense to let them be generated. Yet the article he linked to about adaptive coloring for syntax highlighting will be helpful I think! Thanks!

Yet it currently works quite well I’ll first try to fix the colors in the Declaration tooltip, quickopen and codecompletion lists.

So stay tuned for more KDevelop goodness throughout the next days.