kate Syndicate content

» Kate/KDevelop Sprint 2014: First Days

Tue, 01/21/2014 - 19:24

Hey all! Greetings from the joint Kate/KDevelop sprint at the Blue Systems office in Barcelona!

I only arrived yesterday but already I have great news for you: After months of work I finally merged the sharedurls branches into master for KDevelop/KDevplatform etc. pp. There I worked on a optimization in our handling of file paths.

The status quo up until know was the following: When importing a folder as a project in KDevelop, we filled a model with every file and folder in the project (recursively). For every item we also stored its path as a KUrl to the potentially remote location. KUrl and QUrl are awesome when you have to work with paths and urls, but as soon as you store potentially thousands of them at the same time it becomes quite inefficient. Assume e.g. you open /foo/bar/blub/ which contains /foo/bar/blub/bla.h. When you use KUrl/QUrl to store these paths, you cannot share any memory between the two, as internally basically a QString is used. Thus, when you import deep folder trees or folders with many files, you’ll waste a lot of memory for common sub-paths. Furthermore, due to the amount of allocations required, reading the tree is pretty slow.

So in the sharedurls branch I worked on a internal replacement for KUrl in KDevelop: It’s called KDevelop::Path and is a glorified QVector<QString> with convenience API to simulate a KUrl and simplify porting. Every entry in the vector contains a path segment. It leverages QStrings implicit sharing to minimize the memory overhead. Furthermore, when you parse a tree structure recursively, all you do is copying vectors and appending strings to them - which is rather cheap as a QString is a small handle structure.

So all in all this should greatly improve the performance of opening projects in KDevelop. Especially for large sessions containing thousands of files (eg.: Qt 4, multiple Qt 5 modules, LibreOffice, Kernel, WebKit, …) the new code is much faster and consumes less memory. I’ve seen time savings in the order of multiple seconds in total as well as memory consumption going down in the order of 100MB.

While this sounds like a fairy-tale, I have to admit that it was/is a lot of work: By using a custom class, you have to convert to KUrl/QUrl or QString quite often when interacting with existing API. This of course is costly and potentially marginalizes or even pessimises the potential performance gains. Hence one needs to pay special attention and port code such that it minimizes these conversions. As such I can only recommend anyone doing something like that when you have similar extreme usecase. For a normal file browser or web browser I doubt the you’ll gain much if anything.

So please compile the current master branches and take a look for yourself. My tests and benchmarks look all good, yet I might have overlooked something. If you spot any regressions, please shout!

Now that this is mostly done and polished, I’ll continue working on Clang integration in KDevelop. Stay tuned for the next blog entry about that topic :) And already a huge thank you to Aleix Pol for organizing this sprint, to Blue Systems for having us, and to the KDE e.V. for sponsoring the trip and accommodation!

» KDevelop 4.2 Beta 1 released

Fri, 12/17/2010 - 22:49

Hey all,

I’m happy to announce the immediate availability of KDevelop 4.2 Beta 1. This is a testing version and any kind of feedback would be welcome and appreciated. Please use our bugtracker. You can download the sources or wait for your distribution to create packages.

Note: Anyone who runs KDE 4.6 or later needs this version (or any later). This is because the SmartRange interface in Kate got nuked and replaced with MovingRange interface. Many thanks to David Nolden for more or less single handedly porting KDevelop to this new architecture.

The good news doesn’t end here though, KDevelop 4.2 ships with lots of notable changes and new features:

  • MovingRange` support (Thanks to David Nolden)
  • Grep Plugin rewrite with replace functionality (Thanks to Silvere Lestang, Julien Desgats, Benjamin Port)
  • QtHelp plugin now supports arbitrary .qch files (Thanks to Benjamin Port)
  • ManPage plugin (Thanks to Benjamin Port, Yannick Motta)
  • new look for code assistants (Thanks to Olivier JG)
  • filter for the project model view (Thanks to Eugene Agafonov)
  • improved “Problems” toolview, along with “TODO/FIXME” browsing (Thanks to Dmitry Risenberg)
  • better standards compliance in the C++ macro implementation (Thanks to Dmitry Risenberg)
  • argument dependent lookup for the C++ plugin (Thanks to Ciprian Ciubotariu)
  • open with external application by default (esp. useful for Qt Designer .ui files)
  • PHP: rainbow colors for all vars (and no nested functions or similar), esp. useful for scripts without Classes/Functions
  • code cleanup for working sets
  • better handling of locked sessions (locked session gets activated or the user gets asked whether he wants to remove the lock file)
  • improved Snippets editing usability

Along with these big changes, lots of small improvements have been made and we will continue to polish and harden out this branch in order to give you a stable, fast KDevelop 4.2 soon. Considering the impact and amount of changes, testing would be very welcome in order to prevent regressions and similar in the final 4.2 version.

Bye and thanks again to all contributors who made this possible. I’m especially thrilled to welcome so many new people, well done!

» KDevelop and Kate advancements over the last weeks

Sun, 11/21/2010 - 17:38

Hey all,

I didn’t blog in a long while so I thought I’d dump some notable things I did over the last weeks. It’s probably neither complete nor thorough - you should try it all out to see it for yourself :)

Kate

Lets start small with my Kate contributions. I really concentrate on KDevelop nowadays since Kate works quite well for me and I don’t have many itches to scratch anymore :) So, what did I do these last weeks? Mostly cleanup and performance work, especially regarding MovingRanges which of course is mostly needed for a good experience in the upcoming KDevelop 4.2. With the help of Dominik Haumann I also went through our indentation unit tests and made sure we expect all currently failing test cases, as well as fixing those I could. Now we (finally!) have a passing test suite and get noticed about introduced regressions. Awesome!

KDevelop

Of course I still concentrate my contributions on KDevelop and related plugins. What I find quite interesting is that I’m now for over two months in a “polish-and-bugfixing”-mode, without adding much new features. I think that shows that I’m quite satisfied with the existing features, just not with the way they are integrated. E.g. the Kate performance improvements above were done in order to make “reformat source” not hang the IDE for a couple of seconds anymore, in KDevelop 4.2 it will only be like one second for really big source files. I’ve also tackled our black sheep, the “launch configuration” dialog. I didn’t rewrite it, but I did some polishing and have some more ideas to hopefully get it into a useable state. Oh, I just remember: I also spent some time on the “Snippets” plugin, restructured the editor dialogs a bit and made them use Katepart as well, for syntax highlighting and proper editor features :)

I generally care about small things nowadays, like e.g. that all widgets have a proper tooltip. This is also something where you all could help: If you find a widget in KDevelop master (4.2) that has no tooltip, please notify me! I’ve added those e.g. to the outline and quickopen toolbar widgets among others.

And since KDevelop 4.1 was in beta stage I’ve fixed lots of bugs and continue to do that. I triage each new bug for KDevelop/KDevplatform, request feedback and valgrind logs and try to fix whatever I can. Even now I can safely assume that KDevelop 4.2 will be more stable than 4.1, even though we did lots of architectural changes in the platform, esp. regarding MovingRanges.

And now to the last and to me most pleasing part: New features! Remember how I said I didn’t do lots of feature development at all these past days? This is basically true, instead I let others do the work :) We really get more and more patches and merge requests by other people, and I did merge quite some changes already. We have an improved “find in files” plugin now, and the French students are working hard on giving it replace functionality as well! We already merged their improved QtHelp plugin that gives you the ability to integrate arbitrary .qch files into KDevelop, e.g. the KDE Api documentation. They also work on [man integration](, which will be a joy for all STD C / C++ developers.

Furthermore we now have a “Filter…” line edit above the project manager view, “Argument Dependent Lookup” for the C++ plugin, a much improved “Problems” toolview, with the ability to show TODOs etc. pp.

This is just from the last few weeks, and it’s work done by many new contributors. It’s really very cool to see this happening.

Oh, and I personally worked a bit on the QMake support for KDevelop, I’m not sure whether I will manage to get this into a usable state for KDevelop 4.2, but I already use it at work.

Which brings me to the last part of this blog post, release schedules! I’ll finally release a KDevelop 4.1.1 bug fix version next week, and will make sure that there is a 4.2 release just before / shortly after the KDE 4.6 release in January. The big “issue” there is that KDevelop 4.1 will not work with KDE 4.6+, since there are no SmartRanges in Kate from 4.6 anymore. So a big fat warning to any adventurous tester out there: If you want to try KDE 4.6 betas, remember that you’ll have to get KDevelop master (== 4.2) as well somehow!

So, that’s it for now, have a nice sunday everyone :)

» Beware of KDevelop Master and KDELibs 4.5.1 or lower

Wed, 09/08/2010 - 17:57

Hey all,

please don’t use KDevelop master with KDELibs 4.5.1 or lower. Katepart in that version misses a crucial commit that makes KDevelop crash. It is fixed for 4.5.2. In the meantime you have on of the following options:

  1. Build kate from sources
  2. Switch to the stable branches, i.e.: KDevplatform 1.1, KDevelop 4.1, PHP 1.1, …
  3. wait for the 4.5.2 release

bye

» Spotlight: Kate Scripting

Tue, 07/27/2010 - 00:51

Hey ho everyone.

Dominik asked me to blog about a feature in Kate that is still (sadly!) pretty unknown and seldom used: Kate Scripting. As you should know you can script KatePart completely via JavaScript. As those articles explain, it’s rather simple to write functions and put them into a file to have them reusable. But what for those write-use-throwaway kind of cases, where you simply need to get a job done quickly and don’t want to go through the overhead of writing some full fledged, documented, action-binded, localized script?

Utility Functions and why JavaScript rocks

Note: Neither map nor filter will be shipped with 4.5 to my knowledge, sorry about that. But you can still use the each helper (see below) to achieve the same with a bit more typing…

Take a look at utils.js on current git master: http://gitorious.org/kate/kate/blobs/master/part/script/data/utils.js

Put a special note on the helper functions map, filter and each and how they are used to implement e.g. rmblank, [rl]trim and the other functions. Cool eh? And the best part, you can reuse them directly from inside KatePart to get a job done:

mail-style quoting

Lets assume you write an email or use something like Markdown or Textile and want to quote. You’ll have to prepend a few lines with the two chars ‘> ‘. Instead of copy’n’pasting like a maniac do this instead and save yourself some breath:

  1. press F7 to open the Kate command line
  2. write e.g. map "function(l) { return '> ' + l; }"
  3. execute

Note: When you don’t have anything selected, the whole document will get “quoted”.

remove lines that match a pattern

This is something everyone needs to do sooner or later, happened quite a few times to me already. I bet vim has some esoteric command I cannot remember and emacs has something like C-x M-c M-butterfly. But with Kate most users only see search & replace and forfeit to the command line. Well, now it’s again a good time to use the command line:

  1. press F7 again
  2. write e.g. filter "function(l) { return l.indexOf('myNeedle') == -1; }"
  3. execute

Now all lines that contain ‘myNeedle’ will get removed. Sure, this is “verbose” but assuming you know JavaScript it’s actually quite easy, expendable and - best of all - good to remember. At least for me, YMMV.

shortcuts

For simple cases I’ve now introduced a shortcut way of doing the above, that saves you even more typing, but is limited to simple evaluations like the ones above. If you need something fancy, you’ll have to stick to the type-intensive way. Aynhow, here’s the shortcut version of the two scripts:

  1. map "'> ' + line"
  2. filter "line.indexOf('myNeedle') == -1"
the guts: each (interesting for users of KDE 4.x, x < 6)

Both of the above are implemented using the each helper I introduced even before KDE 4.4 afair. If you are using KDE 4.5 and want to do one of the above, a bit more typing is required:

  1. for map, you write something like this:
    each "function(lines) { return lines.map(function(l){ /** actual map code **/ }); }"
  2. for filter you do the same but replace map with filter:
    each "function(lines) { return lines.filter(function(l){ /** actual filter code **/ }); }"
Conclusion

You see, it’s quite simple and powerful. I really love map-reduce and how easy it is to use with JavaScript. Hope you like it as well.

PS: I actually think about making it yet even easier, by allowing some syntax like this: map '> ' + line or filter line.indexOf('myNeedle') == -1, must take a look on how hard it would be (beside the need for extensive documentation, but hey we have the help command in the Kate CLI, who can complain now? :) Implemented

Bye

» Kate Highlighting for QML, JavaScript

Fri, 02/26/2010 - 16:45

Hey everyone!

I’ve started my internship at KDAB this week, it’s great fun so far! Though I spent most of my time this week on Bertjans KDevelop plugin, I couldn’t resist on a bit of Kate hacking:

steveire is experimenting with QML so I couldn’t stop but notice that there is no highlighting for it in Kate. Well, there was none ;-) Now you get pretty colors, rejoice!

Note: Since QML is basically JSON with some added sugar, I reused the existing JavaScript highlighter and improved it. Hence you get imrpoved JSON and member highlighting in plain.js as well. Enjoy!

» Kate/KDevelop HackSprint - Up To Day 4

Wed, 02/17/2010 - 19:43

Woha, quite a few days flew by without me blogging about anything. Thankfully the others started to write so I don’t have to repeat it all ;-) Instead I’ll concentrate on stuff I did or learned.

GHNS for Snippets

Well, first I think an excuse is in oder: There is a GHNS button for Kate Snippets in 4.4.0 but it’s broken, neither me nor Joseph had time to acutally use and fix it… But anyways, I fixed it now for 4.4.1. For 4.5 we’ll also have an Upload Dialog.

I also added both now to KDevelop, you can now upload and download snippets from it. I added a few dump examples but will probably improve it steadily.

Kate Performance

On Saturday and Sunday I started to profile Kate highlighting for a large MySQL dump and managed to greatly improve the speed. Actually the funny thing is that I could improve RegExp based highlighting (you still should try to prevent using it, it will always be slower than simple char/string based highlighting). And the knowledge for this optimization I had from my time as an active contributor for GeSHi. I feel like it was ages ago, he funny :)

So if anybody has a big file that takes ages to load (but only if you use KDE 4.5 trunk or higher), tell me and give me the file. I might find some more ways to optimize different languages.

Other Stuff

Other than that I also managed to fix a few more bugs in Kate and KDevelop and had a good time with the other guys here in Berlin. Yesterday I showed some others how I spent quite a few nights: Partying in Berlin is nice :)

Oh and Bernhard showed me a little gem of his: Kate Standalone which you can use to build only KatePart from kdelibs trunk. This actually works very well (up until some new API from KDE trunk is used).

» Kate/KDevelop HackSprint Day 1

Sun, 02/14/2010 - 01:58

So, first day of the Kate/KDevelop hacksprint.

We just talked and hacked at the rented flat,got to know each other and had a fun time. Everybody made it more or less in time, even last minute attendee Adymo from Ukraine, nice! Hacking-wise the productivity wasn’t that high, esp. for me, but a few patches got committed here and there.

Right now I’m working on a little speedup for Kate, esp. for big MySQL files - lets see how it turns out. Cullmann showed me a few things I could do so maybe it works out, lets see.

Over the next week I plan to push in user configurable include paths for the PHP plugin and do some more Snippets & Scripting work in Kate, lets see how it turns out. I’ll go home now, kinda sucks that I don’t stay with the others here at the flat but have to take a 1h ride into the city… Berlin is definitely too big :D

» Snippets In KDevelop / Kate

Wed, 02/03/2010 - 17:59

Hey all!

Just wanted to give you a little rundown on Snippets in Kate 4.4 (via the snippets_tng plugin) and KDevelop Beta 8 (soon to be released).

Note: The Kate plugin was written by Jowenn and introduced me to all these nice features. For KDevelop I wrote a somewhat simpler yet imo better implementation. We will try to get the best of both worlds into KDE 4.5. Stay tuned!

General Usage & Features
  • create a snippet repository (or download via GHNS [see below])
  • create snippets in that repository
  • insert snippets via the snippets view (i.e. double click), or (imo better/faster) insert them via code-completion (remember: CTRL + Space requests code completion at the current cursor position!).
  • snippet gets inserted (properly indented) and potential placeholders/variables get expanded. A variable is something like %{date} or ${email}. Also take a look at the API documentation.
  • variables that get inserted via “${…}” will be “selectable”, meaning you can jump from one var to the other by hitting TAB / Shift TAB
  • the %{...} vars will only get expanded and inserted, without getting selectable.
  • multiple occurrences of the same variable will be updated once one of them gets edited, something that is called “mirroring” in other editors.
  • once one edits ESC the cursor is placed at the end of the snippet or to the first occurrence of ${cursor} or %{cursor} and the user types something, the snippet-handler quits and you are left with your normal editor until you insert the next snippet
  • nested snippets (i.e. insert snippet than insert another one) should “just work”.
Snippet Management
  • group snippets by file type, i.e. PHP snippets will only be offered during code completion when one edits a PHP file.
    Note: In KDevelop and KDE 4.4 nested documents are supported, e.g. create a CSS snippet and it will be shown inside the CSS parts of a HTML document or similar. This uses my HighlightInterface I wrote for KDE 4.4. I still have to rewrite some parts of the snippets_tng plugin for Kate so that it works there as well
  • group snippets in repositories, set an Author and a License of your choice
  • publish snippet repositories via GHNS: In Kate you can already download snippets from GHNS but we sadly don’t have any repos up on opendesktop… I’ll have to add some prior to KDE 4.4. Also we didn’t have enough time to implement uploading of Repos from inside Kate in time for KDE 4.4. So stay tuned for KDE 4.5. KDevelop currently has no support for GHNS, but I plan to fix this tomorrow or the next days - together with uploading from inside KDevelop, i.e. all the nice features of GHNS v3.
  • in KDevelop (and someday in Kate as well) you can simply select a part of your currently opened document and select the “create snippet from selection” in the ContextMenu - easy & fast!
TODO

There’s much to do.

  • Highest priority right now for me is to get GHNS with all bells and whistles supported for KDevelop.
  • Then I’ll merge and integrate the Kate & KDevelop plugins as much as possible, so we have a reduce code base.
  • Also important is to make all shortcuts configurable
  • Another thing is: How could we improve interoperability even between e.g. editors like TextMate or Gedit? Both have snippets features and their bundles are available in the net. If we can support those we’d save us a lot of work

Also, I should probably do a screencast… Not now though ;-)

» Kate polishing

Thu, 01/21/2010 - 02:09

Phew, I just finished some last-minute backports to the KDE 4.3.5 branch. Lets hope the bug fixes I and pletourn did are as good as they look. Expect a much more stable Kate for 4.3.5 & 4.4! We managed to fix two bugs which are potentially the cause for dozens of bug reports, all seemingly random. Lets see whether our fixes hold up to our hopes!

Other than that: You should look forward to Kate scripting (with JavaScript) in 4.4. It’s dead simple but actually useful. In the utils.js file we ship with Kate there are now the following tools (all operate on the selection or - if none exists - on the whole document):

  • sort - simple sorting
  • natsort - natural sorting
  • uniq - filter duplicates
  • trim - remove leading & trailing whitespace
  • ltrim - remove leading whitespace
  • rtrim - remove trailing whitespace

Do you have more ideas for such simple helper functions?