Kasperian Moving Parts

kinda like Batman, but with a wife and 3 kids

Monday June 8, 2009
by Jason 'vanRijn' Kasper

I Can Haz a Palm Pre AND IT IS AWESOME!

Woot, I got one! I’ll post again once I get the pictures off of the camera, but I got to the local Sprint store at 7:00 (AM!!!!) this last Saturday, waited for an hour, and managed to snarf up 2 Palm Pres– one each for my beautiful bride and I.

I realize I need to put up something more deep than “WOW! AWESOME! OUTSTANDING! I LOVE IT!”, but honestly, that’s about all I can muster right now. This is definitely the device I’ve been waiting for the last 10 years. Amazing interface. Excellent graphics. Beautiful applications. Perfect synchronization with my Google information (it does desktop syncing too, iirc, but that’s so early 90’s!). REALLY fast USB data transfer (like, I’ve never seen anything this fast before). The Touchstone charging thingey is mind-blowingly cool and simple and elegant (induction charging == no wires for charging!). Music and Video playing is dead simple and being able to transfer my music/videos from Linux is a must-have (sorry, Apple, you intentionally suck at this, so go away). In a word: perfect. Seriously.

The apps I’ve been using thus far are Pandora, Accu-Weather, Tweed, YouTube, and of course all the built-in goodness (calendar, contacts, mail, clock, calculator, google maps, GPS navigation, etc.), and I am 100% impressed and satisfied. The WebKit-based Pre browser does an extremely nice job, and the multi-touch zooming, auto-screen rotation, and scrolling is beautiful. Though Adobe hasn’t seen fit to provide their proprietary Flash browser plugin for the Pre yet, the Palm guys have done a really nice job at hiding that fact, from what I’ve seen so far (open the browser and go to youtube.com, for instance, and click on a video and it launches the video in a separate application). I believe I’ve heard that Adobe/Palm have said that Flash will be available for the Pre before the year’s end, so it may be a non-issue soon enough.

One final thought I had while playing with the Pre for the last couple of days…. As I mentioned previously, I’m a gadget geek, and have been using Palm devices for longer than I’d care to remember. People have made a big deal about Palm’s demise and how bad they have been doing and how it seems like they’ve been intentionally trying to drive people away from them and make themselves go out of business. And all of that seems true, especially having been one of the few, ashamed, die-hard Palm geeks myself. And while the Palm 5 OS seems gosh-darned ancient and ugly and ridiculous, especially compared to the sexiness of the iPhone, Android, and now especially their new WebOS, I think it’s important to remember the fact that for a LONG while, Palm was truly the innovator of the handheld/PDA world. Unfortunately, Palm has had a rough ride in recent years and hasn’t been able to kill off the old Palm OS (and don’t think they haven’t tried!) with anything revolutionary… until now.

As I was playing with my new Palm Pre, it struck me that a lot of the innovations that you see on it and similar devices these days were first done by Palm. Flip the screen sideways and go from portrait to landscape? Palm did that YEARS ago. Of course, the accelerometer wasn’t around then to make it automatic, and PDA hardware couldn’t do the fancy and smooth transition animations, but still, Palm had that idea a while ago and implemented it nicely. Multi-layer launchers? Yep, Palm did that a long time ago. Web browser on your PDA? Yep, Palm did that too.

My point is this: it seems that people are hesitant to think that Palm could actually be breathing new life again… that they’re tired of waiting for Palm to get their act together and do something meaningful… that they’re just copying the iPhone and the Android. Well, as one who was thinking exactly these things before I got my hands on the Palm Pre a couple of days ago, I can tell you that Palm is breathing new life again, that they have finally been able to rise above the old Palm 5 OS, that they have definitely gotten their act together, and most importantly, they have done something extremely meaningful with the Palm Pre. And in retrospect, I don’t think we should be surprised at this. Palm has been innovating handlheld computing for longer than anyone else, and with the Palm Pre, I think they have shown that they are still REALLY good at it.

I have been waiting for a long time for “the one device to rule them all”–for one device that intentionally plays nicely with whatever operating system I want to use, that handles synchronization with my calendar, contacts, and email flawlessly, that has a built-in GPS, that has a good built-in music player, a good built-in movie player, has built-in internet access, a good web browser, is extensible via apps, and has a good screen. I am really excited with the Palm Pre because it is finally “the one device to rule them all,” and it does it REALLY well. The GPS is awesome (better than that on the Nokia N810), by the way, and the music and movie players are gorgeous. And everything you’ve heard about the responsiveness, multi-tasking, beautiful screen, and nice little details are true. =:)

Anyway, that’s about all the words I can muster. I’m very happy with the Palm Pre, and feel like even though I “missed out” of being a gadget geek for the last couple of years with the iPhone, the wait has been worth it. =:) I really think Palm has done something special here with the Pre, and I wish them the best of luck. Being that every store in the state of Massachusetts was completely sold out of Palm Pre’s and their new Touchstone chargers within 4 hours of opening, I have a hunch that they’ve knocked this one out of the park. Well done, Palm!

Thursday May 28, 2009
by Jason 'vanRijn' Kasper

The Palm Is Dead. Long Live The Palm!

KDE and Qt Developers Meet Android

I believe I am one of the last few die-hard nutjobs on the face of this earth who still use (and “use” here is a highly subjective word meaning that I have a bunch of Palm devices lying around, am currently the only semi-active (and “semi-active” means that I get probably a good 2 hours of KPilot hacking in per year =:( ) KPilot developer, and occasionally even turn some of them on) Palm PDA devices. I have successfully resisted the siren call of the iPhone for the last 2+(?) years–partly because there is no functional synchronization solution between my Linux desktop and it, partly because it’s pretty bloody expensive, partly because Cingular has atrociously high data plans compared to Sprint, partly because I’ve endured the lunacy of FLOSS developers trying to keep re-figuring out Apple’s iPod/iTouch/iPhone database structures that would otherwise allow me to synchronize my music and movies with said Apple devices and have an extremely bad taste in my mouth from said frustrations, and partly because I’m one of the cheapest geeks you’ll ever meet (also, being the sole income-provider for a family of 5 only solidifies my inborn cheap nature). All that being said, however, I hereby declare the good old Palm OS officially dead and uninteresting to me anymore. Okay, truth be told, that was an obvious statement to make 2 years ago, but I’ve been in denial since then and am only now trying to face reality and get help. =;P

I am a gadget geek–I always have been–and I have wasted more money on Palm gadgets than I care to remember. I clearly remember agonizing over spending $400 or so for the Palm IIIc when it came out (but OOH, it had a nice color screen!). And the $400 or so I spent on the Clie NX70v was a week-long ordeal that involved me hemming and hawing and spending many an angst-filled evening at the local Circuit City. And the $400 or so I spent on my Treo 650 (which magically turned into a Treo 700p in a couple of years after the 650 became deathly ill) was also quite the emotional ordeal. And yes, I realize that these series of purchases contradict my statement that I’m a cheap geek, so I’ll defend my previous statement by saying that I’m apparently a selectively cheap geek.

Palm was a GREAT gadget and a good OS that allowed me to sync my data with my Linux desktop and enjoy being cool and geeky. In fact, it was (and still is) the only PDA solution that I have found that synchronizes (for the most part) very smoothly with my  Linux desktop. It was never as flashy as the Windows-based devices, but it sure was more stable. And there were a huge number of applications for the Palm OS. But seeing the spartan Palm OS 5 interface nowadays, especially when compared with the iPhone bling, or even the Maemo interface… it’s like looking at the old OLWM Window Manager compared with the current KDE4 sexiness. There’s just no comparison. Unless you’ve been living under a rock for the last 2 years (or have been cheap and/or in denial like me and/or just so in love with the old Palm OS), it’s painfully obvious that very few care about the old Palm OS anymore. Everybody and their pet turtle has an iPhone now (or so it surely seems). And being a FLOSS advocate/hacker/supporter/proponent/religious nutjob, that concerns me and I’d like to again put my money where my mouth and soapbox are.

So what’s my point with all of this? Well, it’s time for me to get a new phone/geek toy, I think. I want to be as FLOSS-supportive and interopable as possible, and I’m really curious what other people who are FLOSS-conscious are thinking about this and have done about it. While none of the options that I see are 100% FLOSS-perfect (being that we’re still dealing with proprietary bits/pieces/networks/hardware with cell phone companies), Android seems the closest, while the Palm Pre (assuming it runs on Linux and allows itself to be open enough to be hackable/customizable/extensible) seems a strong second, whilst the “what do you mean you don’t have an iPhone yet” seems a distant third, being that you’re totally under Apple’s friendly-dictatorship-and-heavily-taxed thumb.

Here’s my short list so far, with my take on positives/negatives. I’m very curious to see what people (especially Planet KDE people who are actively working on providing/improving/supporting FLOSS) have done and are thinking with regards to their cell phones.

  • The Apple iPhone. The current definition of sexiness–just ask the entire planet. Unfortunately, from my understanding, it’s very tightly controlled by Apple, and while you can jailbreak it, you’re still under Apple’s thumb as far as transferring music/videos (at the very least) to it from a Linux desktop. Want to back it up? You’d better have either a physical Windows or Mac machine or a very good virtual machine provider like VMware Workstation or Player and cross your fingers a lot. The other major downside in my book is that you can only get an iPhone if you use AT&T/Cingular as your cell phone provider, and their data plans are the highest in the industry ($30 per phone–and that’s if you don’t want to be able to connect to it via Bluetooth for laptop internet accesss???). Also, it doesn’t look like they have a shareable family data plan, so I’d be looking at $90 per month, at least, for just data access?? And then there’s the fact that it’s the absolutely least FLOSS-friendly geek toy of the bunch, from my understanding. There are a lot of positives, of course, and there seems to be no shortage of 3rd party application developers and applications. Of course, if you want to develop for the iPhone, don’t you have to pay the Apple tax and buy a physical Apple computer as well??
  • Android phones. Really slick and awesome looking! I would LOVE to just have the PIM applications from it be able to run on my Nokia N810 and be able to sync flawlessly with my Google calendar/contacts data! The biggest downside for the Android for me is that only T-Mobile has an Android phone–and I can’t get T-Mobile service in my neck of the woods (literally). I’d really like to leave myself as open as possible to being able to get an Android phone as quickly as possible, so I’m thinking that I’d like to sign up with whichever cell phone service provider will give me the best shot at that. I think that because Google is backing this platform, it has the second-best chance of attracting application developers/hackers and should mean that it’s pretty future-proof from the standpoint of being able to look forward to years ahead of good, solid applications for the Android platform. Is it reasonable to think that even if cell phone companies don’t sell Android phones themselves, that one would be able to pick up an HTC Touch or Diamond or similar and slap the Android OS on it and have a fully functional Android phone?
  • And finally, the soon-to-be-released Palm Pre (and here we tie in nicely with the title… the Palm is dead! Long live the Palm!) I cannot find a whole lot of information about the Palm Pre right now, but what little I see looks good. Based on Linux(?), supports a bunch of audio/video codecs out of the box, sports a slick new interface that looks very much like the iPhone/Android UIs, has a built-in GPS, and is being aimed squarely at the iPhone/Blackberry camps. The things that concern me: it has a custom web browser (why didn’t they use one of the existing FLOSS browsers???), lack of information regarding add-on external storage (does it use microSD?), will it support Bluetooth tethering/DUN(?), and the fact that this is yet another new platform that requires a healthy influx of 3rd-party app developers/hackers. Can Palm pull in a huge number of app developers to breathe life into the Pre and its new WebOS? To me, that’s the biggest question, since if they cannot, I don’t think they can stem the tide of iPhone-exclusive applications and developers. On the positive side, my current cell phone provider (Sprint) will be offering the Pre in another week or so, and they have pretty attractive data plans.

So I’d love to get comment feedback from folks about this. What are you currently using if you’re using one of these solutions? What are you planning on doing going forward?

Saturday April 4, 2009
by Jason 'vanRijn' Kasper

Need New Linux/KDE4-friendly Laptop

Dear Lazyweb,

I need to buy a laptop for personal use. I’ve been using Thinkpads as work laptops for so long that I think I’d like to try something else out for a change. It needs to be Dual Core/Core 2 Duo/whatever. Would be nice if it had a fast 200+GB drive in it. And it absolutely MUST have a fast, awesome graphics card in it that has zero problems with compositing, Linux, KDE4, suspend/resume, or anything else. Having dealt with nVidia cards for quite a while now, I’m guessing this means that the new laptop shouldn’t have an nVidia graphics card in it. I am so tired of the constant problems I have with KDE4 and the nVidia Quadro NVS 140M I have on my work Thinkpad T61–can’t use compositing for more than a day before the system becomes totally unstable and invariably X crashes, etc. =:( Maybe an Intel or ATI card? Also, it would be really groovy if the battery lasted longer than 3 hours, consistently.

Anyone have any suggestions? What have people had good success with in a laptop, been able to do compositing and full desktop effects in KDE4 without having any problems or system instability, etc., etc.? I was thinking of maybe trying a new MacBook, but having tried that before and absolutely hated the keyboard (wth, Steve, no home/end/page-up/page-down keys???), I’m not sure how long I’d last on it before pounding my forehead into it. And that’s about where my list of ideas ends. Any suggestions would be most appreciated.

Tuesday March 3, 2009
by Jason 'vanRijn' Kasper

A Tale of a Flat Tire and God’s Goodness

I pray a lot for my kids. I always have. I ask God to protect them, shield them, guide them, and watch over them when I cannot. Watching your oldest child reach 18 years of age, start college, start driving, and be out of your line of sight and protection for large parts of the day make you pray even more. And so it is that I missed God’s blessing entirely and went straight to angry, pissed off, frustrated, groaning, belly-aching, and in general no fun to be around this morning when my darling bride and I drove both cars down the driveway to get the annual vehicle inspections done.

Thankfully, I noticed the flat tire about 30 yards down the road from our driveway, so I pulled over, checked it, (went straight into pissed-off-mode), and turned around and went back home. It’s snowy, slushy, icey, and cold right now in Douglas, MA, so it was all that I could do to barely get the car into the bottom of the driveway. I’ve never had to drive on a flat tire before, but this time I had no choice. I’ve heard all kinds of bad things can happen as a result, and I was most worried about ruining the wheel or axle or something. So I pull into the driveway, open the trunk, and discover that the trunk has been leaking water again (or allowing it in through the bottom… I don’t know which), and now that it’s winter in MA, it has frozen solid inside the trunk. The trunk wheel well was about half full of frozen water, surrounding, enclosing, and including the spare tire. An hour and a half later, after failed attempts with a heat gun and a steam cleaner, I used a 20-pound steel pry bar to pry the spare tire loose from the trunk. I now have a Flinstone-car-looking ice block that looks like a tire. Another 15 minutes with the steam cleaner to get spare tire free of the ice, and I put the spare on and inflated it.

Now we’re on our way to the inspection station. Turns out that (I think) the reason the tire was flat is because it was so under-inflated that the bead seal just failed and all the air leaked out. However, being that I drove on it for a couple of minutes while it was flat, the tire is now unusable (steel belts destroyed and air bubbles are now in the sidewall). $70 later for a replacement tire, and a Russian immigrant mechanic to tell me “your tire no good”, and we’re on our way again.

And then it hit me. This was what I asked God for. This is what I’ve been praying for. Not necessarily to have a flat tire, ruin an whole day of work, not be able to get the spare tire out because the entire trunk is a big ice cube, and to waste $70 on a replacement tire. No, I distinctly do not remember asking God for that. But I’ve been praying for my kids’ safety for their whole lives, and that is exactly what God did for me. If this would not have happened to me today, it would have happened to my eldest daughter tomorrow, who likely would not have known to check the tire because it felt smushy, and who very well may have gotten into a really bad car accident, or been stranded, or worse, as a result. Thank God that this happened to me today instead of her, tomorrow. And as I was moaning, griping, cursing under my breath, freezing, frustrated, and angry, it occurred to me that God was being extremely gracious to me and my family once again. Who am I to question how God answers my prayers and keeps my family safe?

Thank you, God. And I’m sorry for going straight to anger, frustration, and pissiness and missing your blessings.

Oh, and you cannot tell me that God does not have a sense of humor, after a day like this. =:P

[Update] Wow, people can be incredibly rude, insensitive, and insulting. I’ve deleted the idiotic vitriole that found its way to this posts’s comments, and have turned further comments off. Thanks, idiots!

[Update2] I’ve decided to turn comments back on here. Unfortunately, I deleted the previous hateful, rude, insensitive, insulting comments instead of moderating them, so I am unable to bring them back for your enjoyment.

Thursday February 12, 2009
by Jason 'vanRijn' Kasper

My Childhood Guinea Pig Named Squeaker

<kergoth> anyone messed with protobuf?
<vanRijn> kergoth: i once had a hamster i named protobuf, does that help?
<kergoth> fraid not
<vanRijn> actually, it was a guinea pig
<vanRijn> and it was named Squeaker
<vanRijn> but still
<darth_mall> vanRijn: that’s practically the same thing!
<kergoth> hehe
<vanRijn> darth_mall: I KNOW RIGHT!

I think I’m getting too in touch with my emotions as of late. I actually got a little teared up, thinking about my cute little childhood Guinea Pig named  Squeaker and his untimely demise.


Wednesday February 11, 2009
by Jason 'vanRijn' Kasper

Screencasting in Linux!

I’m excited. I love learning stuff, I really do. I just did  a 12-minute screencast for work, and I think I’ve finally figured out how to get everything to fit together nicely. I’d never done a screencast before–not on any platform–but being that I needed to showcase some development work that I’ve done for the next release of VMware Workstation/Player, and being that I’m working from home for the time being, I needed to get this all working in Linux, and as I said, I think I’ve finally figured it out, woot!

For starters, I used qt-recordMyDesktop to capture the full-screen (1600×1200 resolution) video. I wanted to use it to also capture the audio portion of the screencast at the same time, but when I tried doing so, the audio was really choppy and out of sync. I mostly blame pulseaudio, but also the fact that I did this all on my puny little laptop, and I think that the system just wasn’t able to keep up with me, recording a nested Xephyr session with 4 fake Xinerama monitors (thanks again for that beauty, Lubos!), at 1600×1200 resolution. So I told qt-RecordMyDesktop to not capture audio and what I ended up with was a beautiful 1600×1200 Ogg/Theora .ogv file. We’ll call it demo-video.ogv.

Next, I recorded my voice, doing a monologue of what was happening in the screencast, using my laptop’s internal mic (not the greatest quality, but I don’t have a real microphone, *sigh*), and audacity (oh, and this is nice… audacity doesn’t work with pulseaudio whatsoever). This I saved in mp3 format. We’ll call it demo-audio.mp3.

The next magical trick, obviously, would be to combine the audio and video files into a single movie file, right? Well, all of the questions/answers that Google found me (even though I searched for “mencoder combine audio video”) were examples using ffmpeg. So I gave it a shot. And I’m sure there must be a way to do it, but for the life of me, I couldn’t get ffmpeg to combine my 80-meg demo-video.ogv file and my 10-meg demo-audio.mp3 file in a high quality and problem-free output file. The closest I think I got was this: “ffmpeg -sameq -i demo-video.ogv -i demo-audio.mp3 demo_full.mp4”, but that combined my 80-meg video and 10-meg audio file into a 350-meg mp4 file. Zoinks, Shaggie!! That’ll never do!

I finally stumbled upon the “-audiofile” parameter to mencoder and there was much rejoicing in Agrabah (not to mention Massachusetts). What I ended up with is this little mencoder incantation that seems to work beautifully. And, the resultant file is only 62 megs (80m + 10m == 62m !?!), so I’m sure there’s some loss of quality in there somewhere, but for the life of me, I can’t see it. Here’s what I used:

mencoder -sws 9 -vf pullup,softskip,scale=1600:1200,harddup,unsharp=l3x3:0.7 -oac faac -faacopts br=128:mpeg=4:object=2:raw -channels 2 -srate 48000 -ovc lavc -lavcopts aglobal=1:vglobal=1:vcodec=mpeg4:acodec=libfaac:abitrate=128:vbitrate=1000 -of lavf demo-video.ogv -audiofile demo-audio.mp3 -o demo_full.mp4

So, there you have it. Screencasting, done 100% in Linux. I wish I could show you the results, because I’m pretty darned please with them, but sadly, I cannot (nor do I have a spot to stick 62 megs of mp4 =;P).

I hope this helps some other poor soul, ’cause I couldn’t find much in the way of tutorials for doing this. I’d be very interested to hear what others think of this, as well as any other suggestions for doing screencasting in Linux. I know Aaron’s been doing something along these lines, and I’d be curious to see how this compares to his method. Also, any improvements to my mencoder line (yeah, I’m sure some stuff in there might be redundant or weird), or finding out what the ffmpeg equivalent of my mencoder line is would be greatly appreciated.

Tuesday January 27, 2009
by Jason 'vanRijn' Kasper
1 Comment

KDE 4.2 Released!!

Woohoo! KDE 4.2 is released!! I only wish the last few KPilot bug fixes would have made it into the 4.2.0 release, but we were too late. Still, if you think you knew what KDE4 was all about, think again. Check out KDE 4.2.0. =:)

Monday January 19, 2009
by Jason 'vanRijn' Kasper

KDE 4.2 KPilot coming! (how to both be excited and have realistic expectations)

I just sent this to the KDE PIM mailing lists, but I know not everyone who uses KPilot subscribes, so I’ll re-post it here…

Howdy all,

I just wanted to get a note out to as wide a distribution list as possible to spread some important news about the upcoming KDE 4.2 release and KPilot’s (exciting!!) part in it. If you don’t care about KDE PIM, data syncing, Palm devices, or KPilot, you may stop reading now and I won’t be offended. =;P

For the last 2 years, the talented Bertjan Broeksema and I have spent our Google Summer of Code months doing some major rework and redesign for KPilot. You can see our sync algorithm redesign Use Case here: http://snurl.com/acftl and some UML sequence and class diagrams that we used here: http://snurl.com/acfwd. Previous to our redesign, each conduit in KPilot contained all of the logic necessary for the syncing of record-based data, as well as the transformation, comparison, and resolution of that data. This means that each conduit did things its own way. We didn’t share any code between the conduits for syncing, and we weren’t consistent about how the data was synced. In addition, NONE of our conduits had the ability to synchronize category information from PC to Palm. We had inconsistent smatterings of success going from Palm to PC, but again, we were inconsistent and buggy. And obviously with this duplication of work, whenever a bug popped up in one conduit, it likely existed and had to be fixed (and most probably did not get fixed) in the other conduits. In addition, we did a very poor job of ensuring that we didn’t lose our users’ data, and subsequently, we had a bunch of “KPilot ate my future” e-mails (http://snurl.com/acgf0) that nobody was happy about.

So one of our most important goals in our KPilot redesign was the safety of our users’ data. We now go to great lengths at the beginning and end of the sync process to make sure that our record counts match between Palm and PC, and that we’re able to save all of our data successfully. If anything breaks in the process, we do not commit or save our changes to the Palm and PC sides.

We also didn’t want to keep writing and maintaining “how to sync” logic in our conduits anymore. This was a major design flaw previously and led to a lot of rework and problems. So we designed a record-based (since all of our syncing is from record-based sources) syncing conduit set of base classes that all of our new conduits inherit from. This means that now all of our conduits use the same syncing logic and flow. If we got it wrong, we fix it and now all of our conduits are fixed. If we need to improve the performance and efficiency (which we do, still), we fix it in one place and now it’s fixed and improved in all of our conduits. This is a much cleaner design and it means that our implementing conduit classes are now MUCH simpler, have MUCH less code in them, are MUCH easier to understand, and all follow the same patterns, meaning the learning curve for debugging, writing and contributing conduit code just became darned near close to flat. The job of our conduits is reduced to data transformation, comparison, and record domain knowledge, as it should be. The syncing logic all happens in the base classes.

Another nice feature that we added, although it’s not completely finished, is a final “volatility” check in the base conduit classes. At the end of the sync, we look at how many records we have created/updated/deleted in comparison to starting and ending counts, and if the changes are too volatile, we’ll ask our users whether they want us to save the sync or undo it.

Anyone familiar with programming in professional environments should be familiar with the idea of commit and rollback. Well, previous to KDE 4.2’s KPilot, we had one mode: change live data on the fly and don’t bother with commit/rollback. This made things nice and fast, to be sure, but we also lost a lot of data that we shouldn’t have. In fact, we didn’t even have a way to not lose data, previously. So this was another important design decision we made in the new version of KPilot. We do all of our syncing work in temporary storage and only at the end of a successful sync process do we commit (or rollback) our changes to the real (PC and Handheld) data stores.

We also have a nice new Keyring (http://gnukeyring.sourceforge.net/) conduit for KPilot which uses our new base syncing classes.  Unfortunately, we didn’t get a chance to finish polishing it off, and thus it will most likely be disabled for the final KDE 4.2 release. We’d love your help in getting it finished, though. Bertjan has even written a nice Qt4-based PC Keyring data file viewer/editor that we’d love to get finished, polished, and released. Please help!

So there’s a LOT of coolness in the upcoming KDE 4.2 version of KPilot. Not to mention the fact that it’s the first KDE4 version of KPilot. But I also want to set realistic expectations so that we’re all on the same level.

This will be the first release of a major rewrite of KPilot, and we have had very little user feedback thus far (although there have been a few very helpful folks and I really appreciate them!!!). This means that there are guaranteed to be as-yet-undiscovered bugs. I have been doing a bunch of testing and bug fixing, even since the last RC1 release of KDE 4.2, but there’s just no way I can hit all of the use cases we have. Things are looking extremely smooth and stable and I have not once lost data (yay!), but I’m sure bugs are in there, so make sure you make and keep a good backup of your Palm and PC data before trying new software.

Previous to KDE 4.2, KPilot relied on the external data sources to know about the condition of that data source’s records and PC->Palm  mappings, etc. With KDE 4.2, KPilot now keeps this information itself. This is a good thing! We had a bunch of problems in KDE3 which were caused by the mapping being wrong and/or the state being wrong (X-PILOTID and X-PILOTSTAT).  So we have our own XML-based mapping files now that are kept in $KDEDIR/apps/kpilot/conduits/<

username>/mapping/ and these help us to keep track of the mappings as well as per-conduit last-synced times, etc. We currently rely on this mapping to be valid at the start of a sync (which is also a good thing), but which has one unfortunate side effect that we’ll most probably not be able to get fixed before 4.2 is release: CopyHHtoPC and CopyPCtoHH don’t work as advertised right now. This also means that the first time you sync with the new KDE 4.2 KPilot, we will do a “first sync”, regardless of whether you’ve already synced with KPilot so that we can correctly establish this mapping file, and you’ll end up with a combination of PC and Palm data on both sides at the end.

We’re using Akonadi for syncing your data in KDE 4.2! This is also really good goodness! This will allow us (eventually, once the Akonadi back ends are written) to sync your Palm pilot with an Exchange data source, or a Google calendar, etc., etc. The realistic immediate view, though, is that KPilot is using and relying on a really new Akonadi solution. KDE PIM 4.2 is still not fully ported to using Akonadi purely, although we have bridge resources as an interim solution. This means that before you can sync with KDE’s 4.2 KPilot, you need to add an Akonadi resource for each data type (Calendar, Contacts, etc.), then change your KDE PIM applications to use that (korganizer, kaddressbook, etc.), and then reconfigure KPilot’s conduits to use those new Akonadi data sources. There are glitches to be found in all these layers, to be sure!!  In my testing, it looks like you have to hit reload in korganizer to see kpilot changes to akonadi’s cache after a HotSync, for example. Also, it’s a manual process to convert your PIM data sources to Akonadi resources so KPilot can sync with them. This MUST be done, since KPilot now syncs only with Akonadi data sources. Together, we’ll be finding (hopefully just a few) bugs in KPilot, the new KDE PIM code, and Akonadi. The pay-offs for all of this work are immense, but we need to have realistic expectations about it too.

We have a serious developer shortage in KPilot–both real-person-resource as well as available-time-resource–and as a result, we were only able to bring so much forward from KDE3’s KPilot into KDE 4.2. We tried to focus on the most widely used functionality and, to be honest, that which we the KPilot developers actually use. This means that KDE 4.2’s KPilot will not have: the Knotes conduit (which was badly broken anyway), the DOC conduit (which I’d love to get working again but we’ll need someone to help code it to get it done), the Avantgo/MAL conduit (does _anyone_ actually use Avantgo anymore???), the popmail conduit, or the notepad conduit. So, if you absolutely have to have one of those, please continue to use the KDE3 KPilot, or better yet, help us port your favorite conduit to KDE4!!! What we do support in KDE 4.2’s KPilot is: Calendar conduit (syncs against Akonadi), Contacts conduit (syncs against Akonadi), Memofile conduit (syncs to a directory/files on your hard drive), the Time conduit, the Palm database install conduit, and the ToDo conduit (syncs against Akonadi). These are by far the biggest heavy hitters (the 4 main Palm apps), but again, if you absolutely must have one of the others (or a new conduit altogether), please help us make it a reality by contributing.

Our little First Time Setup Wizard’s device detection functionality is broken.  Actually, it’s been broken for a while. But we didn’t have time to try to get it working, so we disabled that functionality. Really, this is ONLY a one-time setup issue, and there are far better ways to figure out what connection string you should be using, and they’re pretty standard now (“/dev/ttyUSB1” if you’re using the visor kernel module, “usb:” for libusb syncing, /dev/ttyS* for serial devices, etc.).

Previously, KPilot tried to be a “anything that has to do with a Palm device” solution, and we had built-in viewers and editors for Palm databases. These were buggy and error-prone, and confusing, since it looked like KPilot could sync with your Palm, KDE PIM apps, and any changes you made to the databases through the viewers/editors. Well, no more. We’ve removed these viewers and editors to fix a bunch of bugs, to cut down on the porting and maintenance efforts, and to provide a cleaner user interface. KPilot is now focused on being a syncing solution between your Palm and KDE’s PIM back ends. If you need a generic palm database viewer/editor solution, JPilot is much-better suited to that.

Unfortunately, using “usb:” (libusb) to sync your USB palm still has some weirdness to it. I have not had time to dig until I found the solution, but even though we do the exact same thing with “usb:” devices through pilot-link that we do with old-style USB devices (/dev/ttyUSB1, etc.), we periodically hit a crash when closing libusb sockets. Sometimes this shows up as a double-free bug down inside libusb, and sometimes there’s other weirdness. Bottom line is this: “usb:” should be fine for syncing your Palm with KPilot, but you may see crashes after your data has been synced or when KPilot closes the socket handles to libusb (pi_close, in paricular, causes this). This is an area I’d REALLY like some help with, since I don’t see anything that KPilot is doing wrong.

One of the challenges we’ve always had is in getting our users to help us understand what’s going wrong when something goes wrong. In the past, this has meant that we’ve had to tell them to get KPilot’s source code, reconfigure it to make sure debugging is turned on, recompile, re-install, and test again before we could get ANY useful information for as to what might be going wrong. This presented an insurmountable barrier to entry for 99% of our users and has resulted in a lot of bugs never getting fixed and a lot of frustration and wasted time, all around. Well, no more. In KDE 4.2, KPilot now should have debugging (with tunable output levels) turned on by default. The positive to this is that we should be able to ask our users to just run their already-installed version of “kpilotDaemon –debug=9” and send us the output for us to be able to get a much better look at what might be going wrong inside the wee beastie. The down-side is that we will be producing more console spew as a result. I’ve tried to keep this to a minimum, but I’m sure this will upset some people anyway. For those folks: workaround would be to run “kpilotDaemon >/dev/null 2>&1”. We can definitely improve in this area and prune down what gets printed by the default debug level, and I’d greatly appreciate any patches to help do so.

And lastly, as I mentioned previously, we made some intentional design decisions that favored the protection of our users’ data first and foremost. As a result, some things will take longer in KDE 4.2’s KPilot than they used to in KDE3. Most notably in this area is our “first sync”, where we take in both collections of data (Palm and PC) and match them up and result in a combined set. This area has not been performance-tuned (we’re currently at an operational efficiency of O(n^2) in our data matching code, I believe, which is obviously bad), and there are definitely things we need to improve on (and would REALLY appreciate the community’s help on!!!). If you have a database with thousands of records, then “first sync” (which happens obviously the first time, but also if the record mappings are invalid) will take a while and your Palm may time out. Less than optimal? Yes. Slower? Maybe, the first time. You lose data? Definitely not. Can you help make it better? YES! =:)

So, in closing, I hope this helps to get all you Palm users excited about our new version of KPilot. Never before has KPilot looked so good, been as careful with your data, or been easier to help debug, extend, and help out with. There will be some challenges, as there are with any new major version of software (and this is a doozie, all around!!!). But I’m really pleased with what we have and where we can go from here.

Thanks, enjoy, and please give us feedback!! As always, bugs should be reported to bugs.kde.org, e-mails should be sent to kdepim-users@kde.org, and more-or-less real-time help can be found in #kpilot on FreeNode IRC.=:)

Monday January 19, 2009
by Jason 'vanRijn' Kasper

KPilot 4.2 Progress (woot!)

I spent a crapload of time this weekend, going through all the old and crufty KPilot bugs we’ve done a really horrible job of keeping up-to-date on, and triaged the bejeebers out of the list. I think we had ~ 150+ a few months ago. I went through the last 100 of them individually today, and was able to close out 93 of them, woot! A lot of them were problems that had been fixed in KDE 3.5.x, or had been directly addressed  more recently in our KDE 4.2 work, or had been indirectly fixed via our KDE 4.2 work. A lot of them were also the low-hanging fruit you’d expect with “it don’t work so good” and not much more to go on. But all told, we now have a very clean KPilot bugzilla bucket, with 12 open bugs (of which, 10 are wishlist items). We’re all set for the next onslaught of bugs from the upcoming KDE 4.2 release. =;D

I also did some more sync testing with KPilot today and am quite pleased with the results. Thanks to Bertjan’s excellent GSOC work from the last 2 years, we have a really solid syncing solution, and my confidence in it is only growing stronger, the more testing that I do.

So, with this little memento, and a renewed resolve to not be jealous of all the Camp KDE folks enjoying Jamaica whilst I spent 3 hours, shoveling snow today, I now turn in for the night.

KPilot Bug Fixing Mayhem!

Tuesday January 13, 2009
by Jason 'vanRijn' Kasper

KPilot 4.2 progress

I discovered a nasty little data corruption bug in KPilot last night and have put some fixes in for it just this morning. The good news is that we didn’t lose any data. We just gave you a lot more data. =:) So, if you’re helping to test KPilot for our KDE 4.2 release looming Any Day Now (TM), please update from svn (branches/KDE/4.2/kdepim/kpilot) and test again. There is still one little nasty behavior that I see that I need to find a fix for tonight, though. With our new core conduit design for KDE 4.2, KPilot keeps its Handheld -> PC mappings in its own XML file–one per conduit. This is a Really Good Thing (also TM). However, it seems that we don’t do a 100% perfect job just yet in being rigorous about validating this XML mapping file and when it gets messed up, bad things can happen. So, if you’re hitting weird problems with KPilot from KDE 4.2, try removing the XML mapping file for that particular conduit in ~/.kde/share/apps/kpilot/conduits/<PalmUserName>/mapping and re-syncing. That will force KPilot to do a “first sync” and re-create this mapping from both data sources. I’m not saying that’s a good answer, and I’m going to look at finding a solution for the real problem tonight, but it may come in handy for someone out there who’s seeing weirdness.