Launchpad is now an automatic, magical translation factory!

I've been using Launchpad to host my personal finance application wxBanker for a few years now. The thing I was hearing most often was that it wasn't localized; people wanted currencies to look the way they should in their country, and the application to be in their language. Let me explain how Launchpad helped me provide translations for my application, and how much of an utterly slothful breeze it has recently become.


Normally to handle translations, an application has to wrap strings with gettext, create a template, find translators and give the template to them, collect translation files back, and integrate them into the project. This is painful and is why many applications aren't localized, and shut out most of the world as a result. One of the amazing features of Launchpad however, happens to be Rosetta, which brings translators TO your translations via a simple web interface, and then makes those translations available to the developer. With Rosetta, translators don't need to understand gettext or translation file formats; they just need to know two languages!



So that's what a translator sees. Notice how Launchpad even shows how other applications translated the same string. So once you generate a template and upload it, you can assign a translation group to your project such as "Ubuntu Translators" so that your strings will be translated by volunteers on the Ubuntu project; if your project isn't relevant to the Ubuntu project, you can use the more generic Launchpad Translators group. Now all you have to do is wait for some translations, then download them and check it in to your code. Not too bad, right?

It isn't, but Launchpad has recently made it so much better. They started by adding an option to automatically import translation templates from your project. This means as you are developing, all you have to do is regenerate the template and commit, and new strings will show up for translators in Rosetta and be translated automatically (from the developer's perspective). Then today, they announced the other side of this, which is automatically committing those translations back into your code on a daily basis. This means that all I have to do is commit templates as I change strings, and Launchpad handles everything else. This is a profound tool for developers.

What's the next step? Well, from a developer's perspective the translation template is a tool to give to the translators or in this case Launchpad. In the future Launchpad could eliminate this by generating the template itself from the code (this is what developers are doing themselves, after all), so that truly all you have to do after you set up the initial i18n/l10n framework is commit code as normal, and Launchpad magically commits back translations.

All this work Launchpad is doing gives developers more time to develop while still having localized applications at a very minimal cost. This is continuous translation integration, and boy is it cool!

Simple timing of Python code

Often when I am writing in Python, I find that I want to see how long a particular function call or set of statements are taking to execute. Let's say I have the following code that gets executed frequently:

for i in range(10000000):
   x = 934.12 ** 32.61 * i / 453.12 ** 0.23

and I want to know how long it takes to execute to see if it is slowing down my app and should be optimized. Previously I would surround it as such:

import time; x = time.time()
for i in range(10000000):
   x = 934.12 ** 32.61 * i / 453.12 ** 0.23
print time.time() - x

This will print out the duration in seconds of that code segment, but is more work and typing than I want, and more cleaning up later. I realized that the new "with" statement in Python could probably help me out. Let's create a little timer class that cooperates with it:

class Timer():
   def __enter__(self): self.start = time.time()
   def __exit__(self, *args): print time.time() - self.start

Now all we have to do is:

with Timer():
   for i in range(1000000):
       x = 934.12 ** 32.61 * i / 453.12 ** 0.23

You can also try:

with Timer():
   time.sleep(1.5)

For these, 0.28738 and 1.50169 are what I get, respectively. While something like this couldn't really replace application-wide profiling via a module like cProfile, it can be an extremely useful and quick way to see if your prototype is scalable or not. I usually end up having a debug.py or helpers.py file in my larger projects with little tools like this, and I'll probably end up adding this one as well.

Let me know if you are doing something similar, or if I've reinvented something that already exists. I'd also love to hear from people profiling their python code and what techniques they are using, as I am just starting to learn about it.

Karmic Desktop UDS run-down!

I just got back from a wonderful UDS in beautiful Barcelona and thought I would provide a summary of what we can expect in the Karmic Koala 9.10 desktop. Keep in mind that I don't speak for Canonical and what follows is just my understanding of what is on the table for Karmic.

Overall it is gearing up to be a pretty radical and exciting release; there are some changes to the default application set as well as some major version upgrades of existing core components. We are trying to be fairly aggressive in terms of new stuff so that if Canonical wants Karmic+1 to be an LTS (Long Term Support) release, we can have fairly stabilized new technologies by then (thanks to 6 months of stabilization in the Karmic cycle) instead of having to wait until after the LTS (Karmic+2) to introduce them. Since many of these changes would be too radical to first appear in an LTS, if we don't upgrade now we may not be able to for a year, and have to maintain old versions for 3-5 years in the LTS.

On the messaging front, Ekiga will be dropped from the CD to save 10 megs, and Empathy will likely replace Pidgin due to a responsive upstream, voice/video support, and better GNOME integration. It also now has the ability to import accounts from Pidgin, so this should help out with migration. I checked it out a bit at UDS and was impressed with how useful it is with absolutely zero configuration. It will pull your name from the system and enable avahi (auto-discovery of people nearby, like bonjour) with no set up, which made it quite easy to get in contact with people at the conference. You can also supply your email and Jabber ID to the avahi interface to allow other people to discover that info as well.

It also sounds likely that Banshee will replace Rhythmbox as the default media player, and it is the official default of UNR (Ubuntu Netbook Remix) Karmic. This will bring a snazzier interface, better device support including iPods and Androids, and quite importantly an active and responsive upstream. I will admit to not being a huge fan of this transition for Karmic as it seemed too early for me (the lack of a folder watch is quite a regression for me, and it has been promised for the last 3 releases or so, so I'm not holding my breath), but after checking out 1.5 for a bit I will admit that it is growing on me. The user interface does seem nicer, and the lightweight video library which keeps track of what you haven't watched is nice. However, it does seem to use 3-10x more memory than RB which is very troubling (60-300MB compared to RBs fairly consistent 25MB), especially on the netbook scene. I've also had issues with it skipping occasionally, which is very unfortunate. Hopefully the UNR switch will put pressure on better memory management for Banshee.

Banshee syncing with an Android G1

Empathy and Banshee will probably replace their predecessors around alpha 2 of Karmic, and will be either left as default or reverted based on reported regressions and bugs. Keep in mind that if you end up not preferring these applications, the other ones still exist and you can continue to use them.

There are also going to be a bunch of underlying speed improvements, with the boot speed goal being 10-12 seconds. When Ubuntu talks about boot times, we are referring to the time from when grub starts (when Linux first gets control of the machine) to when the user is at a fully loaded desktop with no I/O. The main test machines being used by Canonical here are Dell Mini 9s, with auto-login enabled to get a consistent log-in time. This is pretty impressive as the boot goal was 25 seconds in Jaunty, which was met, and was aggressive itself as Intrepid booted in about 65 seconds on the Dell Mini 9.

grub2 is likely to be default for new installations (upgrades will have grub1 chainloaded to grub2), with ext4 as the default filesystem. The boot process will also be streamlined, with the grub timeout set to 0 and the boot menu hidden. There will instead be two new ways to boot into a different system now. First, there will be a key that can be pressed while booting to bring up an OS chooser, which will halt the current boot and restart into the chosen one. Another goal is to have the restart menu item in GNOME aware of installed OSes and allow the choice there, so you could select for example "Restart into OSX". All in all this means no racing to select the OS for dual-booters, and a faster boot process as well. /tmp is also hopefully going to be made a tmpfs, which means it will reside in RAM and overflow to a swapfile (which in recent Linuxes have on par performance with swap partitions). This means power savings, less disk I/O (especially great for SSDs), and of course blazingly fast performance which should help out a lot especially when, say, loading files from inside an archive The Gnome Display Manager (GDM, which handles the login screen) will also likely be upgraded to GDM2.

Finally let me fire off a few more changes. Power management is being improved all around, with one change already landed being that audio cards will be automatically powered down after 10 seconds of no sound. Encrypted Home directories will hopefully be easier to set up now with an option right in the graphical installer, and I'm working on a UI for managing this and encrypted Private directories in Karmic, more on that later. Firefox 3.5 should be the default version of Firefox. For notifications which want to display actions if the user is interested, there is work on morphing windows:

Source: https://wiki.ubuntu.com/NotificationDesignGuidelines

Ubuntu is also working on being social from the start (see desktop-karmic-social-from-the-start on gobby.ubuntu.com), perhaps installing Gwibber by default and asking the user if they want to integrate social sites (twitter, facebook) into the desktop when they visit them in Firefox, via an extension. There has also been work in looking for a better scanning application to replace xsane (perhaps GnomeScan), some look into using Gnome Control Center, and a common printing dialog.

Okay phew, that's what I've got to report! Let me know what you think of these decisions and changes, and if there is anything you were hoping for that didn't make it, or really anything else you've got to say!

A teaser: Desktop integration with encrypted directories for Karmic

Recently I've been working on desktop integration with ecryptfs. Dustin Kirkland has done some awesome work enabling encrypted Private directories, as well as entirely encrypted Home directories, and I want to bring a UI to that goodness for the Karmic desktop.

UbuntuOne displays a banner at the top of its shares, and this inspired me to borrow the code for use with encrypted Private directories. After a bunch of hacking and debugging, I finally got something to show up:


Pretty exciting! There is much work to be done behind the scenes but this is an encouraging start. After I get this working I plan on making a UI for installing ecryptfs-utils (the necessary package), setting up an encrypted Private directory, and managing/configuring one (or an encrypted Home). This UI would perhaps be available from System -> Administration -> Encrypted Directories, and would allow a user to have a directory of encrypted files available in a few clicks, which is mounted/unmounted transparently at login/logout.

What do you think? Are you currently using an encrypted Home or Private directory? Would you be more likely to if there was a UI to set it up? Please share your thoughts and comments :) I'll be at UDS and can schedule a session on this if there is interest, as well.

Counting the number of Ubuntu users

There have been a few articles recently trying to estimate the number of Linux users, which is apparently a challenging problem. However I have to wonder why it can't be figured out at least at the distro level by simply storing hashes of IP addresses that hit Canonical's update site, and looking at the number of unique ones each week/month.

There are going to be people using mirrors, but this is a small percent to lose to at least get something in the right magnitude, and the most popular mirrors could probably do a similar thing and contribute their numbers anyway. The only other main drawback would be multiple Ubuntu machines under the same IP, which again seems like it would only result in a slight inaccuracy. You'd also lose a small percent to users infrequently using their computers such that they aren't updated on a monthly basis, but yearly results would pull back in any of these people using their computers frequently enough to warrant counting.

Alternatively, as others have suggested as well, if Google would just release their numbers for browsers hitting google.com, we'd probably have a solid idea as well.

Are there already accurate numbers for Ubuntu and if not, am I missing something with my proposal?

UPDATE: Jef pointed out that Fedora is already doing this at http://fedoraproject.org/wiki/Statistics#Yum_Data, which is pretty awesome! That shows about 14 million unique repository connections, so making a VERY rough, not remotely scientific estimate, we could use distrowatch to estimate that Ubuntu has 1.68 times the number of users as Fedora, and get something around the order of 24 million users that have connected.

Extending Java’s Semaphore to Allow Dynamic Resizing

Marshall wrote an excellent post explaining how to dynamically change the number of permits when using semaphores in Java, which I thought I'd share for anyone interested. This can be particularly useful if you have a long-running daemon which you don't wish to restart for changes such as this. If you are using semaphores in Java, or if you don't even use Java but just want to learn more about semaphores, I'd recommend giving it a read.

Email Deliverability & RFC 2142: Everything you wanted to know and never dared to ask

Today Franck Martin wrote an interesting post regarding RFC 2142 ("Mailbox names for common services, roles and functions") and how it relates to email deliverability. If you are running your own email server or own a domain, you may interested in reading it as it sums up what email addresses are expected to be manned at any domain and for what purposes.

For example, did you know that "if an Internet service provider’s domain name is COMPANY.COM, then the ABUSE@COMPANY.COM address must be valid and supported"? And are you manning your postmaster and hostmaster addresses?