The summer is over

Dependent on where you are located on this planet, the summer season may still go on for some weeks or is yet some month away, but at least Google's Summer of Code 2006 is over.

I had the privilege to be one of three students, who where allowed to work on projects for the Plone Foundation. A big thanks to Google for making this happen and to all the people in the Plone community for supporting the idea and me specifically. A special thanks goes to my mentor Alec Mitchell, who has given me all the support I wanted.

Now it is time to look back at the last few months and see how much progress I made and how many of my goals I actually reached. For those of you not familiar with my project, the short form was: Reimplement as much as possible of the existing i18n/l10n infrastructure found in the core Plone project based on Zope3.

To be honest I did not achieve as many of my goals than I had hoped for myself. But the good news is that while some of the work is still to be done, on the one hand I will continue to work on these and on the other hand that quite a lot of my time has gone into creating or refining more infrastructure and process for developing with Zope3 in the context of Plone, which will hopefully benefit all of you.

But here are some concrete things I have been working on:

1. Finish the move to use Zope3 Messages to internationalize Python code.

This has been started in Plone 2.5 as detailed in PLIP 108 and now has been extended to the world of Archetypes. On the current Plone 3.0 bundle you will find that both Archetypes and all of ATContentTypes use Messages now to mark any textual data which needs translation as part of the UI. For an example on how this looks like you can view the event schema definition in ATCT. The two most prominent benefits of this are the ability to automatically extract all these messages from your product to build the Gettext translation templates and the ability to define these messages in your products own translation domain. Both of these should make it considerably easier to internationalize your products resulting in more products being available in multiple languages.

2. Sanitizing the lists of available languages and countries

So far we had three different implementations in AT, Plone and PloneLanguageTool used for various language lists and they where heavily out of sync. I reimplemented these as Zope3 utilities as part of my plone.i18n product and changed all the various places to use these centralized lists instead. Of course there are some different utilities for meta data languages or content languages now and you have the ability to customize these lists easily by adding local persistent variants of these utilities as implemented in plone.app.i18n. If you don't understand what I mean by that: Don't worry - there'll be a nice simple UI for you, which just allows you to select the languages you want to be available. After all this is still Plone - nice defaults and ease of use ;)

3. New reusable and extensible text normalization

When creating new items in Plone the technical id which ends up being used in the URL is usually autogenerated from the title of your object. As the URL is currently still limited to pure ASCII characters, you will have to normalize non-ASCII chars in some way to an ASCII equivalent for all languages except English. So far there was a hardcoded list of character mappings in CMFPlone itself, which you had no chance to enhance or customize easily. Now the whole process is based on Zope3 utilities as well allowing for easy customization and re-use. You can find the code in the plone.i18n package as well. As an added bonus the title normalization is now language dependent, so you can have meaningful mappings regarding your language and the normalization process is not slowed done by having to look at mappings for the whole Unicode character range at once.

4. Making our nice country flags available as Zope3 resources

If you have installed PloneLanguageTool so far or have a multi-language site based on LinguaPlone you will have noticed some nice icons that use country flags to symbolize the languages you want to read content in or translate to. These are now part of plone.i18n as well and thus available in any Zope3 based project rather than being stuck in a Plone specific product.

5. Lots of bug fixes...

Developing new things while there are still critical bugs for the most recent version didn't make much sense to me, so I tried to kill all serious internationalization related bugs for Plone 2.5 and some more as I was on it ;)

And now some notes on the infrastructure side

6. New ZopeSkel templates to quickstart your Plone product

As the Plone community is currently adopting Zope3 development practices, one of the new things you have to learn about, is writing your products not as Zope Products but as regular Python packages. These days this involves providing some metadata to enable them to be packaged as eggs. As this is quite some boilerplate to remember Daniel Nouri has created a new product called ZopeSkel based on Ian Bickings PasteScript to generate product skeletons. After some extensive discussions I have added some nice templates to quickstart products in the plone and plone.app namespaces.

7. New GenericSetup support for local components

As I already blogged about some days ago, followed up by Martin Aspeli's blog entry about local adapters and their use in plone.portlets I have some fresh news from this front. The whole export/import handlers are now part of their own project called GSLocalAddons and I have extended the handlers quite a bit. They are now able to create registrations for utilities and adapters and as a special mode, they can even create utility registration from your already existing content in the ZODB. So making your CMF tools available as utilities shouldn't be any harder as to write two lines of XML now.

8. CMF 2.1 support for Plone 3.0

Only partially related to my SoC project has been my work on CMF 2.1 for Plone. At least this adds one of the last missing pieces to the i18n-puzzle by using Zope3 Messages for things like action titles and some more CMF specific things. I had stopped working on this to concentrate my efforts on the more vital parts of my SoC project, but now that this is over, I can start working on it again.

What remains to be done?

As you can read in my project description there are still some features currently available in PloneLanguageTool and PlacelessTranslationService not yet available in Zope3. After I cleaned up my current projects, I will start working on these again, so we might be able to retire both of these products in Plone 3.5 in the next year. So stay tuned, as the journey to a better world of internationalization isn't over yet :)

We are getting local...

As you may have noticed Plone developers are currently pushing hard to use new Zope3 technologies to bring sanity again to the world of Plone code.

So far we have been using interfaces and browser views in Plone 2.5, but for Plone 3.0 we are going to rely on Zope 2.10 which has a far easier local component story. So you can expect to see not only global utilities and adapters but also their local counterparts.

But how do we actually create those in our persistent CMF-based portals? Well, why not do the obvious and use our existing configuration setup called GenericSetup to create them for us?

Imagine a simple file called sitemanager.xml with the following code:

< ?xml version="1.0"? >
< utilities >
< utility
 interface="plone.app.i18n.locales.interfaces.ICountries"
 class="plone.app.i18n.locales.countries.Countries"
 />
< /utilities >

Wouldn't it be nice if you could use something like it? Well you can, I made the first prototype as part of my Google Summer of Code project and it's working just fine.

You can take a sneak peak at the code at: http://dev.plone.org/collective/browser/GSLocalAddons/trunk/exportimport/sitemanager.py. Once this has a decent amount of tests, I would consider this a valuable addition to GenericSetup itself.

P.S. I should note that I stole some ideas from Lennart Regebro's work on CPSSharedCalendar, thx mate ;)