Today marked an important milestone in the history of Zope2. With the first alpha release of the next major version of Zope2, it proves once again to be still a serious player in the Python world. Thanks to a focused effort of its active community we have taken Zope2 to the next level.
This release not only introduces support for both Python 2.5 and 2.6, includes the latest versions of the ZODB and the Zope component libraries, both still under heavy development each, but also redefines its installation story: Zope2 is now fully eggified and plays nicely with the standard way of installing Python packages today: setuptools - easy_install Zope2 at will.
If this wouldn't be enough in itself, the last weeks have seen an incredible uptake in updating Zope2's documentation and http://docs.zope.org/zope2/ is shaping up nicely. You can read more about the new release at: http://docs.zope.org/zope2/releases/2.12/.
In order to include the latest versions of the ZODB and the Zope components, a final release is not expected before Summer 2009.
The next major version of Plone is developed based on Zope 2.12 and we are more than happy to have such a strong foundation for our work.
Thursday, February 26
Tuesday, February 17
Berlinale Sprint 2009
I'm back for a couple of days from Jarn's first 10% sprint.
While everybody else was working on cool and exciting things, I happened to be in the wrong IRC channel at the wrong time. As a result Tres Seaver and me started to do something about Chris McDonough wining about old and unmaintained artifacts.
After a couple of days and nights of more work, you can now witness the new Zope2 Book - 2.12 edition. Finally we put an end to the confusing situation of the 2.6 edition being hosted at zope.org versus the 2.7 edition being generously hosted by Chris. The book has come home and is now available in a new and shiny form at http://docs.zope.org/zope2book/ thanks to Jens Vagelpohl.
Most of the time was spent converting the book from its old structured text source into a more modern reStructuredText format. With Sphinx and the design taken from docs.zope.org it was easy enough to produce the nice HTML output you see now.
We also did spent quite some time to update the actual content of the book and it is starting to be a valuable source again. The purpose of the book is still to introduce newcomers to Zope, who arent't full-fledged developers. It won't ever be a good guide on how to develop complex applications with Zope2 today. We weren't able to catch up on five years of history in the matter of days, so don't expect the book to be all new and shiny, but you will neither find ZClasses in it anymore and might be surprised to see links to the Python Package Index.
If you want to contribute to the book, the source material is available in the Zope SVN repository at svn+ssh://svn.zope.org/repos/main/zope2book/trunk. The web content is updated directly from SVN once a day. If you don't have commit rights, you can send patches or corrections to me and I'll gladly apply them for you.
Inspired by those activities Tres and Baiju M have also started to update the Zope Developers Guide and are busy converting it to reStructuredText as we speak.
The future of Zope2 is still looking bright ;)
Update: Baiju finished the ZDG conversion and it's available at http://docs.zope.org/zdg/. The links at http://www.zope.org/Documentation/Books/ have been updated by Jens.
If anyone feels strongly about the design, please get in touch with the Zope Foundation and the people behind new.zope.org. I'll not start tweaking CSS for one book here.
While everybody else was working on cool and exciting things, I happened to be in the wrong IRC channel at the wrong time. As a result Tres Seaver and me started to do something about Chris McDonough wining about old and unmaintained artifacts.
After a couple of days and nights of more work, you can now witness the new Zope2 Book - 2.12 edition. Finally we put an end to the confusing situation of the 2.6 edition being hosted at zope.org versus the 2.7 edition being generously hosted by Chris. The book has come home and is now available in a new and shiny form at http://docs.zope.org/zope2book/ thanks to Jens Vagelpohl.
Most of the time was spent converting the book from its old structured text source into a more modern reStructuredText format. With Sphinx and the design taken from docs.zope.org it was easy enough to produce the nice HTML output you see now.
We also did spent quite some time to update the actual content of the book and it is starting to be a valuable source again. The purpose of the book is still to introduce newcomers to Zope, who arent't full-fledged developers. It won't ever be a good guide on how to develop complex applications with Zope2 today. We weren't able to catch up on five years of history in the matter of days, so don't expect the book to be all new and shiny, but you will neither find ZClasses in it anymore and might be surprised to see links to the Python Package Index.
If you want to contribute to the book, the source material is available in the Zope SVN repository at svn+ssh://svn.zope.org/repos/main/zope2book/trunk. The web content is updated directly from SVN once a day. If you don't have commit rights, you can send patches or corrections to me and I'll gladly apply them for you.
Inspired by those activities Tres and Baiju M have also started to update the Zope Developers Guide and are busy converting it to reStructuredText as we speak.
The future of Zope2 is still looking bright ;)
Update: Baiju finished the ZDG conversion and it's available at http://docs.zope.org/zdg/. The links at http://www.zope.org/Documentation/Books/ have been updated by Jens.
If anyone feels strongly about the design, please get in touch with the Zope Foundation and the people behind new.zope.org. I'll not start tweaking CSS for one book here.
Monday, February 2
Baarn Plone UI Sprint 2009
I'm back in Norway after four most productive days spent in Baarn on this years Plone UI sprint. Many thanks to Danny Bloemendaal, Wichert Akkerman, Cornelis Kolbach and Informaat for hosting this event once again! It was wonderful and I'll happily come back next time to try out some more different pancakes ;-)
The work part of the sprint had more presentations and large group discussions than the average Plone development sprint, but these have been both interesting and very productive. I think we got a lot closer to getting a common vocabulary and ideas about how to approach a next generation theming and UI story for Plone.
Luckily we didn't start from scratch, but could base our work on the ideas and quite detailed proposals that have been made in recent month. Especially the Deco and Blocks proposals and the prototype implementations of Rob Gietema from Four Digits have been of tremendous help.
Wichert has already written about how we divided the whole group into five smaller teams. I'll try to give a short summary of what has been worked on by each team.
Content editing
When it comes to content editing one of Plone's distinguishing features is its approach to do real inline WYSIWYG content editing. Most other CMS's have a separate administration interface, that has little to do with the website driven by the system. Plone's integrated approach has been a huge success, but has proven to have various difficulties over the years.
Based on the experience and proposals the team looked into different ways of more clearly separating the administrative UI from the website itself. Examples of this are moving the administrative controls to the edge or most commonly the top of the browser window, lowering the number of controls shown by default and consolidating the controls into a single area.
Another area has been the support for so-called working modes. So far Plone determines what kind of controls the user sees primarily based on the permission set of the user in a given context. The standard view of a page mixes in inline editing controls and directly offers access to even the most uncommon operations with equal priority (think of View, Edit, Rules, History, Sharing but also assigning portlets).
The idea behind working modes is to make it an explicit choice of the user to go into any kind of editing operation. Usually most users will only view information, even if they have the right to edit. The switch to edit mode should be fast, easy and seamless, but nonetheless explicit. Once the user sees the editing controls, he should still see only the most common controls, to make them more effective to use. More uncommon options can be hidden on different pages or using modal dialogs.
We have seen various prototypes of how such separation of concerns could look like and what is technically feasible in modern browsers. One of the things everybody agreed on, is the desperate need for user testing of these approaches, to see what works in the real world for normal users.
Page assembly / Page layout
One of the main ideas behind this work, which is based in large parts on the Deco proposal, is to make it possible for normal users to create rich pages including rich layouts through the standard Plone UI in an intuitive but powerful fashion. It also unifies the editing story for the main content area and any kind of secondary columns, taking away the complexity of entirely different ways of working with these as seen today.
The team gathered around Rob to continue the work on the existing prototypes and incorporate feedback from the other teams into the ongoing work. I'm sure Rob will give us more details in the next days on the progress that has been made.
In the daily wrap-up presentations we have seen a lot of work being done on the modal dialogs, like integration of the actual image browser, examples of tabbed settings in the dialogs and save dialogs including more advanced options like the change note field or the possibility to do workflow changes. The prototype from before the sprint is visible online and lets you have a sneak peak at the kind of changes we are working on.
Control panels
The control panel or global administration UI team gathered around David Convent and part of the GW20E crowd and had time to start implementing many of the ideas and sketches done in the first day. If you followed the SVN commit messages you will have noticed the new plone.app.users package growing over the last days. As seen by the name the main focus has been on the user related settings and general user related data.
The entire range of user dashboard, profile or author page, personal settings, password change forms but also join forms and general member data administration have gotten an overhaul. The goal is to clean up the confusing UI in this area (navigation between the different pages is quite unintuitive today) and provide more flexibility to adjust parts of the pages in question. At the end of the day it should be easy to add for example the country to the member data, make it a required field on the join form and get it into the users data screen.
Backend / Site Theming
The team gathered around Laurence Rowe, Eric Steele and Denys Mishunov and worked and putting the Blocks proposal into action and refine it based on actual experience from designers.
The goal in this area is to redefine the entire theming story for Plone in a way that makes it vastly easier. The focus is on making it possible to theme a Plone site effectively with little knowledge about Plone itself using HTML, CSS and images as the main tools. Knowledge about any kind of Zope component architecture or various forms of METAL macros, viewlets, portlets, skin templates or browser pages should no longer be required.
Towards the end of the sprint we had a quite advanced demo that showed this in action and we were able to see how even a Plone 3 site was themed in this way. While the new approach uses a unified concept of tiles as "possibly dynamic chunks of HTML" instead of all the concepts we had before, bridging code to provide viewlets and portlets into this new model has been written and many more of the older concepts should be easy to bridge into the new model. An evolutionary upgrade path is going to be possible with this, not causing a major break in knowledge and technology investments already being made. Denys captured one of the demos on video and I hope he and Eric will tell the world more about this exciting new story.
Testing AJAX
With all exciting new technology and especially a focus on a much richer client side editor environment, comes the dreaded question on how to do effective testing of AJAX applications. The team gathered around Balazs Ree and Godefroid Chapelle of KSS-fame, who brought extensive knowledge in the area with them. Various different approaches on how to effectively test the real application, doing so in a repeatable and automated way, while making it easy to write tests have been discussed and some of them prototyped.
The tools of choice today are still Selenium and with Selenium RC and its Python bindings a way to instruct a browser from a Python test. Thanks to collective.buildbot and buildout setting up an automated testing infrastructure on multiple platforms using different browsers has become vastly simpler. Progress has been made in various directions, picking up old work from the early KSS days as well as utilizing new infrastructure like WSGI middleware. I hope Jean-Paul Ladage and Balazs Ree can tell more about the concrete achievements being made.
Summary
As you can read there's been work done in various directions and I'm excited about all of them. With the progress being made on the sprint I'm looking forward to see the wider community feedback on these topics. Now that we have gotten actual prototypes to play with and are starting to see concrete implementations of the different pieces this no longer is an abstract exercise.
The work part of the sprint had more presentations and large group discussions than the average Plone development sprint, but these have been both interesting and very productive. I think we got a lot closer to getting a common vocabulary and ideas about how to approach a next generation theming and UI story for Plone.
Luckily we didn't start from scratch, but could base our work on the ideas and quite detailed proposals that have been made in recent month. Especially the Deco and Blocks proposals and the prototype implementations of Rob Gietema from Four Digits have been of tremendous help.
Wichert has already written about how we divided the whole group into five smaller teams. I'll try to give a short summary of what has been worked on by each team.
Content editing
When it comes to content editing one of Plone's distinguishing features is its approach to do real inline WYSIWYG content editing. Most other CMS's have a separate administration interface, that has little to do with the website driven by the system. Plone's integrated approach has been a huge success, but has proven to have various difficulties over the years.
Based on the experience and proposals the team looked into different ways of more clearly separating the administrative UI from the website itself. Examples of this are moving the administrative controls to the edge or most commonly the top of the browser window, lowering the number of controls shown by default and consolidating the controls into a single area.
Another area has been the support for so-called working modes. So far Plone determines what kind of controls the user sees primarily based on the permission set of the user in a given context. The standard view of a page mixes in inline editing controls and directly offers access to even the most uncommon operations with equal priority (think of View, Edit, Rules, History, Sharing but also assigning portlets).
The idea behind working modes is to make it an explicit choice of the user to go into any kind of editing operation. Usually most users will only view information, even if they have the right to edit. The switch to edit mode should be fast, easy and seamless, but nonetheless explicit. Once the user sees the editing controls, he should still see only the most common controls, to make them more effective to use. More uncommon options can be hidden on different pages or using modal dialogs.
We have seen various prototypes of how such separation of concerns could look like and what is technically feasible in modern browsers. One of the things everybody agreed on, is the desperate need for user testing of these approaches, to see what works in the real world for normal users.
Page assembly / Page layout
One of the main ideas behind this work, which is based in large parts on the Deco proposal, is to make it possible for normal users to create rich pages including rich layouts through the standard Plone UI in an intuitive but powerful fashion. It also unifies the editing story for the main content area and any kind of secondary columns, taking away the complexity of entirely different ways of working with these as seen today.
The team gathered around Rob to continue the work on the existing prototypes and incorporate feedback from the other teams into the ongoing work. I'm sure Rob will give us more details in the next days on the progress that has been made.
In the daily wrap-up presentations we have seen a lot of work being done on the modal dialogs, like integration of the actual image browser, examples of tabbed settings in the dialogs and save dialogs including more advanced options like the change note field or the possibility to do workflow changes. The prototype from before the sprint is visible online and lets you have a sneak peak at the kind of changes we are working on.
Control panels
The control panel or global administration UI team gathered around David Convent and part of the GW20E crowd and had time to start implementing many of the ideas and sketches done in the first day. If you followed the SVN commit messages you will have noticed the new plone.app.users package growing over the last days. As seen by the name the main focus has been on the user related settings and general user related data.
The entire range of user dashboard, profile or author page, personal settings, password change forms but also join forms and general member data administration have gotten an overhaul. The goal is to clean up the confusing UI in this area (navigation between the different pages is quite unintuitive today) and provide more flexibility to adjust parts of the pages in question. At the end of the day it should be easy to add for example the country to the member data, make it a required field on the join form and get it into the users data screen.
Backend / Site Theming
The team gathered around Laurence Rowe, Eric Steele and Denys Mishunov and worked and putting the Blocks proposal into action and refine it based on actual experience from designers.
The goal in this area is to redefine the entire theming story for Plone in a way that makes it vastly easier. The focus is on making it possible to theme a Plone site effectively with little knowledge about Plone itself using HTML, CSS and images as the main tools. Knowledge about any kind of Zope component architecture or various forms of METAL macros, viewlets, portlets, skin templates or browser pages should no longer be required.
Towards the end of the sprint we had a quite advanced demo that showed this in action and we were able to see how even a Plone 3 site was themed in this way. While the new approach uses a unified concept of tiles as "possibly dynamic chunks of HTML" instead of all the concepts we had before, bridging code to provide viewlets and portlets into this new model has been written and many more of the older concepts should be easy to bridge into the new model. An evolutionary upgrade path is going to be possible with this, not causing a major break in knowledge and technology investments already being made. Denys captured one of the demos on video and I hope he and Eric will tell the world more about this exciting new story.
Testing AJAX
With all exciting new technology and especially a focus on a much richer client side editor environment, comes the dreaded question on how to do effective testing of AJAX applications. The team gathered around Balazs Ree and Godefroid Chapelle of KSS-fame, who brought extensive knowledge in the area with them. Various different approaches on how to effectively test the real application, doing so in a repeatable and automated way, while making it easy to write tests have been discussed and some of them prototyped.
The tools of choice today are still Selenium and with Selenium RC and its Python bindings a way to instruct a browser from a Python test. Thanks to collective.buildbot and buildout setting up an automated testing infrastructure on multiple platforms using different browsers has become vastly simpler. Progress has been made in various directions, picking up old work from the early KSS days as well as utilizing new infrastructure like WSGI middleware. I hope Jean-Paul Ladage and Balazs Ree can tell more about the concrete achievements being made.
Summary
As you can read there's been work done in various directions and I'm excited about all of them. With the progress being made on the sprint I'm looking forward to see the wider community feedback on these topics. Now that we have gotten actual prototypes to play with and are starting to see concrete implementations of the different pieces this no longer is an abstract exercise.
Sunday, February 1
Profiling with kcachegrind made easier
During the third day of this years Baarn UI sprint, I couldn't resist to look into profiling Plone's startup speed again. As I've been recently getting used to using kcachegrind for visualizing profiling data, I wanted to make it easier to get the profiling data right in this tool.
When doing profiling on full application requests the WSGI middleware repoze.profile is an excellent tool to use. But for profiling startup speed I needed to get back to profiling an individual function call. profilehooks to the rescue? Sadly not.
In order to convert the profiling data into a format understandable to kcachegrind I needed the data to be in the cProfile format produced by Python 2.5's new profiling library. profilehooks still lives in the Python 2.4 world and has profile and hotshot support.
So I made a new tiny package called profilestats which will use the cProfile module instead. It works on both Python 2.5 and 2.6 and writes out the profiling data directly into a file called cachegrind.out.profilestats for kcachegrind to grok.
On the actual Plone startup speed front there isn't any real news. Parsing and executing ZCML, deepcopying Archetypes schemata and just the sheer number of imports and packages to load are still the slowest parts. Especially the recursive algorithms used in zope.interface and zope.component are still the slowest individual parts by a large offset.
When doing profiling on full application requests the WSGI middleware repoze.profile is an excellent tool to use. But for profiling startup speed I needed to get back to profiling an individual function call. profilehooks to the rescue? Sadly not.
In order to convert the profiling data into a format understandable to kcachegrind I needed the data to be in the cProfile format produced by Python 2.5's new profiling library. profilehooks still lives in the Python 2.4 world and has profile and hotshot support.
So I made a new tiny package called profilestats which will use the cProfile module instead. It works on both Python 2.5 and 2.6 and writes out the profiling data directly into a file called cachegrind.out.profilestats for kcachegrind to grok.
On the actual Plone startup speed front there isn't any real news. Parsing and executing ZCML, deepcopying Archetypes schemata and just the sheer number of imports and packages to load are still the slowest parts. Especially the recursive algorithms used in zope.interface and zope.component are still the slowest individual parts by a large offset.
Subscribe to:
Posts (Atom)
