You are here: Home CS-Workshop CS-Workshop Blog

CS-Workshop Blog

I can't reorder elements in a Plone folder!

by Mikel Larreategi — last modified Nov 11, 2008 05:25 PM
Filed Under:

Recently I had a problem in a website we are currently developing to reorder some custom content-types in Plone folder.

We are currently building a website that has many customized content-types. In the main folder of the structure we are building, the client wants to reorder elements freely, like in a normal Plone folder. Well, the main folder of the structure is currently a Plone folder, so "no problem" I said, just go to Contents tabs and reorder the elements. But... aghh !!! I couldn't reorder the items !!

I started investigating what was going on, putting a pdb here and there, and found a curious thing: the items stored in the folder were some archetypes based custom objects with a portal type with a space in its name. This portal type was "Product Line". The reordering code compared the items inside it and checked whether the item was actually in it looking at its _objects attribute, nothing special til here. But the check was not safe enough.

The code looked at the Metatype of the FTIs in the portal_types tool to check whether the element which was being reordered was currently a CMF object. But the Metatype of the FTI wasn't the same as the meta type of the object, because the Metatype of the FTI was the actual portal type of the object.

Moreover, the _objects attribute wasn't modified when the meta type of the object was changed, so it wasn't enough the change just the meta_type. I had to write a migration script to change the _objects attribute of the folder to change "ProductLine" with "Product Line". That change made my custom objects orderable.


World Plone Day in the Basque Country

by Luistxo Fernandez — last modified Nov 07, 2008 10:20 AM
Filed Under:

Our company office space hosts today the World Plone Day.

World Plone Day is happening today in over 30 cities around the world. We organised a local meeting in the Basque Country, and talks and mini-workshops are happening around me just as I write these lines. | World Plone Day 2008 © cc-by-sa: codesyntax

Some 30 people gathered here today, and we are happy to see webmasters that edit some of the Plone websites that CodeSyntax developed. | Oier, Joxe, Iban © cc-by-sa: codesyntax

Above, editors of community website, Oier Araolaza; from Mondragon University, Joxe Aranzabal; and from local newssite, Iban Arantzabal.

One of our customers also volunteered to talk. Jabier Santamaria is the engineer behind a Plonegov / Udalplone (our local brand for this Plone project to create websites for the public administration) website, the one for the town council of Amorebieta-Etxano. | Jabier Santamaria © cc-by-sa: codesyntax

We've been happy to welcome as well other Plone creators in the Basque Country, not only sites developed by us. Jesus Romo, of the virtual campus of the University of the Basque Country, has explained how they use Plone in their eKasi elearning project. | Jesus Romo, EHU © cc-by-sa: codesyntax

More pictures from this day in Flickr. And the slides from the presentations made by CodeSyntax coworkers today.

Open Source World Conference Malaga 2008

by Eneko Astigarraga — last modified Oct 20, 2008 09:30 AM
Filed Under:

We are taking part in the Open Source World Conference that will be held at the Malaga's Trade Fairs and Congress Center on 20-22 October.  A great Conference, more than 8,000 people are expected to attend to the conference, the exhibition and complementary activities area is to extend to over 6500 m² and with world-level open source software communities participation.

I hope to attend to some events in which we participate.


OSOR, Open Source Observatory

Observatorio Software Libre OSOR

On Monday is the OSOR, Open Source Observatory and Repository for European Public Administrations  Launch Event at the Open Source World Conference.

The OSOR is a platform actively supporting the sharing of OSS-based eGovernment applications and experiences across Europe, and PloneGov project is a best practice example in the area. Xavier Heymans, ZEA Partners, presents the PloneGov initiative.


Morfeo project

Morfeo project and community is collaborating with a stand and 8 Morfeo-related taMorfeolks and workshops.

Morfeo community is a well open innovation model, and tomorrow you have a interesting speech about  "MORFEO: the development of open source software as part of the strategy for materializing the future Internet of Services" or the Miguel Ángel Cañas and Marcos Reyes speech about EzWeb platform: "EzWeb: an open source application mashup web platform for the next-generation SOA and the future Internet of Services"


Morfeo Project Malaga


Plone everywhere

Btw, I'm talking about  PloneGov , our e-government solution, but Plone is everywhere present in Malaga: for example, the Open Source World Conference site (1) and the OSOR open source observatory (2) are developped  on Plone, our main CMS choice, based on Zope and Python, That's a great choice! 

Welcome Plone Conference 2008

by Mikel Larreategi — last modified Oct 08, 2008 08:31 PM
Filed Under:

We landed in the USA yesterday afternoon, after a long-long trip from Eibar. Now we are waiting for the afternoon talks to be started. Welcome !

We managed to come to the Ronald Reagan Center at 9:00 more or less, got our registration badges and switched on the laptop... but the wifi was down in the main hall... That's the only bug we found in the Conference.

The Conference has started with the logistics information and the Keynote by Alex Limi and Alan Runyan.

Well, we are just recovering from the lunch (mmm, excellent sandwitch!) and waiting for the next talk about Plone 3.1 product migration to start.

More to come later... with photos if we get a stable net conection and upload some photos to Flickr.

Case study: Improving the performance of a Plone Site

by Mikel Larreategi — last modified Sep 17, 2008 12:00 AM
Filed Under:

We published in early June. EuskalKultura is a website for Basque people living far away from is homeland, mainly in America (both southern and northern America). It's just a Plone 2.5 with some custom products such as birthday greetings and other one or two custom archetypes with 3 or 4 fields.

But the main work before publishing it was to import the information of the old website. The old site was a PHP based website with lots and lots of items (mainly news items but also events, restaurant information, interviews, ...), all of then multilingual, mainly in Basque and Spanish. So we have to write some scripts to pull the data from the MySQL database and create the content in Plone throug invokeFactory, with the usual UnicodeDecodeErrors :)

After testing it we managed to import all the data and create all that newsitems and events, all of them properly linked thanks to LinguaPlone to have fully translated website.

Short after publishing the website, we discovered that it was consuming a lot of RAM. We hosted it in a memory limited account in a FreeBSD account at HighSpeedRails, but we have both excessive use of RAM and constant restarts. HighSpeedRails provides some scripts to control the memory consumption of your Zope applications, and restarts it when passes the established limit. We also tried to upgrade the memory limit, and put it higher, but our Plone started to eat all available memory in the hosting service.

So, we decided to reproduce the situation locally, fix it if possible and reproduce again the fixed website.

Thanks to zc.buildout, reproducing the environment was quite easy, I just had to checkout the corresponding buildout from our svn server and run ./bin/buildout. We downloaded the 1.5 GB size Data.fs and Lur wrote a python script to try to reproduce server's load parsing Apache logs. In the meantime, I wrote a harder test-plan, using also Apache logs, to use it with JMeter, taking many ideas from the Plone Performance Sprint 2007.

In our initial tests, we easily got our Plone site consume 700 MB (and growing) of RAM after running it for half an our (or less).

It was our, and client's idea, to be able to select the content featured at the home, so we used CompositePack to get it. We created a new layout for it, and write a browser view to avoid featured newsitems appear in the news listing. The code under the hood, was proved to be totally inefficient, and after some analysis of the website, we realized that the home page was automatic, I mean, our client wasn't using that feature.-

Our client, also used newsitems (all of them saved inside a folder) to create diferent kind of newsitems: short articles, featured newsitems, common newsitems, ... and wanted to show in the home page all newsitems except the ones keyworded as XXXX. Again, the code to get that newsitem was highly inneficient.

So, we decided to get rid of CompositePack for the home page, and to use AdvancedQuery to be able to make not queries to the Plone Catalog.

Those minor changes, proved to be great, because the navigation on the website improved a lot, it was quite faster, and the response time of the home paged decreased notably.

Another bottleneck was found in the keyword portlet. Our client uses keywords to tag news items and events, and wanted a way to have a list of all keywords used in news items and events, each one in the corresponding section. We fastly created a view getting those keywords from the catalog (with a catalog query and a inneficient algorithm :)), and using the strategy copied from Quills, we created a traversal adapter to have a view with all the news items keyworded with the selected one. The process of getting the list of needed keywords was slow, so we changed it with a static list of keywords, updated on daily-basis through a cronjob.

We also moved all news items stored in plain Plone folders to Plone Large Folders (based on BTrees and disabled out-of-the-box in Plone 2.5). We had no ordering requirements in the news items or events folder, because their main view is a properly configured Topic/Collection, so the change wasn't dramatic, but time-consuming. It took hours to cut-and-paste and recatalog all that news item and events.

We also took a look on all customized templates, and made some improvements to avoid common Plone problems: avoid using getObject once and again, use existing views instead of home-made-scripts, ...

Finaly, we also removed from our buildout some unneeded products, improved the packaging of our base products and put the zodb-object-cache option to the default (5000 objects).

After running again the python script and the JMeter test plan together, we found that our Plone site wasn't consuming more than 300 MB of RAM, although having the script running all the night. Incredible!!!

We made some more tests, changin the zodb-object-cache option of our Zope instance, to a more significant value (following Hector Velarde's advice). We tried with 10000, 20000, 30000 or 50000, but we didn't get any improvement neither in memory consumption nor in CPU usage according to top (OK, perhaps this is not the way to monitor a process, but that's the way we were taught ;)), so we decided to set the value to 5000.

So, we downloaded the live Data.fs on friday (thanks cron), run the news items and events cut-and-paste scripts on saturday and sunday, make all the changes on monday and re-upload the Data.fs again on monday midnight. On tuesday, we just made the last configuration changes, and got it running.

After 8 hours running, it's just consuming 340 MB of RAM, some more than in our local tests, but far away from the 700 MB-after-5-minutes, we had in the previous situation.