Skip to main content

Plone RPM deployment

It's been ages since I wrote my last post. I guess now it's time to make that up with post about Plone deployment. In a last few months I've had chance to invest part of my time at work to develop our Plone deployment model and since there hasn't been that many posts about the subject I thought I'd share my experiences for the community.

Before I get to the details here's some background information about the environment I've been dealing with. I'm working at the University of Jyväskylä, Finland. Our university is a heavy weight Plone adopter here in Finland. Plone is in use by every faculty and department. We have about 500 - 700 content managers and about 50 - 60 separate Plone instances. Our front page gets about 2 - 3 million hits page views / month. The important information in previous data is the amount of instances. 50 - 60 is just my estimation about our current status and that amount is increasing every year. I'm sure you got the idea so I'll continue to our previous deployment model which I'm sure is familiar to anyone who has had the pleasure to deploy Plone sites.

Current setup

We've got quite normal setup of RHEL servers, Apache + Varnish combination and loads of Plone sites. Our sites use mainly Plone 3.3.5, but we have increasing amount of Plone 4 sites and also few Plone 2.1 and 2.5 sites. We've used buildout to deploy new sites and update the old ones and as I've mentioned in my previous posts (Managing multiple plone buildouts, problems with plone version pinnging) it hasn't always been enjoyable experience. Don't get me wrong - buildout is awesome tool if you need to put up few sites every now and then and don't need to look after them, but running about 30 buildout scripts within few hours scheduled update window isn't fun. I know many of you think now that why didn't they script that? We did. We also developed tools to update packages automatically to avoid all the manual steps - yet we hit problems in updates, some of them were plain human errors (eg. wrong version pinnings) and some of them were because of some unpredictable behavior of software.

Reinventing the wheel?

Every now and then we've been thinking better deployment story for Plone-sites until we realized that the things we'd want to achieve sounded just like package management software. As we're using Red Hat os in our servers I hold my breath and jumped head first to the RPM world. There isn't too much information about deploying Plone sites as RPM-package on WWW. I knew folks at Weblion have developed environment for Debian based distros, but that didn't help much. Luckily Google revealed that Nikolay Kim from Enfoldsystems had been in talks with Fedora packagers about packaging Plone as RPM. As far as I understood from the mailing list messages this didn't work out well and Enfoldsystems packages never ended up to Fedoras repository due to packaging policy disagreements. Nevertheless Enfoldsystems have kindly published their Plone buildouts and RPM specs in their Subversion repository.

RPM setup

I used Nikolay Kims work as a base structure to fit things to our environment. Nikolay had divided Plone to two separate RPMs. First one (plone-base) contains all the common packages Plone needs. Another package contains the Plone-instance and its eggs. Nikolays packages should work out of the box as they are, but we needed to modify them a bit to suit our needs. I created own RPM-package for the Python virtualenvs - one for Python 2.4.6 and one for Python 2.6.5. I also created RPM-packages for few python libraries (python-imaging, python-lxml, python-ldap) so that they install to virtualenvs instead of system python. With these ready I modified Nikolays spec-files to use my virtualenv packages and modified buildout.cfg file so that it extends to our shared configuration and the initial setup was done. There was lots of manual work to do to add and test RPM-specs for our buildouts but now when it's done life seems to smile at me. To make my workload even lighter and to avoid dull copy-paste work I created custom paster template which creates buildout folder with RPM-spec ready for rpmbuild to do it's magic. I attached the rpmbuild process to Hudson by creating parametrized builds - when our developers includes certain commit message and pushes his changes to Mercurial this launches hudson build, tests and if everything went ok, finally rpmbuild. This is done with the help of hghudson package which I customized a bit to allow parametrized Hudson builds.

What we achieved?

You may wonder what makes RPM deployment better than buildout?

Here are some pros listed in no particular order:
  • It fits well with the regular operating system maintenance.
  • Our system administrators can now update servers/packages automatically without any knowledge about Plone.
  • If something goes wrong we can easily rollback to previous package version.
  • Deploying new site happens just like installing any other package with package management software.
  • We can be sure there isn't going to be any trouble with pypi or some other critical site being down when we're updating/creating new instance.
  • We still have buildout in the background to make developers life easy.
  • It saves time, nerves and makes it possible for me to once again enjoy Wednesdays (guess which day our scheduled maintenance window is?) :)

I know some of these can be achieved by using buildout as well - eg. by mirroring pypi, using eggproxy etc., but nevertheless my experience is that running buildouts on a production server has always a bigger chance to fail than updating tested RPM-package. I still have to admit that at the moment this deployment model is new to us and I probably won't know all the cons. Here are few which are more related to the setup process of RPM deployment model:

  • You'll need a way to create RPM-spec automatically.
  • RPM deployment isn't as straightforward than using buildout.
  • You'll get most out of the pros only when you're serving several instances.
  • You'll need to know at least basics about how to create RPM-packages to set up this kind of environment.

I hope this post will be useful to someone struggling with similar problems. I will be posting updates about the subject when our new deployment model has seen more use. I'd also like to thank Nikolay Kim and Enfoldsystems for the excellent work they have done to make Plone RPM-packaging being as easy as it is now. Thank you!

Popular posts from this blog

Domain name registration through Google - when things go wrong.

Not too many people know, that you can register new domains through Google. This can be done when you're registering for Google Apps Standard Edition which is free and somewhat stripped version of their Google Apps Premium Edition. Latter one is tailored more to suit business needs.

With $10/year prize tag it's not cheapest option, but you'll get "private domain registration to protect against spam at no extra cost, full DNS control and domain management, automatically configured to work with Google services, email, calendar, instant messaging, web pages and more also at no extra charge".

As a comparsion GoDaddy offers .org domains at $14.99/year so it's actually not that bad offer. Google actually is just collecting the data and sending it to their partners (godaddy, enome) which does the registration. I decided to give it a try at January 11th. To my big surprise things didn't went that smoothly. It's been now one week since my order - the domain I …

Plone 4, ZEO and supervisor

This post belongs also to the "lessons learned" category.

With Plone 3, ZEO and supervisor combination you've probably configured your supervisor to start plone instances by running $BUILDOUT/parts/client1/bin/runzope.

Problem is that with Plone 4 your $BUILDOUT/parts/client folder doesn't contain anything else than etc folder. You know starting instances by targeting supervisor to use $BUILDOUT/bin/client1 fg doesn't work like you'd expect (supervisor would control the client1 script - not the actual plone process).

My colleague Jussi Talaskivi figured that using 'console' argument instead of 'fg' for bin/client1 script should do the trick. With 'console' argument stopping, starting and restarting Plone 4 instances with supervisor works like a charm.

Below is full example of working supervisor configuration.

[buildout] parts = supervisor [supervisor] recipe = collective.recipe.supervisor port = 8200 user = xxxx password = xxxx pr…

cd under development packages nested folders without repeating yourself

To me Plone development environment means Fedora 12, terminator and vim. As I do lots of development in the shell there has been one annoying little thing which I'm tired of - cd:ing under development packages inner structure to where the actual code lives.

Doesn't seem much but if you repeat that many times a day you'll get tired of it. Of course you can always launch vim for the package root and open the file you wanted from there, but it's still extra effort.

Few weeks ago me and my colleague Jussi Talaskivi started to think that there has to be a script which takes you to your destination simply by parsing dotted name and using information from there to cd you to the correct place. After spending some time googling around we found none. Annoyed by this we decided to do this missing script by ourself.

Soon it turned out that this wasn't that easy task as it sounds - simple shell or python script where you just parse correct path from arguments wouldn't do t…