1. Migrating is fun fun fun!
Let's face it - migrating Plone site from old version to new one can be either painful or very painful experience. If you need to do it for one site only, you don't even get the reward of the fact that second time will be so much easier. Third time you'll start wondering what was the hard part in it.
This post is intended to be description how we did plone migration from older version to new one with big, live and heavy traffic site. We're still middle of migrating rest of our sites and therefore I'm going to upgrade this description when I'm having more time. Also note that this is written with one go from memory and as doing the migration is quite complex thing to do including many tasks there might be some errors.
I hope this gives some ideas to others struggling with the same topic and also raises some discussion why this is so complex to do when it shouldn't. I'm also happy to get comments how we could have done this better.
This post is intended to be description how we did plone migration from older version to new one with big, live and heavy traffic site. We're still middle of migrating rest of our sites and therefore I'm going to upgrade this description when I'm having more time. Also note that this is written with one go from memory and as doing the migration is quite complex thing to do including many tasks there might be some errors.
I hope this gives some ideas to others struggling with the same topic and also raises some discussion why this is so complex to do when it shouldn't. I'm also happy to get comments how we could have done this better.
2. Backgrounds
In our case we had quite big Plone 2.5.3 site with over 100 000 objects and about 10 GB Data.fs file (zodb database). We have about 500 active users who create content and our front page gets over million hit/month. Amount of data/objects/activity started to have it's effect on our site and we decided it's time to split the site to smaller individual plone sites - and while we're at it - also update hardware behind all this. At those times Plone 3 had matured enough, se we decided to upgrade the Plone version as well. This is where all the fun starts.
3. Old products
Our site has several custom products developed for Plone 2.1 and Plone 2.5 series - we even had some old products from pre-archetypes times (mxmDynamicPage), so we knew we had some migration to do.
3.1 Evaluation of available tools
There are quite a few tools available for migrating Plone content, but at least by those times none of them seemed mature enough for our needs. Some of them didn't work at all with Plone 2.5 versions and other didn't work with Plone 3. Luckily we found json_migrator product which wasn't silver bullet, but at least it was easily modified to suit our needs. Our special thanks goes to json_migrators developer Max M who kindly helped us to get started with it.
3.2 Writing new versions of our own custom products so that they work in Plone 3
This was good chance to get to know Plone 3 world and start doing things Plone 3 way. Besides our old products, we also wrote policy product which has proven to be a life saver when we were actually doing migrations. Author is now a fun of policy products!
3.3 Writing migration scripts
Some parts of the migrations we did with json_migrator and some by using Product.contentmigrator product. Reason for this divide was that we had products with no actual replacements in Plone 3 (well other than Plone 3 itself) like Simple Portlet which functionality is in Plone 3 already. This is why we needed to take data from our old site and create a script which could import the data as appropriate to environment. Eg. in Simple Portlets case we did a script which took the data from saved json-files and added static text portlets to certain folders right or left portlet.
4. Data.fs or zexp export?
Traditionally migration is done by copying old sites Data.fs file under new site and starting a site, running the portal_migration and hoping there won't be any problems and finally be happy migrator.
4.1 ..or both?
Our case was different as we were splitting our site - not migrating the whole thing. So what we had was the following:
- Live site which was no no area to tweak
- Identical "preview" site which had latest backup from our live-site
- Helper site with identical versions from everything in above two
- Brand new Plone 3.1.7
While doing the migration we didn't touch our live site at all - it was all the time in use and users knew what was going on and were ready for updating same info to two sites at the same time. On our preview site we created new plone site and named it with the final name it was going to be in our new site. Then we cutted from the preview-site all the content we were going to split and pasted it in this - let's call it preview2 - site.
This took awhile... even with pretty heavy class server.
4.3 Cleaning out the mess and doing the export
After cutting and pasting was ready we ran json_export script which took care of all the data we couldn't migrate other ways. When this was ready we ran scripts which deleted all that content which was now safe in json-files on the server. This was also good point for removing products which were unneeded in Plone 3. In our experience eg. Plone Language Tool caused some random problems for us while we were doing first migrations so this is where we now get rid of it. We also removed all those products whose data we exported as json-files - there's no reason to take dead weight to new Plone :) After cleanup we're ready to export our preview2 site as zexp to server. Luckily this didn't take that long.
5. w00t.. what to do with this zexp
We moved zexp-file inside our helper sites var -folder and started the site. Now all we needed to do was log in to ZMI and go to export/import form. From there under import section we had our zexp-file. We selected it from the dropdown box and made sure we had the "Retain existing ownership information" checked and clicked import-button.
This can also take some time to complete.
After import is ready you can shutdown this helper site and copy its Data.fs file under you new Plone 3 site.
6. Running the final steps of migration
Now we're finally in the point where things either work or they don't. If luck isn't on your side today you can end up in the situation where all the work and waiting was for nothing. We were optimistic and started the new Plone 3.1.7 site and hopeed for the best. While site is starting it is building the indexes so this takes awhile, but nothing compared to the paste or import times mentioned in previous chapters.
After awhile we had zope running but when we tried to see Plone we're greeted with horrible errors. We logged in as admin to ZMI and opened our plone site in the ZMI and clicked portal_migration object. Now we had portal_migrations form open in our web browser where there are some version information in the upper part of the window. There was old plone sites version and current plone sites version. Now we simply just clicked the migrate-button which started the actual migration. Luckily everything went fine and we had migrated our old Plone site to new version.
7. Final cleanup
7.1 Old products
Migration leaves all sort of stuff from old plone still around. There are probably some products in the ZMI Control panels product management page which are remainings from the old Plone. Easy solution to clean them is to select everything from that page, click delete, stop and restart the site.
7.2 Workflows
For some reason migrations doesn't touch workflows so if you have used default plone workflow/folder workflow in your old plone you'll likely want to create new temporary site under Plone 3 and copy its plone and folder workflows under your actual sites portal_workflow (after you've deleted the old ones). Reason for this is that in the Plone 3.1 you don't give owner, or manager roles anymore via sharing tab - instead you give them roles like can edit, can add etc. Old plones workflows doesn't give a damn about those new roles and likewise doesn't have owner or manager roles in sharing tab. So unless you don't do this your newly assigned roles won't work. After replacing those workflows you'll be still blind of what roles users have in your site as sharing tab doesn't include old roles. Our solution to this was custom script which walked through all folderish objects and converted owner/manager roles to can add/can edit roles and Member roles to can read. Problem solved.
7.3 Policy product
Now it's time for our shiny new policy product which installs our products to our site and takes care of turning on/off every little switches. After installing this you should have almost ready site, but there are still few things to do. Now when we have all the products installed we still have one migration to run - contentmigration.
If you still remember we decided to migrate some of the contents by using contentmigrator product. We still have some content on our site using old product and we need to get rid of it so we create external method to portal_skins/custom folder and fill the needed information to point our external method of our new replacing product. We save the external method and hit the test tab. When done we can remove remainings of our old product by deleting the product from products folder and restarting site.
8. Enjoy
What? It's already done? Whew.. it wasn't so bad was it?
Comments
Post a Comment