CMS evaluation
- Paul Seiler (Unlicensed)
CMS options
The following CMS were identified for evaluation in this process:
- Drupal: https://www.drupal.org/
- Silverstripe: https://www.silverstripe.org/
- Squiz: https://www.squiz.net/
- Weebly: https://www.weebly.com/
- Wix: http://www.wix.com/
- Wordpress: https://wordpress.com/
Initial elimination/shortlisting
Four of the six CMS were eliminated from full evaluation as the evaluation panel compared each to the evaluation criteria (see table below). A summary of the reasoning is provided below.
Wix
- Suitable only for low complexity sites
- Lacking the powerful developer interface that we require for the complex quadrant
- Weak at supporting multiple sites on the same instance, a requirement to make the management of the service realistic and sustainable
- Limited integration to support user management
- Limited integrations with other 'services' that we desire
Weebly
- Suitable only for low complexity sites
- Lacking the powerful developer interface that we require for the complex quadrant
- Weak at supporting multiple sites on the same instance, a requirement to make the management of the service realistic and sustainable
- Limited integration to support user management
- Limited integrations with other 'services' that we desire
Wordpress
- Suitable only for low complexity sites
- Lacking the powerful developer interface that we require for the complex quadrant
- Weak at supporting multiple sites on the same instance, a requirement to make the management of the service realistic and sustainable. Although there is multi-site capability, it is limited.
- Limited integration to support user management
- Limited integrations with other 'services' that we desire
Squiz
- UI is too complex and difficult for untrained users to manage content
- Lacks the powerful developer interface that we require for the complex quadrant
- Too few experienced developers, especially in Wellington, but even NZ
- Limited, and expensive, user support, unless provided by the Web Team
- Too few plug-ins
Scored evaluation of short-listed CMSs
Both Drupal and Silverstripe were scored against the evaluation criteria below, using a scale of 0-5, where 0 means that the product/service doesn't meet the criteria at all and 5 means that it meets it fully or exceeds our expectations. The following table shows the average scores across all panel members.
| CMS feature or attribute | Importance | Weighting | Drupal | Silverstripe |
|---|---|---|---|---|
Simple/usable editing interface for content editors/maintainers (CMS) Users with low level of technical skills are easily able to create and maintain content with no/minimum training. Most content staff are not full time web editors, but perform this function as one of many. | Essential | 15% | ||
Powerful developer interface (CMS) Skilled developers can work efficiently using a development interface. Enables the development of new integrations and functionality by skilled staff. | Essential | 15% | ||
Developers readily available (CMS) There is a large (and growing?) developer pool, especially in Wellington, with experience in the CMS. Allows client units to engage development resource however they want (hire/contract directly or through the service provider) and the Web Team to bring people in as required. | Essential | 15% | ||
Support and self-service (CMS) Active user community based around the CMS, ideally in Wellington. Users can access suitable and affordable training resources. | Essential | 10% | ||
Multi-tenanted (CMS) The service can support multiple sites and multiple domains, each with multiple levels of access. Keen to explore the boundaries or limits of this (i.e. when a second instance is preferential) | Essential | 10% | ||
User management (CMS) Authentication must be with ADFS for write/editor access Ideally, the ADFS integration extends to authorisation/group membership What about provisioning and deprovisioning flows? | Essential | 10% | ||
Plug-in library (CMS) There should be many plug-ins available to extend the core functionality with the minimum of development effort. Including:
| Essential | 5% | ||
Backing up, restoring and versioning (CMS) The CMS supports versioning of the content/sites and the associated review and rollback. The service provider either offers a back-up service or the service allows for auto backup via cron job or other program. We can restore our own files by yourself. | Essential | 5% | ||
Simple/usable control panel (CMS) Web Team (i.e. super admin) access to a useful and usable control panel to monitor and manage all sites on our instance(s). | Desirable | 5% | ||
Templates (CMS) Easy templating to enable reuse of look and feel aspects. Readily available pre-built templates, even if pay-to-use. Good native support for responsive sites. | Desirable | 5% | ||
Integrates with our other shared services (CMS) The site can draw on our CDN The pages on these sites will display in home site search results (if required). Google analytics is available across all the sites. | Desirable | 5% | ||
| 100% |
Other criteria
The following criteria were best scored as binary (i.e. Yes/No), rather than as a percentage. The values below represent the consensus of the panel.
| Feature or attribute | Importance | Scoring | Drupal | Silverstripe |
|---|---|---|---|---|
Security The CMS must meet the University's minimum security standards. See DIA Cloud Checklist Template - To Vendor.xlsx | Essential | Yes/No | ||
Price We must be able to afford the price. Check sign-on cf renewal. | Essential | Yes/No | N/a* | N/a* |
*Not applicable in this case, as the CMS is open source and therefore use of it incurs no licence fees.