Automatic security updates and a support lifecycle: the missing links to Backdrop affordability

dev
drupal
php
Published

July 2, 2016

I saw Nate Haug and Jen Lampton give their Backdrop CMS intro talk last weekend at the Twin Cities Drupal Camp. They have already done much to identify and remedy some big functionality and UX issues that cause organizations to leave Drupal for WordPress or other platforms, but you can read more about that elsewhere or experience it yourself on Pantheon.

Their stated target audience is organizations who need more from their website than is WordPress’s primary use case, but still don’t need all the capability that Drupal 8’s software engineering may theoretically enable – everyone doesn’t need a CMS that supports BigPipe or compatibility with multiple heterogeneous data backends.

That positions them squarely in the same space in the same target market as Squarespace – but whereas Squarespace is a for-profit business, Backdrop is open-source software, and affordability to the site owner is so important to the project that it gets a mention in Backdrop’s single-sentence mission statement.

Simplified site building for less-than-enterprise organizations is a crowded space already. The Backdrop philosophy identifies a small area of this market where a better solution is conceivable, but I think a big reason so much emphasis is given to goals and philosophy on the Backdrop website and in their conference session is that Jen and Nate recognize their late entry in this crowded market leaves little room for Backdrop to miss in achieving its goals. Backdrop’s decision makers definitely need to keep a clear view of the principles the project was founded for in order for Backdrop to positively differentiate itself.

Affordability is one potential differentiator, and I am personally happy to see it’s one already embodied by Backdrop’s promises of backwards compatibility and low server requirements. But, frankly, WordPress has got those things already. An objective evaluation of Backdrop’s affordability in its current form would put it on par with, but not appreciably better than a more established competitor. But there is hope, because:

Current CMSs don’t give site owners what’s best for them

To make converts, Backdrop could truly differentiate itself with a pledge to offer two new things:

  1. Security backports for previous releases, with a clearly published end-of-support date on each release, so site owners can address security issues with confidence that nothing else about their site will change or stop working.
  2. A reference implementation for fully automated updates, so that installed sites stay secure with zero effort.

Here’s why.

Let’s assume there’s a certain total effort required to create and maintain that highly customized website described by Backdrop’s mission statement. Who exerts that effort is more or less transferable between the site builder and the CMS contributors. On one extreme, the site owner could eschew a CMS entirely and take on all the effort themselves. Nobody does this, because of the colossal duplication of exertions that would result from all the sites re-solving many of the same problems. Whenever a CMS takes over even a small task from site builders, a mind-boggling savings in total human hours results.

Consider this fact in the context of CMS security. From a site owner’s perspective, here’s the simplest-case current model in Drupal, Backdrop, WordPress, and other popular open-source CMS’s for keeping the sites they run secured over time. This model assumes a rather optimistic site owner who isn’t worried enough about the risk of code changes breaking their site to maintain a parallel development environment; organizations choosing to do this incur even more steps and costs.

  1. A responsible individual keeps an eye out for security updates to be released. There are myriad ways to be notified of them, but somebody has to be there to receive the notifications.
  2. When the individual and/or organization is ready, an action is performed to apply the latest version of the code, containing the security update as well as non-security changes the developers have made since this site was last updated. Sometimes this action is as simple as clicking a button, but it cannot be responsibly automated beyond this point due to the need to promptly perform step 3.
  3. The site owner browses around on their site, checking key pages and functions, to find any unintended consequences of the update before their customers do. (For non-enterprise, smaller sites, let’s assume an automated testing suite providing coverage of the highly customized features does not exist. Alternatively, if it did exist, there would be a cost to creating it that needs to be duplicated for each instance of the CMS in use.)
  4. In practice, the developers usually did their job correctly, and the update does not have unintended consequences. But sometimes, something unexpected happens, and a cost is incurred to return the site to its past level of operation, assuming funds are available.

Once a site has been developed, unsolicited non-security changes to the code rarely add business value to the organization operating the site. In the current model, however, changes are forced on organizations anyway as a necessary component of maintaining a secure site, merely because they are packaged along with the security update. In my opinion, the boldface observation above ought to be recognized as one of the principles guiding Backdrop’s philosophy. In the classic model, the CMS avoids a small task of backporting the security fix to past releases and the work is transferred to site owners in the form of the above steps. That’s expense for the site owner, and in total it is multiplied by each site the CMS runs – a much larger figure than offering a backport would have amounted to.

This is a clear shortcoming of current offerings, and Backdrop’s focus on affordability makes it a ripe candidate for breaking this mold. Not to mention the value proposition it would create for organizations evaluating their CMS options. Heck, make a badge and stick it on backdropcms.org/releases:

Supported

3 years

Stable

Backdrop could guarantee security updates will not disrupt the sites they run; the competition could only say “A security update is available. You should update immediately. It is not advisable to update your production site without testing the update first, because we introduced lots of other moving parts and can’t make any guarantees. Good luck.”

That’s something I think developers and non-technical decision makers alike can appreciate, and that would make Backdrop look more attractive. Don’t want to pay monthly dues to Squarespace? Don’t want to pay periodic, possibly unpredictable support fees to a developer? Now you can, on Backdrop.

The above case that a software support lifecycle would make site maintenance more affordable to site owners does not even begin to take into consideration the reality that many sites simply are not updated in a timely fashion because the updates aren’t automated. If you are not an enterprise with in-house IT staff, and you are not paying monthly premiums to an outfit like Squarespace or a high-end web host with custom application-layer firewalls, history shows a bot is pretty guaranteed to own that site well before you get around to fixing it. Exploited sites are in turn used to spread malware to their visitors, so adding automated updates to the CMS that can be safely applied, rapidly, without intervention would have a far-reaching overall impact on Internet security.

But how achievable is this?

Isn’t extended support boring to developers volunteering their time?

Yes, probably. Top contributors might not be too psyched to do backports. But just as in a for-profit development firm, developers with a range of abilities are all involved in creating open-source software. Have top contributors or members of the security team write the fix and corresponding tests against the latest release, and let others merge them back. The number of patches written for Drupal core which have never been merged has I think demonstrated that developer hours are eminently available, even when the chance of those hours having any impact is low. Propose to someone that their efforts will be reflected on hundreds or thousands of sites across the Internet in a matter of days, and you’ll get some volunteers. Novice contributors show up weekly in #drupal-contribute happy to learn how to reroll patches as it is. Security issues might be slightly more tricky in that care needs to be taken to limit their exposure to individuals whose trust has been earned, but this is totally doable. Given the frequency of core security releases in Drupal 7, a smaller pool of individuals known personally by more established community members could be maintained on a simple invite-only basis.

Update, April 2017
I discussed how achievable backports within major versions of Drupal could be in a core conversation at DrupalCon Baltimore. The focus related especially to the possibility of extending the model where official vendors have access to confidential security information to support Drupal 6 LTS. Participants included many members of the security team and a few release managers; the youtube is here.

Some interesting possibilities exist around automating attempts to auto-merge fixes through the past releases and invoke the test suite, but rigging up this infrastructure wouldn’t even be an immediate necessity.

Also, other projects in the wider, and even PHP FOSS world show us it can be done. Ubuntu made software lifecycles with overlapping supported versions famous for an entire Linux distribution (though most all of the other distros also pull it off with less fanfare, chalking it up as an implicit necessity), and it’s even been embraced by Symfony, the PHP framework deeply integrated into Drupal 8. While Drupal adopted  Symfony’s software design patterns and frequently cites this as one of Drupal 8’s strengths, they didn’t adopt Symfony’s software lifecycle practices. In this regard, Drupal is increasingly finding itself “on the island.” Hey, if Backdrop started doing it, maybe Drupal would give in eventually too.

What about contributed modules?

I would argue that a primary strategy to handle the issue of contrib code should be to reduce the amount of contrib code.  This fits well with a Backdrop initiative to put the things most people want in the core distribution of the product.  A significant number of sites – importantly, the simplest ones that are probably least inclined to think about ongoing security – would receive complete update coverage were backports provided only for core. The fact that contrib is there in CMS ecosystems is sometimes cited as a reason security support lifecycles would not be possible, but it’s no excuse not to tackle it in the CMS’s core.

Contrib will always be a major part of any CMS’s ecosystem though, and shouldn’t be left out of the opportunity to participate in support lifecycles.  I would propose that infrastructure be provided for contrib developers to clearly publish their own support intentions on their project pages. Then, when a security issue is disclosed in a contrib module, the developer would identify, by means of simple checkboxes, the versions that are affected. There would be no obligation to actually produce a patched version of old releases identified as vulnerable, however, regardless of previously published intentions. This would have two effects: A) the developer would be reminded that they committed to do something, and therefore might be more likely to do it, and B) sufficient data would be available to inform site owners of a security issue requiring their attention if the contrib module chose not to provide a backported fix. Eventually, the data might also be used as a statistic on project pages to aid site builders in selecting modules with a good support track record.

Aren’t automated updates inherently insecure?

No, although some web developers may conflate performing an automatic update with the risks of allowing code to modify other code when it can be invoked by untrusted users. A reference implementation of an automatic updater would be a separate component from the CMS, capable of running with a different set of permissions from the CMS itself.

Brief case study: “Drupalgeddon”

Here’s the patch, including its test coverage, that fixed what probably proved to be the most impactful security vulnerability in Drupal 7’s history to date:

drupalgeddon

The fix itself is a one-line change in database.inc. Security patches are, as in this case, often very small and only have any impact on a site’s behavior in the face of malicious inputs. That’s why there’s value in packaging them separately.

Drupal 7.32, the version that fixed this vulnerability, was released in October 2014. A

git apply -3 drupalgeddon.patch

is able to automatically apply both the database.inc and database_test.test changes all the way back to Drupal 7.0, which was released almost four years earlier in January 2011. Had the infrastructure been in place, this fix could have been automatically generated for all earlier Drupal versions, automatically verified with the test suite, and automatically distributed to every Drupal 7 website with no real added effort on the part of the CMS, and no effort on the part of site owners. Instead, in the aftermath, tremendous time energy and money was expended by the site owners that were affected or compromised by it, with those that didn’t patch in a matter of hours facing the highest expenses to forensically determine if they were compromised and rectify all resulting damages.

You better believe botnet operators maintain databases of sites by CMS, and are poised to use them to effectively launch automated exploits against the bulk of the sites running any given CMS within hours of the next major disclosure. So, unless the CMSs catch up, it is not a matter of if this will happen again, but when.

The only way to beat the automation employed by hackers is for the good guys to employ some automation of their own to get all those sites patched. And look how easy it is. Why are we not doing this.

Final thoughts

A CMS that offered an extended support lifecycle on their releases would make site ownership more affordable and simpler, and would improve overall Internet security. Besides being the right thing to do, if it made these promises to its users, Backdrop would be able to boast of a measurable affordability and simplicity advantage. And advantages are critical for the new CMS in town vying for market share in a crowded and established space.