Monthly Archives: June 2013

Harnessing The Power of Versions On App Engine

One powerful, but often overlooked, feature of running applications on App Engine is the ability to have multiple versions of your application deployed and running simultaneously. This feature is invaluable when it comes to being able to “smoke test” a new version of your site live in production before your users can see it. Or you can use traffic splitting to allow you to test different versions of your site against live traffic in a controlled and deterministic fashion.

The release of WordPress 3.5.2 is a great opportunity for us to demonstrate how to use versions to do a controlled upgrade of our WordPress blog to the latest version.

A Note About App Engine Versions

Before we get started on the steps for running multiple versions of your application, we should briefly describe application versions and the default version. App Engine allows you to deploy up to 10 versions of your application to the production servers. Versions are completely independent of each other, and in fact versions can even be using different runtimes – so you can deploy a PHP application and a Java application as different versions of the same appspot application and run them simultaneously. Versions are identified by the version string that you specify either in the app.yaml file or as a command line paramater when using the appcfg.py tool. If you specify the version as a command line option then it will override the version that is supplied in the app.yaml file.

The first time that you upload your application to the production servers, it will become the default version, and all user requests will be served using that version. If you upload a new version of your application with a new version identifier, then it will not serve any user requests until you go into the admin console and either mark it as the default version or enable traffic splitting (more detail about both are later in this post). If you use a version identifier that matches an already uploaded version, then it will overwrite the old version with the new one. If that version happens to be the default version, then it will start serving immediately.

Now we know a little about versions, we can go about using them to upgrade our version of WordPress we’re using.

Step 1: Uploading The New Version Of Your Site.

The first step to running a new version is actually uploading the updated version of your site to the production servers. The steps outlined below assume that you have setup your WordPress source code following the instructions on the Running WordPress page. We will get the latest version of WordPress and unpack it to the APPLICATION_DIRECTORY directory as mentioned in that article, using the following steps.

$ wget http://wordpress.org/latest.zip
$ unzip -o latest.zip -d APPLICATION_DIRECTORY

You should now run the site in your development environment to ensure that it behaves as you expect it to (once again, following the instructions on the Running WordPress page).

$ APP_ENGINE_SDK_PATH/dev_appserver.py --php_executable_path=PHP_CGI_EXECUTABLE_PATH APPLICATION_DIRECTORY

Once you’re satisfied that the site is running as you would expect in the development environment, you can upload it to the production servers. We’ll use the --version flag of appcfg.py to upload the site with a new version number, that differs from the exiting version. In this example, we’ll use today’s date and the postfix “-352″ as the version identifier.

$ appcfg.py -R --version 20130624-352 update .

The appcfg tool will now upload a new version of your site. One interesting thing to note is that the tool will only upload files to the server that are different to existing files that you’ve previously uploaded in an existing version. This results in very fast uploads for new versions of large site that have only a small number of changes to the source.

You should see output similar to what’s shown below

10:53 AM Host: appengine.google.com
10:53 AM Application: gae-php-tips; version: 20130624-352 (was: wp-0501)
10:53 AM Starting update of app: gae-php-tips, version: 20130624-352
10:53 AM Getting current resource limits.
10:53 AM Scanning files on local disk.
...
10:53 AM Checking if deployment succeeded.
10:53 AM Deployment successful.
10:53 AM Checking if updated app version is serving.
10:53 AM Completed update of app: gae-php-tips, version: 20130624-352
10:53 AM Uploading cron entries.

You can now go to the admin console for you site and view the new version that was just uploaded.

Screen Shot 2013-06-24 at 11.02.41 AM

As you can see from the screen shot, the new version has successfully uploaded, but our site is still serving live traffic from the existing version labelled “17062013”.

Step 2: Manually Testing The New Version

App Engine versions allow you to access any of the deployed versions of your site by prepending the correct version specifier to the site address. In our example here, I can access the new version of http://gae-php-tips.appspot.com/ by entering the url http://20130624-352-dot-gae-php-tips.appspot.com/ in the address bar of my browser.

Note: We use this feature to deploy phpMyAdmin side by side with WordPress in our application. You can see a version named “phpmyadmin” in the list of deployed versions.

Step 3: Traffic Splitting Live Traffic To The New Version.

Now that we’ve deployed the new version of the site and manually tested that it serves as we expect, we can now start sending it live traffic. We have two options for serving traffic from this applcation.

  • Make the new version the default version so that it handles all requests for the site.
  • Use traffic splitting to send a percentage of users to the new version, to assess its stability before making it the default.

In this example, we’ll use traffic splitting to ensure that the new version of the site is stable before making it the default. To get started, click on the traffic splitting link on the versions page for your site. It will present the option to enable traffic splitting which will look like this.

Screen Shot 2013-06-24 at 11.17.03 AM

From here click on the “Add Traffic Split…” button to configure traffic splitting. You will be asked to select the version that you want to split traffic to, and the percentage of traffic to split to the new version. Select the version that you just uploaded and the amount of traffic you want to send it (in my case, 25%).

Screen Shot 2013-06-24 at 11.17.26 AM

Once you click on OK App Engine will enable traffic splitting using the configuration that you’ve supplied. A reminder that traffic splitting is enabled will be shown on your admin console page for the site.

Screen Shot 2013-06-24 at 11.17.42 AM

Your site is now serving a percentage of user requests with the new version of your site.

Step 4: Monitoring The New Version.

You can use the application logs to verify that the new version of your site is serving without errors. At the top of the logs page it allows you to view logs for a specific  version. Using this, you can quickly drill down if the new version of your site is serving correctly.

Screen Shot 2013-06-24 at 11.31.34 AM

You can see that these are the logs for the versions 17062013 which is currently serving 75% of the traffic for the site, while below are the logs for a new version 20130624-352 which is serving 25% of the traffic for our site.

Screen Shot 2013-06-24 at 11.30.58 AM

Step 5: Making The New Version The Default

Once we’re satisfied that the new version of the site is performing correctly, we can switch it over to be the default for all user requests.To do this, we go back to the versions page in the admin console for our site. By selecting the radio button next to the new version and pressing Make Default, the traffic split will be dropped and the new version will handle all live traffic.

Screen Shot 2013-06-24 at 11.38.29 AM

You could also roll back to the older version of the app by removing the traffic split, or change the split percentage if you want to gradually increase the traffic load on the new version over time.

Notes About Running Multiple Versions

One important thing to consider when running multiple versions of your site, is that each different version of your site requires at least one running instance to serve from. These instances are charged independently, so that if you are traffic splitting on a free site you are likely to run out of the free quota during one 24 hour billing period.

Another thing to consider when running multiple versions is that all versions share the same database. If there is a schema change to the database for a new version, the older versions may not function correctly. In our example we upgraded from WordPress 3.5.1 to 3.5.2 which uses the same schema so we could serve both versions from the same database. If you want to run multiple versions that have different schemas then you use separate databases for each version. You could use the backup and restore functionality of CloudSQL create a copy of your live database for the new version.

Announcing the Google App Engine WordPress plugin

Screen Shot 2013-06-13 at 5.23.09 PM

Today we officially launched the Google App Engine Plugin for WordPress. We’ve also put the source of the plugin on github for those of you who might be inclined to want to modify of add functionality.

The plugin wires together some code WordPress features with App Engine APIs, for now including:

  • Sending e-mail using the Mail API.
  • Uploading and serving media files to Google Cloud Storage.

Full details on how to install the plugin, as well as optimally configure your WordPress site can be found in this detailed post on developers.google.com.

 

Tips For Configuring The Plugin

The plugin is very easy to configure. Currently there are only two options that you need to be concerned with, EMail Settings and Upload Settings.

Email Settings

Screen Shot 2013-06-13 at 8.19.42 PM

To enable sending of Email using the App Engine Mail API select the checkbox “Use App Engine Email Service”. By default, mail will be sent from wordpress@.appspot.com, but you can change the address to one of the allowed sender addressed as described here.

Upload Settings

Screen Shot 2013-06-13 at 8.24.47 PM

To configure the upload settings, all you need to provide the name of a Google Cloud Storage bucket that the application has permission to write to. You can follow the instructions here to correctly associate your application to a bucket. Once you have done this, simply enter the bucket name in the correct field to enable uploads to that bucket.

When you try and save the settings, the plugin will attempt to write a test file to the bucket to ensure that your application has write permissions. If your application cannot write to the bucket you will be shown a warning at the top of the configuration screen, as shown below.

Screen Shot 2013-06-13 at 8.24.37 PM

Once your configuration settings are correct press save, and your WordPress site will be ready to send mail and handle media uploads.

What’s new for PHP in the App Engine 1.8.1 Release

Today we publicly announced the 1.8.1 release of Google App Engine, and as part of this release we’ve added a bunch of new features for the PHP runtime. The highlights of what’s new includes:

  • Three more runtime extensions:
  • The ability to include and/or require files PHP scripts from Google Cloud Storage.
  • Better support for url_stat of items in Google Cloud Storage.
  • Support for application files with ‘@’ in their name.
  • Honor the default_stream_context() when using the Google Cloud Storage stream wrapper.
  • Added the CloudStorageTools::deleteImageServiceUrl() call for the high speed image serving service.
  • Fixed a bug where $_SERVER[“PHP_SELF”] was including the query string of the URL.
  • Removed the Capabilities API.
  • Simplified the types of exceptions that can be thrown from the Users API.

Including Files From Google Cloud Storage

As mentioned, with this release we’ve made it possible to include and/or require script files from Google Cloud Storage.

To enable this feature, you must list the buckets that you want to include files from in your applications php.ini file. The bucket names can be specified as a comma separated list to the google_app_engine.allow_include_gs_buckets ini setting.

google_app_engine.allow_include_gs_buckets = "my_bucket"

Once you have configured the bucket, you can then include the files directly in your PHP application code.

<?php

require_once "gs://my_bucket/file.php";

The ideal use case for storing script files in Google Cloud Storage is for the intermediate files generated by various templating frameworks. Accessing script files from Google Cloud Storage is slower than accessing files that have been uploaded as part of you application and these files cannot take advantage of the powerful versioning system that App Engine provides.

Caching WordPress for App Engine

Caching is one of the simplest ways to improve the performance of your WordPress site. There are two simple, easy to use caching techniques that you can leverage when hosting a WordPress site on App Engine.

Memcache

App Engine has a zero configuration memcache service that can be used in conjunction with the memcached plugin, that reduces the number of database calls that are required to display a page. Batcache uses the memcache service to store and serve fully rendered pages. It has a number of tuning options that can be used to configure caching to not only reduce the time taken to serve pages, but the overall cost to run your site.

To get started using memcache and WordPress, download and install the plugins like this

$ wget http://downloads.wordpress.org/plugin/memcached.2.0.2.zip
$ unzip memcached.2.0.2.zip -d /path/to/wordpress/wp-content/
$ wget http://downloads.wordpress.org/plugin/batcache.1.2.zip
$ unzip batcache.1.2.zip -d /path/to/wordpress/wp-content/plugins/

Now edit your wp-config.php file and enable caching, and tune the batcache settings if you want.

define('WP_CACHE', true);
$batcache = [
    'seconds' => 0,
    'debug' => false,
    'max_age' => 60 * 30,  // 30 minutes.
];

Setting a large max_age value will dramatically reduce the database access of your application, at the price of clients potentially seeing stale pages. If you site has rarely changing content then this can be an effective way to reduce the overall cost of your application as it will reduce the usage of the CloudSQL database.

Now all that’s left is to use appcfg.py to push your application to production, and your WordPress site will start using the memcache service.You can use the admin console memcache viewer to see how many objects have been cached, how often there is a cache hit and flush the cache if required.

Static File Caching

When serving static files, App Engine will automatically add caching headers so that the resources can be cached by the users browser and any intermediate proxies. By default, the cache expiration time is 10 minutes. However, you can set this expiration time to a higher value if you have resources that rarely, if ever change.

The easiest way to do this is to set the default_expiration value in the app.yaml file of your WordPress site.

application: YOUR_PROJECT_ID
version: wp
runtime: php
api_version: 1

default_expiration: "1d"

Note: Be careful how you set the default expiration value. If you set a large expiration time like 365d then any changes to the content of the static resources will likely not be visible to users as the old content will still be cached in upstream proxies.

Using both of these techniques can result in vastly improved page load times for your site, combined with a reduction in the CloudSQL usage and therefor cost.