Author Archives: slangley

What’s new for PHP runtime in the App Engine 1.8.4 release

Today we announced the 1.8.4 release of App Engine and although we’ve been busy working behind the scenes on a bunch of under the covers improvements, we’ve still managed to roll out a few nice new features and fixes for the PHP runtime. Once again, it’s worthwhile highlighting a few of these items.

The PHP interpreter was upgraded from PHP 5.4.8 to PHP 5.4.17

This picks up a lot of bug fixes from the PHP core team. For a detailed list please check the PHP5 Change Log.

The is_writable() method now supports Google Cloud Storage files and buckets.

This new feature brings us close to having Google Cloud Storage working similar to what is expected of a local file system for PHP applications. Supporting is_writable is an interesting use case – to be absolutely sure that the application has permission to write to a bucket, we have to start the process of writing an actual file to the bucket, and then check the result. As this is a comparatively time-consuming process, we cache the result in memcache the first time you call is_writable on a bucket. For example:

$my_bucket = 'gs://my_bucket/';
// Will make a request to GCS and cache the result.
$check1 = is_writable($my_bucket);
// Will retrieve the value of the previous call from the cache.  
$check2 = is_writable($my_bucket); 

Naturally this means that if you change the ACL on your Google Cloud Storage bucket, you app could read the incorrect value of is_writable from the cache. ACLs on Google Cloud Storage buckets are usually changed infrequently, so the likelihood of your app using a stale cached result is low. However, you can always flush memcache from your app’s admin console to clear the cached result.

By default the value is cached for 10 minutes, however you can change this by setting the 'writable_cache_expiry_seconds' value in the stream context configuration for Google Cloud Storage.

// Set a long writable cache timeout, as we never change the ACL on the bucket.
stream_context_set_default(['gs' => ['writable_cache_expiry_seconds' => '86400']]);

You no longer need to specify the PHP runtime when deploying applications

You longer need to specify -R when using appcfg.py to deploy php apps.

# Before
$ appcfg.py -R -A myapp update path/to/my/app/

# Now
$ appcfg.py -A myapp update path/to/my/app/

What’s new for PHP runtime in the App Engine 1.8.3 release

Today we announced the 1.8.3 release of App Engine, which contains a bunch of spiffy new features and fixes for the PHP runtime. It’s worthwhile highlighting some of the new features and changes in this release.

Whitelisted users are now able to deploy PHP to any of their apps.

If your user account has previously been whitelisted for a PHP application, then you can now use PHP with any of your applications, including new ones. This includes applications located in the European Union.

Environment variables that are set in app.yaml are now available in the PHP runtime.

You can now pass environment variables to your application by defining them in your app.yaml file. These variables will be available using the getenv function call or from the $_SERVER superglobal.

To configure environment variables, add them to your app.yaml file using the env_variables section.

env_variables:
foo: 'bar'
baz: 'zoo'

Your application will then be able to read these environment variables back at runtime.

>>> echo getenv('foo');
bar

Read-through cache for Google Cloud Storage

We’ve added a read-through cache for Google Cloud Storage files that are being accessed through our stream wrapper. By default, we cache files as you read them in memcache, and on subsequent reads we will retrieve the file from memcache and return the contents if they have not changed.

To determine if the contents of a file have changed or not, we make a request to Google Cloud Storage. So in the case of a file that has not changed, we don’t need to retrieve the contents, but we still need to incur the penalty of at least 1 round trip to Google Cloud Storage.

If you have an application where you know the files are not going to change once they’re written, or you’re prepared to occasionally be served stale file contents, then you can skip the contents change check by enabling optimistic caching of the file. With optimistic caching on, if we find the file in memcache, we will return the contents without checking that the file is still current, so no additional roundtrip call to Google Cloud Storage is performed.

You can configure read caching using the stream context options that you pass when opening the file.

$gs_options = [ 'gs' => [
      "enable_cache" => true,  // true by default, false disables all read caching
      "enable_optimistic_cache" => true,  // false by default, true enables optimistic caching
      "read_cache_expiry_seconds" => 3600,  // Number of seconds that reads are cached in memcache.
]];
stream_context_set_default($gs_options);

We will invalidate the read cache if you write to the file from your application, so that it is cached again on the next read. Note that we don’t invalidate the cache if the file is changed from outside of your application by directly uploading to the bucket, or on a rename.

Using the Google APIs Client Library for PHP with App Engine

Important Note: Newer releases of the Google APIs client have support for App Engine out of the box, making this article obsolete.

Download the latest version from the github repository.

The Google API Client library allows you to work with Google APIs such as Analytics, Adsense, Google+, Calendar, Drive or Games easily from your application. Recent versions of the client library include support for HTTP streams, which makes it simple to use the library from an App Engine application just by tweaking the configuration file.

Follow these steps to get the client library up and running.

  1. Download the latest release package of the client.
  2. Extract the library from the archive and copy the google-api-php-client directory to your project root.
  3. Edit the file google-api-php-client/src/config.php, and configure the client to use HTTP streams and memcache by making the following changes to the $apiConfig array.
        // Which Authentication, Storage and HTTP IO classes to use.
        'authClass'    => 'Google_OAuth2',
        'ioClass'      => 'Google_HttpStreamIO',
        'cacheClass'   => 'Google_MemcacheCache',
    
        // We need to configure fake values for memcache to work
        'ioMemCacheCache_host' => 'does_not_matter',
        'ioMemCacheCache_port' => '37337',
    
  4. Test that the client is working correctly by following the Google Plus example on the client home page.

Note that to use the API client from your application, you’ll need to follow the instructions for using OAuth 2.0 with your application.

In a follow-up post we’ll show you how to use the Google API client to access the Google Cloud Datastore from your application.

Harnessing The Power of Versions On App Engine

One powerful, but often overlooked, feature of running applications on App Engine is the ability to have multiple versions of your application deployed and running simultaneously. This feature is invaluable when it comes to being able to “smoke test” a new version of your site live in production before your users can see it. Or you can use traffic splitting to allow you to test different versions of your site against live traffic in a controlled and deterministic fashion.

The release of WordPress 3.5.2 is a great opportunity for us to demonstrate how to use versions to do a controlled upgrade of our WordPress blog to the latest version.

A Note About App Engine Versions

Before we get started on the steps for running multiple versions of your application, we should briefly describe application versions and the default version. App Engine allows you to deploy up to 10 versions of your application to the production servers. Versions are completely independent of each other, and in fact versions can even be using different runtimes – so you can deploy a PHP application and a Java application as different versions of the same appspot application and run them simultaneously. Versions are identified by the version string that you specify either in the app.yaml file or as a command line paramater when using the appcfg.py tool. If you specify the version as a command line option then it will override the version that is supplied in the app.yaml file.

The first time that you upload your application to the production servers, it will become the default version, and all user requests will be served using that version. If you upload a new version of your application with a new version identifier, then it will not serve any user requests until you go into the admin console and either mark it as the default version or enable traffic splitting (more detail about both are later in this post). If you use a version identifier that matches an already uploaded version, then it will overwrite the old version with the new one. If that version happens to be the default version, then it will start serving immediately.

Now we know a little about versions, we can go about using them to upgrade our version of WordPress we’re using.

Step 1: Uploading The New Version Of Your Site.

The first step to running a new version is actually uploading the updated version of your site to the production servers. The steps outlined below assume that you have setup your WordPress source code following the instructions on the Running WordPress page. We will get the latest version of WordPress and unpack it to the APPLICATION_DIRECTORY directory as mentioned in that article, using the following steps.

$ wget http://wordpress.org/latest.zip
$ unzip -o latest.zip -d APPLICATION_DIRECTORY

You should now run the site in your development environment to ensure that it behaves as you expect it to (once again, following the instructions on the Running WordPress page).

$ APP_ENGINE_SDK_PATH/dev_appserver.py --php_executable_path=PHP_CGI_EXECUTABLE_PATH APPLICATION_DIRECTORY

Once you’re satisfied that the site is running as you would expect in the development environment, you can upload it to the production servers. We’ll use the --version flag of appcfg.py to upload the site with a new version number, that differs from the exiting version. In this example, we’ll use today’s date and the postfix “-352″ as the version identifier.

$ appcfg.py -R --version 20130624-352 update .

The appcfg tool will now upload a new version of your site. One interesting thing to note is that the tool will only upload files to the server that are different to existing files that you’ve previously uploaded in an existing version. This results in very fast uploads for new versions of large site that have only a small number of changes to the source.

You should see output similar to what’s shown below

10:53 AM Host: appengine.google.com
10:53 AM Application: gae-php-tips; version: 20130624-352 (was: wp-0501)
10:53 AM Starting update of app: gae-php-tips, version: 20130624-352
10:53 AM Getting current resource limits.
10:53 AM Scanning files on local disk.
...
10:53 AM Checking if deployment succeeded.
10:53 AM Deployment successful.
10:53 AM Checking if updated app version is serving.
10:53 AM Completed update of app: gae-php-tips, version: 20130624-352
10:53 AM Uploading cron entries.

You can now go to the admin console for you site and view the new version that was just uploaded.

Screen Shot 2013-06-24 at 11.02.41 AM

As you can see from the screen shot, the new version has successfully uploaded, but our site is still serving live traffic from the existing version labelled “17062013”.

Step 2: Manually Testing The New Version

App Engine versions allow you to access any of the deployed versions of your site by prepending the correct version specifier to the site address. In our example here, I can access the new version of http://gae-php-tips.appspot.com/ by entering the url http://20130624-352-dot-gae-php-tips.appspot.com/ in the address bar of my browser.

Note: We use this feature to deploy phpMyAdmin side by side with WordPress in our application. You can see a version named “phpmyadmin” in the list of deployed versions.

Step 3: Traffic Splitting Live Traffic To The New Version.

Now that we’ve deployed the new version of the site and manually tested that it serves as we expect, we can now start sending it live traffic. We have two options for serving traffic from this applcation.

  • Make the new version the default version so that it handles all requests for the site.
  • Use traffic splitting to send a percentage of users to the new version, to assess its stability before making it the default.

In this example, we’ll use traffic splitting to ensure that the new version of the site is stable before making it the default. To get started, click on the traffic splitting link on the versions page for your site. It will present the option to enable traffic splitting which will look like this.

Screen Shot 2013-06-24 at 11.17.03 AM

From here click on the “Add Traffic Split…” button to configure traffic splitting. You will be asked to select the version that you want to split traffic to, and the percentage of traffic to split to the new version. Select the version that you just uploaded and the amount of traffic you want to send it (in my case, 25%).

Screen Shot 2013-06-24 at 11.17.26 AM

Once you click on OK App Engine will enable traffic splitting using the configuration that you’ve supplied. A reminder that traffic splitting is enabled will be shown on your admin console page for the site.

Screen Shot 2013-06-24 at 11.17.42 AM

Your site is now serving a percentage of user requests with the new version of your site.

Step 4: Monitoring The New Version.

You can use the application logs to verify that the new version of your site is serving without errors. At the top of the logs page it allows you to view logs for a specific  version. Using this, you can quickly drill down if the new version of your site is serving correctly.

Screen Shot 2013-06-24 at 11.31.34 AM

You can see that these are the logs for the versions 17062013 which is currently serving 75% of the traffic for the site, while below are the logs for a new version 20130624-352 which is serving 25% of the traffic for our site.

Screen Shot 2013-06-24 at 11.30.58 AM

Step 5: Making The New Version The Default

Once we’re satisfied that the new version of the site is performing correctly, we can switch it over to be the default for all user requests.To do this, we go back to the versions page in the admin console for our site. By selecting the radio button next to the new version and pressing Make Default, the traffic split will be dropped and the new version will handle all live traffic.

Screen Shot 2013-06-24 at 11.38.29 AM

You could also roll back to the older version of the app by removing the traffic split, or change the split percentage if you want to gradually increase the traffic load on the new version over time.

Notes About Running Multiple Versions

One important thing to consider when running multiple versions of your site, is that each different version of your site requires at least one running instance to serve from. These instances are charged independently, so that if you are traffic splitting on a free site you are likely to run out of the free quota during one 24 hour billing period.

Another thing to consider when running multiple versions is that all versions share the same database. If there is a schema change to the database for a new version, the older versions may not function correctly. In our example we upgraded from WordPress 3.5.1 to 3.5.2 which uses the same schema so we could serve both versions from the same database. If you want to run multiple versions that have different schemas then you use separate databases for each version. You could use the backup and restore functionality of CloudSQL create a copy of your live database for the new version.

Announcing the Google App Engine WordPress plugin

Screen Shot 2013-06-13 at 5.23.09 PM

Today we officially launched the Google App Engine Plugin for WordPress. We’ve also put the source of the plugin on github for those of you who might be inclined to want to modify of add functionality.

The plugin wires together some code WordPress features with App Engine APIs, for now including:

  • Sending e-mail using the Mail API.
  • Uploading and serving media files to Google Cloud Storage.

Full details on how to install the plugin, as well as optimally configure your WordPress site can be found in this detailed post on developers.google.com.

 

Tips For Configuring The Plugin

The plugin is very easy to configure. Currently there are only two options that you need to be concerned with, EMail Settings and Upload Settings.

Email Settings

Screen Shot 2013-06-13 at 8.19.42 PM

To enable sending of Email using the App Engine Mail API select the checkbox “Use App Engine Email Service”. By default, mail will be sent from wordpress@.appspot.com, but you can change the address to one of the allowed sender addressed as described here.

Upload Settings

Screen Shot 2013-06-13 at 8.24.47 PM

To configure the upload settings, all you need to provide the name of a Google Cloud Storage bucket that the application has permission to write to. You can follow the instructions here to correctly associate your application to a bucket. Once you have done this, simply enter the bucket name in the correct field to enable uploads to that bucket.

When you try and save the settings, the plugin will attempt to write a test file to the bucket to ensure that your application has write permissions. If your application cannot write to the bucket you will be shown a warning at the top of the configuration screen, as shown below.

Screen Shot 2013-06-13 at 8.24.37 PM

Once your configuration settings are correct press save, and your WordPress site will be ready to send mail and handle media uploads.

What’s new for PHP in the App Engine 1.8.1 Release

Today we publicly announced the 1.8.1 release of Google App Engine, and as part of this release we’ve added a bunch of new features for the PHP runtime. The highlights of what’s new includes:

  • Three more runtime extensions:
  • The ability to include and/or require files PHP scripts from Google Cloud Storage.
  • Better support for url_stat of items in Google Cloud Storage.
  • Support for application files with ‘@’ in their name.
  • Honor the default_stream_context() when using the Google Cloud Storage stream wrapper.
  • Added the CloudStorageTools::deleteImageServiceUrl() call for the high speed image serving service.
  • Fixed a bug where $_SERVER[“PHP_SELF”] was including the query string of the URL.
  • Removed the Capabilities API.
  • Simplified the types of exceptions that can be thrown from the Users API.

Including Files From Google Cloud Storage

As mentioned, with this release we’ve made it possible to include and/or require script files from Google Cloud Storage.

To enable this feature, you must list the buckets that you want to include files from in your applications php.ini file. The bucket names can be specified as a comma separated list to the google_app_engine.allow_include_gs_buckets ini setting.

google_app_engine.allow_include_gs_buckets = "my_bucket"

Once you have configured the bucket, you can then include the files directly in your PHP application code.

<?php

require_once "gs://my_bucket/file.php";

The ideal use case for storing script files in Google Cloud Storage is for the intermediate files generated by various templating frameworks. Accessing script files from Google Cloud Storage is slower than accessing files that have been uploaded as part of you application and these files cannot take advantage of the powerful versioning system that App Engine provides.

Caching WordPress for App Engine

Caching is one of the simplest ways to improve the performance of your WordPress site. There are two simple, easy to use caching techniques that you can leverage when hosting a WordPress site on App Engine.

Memcache

App Engine has a zero configuration memcache service that can be used in conjunction with the memcached plugin, that reduces the number of database calls that are required to display a page. Batcache uses the memcache service to store and serve fully rendered pages. It has a number of tuning options that can be used to configure caching to not only reduce the time taken to serve pages, but the overall cost to run your site.

To get started using memcache and WordPress, download and install the plugins like this

$ wget http://downloads.wordpress.org/plugin/memcached.2.0.2.zip
$ unzip memcached.2.0.2.zip -d /path/to/wordpress/wp-content/
$ wget http://downloads.wordpress.org/plugin/batcache.1.2.zip
$ unzip batcache.1.2.zip -d /path/to/wordpress/wp-content/plugins/

Now edit your wp-config.php file and enable caching, and tune the batcache settings if you want.

define('WP_CACHE', true);
$batcache = [
    'seconds' => 0,
    'debug' => false,
    'max_age' => 60 * 30,  // 30 minutes.
];

Setting a large max_age value will dramatically reduce the database access of your application, at the price of clients potentially seeing stale pages. If you site has rarely changing content then this can be an effective way to reduce the overall cost of your application as it will reduce the usage of the CloudSQL database.

Now all that’s left is to use appcfg.py to push your application to production, and your WordPress site will start using the memcache service.You can use the admin console memcache viewer to see how many objects have been cached, how often there is a cache hit and flush the cache if required.

Static File Caching

When serving static files, App Engine will automatically add caching headers so that the resources can be cached by the users browser and any intermediate proxies. By default, the cache expiration time is 10 minutes. However, you can set this expiration time to a higher value if you have resources that rarely, if ever change.

The easiest way to do this is to set the default_expiration value in the app.yaml file of your WordPress site.

application: YOUR_PROJECT_ID
version: wp
runtime: php
api_version: 1

default_expiration: "1d"

Note: Be careful how you set the default expiration value. If you set a large expiration time like 365d then any changes to the content of the static resources will likely not be visible to users as the old content will still be cached in upstream proxies.

Using both of these techniques can result in vastly improved page load times for your site, combined with a reduction in the CloudSQL usage and therefor cost.
 

Setting up phpMyAdmin on App Engine

Note: This article is obsolete. Current instructions for installing phpMyAdmin can be found here.

I’ve had a few requests for publishing how to get phpMyAdmin up and running. No doubt we’ll add this as a tutorial to the developers.google.com soon but in the meantime I thought I’d add it here.

    1. Download the phpMyAdmin source code from here.
    2. Create a directory named phpMyAdmin and move the downloaded zip file to it.
    3. Unzip the package. The following instructions assume that you unzip the contents of the downloaded package into a subfolder named ‘phpMyAdmin’ for simplicity.
    4. Create a file ‘app.yaml’ in the root folder and add the following contents.

 

application: myapplication
version: phpmyadmin
runtime: php
api_version: 1

handlers:
- url: /(.*\.(ico$|jpg$|png$|gif$))
static_files: phpMyAdmin/\1
upload: phpMyAdmin/(.*\.(ico$|jpg$|png$|gif$))
application_readable: true

- url: /(.*\.(htm$|html$|css$|js$))
static_files: phpMyAdmin/\1
upload: phpMyAdmin/(.*\.(htm$|html$|css$|js$))
application_readable: true

- url: /(.*\.(php$))
script: phpMyAdmin/\1
secure: always
login: admin

- url: /(.+)
script: phpMyAdmin/index.php
secure: always
login: admin

- url: /
script: phpMyAdmin/index.php
secure: always
login: admin

 

    1. You should end up with a directory hierarchy something like

 

-- phpMyAdmin
+ app.yaml
+ phpMyAdmin/
-- browse_foreigners.php
-- bs_disp_as_mime_type.php

 

    1. Now you can create the phpMyAdmin configuration file.

 

$ cp phpMyAdmin/config.sample.inc.php phpMyAdmin/config.inc.php

 

    1. Edit the phpMyAdmin/config.inc.php file and configure the first server to use the Cloud SQL instance that you already have.

 

// Change this to use your project and instance that you've created.
$host = '/cloudsql/my-project:my-instance';
$type = 'socket';

/*
* First server
*/
$i++;
/* Authentication type */
$cfg['Servers'][$i]['auth_type'] = 'cookie';
/* Server parameters */
$cfg['Servers'][$i]['socket'] = $host;
$cfg['Servers'][$i]['connect_type'] = $type;
$cfg['Servers'][$i]['compress'] = false;
/* Select mysql if your server does not have mysqli */
$cfg['Servers'][$i]['extension'] = 'mysqli';
$cfg['Servers'][$i]['AllowNoPassword'] = true;

 

    1. Add the following entries to the end of config.inc.php.

 

$cfg['McryptDisableWarning'] = true;
$cfg['PmaNoRelation_DisableWarning'] = true;
$cfg['ExecTimeLimit'] = 60;
$cfg['CheckConfigurationPermissions'] = false;

 

    1. Now you can push phpMyAdmin as a version of your application.

 

$ appcfg.py -R -A my_application_id -V phpmyadmin update .

 

    1. Now you can access phpMyAdmin using version addressing

 

http://phpmyadmin-dot-my_application_id.appspot.com

phpMyAdmin has a plethora of options, and what is listed here is only the basics to get it up and running. If you find any problems please file an issue here.

PHP runtime new feature requests

It’s great to see so much interest in the PHP runtime we released at I/O – and there are some cool blog posts starting to appear that showing how to get various frameworks and applications up and running.

When we were building the runtime we needed a way to prioritize what PHP extensions we should support at launch. The obvious way to do that is to pick one of the popular PHP apps and get it to work, and then build from there. As you can see from our I/O session we picked WordPress.

So now we get to the inevitable “Hey you forgot popular extension ‘X’ that I need to run popular framework ‘Y'” types of questions and discussions. Never fear comrades, the extensions that we include in the runtime are far from set in stone – and we’re really looking now for the community to tell us what it is they need to get their applications working well on App Engine.

To this end, the best place to tell us what’s missing is in the issue tracker. You can see what people have already raised here, and if what you need is already listed then “star” the issue to let us know it’s important.

If what you need is not listed, then go and add a new feature request and let us know what you need and why it’s important. Other developers can then star your issue as well and we can easily see what the majority of the community requires. Make sure you add “Language-PHP” to the labels of the issue your create.

Of course, some items might not be PHP specific but overall feature requests or bugs for App Engine – you should use the same process to file issues for the overall platform, using the label field of the issue to categorize it as best you can.