jeremykendall.net

Composer Platform Packages

Here’s something about Composer that I can never remember, I always have to look up, and I always have a hard time finding where it is in the documentation. Ladies and gentlemen, I give you platform packages:

Platform packages

Composer has platform packages, which are virtual packages for things that are installed on the system but are not actually installable by Composer. This includes PHP itself, PHP extensions and some system libraries.

  • php represents the PHP version of the user, allowing you to apply constraints, e.g. >=5.4.0. To require a 64bit version of php, you can require the php-64bit package.

  • hhvm represents the version of the HHVM runtime (aka HipHop Virtual Machine) and allows you to apply a constraint, e.g., ‘>=2.3.3’.

  • ext-<name> allows you to require PHP extensions (includes core extensions). Versioning can be quite inconsistent here, so it’s often a good idea to just set the constraint to *. An example of an extension package name is ext-gd.

  • lib-<name> allows constraints to be made on versions of libraries used by PHP. The following are available: curl, iconv, icu, libxml, openssl, pcre, uuid, xsl.

You can use composer show --platform to get a list of your locally available platform packages.

So that’s the relevant portion of the documentation, and its composer show --platform that lists local platform packages. Maybe now I’ll remember.

UPDATE

Here’s a really killer Composer Cheat Sheet I ran across after writing this post. Enjoy.

PHP Password Hashing: A Dead Simple Implementation

[UPDATE: Added a new section at the end of the post]

[UPDATE 2: Added a section RE: StorageDecorator]

tl;dr: Install Password Validator and all of your password troubles will be solved. All of them. It’ll even upgrade your old hashes transparently. Sup?

Hashing Done Wrong

We all know to encrypt passwords for highest level of security. Unfortunately, too many do it like this:

1
2
3
4
5
6
7
8
class SecurityFail
{
    // Encrypt Passwords for Highest Level of Security.
    static public function encrypt($pword)
    {
        return md5($pword);
    }
}

While there was never any excuse for getting it that wrong, there’s now no excuse for getting it wrong at all. Developers, meet the new(-ish) PHP password hashing functions (and the userland implementation password-compat).

Hashing Done Right

First, alter the password column in your user database to VARCHAR(255). Current BCRYPT passwords are 60 characters in length, but when PHP upgrades the default hash (which will happen at some point), you want to be ready. Really, just do it.

When it’s time to create a new user password, throw the plain text password into password_hash():

1
$hash = password_hash($plainTextPassword, PASSWORD_DEFAULT);

The next time a user logs in, use a little password_verify() action:

1
$isValid = password_verify($plainTextPassword, $hashedPassword);

If the password is valid, check to see if it needs to be rehashed with password_needs_rehash():

1
$needsRehash = password_needs_rehash($hashedPassword, PASSWORD_DEFAULT);

If the password needs to be rehashed, run it through password_hash() again and persist the result.

Trivial, right? Right!

Even Trivial-er

Since implementing the code above might take as many as two or three hours out of your day, I went ahead and implemented it for you. Behold, Password Validator!

Password Validator

The Password Validator library validates password_hash generated passwords, rehashes passwords as necessary, and can upgrade legacy passwords (if configured to do so).

The really big deal here is the ease of upgrading from your current legacy hashes to the new, more secure, PHP generated hashes. More on that later.

Usage

Password Validation

If you’re already using password_hash generated passwords in your application, you need do nothing more than add the validator in your authentication script. The validator uses password_verify to test the validity of the provided password hash.

1
2
3
4
5
6
7
8
use JeremyKendall\Password\PasswordValidator;

$validator = new PasswordValidator();
$result = $validator->isValid($_POST['password'], $hashedPassword);

if ($result->isValid()) {
    // password is valid
}

If your application requires options other than the password_hash defaults, you can set both the salt and cost options with PasswordValidator::setOptions().

1
2
3
4
5
$options = array(
    'salt' => 'SettingYourOwnSaltIsNotTheBestIdea',
    'cost' => 11,
);
$validator->setOptions($options);

IMPORTANT: PasswordValidator uses a default cost of 10. If your existing hash implementation requires a different cost, make sure to specify it using PasswordValidator::setOptions(). If you do not do so, all of your passwords will be rehashed using a cost of 10.

Rehashing

Each valid password is tested using password_needs_rehash. If a rehash is necessary, the valid password is rehashed using password_hash with the provided options. The result code Result::SUCCESS_PASSWORD_REHASHED will be returned from Result::getCode() and the rehashed password is available via Result::getPassword().

1
2
3
4
if ($result->isValid() && $result->getCode() == Result::SUCCESS_PASSWORD_REHASHED) {
    $rehashedPassword = $result->getPassword();
    // Persist rehashed password
}

IMPORTANT: If the password has been rehashed, it’s critical that you persist the updated password hash. Otherwise, what’s the point, right?

Upgrading Legacy Passwords

You can use the PasswordValidator whether or not you’re currently using password_hash generated passwords. The validator will upgrade your current legacy hashes to the new password_hash generated hashes. All you need to do is provide a validator callback for your password hashing scheme and then decorate the validator with the UpgradeDecorator.

1
2
3
4
5
6
7
8
9
10
11
12
use JeremyKendall\Password\Decorator\UpgradeDecorator;

// Example callback to validate a sha512 hashed password
$callback = function ($password, $passwordHash) {
    if (hash('sha512', $password) === $passwordHash) {
        return true;
    }

    return false;
};

$validator = new UpgradeDecorator(new PasswordValidator(), $callback);

The UpgradeDecorator will validate a user’s current password using the callback. If the user’s password is valid, it will be hashed with password_hash and returned in the Result object, as above.

If the callback determines the password is invalid, the password will be passed along to the PasswordValidator in case it’s already been upgraded.

Persisting Rehashed Passwords

Whenever a validation attempt returns Result::SUCCESS_PASSWORD_REHASHED, it’s important to persist the updated password hash.

1
2
3
4
if ($result->getCode() === Result::SUCCESS_PASSWORD_REHASHED) {
    $rehashedPassword = $result->getPassword();
    // Persist rehashed password
}

While you can always perform the test and then update your user database manually, if you choose to use the Storage Decorator all rehashed passwords will be automatically persisted.

The Storage Decorator takes two constructor arguments: An instance of PasswordValidatorInterface and an instance of the JeremyKendall\Password\Storage\StorageInterface.

StorageInterface

The StorageInterface includes a single method, updatePassword(). A class honoring the interface might look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
<?php

namespace Example;

use JeremyKendall\Password\Storage\StorageInterface;

class UserDao implements StorageInterface
{
    public function __construct(\PDO $db)
    {
        $this->db = $db;
    }

    public function updatePassword($identity, $password)
    {
        $sql = 'UPDATE users SET password = :password WHERE username = :identity';
        $stmt = $this->db->prepare($sql);
        $stmt->execute(array('password' => $password, 'username' => $identity));
    }
}

Storage Decorator

With your UserDao in hand, you’re ready to decorate a PasswordValidatorInterface.

1
2
3
4
5
6
7
8
use Example\UserDao;
use JeremyKendall\Password\Decorator\StorageDecorator;

$storage = new UserDao($db);
$validator = new StorageDecorator(new PasswordValidator(), $storage);

// If validation results in a rehash, the new password hash will be persisted
$result = $validator->isValid('password', 'passwordHash', 'username');

IMPORTANT: You must pass the optional third argument ($identity) to isValid() when calling StorageDecorator::isValid(). If you do not do so, the StorageDecorator will throw an IdentityMissingException.

Validation Results

Each validation attempt returns a Result object. The object provides some introspection into the status of the validation process.

  • Result::isValid() will return true if the attempt was successful
  • Result::getCode() will return one of three possible int codes:
    • Result::SUCCESS if the validation attempt was successful
    • Result::SUCCESS_PASSWORD_REHASHED if the attempt was successful and the password was rehashed
    • Result::FAILURE_PASSWORD_INVALID if the attempt was unsuccessful
  • Result::getPassword() will return the rehashed password, but only if the password was rehashed

Database Schema Changes

As mentioned above, because this library uses the PASSWORD_DEFAULT algorithm, it’s important your password field be VARCHAR(255) to account for future updates to the default password hashing algorithm.

Helper Scripts

There are two helper scripts available, both related to the password hash functions (these functions are only available after running composer install).

version-check

If you’re not already running PHP 5.5+, you should run version-check to ensure your version of PHP is capable of using password-compat, the userland implementation of the PHP password hash functions. Run ./vendor/bin/version-check from the root of your project. The result of the script is pass/fail.

cost-check

The default cost used by password_hash is 10. This may or may not be appropriate for your production hardware, and it’s entirely likely you can use a higher cost than the default. cost-check is based on the finding a good cost example in the PHP documentation. Simply run ./vendor/bin/cost-check from the command line and an appropriate cost will be returned.

NOTE: The default time target is 0.2 seconds. You may choose a higher or lower target by passing a float argument to cost-check, like so:

1
2
$ ./vendor/bin/cost-check 0.4
Appropriate 'PASSWORD_DEFAULT' Cost Found:  13

Wrapping Up

The addition of native password hashing functions is the most important security update to PHP since, well, I don’t know when. There’s no excuse for not implementing them in your applications, and the Password Validator library makes it trivial. That’s especially true when it comes to updating your legacy password hashes, which many of us need to do. Even if you only use the Password Validator as a roadmap for your own implementation, I strongly recommend upgrading ASAP.

Kudos

I was remiss to not add this bit of kudos when I originally published this post. Better late than never.

Credit for the new password hashing functions goes to Anthony Ferrara. He submitted the original RFC and created the password-compat library. The PHP community owes Anthony a debt of gratitude for making password hash security so ridiculously simple. Seriously, if I can grok it, you know it’s idiot proof :–)

Without Anthony’s hard work (and PHP core’s unanimous ‘Yes’ votes, and the password-compat contributors), my small contribution wouldn’t have been possible. Kudos to you all.

Installing Phpass From Openwall via Composer

[UPDATE: Added a PHP version clarification at the end of the post.]

Managing dependencies via Composer is one of the most revolutionary advancements in the history of PHP. Composer packages are frequently hosted on Github, listed on Packagist, and required in your project via the require field in composer.json.

So Where is phpass?

What happens when that’s not the case? One library of note, phpass, is not available on Github (or any other supported VCS)1 and therefore can’t simply be added to the require field for easy installation. All is not lost, however, thanks to Composer’s package repository feature2.

Behold, Composer’s ‘Package’ Repository!

After reviewing the package repository docs, I found it ridiculously easy to require phpass in my project. Here’s what you have to do.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
{
    "repositories": [
    {
        "type": "package",
        "package": {
            "name": "openwall/phpass",
            "version": "0.3",
            "dist": {
                "url": "http://www.openwall.com/phpass/phpass-0.3.tar.gz",
                "type": "tar"
            },
            "autoload": {
                "classmap": ["PasswordHash.php"]
            }
        }
    }
    ],
    "require": {
        "openwall/phpass": "0.3"
    }
}

Now you can run composer install (or composer update, as appropriate) and Composer will install phpass as a project dependency. Sweet!

UPDATE – CLARIFICATION: Using phpass is only advisable for PHP versions that won’t support the new password hashing functions. That’s any version of PHP less than 5.3.7:

If you’re at PHP >= 5.3.7, enjoy this article as a Composer tip you might not have know about until now and use password_compat. If you’re at PHP < 5.3.7, this is both a Composer tip and an admonition to upgrade you password security. Do it!

Many thanks to Meroje and @craig_bass for pointing out that password_compat is superior, making it clear that I needed to post a clarification.


  1. Yes, there are phpass repos on Github, but Anthony Ferrara recommends against them. When Anthony talks security, I listen.

  2. Be aware, there are significant drawbacks to this method (noted at the bottom of the Package documentation), but sometimes it’s the only way.

Creating Case-insensitive Routes With the Slim Framework

This Stack Overflow question about case-insensitive routing with the Slim Framework caught my eye recently. The question asks, in part:

How can I avoid setting up two separate routes [with different cases] that trigger the same callback function?

I found the question intriguing because 1) I love Slim and 2) I’ve never really thought about whether or not URLs are case-sensitive to begin with. My immediate thought was, “They’re already case-insensitive! Has dude even tested this?”

Are URLs Case-sensitive?

Well, kinda. It frequently, but not always, boils down to whether or not the web server’s filesystem is case sensitive. The HTTP server (Apache, nginx, etc) can get involved, as they can (always?) be configured to serve URLs with or without regard to case sensitivity. The web application and/or framework is involved too. If all other factors are case-insensitive but your application’s router is case-sensitive, then your application’s URLs will be too. This is the case with the Slim Framework.

Should URLs Be Case-sensitive?

Again, kinda. In “HTML and URLs”, the w3c has this to say:

URLs in general are case-sensitive (with the exception of machine names). There may be URLs, or parts of URLs, where case doesn’t matter, but identifying these may not be easy. Users should always consider that URLs are case-sensitive.

In general and should are the operative words there. It’s OK either way1, but assuming case-sensitivity is prudent.

So What About Slim?

URLs in the Slim Framework are case-sensitive2. That’s correct based on what we learned about URL case-sensitivity from the w3c, but what if you have a use case that requires case-insensitive URLs? What do do then?

The Magic of Slim Hooks

What are Slim Hooks? From the documentation:

A “hook” is a moment in the Slim application lifecycle at which a priority list of callables assigned to the hook will be invoked. A hook is identified by a string name.

Slim provides six default hooks, three invoked before the current route is dispatched and three invoked after. By using one of the hooks that’s invoked before the route is matched, we can alter the incoming URL’s path to match the case of the routes we’ve defined in our Slim application.

The Case-insensitive Route Hook

First, if there are any routes with mixed case, change them all to lower case. That’s the de facto standard for creating routes anyhow, and if any requests come in that match the old, mixed case routes, we’ll take care of those with the hook in the below example. Same issue, we’re just flipping it on its head. Seriously, don’t get weird about changing those routes.

Next, register this callback on the slim.before.router hook:

1
2
3
$app->hook('slim.before.router', function () use ($app) {
    $app->environment['PATH_INFO'] = strtolower($app->environment['PATH_INFO']);
});

This works because Slim matches the routes you’ve defined against the Slim Environment’s PATH_INFO (originally found in $_SERVER['PATH_INFO']). Since your routes are lower case and the incoming request paths are lower case, you’ve accomplished case insensitive routing in 3 lines of code. BOOM.


  1. It’s interesting to note that domain names are case-insensitive, regardless of whether or not that site’s URLs are case-insensitive.

  2. A pull request against Slim 2.4 is in the works that will allow case-insensitive routes via a config setting.

Using Callbacks to Bypass Guzzle’s Cache Plugin

I’m a big fan of the Guzzle PHP HTTP client. I use it whenever I need to make requests of 3rd party APIs from my applications. If you’re still writing cURL requests by hand or have rolled your own HTTP client, I highly recommend checking out Guzzle.

I’m currently making heaviest use of Guzzle in my photo-a-day project, Flaming Archer, in order to get photo data from Flickr. To keep from hammering the Flickr API, I’m caching all of those requests. Guzzle makes caching ridiculously easy by way of their plugin system and their HTTP Cache plugin.

The problem with the caching plugin, at least at first blush, is how to bypass the cache in certain specific instances where caching might not be appropriate. The docs are a little light in this area, so it took me a few minutes to get it sorted out. Let’s start at the top.

The Guzzle Client

“Clients create requests, send requests, and set responses on a request object. When instantiating a client object, you can pass an optional “base URL” and optional array of configuration options.”

Here’s an example of creating a Guzzle Client, based on my use case of making requests against the Flickr API.

1
2
3
4
5
6
7
8
use Guzzle\Http\Client;

$client = new Client('http://api.flickr.com');
$client->setDefaultOption('query', array(
    'api_key' => 'EXAMPLE_API_KEY',
    'format' => 'json',
    'nojsoncallback' => 1,
));

I use the client for the GET requests I need to make against the Flickr API. Each request will include the above default options in the query string. Nice!

Adding Caching

Since I don’t want to hammer the crap out of the Flickr API and start hitting the rate limit1, I wanted to cache each request. Thankfully, Guzzle has an awesome plugin system that includes an HTTP Cache plugin.

“Guzzle can leverage HTTP’s caching specifications using the Guzzle\Plugin\Cache\CachePlugin. The CachePlugin provides a private transparent proxy cache that caches HTTP responses.”

Rather than rolling my own caching strategy (My first solution was to write a decorator for caching), I decided to use Guzzle’s native plugin and leave all the caching work to them.

1
2
3
4
5
6
7
8
9
10
11
use Guzzle\Cache\Zf2CacheAdapter;
use Guzzle\Plugin\Cache\CachePlugin;
use Guzzle\Plugin\Cache\DefaultCacheStorage;
use Zend\Cache\Backend\TestBackend;

$backend = new TestBackend();
$adapter = new Zf2CacheAdapter($backend);
$storage = new DefaultCacheStorage($adapter);
$cachePlugin = new CachePlugin($storage);

$client->addSubscriber($cachePlugin);

The cache plugin will now intercept and cache GET and HEAD requests made by the client.

Custom Caching Decisions

So what if, now that you’re caching each GET request, there’s a request or requests you don’t want cached? Guzzle makes solving that problem trivial by allowing for “custom caching decisions”, but the documentation on how to make those custom decisions is decidedly light.

“… you can set a custom can_cache object on the constructor of the CachePlugin and provide a Guzzle\Plugin\Cache\CanCacheInterface object. You can use the Guzzle\Plugin\Cache\CallbackCanCacheStrategy to easily make a caching decision based on an HTTP request and response.”

Wat?

That was clear as mud to me, so I spent a few minutes digging through the source. This is what I came up with:

  • The CallbackCanCacheStrategy provides a method of providing callbacks to the cache plugin that, based on a boolean response, determine whether or not a particular request or response should be cached.
  • The CallbackCanCacheStrategy accepts two optional arguments to its constructor: a callable that will be invoked for requests and a callable that will be invoked for responses. The request callback gets an instance of Guzzle\Http\Message\RequestInterface, and the response callback gets an instance of Guzzle\Http\Message\Response.

Bypassing Cache

In my case, I want to cache everything except for calls to the flickr.photos.search API method. Since all of the GET requests I’m making include a method query string param, it was trivial to write the callback that got me where I needed to go.

1
2
3
4
5
6
7
8
9
$canCache = new CallbackCanCacheStrategy(
    function ($request) {
        if ($request->getQuery()->get('method') === 'flickr.photos.search') {
            return false;
        }

        return true;
    }
);

Putting It All Together

Now that I’ve built my $client, $storage, and $canCache strategy, here’s how I put it all together.

1
2
3
4
5
6
$cachePlugin = new CachePlugin(array(
    'can_cache' => $canCache,
    'storage' => $storage,
));

$client->addSubscriber($cachePlugin);

Now all of my GET requests are cached except for those using the flickr.photos.search method. BOOM.


  1. I can’t find the documentation on API rate limiting right now, but I know it’s limited and I don’t want to hit that limit.

PHP and Capistrano 3: Notes to Self

I spent quite a bit of my day yesterday trying to work out a painless, scripted, idiot-proof deployment process with Capistrano for my photo-a-day website. I’ve been doing a lot of work on the site lately, which means a lot of deployments, and I’ve been very unhappy with myself for implementing what amounts to “deployment worst practices” when it comes to my personal projects.

The last time I worked with Capistrano was about two years ago, and a lot has changed since then. Capistrano v3 was released in June of 2013 and brought with it a lot of great changes, but for a guy who doesn’t know ruby and relies on tutorials and Stack Overflow questions for help, the version bump brought a lot of pain as well.

Challenges

Every Tutorial is Wrong

Just know going into this that (almost) every tutorial you find is going to be a Capistrano v2 tutorial. Enough has changed between v2 and v3 to make those tutorials just misleading enough to cause a good amount of pain.

Stack Overflow is a Capistrano v3 Desert

As of this writing, there are ten questions tagged capistrano3 on Stack Overflow. Seriously. Ten. And only four of those include accepted answers.

Capistrano v3 Documentation is Lacking

The documentation available for v3 is seriously lacking, although the problem is more one of quantity than quality. What’s available is good, there’s just nowhere near enough of it.

Caveat Emptor

These are indeed “Notes to Self”. I hope they help you out, but if they don’t, I’m giving you fair warning. Please feel free to add what’s missing to the comments.

Reading the Capistrano docs is highly recommended. These notes are supplemental.

PHP + Capistrano v3

NOTE: You can find the application source, which now includes Capistrano v3, on GitHub.

Ruby

First, I had to reinstall rvm, a ruby version manager. What I don’t know about installing a ruby dev environment could fill books, so I let rvm take care of that for me.

1
\curl -L https://get.rvm.io | bash -s stable --ruby

(Capistrano requires Ruby 1.9 or newer, so the current stable Ruby from rvm will work fine.)

Installation

Install Capistrano. There are a few options, but I used gem install.

1
gem install capistrano

NOTE: All of the PHP tutorials I found instruct you to install the railsless-deploy gem. As Capistrano v3 “doesn’t ship with any Railsisms”, this is no longer necessary and the railsless-deploy project is obsolete.

I’ll probably go back and add a Gemfile since I’ll be adding the composer gem and want everything I need in one place.

Preparing Your App

The capify command is no longer, it’s now cap install.

1
2
cd /path/to/project
cap install

The documentation on this portion, “Preparing Your Application”, is one of the places the documentation shines, IMHO.

Capistrano Files

  • Capfile: Kind of like a bootstrap. Takes care of necessary required configs and globs for custom tasks.
  • config/deploy.rb: Common items across all deployments go here
  • config/deploy/{staging, production}.rb: Environment specific deployment settings

Roles

I’m still not 100% clear on this, but roles don’t seem to be roles in the ACL sense, but rather roles in the “division of server responsibility” sense, hence the roles :web, :app, and :db.

The docs say that you can dump the :app and :db roles if you like, but if you’re going to use the :linked_files and :linked_dirs feature (which are pretty cool) you’ll need to leave the :app role in place. I’m obviously doing something wrong or missing something here, but that’s what I had to do.

I found it extremely helpful to refer to the deploy.rb.erb template. I removed most of the example text before realizing I needed to use some of it, and referencing the template was nice.

SSH Forwarding

SSH Agent Forwarding is the way to go here, IMHO. I already had a key agent running so the forwarding was dead simple, almost like magic. If you don’t already have an agent running, here’s some good info from GitHub Help.

Server Config

The section on how to set up the proper Capistrano directories on your server is way down deep in the Authorisation portion of the docs. Short version: you need two dirs in the root of your project (on the server): releases and shared, and they must be readable and writeable by both the webserver and deploy user.

Getting permissions right is easy for some, but I always return to the Setting Up Permissions section of the Symfony2 docs to get them set up properly.

Making Composer Work

Once I got my deploy.rb and deploy/production.rb right and tested (by deploying, natch), I needed to create a task to run composer install. Getting that right turned out to be pretty difficult because of a design decision in SSHKit.

Short story: No spaces in command line commands.

I finally got my composer command running by doing this:

1
2
3
4
5
6
7
8
9
10
desc 'composer install'
task :composer_install do
    on roles(:web) do
        within release_path do
            execute 'composer', 'install', '--no-dev', '--optimize-autoloader'
        end
    end
end

after :updated, 'deploy:composer_install'
  • The within release bit in the command tells the task to cd into the release directory before running composer.
  • There are before and after hooks you can apply to the deploy flow. The final line above is a hook that runs after deploy:updated.

Composer Gem for Capistrano v3

Of course, as soon as I got it working, Peter Mitchell sent me this tweet:

I haven’t yet replaced my hacked version with the gem, but I’ll use it as soon as I do any refactoring.

Deployment Annoyances

cap production deploy --dry-run never worked for me, although cap production deploy worked fine. No idea why, who cares at this point. Maybe I’ll dig in later.

Also, cap production deploy really wants to run deploy:restart, even though it doesn’t show up anywhere in the deploy flow. I replaced the :restart task that’s in the default deploy.rb template, made sure it was empty, and deploy finally worked.

Linked Files and Directories

The :linked_files and :linked_directories feature is really nice. I’ve used it for logs, a local.php config, Twig caching, my SQLite database, and my generated rss file. The linked items are for files and dirs that should be shared between deploys.

Also, those files and dirs need to be present in the shared dir before deploying. Deploy will puke if they’re not present.

Capistrano Variables

I couldn’t find a listing of these anywhere, but release_path is one of them, and it points to the latest release path.

That’s All For Now

I hope the notes help when you get ready to write your own Capistrano v3 scripts for use with PHP. I’ll update this as I learn more, and I’ll make sure to point out any excellent point made in the comments.

Restarting VirtualBox on OSX

My Vagrant + VirtualBox VM workflow was distrupted late this afternoon by an error during vagrant up:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
[default] Importing base box 'precise64'...
[default] Matching MAC address for NAT networking...
[default] Setting the name of the VM...
[default] Clearing any previously set forwarded ports...
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.

    Command: ["hostonlyif", "create"]

    Stderr: 0%...
    Progress state: NS_ERROR_FAILURE
    VBoxManage: error: Failed to create the host-only adapter
    VBoxManage: error: VBoxNetAdpCtl: Error while adding new interface: failed to open /dev/vboxnetctl: No such file or directory

    VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component HostNetworkInterface, interface IHostNetworkInterface
    VBoxManage: error: Context: "int handleCreate(HandlerArg*, int, int*)" at line 68 of file VBoxManageHostonly.cpp

Googling for Failed to create the host-only adapter didn’t seem to return anything useful, so I tried with failed to open /dev/vboxnetctl and immediately found an answer.

The trick, according to GitHub user lslucas, is to restart VirtualBox from the command line like so:

1
 $ sudo /Library/StartupItems/VirtualBox/VirtualBox restart

That worked like a champ for me, and I was immediately able to get back to work.

For what it’s worth, here’s my current software config:

  • VirtualBox 4.2.16
  • Vagrant 1.2.2
  • Mac OS X 10.8.4

Nashville, I Hardly Knew Ye

tl;dr:

  • My wife Megan and I are moving back to Memphis, TN
  • All of our family is there
  • The pregnancy has been tough, we’d like to raise our son around family
  • I’m leaving Nashville, but I’ll still be working for OpenSky
  • I’ll miss you all terribly, but I’ll be back regularly

So Long, And Thanks For All The Awesome

I might as well get right to it, especially since I ruined the surprise above: Megan and I have decided to move back to Memphis. The decision to leave Nashville, especially after spending such a short time here, was a really difficult one, but I’m convinced this is the best decision for me and my family.

A Little Background

Megan and I are both Memphis natives. We’ve got deep roots in the “Bluff City”, and both of our families (excepting one of my brothers and his family) are still there. We moved to Nashville late last year after the incomparable Scott Gordon hooked me up with an awesome gig here in town.

In April of this year we found out we’d be having a baby! That was amazing, wonderful news, but the pregnancy has been difficult from day one. Recently, things got really, really scary. After a lot of discussion and soul searching, Megan and I decided the best decision for us and our new family would be moving home to be around the support, care, and help of our families.

I’m Leaving, But Only Mostly

So come mid- to late-October, when our current lease is up, we’re heading back to M-town. There is a huge Nashville-related silver lining, however. OpenSky is letting me move to Memphis while staying on staff with them! That means regular trips back to Nashville to work in the local office, get some face time with the team, and hang out with all my Nashville friends.

So Let’s Do Lunch

Nashville friends, we need to do lunch, pronto. Two months seems like a long time, but I’ll be driving that moving truck west on I-40 before we know it. I want to get together while it’s still relatively easy to do so. I’ve spent to little time with y’all as it is.

MEMPHIS, I AM ALMOST BACK IN YOU

Leaving Nashville is sad, coming back to Memphis is going to be awesome. I’m looking forward to hanging out with my Memphis people, getting involved in the Memphis PHP and the #memtech communities again, reconnecting with Memphis Roller Derby, and being able to see the fam whenever.

Exciting Times Ahead

Things have been wild the past few years, and they’re about to get a lot more exciting. Change is tough, and I’m feeling the pressure, but I feel great about where I’m at, where Megan and I are at, and our future. I can’t wait to see what comes next.

Trending on GitHub

Thanks to yesterday’s link love from PHPDeveloper, I’ve made both the Trending PHP Developer list and the Trending PHP Repo list on GitHub.

With internet fame being so fleeting, I took some screenshots for posterity. Now please excuse me while I get some ice for my arm. I seem to have injured myself while patting myself on the back.

API Query Authentication With Query Auth

Most APIs require some sort of query authentication: a method of signing API requests with an API key and signature. The signature is usually generated using a shared secret. When you’re consuming an API, there are (hopefully) easy to follow steps to create signatures. When you’re writing your own API, you have to whip up both server-side signature validation and a client-side signature creation strategy. Query Auth endeavors to handle both of those tasks; signature creation and signature validation.

Philosophy

Query Auth is intended to be — and is written as — a bare bones library. Many of niceties and abstractions you’d find in a fully featured API library or SDK are absent. The point of the library is to provide you with the ability to focus on writing the meat of your API while offloading the authentication bits.

What’s Included?

There are three components to Query Auth: request signing for API consumers and creators, request signature validation for API creators, and API key and API secret generation.

Request Signing

1
2
3
4
5
6
7
8
9
10
11
12
$collection = new QueryAuth\NormalizedParameterCollection();
$signer = new QueryAuth\Signer($collection);
$client = new QueryAuth\Client($signer);

$key = 'API_KEY';
$secret = 'API_SECRET';
$method = 'GET';
$host = 'api.example.com';
$path = '/resources';
$params = array('type' => 'vehicles');

$signedParameters = $client->getSignedRequestParams($key, $secret, $method, $host, $path, $params);

Client::getSignedRequestParams() returns an array of parameters to send via the querystring (for GET requests) or the request body. The parameters are those provided to the method (if any), plus timestamp, key, and signature.

Signature Validation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
$collection = new QueryAuth\NormalizedParameterCollection();
$signer = new QueryAuth\Signer($collection);
$server = new QueryAuth\Server($signer);

$secret = 'API_SECRET_FROM_PERSISTENCE_LAYER';
$method = 'GET';
$host = 'api.example.com';
$path = '/resources';
// querystring params or request body as an array,
// which includes timestamp, key, and signature params from the client's
// getSignedRequestParams method
$params = 'PARAMS_FROM_REQUEST';

$isValid = $server->validateSignature($secret, $method, $host, $path, $params);

Server::validateSignature() will return either true or false. It might also throw one of three exceptions:

  • MaximumDriftExceededException: If timestamp is too far in the future
  • MinimumDriftExceededException: It timestamp is too far in the past
  • SignatureMissingException: If signature is missing from request params

Drift defaults to 15 seconds, meaning there is a 30 second window during which the request is valid. The default value can be modified using Server::setDrift().

Key Generation

You can generate API keys and secrets in the following manner.

1
2
3
4
5
6
7
8
9
$randomFactory = new \RandomLib\Factory();
$keyGenerator = new QueryAuth\KeyGenerator($randomFactory);

// 40 character random alphanumeric string
$key = $keyGenerator->generateKey();

// 60 character random string containing the characters
// 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ./
$secret = $keyGenerator->generateSecret();

Both key and secret are generated using Anthony Ferrara’s RandomLib random string generator.

That’s Kinda Ugly, Dude

As I pointed out, the Query Auth library is pretty bare bones. There are a lot of opportunities for abstraction that would make the library much easier to use and much nicer to look at. If I added them to Query Auth, however, that would lock library users into whichever HTTP client I chose to use. The same concern would go for whatever other abstractions I decided on. The point here is to offload query authentication, and only query authentication, to the Query Auth library.

Sample Implementation

In order to demonstrate how one might implement the Query Auth library, I’ve whipped up a sample implementation for you.

The sample uses Vagrant and VirtualBox to allow you to see the whole thing in action. Slim Framework runs the API, Guzzle is used to make requests to the API, and both a GET and POST request are implemented. JSend, Jamie Schembri’s PHP implementation of the OmniTI JSend specifiction, is used to send messages back from the API, and Parsedown PHP, Emanuil Rusev’s Markdown parser for PHP, is used to render the sample implementation’s documentation.

Request Signing

In the sample implementation, request signing has been abstracted in the Example\ApiRequestSigner class. Signing requests is now as simple as passing the request object and credentials object to the signRequest method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
/**
 * Signs API request
 *
 * @param RequestInterface $request     HTTP Request
 * @param ApiCredentials   $credentials API Credentials
 */
public function signRequest(RequestInterface $request, ApiCredentials $credentials)
{
    $signedParams = $this->client->getSignedRequestParams(
            $credentials->getKey(),
            $credentials->getSecret(),
            $request->getMethod(),
            $request->getHost(),
            $request->getPath(),
            $this->getParams($request)
            );

    $this->replaceParams($request, $signedParams);
}

Signature Validation

In the sample implementation, signature validation has been abstracted in the Example\ApiRequestValidator class. Validating request signatures is now as simple as passing the request object and credentials object to the isValid method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
/**
 * Validates an API request
 *
 * @param  Request        $request     HTTP Request
 * @param  ApiCredentials $credentials API Credentials
 * @return bool           True if valid, false if invalid
 */
public function isValid(Request $request, ApiCredentials $credentials)
{
    return $this->server->validateSignature(
        $credentials->getSecret(),
        $request->getMethod(),
        $request->getHost(),
        $request->getPath(),
        $this->getParams($request)
    );
}

Signing a GET Request

Signing a request is now extremely clean and simple. Here’s the GET example from the sample implementation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
/**
 * Sends a signed GET request which returns a famous mangled phrase
 */
$app->get('/get-example', function() use ($app, $credentials, $requestSigner) {

    // Create request
    $guzzle = new GuzzleClient('http://query-auth.dev');
    $request = $guzzle->get('/api/get-example');

    // Sign request
    $requestSigner->signRequest($request, $credentials);

    $response = $request->send();

    $app->render('get.html', array('request' => (string) $request, 'response' => (string) $response));
});

Validating a GET Request

Validating a GET request is equally clean and simple. Note the try/catch that handles possible exceptions from the validation class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
/**
 * Validates a signed GET request and, if the request is valid, returns a
 * famous mangled phrase
 */
$app->get('/api/get-example', function () use ($app, $credentials, $requestValidator) {

    try {
        // Validate the request signature
        $isValid = $requestValidator->isValid($app->request(), $credentials);

        if ($isValid) {
            $mistakes = array('necktie', 'neckturn', 'nickle', 'noodle');
            $format = 'Klaatu... barada... n... %s!';
            $data = array('message' => sprintf($format, $mistakes[array_rand($mistakes)]));
            $jsend = new JSendResponse('success', $data);
        } else {
            $jsend = new JSendResponse('fail', array('message' => 'Invalid signature'));
        }
    } catch (\Exception $e) {
        $jsend = new JSendResponse('error', array(), $e->getMessage());
    }

    $response = $app->response();
    $response['Content-Type'] = 'application/json';
    echo $jsend->encode();
});

Sample Request and Response

The code above produces the below request and response:

Request

1
2
3
GET /api/get-example?key=ah5yEgQzjuFsC9nWsRI4Nar3ikOqWVPcD3OntHpg&timestamp=1376416267&signature=3DqimkvigYBorGi8wHfil9lB8oCWhB%2BHYt6rVfE4zx4%3D HTTP/1.1
Host: query-auth.dev
User-Agent: Guzzle/3.7.2 curl/7.22.0 PHP/5.5.1-2+debphp.org~precise+2

Response

1
2
3
4
5
6
7
8
HTTP/1.1 200 OK
Date: Tue, 13 Aug 2013 17:51:07 GMT
Server: Apache/2.4.6 (Ubuntu)
X-Powered-By: PHP/5.5.1-2+debphp.org~precise+2
Content-Length: 75
Content-Type: application/json

{"status":"success","data":{"message":"Klaatu... barada... n... necktie!"}}

Wrapping Up

So there you have it: QueryAuth to sign and validate API requests (and generate keys and secrets!) and a sample implementation to get you going. If you find this helpful, or have any questions or comments, please let me know. If you find any horrible mistakes, please feel free to submit an issue or a pull request, or you can always submit the offending code to CSI: PHP :–)