After two amazing, challenging, crazy, exhausting years with Graph Story, I’ve decided that it’s time to find a new challenge and a new workplace to call home.

I’ve grown more as a developer in the past two years than I have in my entire career. I’ve also worked harder than I’ve ever worked before. At the end of these past two years I find myself utterly exhausted and in need of a change of pace.

Available for Hire

If you’re in search of a Senior Engineer to join your dev team, I’m looking and immediately available! I bring ~15 years of web development experience to the table, a passion for testable, maintainable code, an insatiable appetite for learning new things, and some leadership experience.

If you’d like to know more, here’s my resume (a printable version is available here), you can take a look at my open source work on GitHub, or even spend a few minutes browsing through some of my photos.

What I’m Looking For

I want to get out of my comfort zone and learn and do some new things. Don’t need a PHP dev but you’re looking for a great Senior Engineer? If you’re willing to teach, I’m willing to learn (with the caveat that I’d much prefer an open source shop).

I need to work remotely. My wife and I are about to have kiddo #2 and both of our families are here in Memphis. Relocating right now would be extremely difficult, although I would consider relocating at some point in the near future for the right opportunity.

Additionally, here are the top three things I’m looking for in a new employer:

1. Work-life balance: I’m a crazy hard worker, and I can and will work some really long hours, but I’m looking for 40-50 hours/week as a regular thing.
2. A strong engineering culture working on big things: I’m less concerned about what you do as a company than I am with your engineering culture and practices. If you have a medium-large group of brilliant engineers that walk the best-practices walk, I’m in.
3. Tech community support and involvement: I want to work for a company that values sending engineers to conferences, supports and uses open source software, gives back to the community by means of supporting user groups, open sourcing tools where possible, and contributing to open source projects as a matter of course.

Contact Info

If you’d like to talk turkey, hit me up at jeremy -at- jeremykendall -dot- net. I look forward to hearing from you!

Encoding Detection

I was quickly able to determine that the CSV file in question was encoded with utf-16le by using the following command:

1
2

$ file -I unknown-encoding.csv
unknown-encoding.csv: text/plain; charset=utf-16le

Converting to UTF-8

Converting the file to a new encoding was just as easy:

1

iconv -f utf-16le -t utf-8 unknown-encoding.csv > new-encoding.csv

References

The commands above were sourced from the following superuser questions and accepted answers:

• Determining the encoding of a file on Mac OS X?
• Converting the encoding of a text file (Mac OS X)

UPDATE 2: Oops. I didn’t explain that correctly. Here’s Dan to set the matter straight:

@JeremyKendall that is slightly wrong. It sets the drift threshold, so it only syncs if it is 10 seconds or more off.

— Dan Horrigan (@dhrrgn) October 6, 2014

ORIGINAL POST: I frequently find myself needing to update the time on my Vagrant boxes. This is especially true when I’m testing Query Auth between two different Vagrant boxes (mimicking a client/server relationship) as Query Auth will fail a request with a timestamp that varies too greatly from the server’s timestamp (+-15 seconds by default). Even though I run NTP on all my virtual servers, time drift is a frequent problem. Furthermore, when the drift is too great (can’t find a reference to how large the drift needs to be), NTP won’t correct it all at once.

A quick Google search later and, once again, it was Stack Overflow the rescue, courtesy of Martin Schröder’s answer to the question “How to force a clock update using ntp?”

One quick bash script later and excessive clock drift is now a trivial issue.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

#!/bin/bash
# Forces an ntp update
#
# Based on SO user Martin Schröder's answer to "How to force a clock update
# using ntp?": http://askubuntu.com/a/256004/41943

# Fail fast (set -e will bail at first error)
set -e

if [ "$EUID" -ne 0 ]; then
    echo "ERROR: '$0' must be as root."
    exit 1
fi

service ntp stop

echo "Running 'ntpd -gq'"
ntpd -gq

service ntp start

Postscript

If you’re unfamiliar with the particulars of creating bash scripts, here are the steps you can take to create your own version of the above script. You may need to preface the commands using sudo.

• Copy and paste the above script into a text file (I’ve named mine force-ntp-update).
• On the command line, call chmod a+x /path/to/force-ntp-update to allow all users to execute the command.
  • If you’re the only user who should be able to execute the command, change the above to chmod u+x /path/to/force-ntp-update.
• Move the script somewhere in your path: mv /path/to/force-ntp-update /usr/local/bin.
• Done!

Now anytime you need to force an update, simply call sudo force-ntp-update.

Full disclosure: I can only confirm the existence of this bug in the above mentioned environment. I have not tested for the bug in any other configuration.

Bug Report - TL;DR

There is a bug report filed by Artyom Nosov which details the issue and includes a patch. The patch diff is linked from the bug report and is included below:

1
2
3
4
5
6
7
8
9
10
11
12

diff -urN gearmand-1.0.6-old/debian/gearman-job-server.upstart gearmand-1.0.6/debian/gearman-job-server.upstart
--- gearmand-1.0.6-old/debian/gearman-job-server.upstart2013-11-11 01:58:42.000000000 +0400
+++ gearmand-1.0.6/debian/gearman-job-server.upstart20132013-12-13 22:30:32.392281779 +0400
@@ -9,4 +9,7 @@

  respawn

   -exec start-stop-daemon --start --chuid gearman --exec /usr/sbin/gearmand -- --log-file=/var/log/gearman-job-server/gearman.log
   +script
   +    . /etc/default/gearman-job-server
   +    exec start-stop-daemon --start --chuid gearman --exec /usr/sbin/gearmand -- $PARAMS --log-file=/var/log/gearman-job-server/gearman.log
   +end script

The Fix

Gearman runtime configuration is handled by /etc/default/gearman-job-server in Ubuntu. The default upstart job does not use the Gearman config file, causing Gearman to run with its defaults, which may or may not be acceptable to you. To resolve the issue, update /etc/init/gearman-job-server.conf using the diff included above. For reference, here is my entire gearman-job-server.conf file:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

# -*- upstart -*-

# Upstart configuration script for "gearman-job-server".

description "gearman job control server"

start on (filesystem and net-device-up IFACE=lo)
stop on runlevel [!2345]

respawn

# exec start-stop-daemon --start --chuid gearman --exec /usr/sbin/gearmand -- --log-file=/var/log/gearman-job-server/gearman.log

# PATCH: https://bugs.launchpad.net/ubuntu/+source/gearmand/+bug/1260830
script
    . /etc/default/gearman-job-server
    exec start-stop-daemon --start --chuid gearman --exec /usr/sbin/gearmand -- $PARAMS --log-file=/var/log/gearman-job-server/gearman.log
end script

The Benefit

Gearman will now respect the settings found in /etc/default/gearman-job-server, allowing you to configure Gearman with command line options that you’d otherwise have to add to the init script. In the case of the Graph Story implementation, I’m binding Gearman to localhost and using MySQL as a persistent queue. Here’s what that config file looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

# This is a configuration file for /etc/init.d/gearman-job-server; it allows
# you to perform common modifications to the behavior of the gearman-job-server
# daemon startup without editing the init script (and thus getting prompted by
# dpkg on upgrades).  We all love dpkg prompts.

# Examples ( from http://gearman.org/index.php?id=manual:job_server )
#
# Use drizzle as persistent queue store
# PARAMS="-q libdrizzle --libdrizzle-db=some_db --libdrizzle-table=gearman_queue"
#
# Use mysql as persistent queue store
# PARAMS="-q libdrizzle --libdrizzle-host=10.0.0.1 --libdrizzle-user=gearman \
#                       --libdrizzle-password=secret --libdrizzle-db=some_db \
#                       --libdrizzle-table=gearman_queue --libdrizzle-mysql"
#
# Missing examples for memcache persitent queue store...

# Parameters to pass to gearmand.
PARAMS="--listen=localhost \
        -q mysql \
        --mysql-host=localhost \
        --mysql-port=3306 \
        --mysql-user=redacted \
        --mysql-password=redacted \
        --mysql-db=gearman \
        --mysql-table=gearman_queue"

Side Note and Pro Tip: MySQL Persistent Queue

Creating the MySQL database and providing privileges to your database user allows Gearman to create its own gearman_queue database table. Don’t risk creating an incompatible database table; let Gearman do that work for you.

Requirements

While I’m not going to make any general knowledge assumptions (call me out in the comments if I miss the mark), I am going to set a few requirements. In short, we’ll be using virtualization technology (Vagrant and VirtualBox), the Ubuntu 14.04 LTS operating system, and Composer, a dependency management tool for PHP.

Why Do This All “By Hand”?

While there’s an “easier, softer way” (think Phansible or PuPHPet), I think it’s important to know what’s going on behind the scenes. What if you need to fix something on your server once it’s in production? What if you have to tweak a setting here or there, or create a new vhost or nginx site? It’s a good idea to have done it at least once by hand before moving on to automated solutions. You’ll have a better feel for what’s happening, why it’s happening, and how to fix any problems that might arise in the future.

Preparing the Host Environment

By “host”, I mean your computer. These are the first steps we’ll take to prepare your computer to host the virtual machine we’ll use to run the tutorial code.

• Install Vagrant: Grab an installer for your OS here.
• Install VirtualBox: Grab an installer for your OS here.

Create Your VM

• Create a directory for your project, and then change into your project directory
• Make a sub-directory for your Slim project within your new project directory: mkdir zero-to-slim.dev
• Now run vagrant init ubuntu/trusty64 from your project directory

vagrant init will create a Vagrantfile, the file that tells Vagrant how to build your VM. We’ll need to edit that file and add a few settings.

Edit /path/to/project/Vagrantfile and add the following lines after config.vm.box:

1
2
3
4
5
6

  config.vm.hostname = "zero-to-slim.dev"
  config.vm.network :private_network, ip: "192.168.56.103"
  config.vm.synced_folder "./zero-to-slim.dev", "/var/www/zero-to-slim.dev", id: "web-root",
      owner: "vagrant",
      group: "www-data",
      mount_options: ["dmode=775,fmode=664"]

Save and close your Vagrantfile, and then run vagrant up. You’ll be treated to some output as your VM is built. Once that’s done, the VM is ready to configure.

Connect and Configure VM

Run vagrant ssh from your project directory. The following steps will be completed on the VM, not your host machine.

If you’re on Windows, the ssh command won’t be available to you. You’ll need to use a program like PuTTY instead. Here are some step-by-step instructions for configuring your Windows box to connect to your new VM.

• sudo apt-get update
• sudo apt-get install curl vim wget python-software-properties -y
• sudo add-apt-repository ppa:ondrej/php5 -y
• sudo apt-get update

Choose a Web Server and a PHP Version

You must choose one or the other, not both. nginx is extremely popular, but if you’re more comfortable with Apache I’ve included instructions below.

nginx and PHP-FPM

• sudo apt-get install php5-fpm php5-cli php5-xdebug nginx -y

Apache2 and PHP

• sudo apt-get install php5 php5-cli php5-xdebug apache2 -y

Install Composer Globally

See Composer’s Getting Started section for the most up-to-date installation instructions (There’s an installer available for those of you on Windows). Below is how I install Composer on both Mac and Linux.

• curl -sS https://getcomposer.org/installer | php
• sudo mv composer.phar /usr/local/bin/composer

Use Composer to Install Slim Skeleton

Heads up, this is going to take a while. Composer is an amazing technology, but in this case it’s pretty slow (this has more to do with the VM than with Composer). Run the command, grab a cup of coffee, and come on back to finish up.

• composer create-project slim/slim-skeleton /var/www/zero-to-slim.dev

If you suspect there’s a problem with the composer create-project command due to how long it takes to get feedback, you can add the verbose option (-vvv) to the command, like so: composer create-project slim/slim-skeleton /var/www/zero-to-slim.dev -vvv

Configure Your Web Server

Since the following files aren’t in a synced folder, which would allow you to edit them from your host machine, and since you need to be root to edit them, the best way to do so is directly on the server using a text editor. I’ve chosen Vim in this case. Once again, here’s an opportunity to learn about a tool that you may find yourself needing at some point in the future, even if you only ever use it to edit a file or two on your server(s).

Editing a Config File with Vim

• Type sudo vim <server_config_file> (either /etc/nginx/sites-available/default or /etc/apache2/sites-available/000-default.conf, depending)
• Type gg on the keyboard to ensure you’re at the top of the file
• Delete everything in the file with dG
• Type i to enter insert mode
• Copy the appropriate config from below and paste it into the config document
• Hit ESC to exit insert mode
• Type :wq to write your changes and quit the file.

Congrats! You’ve just edited your server config using Vim, an accomplishment in itself.

nginx and PHP-FPM

Replace the contents of /etc/nginx/sites-available/default with the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

server {
    listen      80;
    server_name zero-to-slim.dev;
    root        /var/www/zero-to-slim.dev/public;

    try_files $uri /index.php;

    # this will only pass index.php to the fastcgi process which is generally safer but
    # assumes the whole site is run via Slim.
    location /index.php {
        fastcgi_connect_timeout 3s;     # default of 60s is just too long
        fastcgi_read_timeout 10s;       # default of 60s is just too long
        fastcgi_pass unix:/var/run/php5-fpm.sock;
        include fastcgi_params;
    }
}

Save and close the default config, and then restart nginx: sudo service nginx restart

Apache2

Replace the contents of /etc/apache2/sites-available/000-default.conf with the following:

1
2
3
4
5
6
7
8
9
10

<VirtualHost *:80>
    DocumentRoot "/var/www/zero-to-slim.dev/public"
    ServerName zero-to-slim.dev

    <Directory "/var/www/zero-to-slim.dev/public">
        AllowOverride All
        Order allow,deny
        Allow from all
    </Directory>
</VirtualHost>

• Save and close the default config
• Enable mod_rewrite for URL rewriting: sudo a2enmod rewrite
• Restart Apache: sudo service apache2 restart

Configure PHP for Dev Environment

Since editing php.ini with Vim would require quite a few detailed instructions, I’ve opted to show you how to override php.ini settings by adding an additional config file.

On your host machine, create a file in your project directory name 00-php.ini. Paste the following contents into the file:

1
2
3
4

error_reporting = -1
display_errors = On
display_startup_errors = On
html_errors = On

These settings ensure that PHP will report any and all errors it encounters, and that PHP will display those errors on screen. The html_errors setting will provide a nice looking error formatted by xdebug.

Since your project directory is synced to /vagrant on your VM, the file you just created will be available on your VM. Copy the new config file into the directory scanned by PHP for additional config files and restart your web server. The following commands should be executed on your VM, not your host machine.

nginx and PHP-FPM

• sudo cp /vagrant/00-php.ini /etc/php5/fpm/conf.d/
• sudo service php5-fpm restart

Apache2 and PHP

• sudo cp /vagrant/00-php.ini /etc/php5/apache2/conf.d/
• sudo service apache2 restart

Final Configuration

• Add 192.168.56.103 zero-to-slim.dev to /etc/hosts on the host machine (Windows users, see: How do I modify my hosts file)
• Visit http://zero-to-slim.dev in your web browser

You should see the Slim welcome page. If you don’t, there should be an error displayed telling you exactly what went wrong. Double check the steps above to make sure you didn’t miss anything. If you still have problems, drop me a line in the comments and I’ll help you get them sorted out.

Wrapping Up

So there you have it. You started with nothing at all and now have a working Slim Framework application. Congrats!

FYI, I highly recommend basing your Slim apps on the Slim Skeleton. Install it, modify it for your specific application needs, and you’ll have a much easier time getting up and running, I promise. That’s what I did when I first started with Slim.

Next time we’ll talk about how to automate this process. Now that you know what’s involved, there’s no point in wasting precious dev time manually configuring new machines for each new project.

Platform packages

Composer has platform packages, which are virtual packages for things that are installed on the system but are not actually installable by Composer. This includes PHP itself, PHP extensions and some system libraries.

• php represents the PHP version of the user, allowing you to apply constraints, e.g. >=5.4.0. To require a 64bit version of php, you can require the php-64bit package.

• hhvm represents the version of the HHVM runtime (aka HipHop Virtual Machine) and allows you to apply a constraint, e.g., ‘>=2.3.3’.

• ext-<name> allows you to require PHP extensions (includes core extensions). Versioning can be quite inconsistent here, so it’s often a good idea to just set the constraint to *. An example of an extension package name is ext-gd.

• lib-<name> allows constraints to be made on versions of libraries used by PHP. The following are available: curl, iconv, icu, libxml, openssl, pcre, uuid, xsl.

You can use composer show --platform to get a list of your locally available platform packages.

So that’s the relevant portion of the documentation, and its composer show --platform that lists local platform packages. Maybe now I’ll remember.

[UPDATE 2: Added a section RE: StorageDecorator]

tl;dr: Install Password Validator and all of your password troubles will be solved. All of them. It’ll even upgrade your old hashes transparently. Sup?

Hashing Done Wrong

We all know to encrypt passwords for highest level of security. Unfortunately, too many do it like this:

1
2
3
4
5
6
7
8

class SecurityFail
{
    // Encrypt Passwords for Highest Level of Security.
    static public function encrypt($pword)
    {
        return md5($pword);
    }
}

While there was never any excuse for getting it that wrong, there’s now no excuse for getting it wrong at all. Developers, meet the new(-ish) PHP password hashing functions (and the userland implementation password-compat).

Hashing Done Right

First, alter the password column in your user database to VARCHAR(255). Current BCRYPT passwords are 60 characters in length, but when PHP upgrades the default hash (which will happen at some point), you want to be ready. Really, just do it.

When it’s time to create a new user password, throw the plain text password into password_hash():

1

$hash = password_hash($plainTextPassword, PASSWORD_DEFAULT);

The next time a user logs in, use a little password_verify() action:

1

$isValid = password_verify($plainTextPassword, $hashedPassword);

If the password is valid, check to see if it needs to be rehashed with password_needs_rehash():

1

$needsRehash = password_needs_rehash($hashedPassword, PASSWORD_DEFAULT);

If the password needs to be rehashed, run it through password_hash() again and persist the result.

Trivial, right? Right!

Even Trivial-er

Since implementing the code above might take as many as two or three hours out of your day, I went ahead and implemented it for you. Behold, Password Validator!

Password Validator

The Password Validator library validates password_hash generated passwords, rehashes passwords as necessary, and can upgrade legacy passwords (if configured to do so).

The really big deal here is the ease of upgrading from your current legacy hashes to the new, more secure, PHP generated hashes. More on that later.

Usage

Password Validation

If you’re already using password_hash generated passwords in your application, you need do nothing more than add the validator in your authentication script. The validator uses password_verify to test the validity of the provided password hash.

1
2
3
4
5
6
7
8

use JeremyKendall\Password\PasswordValidator;

$validator = new PasswordValidator();
$result = $validator->isValid($_POST['password'], $hashedPassword);

if ($result->isValid()) {
    // password is valid
}

If your application requires options other than the password_hash defaults, you can set both the salt and cost options with PasswordValidator::setOptions().

1
2
3
4
5

$options = array(
    'salt' => 'SettingYourOwnSaltIsNotTheBestIdea',
    'cost' => 11,
);
$validator->setOptions($options);

IMPORTANT: PasswordValidator uses a default cost of 10. If your existing hash implementation requires a different cost, make sure to specify it using PasswordValidator::setOptions(). If you do not do so, all of your passwords will be rehashed using a cost of 10.

Rehashing

Each valid password is tested using password_needs_rehash. If a rehash is necessary, the valid password is rehashed using password_hash with the provided options. The result code Result::SUCCESS_PASSWORD_REHASHED will be returned from Result::getCode() and the rehashed password is available via Result::getPassword().

1
2
3
4

if ($result->isValid() && $result->getCode() == Result::SUCCESS_PASSWORD_REHASHED) {
    $rehashedPassword = $result->getPassword();
    // Persist rehashed password
}

IMPORTANT: If the password has been rehashed, it’s critical that you persist the updated password hash. Otherwise, what’s the point, right?

Upgrading Legacy Passwords

You can use the PasswordValidator whether or not you’re currently using password_hash generated passwords. The validator will upgrade your current legacy hashes to the new password_hash generated hashes. All you need to do is provide a validator callback for your password hashing scheme and then decorate the validator with the UpgradeDecorator.

1
2
3
4
5
6
7
8
9
10
11
12

use JeremyKendall\Password\Decorator\UpgradeDecorator;

// Example callback to validate a sha512 hashed password
$callback = function ($password, $passwordHash) {
    if (hash('sha512', $password) === $passwordHash) {
        return true;
    }

    return false;
};

$validator = new UpgradeDecorator(new PasswordValidator(), $callback);

The UpgradeDecorator will validate a user’s current password using the callback. If the user’s password is valid, it will be hashed with password_hash and returned in the Result object, as above.

If the callback determines the password is invalid, the password will be passed along to the PasswordValidator in case it’s already been upgraded.

Persisting Rehashed Passwords

Whenever a validation attempt returns Result::SUCCESS_PASSWORD_REHASHED, it’s important to persist the updated password hash.

1
2
3
4

if ($result->getCode() === Result::SUCCESS_PASSWORD_REHASHED) {
    $rehashedPassword = $result->getPassword();
    // Persist rehashed password
}

While you can always perform the test and then update your user database manually, if you choose to use the Storage Decorator all rehashed passwords will be automatically persisted.

The Storage Decorator takes two constructor arguments: An instance of PasswordValidatorInterface and an instance of the JeremyKendall\Password\Storage\StorageInterface.

StorageInterface

The StorageInterface includes a single method, updatePassword(). A class honoring the interface might look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

<?php

namespace Example;

use JeremyKendall\Password\Storage\StorageInterface;

class UserDao implements StorageInterface
{
    public function __construct(\PDO $db)
    {
        $this->db = $db;
    }

    public function updatePassword($identity, $password)
    {
        $sql = 'UPDATE users SET password = :password WHERE username = :identity';
        $stmt = $this->db->prepare($sql);
        $stmt->execute(array('password' => $password, 'username' => $identity));
    }
}

Storage Decorator

With your UserDao in hand, you’re ready to decorate a PasswordValidatorInterface.

1
2
3
4
5
6
7
8

use Example\UserDao;
use JeremyKendall\Password\Decorator\StorageDecorator;

$storage = new UserDao($db);
$validator = new StorageDecorator(new PasswordValidator(), $storage);

// If validation results in a rehash, the new password hash will be persisted
$result = $validator->isValid('password', 'passwordHash', 'username');

IMPORTANT: You must pass the optional third argument ($identity) to isValid() when calling StorageDecorator::isValid(). If you do not do so, the StorageDecorator will throw an IdentityMissingException.

Validation Results

Each validation attempt returns a Result object. The object provides some introspection into the status of the validation process.

• Result::isValid() will return true if the attempt was successful
• Result::getCode() will return one of three possible int codes:
  • Result::SUCCESS if the validation attempt was successful
  • Result::SUCCESS_PASSWORD_REHASHED if the attempt was successful and the password was rehashed
  • Result::FAILURE_PASSWORD_INVALID if the attempt was unsuccessful
• Result::getPassword() will return the rehashed password, but only if the password was rehashed

Database Schema Changes

As mentioned above, because this library uses the PASSWORD_DEFAULT algorithm, it’s important your password field be VARCHAR(255) to account for future updates to the default password hashing algorithm.

Helper Scripts

There are two helper scripts available, both related to the password hash functions (these functions are only available after running composer install).

version-check

If you’re not already running PHP 5.5+, you should run version-check to ensure your version of PHP is capable of using password-compat, the userland implementation of the PHP password hash functions. Run ./vendor/bin/version-check from the root of your project. The result of the script is pass/fail.

cost-check

The default cost used by password_hash is 10. This may or may not be appropriate for your production hardware, and it’s entirely likely you can use a higher cost than the default. cost-check is based on the finding a good cost example in the PHP documentation. Simply run ./vendor/bin/cost-check from the command line and an appropriate cost will be returned.

NOTE: The default time target is 0.2 seconds. You may choose a higher or lower target by passing a float argument to cost-check, like so:

1
2

$ ./vendor/bin/cost-check 0.4
Appropriate 'PASSWORD_DEFAULT' Cost Found:  13

Wrapping Up

The addition of native password hashing functions is the most important security update to PHP since, well, I don’t know when. There’s no excuse for not implementing them in your applications, and the Password Validator library makes it trivial. That’s especially true when it comes to updating your legacy password hashes, which many of us need to do. Even if you only use the Password Validator as a roadmap for your own implementation, I strongly recommend upgrading ASAP.

Kudos

I was remiss to not add this bit of kudos when I originally published this post. Better late than never.

Credit for the new password hashing functions goes to Anthony Ferrara. He submitted the original RFC and created the password-compat library. The PHP community owes Anthony a debt of gratitude for making password hash security so ridiculously simple. Seriously, if I can grok it, you know it’s idiot proof :-)

Without Anthony’s hard work (and PHP core’s unanimous ‘Yes’ votes, and the password-compat contributors), my small contribution wouldn’t have been possible. Kudos to you all.

Managing dependencies via Composer is one of the most revolutionary advancements in the history of PHP. Composer packages are frequently hosted on Github, listed on Packagist, and required in your project via the require field in composer.json.

So Where is phpass?

What happens when that’s not the case? One library of note, phpass, is not available on Github (or any other supported VCS)¹ and therefore can’t simply be added to the require field for easy installation. All is not lost, however, thanks to Composer’s package repository feature².

Behold, Composer’s ‘Package’ Repository!

After reviewing the package repository docs, I found it ridiculously easy to require phpass in my project. Here’s what you have to do.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

{
    "repositories": [
    {
        "type": "package",
        "package": {
            "name": "openwall/phpass",
            "version": "0.3",
            "dist": {
                "url": "http://www.openwall.com/phpass/phpass-0.3.tar.gz",
                "type": "tar"
            },
            "autoload": {
                "classmap": ["PasswordHash.php"]
            }
        }
    }
    ],
    "require": {
        "openwall/phpass": "0.3"
    }
}

Now you can run composer install (or composer update, as appropriate) and Composer will install phpass as a project dependency. Sweet!

UPDATE - CLARIFICATION: Using phpass is only advisable for PHP versions that won’t support the new password hashing functions. That’s any version of PHP less than 5.3.7:

• PHP >= 5.5: Password hashing functions available natively
• PHP >= 5.3.7, < 5.5: password_compat provides forward compatibility
• PHP < 5.3.7: phpass is the gold standard

If you’re at PHP >= 5.3.7, enjoy this article as a Composer tip you might not have know about until now and use password_compat. If you’re at PHP < 5.3.7, this is both a Composer tip and an admonition to upgrade you password security. Do it!

Many thanks to Meroje and @craig_bass for pointing out that password_compat is superior, making it clear that I needed to post a clarification.

How can I avoid setting up two separate routes [with different cases] that trigger the same callback function?

I found the question intriguing because 1) I love Slim and 2) I’ve never really thought about whether or not URLs are case-sensitive to begin with. My immediate thought was, “They’re already case-insensitive! Has dude even tested this?”

Are URLs Case-sensitive?

Well, kinda. It frequently, but not always, boils down to whether or not the web server’s filesystem is case sensitive. The HTTP server (Apache, nginx, etc) can get involved, as they can (always?) be configured to serve URLs with or without regard to case sensitivity. The web application and/or framework is involved too. If all other factors are case-insensitive but your application’s router is case-sensitive, then your application’s URLs will be too. This is the case with the Slim Framework.

Should URLs Be Case-sensitive?

Again, kinda. In “HTML and URLs”, the w3c has this to say:

URLs in general are case-sensitive (with the exception of machine names). There may be URLs, or parts of URLs, where case doesn’t matter, but identifying these may not be easy. Users should always consider that URLs are case-sensitive.

In general and should are the operative words there. It’s OK either way¹, but assuming case-sensitivity is prudent.

So What About Slim?

URLs in the Slim Framework are case-sensitive². That’s correct based on what we learned about URL case-sensitivity from the w3c, but what if you have a use case that requires case-insensitive URLs? What do do then?

The Magic of Slim Hooks

What are Slim Hooks? From the documentation:

A “hook” is a moment in the Slim application lifecycle at which a priority list of callables assigned to the hook will be invoked. A hook is identified by a string name.

Slim provides six default hooks, three invoked before the current route is dispatched and three invoked after. By using one of the hooks that’s invoked before the route is matched, we can alter the incoming URL’s path to match the case of the routes we’ve defined in our Slim application.

The Case-insensitive Route Hook

First, if there are any routes with mixed case, change them all to lower case. That’s the de facto standard for creating routes anyhow, and if any requests come in that match the old, mixed case routes, we’ll take care of those with the hook in the below example. Same issue, we’re just flipping it on its head. Seriously, don’t get weird about changing those routes.

Next, register this callback on the slim.before.router hook:

1
2
3

$app->hook('slim.before.router', function () use ($app) {
    $app->environment['PATH_INFO'] = strtolower($app->environment['PATH_INFO']);
});

This works because Slim matches the routes you’ve defined against the Slim Environment’s PATH_INFO (originally found in $_SERVER['PATH_INFO']). Since your routes are lower case and the incoming request paths are lower case, you’ve accomplished case insensitive routing in 3 lines of code. BOOM.

I’m currently making heaviest use of Guzzle in my photo-a-day project, Flaming Archer, in order to get photo data from Flickr. To keep from hammering the Flickr API, I’m caching all of those requests. Guzzle makes caching ridiculously easy by way of their plugin system and their HTTP Cache plugin.

The problem with the caching plugin, at least at first blush, is how to bypass the cache in certain specific instances where caching might not be appropriate. The docs are a little light in this area, so it took me a few minutes to get it sorted out. Let’s start at the top.

The Guzzle Client

“Clients create requests, send requests, and set responses on a request object. When instantiating a client object, you can pass an optional "base URL” and optional array of configuration options."

Here’s an example of creating a Guzzle Client, based on my use case of making requests against the Flickr API.

1
2
3
4
5
6
7
8

use Guzzle\Http\Client;

$client = new Client('http://api.flickr.com');
$client->setDefaultOption('query', array(
    'api_key' => 'EXAMPLE_API_KEY',
    'format' => 'json',
    'nojsoncallback' => 1,
));

I use the client for the GET requests I need to make against the Flickr API. Each request will include the above default options in the query string. Nice!

Adding Caching

Since I don’t want to hammer the crap out of the Flickr API and start hitting the rate limit¹, I wanted to cache each request. Thankfully, Guzzle has an awesome plugin system that includes an HTTP Cache plugin.

“Guzzle can leverage HTTP’s caching specifications using the Guzzle\Plugin\Cache\CachePlugin. The CachePlugin provides a private transparent proxy cache that caches HTTP responses.”

Rather than rolling my own caching strategy (My first solution was to write a decorator for caching), I decided to use Guzzle’s native plugin and leave all the caching work to them.

1
2
3
4
5
6
7
8
9
10
11

use Guzzle\Cache\Zf2CacheAdapter;
use Guzzle\Plugin\Cache\CachePlugin;
use Guzzle\Plugin\Cache\DefaultCacheStorage;
use Zend\Cache\Backend\TestBackend;

$backend = new TestBackend();
$adapter = new Zf2CacheAdapter($backend);
$storage = new DefaultCacheStorage($adapter);
$cachePlugin = new CachePlugin($storage);

$client->addSubscriber($cachePlugin);

The cache plugin will now intercept and cache GET and HEAD requests made by the client.

Custom Caching Decisions

So what if, now that you’re caching each GET request, there’s a request or requests you don’t want cached? Guzzle makes solving that problem trivial by allowing for “custom caching decisions”, but the documentation on how to make those custom decisions is decidedly light.

“… you can set a custom can_cache object on the constructor of the CachePlugin and provide a Guzzle\Plugin\Cache\CanCacheInterface object. You can use the Guzzle\Plugin\Cache\CallbackCanCacheStrategy to easily make a caching decision based on an HTTP request and response.”

Wat?

That was clear as mud to me, so I spent a few minutes digging through the source. This is what I came up with:

• The CallbackCanCacheStrategy provides a method of providing callbacks to the cache plugin that, based on a boolean response, determine whether or not a particular request or response should be cached.
• The CallbackCanCacheStrategy accepts two optional arguments to its constructor: a callable that will be invoked for requests and a callable that will be invoked for responses. The request callback gets an instance of Guzzle\Http\Message\RequestInterface, and the response callback gets an instance of Guzzle\Http\Message\Response.

Bypassing Cache

In my case, I want to cache everything except for calls to the flickr.photos.search API method. Since all of the GET requests I’m making include a method query string param, it was trivial to write the callback that got me where I needed to go.

1
2
3
4
5
6
7
8
9

$canCache = new CallbackCanCacheStrategy(
    function ($request) {
        if ($request->getQuery()->get('method') === 'flickr.photos.search') {
            return false;
        }

        return true;
    }
);

Putting It All Together

Now that I’ve built my $client, $storage, and $canCache strategy, here’s how I put it all together.

1
2
3
4
5
6

$cachePlugin = new CachePlugin(array(
    'can_cache' => $canCache,
    'storage' => $storage,
));

$client->addSubscriber($cachePlugin);

Now all of my GET requests are cached except for those using the flickr.photos.search method. BOOM.

The last time I worked with Capistrano was about two years ago, and a lot has changed since then. Capistrano v3 was released in June of 2013 and brought with it a lot of great changes, but for a guy who doesn’t know ruby and relies on tutorials and Stack Overflow questions for help, the version bump brought a lot of pain as well.

Challenges

Every Tutorial is Wrong

Just know going into this that (almost) every tutorial you find is going to be a Capistrano v2 tutorial. Enough has changed between v2 and v3 to make those tutorials just misleading enough to cause a good amount of pain.

Stack Overflow is a Capistrano v3 Desert

As of this writing, there are ten questions tagged capistrano3 on Stack Overflow. Seriously. Ten. And only four of those include accepted answers.

Capistrano v3 Documentation is Lacking

The documentation available for v3 is seriously lacking, although the problem is more one of quantity than quality. What’s available is good, there’s just nowhere near enough of it.

Caveat Emptor

These are indeed “Notes to Self”. I hope they help you out, but if they don’t, I’m giving you fair warning. Please feel free to add what’s missing to the comments.

Reading the Capistrano docs is highly recommended. These notes are supplemental.

PHP + Capistrano v3

NOTE: You can find the application source, which now includes Capistrano v3, on GitHub.

Ruby

First, I had to reinstall rvm, a ruby version manager. What I don’t know about installing a ruby dev environment could fill books, so I let rvm take care of that for me.

1

\curl -L https://get.rvm.io | bash -s stable --ruby

(Capistrano requires Ruby 1.9 or newer, so the current stable Ruby from rvm will work fine.)

Installation

Install Capistrano. There are a few options, but I used gem install.

1

gem install capistrano

NOTE: All of the PHP tutorials I found instruct you to install the railsless-deploy gem. As Capistrano v3 “doesn’t ship with any Railsisms”, this is no longer necessary and the railsless-deploy project is obsolete.

I’ll probably go back and add a Gemfile since I’ll be adding the composer gem and want everything I need in one place.

Preparing Your App

The capify command is no longer, it’s now cap install.

1
2

cd /path/to/project
cap install

The documentation on this portion, “Preparing Your Application”, is one of the places the documentation shines, IMHO.

Capistrano Files

• Capfile: Kind of like a bootstrap. Takes care of necessary required configs and globs for custom tasks.
• config/deploy.rb: Common items across all deployments go here
• config/deploy/{staging, production}.rb: Environment specific deployment settings

Roles

I’m still not 100% clear on this, but roles don’t seem to be roles in the ACL sense, but rather roles in the “division of server responsibility” sense, hence the roles :web, :app, and :db.

The docs say that you can dump the :app and :db roles if you like, but if you’re going to use the :linked_files and :linked_dirs feature (which are pretty cool) you’ll need to leave the :app role in place. I’m obviously doing something wrong or missing something here, but that’s what I had to do.

I found it extremely helpful to refer to the deploy.rb.erb template. I removed most of the example text before realizing I needed to use some of it, and referencing the template was nice.

SSH Forwarding

SSH Agent Forwarding is the way to go here, IMHO. I already had a key agent running so the forwarding was dead simple, almost like magic. If you don’t already have an agent running, here’s some good info from GitHub Help.

Server Config

The section on how to set up the proper Capistrano directories on your server is way down deep in the Authorisation portion of the docs. Short version: you need two dirs in the root of your project (on the server): releases and shared, and they must be readable and writeable by both the webserver and deploy user.

Getting permissions right is easy for some, but I always return to the Setting Up Permissions section of the Symfony2 docs to get them set up properly.

Making Composer Work

Once I got my deploy.rb and deploy/production.rb right and tested (by deploying, natch), I needed to create a task to run composer install. Getting that right turned out to be pretty difficult because of a design decision in SSHKit.

Short story: No spaces in command line commands.

I finally got my composer command running by doing this:

1
2
3
4
5
6
7
8
9
10

desc 'composer install'
task :composer_install do
    on roles(:web) do
        within release_path do
            execute 'composer', 'install', '--no-dev', '--optimize-autoloader'
        end
    end
end

after :updated, 'deploy:composer_install'

• The within release bit in the command tells the task to cd into the release directory before running composer.
• There are before and after hooks you can apply to the deploy flow. The final line above is a hook that runs after deploy:updated.

Composer Gem for Capistrano v3

Of course, as soon as I got it working, Peter Mitchell sent me this tweet:

I haven’t yet replaced my hacked version with the gem, but I’ll use it as soon as I do any refactoring.

Deployment Annoyances

cap production deploy --dry-run never worked for me, although cap production deploy worked fine. No idea why, who cares at this point. Maybe I’ll dig in later.

Also, cap production deploy really wants to run deploy:restart, even though it doesn’t show up anywhere in the deploy flow. I replaced the :restart task that’s in the default deploy.rb template, made sure it was empty, and deploy finally worked.

Linked Files and Directories

The :linked_files and :linked_directories feature is really nice. I’ve used it for logs, a local.php config, Twig caching, my SQLite database, and my generated rss file. The linked items are for files and dirs that should be shared between deploys.

Also, those files and dirs need to be present in the shared dir before deploying. Deploy will puke if they’re not present.

Capistrano Variables

I couldn’t find a listing of these anywhere, but release_path is one of them, and it points to the latest release path.

That’s All For Now

I hope the notes help when you get ready to write your own Capistrano v3 scripts for use with PHP. I’ll update this as I learn more, and I’ll make sure to point out any excellent point made in the comments.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
[default] Importing base box 'precise64'...
[default] Matching MAC address for NAT networking...
[default] Setting the name of the VM...
[default] Clearing any previously set forwarded ports...
[default] Creating shared folders metadata...
[default] Clearing any previously set network interfaces...
There was an error while executing `VBoxManage`, a CLI used by Vagrant
for controlling VirtualBox. The command and stderr is shown below.

    Command: ["hostonlyif", "create"]

    Stderr: 0%...
    Progress state: NS_ERROR_FAILURE
    VBoxManage: error: Failed to create the host-only adapter
    VBoxManage: error: VBoxNetAdpCtl: Error while adding new interface: failed to open /dev/vboxnetctl: No such file or directory

    VBoxManage: error: Details: code NS_ERROR_FAILURE (0x80004005), component HostNetworkInterface, interface IHostNetworkInterface
    VBoxManage: error: Context: "int handleCreate(HandlerArg*, int, int*)" at line 68 of file VBoxManageHostonly.cpp

Googling for Failed to create the host-only adapter didn’t seem to return anything useful, so I tried with failed to open /dev/vboxnetctl and immediately found an answer.

The trick, according to GitHub user lslucas, is to restart VirtualBox from the command line like so:

1

 $ sudo /Library/StartupItems/VirtualBox/VirtualBox restart

That worked like a champ for me, and I was immediately able to get back to work.

For what it’s worth, here’s my current software config:

• VirtualBox 4.2.16
• Vagrant 1.2.2
• Mac OS X 10.8.4

• My wife Megan and I are moving back to Memphis, TN
• All of our family is there
• The pregnancy has been tough, we’d like to raise our son around family
• I’m leaving Nashville, but I’ll still be working for OpenSky
• I’ll miss you all terribly, but I’ll be back regularly

So Long, And Thanks For All The Awesome

I might as well get right to it, especially since I ruined the surprise above: Megan and I have decided to move back to Memphis. The decision to leave Nashville, especially after spending such a short time here, was a really difficult one, but I’m convinced this is the best decision for me and my family.

A Little Background

Megan and I are both Memphis natives. We’ve got deep roots in the “Bluff City”, and both of our families (excepting one of my brothers and his family) are still there. We moved to Nashville late last year after the incomparable Scott Gordon hooked me up with an awesome gig here in town.

In April of this year we found out we’d be having a baby! That was amazing, wonderful news, but the pregnancy has been difficult from day one. Recently, things got really, really scary. After a lot of discussion and soul searching, Megan and I decided the best decision for us and our new family would be moving home to be around the support, care, and help of our families.

I’m Leaving, But Only Mostly

So come mid- to late-October, when our current lease is up, we’re heading back to M-town. There is a huge Nashville-related silver lining, however. OpenSky is letting me move to Memphis while staying on staff with them! That means regular trips back to Nashville to work in the local office, get some face time with the team, and hang out with all my Nashville friends.

So Let’s Do Lunch

Nashville friends, we need to do lunch, pronto. Two months seems like a long time, but I’ll be driving that moving truck west on I-40 before we know it. I want to get together while it’s still relatively easy to do so. I’ve spent to little time with y'all as it is.

MEMPHIS, I AM ALMOST BACK IN YOU

Leaving Nashville is sad, coming back to Memphis is going to be awesome. I’m looking forward to hanging out with my Memphis people, getting involved in the Memphis PHP and the #memtech communities again, reconnecting with Memphis Roller Derby, and being able to see the fam whenever.

Exciting Times Ahead

Things have been wild the past few years, and they’re about to get a lot more exciting. Change is tough, and I’m feeling the pressure, but I feel great about where I’m at, where Megan and I are at, and our future. I can’t wait to see what comes next.

As of 06:58 CDT, I'm on the trending dev and trending repo lists on GitHub. Two thumbs, internet famous, this guy. https://t.co/stWaUkV34e

— Jeremy Kendall (@JeremyKendall) August 16, 2013

Thanks to yesterday’s link love from PHPDeveloper, I’ve made both the Trending PHP Developer list and the Trending PHP Repo list on GitHub.

With internet fame being so fleeting, I took some screenshots for posterity. Now please excuse me while I get some ice for my arm. I seem to have injured myself while patting myself on the back.

Philosophy

Query Auth is intended to be – and is written as – a bare bones library. Many of niceties and abstractions you’d find in a fully featured API library or SDK are absent. The point of the library is to provide you with the ability to focus on writing the meat of your API while offloading the authentication bits.

What’s Included?

There are three components to Query Auth: request signing for API consumers and creators, request signature validation for API creators, and API key and API secret generation.

Request Signing

1
2
3
4
5
6
7
8
9
10
11
12

$collection = new QueryAuth\NormalizedParameterCollection();
$signer = new QueryAuth\Signer($collection);
$client = new QueryAuth\Client($signer);

$key = 'API_KEY';
$secret = 'API_SECRET';
$method = 'GET';
$host = 'api.example.com';
$path = '/resources';
$params = array('type' => 'vehicles');

$signedParameters = $client->getSignedRequestParams($key, $secret, $method, $host, $path, $params);

Client::getSignedRequestParams() returns an array of parameters to send via the querystring (for GET requests) or the request body. The parameters are those provided to the method (if any), plus timestamp, key, and signature.

Signature Validation

1
2
3
4
5
6
7
8
9
10
11
12
13
14

$collection = new QueryAuth\NormalizedParameterCollection();
$signer = new QueryAuth\Signer($collection);
$server = new QueryAuth\Server($signer);

$secret = 'API_SECRET_FROM_PERSISTENCE_LAYER';
$method = 'GET';
$host = 'api.example.com';
$path = '/resources';
// querystring params or request body as an array,
// which includes timestamp, key, and signature params from the client's
// getSignedRequestParams method
$params = 'PARAMS_FROM_REQUEST';

$isValid = $server->validateSignature($secret, $method, $host, $path, $params);

Server::validateSignature() will return either true or false. It might also throw one of three exceptions:

• MaximumDriftExceededException: If timestamp is too far in the future
• MinimumDriftExceededException: It timestamp is too far in the past
• SignatureMissingException: If signature is missing from request params

Drift defaults to 15 seconds, meaning there is a 30 second window during which the request is valid. The default value can be modified using Server::setDrift().

Key Generation

You can generate API keys and secrets in the following manner.

1
2
3
4
5
6
7
8
9

$randomFactory = new \RandomLib\Factory();
$keyGenerator = new QueryAuth\KeyGenerator($randomFactory);

// 40 character random alphanumeric string
$key = $keyGenerator->generateKey();

// 60 character random string containing the characters
// 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ./
$secret = $keyGenerator->generateSecret();

Both key and secret are generated using Anthony Ferrara’s RandomLib random string generator.

That’s Kinda Ugly, Dude

As I pointed out, the Query Auth library is pretty bare bones. There are a lot of opportunities for abstraction that would make the library much easier to use and much nicer to look at. If I added them to Query Auth, however, that would lock library users into whichever HTTP client I chose to use. The same concern would go for whatever other abstractions I decided on. The point here is to offload query authentication, and only query authentication, to the Query Auth library.

Sample Implementation

In order to demonstrate how one might implement the Query Auth library, I’ve whipped up a sample implementation for you.

The sample uses Vagrant and VirtualBox to allow you to see the whole thing in action. Slim Framework runs the API, Guzzle is used to make requests to the API, and both a GET and POST request are implemented. JSend, Jamie Schembri’s PHP implementation of the OmniTI JSend specifiction, is used to send messages back from the API, and Parsedown PHP, Emanuil Rusev’s Markdown parser for PHP, is used to render the sample implementation’s documentation.

Request Signing

In the sample implementation, request signing has been abstracted in the Example\ApiRequestSigner class. Signing requests is now as simple as passing the request object and credentials object to the signRequest method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

/**
 * Signs API request
 *
 * @param RequestInterface $request     HTTP Request
 * @param ApiCredentials   $credentials API Credentials
 */
public function signRequest(RequestInterface $request, ApiCredentials $credentials)
{
    $signedParams = $this->client->getSignedRequestParams(
            $credentials->getKey(),
            $credentials->getSecret(),
            $request->getMethod(),
            $request->getHost(),
            $request->getPath(),
            $this->getParams($request)
            );

    $this->replaceParams($request, $signedParams);
}

Signature Validation

In the sample implementation, signature validation has been abstracted in the Example\ApiRequestValidator class. Validating request signatures is now as simple as passing the request object and credentials object to the isValid method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

/**
 * Validates an API request
 *
 * @param  Request        $request     HTTP Request
 * @param  ApiCredentials $credentials API Credentials
 * @return bool           True if valid, false if invalid
 */
public function isValid(Request $request, ApiCredentials $credentials)
{
    return $this->server->validateSignature(
        $credentials->getSecret(),
        $request->getMethod(),
        $request->getHost(),
        $request->getPath(),
        $this->getParams($request)
    );
}

Signing a GET Request

Signing a request is now extremely clean and simple. Here’s the GET example from the sample implementation.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

/**
 * Sends a signed GET request which returns a famous mangled phrase
 */
$app->get('/get-example', function() use ($app, $credentials, $requestSigner) {

    // Create request
    $guzzle = new GuzzleClient('http://query-auth.dev');
    $request = $guzzle->get('/api/get-example');

    // Sign request
    $requestSigner->signRequest($request, $credentials);

    $response = $request->send();

    $app->render('get.html', array('request' => (string) $request, 'response' => (string) $response));
});

Validating a GET Request

Validating a GET request is equally clean and simple. Note the try/catch that handles possible exceptions from the validation class.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

/**
 * Validates a signed GET request and, if the request is valid, returns a
 * famous mangled phrase
 */
$app->get('/api/get-example', function () use ($app, $credentials, $requestValidator) {

    try {
        // Validate the request signature
        $isValid = $requestValidator->isValid($app->request(), $credentials);

        if ($isValid) {
            $mistakes = array('necktie', 'neckturn', 'nickle', 'noodle');
            $format = 'Klaatu... barada... n... %s!';
            $data = array('message' => sprintf($format, $mistakes[array_rand($mistakes)]));
            $jsend = new JSendResponse('success', $data);
        } else {
            $jsend = new JSendResponse('fail', array('message' => 'Invalid signature'));
        }
    } catch (\Exception $e) {
        $jsend = new JSendResponse('error', array(), $e->getMessage());
    }

    $response = $app->response();
    $response['Content-Type'] = 'application/json';
    echo $jsend->encode();
});

Sample Request and Response

The code above produces the below request and response:

Request

1
2
3

GET /api/get-example?key=ah5yEgQzjuFsC9nWsRI4Nar3ikOqWVPcD3OntHpg&timestamp=1376416267&signature=3DqimkvigYBorGi8wHfil9lB8oCWhB%2BHYt6rVfE4zx4%3D HTTP/1.1
Host: query-auth.dev
User-Agent: Guzzle/3.7.2 curl/7.22.0 PHP/5.5.1-2+debphp.org~precise+2

Response

1
2
3
4
5
6
7
8

HTTP/1.1 200 OK
Date: Tue, 13 Aug 2013 17:51:07 GMT
Server: Apache/2.4.6 (Ubuntu)
X-Powered-By: PHP/5.5.1-2+debphp.org~precise+2
Content-Length: 75
Content-Type: application/json

{"status":"success","data":{"message":"Klaatu... barada... n... necktie!"}}

Wrapping Up

So there you have it: QueryAuth to sign and validate API requests (and generate keys and secrets!) and a sample implementation to get you going. If you find this helpful, or have any questions or comments, please let me know. If you find any horrible mistakes, please feel free to submit an issue or a pull request, or you can always submit the offending code to CSI: PHP :-)

Having trouble getting your Synced Folders permissions just right in your Vagrant + VirtualBox VM? They’ve been giving me some grief lately. Here are the (undocumented) Vagrantfile options that finally got it sorted out.

Permissions Challenges

The issue that got me digging into this was trying to get permissions just so to allow my web apps to write logs. As you probably know, apache (or your web server of choice), sometimes needs write access to certain web application directories. I’ve always take care of that by adding the apache user group to the directories in question and then giving that user full access. Doing that in Ubuntu looks something like:

1
2

chown -R jeremykendall.www-data /path/to/logs
chmod 775 /path/to/logs

If you’ve tried doing something similar in your Vagrant shared folders, you’ve likely failed. This, as it turns out, doesn’t work with VirtualBox shared folders – you have to make the changes in your Vagrantfile.

Setting Permissions via the Vagrantfile

UPDATE: Thanks to Joe Ferguson for pointing out in the comments that Vagrant has been upgraded and my example was no longer current. Below are both examples marked by Vagrant version.

Here’s my new synced_folder setting in my Vagrantfile:

Vagrant v1.1+:

1
2
3
4
5

  # Vagrant v1.1+
  config.vm.synced_folder "./", "/var/sites/dev.query-auth", id: "vagrant-root",
    owner: "vagrant",
    group: "www-data",
    mount_options: ["dmode=775,fmode=664"]

Vagrant 1.0.x:

1
2
3
4
5

  # Vagrant v1.0.x
  config.vm.synced_folder "./", "/var/sites/dev.query-auth", id: "vagrant-root",
    :owner => "vagrant",
    :group => "www-data",
    :extra => "dmode=775,fmode=664"

I’m sure you can immediately see what resolved the issue. Lines 3 and 4 set the owner and group, respectively, and line 5 sets directory and file modes appropriately. That simple fix was frustratingly difficult because I couldn’t find it documented anywhere. After much searching and opening far too many browser tabs, I cobbled together the info above. A quick vagrant reload later and I was off to the races.

UPDATE: Alternate Method

An alternate method that doesn’t include modifying your synced folder permissions is changing the web user to the vagrant user. Bad idea? Security problem? Not on your dev VM it ain’t, and that’s good enough for me. Big thanks to Chris Tankersley for all the help getting this one figured out.

Chris and I both put together gists, and this is how I’m currently doing it in Flaming Archer, but probably the best method for changing the apache user to the vagrant user comes from the Intracto Puppet apache manifest.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

# Source https://raw.github.com/Intracto/Puppet/master/apache2/manifests/init.pp

# Change user
exec { "ApacheUserChange" :
    command => "sed -i 's/APACHE_RUN_USER=www-data/APACHE_RUN_USER=vagrant/' /etc/apache2/envvars",
    onlyif  => "grep -c 'APACHE_RUN_USER=www-data' /etc/apache2/envvars",
    require => Package["apache2"],
    notify  => Service["apache2"],
}

# Change group
exec { "ApacheGroupChange" :
    command => "sed -i 's/APACHE_RUN_GROUP=www-data/APACHE_RUN_GROUP=vagrant/' /etc/apache2/envvars",
    onlyif  => "grep -c 'APACHE_RUN_GROUP=www-data' /etc/apache2/envvars",
    require => Package["apache2"],
    notify  => Service["apache2"],
}

Additionally, if you’re copying and pasting from anywhere, don’t forget to change the apache lockfile permissions:

1
2
3
4
5
6
7

# Source https://github.com/Intracto/Puppet/blob/master/apache2/manifests/init.pp

exec { "apache_lockfile_permissions" :
    command => "chown -R vagrant:www-data /var/lock/apache2",
    require => Package["apache2"],
    notify  => Service["apache2"],
}

ACL on Shared Folders That Are Not NFS

One of the reasons the above methods are necessary is that you can’t use ACLs on shared directories. If none of the above options appeal to you, it’s possible to use ACLs on your VM as long as the directories aren’t shared. For more information, see Frank Stelzer’s comment regarding setfacl on a Vagrant box.

Woe to you, o earth and sea, for the devil sends the beast with wrath, because he knows the time is short … Let him who hath understanding reckon the number of the beast: for it is a human number; its number is six hundred and sixty six.

Before I delve into why I’m shocked and dismayed, I want to say a few things about Composer and the Composer team. In all of my years of programming in PHP, I’m not sure there’s been a more important, more game changing, or more exciting project than Composer. Being able to easily manage project dependencies has revolutionized the way I develop. Composer, and the related packagist.org, have been a larger quality-of-life improvement for me than any other tool I’ve added to my toolkit over the years. I’d like to extend my sincerest thanks to Nils Adermann, Jordi Boggiano and the many community contributors who have worked so hard and so diligently to make Composer a reality.

My Beef with the Change

1. There was never any public discussion about the change

Beyond a few brief asides in a couple of github issues and pull requests, I can’t find anywhere this change discussed publicly. For a project of this size and this importance, removing the community from the decision making process was a terrible mistake. I’d much prefer to have argued all of this before the change than after.

Yesterday, Jordi posted “Composer: Installing require-dev by default” to explain his rationale for the change. One of his points is that he made a note in the 1.0.0-alpha changelog regarding the upcoming change. While this is true, I find it insufficient. I rarely read changelogs (my bad), but I certainly don’t read changelogs to discover what’s coming in the future. That’s what road maps, blog posts, and PRs are for. Putting that note in the changelog was “too little, too early”.

2. Composer philosophy and workflow has always been, “install for production, update for development”

The longstanding Composer rule of thumb has always been “install for production, update for development”. Adam Brett’s post on Composer workflow and the difference between update and install is a good example of this rule of thumb. Humorously enough, Jordi reinforces that rule of thumb (while defending the change that composer update installs dev requirements by default) in his blog post immediately prior to the post defending the composer install change:

“The install command on the hand remains the same. It does not install dev dependencies by default, and it will actually remove them if they were previously installed and you run it without –dev. Again this makes sense since in production you should only run install to get the last verified state (stored in composer.lock) of your dependencies installed.”

That rule of thumb is now turned on its head, and the default “composer install in production” advice now needs updating, careful warnings, and caveats.

3. Tools should never default to dev (unless they’re meant for dev, of course)

This is a philosophical point on my part, but it’s one I don’t think is unique to me and it’s one that I think can be well defended. My point here is that one should always write code and tools in such a way that deployments to production will only ever result in production code being deployed. Clear as mud? Let me try with an example.

I frequently use environment variables to allow my applications to detect which environment they’re running in. If those environment variables don’t exist, then the application should default to production. Why? Because dev environments are the special case, not production, and it’s far too easy to forget to add those environment variables when deploying. I make my life easier by making the production environment as idiot-proof as possible, and not the other way around.

This philosophy was in place in the prior behavior of composer install (and composer update, for that matter). Now that it’s changed, the production environment is far more likely to suffer than the development environment. Forgetting to add the –dev flag in development is a lot less (potentially) costly than forgetting to add the –no-dev flag in production.

In a seeming contradiction, I’ve said that I have no problem with composer update defaulting to installing dev dependencies. I’ve gone back and forth on that a bit when considering my “tools should never default to dev” position, but I don’t think I’m being inconsistent here. Since the rule of thumb encourages using update in development and never in production, then update becomes a dev tool which can safely default to installing development dependencies. Having said that, if consistency between commands is important, then composer update should no longer default to dev and the changes to both install and update should be reverted.

In Closing

Composer has become an integral part of my workflow, and a critical piece of the PHP development process in general. I loved Composer before this change and I’ll love Composer after. That said, changing the Composer command that is intended primarily for production use is extremely disruptive and a very bad call, especially considering how the change came about.

As I write this follow-up post, the gas has been turned back on, my hot water heater is working away, and my wife and I find ourselves on the far side of a bad situation turned good.

We last left off with a phone call from Piedmont corporate and a promise to have our gas turned back on by 5 pm CST. Not only did Piedmont come through, they came through big. The technician they sent was early to the appointment and went above-and-beyond to make sure we were taken care of. While I wish this entire situation never happened, the outcome is the very best of a bad situation, and I want to close this out with a big thanks to Piedmont Natural Gas.

Once Piedmont corporate learned about our problem, they went the extra mile to resolve it to our satisfaction as quickly as possible. I don’t know everything they did, but they went as far as reviewing the recordings of both our Friday and Saturday customer service phone calls, calling both my wife and myself to schedule reconnection, and taking the time to explain exactly what happened, why it happened, and then take responsibility for it. Kudos to Piedmont.

Thanks also to all of you for being so supportive. In the grand scheme of things, this was small potatoes, but that didn’t make it any less upsetting. The retweets, supportive comments, and personal commiserations went a long way towards making this a lot easier.