My WordPress Development Workflow by Tim Nash

I love reading about people’s workflows, and each time I read about someone else’s setup I take something away. I’m inspired by folks who spend half their day at standing tables (not inspired enough to actually give standing a go mind you) and who have built contraptions to aid their day. Yet workflow is about more than just hardware and posture. In this post I thought I would take you through the tools that I use day-to-day.

This post will be the first in a series of posts that are a sort of – How I do X? What stuff I use day in day out? Along with why I do.

WARNING it’s worth remembering I don’t do much full on development any more and when I do I tend to be working on either more complex systems or working with lots of unfamiliar legacy code. In addition to this I have my own sites and a few legacy clients who have unique and individual setups. Normally the remainder of my work with clients is based around AWS or a similar infrastructure.

Operating System & Apps

My main computer is a Macbook Pro 2010 and my day-to day OS is OSX Mavericks which I jokingly say is the prettiest terminal GUI going. I have MacPorts on, and a range of small tools and other bits more often found on a typical Unix box. I use MacPorts over HomeBrew for no reason other then that was the one I used first.

In terms of Apps it’s pretty light development wise:

Sublime Text 2
Chrome
Firefox Developer Edition
Compass.app
Sequel Pro & MySQLWorkbench

Sublime Text

My main text editor is Sublime Text 2, there is Sublime Text 3 but I have stuck with version 2 as all my plugins are compatible along with a few custom plugins. I don’t think there is much work in migrating plugins over but I haven’t investigated. In terms of other IDEs and text editors I have worked with or tried a range in the past including hip new offerings like Atom to mature products like PHPStorm. On the whole Sublime Text offers the right balance for me, but text editors are such personal choices. I still have a copy of Eclipse up to date and setup for occasional weird programming tasks, but prefer not to fire it up unless needed.

On the command line, I use Nano over VIM (I just heard the audible gasp, from 90% of readers) my first Unix experience (which was in the early 90s, coming from a BBC/Acorn background) made use of Pine (Email client) which had Pico built in which became Nano. So I just used Nano, while I am utterly convinced by the superiority of emacs and VIM, I have never had a need to learn to use them properly.

With that admission out, my sublime plugin list (or vaguely relevant/stuff I use) is:

Sublime Code Intel
Sublime Text 2 WordPress
Hack Sublime Text
We Love WordPress fork of SublimePHPTidy
Sublime Linter with a custom HHVM linter
X-Debug Client
VCS Gutter
PHPCodesniffer

Several people have suggested, that with most of these plugins built in what I really want is PHPStorm and they may be right, it’s certainly heading in an interesting direction but at the moment I’m happy with my Sublime setup.

Where possible each project I work on is given its own Sublime project as most of the plugins have config that is project dependent as well as VM dependent. Where a project doesn’t have a vagrant instance setup for debugging I make use of the OSX version of software, though normally I have updated via MacPorts for example I have 3 separate versions of PHP running locally. Two of these are pre-vagrant days, in fact a lot of legacy software on this machine could be stripped off.

Most of the Sublime plugins are related to either tidying code (or making it fit a certain coding style) or debugging and are all pretty standard I think with nothing too exotic. A couple of Hack specific plugins for working with HHVM perhaps would not be in a normal WordPress dev sublime install.

A few plugins/things I don’t use within Sublime:

Git – I use from the command line, while I understand and am a proponent of commit often I still don’t like the idea of committing via an IDE. Plus the few second’s break to type git commit gives me a little reset moment. There are however I believe a couple of good git plugins for Sublime. I do also make use of VCS Gutter which shows Git/SVN Diffs, which I find handy.

In a similar vein, I don’t have Grunt or any work tasks setup to work automatically from Sublime, again preferring to fire up the command line as needed.

I have several different projects which use different workflows for working with Grunt and similar. Nearly all now run grunt tasks as part of provisioning/deploying of code either on Vagrant or remote server. This means precompiled and minified code is not held in the Repository, the downside is that everything needs to go through a provisioning tool or have its own version of the grunt tasks. I do (out of laziness) also have a copy of Compass for quickly working with Less and SASS files, normally where the project isn’t something I’m working on and I just want to get it up and running.

Compass.app

I mentioned Compass.app, it’s not something constantly open, but while I use compass from grunt, having a pretty GUI to one hand is useful for example if I’ve downloaded something to play with or when I just need to make a quick change to something that I don’t have a project for. In such cases I use compass.app, I don’t make heavy use of it and barely touch any of its features. Still it was a good purchase for a few pounds.

Chrome and Firefox

Browser wise, my main browser is Chrome, with my second browser being Firefox Developer Edition (which is also the one Selenium fires for some acceptance testing with Codeception when not using Webdriver).

Within Chrome I use the following extensions (for Dev work)

While there are a fair few there, most are pretty self explanatory. The WordPress.org SVN button extension is surprisingly useful providing one click access to the SVN repo of plugins on w.org from their plugins page. Postman is the de facto REST client for Chrome, while Form Editor allows modification of form data being sent to server, along with bits in the Chrome toolbar replacing Firefox Tamper Data extension. Appspector provides a one click glance at what technology a site is using, though it does occasionally bring back false positives.

With Firefox, I really only have Firebug and TamperData plus the default install for the Developer Edition.

Sequel Pro & MySQL Workbench

For managing databases I use Sequel Pro for Mac for most day-to-day commands, or switch to command line if I think it’s going to be process intensive. I prefer Sequel Pro over PHPMyAdmin even though it’s running locally. When working with DB design I use MySQL Workbench for EER diagrams and working out relationships. Though when it comes to working with WordPress this is rarely needed, to be honest I can’t remember the last time I used MySQL Workbench in anger.
Back to Top

Virtualisation

For development I use Vagrant to manage my virtualised instances, I run 4 separate vagrant setups at the moment:

VVV
Amazon AMI Mimic Build
Legacy Client Mimic
TimNash.co.uk Mimic

My various instances are setup for different tasks, for example VVV which has almost become the de facto setup for WordPress Development I primarily use for teaching on, so most of my screencasts are done using VVV. It’s also a good general environment to trash on a regular basis.

The Amazon AMI Mimic, represents a fairly standard EC2 instances designed for PHP, but with HHVM, Redis and Beanstalkd setup as well as some basic testing and development tools.

The Legacy Client is a fairly standard LAMP stack configured for Magento with some occasional tweaks and an Elastic Search on the machine, it represents a legacy client dedicated box.

TimNash.co.uk Mimic, well mimics this site which sits on Ubuntu Trusty with a slightly different setup to the Amazon AMI client as it’s running, Memcache, Gearman and HHVM. Like the Amazon AMI Mimic it has a few extra development tools as standard. While I call it the timnash.co.uk mimic I actually have probably half a dozen sites on a couple of servers with a similar configuration. Over time I’m porting them over to use AWS and switching which Vagrant instance I use.

If you’re getting into virtualisation, specifically Vagrant for WordPress development and are looking for a good intro, then I would suggest looking at VVV to get up and running, but don’t worry about having to use VVV pick what is comfortable for you to use. Installing and configuring a virtual machine, even using Vagrant can at times be a total pig, especially on Windows machines but once you start using virtualisation you won’t go back to using XAMP and similar.

It’s also worth remembering one of the big benefits of Vagrant is matching your production servers, or providing multiple testing environments while solutions like VVV get you going quickly you may want to build tailored images to match your existing setups.

One area where I tend to work differently to say how VVV is setup is I don’t make heavy use of shared folders. For example in VVV the /srv/www folder is accessible both within Vagrant and the host system. While I do use shared folders so I can make use of remote x-debug amongst other features and so I can put breakpoints into code without committing almost certainly broken code. When working with version controlled projects I maintain a copy of the provisioning tools grunt scripts so on a post commit hook locally, the grunt scripts are run on Vagrant’s side. I also can always manually run these scripts. What it does mean is that if I’m working on CSS styling, I either have to run Grunt manually or wait until I commit to see it processed. This isn’t an issue for me as I rarely work with CSS but I can imagine such workflows being problematic for front end devs. What it does mean is I have far more faithful Vagrant setups and I only have to maintain one set of scripts.
Back to Top

Testing & Performance Monitoring

When it comes to testing, performance and monitoring I use a pretty common suite of programs:

xdebug/HHVM implementation
xhprof/xhgui
WP-CLI
Codeception
CodebaseHQ Exceptions/Airbrake

X-Debug

Like most PHP developers I swear by x-debug for debugging applications, if you don’t make use of x-debug in your workflow and work with PHP you should go and check it out. Not only does it provide full stack traces, but allows you to set break points so you can step through your code. With integration with Sublime Text this is a key component to my general workflow. With recent versions of HHVM certain parts of x-debug have been included in HHVM by default though with minimal documentation. Still this has allowed me to make heavy use of Hack in the last couple of projects even if x-debug doesn’t always work within HHVM as you would expect.

XHprof

XHprof is a passive profiler, created by Facebook and designed for production sites to monitor performance of PHP applications. It’s particularly useful when looking to track down stray processes causing mayhem on your server. Combined with XHGui it gives you a visual representation, it’s not pretty, but is functional which is all you actually need. I have played with a fork of XHprof called Uprofiler however haven’t switched over.

PHP Unit & Codeception

For testing I use a combination WP-CLI for accessing PHPUnit Tests within WordPress and Codeception for acceptance and some general functional tests. I make far more use of acceptance and regression testing then I do of unit tests. I also make use of Codeceptions acceptance tests beyond standard development, for example using them as part of the provisioning process and for testing backups.

Codebase Exceptions / Airbrake

For handling exceptions, mainly on live sites, I push them into CodeBaseHQ which is the service I use for git repos and ticketing once a site is live. CodeBase uses an Airbrake compatible solution for receiving exceptions. Basically this is a HTTP request with information and partial stack trace of any error I choose to send. I use the Airbrake WordPress plugin, but as they hard code the Airbrake URL in I have simply replaced with CodeBase credentials.
Back to Top

Deployment/Provisioning & Code Management

Codebase
Github
DeployHQ – Legacy
Jenkins
PHPCI
Anisble/Grunt
Puppet

Codebase & Github

Most of my code is still held in CodebaseHQ which provides Git/SVN/Mercurial repos along with ticketing, wiki, exception monitoring and a pile of features I have never used. At Coding Futures we use to make heavy use of Codebase and it was at the core of all our development. With Coding Futures no more, I kept using Codebase but other than exception handling I’m barely using any features to justify continuing to pay for it and it’s on my to migrate list. Alternatives include bitbucket which would be free or using something like Gitlab on an EC2 instance pulling from S3, though this is one of those where it’s so much easier to use a hosted solution.

Codebase is however a useful system if you are looking for an all in one system, it’s a shame ATech the guys behind it seem to have left it to rot a bit as there really hasn’t been any noticeable development on it for a couple of years.

For public work related to the site I tend to use Github, I’m not a prolific githubber, and simply use it as a public repo for work that is normally worked on in a private repo on Codebase. Not for any other reason than that’s how I have always done it, so not something I would recommend to others. As I am planning a couple of open source, totally free plugins I do intend to make more heavy use of Github/Travis/ScrutinizerCI combination.

DeployHQ / Jenkins / PHPCI

For code deployment I use a mix of tools, I still have a couple of legacy sites making use of DeployHQ, another ATech product which I believe you can get for free through Codebase. DeployHQ does automatic and manual deployments and is a perfectly fine deployment tool. Most of my stuff works on a CI approach and is a combination of Jenkins and a few shell scripts, again nothing fancy, combined with Grunt run on the server rather then from Jenkins for precompiling and minification. If Ruby and Capistrano are more your thing then there is WordMove a Capistrano setup for WordPress deployments.

For Jenkins I make use of a few plugins including:

Build Pipeline View
Publish over SSH
Git or Github Plugin (or DVCS of your choice)
Codesniffer Plugin

The Publish over SSH is pretty much used for a couple of weird legacy cases, most deployment and provisioning is done with shell scripts.

I have experimented with PHPCI which is a deployment and continual integration server written in PHP and aimed at deploying PHP based applications. It’s an interesting project and well worth following, I fully intend to migrate at least one project (probably this site) to using PHPCI during the year.

For stuff on Amazon, deployment is slightly different, rather then pushing content to an existing EC2 instance, I tend to provision a new base image, update and pull from the repo. Run any Codeception tests against it to check everything is ok, and then kill off the server it’s replacing. The process is fairly seamless but does mean deployments can be 20/30 minutes plus, so it doesn’t lend itself to quick hot fixes or lots of little changes.

Puppet / Anisble / Grunt

For provisioning I use Puppet and Anisble along with good old Bash Scripts and WP-CLI, I have played with Chef and looked into Docker (which works slightly differently) again the choice of Puppet was based on the thing I looked at when I started doing provisioning.

For new projects I always now start with a base image, and setup provisioning and deployment for the live site at the start where possible as it gives me the flexibility of destroying and recreating projects with ease, it also makes migration from Vagrant to test to live easier. As most of my new projects are primarily AWS based these days this process is even more critical.

Puppet doesn’t have to just be for server management either, as I look to change hardware for my laptop and look at maintenance for the beast, I keep considering using Puppet to provision for local machines as well. Though it does seem a lot of effort, and I have only reinstalled OSX once in 4 years which again is putting me off. If you are in a small or mid size business then this is definitely something to consider though.

With Grunt I make use of compass, and uglify along with a custom plugin, I don’t run any PHP validation scripts with Grunt though it might be useful to do so, instead I do this on the pre-commit and as part of general debugging. In addition I use Codeception to run acceptance tests which while in itself will not specifically spot parse errors is designed to provide confidence that everything is ok. That said if working in large teams or with mixed dev setups having a code sanity check at this stage is not a bad idea. I use Grunt Watch on the Vagrant box which triggers other grunt plugin runs whenever files change within watched folders.
Back to Top

Other Tools

In addition to the above I do have a few additional tools which primarily aid development:

Balsamiq
Vagrant Manager
Alfred
Spotify / Coffitivity
XCode
Eclipse
Apache Condova/PhoneGap

These tools, while apps just didn’t fit into my other categories or are used less frequently they are in their own ways really important to my Dev workflow.

Balsamiq

Balsamiq is a mocking tool, for layouts, I am not a front end dev but do find it useful to have a rough visual idea of what things should look like. I don’t use Balsamiq for anything but drawing very rough sketches often to pass to designers or to provide the very basic of mockups. Still a useful tool.

Vagrant Manager

For monitoring Vagrant instances and also to keep them under control I use Vagrant Manager, while I don’t use it for launching instances to often, preferring to do that from Alfred, I do have it running and keeping an eye on what’s up.

Alfred

Alfred is a workflow tool, similar to spotlight on OSX, alt space brings up a large search box and you can search for pretty much anything. Alfred’s real power comes from its workflows the ability to run scripts within Alfred to control and retrieve information. For example managing Vagrant instances, opening Sublime Text, searching the WordPress codex or searching for a specific hook/action/filter within a folder. All of this can be done in the terminal, and each needs to be added as a workflow. But Alfred is a powerful OSX tool that if not currently in a terminal (for example writing a blog post) makes finding and doing tasks quick and easy.

Spotify and Coffitivity

Spotify provides music, enough said really however you tend to work on your own, often in an empty house you can if you are not careful go stir crazy. The solution is to get out and about, much is made of going and sitting in a coffee shop, which is fine, except I find coffee shops uncomfortable and distracting and for my nearby coffee shops to be breaching trading standards by calling themselves COFFEE shops. The answer at least for me is coffitivity.com which provides a range of audio loops of coffee shops. Simply fire up the site put head phones on along with music and there you have it, an instant coffee shop ambiance with out the need to actual go to a coffee shop.

Mobile Development

The final 3 things are all for working with Mobile apps, xcode/Eclipse and Apache Condova/Phonegap provide IDE and build tools to generate application from HTML/CSS/JS. I don’t do too much mobile dev work, but when I do this is my tool set. If I’m cursing about Java or ObjectiveC this is normally because I haven’t been able to achieve what I want to do in Condova.
Back to Top

Workflow in Practice

So here is how my workflow goes:

Setup

While every project is different, normally it will start with a scoping document, giving me an idea of technologies and server setup. A document sounds formal and for some people this is a very formal document, but it’s as likely to be a set of Trello cards or scribbled notes in my notebook. With scoping done, I will use the closest preselected image I have and setup Vagrant.

Once Vagrant is configured and a new instance is fired up for the first time, I will fire up Sublime and create a new project in an empty folder. Next is a git checkout of my default project setup which creates a folder structure and pulls relevant composer bits. Modify the JSON file and fire up composer to start building the basic project.

While that’s happening I create a new repo in Codebase, to commit this shiny code to and adding the project to Jenkins (even though at this stage there’s no remote server I still put it in Jenkins). Finally edit the Sublime Project file with relevant end points and off we go. If I’m also working with a staging server or using AWS then I would set up and associate with Jenkins.

Finally I edit my Alfred workflow, which when it fires opens a Vagrant Instance and Sublime with the project loaded. A second command halts vagrant and saves and closes all files before running git stash.

Editing and Committing

With everything setup, I fire up Sublime from Alfred, which also kicks in the Vagrant Instance (This is not always perfect which is where Vagrant Manager comes into its own to stop, start and spot when Vagrant instances haven’t halted accidentally.

I then edit away to my heart’s content, when I want to commit I simply tab and commit, when I want to push likewise. After pushing the Vagrant install normally becomes unresponsive for 20 seconds or so (bare in mind this is an older machine). So similarly there is a few second’s delay when using x-debug’s remote features within Sublime.

When happy/ready to push to a production or staging server, is simply a case of committing into the relevant branch (a throw back to the old SVN days, I really do use Git more like SVN then I should).

Deployment/Provisioning

Depending on the setup 1 of 3 things happens.

If using DeployHQ then on committing to branch, a post notification hook is sent to DeployHQ from Codebase and DeployHQ runs, pre deploys script on remote server, SFTP content and then runs post Deploy work scripts, including any Grunt scripts.

If using Jenkins (non AWS) it’s checking Git repo every 60 seconds, if a change to branch is detected that is being monitored it does a git checkout to a temporary holding area, performances relevant tests if they pass runs, script to SCP to remote machine and runs clean up script.

If using AWS Jenkins checks, fires generating a new AWS instance, which triggers Puppet and Anisble to build the server from image with latest packages and connects to remote DB, and pulls the latest of the branch Jenkins was monitoring, runs any workflow tests and any remote tests before being added to the live group. Once live a second process destroys the older server.

Once live, Codeception acceptance tests are run to confirm everything is working before a build complete notice is sent.

Make coffee

While code is deploying it’s the perfect time to make coffee or look at Google Analytics and wait for everything to go horribly wrong. Actually normally deploying from DeployHQ or from Jenkins not on AWS takes seconds, but provisioning on AWS takes a fair bit of time in comparison, the advantage though is it means ease in upgrading, managing and much simpler to test against before truly releasing into the wild.

Pulling content from Live

If I’m working on a project which is already live, it’s handy to have the latest version of content regardless of where I host sites, I nearly always back up to S3 so make use of S3fs (https://code.google.com/p/s3fs/) to mount S3 buckets. When working locally I mount the S3 bucket and then use rsync for the uploads folder and a small bash script to pull the SQL Dump and import with WP-CLI followed by search-replace. To finish the job, I remove my user and recreate the user with local credentials (and therefore local salt).

Weird Development Workflows

I’m not sure if anything in this post is particularly weird or different, where possible I have tried to automate the process. A few steps I have kept deliberately separate such as stepping outside Sublime to use Git.

This workflow is far from perfect, and is very messy in places it also can be prone to breaking. It’s changed over time and has grown organically often from reading other peoples posts like this one. When I get new hardware I intend to sit down and rethink the whole flow, even just writing this article has given me a few ideas and I would also love to hear yours.

So that’s my Dev workflow tools, tell me about yours, suggest areas for improvements and let me know what you think?

My WordPress Development Workflow