At Metal Toad we have been expanding our testing to include more behavior driven testing.
At Metal Toad we have been expanding our testing to include more behavior driven testing. The end goal is to bridge the gap between the languages that we developers speak (which tend to be more logic than spoken) and the languages that our clients speak (commonly English or some other language that normal people use for communication). One interesting challenge we found was taking all the "awesome" that behat has to offer and applying it to a site that has several domains being served from a single Drupal install running the domain access module.
One of the tricks of working on a Drupal site that shares most of its guts and glory with multiple domains is making sure that the changes you are making to one domain are not having any adverse effects on the others. As much as we all love clicking through multiple sites over and over again in search of things that should not be there, this seemed like a great task for automation and our ol' pal behat.
Sure we could simply create completely separate behat tests for each domain, but that would be repetitive and require some extra effort for all involved to run and review the results.
In the case shown in this example the domains share a lot of the same modules and theme elements. 
To make the most of our behat context we wanted to make sure all of our tests could reference the same custom testing functions. Since we also want to keep all of our tests in one central location, the simple solution was to create sub-directories within our tests/ directory for each domain.
Within each domain's directory there is a config/behat.yml that has the settings for that particular domain and points it back to our shared context file. The features/ directory contains all of the test features and scenarios that are written for that domain.
But when it came time to run our tests we wanted to avoid having to go into each directory and manually running behat. In the spirit of "old tricks are the best tricks" we put together a simple shell script that would do it for us (see code below). The steps are relatively simple and of course you could easily substitute with any programming language of your choice.
Then we decided to take it a step further. Having all of the tests run from one command is great, but it still does nothing in the eyes of the client. In this type of testing, if you're not making things better for the client, then you're sort of missing the mark.
Since we are already looping through the files while generating the behat reports we figured, why not also pull it all together into a single html page that displays the results of each domain all just a few scrolls away from one another.
The end result will give you a style-able presentation page to see the results of all tests run against all domains.
Then just to add a little extra "gravy", let's archive off the entire directory of results into a zip file that can easily be sent off to any and all stakeholders in the project.
Let's take a look at the quick little shell script that is looping through each folder looking for behat to run, archiving off the test results, and pulling it all together into a presentable format:
# File and path names, change as desired
resultDir="results"
resultFile="results.html"
# Store some date information for later use in the presentation file
fancyDate=$(date +"%a, %b. %d %Y %I:%M %p")
# Create a new directory for the results based on the current date and time
newDir=$(date +"%y-%m-%d-%H%M%S")
dirPath=$resultDir/$newDir
# Echo the start time out to the terminal (for those who may be watching)
start=$(date +"%m-%d-%y %T")
echo "Start time : $start"
# Make the destination directories
mkdir $dirPath
mkdir $dirPath/xml
# If available copy over style and image assets to make the presentation file more visually pleasing
if [ -d assets ]; then
mkdir $dirPath/assets
cp assets/* $dirPath/assets/
fi
# Create the presentation file and write the head and top contents
indexFile=$dirPath/index.html
touch $indexFile
echo "<!DOCTYPE HTML><html><head><title>Behat Results</title><link rel=\"stylesheet\" type=\"text/css\" href=\"assets/styles.css\" /></head><body><div id=\"head\"><h1>Behat : Behavior Driven Test Results</h1><div>Metal Toad Media</div><div>$fancyDate</div></div><div id=\"content\">" >> $indexFile
# Loop through the directories looking for behat to run
i=1
for item in */
do
cd $item
# If a behat config file is found, execute the behat tests
if [ -f config/behat.yml ]; then
echo "########## RUNNING BEHAT FOR $item ##########"
# Make a new directory for the domain
mkdir ../$dirPath/$item
# Create an iframe within our presentation page to show this domains results.html
echo "<section><h2>$item</h2><iframe src=\"$item$resultFile\"></iframe></section>" >> ../$indexFile
# Run the behat tests for this domain
# Here we are outputting the
# - pretty format to the terminal
# - html format to our results.html files
# - junit-like xml into the xml directory (used for other test results analysis)
behat -f pretty,html,junit --out ,../$dirPath/$item$resultFile,../$dirPath/xml --expand --no-paths
fi
cd ../
done
# Close out the presentation page html
echo "</div></body></html>" >> $indexFile
# Zip up the test result contents
zip -rq $dirPath.zip $dirPath/*
# output the finish date time to the terminal
finish=$(date +"%m-%d-%y %T")
echo "Finish time : $finish"
For more general information about behat and how it can be used for behavior driven testing, be sure to checkout behat.org
A/B and Multivariate Testing is a staple of the product development lifecycle. Whatever your goal, testing should be a frequently used tool in your...
I've been doing a lot more Behat testing recently.
In the testing world it is very common that Quality Engineers talk about defects, bugs and errors, during the software development life cycle.