How I got Twitter, UNIX Timestamps, and Drupal 7 to all play nice

Recently I had an opportunity to work with Twitter module for Drupal 7 on one of the projects. Most of the functionality I needed for the project was already available within the module and with few tweaks here and there I can get the task done.

Here at Metal Toad Media we take a lot of pride in our work. We try to do everything we can in order to ensure that by the time our code gets into hands of our client, it is a solid product. There are several stages in our development cycle to aid the process of QA. Each developer has a local server running on their machine and this is where most of the magic happens. Once a feature has been implemented and thoroughly tested on the local machine, we proceed and deploy the code onto a development server. Once our code makes it onto development server, project managers (PM's) can check out our work and do some testing of their own. After PM's are done with the QA and give us the green light, we deploy this code onto a staging server where the client can assess our work and give us feedback if they want any further changes to their website.

By this point, our code has been evaluated by development team, project manager(s), client(s) and whoever they share the link to the staging website with. That is a pretty decent sample group of testers and at this point we can pass the code to the client feeling good about our work.

This is the process we use on all of our projects including this one. Social networking integration was implemented and tested, passed the project manager QA on development server, client QA on the staging server. We passed the code to the client and everyone lived happily ever after. Well, that's until the client deployed the code on their server and all of the sudden, Twitter OAuth wasn't working anymore.

Symptom

When user would try to authenticate using Twitter OAuth they would get an error due to oauth_token not being populated by the module. During failed authentication handshake user is forwarded to:

https://api.twitter.com/oauth/authenticate?oauth_token=

Troubleshooting

My first hunch was that either OAuth key pair was off or the Twitter app permissions were not set up properly. However, upon further investigation I found out that this was not the case. It was weird as no matter what I used as OAuth keys, I would get the same thing. I checked out Drupal and server logs, but nothing was reported in either of them. I cannot replicate this bug on development or staging servers and production server is not reporting anything wrong in the logs. Now I'm in a big no-no situation. I have to start messing with the code in the production environment. We talked to the client, explained what needs to be done and that there might be potential outages across the website during the troubleshooting process. I was going line by line through the Twitter module, print statement here, print there until I located point of failure.

During the authentication handshake, one of the parameters passed to Twitter API is the UNIX timestamp. This is a security measure to reduce the possibility of someone re-using the authentication token by having it expire after a certain period. Since the server time was not properly synchronized (it was behind several hours), each OAuth token was generated expired, thus the authentication failure. This seemed to be one of the expected scenarios by Twitter module developers as this call was wrapped in a try block. Except the catch block was left empty, probably an oversight.

// From twitter.lib.php - Lines 310-320 v7.x-3.1
public function get_access_token() {
  $url = $this->create_url('oauth/access_token', '');
  try {
    $response = $this->auth_request($url);
  }
  catch (TwitterException $e)  {
  }
  parse_str($response, $token);
  $this->token = new OAuthConsumer($token['oauth_token'], $token['oauth_token_secret']);
  return $token;
}

As a result, I have no feedback of what is failing or why it is failing and am left poking around in the production environment.

Lessons Learned

  • If you are ever in a situation where you are setting up a new server environment, make sure that server time being synchronized is on your checklist of TO-DO things. Otherwise some obvious and not so obvious, time sensitive operations will cause you headaches.
  • Always make sure that your exception catch block is implemented, and exception is properly handled. It is OK not to print this error back to the user, especially when the report might have security implications. However, at the very least, print something to the logs so you leave a trace for the developer to troubleshoot the issue.

The URL to the live website was not advertised to the public yet, so the consequences of any downtime caused by poking around in the production environment were fairly mild. But this didn't have to be the case. This could have been a website with high traffic volume, where any extended downtime translates in significant revenue loss. Debugging in production will not be an option most of the time, so do yourself and everyone else a favor: sync your clocks and handle your exceptions!

Filed under:

Comments

I am curious, you mention to pass the work to Project managers to do initial testing once it's on the development server and then to the client for further testing.

What is you process for test procedures. Do you outline to them what is to be tested and how to test it? Do they get a checklist to be completed.

I would appreciate some of your insight into how to manage the testing of a bit of work. I have experience in the past of trying to accomodate testing especially client testing and it all being a bit hit and miss sometimes.

I have been trying to come up with a way to set an agreed test requirement prior to sign off, where the client buys into testing certain things by taking certain actions and while it won't test everything other aspects can be assumed in their being able to complete certain tasks. e.g. OAuth works because the client has been able to sign in, which is great as long as the client knows they have to try to log in via twitter to test that part.

Gareth,

Our testing procedure is roughly as follows:
a) A developer will complete a task/ticket per the specifications. They are also expected to perform their own self-QA to look for major bugs and unexpected bugs introduced across the site related to the task.
b) In the ticketing system, the developer will pass the ticket to a project manager, who reviews and provides QA. Our project managers generally have the best sense of client requirements for a task, because they're the ones working to write the specs to begin with and discussing with the client and have the best understanding of their expectations.
c) Once we have reviewed internally, we generally batch a list of completed items for a client to review at once (or in the case of a new site launch, we hand over the entire client site following our rounds of internal QA). They will review the site to make sure it meets their expectations. Generally at the time of site hand-off, we'll perform training and walk through content creation and any other major items of note that will be required to perform QA and content entry. Usually client QA becomes a mix of in-scope bugs that we need to address (resulting in the addition of new tickets) and new feature requests, which are addressed either on a future development wishlist or change order basis.
d) Independent of client QA and approval, our developers perform code review on commits to QA the code itself and make sure there aren't any errors.
e) Following all QA rounds and approvals, we'll confirm approval to deploy updates to the live site from our development/staging sites.

In some cases where clients wish to minimize hours spent against their budget, we'll make arrangements to minimize our own internal QA and put the majority of review on the client, but usually that only happens in cases where are clients are fairly technical and understand that we may hand over buggy features from time to time.

Hopefully that helps clarify?

Add new comment

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • You can enable syntax highlighting of source code with the following tags: <code>, <blockcode>, <cpp>, <java>, <php>. The supported tag styles are: <foo>, [foo].
  • Web page addresses and email addresses turn into links automatically.
  • Lines and paragraphs break automatically.

Ready for transformation?