Stop talking about scaling Drupal and make it easier to use

It seems that I hear a lot of people within the Drupal community focusing on how to make Drupal scalable. This is all well and good, but Drupal does scale; the Emmys know this, the Grammys know this and so does the Economist. Drupal can handle millions of page views in single day and frankly this isn't an issue that most websites face. This single biggest argument people have against building a website in Drupal is that it too complicated.

So does Drupal need to be complicated? Absolutely not! As Drupal service providers our job is to build something that's not only high quality but also to make the user experience during and (maybe more importantly) after development easy. Drupal 7 is a great step in this direction, so let's get it out the door. Let's focus more attention on administrator workflow and making some of the more commonly used features of Drupal (like blogging) more easy to use and more intuitive. Let's attract more designers and non-developers to this fantastic platform.

The usability of Drupal affects every Drupal user. So let's leave the scalability question to Dries and Acquia and start focusing more on making Drupal sites that are intuitive, easy and a joy to use.

Addendum (7/23/10):

I'd like to start out by apologizing for not recognizing the contributions of more people in my initial post; I don't want to under value the work that has been done on Drupal to-date. Notable companies/people doing scalability work include Chapter 3, Four Kitchens, and countless others.

As for UX, as merlinofchaos mentioned you can get a sense of who's active in this movement in terms of improving the codebase at:

http://groups.drupal.org/drupal7ux

As I already mentioned, I think that Drupal7 makes great strides in improving the usability of Drupal.

With that said, the reason that I posted this article is that I think that during site building by service providers (this is important) strides forward in UX tends to take a back seat to how many visitors the site can support. I hear a lot of buzz about high profile sites switching to Drupal and not as much about how Drupal is making people's lives easier.

I know that people who are excited by scalability are going to continue to work on this and continue to push the envelope getting bigger sites on to smaller hardware. That's great! I'd just like creating fantastic user experiences within Drupal to be seen as just as important and get the recognition and attention it deserves. Great users experiences will attract more people and increase Drupal adoption rates - and that is good for everyone.

Comments

Once you have nodes/users/comments/taxonomy terms into five digits each, which is not hard for any moderately busy community site to get to, you will very quickly run into issues with things like taxonomy listings, complex views, tracker or forum module.

Scaling a site like the Emmys is relatively easy - looks like hardly any logged in traffic so nearly every request uses page caching - even with a vast amount of traffic it's fairly simple to scale Drupal to deal with that, this much is true as of the past couple of years.

However if you have logged in users and a lot of dynamic content, then you very quickly need relatively complex caching strategies and some way to denormalize queries and/or you'll be moving off shared hosting or cheap VPS accounts pretty quick.

Most sites Acquia are hosting (that I know of) are not dealing with large user communities, so "leaving [that kind of scaling] to Acquia" would be really idiotic - what they're doing is scaling the provision of lots of small and medium websites, and higher traffic sites with mainly anonymous traffic, at least at the moment I think that's the focus.

Most sites with large user communities cannot afford the hardware and double-figure development teams of sites like the Economist and Examiner.com, and don't get the free expert help of Drupal.org - so it's very important, to me personally as someone who both works on a large dynamic site as my day job, and has spent years running medium-sized community sites on a shoe string in my spare time, that it can handle those kind of sites better without having to use core patches (or Pressflow, which is largely packaged core hacks plus a few extras), or a massive reliance on custom code, or extremely beefy hardware, or all of these at once.

Working with Drupal 7 for the past few months it's going to be much, much better than this than D6 (not good on $5/month shared hosting, but much better on single VPS accounts), but there's a very long way to go before it's as easy to build a complex site that doesn't fall over once more than a few people use it at the same time, as it is to just build the complex site.

I'm also not clear if you want to focus on making Drupal sites easier to use, or Drupal as a site building platform easier to use - the two are not the same thing at all.

Apparently you missed an effort so grand that it actually got its own name: d7ux.

It seems odd that you would post this. Usability has been a giant focus of people working on core right now, with half a dozen people who've basically been working on it exclusively.

So let's leave the scalability question to Dries and Acquia

...

Perhaps you didn't mean to be, but I find this statement kind of insulting.

And what if I say that the backend is harder to replace than the user interface and also the UI is more often site specific so let's get Acquia to build usability? Drupal Gardens, hm?

So, you've already felt the wrath of other devs, no need for me to apply ;-) Here's what I've been doing about usability. Why not go download ctools and context_admin and see if you can't make your site a little easier to maintain. Making drupal easier starts with you (if you're this fired up to make it more usable). Hopefully you can build on other people's work.

Eclipse

http://drupal.org/project/context_admin

I too am surprised about the "start focusing more on making Drupal sites that are intuitive, easy and a joy to use" part. It's up to designers and builders to create great ux for projects, regardless of the platform used.

If you mean better usability of Drupal itself, then did you really miss out on all the UX work of the last 18 or so months? Come to #drupal-usability on IRC or groups.drupal.org/usability and we'll get you up to speed :)

Just because someone can make a Drupal site scale to a million sites doesn't mean that same person can make a site easier for a novice to use. The two skill sets are completely different. Attempting to get developers to shift from performance to usability doesn't make any sense. Focussing on recruiting more UX specialists into the community would make a lot more sense.

People with the interest and the ability to address scalability should absolutely continue talking about scalability. People with the interest and ability to fix usability issues should do so. Sometimes they are the same people but often they are not.

Thanks for the info, I know very little about your company, and thankfully I'm glad that I'll now never have to bother finding out more. Ill informed, idiotic, comes to mind when reading your article. If you are the company, or employed by the company, then you or it deserves to fail with such an attitude.

Drupal does not scale well, even on medium sized sites, Drupal is not enterprise grade, it's not even medium business grade. And yes I'm well aware of all the big companies doing Drupal. Drupal takes a lot of work to get going for any project of worth.

Performance of a Web site, is also a feature of a website. If your website does not scale, it doesn't matter how pretty the front page looks, it's not going to be used.

While I get your point, you have made it in such a way that its meaning is hidden in a forest of objectionable statements.

At the very least Tag1, Chapter3 and 4Kitchens tend not to like leaving the scaling to Acquia.

Hi Joaquin, This post could have done a better job defining it's target audience. I think you are right most people shouldn't be worrying about the scalability of Drupal, it's been proven time and again that Drupal can be scaled. It's still skilled and hard work, but the best practices are shared effectively.

Of course, most people doesn't include contributors like Earl. If views, panels, ctools didn't scale well, most of the sites that rely on his modules would fall over. If catch and chx hadn't been doing so much detail performance work on drupal 7, while working on the d7 launch of examiner.com, performance hits we've taken to improve the D7UX would make D7 unusable for some. So for those core developers, please keep worrying about scalability and performance!

I am glad to see the CEO of a Drupal shop call for more attention to the details that impact every Drupal user, like the administration work flow, and for a better out of the box experience for common applications like blogging when using Drupal.

There are thousands of Drupal development shops and dozens of them are launching really big interactive Drupal sites. Collectively, we rely on the developers of those large sites to find performance and scalability issues in contributed modules and issues with Drupal core to keep the out of the box scalability of Drupal relatively high.

Products like project mercury and Acquia hosting are certainly helping to make scaling Drupal easier from an infrastructure level. Scaling anonymous traffic with pre-configured caching systems still requires significant effort and skill, particularly when scaling a high availability infrastructure with huge bursts, but it is a well worn path.

At Acquia we scale very large anonymous traffic sites as well as authenticated traffic sites, with lots of community features. That is often a combination our support and hosting operations team working with dozens of different customers and the many highly capable Drupal shops who design and architect sites to be scalable. At least one of those community sites is doing more than 20M pv a month, with a significant amount of authenticated traffic.

We are also developing a lot of very large Drupal commons sites, which in many cases has dozens of contributed modules and is exclusively authenticated traffic. Some of these sites are really going to push the envelop for Drupal community site sizes.

For people who are evaluating Drupal and how to scale the hosting infrastructure, see: http://acquia.com/products-services/acquia-hosting

To learn about the state of scaling and performance in Drupal join the high performance group, where dozens of skilled Drupal contributors share their knowledge. http://groups.drupal.org/high-performance

Don't forget to join the D7UX effort as well!

Cheers,
Kieran

I'd like to start out by apologizing for not recognizing the contributions of more people in my initial post; I don't want to under value the work that has been done on Drupal to-date. Notable companies/people doing scalability work include Chapter 3, Four Kitchens, and countless others.

As for UX, as merlinofchaos mentioned you can get a sense of who's active in this movement in terms of improving the codebase at:

http://groups.drupal.org/drupal7ux

As I already mentioned, I think that Drupal7 makes great strides in improving the usability of Drupal.

With that said, the reason that I posted this article is that I think that during site building by service providers (this is important) strides forward in UX tends to take a back seat to how many visitors the site can support. I hear a lot of buzz about high profile sites switching to Drupal and not as much about how Drupal is making people's lives easier.

I know that people who are excited by scalability are going to continue to work on this and continue to push the envelope getting bigger sites on to smaller hardware. That's great! I'd just like creating fantastic user experiences within Drupal to be seen as just as important and get the recognition and attention it deserves. Great users experiences will attract more people and increase Drupal adoption rates - and that is good for everyone.

2bits.com specializes in Drupal performance and scalability.

We presented on this back in April at DrupalCon San Francisco: see 2.4 million page views per day, 60 million per month, one server!. That site has actually hit higher records now, with 3.4 million page views per day and 92 million per month.

On our site you can find plenty of articles, case studies, and presentations on Drupal performance tuning and optimization for large web sites.

Add new comment

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • You can enable syntax highlighting of source code with the following tags: <code>, <blockcode>, <cpp>, <java>, <php>. The supported tag styles are: <foo>, [foo].
  • Web page addresses and email addresses turn into links automatically.
  • Lines and paragraphs break automatically.

About the Author

Joaquin Lippincott, CEO

Joaquin is a 20+ year technology veteran helping to lead businesses in the move to the Cloud. He frequently speaks on panels about the future of tech ranging from IoT and Machine Learning to the latest innovation in the entertainment industry.  He has helped to modernize software for industry leaders like Sony, Daimler, Intel, the Golden Globes, Siemens Wind Power, ABC, NBC, DC Comics, Warner Brothers & the Linux Foundation.

As the CEO and Founder of Metal Toad, an AWS Advanced Consulting Partner, his primary job is to "get the right people in the room".  This one responsibility is cross-functional and includes both external business development functions as well as internal delegation and leadership development.

A UCLA alumni, he also serves in the community as a Board Member for the Los Angeles Area Chamber of Commerce, the Beverly Hills Chamber of Commerce, and Stand for Children Oregon - a public education political advocacy group. As an outspoken advocate for entry-level job creation in tech he helped found the non-profit, P4TH, an organization dedicated to increasing the number of entry-level jobs in the tech industry, and is in the process of organizing an Advisory Board for the Bixel Exchange, a Los Angeles non-profit that provides almost 200 tech internships every year.

 

Ready for transformation?