Why Drupal 8 won't ship with REST content negotiation

Some friends on Twitter were alarmed by this Drupal change record:
Accept header based routing got replaced by a query parameter
curl /node/1 -H "Accept: application/hal_json"
curl /node/1?_format=hal_json

The issues leading to this change are too lengthy to capture on Twitter, so I'll give my perspective here.

What was the problem with accept-routing?

  1. Most critically, not all CDNs and caches support content negotiation (see #2364011)
  2. A secondary issue that I think deserves more attention is that even when caches support content-negotiation, the cache hit rate suffers.

The poor hit rate arises because web browsers do not simply declare Accept: text/html, they request a bizarre mishmash of media types depending on the specific browser, version, OS, and even what extensions are installed. A quick test on this site discovered roughly 100 unique Accept headers in 24 hours.

But isn't content-negotiation required to have "pure" REST?

  • No. Roy Fielding has said "Neither choice has anything to do with REST" (referring to a debate between extensions or query parameters).
  • While I don't deny the aesthetic appeal of the Accept header, there are good reasons why Drupal often has to de-prioritize theoretical purity. As mikl wrote on #2364011, "Drupal should be engineered for the web we have, rather than the one we wish we had."

Is ?_format a Drupalism?

The special _format Routing Parameter comes from Symfony, so I think this would be more accurately called a Symfony-ism, if it's an -ism at all. However, given that REST is an architectural style, and not a specific protocol, any API we design will naturally contain some details unique to Drupal.

Can't Varnish normalize the wide variety of Accept headers?

Yes, but if custom normalization routines in Varnish were the only way to have a high-performing cache, then that would be a bad Drupal-ism. It's better to have broad compatibility with current CDNs and caches.

Could a contrib module re-enable content negotiation?

Probably. The Symfony route filters are plug-ins that can be swapped out.

What alternatives were considered?

Some other options were moving REST resources to an /api path, or using extensions like /node/1.json. In the end, the query parameter won out because it had the least impact on the existing routing system.

The Future

I think an ideal solution to both problems would be widespread support for the Alternates: header (RFC 2295). In this way a server can specify what alternate media formats, languages, or encodings are available, and content negotiation can be efficiently performed at the network's edge. Sadly, this experimental RFC appears largely abandoned since its introduction in 1998, so perhaps the best we can hope for is by the time Drupal 9 is ready, CDNs and browsers will have cleaned up their act.

Filed under:

"Could a contrib module re-enable content negotiation?"

Yes we have tests in core to ensure that they can be added back in a later point release if things get less crazy

Thank you for this great blog post! It describes it really well why we fixed it that way.
I'll use it for future reference :)

Regarding alternates, interesting! But it is tricky, because at least with the current routing system different formats don't know each other.

Add new comment

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • You can enable syntax highlighting of source code with the following tags: <code>, <blockcode>, <cpp>, <java>, <php>. The supported tag styles are: <foo>, [foo].
  • Web page addresses and email addresses turn into links automatically.
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.

About the Author

Dylan Tack, Director of Engineering

Dylan is a software engineer with more than a decade of experience working with a wide variety of clients including the Linux Foundation, PBS, Habitat for Humanity, TV.com and the Emmys. His background includes training as an electrical engineer, but he became passionate about open source through his work with a university genetics lab.