Pantheon Launches Community Resource for Scaling WordPress

When people ask the question, “Can WordPress scale?” they are often pointed to some of the largest websites running on WordPress, such as Time Magazine, TechCrunch, NBC Sports, Playstation, the New York Observer, and others. But how do you get there and what does it take to deliver WordPress at scale to millions of visitors?

Yesterday Pantheon launched a new resource to answer this question with a knowledge base of best practices from experienced developers. WordPress at Scale is a community-driven project that aims to educate site owners and developers about scalable website infrastructure and optimizations that many large scale sites employ.

The site’s content is managed via a GitHub repository where anyone can contribute. Topics currently range from elastic architecture to object caching to recommended development workflows for scaling.

Pantheon Aims to Make Scalability and Performance Commonly Understood Best Practices

“We work with a lot of web agencies and dev shops, and we’ve also spent a lot of time working within the community at WordCamps over the past 18 months,” Pantheon co-founder Josh Koenig said. “It definitely seems like there’s a need for go-to resources when it comes to questions about scale. We wanted to set a serious goal for ourselves to make an impact in the WordPress ecosystem.”

While you can find scattered tutorials around the web about how to scale WordPress, this collaborative community effort is one of the first to aggregate resources into a collection. Knowledge of scalability doesn’t come easily and isn’t always shared with the community.

“Very often learnings about scalability don’t make it out of projects or companies because they can be hard to generalize (or because people think there’s proprietary value in them),” Koenig said. “Our opinion — and this is based on our experience in the Drupal project with our pre-Pantheon work on PressFlow and ​Mercury — is that there’s ​vastly more value to be had for everyone in making scalability and performance commonly understood best practices.”

In order to accomplish this goal, Pantheon set up the WordPress at Scale microsite as a community project.

“We wanted this to be a real contribution,” Koenig said. “Since we’re a platform provider, doing it ‘in house’ would mean it’s ultimately just a marketing piece.

“I’m clearly being up front about the fact that we hope this site has marketing ​value​ to us, but that value should (rightly so) be proportional to how actually useful it is to the community,” he said. “We’d never get very far if this was just a Pantheon thing.”

Weston Ruter, CTO at XWP and contributor to the WordPress at Scale site, agrees on the value of having the site set up as a community project.

“A community-driven resource like this is important because there is a lot to know, and there is a lot of experience to draw on from the community,” Ruter said. “No one agency or consultant has all of the possible tips and tricks for scaling WP, so having a collaborative resource to draw that information together is very helpful to keep it from being isolated in our respective silos.”

Although scaling varies widely based on the type of content being served and user activity on the site, many aspects of optimization are fairly straightforward and can be easily applied as needed.

“I suppose scaling WordPress is more a formula than an art,” Ruter said when asked which it resembles more. “If there is art to it, it is having an eye for designing the architecture to take into account the specifics of the site being scaled. Different sites have different techniques required for scaling. A brochure site with seldom-changing content will be cached very differently than a social network site.”

WordPress at Scale is just a starting point for what Koenig hopes will become a comprehensive resource authored by many contributors. It’s currently missing several topics that he wants to cover in more detail.

“There are a couple good issues in the queue already with specific technical topics I’d love to cover: fragment caching, and ESI,” he said. “We will definitely be getting into those and other topics with more specificity, especially as we can leverage more contributions from the community.

“Once we get the content a little more built-out, I want to create a generic presentation that anyone can use to cover this topic,” Koenig said. “Whether that’s giving a talk at a meetup, or pitching WordPress as a solution to a client, having some good quality materials will really help get the message out there.”