Web Platform Docs API used in Brackets

A basic goal of WebPlatform.org is to be the site where you can come for answers to your trickiest (and simplest) development and design questions about the Open Web Platform. Along with being a site, we also want to be a service — or really to be of service, as the saying goes. Besides being a central location where you can get information, we want the information to get to you. So we’ve provided APIs for retrieving the content in-context.

You’ve seen it before in editors. You select “Help” and documentation pertaining to the code you’re typing shows up in your editing window. It’s referred to on Wikipedia as “Context-sensitive help“. And it’s great, because, right as you’re coding, you can check a method signature, or get an explanation or a sample. The key is the content: Is the feature proprietary? Widely adopted? Is the help current? If Web Platform Docs are being pulled in, you know where you stand: it’s documenting the open web. And you’re getting the latest content we have.

Being on the MediaWiki platform means that we can provide an interface for structured information reuse. You can try it now. In your browser address bar, just type:

http://docs.webplatform.org/w/api.php?action=parse&page=css/properties/box-shadow

OK, it’s not pretty, but you get the idea — or actually, the xml result. Adding a format parameter &format=json gives you the results back as a JSON object. You can read more about the default calls in the MediaWiki developer documentation.

So, the API was lying fallow, when Adobe’s Alan Greenblatt (@agreenblatt) decided he wanted to contribute something useful to the Brackets project. If you aren’t already using Brackets, you should check it out: it’s an “open-source code editor built with the web forthe web,” which is a great match for what we’re doing at WPD. WebPlatform.org individual contributor, David Kirstein (aka: frozenice) worked with Alan to explore and adjust the API. The result is in the latest release of Brackets. With a simple Command/Control K, the reference content for a CSS property appears in the context of your CSS code!

As it stands, it’s a pretty neat little API, but there are considerations when using services. Mindful plugin models should be employed when implementing a documentation-as-a-service. For example, we’re still working on our infrastructure, and so there was a concern about having a lot of individuals hitting the site. Also, the Brackets team wanted to take some basic security measures. So, for this early implementation, he decided to preprocess and package the results.

The Brackets implementation includes a “Read more” button, which sends the user to the full page in the browser. And that’s where the details lie: Right now, we’re working on getting key content up to beta quality. So that button should say: “Read more, and if you think you know better, please don’t hesitate to edit the page, and make it better.” Because after all, WebPlatform Docs is “Your web, documented.” For more information on registering for the site, talking with the community, and contributing, select Editing from the top menu on any page.

That’s the scenario: You’re in the throes of a project; you need a reminder or an explanation or some sample code; you get WebPlatform.org content within your working context; and if you can, you leave that page a little bit better for the next developer.

Building Web Platform’s Infrastructure

For the initial launch of Web Platform, we decided to go for an alpha release with a small and concrete set of platform goals. We used open source software, and we kept the initial set of applications small, to focus on preparing them to handle the initial launch load.

Applications targeted

The initial set of applications targeted were those that launched with the alpha: MediaWiki (docs), WordPress (blog),  Piwik (stats), Question2Answer (talk/forums), qwebirc (talk/chat), and LumberJack (chat logging). The first four are PHP applications, qwebirc is Python, and LumberJack is Python, PHP, and JavaScript.

Scaling targets

Our upper-bound targets for launch day were:

  • 100,000 visitors
  • 200 anonymous requests per second
  • 10 logged-in requests per second.

Our use case assumes 95+% of requests will be reads.

We wanted to be well-prepared for whatever the Internet would throw at us.

Launch statistics

Our actual statistics for the launch day were:

  • 86,000 visitors
  • 720,000 page views
  • 300 requests per second during the US peak
  • 350 requests per second during the Europe peak.

Our application servers combined CPU load was steady between 10-20%. Memory usage was steady between 20-30%. Database wait-io (the only statistic that really showed a blip) was between 5-10%. The storage servers showed no statistics worth mentioning.

Here’s how we managed this…

Continue reading