NetDNA API Docs

Over the past weeks I’ve helped out NetDNA by converting their API docs from hard-coded HTML to auto-generated Markdown with Github Pages.

They asked me to document it, so here’s how it went from start to finish. If you want to skip all this, then go check out their docs for yourself here:

http://docs.netdna.com

Cleanup

The existing docs were OK, but needed cleanup for easier management and a new design. Here’s a screenshot of what they looked like and here’s the HTML.

Key issues:

• Navigation was not friendly (no affix/sidebar that scrolls with page)
• No search for docs, you had to manually do CTRL+F and that took a while
• Everything was hard-coded in HTML, and making changes took a lot of time copy/paste
• Code examples were missing
• Lacked instructions for getting started
• No support info listed for integration/dev help
• API wrappers for languages weren’t anywhere to be found

Migration from HTML to Markdown

We had to convert HTML to Markdown. Not an easy process with custom styling and getting GFM tables parsed correctly. Doing this by hand would take a while (e.g. writing <h1>Blah</h1> to # Blah and so on…) so a script to scrape/parse did the trick.

It took this 100 LOC script to automate it, but it worked flawlessly.

Headaches may have occured while writing this regexp:

md = md.replace(/<table>(?:(?:(?!<\/table>)[\s\S])*?)<\/table>/g, function(html) {
  return parseTable(html)
})

Not too fancy… but it worked!

NPM modules used:

• cheerio - parse HTML with jQuery helper functions and cleanup of HTML
• fs - read existing HTML and write Markdown output
• path - namely path.join to join dirs
• to-markdown - convert HTML to markdown (still needs some touchups after this)
• htmltidy - beautify the HTML for any tab/space issues
• htmlparser2 - parse the HTML for GFM tables

Setup

They wanted to integrate Twitter Bootstrap, and so I had a module called readme-docs which was used for a bit (I recommend it if you want something simple for your Readme.md files).

However, more control over HTML/CSS without having to hack around with a CLI was essential – so we couldn’t use that module.

I decided to use wintersmith to write and finish the new stack. Wintersmith is a blog engine I used in the past for setting up StartupSupper, so setup was straightforward.

This turned out to be the best decision, since this module lets you easily deploy pages to Github (e.g. a blog) while letting you use Node for builds.

Being able to use Node with LESS and Jade for Github Pages was the icing on the cake.

Adding instructions to let future maintainers have sanity was the last step.

Github Pages

Setting up generated docs at http://docs.netdna.com was relatively simple and straightforward.

We added a CNAME in “master” branch, set DNS to point to Github, and had our wintersmith project in the “source” branch.

Grunt was used for the build process, and I pulled the Gruntfile out of StartupSupper (good coders code, great reuse).

Had some issues when I first tried running grunt to deploy to http://netdna.github.io (kept getting storage errors) → but then I soon realized Github was down… and then back up within minutes!

Making updates to their API is now easy.

λ ~/Public/netdna.github.com/ source grunt
Running "setup" task
Creating ./_deploy folder
Initiating git repo
Adding github to remotes as origin
Setting master branch remote
Setting master branch merge

Running "deploy" task
Resetting repo
Building site
Adding all files from ./_deploy
Getting last commit from source branch
Make commit of changes with last commit msg from source branch
Push changes to github

Done, without errors.

Final result…