Node focus on Blogger posts

In Node focus explorer without excel I showed a d3.js app that could navigate a google site using tags and nodes. This was actually pretty generalized, so it was a minor modification to add blogger post navigation capability also. 

Again we'll use Google Apps Script and adapt Analyzing site content with GAS  to take data from blogger posts rather than Google Site content (actually .. as well as .. but more of that later).

Getting the blogger API key

Unlike Google Sites, you do need to get an API key to access the blogger API. You can apply for one here. A day or two later, someone sends you a mail with details of your key.

The API data

This is a straightforward REST api, but it does have a limit of 20 posts, so you have to call it repeatedly until you have them all as follows
    var cb = new mcpher.cBrowser();
    var p, items=[];
    while (!p || p.nextPageToken) {
      p = JSON.parse(cb.get(initBlogger(eArgs,source) + ( p ? '&pageToken='+p.nextPageToken : '')));
      if (p && p.items) items = items.concat (p.items);
      if (!p) break;

Storing your API key securely

One of the challenges about publishing code where there is private information is how to hide it. I use Database abstraction and Google Apps Script and store stuff like this on my private MongoDb store, but you may find it easier to use something l like your property store. Here's how I do it.
function getApiKey(api) {
  // my private store
  var myKey = myStuff.getMyStuff(api, myStuff.myStuffDb());
  if (myKey.status.code == "good") {
    return myKey.result;
  else {
    throw ("unable to find your key for " + api);
    return null;

Creating the dataset

Retrieving the data takes the same structure as for Google Sites, except its a little simpler. There is no need for recursion since the blog posts are flat. In fact - you can see from the blogger and sites version below there is a very minor difference to get the data from different properties, and to recurse or not.

The only other wrinkle is that the blog content comes as html, so it needs to be rendered as plain text to avoid the html markup being considered as linkable data. I found this neat solution for that on stackOverflow by Cory Goldfeder who continually comes up with great stuff like this.

Scheduling data updates

Now I have a Google site version and a blogger version. Scheduling GAS showed how to add a nightly update to write the data to Google Drive. I'll just update that to write the blogger version to Google Cloud store (for more on that see Cloud Storage and Apps Script) and here is the result for the blogger. You can navigate in the same way as in the Google site version, except this time you get to both. You can write it to Drive if you prefer by changing the parameter from gcs to drive, but then you'll need to find somewhere to host it. Using cloud storage I can host it in place from where I write it.

tagoutput :'gcs',  // change this to drive if you want to write to drive instead
The navigator (at takes an argument that is the name of the input data (the default is the data for this site). So you can use it as is, once you have created a data set as described here and in the Google site version

Finally, I've built in a proxy to allow the data to come from some other place if you don't have a site to host your data on. For details see Apps Script as a proxy. If you don't want to use a proxy, you'll need to change a couple of lines to get the data directly instead of via a proxy.

It's embedded below, or you can try it out here.

Combining data sets

This is where it gets real interesting - now you can combine multiple sites or blogs into a single navigation app. I have scheduled a nightly trigger that creates a dataset combing the sites and blogger data, giving us this (it's a bit much but once you start focusing in it can be useful). It will take a while to settle down as there's a huge amount of content between the two sources. You can also run it here

The code

Here is the Google Apps Script code to generate the data

Here is the web app code

For more on this , see GAS and sites and d3.js and this excelramblings blogpost. For help and more information join our forum,follow the blog or follow me on twitter .

For more stuff see my book - Going Gas.  All formats are available now from O'Reilly,Amazon and all good bookshops. You can also read a preview on O'Reilly.