Ephemeral Exchange


Sometime you need to exchange transient data between platforms, but they don't talk to each other very well. For example you could be launching a webapp with some data generated from Excel, but you want to get feedback data back into Excel again. You could use a database, or push/pull solutions, but it can be complex to set up, and the authentication is often difficult to get right.

What is Ephemeral Exchange


It's a JSON API to exchange ephemeral data that doesn't need each platform to authenticate, but relies on using keys that are known by each collaboration party, and issued by the API. Apps knowing the item id, and appropriately authorized access keys can use them to retrieve data written by an entirely different platform (even using a web browser)

Demo 


and here's another showing how to use streetview and generate slides with selected views.


Hosted or open source

The code is open source, but I'm also providing a hosted solution on Google Cloud Platform. All you need to do is register, get a boss key, and you are ready to go. The hosted solution supports CORS so you can use it even from client side JavaScript without worrying about Cross Origin request issues.

Code

Most of the demos and libraries mentioned in these pages are available in their own github repo, but for convenience there is a consolidated repo where I'll push all the associated code. If you'd like to contribute, please make a PR there.

Tutorial

To register, to read about the service in more detail, and to work through an interactive API explorer tutorial, you should visit the site, which is currently in developer preview.

More about Ephemeral Exchange


Security

The hosted solution is of course public, but the data is transient - It only allows data to persist for a short period of time. There is a lot of security built into the access solution, some of which is described below. The key thing is that permissions are on a single item level, so you can allow an app to access one item in your store, but not another.

Access Keys

Keys are used to identify yourself and to enable you to access the exchange. They expire after some period of your choosing, and are associated with a particular registered account, and different kinds of keys can do different things. 

Boss key

It looks like this, and you can use it to generate writer and reader keys for your account. You might want to generate multiple keys so you could give them different expiry times and use different keys to allow access to specific items or types of items.
bab-2f8-15c6get53b36

Here's an example of generating 2 reader keys that will expire after 2 days, plus the response

/bab-2f8-15c6get53b36/reader?count=2&days=2


{ "type": "reader", "plan": "a", "lockValue": "", "ok": true, "validtill": "Thu, 15 Dec 2016 13:38:52 GMT", "keys": [ "rak-4ru-2k19ubf138pb", "rak-4pv-2x19lbf1338b" ], "accountId": "f32" }

Reader key

Looks like this, and you can generate them to share with apps and others who you are allowing to read  data items.
rak-4ru-2k19ubf138pb

This example is using a write key to write some data, and is also assigning some keys that can read that data (Note that in these example I'm using GET to write small amounts of data. This makes it possible to use the Browser or other platforms that don't allow POST, but normally data is written with POST)

/writer/wak-h4l-b1235bf5f3d6?data=goodreaders&readers=rak-4ru-2k19ubf138pb,rak-3fm-2esgkbf13d76

{ "writer": "wak-h4l-b1235bf5f3d6", "ok": true, "id": "3qik236b1gl9hdsaf", "plan": "a", "accountId": "f32", "readers": [ "rak-4ru-2k19ubf138pb", "rak-3fm-2esgkbf13d76" ], "lifetime": 3600, "code": 201 }

and here is one of these keys issuing a read request for that item

/reader/rak-4ru-2k19ubf138pb/3qik236b1gl9hdsaf

{ "reader": "rak-4ru-2k19ubf138pb", "ok": true, "id": "3qik236b1gl9hdsaf", "accountId": "f32", "value": "goodreaders", "code": 200, "modified": 1481636336955 }

Writer key

Looks like this, and you can generate them to share with apps and others who you are allowing to update data items.

wak-h4l-b1235bf5f3d6

This example an item is being written, allowing another write key to update it.

/writer/wak-h4l-b1235bf5f3d6?data=sharedupdate&writers=wak-z4r-b1231bf5f3g6,wak-d4s-b123jne5f3h6,wak-34a-b123pbe5f3k6,wak-34p-b123cpf5f3v6

{ "writer": "wak-h4l-b1235bf5f3d6", "ok": true, "id": "3qzj2i16s3gbzvlzf", "plan": "a", "accountId": "f32", "writers": [ "wak-z4r-b1231bf5f3g6", "wak-d4s-b123jne5f3h6", "wak-34a-b123pbe5f3k6", "wak-34p-b123cpf5f3v6" ], "lifetime": 3600, "code": 201 }

and here is the item being updated by that other write key

/writer/wak-z4r-b1231bf5f3g6/3qzj2i16s3gbzvlzf?data=updatebyanother


{ "writer": "wak-z4r-b1231bf5f3g6", "ok": true, "id": "3qzj2i16s3gbzvlzf", "plan": "a", "accountId": "f32", "lifetime": 3600, "modified": 1481636337242, "code": 201 }


Item keys

When an item is written to the store it generates a unique id, which is associated with your account. You can also specify a lifetime for that item, after which it expires. Every item has a default lifetime of 30 minutes. Before an item can be accessed, you need the item id, along with the access key that created it plus the lock if there is one, and also the key has to be one that is associated with your account. The only key that can read an item by default is the one that created it. However, you can add a list of reader keys that can access an item, or writer keys that can update that item. A public item key looks like this.
dx1fa-ex4-bdasea1fpak1

Alias

Sometimes it's useful to use a consistent name to access some data, even though the underlying data may change. To do this you can assign an alias. An alias is actually something that belongs to a specific key, for example a reader key, and different physical items can be 'slotted into that name'. Imagine that you had a task that regularly needed to find some data written by some other process - let's call it stock-price. 

The task that had the responsibility for creating that data item regularly would do something like this
/writer/wxk-aa1-18jn81akfb2h?updaters=uxk-f1v-b1ana8q5cj6a

and would get this response
writer:"wxk-aa1-18jn81akfb2h"
ok:true
id:"dx1fa-qp5-bpatra1fka61"
plan:"x"
accountId:"1fa"
updaters:["uxk-f1v-b1ana8q5cj6a"]
lifetime:3600
code:201

Assuming that the receiving task already knew the updater key as part of a standard configuration, the problem here is that it won't know the item key. In some cases, where the receiving task is initiated by the creating task, passing the transient item key would be ok, but in other cases, where there is no direct communication, an alias can be used. 

First the creating task assigns a standard alias that is known by the receiving task (in our case stock-price) to the newly created data item. For this he needs the writer key to be able to write to the store, the updater key who is going to use the alias and the item id key to assign to the alias.
/wxk-aa1-18jn81akfb2h/uxk-f1v-b1ana8q5cj6a/alias/stock-price/dx1fa-qp5-bpatra1fka61

Next the receiver task can get the item using its alias, without ever having to know the specific id.
/reader/uxk-f1v-b1ana8q5cj6a/stock-price

Encryption

All the data is encrypted as it is written to the store, but you should not writer personally identifiable info.

Notes for Apps Script developers, and comparison with Apps Script CacheService

If you are Apps Script developer, you probably use the CacheService. You can use Effex instead of that if you want, but here are the main differences. 

  Apps Script Effex
 Scope To a user, document, or script. Cannot be used across multiple scripts Can be used across any platforms, so has no scope boundaries.
 Performance Fast - about 30ms, integrated into Apps Script platformDepending on your location, about 200ms. Runs on Google infrastructure, but is not Apps Script.
 Rate limiting The CacheService seems to be free of rate limits Since you would use UrlFetchApp to access it from Apps Script, you will be using UrlFetch quota and would hit UrlFetchApp rate limits well before the Effex rate limits kicked in.
 Maximum item size 100k max The free tier , 512k. Other tiers not yet available    
 Lifetime 6 hours max     The free tier, 12 hours max. Other tiers not yet available.
 Analytics     none Full usage data by operation and size to the individual key level

In summary, for small caching requirements that are limited to within a single Apps Script, the apps script caching is a better choice. If you need to extend across multiple scripts, or share with other platforms you could consider Effex.







For more like this, see Google Apps Scripts snippets. Why not join our forum, follow the blog or follow me on twitter to ensure you get updates when they are available.