Articles

A project dump

Current endeavors

  • ClarityAd, Creative verification software for the online ad industry

Past experience

  • 2008-2011
    CFO then GM at Fotolog and Allopass USA
  • 2004-2008
    Finance
  • 2000-2004
    Audencia business school-France
Let's talk!

Lifecycle emails with KissMetrics

Growth hacking for the rest of us

Get the source code

Intro

KissMetrics is great. You push a lot of data and pull a lot of information. Unfortunately, there is no way to programatically access this wealth of information. Yes, they let you download your data on S3 in JSON and do whatever you want with it but that sounds very much like reinventing the wheel to me.

User behavior => Reports I'm supposed to look at regularly (I don't)
The KissMetrics flow

Lifecycle emails

After some time with KM, I realized that the best use of my time was to identify some behavior patterns and get in touch with people accordingly: Lifecycle emails!.. at least manually, which was neither achievable nor desirable. I set off to identify any crude solution that could allow me to automate this task, refine it and scale it.

I came up with this model for what I wanted to achieve:
  • Build an insightful report about specific behaviors
    • "2 days ago, user visited the pricing page, hasn't subscribed to paid plan yet",
      let's call it "Offer1 Report"
  • Build an email template with a relevant message for the specific behavior,
    let's call it "Offer1 Template"
  • Run the KM report and send the email
  • Track emails views and email clicks with KM

Close the feedback loop

With so many touch-points now tracked, we can split our user population in as many groups, for example:
  • Visitor never created a user account
  • User never hit the pay-wall
  • User hit the pay-wall and subscribed right away
  • User hit the pay-wall but never opened the offer email
  • User hit the pay-wall, clicked the link but didn't subscribe
  • User hit the pay-wall, clicked the link and subscribed
Going one step further, we're closing the feedback loop by building new KM reports using those events, for example:
  • "Received the Offer1 email, didn't open it, haven't subscribed to paid plan yet",
    let's call it "Offer2 Report"

We now have a scalable model to automatically gather information and act upon it with automatic touch points that we track.

User behavior <=> Automatic touch points
The sit-back-and-relax flow

Abusing KM's JSONP REST API for profit

The KM reporting front-end exposes a convenient JSONP REST API that supports the backbone.js-based application. The queries are sent to https://dcX.kissmetrics.com/q as GET requests.

Sign your requests

Interestingly for us, authentication is not based on the current user's session. Rather, each request is signed with your API key and your secret key using HMAC-SHA512.

The signing process works just like you'd expect:
  • Put all GET parameters in an array
  • Add a timestamp
  • Add your product key
  • Sort the associative array by key
  • URL encode (some shenanigans involved here, see my source code)
  • Concatenate
  • Hash + Base64 and append the resulting signature as an URL parameter "api_sig"

Find your keys

Go to https://www.kissmetrics.com/people then to the console and look up the peopleSearchRouter.queryOptions object. It contains the productKey and productSecret values that will allow you to sign requests to the REST API.

Forge your queries

When you run a report, KM generates some JS code that represents your query.

What works well for me is to build a report in KM, then see what it looks like in the browser's Network inspector. You need to find the "query" parameter in the URL request to "https://dcX.kissmetrics.com/q?sort_by..."

We only care about the "prefilter" JS function, the rest never changes (at least for a "People" report). Then programatically update the timestamps so that you can run this report in a cron.

I played safe and did my best to mimick the query format generated by KM's UI in every single way. Obviously there is a lot more potential to this as the query language is more powerful than what can be done with the report building UI.

(PHP)    
// Date interval
$to = strtotime("00:00:00 UTC");       // last full day
$from = strtotime('-1 day', $to);      // the day before
$to--;

// Query example
$prefilter = 'function(person){
                 return person.get_prop_matches_between(4, \"contains\", \"pricing\",
                        \"case_insensitive\",'.$from.','.$to.').length > 0
                     && person.get_num_times_did_event_between(26, '.$from.', '.$to.') == 0
                     && (person.get_num_times_did_event_between(27, '.$from.', '.$to.') == 0
                     && (person.get_prop_matches_between(11, \"contains\", \"'.$template.'\",
                        \"case_insensitive\", '.$from.', '.$to.').length > 0)) 
             }';

Putting it all together (a PHP implementation)

We have everything we need to talk to KM's REST API.
There are 2 methods:
  • /q: submit a request
  • /u: get a status update or the full result if finished.

I did the heavy-lifting for you and implemented it in PHP.
As I envisioned this for a cron, the /u method is ran synchronously until it gets a result.
The code for this implementation is available for download here.

$kmr = new KM_Report($apikey, $apisecret)
if($kmr->runPeopleQuery($prefilter))
{
    foreach($kmr->results->data as $p)
    {
        // do your stuff, like sending an email.
    }
}

If you're going to send emails, I recommend you use Mandrill, Mailgun, SendGrid, or any other transactional email platform of your choice, rather than using SMTP from your own server...

Track your lifecycle emails

Looking to create new KM reports based on email touch points, measure view rate, A/B test emails?
Let's list the new events that KM will now track for us:
  • Received email

    Track when you send an email (server side), so that users never get it twice by mistake.
        KM::init($apikey);
        KM::identify($email);
        KM::record('Received email', array('name' => $template));
    
  • Viewed email

    Let's set up a <img> beacon to track when users view your emails
    $beacon = "<img src='http://trk.kissmetrics.com/e?_k=$apikey&_n=Viewed+email
               &_p=$umail&name=$utemplate'/>";
    
  • Clicked email

    KM's URL API allows us to track inbound links coming from your emails
        $url = "http://www.mysite.com/landingpage?kme=Clicked%20email&kmi=$umail&km_name=$utemplate";
    

A/B test your lifecycle emails

Now that we have this system in place, we can go one step further and A/B test our email templates. We'll simply create two templates Offer1A and Offer1B, and send each of them 50% of the time. Based on the above, all the tracking is already in place to measure A and B and conclude.

Beyond lifecycle emails

You won't solve all problems by throwing emails at people.

Once you find groups of people that are stuck at different points in the engagement, feed your database with this info! You'll be able to implement any change that you deem relevant for those buckets of users (and measure the impact).

  • Have a subset of your users participate in a survey next time they log in.
  • Target A/B tests to a specific portion of your user base.
  • Automatically hide advanced features for people who seem to get lost in your UI.
  • Would love to hear more ideas from you ;-)

Wrap-up

I found this solution to be a neat way to keep all your marketing data under KM's umbrella. Now some disclaimers and miscellaneous warnings:

  • There is a two-hour delay until any event appears in the reports. If you run a cron for the day before, take it into account or you'll miss some data.
  • There is little chance that this won't break at one time or another, as it relies on KM's undocumented private API. I'll update my code as need be if/when possible without any guarantee.
  • There is some potential for trolling: Don't use the reporting API in ways that could disrupt KM's service, use your common sense.
  • I sincerely hope that KM will support or at least accept the use of this API but this work is not endorsed by them. This is another risk you'll have to bear with when using it.
  • Update: Hiten, cofounder@KM, pinged me to let me know he loved the idea and is looking for feedback to make this API more usable in the future! Thank you Hiten!

Having said that, if you're looking to implement this and need a little help, please don't hesitate to get in touch or join the discussion on HackerNews.

Update

Wow! HackerNews Front page, bandwidth exceeded on site44, enthusiastic message from Hiten, co-founder at KissMetrics, what a day. Thanks for all the feedback!

Share this article



All Rights Reserved. - Hosted by site44.com