The power of session-level analytics reporting in Sitecore Analytics

Some background on session-level analytics in Sitecore:

The conversation often comes up when discussing Sitecore Analytics: "when should I use Sitecore Analytics and when should I use xyz Analytics?".  The use of multiple analytics tools, the coordination between them, and/or the decision to standardize on one analytics tool is a significant topic, but from my perspective it's simply important to consider what the landscape of Sitecore Analytics provides.  On the surface:

1. Sitecore has incredible analytics drill-down capabilities--from a list of recent visits, to all visits from a particular company or source, to an individual visitor, to an individual visit.
2. Sitecore offers standardized OLAP cube data, and a set of sophisticated business intelligence dashboards
3. Probably most importantly, Sitecore provides ACTIONABLE analytics.  Your reports are right there in your power tool of content authoring and publishing.  Now you can DO something with that analysis right away.
4. Sitecore provides incredibly compelling segmentation possibilities.

Together these capabilities span the spectrum of the ultra-specific (an almost Tealeaf style capability of watching an individual session which could be interesting for many situations such as a high-price ecommerce transaction, evaluation of a sample session for usability, etc.) to the ultra-summary (the rolling up and tearing back down based on my own business’ dimensions or facets).

So for today, let’s get ultra-specific.  For those of you that have read other articles about Launch Sitecore, you’ll know that our purpose is pretty specific—create an evaluation package that’s best-in-class so that our prospects can quickly learn that Sitecore is the best platform choice on the planet.  For those of you that understand our Engagement Plans, you’ll know that this process looks like this:

1. Through demonstrations and technical deep dives, our Sales Engineering team builds interest in having a prospect visit www.launchsitecore.net.
2. While enjoying some articles about Sitecore, our visitor is really urged to login or register to get to the good stuff (the download of the package itself).
3. After registering, the visitor wants to download the package (the site itself that they can apply to their local test installation of Sitecore).
4. After a glorious evaluation of the product, the visitor (now our customer) tells us how great the experience was in an incredibly over-simplified feedback form.

So, since this is only intended to be marginally real-world, let’s go through this is a visitor, all in one visit and see what we get on the other end from the reporting side.

The Simulation:

First, visit www.launchsitecore.net.

Since we’re simulating a brand new visitor, we’ll go ahead and register:

We are so excited about Launch Sitecore, we want to get our hands on the package right away, and we choose to download the version for Sitecore 7.x:

As we read a Digital Marketing System focused article, we check out the slide out at the top right of the screen, used to show the type of information Sitecore is collecting about our visit.  We notice that we've been categorized in the Digital Marketer Audience Segment (based on a number of articles we've read, including the "Create a Goal" article.  We also notice that it seems we are investigating the site to a level that shows we have a "Detailed" interest in the content.  We're not just reading the home page--we've investigated some detailed articles about the Sitecore Digital Marketing System.

The Reporting:

In the Sitecore Marketing Center, we can find the Latest Visit report.  Lots more to talk about around this reporting interface itself (including filters, date ranges, etc.), but for today let’s get to the info about our simulation.  First, since I’m in my home office on a snowy New England day, I’m behind my Comcast ISP account.  Beyond the scope of this article is my ability to tag my visitor sessions with additional information (from forms filled in, from authentication events, etc.) that could further identify my visitors.  In addition, I chose to “categorize” all Comcast Cable visitors as “My Company”.  (I could have categorized this as “ISP” or anything else I’d want to use to describe visitors from the Comcast DNS resolution).

I could drill down to all Visits from Comcast Cable by clicking on that link.  That would then show me more specific visitor sessions (possibly with specifically identifying tags).  Since I just ran through the simulation, I know this session is mine and I’ll bypass that by clicking on the link under the Date & Time.



Now at the Visit (Session) Detail report, I get a nice snapshot of my overall visit.  Some things to notice:

• We were able to tag the session with “MikeBlogTest” (the name I used to Register with the site)
• We see that I visited by clicking on a link within my Blog
• We see the total value of the visit was 85 Engagement Value points (with the breakdown of goal achievement following)
• We get a nice breakdown of the Pattern Card shape that is emerging from my visit—the specific content profile key attributes that are accumulating based on my content consumption.

Further down in the report, we start to get even more detail about the visit, including:

• The fact that my Campaign was triggered
• Fields that were filled out in our registration form
• Pages visited and specific duration for those page views
• Goals achieved (Register)

Another page, and more detail to round out the entire session.  More goals and fields filled in.  One interesting thing I want to point out here is that we actually captured an “error” on the site (line 17).  Error is in quotes here since, depending on how we handled it in the application, this could have been completely invisible to the visitor.  While this isn’t in any way a sophisticated error handling technique (it’s not meant to be), it could provide incredibly valuable information to the reviewer of the report in the context of the site visit.  What this tells the reviewer is that when a visitor clicked on a Search Result, there was no presentation assigned to the item clicked on.  A nice bit of usability testing after the fact (hey, we can’t catch everything in QA).

The Summary:

While not every analysis requirement will focus on an individual session or individual visitor, it is important to know that we can drill down to this level.  This gives us an incredible foundation to consider ties to transactional systems (Ecommerce), CRMs and other data sets to develop a full picture of our customer experience and value.  This is the ultimate leaf of the analytics tree, while keeping the ability to aggregate and roll back up to the branches.  By rolling back up, we will be able to get this same insight at the company level (all visits from one of our prospects), at the Pattern level (with Sitecore's ability to model the audience segments, now I can use those segments to hold aggregate analytics), and much more....

Using Sitecore Engagement Plans as an Audience Segmentation Tool

Engagement Plan as Audience Segmenter

I've been kicking around the idea of, instead of strictly mapping out a customer journey with Sitecore Engagement Plans, to instead use them as a visual audience segmentation tool.  With the great analytics that Sitecore is collecting, this seems like a natural extension to be able to divide the visitors within that analytics data into well-defined categories of patterns.  I'll end this post with an idea that could be added to the current Sitecore concept of Pattern Cards (with your great development addition, of course).

An example audience segmentation Engagement Plan

The idea is this: use the States and Conditions of Sitecore’s Engagement Plan to get a visual indication of the distribution of your audience based on any segmentation criteria.  Since the Rules Engine is so nicely involved in the Engagement Plan Condition checks, this to me is an incredibly natural use of this tool.

The Idea in Launch Sitecore

Here’s how it falls out for Launch Sitecore.  Our audience segments are clearly defined by the groups we envision might visit us at www.launchsitecore.net.  However, this definition is strictly based on content consumption—using Sitecore’s Pattern Card matching, we simply watch for the thresholds between these content consumption patterns.  If someone is reading a lot of articles around Sitecore’s Digital Marketing System, we might conclude he appears to be a marketer.  While this is really cool and easy to do, I have always thought that these patterns need to be a bit more descriptive, inclusive of content consumption patterns and potentially a lot more.  In addition, the definition of this pattern should be flexible and open to change on an ongoing basis—without development.

Talking to a lot of smart partners and clients lately has verified my assumptions.  As we get into the real-world use cases, we talk about all the attributes that should come together to fully define a marketing persona—everything from campaigns to keyword use in search to “known” profile attributes in CRMs to landing pages to referral sources.

Enter the Rules Engine

Enter the Sitecore Rules Engine and the perfect modeling environment for this situation.  Back to Launch Sitecore and our example.  As a simple start, all I want to do is segment my visitor traffic between two opposing possibilities—the visitor appears to be a Marketer or the visitor appears to be a Technologist.  We know that a couple of our Patterns nibble at this persona, and through content consumption our visitor might look like Mark the Marketer, or Devon the Developer.  But what we’re shooting for here is a much richer definition of the age-old split between marketer and technologist.  (By the way, if you consider yourself the perfect marriage of both, give me a shout).

First, we begin with a State that everyone is going to start their journey in:

The initial Engagement Plan state that all our visitors start in

We also get to decide which visitors we want in this plan.  For example, we might only want to evaluate visitors from a certain campaign.  Maybe I'm writing lots of articles on digital marketing subjects, and I want to test if the readers of those articles exhibit what I've defined to be "marketer" behavior on the site.

For the visitors we decide to track in this Engagement Plan, the first assumption is that we don’t yet have enough information to define them as Marketer or Technologist.  But we can immediately rectify that by invoking the Rules Engine in our first condition check.  We could check for either, but we chose to to first see if we should deem this visitor as technical.

A set of conditions we can check for to see if some one is "a technologist"

Our Condition invokes the Rules Engine where we can build a very sophisticated grouping of attributes that, for us at Launch Sitecore, peg someone with the Technical label.  We’re going to use the Pattern Card matches as a possibility (now grouping a couple of them together that represent our two technical Patterns), as well as including an additional possibility that someone got to Launch Sitecore using the word “development” in their search phrase (i.e., “Sitecore development”).

While this is a simple example, hopefully you’re seeing the possibilities here.  Never mind how rich the out of the box condition set is in Sitecore (geography, site, where you are in the content hierarchy itself, etc., etc.), we also know that this condition could be absolutely anything you dream up that returns a true/false.

If the condition returns true, we’re done and place the visitor in the Technical bucket:

A state for those we've deemed to be Technologists

If the overall condition returns false, we drop down to the check for the Marketer:

A set of conditions we can check for to see if some one is "a marketer"

As before, we are able to reuse our initial logic around Pattern Cards, with Connie and Mark combining forces as a possible reason to label a visitor as a Marketer.  In addition, we look for a search keyword containing DMS (Digital Marketing System) as well as a check for my MarchBlogBlitz campaign.  Since many of my articles in this time period are covering Analytics, I’m making the assumption that we have some Marketers on our hands.  Keep in mind the ability to replace the “or”s for “and”s, giving us possibly more granular condition definition.  As above for the Technical case, if this multi-clause condition returns true, we dump the visitor in the Marketing bucket.

Nice little trick if both of these checks return false.  We simply put the visitor back into the Undecided state and let other things trigger another check (maybe the next check is done on a follow up Goal, Page Visit, Login Event or anything else we might wait for our visitor to accomplish).  We do this by one last Conditional check, which always returns True:

Send the visitor back until we're sure

Here’s another look at the overall plan:

The full audience segmentation Engagement Plan

Pretty simple, but sky’s the limit.  One really cool idea that fits nicely into this is a very simple geography breakdown.  I could have, for instance, three states representing West, Central, and Eastern United States and immediately drop visitors into those states as they enter the site based on our GEOIP resolution.  (Since I am in the world of ideas only here, be sure to appreciate performance implications of making such wholesale checks).

Manually test how a visitor maps to the plan

One last really cool thing.  Since I have lots of visitor information in our Launch Sitecore site, I can even run a visitor through my new Engagement Plan.  Using the Supervise features of Engagement Plans, I grabbed my user (the Visitor associated with my Facebook authentication).  I moved that User to my new plan and the Undecided State.  In Supervise, I can manually press the trigger:



Since my Conditions are checked at Session End (among other events), I chose to run the Session End Trigger.

Sitecore goes through the entire logic of my plan and determines that my user account should be placed in the Marketing bucket.

The results of manually sending someone through the plan

Way cool.

NOW GO DOWNLOAD LAUNCH SITECORE.  (Need to get that great data for my reporting experiment).  I’ll let you know how our audience trends between this new Marketing and Technologist segmentation.

Extra Credit

For the interested student and the great developer….

Where I think this would be even more interesting is to use this same logic in the Pattern Card feature of Sitecore.  My prototype took all of 2 seconds (since it doesn’t in any way work).  But for an enterprising developer that adds the Rules logic to the Pattern Card check, this seems like a killer feature:

This becomes especially interesting considering Pattern Cards are an automatic dimension of Sitecore’s Analytics reporting.

Modeling a campaign in both Google Analytics and Sitecore Analytics

I’ve been off busy gathering lots of great data for my ongoing investigation into using Google Analytics and Sitecore Analytics side by side.  (OK, I haven’t been writing for a while and that’s my excuse).  Now I am just about ready to tackle the reporting side of my experiment.  Before doing that, I realized that there’s a little bit of cleanup I needed to do to make sure I’m modeling a similar environment in both analytics systems.  In my next set of articles, I’ll be evaluating GA reports against Sitecore’s Engagement Intelligence Analyzer reports for some typical expectations—page views, time spent on page, traffic sources, browsers and devices, etc.  In addition to the typical, I will then try to highlight additional insight that each tool is able to provide (Sitecore profiles, traffic segmentation, conversion funnels, engagement plans).

As described in previous posts, I already have the Conversion (GA) / Goal (Sitecore) of “Download Launch Sitecore” set up in both systems.  Today I will set up one last facet—a campaign.  In follow up articles, I will use campaign codes to see the resulting traffic from this series of articles.  I will call it “MarchBlogBlitz”.  I will be shameless in my attempt to get you to download Launch Sitecore in every article, annoying you to the point where you just have to visit the site and click the download button.  I want a spike in traffic, lots of conversions, and pretty reports.  Goal defined, and thanks for your help.  NOW GO DOWNLOAD LAUNCH SITECORE. (Of course, register or login first).

Welcome back.

First, the campaign in Sitecore:

We have a folder of campaigns (a campaign category) called “SE Driven Traffic”.  Our team can use these campaigns to identify traffic sources (across channels) and we can have a little competition to see who on the Sales Engineering team can drive the most traffic to Launch Sitecore.

In the Marketing Center in Sitecore, I am able to easily insert a new Campaign in this category.  My Campaign is called MarchBlogBlitz.

There are a few additional fields that we could fill out, but for now let’s just concern ourselves with the Campaign Link (which is already filled out for us).  This is the query string that will tell Sitecore that incoming traffic is part of this new campaign.  sc_camp is the default query string parameter that Sitecore uses, but this is technically configurable.  The GUID after the equals sign can also be changed, but for my purposes I’m going to leave it as is.  In fact, my shameless attempts at sending you to Launch Sitecore are already using this Campaign ID.

One REALLY interesting thing here is that we could very easily add a Visitor to an Engagement Plan state.  If I took one small additional step, I could have a very simple and nice Engagement Plan in Sitecore that allows me to watch a Campaign Visitor as he progresses through some states towards my end goal of DOWNLOADING LAUNCH SITECORE.  (Something for the next campaign and experiment).

So off to Google.  Google allows you to define any number of custom campaigns (this is separate from the money making AdWords feature).  Defining a campaign is as simple as adding some predefined query string parameter attributes within your traffic source links.  Google has a handy tool to create your URLs based on these attributes.  For mine, I am including only those that are required (there are many to fully organize your campaigns):

Done.  The resultant URL from Google is:  http://www.launchsitecore.net/?utm_source=blogspot&utm_medium=blog&utm_campaign=MarchBlogBlitz, which I will use everywhere.  Of course, I will add additional query string parameter from Sitecore (sc_camp=E8D2CAF7FDAB4383A473E350F2FA81B3) and I’m off to the races with my comparison.  NOW GO DOWNLOAD LAUNCH SITECORE SO I CAN CREATE SOME COOL REPORTS.

Thanks.

“Dynamic” Goals in Sitecore CEP and Google Analytics

A quick aside as I continue to compare and contrast my experience setting up GA with my current understanding of Sitecore CEP….

I started thinking about Goals in a much more dynamic way.  While I appreciate the Engagement Value that Sitecore assigns to the Goals you can model directly in the system, I often get the feedback that goes something like, “yeah, but we might have one goal that would be worth very different things based on other factors, like where it’s accomplished, on which site, based on whether the visitor has accomplished something else”, etc., etc., etc.

As always, in ECommerce this is usually pretty straightforward.  Goal = Purchase, and Value = Invoice Total.  With most of our Goals, though, the Value can be much harder to agree on.  I like the example here of a site registration Goal.  Two different departments might have this same Goal, but they might weight it very differently in terms of Value.  One department has determined that registration leads to a particularly high ECommerce conversion rate, so they feel the act of registration itself should be high up the Engagement Value pyramid.  Another department, while still wanting to track the Registration goal occurrence, hasn’t seen the same focused correlation towards their interesting conversions.

While we can certainly consider setting up different Goals for the different purposes (and starting down the path of Department 1 Registration, Department 2 Registration), what I like about the opportunity of this use case is to detach the Engagement Value from the Goal itself.  The simplest code sample I could think of is below:

if (Sitecore.Analytics.Tracker.IsActive)
                Tracker.CurrentVisit.Value += 1000;

where we arbitrarily add 1000 Engagement Value points to the current Visit.  While this is oversimplified, it highlights the main point here….let additional logic determine the actual Engagement Value based on additional criteria (in the Sitecore Context or otherwise),  and then promote the simple maintenance of that additional criteria.

Tying to a CRM for an attribute like this could be very interesting (salespeople determining the value of a demo request for their own region).  I also like the idea of modeling this as Sitecore content.  Maybe a Goal should look at where it is in Context to the content hierarchy (if the current item has a Value use it, if not look up the tree, etc.).  Probably better—model this in the Rules Editor (multiple clause condition like “if item is subitem of books section, and visit number is greater than 5”).

I started down this path as it’s taking me longer and longer to figure out how to model complex Goal achievement in Google Analytics.  Of course I’m quite biased towards Sitecore here, but thinking through the Sitecore API possibilities, coupled with allowing content authors easy maintenance has been quite a bit more exciting than considering all the landing pages I’d need so far in my Google Analytics Goal scenarios.  To be fair, I have to dig deeper into GA and see what I get from their API.

Launch Sitecore Goals Modeled in Google Analytics and Sitecore

 

The main purpose of this post is to simply understand how I can, in GA, model the “Download Launch Sitecore” goal that we currently have modeled in our Sitecore site.  Before that, a quick aside that has always confused me—visits, unique visitors and new visitors.  Good to remind ourselves of the simple--yet still sometimes confusing—stuff.

A visitor comes to our site.  The visit is counted as a visit.  Plus:

If this is the first visit from this visitor in this reporting period, the visitor is recorded as a unique visitor.

If this is the first visit from this visitor ever (at least by available information, cookie, etc), the visitor is also recorded as a new visitor.

OK, not the purpose of this post.  For this discussion, we want to end up with a simple goal modeled in both Sitecore Analytics (done), and GA.  We’ve already discussed a full explanation of the goal strategy for Launch Sitecore, so for today we just have to concern ourselves with trying to track how many of our visitors download the Launch Sitecore package from our site.

A quick review of how we accomplished this in Sitecore.  We had some choices here when we considered how to identify that our Download Launch Sitecore goal has been accomplished:

1. Use the fact that our visitor made it to the page where the download is available
2. Use the “onclick” event sparked from clicking the download link
3. Identify the request to the actual Sitecore package (zip file stored in Sitecore’s Media Library) as the actual accomplishment of the goal
4. Somehow figure out when our visitor ACTUALLY installed the Sitecore package

I put this list in order of effectiveness, in my opinion.  Using #1 would give me the least confidence in the validity of the goal accomplishment numbers….#4 the most confidence.  The nice thing with our decision in Sitecore—#3 is actually just as easy to model as #1.  The reason:  since the actual Media Library asset is referenced / stored in Sitecore (the zip file package), we can simply check off the Goal box for that Media Library item:

The same would hold true if we chose #1, since we could apply this goal to the visit of a page rather than a request of a Media Library item.  #2 would actually take a bit more work, most likely setting the PageEvent (Goal) via the API.  A couple lines of code, but more work than simply clicking a check box.

#4 is interesting, but balancing the work it would take with the additional benefits, it’s not yet done on Launch Sitecore (maybe a nice future feature).  We would have to do something within the package install to call in the goal accomplishment.

Off to Google.  To be fair, remember that this is my “out of the box” attempt at setting a Goal in my Google Analytics.  And for now, just trying to achieve parity with the Download Launch Sitecore goal we already have in our Sitecore solution.

Turns out, easy enough.  Since I can include the link to the actual Media Library asset:

http://www.launchsitecore.net/~/media/Files/Packages/LaunchSitecore6622.ashx

….this URL will now represent my Goal in GA.  Less flexible overall, since I will have to remember to change this URL when we change the package file (or always keep the file name the same).

 

For now, this will be good enough for us to start tracking the same goal in GA.  For consistency’s sake, we’ll also assign a Goal Value of 25 (same as the Engagement Value in Sitecore).  My Launch Sitecore boss is already telling me this is too arbitrary, so as we go we better do a better job rationalizing (potentially monetizing) this goal value.  I can see some CRM work in our future.

Off to model campaigns in GA (remember back to our Launch Sitecore exercise, where we have campaigns that represent Social Network Login visitors, visitors who click on an auto signature within a Sales Engineer’s email, and email (for future use).

Launch Sitecore and Google Analytics: Overview Reports

In my investigation of Google Analytics (and I’m new to it), I wanted to understand what you get for table stakes reports.  Logging into to GA gives you a really nice overview:

Hard to see such a rich report in a blog article, but some of the things I’d expect from an analytics are there—visits, unique visitors, page/visit, etc.  In addition, a very intuitive interface let’s me see more specific table-based reports that filter my viewpoint by browser, visitor location, operating system, and more.

Sitecore’s Engagement Intelligence Analyzer (and the full business intelligence suite it enjoys) employs a robust interface strategy, where dozens of prebuilt dashboards are available, each additionally customizable with filter criteria.  The example report below allowed me to easily define date ranges and looks at visitor locale (by country), to see a nice bar/line graph plotting both average visit duration and pages per visit, see a quick list of referring sites comparing the last 30 days against the previous 30 days and more:

There are plenty of additional dashboards that reflect all of the data shown on GA’s opening dashboard (stats covering visit duration, bounce rate, operating systems, browsers, etc.).

In addition, each of the individual graphs above gives me the option to expand / contract levels, add additional criteria, toggle between a chart and the data table behind it and more.

Here’s the same dashboard with the table data toggle on the top right graph:

The available dashboards make readily available all the same information on the Google default dashboard.  Of course, with a full-featured BI environment, the possibilities for customization are much greater here.  The Operating System report, for instance, gives me “live” graphs showing visit, conversion, duration and page per visit information.  As I click on a particular operating system, those graphs are refreshed and updated with the relevant data to that operating system only.  A full “drillpad” gives me even more power to fine tune and specify my criteria:

OK, so that’s just the surface, but I’m now confident that Sitecore Analytics provide me easy access to all the “default” analytics data in Google.  Since I know a lot about Sitecore Analytics, now I’m off to learn about segmented targeting, goal and campaign identification in GA so that I can fully understand the level effort to model these (I already have these marketing assets modeled in Sitecore and our Launch Sitecore site).

Off to consider both systems as I try to leverage analytics to answer the important questions for our Launch Sitecore site—why does our site exist, what do we want our visitors to accomplish, how are they getting to the site and which full path leads to optimal results?

Next up, setting Goals in GA.

Launch Sitecore and Google Analytics

Yes, Google Analytics.  I decided it was time to step back and figure out how the leader of the pack does things so that I can truly understand Sitecore’s offering in this space.  As I’ve been diving more and more into the analytics discussion, my main focus has been on understanding Sitecore’s quickly growing and rich analytics offering and to describe the benefits to partners and prospects.  Invariably, though, the challenging discussion comes up:  “do I use this instead of Google Analytics?”, and: “I have this set of reports from GA….are you saying Sitecore’s reports are different, better?”.

So, now I’m setting off on a journey to understand this a lot better.  The good thing—we have a public Launch Sitecore site that has been up and running now for some time with both Google Analytics and Sitecore Analytics gathering some great data.  Not a huge site, but very effective in considering things like Goals/Conversions, Segmentation, Referral Sources / Campaigns, and more.

My goal with this exercise is to truly define the best of both worlds.  Google Analytics is huge, rich, entrenched, effective.  Sitecore Analytics is laser focused on the the best Customer Engagement Platform on the planet.  Let’s try to answer the questions….where do these tools differ, overlap, complement?  How are they used together and what does each uniquely bring to the table?

For this simple start, one question that often comes up:  how do I integrate Sitecore with Google Analytics.  While this discussion could take off on a few different directions, let’s keep this ridiculously simple for now and see where we need to expand the answer.  For now, because of the great Sitecore presentation strategy, it’s as easy as this:

1. Sign up for a Google Analytics account if you don’t already have one
2. Copy the javascript GA gives you to insert into your pages for tracking
3. In Launch Sitecore, since we have one Layout that governs our entire browser-based design, all we have to do is insert that javascript right before </body>.  This will automatically track every Sitecore item / page that uses this layout (in the case of Launch Sitecore, this means all pages).
4. Sit back and let Google start collecting
5. (oh yeah), Sitecore DMS is already working its collection magic without the javascript.

Next we’ll start to drill into the data a bit and start to understand if our Launch Sitecore site is meeting up to our goals.