Different Survey Score Types and How they are Calculated

SMG focuses on top box scores to differentiate between an exceptional customer experience and an ordinary one. Top Box scoring captures the Overall Satisfaction (OSAT) ratings given by your customers. Although it is easy to assume all customers satisfied to any degree are similar, research has found customers who are highly satisfied are far more loyal and therefore more influential on a company's bottom line. Focusing on top box scores reinforces company-wide standards for quality customer service while uncovering performance differences between units.

A top box score means the customer gave the highest possible rating—there is no room for improvement. Not only do top box scores better predict loyalty intentions, but they also better predict actual loyalty behaviors. The benefit of tracking satisfaction at every unit is identifying variability, correcting deficiencies at poorly performing units, and having high performing units share best practices. Top Box is the highest performance measure. Combining two top box scores limits the ability to distinguish high performing units from units that are simply OK in the customer's mind.

There are at least three main reasons why tracking "top box" scores (e.g., highly satisfied scores) is preferable to tracking "top two box" scores (e.g., highly satisfied combined with satisfied scores):

1. Top Box scores send a message to customers and employees that excellence is the goal.
2. Ratings by highly satisfied customers have stronger relationships with more important outcomes than satisfied ratings.
3. Tracking Top Box scores uncovers important variability in unit performance.

Top box scores not only convey to employees and customers that excellence is important, they also lead to real differences in outcomes important to the business, like customer loyalty and comp sales. The real power of multi-unit measurement is to identify poor performing units that can tarnish the brand and to leverage the knowledge gained from high performing units to improve the customer experience across the company.

The Top Box Score is calculated using the percent of "Top Box" responses (those who rate a 5 - Highly Satisfied on a satisfaction question or Yes on a yes/no question). Highly Satisfied guests are more likely to return, recommend and spend more on average.

The Bottom 2 Box Score is calculated using the percent of "Bottom 2 Box" responses (sum of those who rate a 1 - Highly Dissatisfied and a 2 - Dissatisfied on a 5-scale satisfaction question). Bottom 2 box scores vary across clients and industries and typically account for a small percentage of customers. SMG believes in focusing your efforts on the opportunity that exists to drive satisfied respondents to highly satisfied to achieve the most impact to your business.

B3B reports on the percent of Highly Dissatisfied (1), Dissatisfied (2), and Neither Satisfied or Dissatisfied (3). The B3B is calculated by adding the 1s, 2s, and 3s and dividing by the Total Number of Surveys.

• For example, 20 people take the survey. Of those, 10 were 5s, 6 were 4s, and 4 were 1-3s. The B3B would be 4 (1-3s) / 20 Total Surveys or 20% (4/20 =0.20).

Deciding the right metric(s) for your brand requires exploring the Net Promoter concept in more detail and having a clear understanding of your brand's culture and goals. The logic behind Net Promoter Score is that promoters generate positive word of mouth that helps a brand acquire new customers, and detractors generate negative word of mouth that leads to customers avoiding the brand. Since having more promoters than detractors is good for business, the higher the NPS, the better.

The Net Promoter Score is typically based on a question that asks consumers how likely they are, on an 11-point scale of 0–10, to recommend a company's products or services to a friend or colleague. Ratings on this question are then divided into three categories: 

1. Promoters (those likely to recommend; 9s and 10s on the 11-point scale)
2. Detractors (those unlikely to recommend; 0–6 on the 11-point scale)
3. Passives (those neither likely nor unlikely to recommend; 7s and 8s on the 11-point scale)

The Net Promoter System is a philosophy and set of practices centered around measuring whether brands are treating customers in a way that leads to long-term growth and "good profits" or short-term revenue gain at the expense of long-term growth ("bad profits"). A Net Promoter System contains three critical components:

1. It measures and categorizes customers as promoters or detractors.
2. It puts feedback in the hands of employees and enables them to implement "close the loop" processes with dissatisfied customers.
3. It creates a culture that makes customer loyalty a priority by sharing customer feedback and celebrating loyalty champions.

How is this reflected on the Reporting Website?

• A Full-Scale Report will show the full distribution of responses. For example, how many responded as a promoter (9-10), passive (7-8) or detractor (0-6).
  In the below example, 1s are your detractors, 2s are passives, and 3s are promoters. Therefore, your NPS score is 48%-35%=13%
  
• A comparison report will automatically do that calculation for you (see below):
  
• This calculation is also typically found on many client Dashboards/Home Pages:
  

Rate Count & Rate Sum

The following new score types are available in the Score drop-down in 360 Admin.

These new score types can be configured on KPI, HDIC, and trend cards.

• 10,000 Normalized Rate Count
• 1,000 Normalized Rate Count
• 10,000 Normalized Rate Sum
• 1,000 Normalized Rate Sum
• Count Ratio

Count calculations count the number of responses in the numerator that meet the measure criteria (e.g., count of OSAT 1 or 2 responses) while the denominator is summed.

Sum calculations sum the numerator and denominator (e.g., the sum of the daily number of complaints received per store). There will be scenarios where a client will want to leverage the COUNT score on the denominator measure rather than the SUM score, which is why the Count Ratio was added as a new score type.

Count Ratio

The Rate Count score type utilizes SUM as the denominator (Count of Numerator/Sum of Denominator). However, this provides inaccurate calculations in cases where COUNT as the denominator is more appropriate (Count of Numerator/Count of Denominator). This necessitated the implementation of a new scoring type known as the Count Ratio.

Count Ratio score type was created for the scenario described above. The primary difference between Rate Count and Count Ratio is how the denominator measure is scored. By design, Rate Count uses a SUM score for the denominator, meaning all of the numerical values for the measure in the database are added together to form the denominator of the score. However, since the desired use-case from the above example is to calculate a percentage of Yes answers for a Yes/No measure, a SUM would only add the values for Yes answers since No is mapped to 0 in the database. When combined with the COUNT score of the numerator measure configured to use a Segment on Yes values, this will always result in a Rate Count score of 100% (since the COUNT of 1's is inherently the same value as the SUM of 1's and 0's).

The Count Ratio, however, uses a COUNT score on the denominator measure rather than a SUM score. When configured with a Yes/No measure, this COUNT will result in a value equal to the number of rows in the database for which any value exists for the denominator measure. Since the numerator measure is configured to count only the Yes responses with the Segment, the resulting calculation using the Count Ratio score is (COUNT of 1's / COUNT of all values). This results in a value equal to the percentage of Yes values amongst all responses in the card's source(s).

• Choose the right loyalty metric for your business. SMG recommends focusing on a satisfaction or loyalty metric that is the most appropriate for your business rather than using a one-size-fits-all approach.
• Use a scale that makes sense for your business and customers.
• Determine if a Net Promoter Score or Top Box score is best. When different calculations yield similar predictive results, pick the simplest calculation to explain and justify to stakeholders at all levels of the organization.
• Keep survey length manageable for respondents. SMG recommends asking about specific elements of the customer experience to give operators clear information on improving customer satisfaction and loyalty while keeping the overall length manageable for survey respondents.