How do you measure importance

When methods yield apparently inconsistent results, the role of the researcher in interpreting data and in establishing the meaning and implications of the results of statistical analysis is, however, critical.

Researchers continually strive to find the best methods to use in analysing customer satisfaction data. Significant research has been published communicating various techniques that will help researchers identify the best course of action when evaluating customer satisfaction data. The issue of derived versus stated importance is one that has been debated for many years.

While there is a considerable deliberation over this issue, most researchers agree that no one method is adequate. If forced to choose between derived and stated importance, some prefer stated importance, in part due to the ease with which it is administered and interpreted. Many, however, suggest that derived importance would be their choice primarily due to the confidence afforded to them by the resulting statistical model.

The purpose of the paper is to compare one measure of self-stated importance and three analytical methods of derived importance, and to identify the role of researcher expertise in resolving discrepancies in results. The value of the decision maker's experiential knowledge in reconciling and integrating conflicting results will be discussed. Customer satisfaction surveys are conducted by many organisations, but the utility of this data has been questioned. Even sophisticated approaches may have serious flaws because they tend to be complicated, ambiguous, have poor response rates, and are not associated with company profit and growth.

They suggest that customer satisfaction is an important concept that is significantly related to an organisation's profitability. Whether it is argued that customer satisfaction surveys are relevant or not, marketing researchers recognise that surveying is currently being conducted, and this trend will continue, for many reasons.

The method is reasonably inexpensive, is easy to understand, gives comparable time series data and appears to close a communication gap with the customer. Typically, managers do not look to optimise their decision-making process. Rather, they tend to look for logical answers near the issue. When using stated importance, respondents are asked to explicitly state their perception of the importance of each attribute being measured in a customer satisfaction survey.

Stated importance can be obtained fairly easily by using a rating scale, rank order or a constant sum. Researchers commonly use 5—7- or point scales to obtain importance ratings.

When using rank order to obtain stated importance, respondents are asked to place the attributes in order from most important 1 to least important maximum rank is the number of total attributes examined. The constant sum requires the respondents to divide points among all attributes with the most important attributes receiving the greatest number of points. Attributes that are not important are assigned a value of 0. Any number between 0 and can be used. The same number can be used more than once as long as the sum of all values assigned is These straightforward methods allow for easy collection of data and calculation of an importance value for each attribute.

Such measures, particularly when used to calculate a customer satisfaction index CSI , can be useful when companies are attempting to identify the strengths and weaknesses of their company as compared to their competitors, as perceived by the customers. While providing a simple means of assessing importance, the stated importance techniques are fraught with issues for debate among researchers.

Some of these issues include cost efficiencies relative to survey administration and analysis, variability of importance among attributes relative contribution to overall satisfaction, and impact of the attributes on satisfaction. In order to obtain self-stated importance ratings, the customer satisfaction survey must contain twice the number of questions, one each for importance and satisfaction for each attribute included.

Respondents often view this as a repetitive and tedious task. On telephone surveys, every minute will impact the cost of the study. Additionally, longer surveys are likely to negatively impact response rates. All of these factors ultimately affect the cost and quality of the survey.

A further limitation of the stated importance techniques, identifying relative contribution of attributes, is a strength of some of the derived importance techniques. Simply put, the relationship among attributes and the relationship between attributes and overall satisfaction is unknown when using stated importance.

While respondents may consider such relationships when providing their self-stated importance values, those relationships are not visible or known to the researcher and, therefore, cannot be used in analyses.

Deriving a measure of attribute importance in customer satisfaction studies can be accomplished through the use of various techniques. While multivariate analytical techniques such as conjoint analysis have been used by researchers, 26 correlation and regression analyses are the most commonly used tools in deriving importance.

When using either of these tools, each attribute or predictor variable is related to a broader measure or criterion variable, such as overall satisfaction, to identify the attribute's impact on the broader measure. Three such measures that are closely related include bivariate Pearson correlation, standard regression coefficient or beta weight , and the product of the beta weight and the corresponding Pearson correlation.

In rare cases where all attributes are uncorrelated, the three measures will yield identical measures of relative importance 27 but the use of derived rather than stated importance is not void of its own issues.

Problems can result due to multicollinearity which can be overcome with analytical techniques such as principal component regression, factor analysis, or partial least square regression , missing data and the choice of specific models used in analysis. The primary drawback of using correlation analysis to derive importance is that, while it provides a measure of the impact of the individual attributes on overall satisfaction, it does not take into account the relationship between variables.

In customer satisfaction studies, it is highly unlikely that the attributes do not have any relationship with one another. Research suggests that customers view products and services as a bundle of attributes rather than independent attributes. For example, when examining satisfaction with automobiles, a study might include attributes such as fuel efficiency and cost to operate the vehicle.

While the researcher may consider cost to operate as having to do with issues such as fluid or tyre replacement, the respondent may see the amount of fuel needed to operate the car as a cost of operation. In this case, the two variables would be highly correlated and would ultimately impact the results of regression analysis.

Additionally, respondents do not always provide a response to every question on a survey. Researchers must determine what to do about responses that are missing. The mean of all responses is often substituted for missing data to allow the respondent's other answers to be included in the analysis, but this can provide misleading results.

Furthermore, the type of regression model used and the level of explanation resulting from the model could hinder the use of this technique used in deriving importance. Researchers recommend using a standard regression model rather than stepwise, forward or backward selection techniques for inputting attributes into the model. Use of the various selection techniques can produce differing results. Caution must also be used in interpreting the results of regression analysis when an attribute has a regression coefficient that is nonsignificant.

Often an attribute will be found to have a negative relationship with overall satisfaction giving the appearance that the presence of the attribute results in a decrease in satisfaction when, in fact, the absence of the attribute would result in a greater decrease in satisfaction.

The topic of safety is often discussed as an example of this issue. Looking back at the example of satisfaction with automobiles, a regression analysis might show that safety is not a driver of satisfaction or does not have a significant impact on overall satisfaction. Such findings would imply that a company like Volvo would improve satisfaction if they changed their positioning strategy away from one of providing a high level of safety for their customers.

When asked why they purchased a Volvo, many customers, however, indicate that the company's safety record was a major factor in their decision. This suggests that some attributes are considered minimum requirements and do not contribute to customer satisfaction as long as they exist to the necessary degree. But if all automobiles are relatively similar on this, the most influential factor in determining purchase behaviour may be cost.

This type of influencer is often captured by using correlations and regression analyses. Reasons for using derived importance are in diametric opposition to the reasons for not using stated importance. Derived importance can reduce the cost of obtaining the data yet will likely increase analysis costs as multivariate statistics, rather than descriptive statistics, will need to be used.

In contrast, the shorter questionnaire will reduce demands on respondents and likely increase response rates, consequently decreasing overall survey costs. Also, there are differences in prediction and contribution to take into account when deciding on whether to use derived or stated importance. The results show that a higher CSI is obtained when using stated importance than when using derived importance. Although the attributes that contributed most to customer satisfaction were the same regardless of which importance method was used, Chu perceives that the derived importance method is superior due to its power of prediction and explanation.

One of the primary advantages to using regression analysis to derive importance is that the regression model provides a statistical model of the relative impact of each attribute on overall satisfaction and also assesses the relationships between attributes. Correlation analysis provides a similar view but only examines the relationship of one attribute at a time with overall satisfaction.

Both of these analysis techniques allow the researcher to identify those discriminating or differentiating attributes that will seemingly make the greatest contribution to customer satisfaction. Hanson 50 and Di Paula 51 both favour the use of correlation over regression in deriving importance. Hanson's suggestion is based on theoretical and practical experiences with real-world data sets. Di Paula's recommendation is based on a rank order of both derived and stated importance of survey attributes.

The rankings were then correlated to examine any differences. Although both sets of rankings were statistically correlated, indicating that both measures captured the same information, he suggested using both techniques initially until it can be determined that the two methods yield the same results. Then, he surmises, it would be safe to eliminate the stated importance measure from future administrations of the survey.

In an examination of three data sets, Gustafsson and Johnson 52 compared a variety of methods for statistically deriving attribute importance.

Similar to the debate on whether to use stated or derived importance, the researchers found that each method of deriving importance has its strengths and weaknesses making it difficult to recommend any one technique over others. Press ESC to cancel. Skip to content Home Essay What is the importance of measurement in our daily lives? Ben Davis May 8, What is the importance of measurement in our daily lives?

What is measurement and its importance? Why is it important to measure? What is the function of measurement? What are the characteristics of good measurement? What are the qualities of good research instrument?

What are the three main qualities of a good test? Why it is important to have a good research instrument? What are examples of research instruments? What is difference between validity and reliability? Your analysis will then be the percent of respondents who selected each item in their top 2.

Many of the web survey tools will allow you to limit — or force — the number of items checked to a specific number. The number of items you should ask people to check is driven in good part by the number of items from which they have to choose. Two or three is a reasonable number of items, but if you have one item that you know everyone is likely to select, then you might want to ask for an additional choice.

For example, price would likely be a top-three choice for everyone regarding factors that affect a purchase decision. To enhance your data, you can also pose a follow-up question asking the respondent which of the three choices they just checked would be their number one choice. Then, you could pose a second follow-up question about the remaining two choices. Some web-form survey software will perform a threaded branch in which the selections from the first question are carried forward to the subsequent questions.

A question format that combines the interval scale with the forced ranking is the fixed-sum or fixed-allocation question format. Here, you present the respondent with the same set of items and ask them to allocate points across the items based on their relative importance.

The respondent has to make a trade-off. More points for one selection means fewer points for another. A key decision is how many items to present. Four, five or ten are optimal since they divide evenly into Some web survey tools allow you to set the total to something other than For example, if you decide on 7 items, then have the items total to The tools should provide a running total to the respondent and can force the question to be completed correctly.

Otherwise some data cleansing will be necessary, but that is likely worth the effort for the very rich data the format can generate. One way to measure importance is to not ask it at all! Instead, importance can be derived statistically from the data set. Consider the scenario where you have questions measuring the satisfaction with various aspects of a product or service and you want to know how important each is to overall satisfaction.

Include a summary question measuring overall satisfaction, which you probably would anyway, and skip any questions about importance. Using correlation or regression analysis, you can determine which items align most closely to overall satisfaction. But be sure your manager can understand and will accept the findings. A final method involves a more complex statistical technique, conjoint analysis. This technique is particularly useful in constructing product offerings where various features are combined into bundles, and the researcher wants to know how important each factor is in driving purchase decisions.

Conjoint analysis requires a special research survey where the respondent is presented with pairs of factors asking the relative importance of one over the other. Subsequently, the respondent is presented with two versions of the product, each with different sets of features based upon the previous responses and asked which product version they prefer.

Think about these alternatives when constructing the research program and the survey instrument, and you can generate useful, meaningful data for business decisions. My fall back is the multiple-choice checklist approach. Simple for the respondent, yet it provides meaningful information for management.