Our research on apartment community resident satisfaction shows that nearly 50% of turnover is controllable.

With the increased pressure from investors, Wall Street, competitors and an increasingly savvy consumer, the importance of sound resident service and retention programs has grown.  Over the years, taking steps to minimize controllable turnover has, more and more, been seen to begin with performing resident surveys.  By going straight to the “source” for critical input, firms can use the findings as the foundation for developing sound management and retention plans – that address the true property issues impacting resident satisfaction.  When properly done, these efforts obviously can have a dramatic impact on the bottom line.

A very important point, however, is to not just track satisfaction – but to make sure it is tracked accurately.  And that brings us to the critical issue of how confident you can be with the survey results.  How confident you can be is a function of the reliability of the data – and reliability is directly a function of the “methodology”, or type of survey, used.  Years and years of research have proven that reliability varies depending on which method is used, and the impacts of each are known.

Given the need to increase the effectiveness of resident retention programs, it has become more critical that customer service oriented firms obtain reliable data to assist with making very important property, staffing and program decisions.  Therein, as they say, lies the rub.  Research has shown that the survey method traditionally used in property management – the mailed survey – has serious reliability flaws.  But, not to worry – there is “a better mousetrap”.  Very strong arguments can be made for the significant benefits to utilizing a new approach – random telephone surveys.  Research shows that there are significant differences in reliability between traditional mailed surveys and random telesurveys.

Lee Papa, President of Consultant Providers and an independent industry expert in performance measurement systems, agrees.  Papa states "There are many different and effective performance measurement tools being utilized today for measuring customer satisfaction, but in high-relationship service organizations such as multi-family property management, the trend is moving towards a more personal approach to information gathering.  Instead of doing written resident satisfaction surveys, the shift is leaning toward surveying by phone.”

Do residents object to getting calls from telesurveyors?

The answer is 180 degrees from what you would expect.  At SatisFacts our research shows that over 90% of the time that we reach a resident on the phone – they complete the survey!  That’s at least three times the response rate for mailed surveys.  Why?  There are several main reasons. 

First, the survey is pre-marketed, so as to create awareness of the upcoming survey and build the desire to be surveyed. 

Second, this is not a “sales call” – it’s a customer service call.  We live in a customer service driven society, where most of our target markets expect quality service – and most targets like that their community cares enough to call to see what they think.

Lastly, unlike mailed surveys, completing a survey on the phone takes very little effort!

Harry Beckwith, in his book entitled Selling The Invisible – A Field Guide to Modern Marketing, supports the participation claim and how it relates to written versus oral surveys.  He states, “Typically, 40% of people will respond to a written survey.  (The response can fall well below that.)  In oral surveys, you often can get almost 100 percent response.”

How important is statistical reliability?

When you use the findings from annual resident satisfaction surveys to make important decisions on personnel and capital expenditures – it becomes clear as to how critical it is that you use reliable statistics.  Having confidence in how valid your results are should be a primary objective in the satisfaction research you do.  Mailed surveys, unfortunately, do not provide a great deal of reliability.  Why?  The answer lies in the exciting world – OK, now get your cup of coffee ready – of statistical theory!

What is “margin of error” and why is it important?

The issue of reliability really boils down to what is called “margin of error” (i.e., “Candidate A is leading B by 55% to 45%, +/-5%”).  Margin of error essentially tells you how confident you can be with the statistics.

The larger the margin, the less reliable the statistics are and the less confident you can be with the figures.  To obtain sound statistics to help you make decisions, you need to control the margin of error.

How confident can you be with results? 

The answer is a function of factors covered later.  In the language of statistics, the “confidence interval”, or margin of error, is a function of these factors and is what provides the answer.  The margin measures how precise an estimate from a survey sample approximates the population value.

Suppose we have a bag of 200 balloons, and we want to find the proportion of red ones. We could go through all the balloons, but a more efficient method would be to take a sample of them from random areas of the bag and estimate the proportion.  For instance, in a random sample of 30 balloons, say we find that 50% are red ones. Because the resulting "50%" is based on only some of the balloons, it is subject to some uncertainty or error.  The confidence interval states the margin of error.  In this example, there is a 95% chance that the result falls within +/- 6.44% of the sample estimate of 50%.

The same applies to a random telesurvey for a 200 unit property.   If we want 95% confidence in the results, and complete surveys with 30 (15%) of the 200 residents, the resulting margin of error is the same as above.  For example, if a question has an average score of 4.2 on a particular question (on a 1-5 scale, where scores below a 3.5 are a “red flag”, 3.5 to 4.0 is an average score, and above 4.0 is superior), then you can say that there is a 95% chance that the average is +/-6.44%, or falls within the reasonable range of 3.93 (high above average score) to 4.47 (a superior score).  If the margin was, for example, a very high 30% – which can be the case with mailed

surveys – then your statistics are legitimately meaningless as all you can say is that there is a 95% chance that the average rating falls within the range of 2.94 (a “warning” score) to 5.00 (a perfect score).

What impacts the margin of error?  Why are random phone surveys so much more reliable?

A major benefit random telesurveys like SatisFacts offer over mailed surveys has to do with statistical reliability…with how much confidence can you apply your results to the total population of your clients.  Mailed surveys do not provide statistics that have a great degree of reliability for a number of reasons.  The margin of error, and thus reliability, as well as the volume and depth of information obtained in a survey are impacted by several primary factors, which are covered below.

Random Selection

Random selection insures a cross section of opinions.  When respondents are not randomly selected, as is the case with mailed surveys, you have no control over who responds and results are not representative of the whole as dissatisfied residents are most likely to take the time to complete the written survey.  This translates into negatively “skewed” results.

William M. Trochim (Cornell University), in his web-based textbook Research Methods Knowledge Base, brings home one of the bottom line issues regarding the importance of random selection in surveying – and this helps explain why the claim can be made that the non-random nature of mailed survey responses seriously impacts the reliability of the findings. 

Trochim states, “The difference between non-probability and probability sampling is that non-probability sampling does not involve random selection and probability sampling does.  Does that mean that non-probability samples aren't representative of the population?  Not necessarily. But it does mean that non-probability samples cannot depend upon the rationale of probability theory.  At least with a probabilistic sample, we know the odds or probability that we have represented the population well.  We are able to estimate confidence intervals for the statistic.  With non-probability samples, we may or may not represent the population well, and it will often be hard for us to know how well we've done so.  In general, researchers prefer probabilistic or random sampling methods over non-probabilistic ones, and consider them to be more accurate and rigorous.”

Response Rate

The higher the response rate, the better chance a cross section of opinions is obtained – and there is a significant difference between the response rate to different methods.  Mailed surveys average in the 15% - 30% range.

In other words, the majority of residents are not willing to take the time and effort to complete and mail a survey.  The results for random telesurveys are quite different.  SatisFacts, for example, completes surveys with 90% of the randomly selected residents reached on the phone.  The main reasons for our high completion rate were covered earlier.  Our results mirror Beckwith’s earlier statement that oral surveys can reach nearly a 100% response.

Sample Population

The similarity of the respondents in the population to be surveyed can impact the margin of error and results – although the amount of impact varies depending on what survey type is used.  Again, mailed surveys tend to get more responses from dissatisfied residents.  Thus, with this approach you tend to not receive responses from a cross section of opinions.  And the more diverse the demographics and psychographics are, the more important it is to get a cross section of opinions.

Non-Response Bias

The low response rates and non-random nature of mailed surveys translates into an inability to obtain results from a cross section and to be able to say that results apply to the entire “population” (of the property’s residents).  Research proves that it is invalid to claim that the people who did not respond (“non-respondents”) to a mailed survey have the same opinions and attitudes as the ones who did.  Not being able to apply the results to everyone who lives at a community, for example, is called “non-response bias”.

Gilbert A. Churchill Jr., in his book, Marketing Research-Methodological Foundations, provides strong support for this claim.  The author writes that a primary source for “non-observation bias is the error due to non-response, which represents a failure to obtain information from some elements of the population that were selected and designated for the sample…Study after study has indicated that the assumption that those who did not respond were in fact equal to those who did is RISKY…Some of the most extensive research contrasting those willing to participate with those who refuse their cooperation has been conducted with respect to mail surveys.

The refusal problem is particularly acute with mail surveys because refusals are the main source of non-response bias.”  Thus mailed surveys tend to be responded to by dissatisfied customers, the results are not representative, are skewed to the negative end – and you cannot take the leap of faith and say the results apply to the whole sample.

In-House Versus Third Party Purveying

Whether the surveys are done in-house or contracted out to a third party can impact results in several ways.  Both the willingness to be surveyed, candor and the amount of information obtained increases when respondents are contacted by an outside party.  The quality and quantity of information can expand even more when a third party survey offers the opportunity for anonymity.

Regarding doing surveys in-house versus contracted out, Beckwith agrees, stating “Make it so your clients can talk behind your back, and that you can learn what they’re saying…Your clients will give far more candid answers. Have a third party do your surveys.”  And Consultant Providers’ Lee Papa, who served as Director of Benchmarking Services for a major national satisfaction research firm agrees, stating ”It is crucial to utilize an independent third party in any performance measurement system for validity and reliability of the data."

What are additional benefits of telesurveys?

There are numerous other benefits telesurveys offer over mailed surveys. Harry Beckwith shares some very important insights, stating “phone surveys usually produce more revealing answers…on the phone, people will open up and reveal the information you need…Time after time, oral surveys work better.”

Why?

Beckwith continues by stating, “For one thing it’s easier to talk than write.  So people say more in oral surveys than they write on written ones…An experienced interviewer can be more conversational and relaxed with subjects and can go outside the script to probe…All of this helps produce more information…An oral interviewer makes a personal contact on your behalf.  This shows a greater interest…and conveys a stronger service message about your company…Finally, a person’s voice conveys feelings that her written words often obscure.”

The ability to clarify and probe are additional significant telesurvey benefits.  Beckwith adds “But no written survey can clarify every word or use words that need no clarification, and no researcher can accurately interpret each word a person being surveyed writes down…When you conduct written surveys, you cannot correct this problem; too often, you cannot even see it.  But when you conduct oral surveys, you can clarify your questions and ask people to clarify their answers…beware of written surveys.”  While you can’t read between the lines of what a client means when they check off a score of “3” on a written survey, a telesurveyor can react to the resident’s tone and comments – “You said ‘I guess I’ll rate that, maybe, a 3’.  What do you mean by that?” – and report the resident’s reply.

Another advantage telesurveys offer are dramatically shorter turnaround times.  While it typically takes at least 2 months to prepare and mail a survey, wait until an adequate number of responses are received, complete data entry and then generate the report –telesurveys reports are typically turned around in a mere 3 to 4 weeks!

The primary benefit mailed surveys offer is cost.  However, while the cost is typically less than several hundred dollars lower than telesurveys for a 200-300 unit property, there is one significant issue: the cost for mailed surveys (mail out postage and envelopes, return postage and envelopes, survey printing, stuffing) is fixed regardless of the response rate!  And if a follow-up mailing or completion incentives are required, the cost obviously increases.  Plus, this figure does not include the soft cost involved if your staff has to enter the data into a database or spreadsheet, type up all of the comments, generate the report, graphs and summary.  With telesurveys there is a guaranteed response rate and you only pay for surveys completed.  To put the discussion in perspective, the typical annual telesurvey cost for a 200 unit property is no more than $600 – we’re not talking about a budget buster!

Random telesurveys generate more positive results Mailed survey results tend to be more negative due to the nature of who typically takes the time to respond.  Common sense tells you people with an issue are going to be most likely to respond – and this explains why mailed survey results “skew negative”.  A perfect example comes from one of our clients.

This “resident satisfaction and retention focused” firm used a third party mailed survey service for several years, but were ready to stop doing surveys because of how disappointing the results were.  While they knew they were not perfect, they were shocked by how low their scores were.

Making the switch from mailed to random phone surveys made sense to them for several reasons.  First, the personal touch fit in nicely with their approach to service.  Second, they were impressed with our 90% completion rate.  Third, they understood that randomly selecting residents meant a cross section of residents would be surveyed and this would thus insure that an accurate picture was painted.  Due to this they knew that any potential issues would be put into a proper perspective - if there was an issue, it would show up on the radar screen but would not be “blown out of proportion”.  Fourth, they wanted to be able to use the findings in their properties’ leasing, marketing and advertising efforts.  Lastly, they wanted to be able to show potential investors how good they really were and the mailed survey results were too low to share!

The prior mailed survey’s overall portfolio score was barely average, coming in just above 3.5 on a 5 point scale.  A 3.5 is like a 70% rating – in other words, a low average test score.  Before we conducted the portfolio surveys, we surveyed their Resident Managers to see what they expected their average scores would be.  The result was clearly a function of the Managers’ expectations based on past survey results – given how low results had been with mailed surveys, the combined overall score Managers expected was a 3.65, or a 73% score.

Using our random telesurveys the portfolio came in at an above average score of 3.97, well above our database’s national average score of 3.79, nearly 10% higher than what the managers expected and about 15% higher than the past mailed surveys!

Summary

Random telesurveys offer distinct and statistically proven advantages over mailed surveys, and paint a much more accurate picture of issues and attitudes.  A cross section of views is possible due to taking control of who responds by randomly sampling residents, and because of very high response/completion rates.  This permits for a low non-response bias, high levels of confidence and low margins of error.  And there are a number of qualitative benefits of having personal contact, from the ability to clarify and probe, to the strong customer service orientation it communicates. Beckwith concludes the survey section of his book, Selling The Invisible – A Field Guide to Modern Marketing, in very certain terms: “For a dozen reasons, conduct oral surveys, not written ones.”