Measuring Service Quality

As quality practitioners, we’re accustomed to measuring the physical attributes of a product: dimensions, angles, power, hardness, tensile strength, color, and many other characteristics. Getting a handle on services can be more difficult. Often there are no physical attributes to measure, or they don’t clearly affect the essential nature of the service. We have to think about what really matters to the customer about the service. Although this is the case with both goods and services, it takes on special significance with a service. Let’s examine the nature of services and discuss the most effective ways of gauging their effectiveness.

The first thing to keep in mind about the service sector is that it is completely different from manufacturing. The things that you take for granted in manufacturing simply don’t exist in many service situations. Consider:

• You control the environment. Even if you subcontract manufacturing to an outside firm, you can still stipulate the environmental conditions. With a service, the environment is often dictated or strongly influenced by the customer. You must adapt to these environmental issues, which can be a huge challenge.

• The customer usually isn’t present. Sure, the customer’s presence is felt through specifications, tolerances, and product requirements, but the customer isn’t standing in front of you or peering over your shoulder. With a service, on the other hand, the customer is front and center. He or she is right there, throwing curve balls and changing requirements midstream.


For these reasons, service provision is radically different from manufacturing. Output measurements that are applied in manufacturing make no sense in a service situation because the customer has such a strong influence over our environment. Think about these traditional measures of output:

• Number of customers processed per hour
• Minutes spent on each call
• Reports generated per day
• Average time per repair
• Rooms cleaned per shift

I’ve seen all of these measures applied aggressively in service environments, and all of them frequently backfired because you can meet output objectives and still generate very low customer satisfaction. That’s why the best way to understand service effectiveness is through customer perceptions.

Customer perceptions are key

Customer perceptions are critical in any product context. In the world of service delivery, they’re especially important due to the personal and interactive nature of services. You may satisfy every stated requirement and still fail to satisfy the customer in a profound way. The ground is shifting as the service is performed, and what you think was perfect may be far from satisfactory. That’s why you must specifically ask your customer what he or she thinks about your services. Don’t provide a long survey that probes every aspect of the service experience; just start with two simple questions: “How satisfied are you with the quality of our services?” and “How likely are you to recommend our services to a colleague?”

These two questions apply to nearly any service situation and industry. The first question addresses basic satisfaction, essentially asking if the services met all requirements. The second question takes this a step further and addresses true commitment: Do you feel strongly enough to recommend our services to somebody else? These represent two different places on the same continuum (as seen in figure 1), and both arenecessary for long-term success.



It’s worth noting that satisfaction falls only in the middle range of the continuum. The blunt reality of business is that basic customer satisfaction is no longer adequate for businesses to remain successful. Basic satisfaction simply means that they might use your services in the future--unless a better offer comes up. Satisfaction is little more than the absence of dissatisfaction, and there’s no glory in just squeaking by. Satisfaction is a reasonable starting point, but the ultimate goal is the kind of commitment that results in customers telling their friends and colleagues about your organization and recommending your services. That’s what you should be striving for.

The two survey questions include a four-point response scale. Some data gurus might question whether this provides much constructive information. Keep in mind, however, that people aren’t reliable measuring instruments. With subjective judgments, four or five degrees of resolution are about as precise as you can expect. Combine the preceding questions with the following two open-ended questions and you’ll have a very useful tool for measuring your services:

• How can we improve our existing services? This is one of the simplest yet most effective questions ever conceived. It strikes at the heart of quality: improvement. It gives customers control of the dialogue, and they can do with it what they will. The responses will provide a clear path to making improvements that your customers value.

• What services would you like to see us offer in the future? Innovation is the key to long-term survival, and this question enlists your customers’ help in making you an innovator. The range of responses is limited only by your customers’ imaginations.

In the case of the open-ended questions, the results can be sorted into similar categories. These can then be plotted on a Pareto diagram to provide guidance on the actions that should be taken. Many quality practitioners bristle at open-ended questions because they don’t produce data in a traditional sense. The responses can be converted to data, however, without much difficulty. Even more important, the results point the way to exactly the improvements and innovations that your customers desire.

You now have a dynamic tool that will take less than a minute of somebody’s time. The scaled questions probe two timeless issues--satisfaction and commitment--and produce solid data that can be tracked, while the open-ended questions provide direction for your improvement efforts. Together you have one of the most streamlined and effective service surveys imaginable.

When to capture perceptions

Ask customers for their feedback as soon as the effects of the service are felt. This might be immediately after performing the service or six months later; it all depends on the type of product you’re addressing and the sorts of contractual obligations that were made with the customer. Consider these service scenarios:

• Restaurant. Feedback could be provided immediately following the experience, or certainly within a day or two of it.

• Appliance repair. Feedback could be provided immediately on certain aspects of the service, but it would probably take weeks to know how effective the repair was. Most appliance-repair companies warranty their repairs for a certain length of time, so the feedback horizon could follow a similar time frame.

• Management consulting. Complex consulting projects that aim to increase a company’s profitability and competitiveness might take up to a year to evaluate. Asking for feedback any sooner would be premature.

These three examples illustrate a range of time frames for feedback, from immediately after the service to a year later. Each organization must decide for itself when the effects of its services can be determined and, thus, when it’s appropriate to solicit feedback.

Once you’ve determined when to capture feedback, the next logical question is how to do it. Yes, you already have the tool, but how exactly will it be administered? Your choices are many: in person or by telephone, e-mail, web site, fax, postal mail, or text message. The chosen method should reflect the most convenient process for your customers. In general, try not to add another communication burden to your customers. If you have frequent face-to-face contact with them, use these interactions for getting their feedback. This also goes for existing communications via telephone and e-mail. If it’s already happening, use it. Providing feedback will only add a minute of extra time, and that’s an investment that most customers are glad to make.

Objective measures

Everything we’ve discussed so far is related to subjective measures of service quality. In other words, we’re asking someone’s opinion of how we performed. They probably don’t have data to back up their opinions, and they may not even be able to provide specific examples. These opinions are the basis for making buying decisions, however, so they’re valuable to you as a service supplier.

Besides subjective performance measures, there are also many objective measures that can be applied to your services. You need only look as far as your service guarantees and contracts to find some effective metrics. Nearly every service provider commits to performing its service within a certain time frame. This naturally gives rise to the question: Was the service performed on time? No opinions are necessary here; you either met your commitments or you didn’t. The data can easily be gathered, charted, and analyzed by your own organization. Hard data provide an excellent counterpoint to customer feedback, and they usually substantiate the themes revealed through customer feedback. When data don’t support these themes, it’s useful to explore the reason for the gaps; e.g., “Our customers think we’re always late, yet our data show this isn’t the case. What’s causing this difference in perceptions?” When there’s a difference of this sort, one of two things typically must happen:

1. The data-collection method must be changed to better match what the customer experiences.

2. The customer must be educated at the performance level. Sometimes providing objective data can shape people’s perceptions, and there’s nothing wrong with doing this.

So, what sorts of measures are helpful in managing service quality? Here are some of the most common:

• On-time delivery. The scheduled date and time is agreed upon between the customer and services provider, and deviations from this schedule can cause serious problems. On-time delivery is an excellent measure that’s usually easy to track.

• Responsiveness. This means your ability to respond to the customer within a reasonable amount of time. The response could be related to a question, problem, quote, inquiry, or order change. Organizations that cultivate “customer intimacy” are usually concerned about how responsive they are.

• Effectiveness. All services are supposed to accomplish something: provide information, repair an appliance, process a transaction, or develop a program, among others. If you’re able to determine if your service was effective, then this is an important measure. Keep in mind that I’m talking about an objective measure of effectiveness, not the customer’s perception of effectiveness.

• Availability. Services that are up and running must be concerned with availability. Examples include utilities providing water, electricity, gas, telephone, or other resources exactly when they’re needed. Being down for a few hours can cause millions of dollars in losses and huge claims.

• Audit results. Processes that provide a service can usually be audited. Either through in-person observation or by examining records, an audit can reveal whether the service was performed as planned. Ideally, conformity with the plan would mean that the service is effective, though this isn’t always the case.

• Cost control. This means adhering to established budgets and spending plans while meeting other service objectives. Notice I didn’t say “cost reduction,” which often is used to justify a reduction in service quality.

In summary, a two-pronged approach is the most effective way to measure service quality. Gauge service effectiveness through customer perceptions and through objective data, and remember that measures are worthless unless you take action.

Five Keys to Successful Internal Auditing

Internal auditing is one of the most routine improvement tools available to organizations. In fact, it’s so ordinary that auditors sometimes forget the underlying principles of auditing. Auditors must be periodically reminded of these underlying truths or the entire audit process can begin to backfire. Keep these in mind as you audit and you’ll nearly always be successful.

Principle 1: The customer of the internal audit is the one being audited

That’s right; the people you’re auditing are your customers. Internal auditing is a service you perform to help make your organization more successful and identify problems before they spiral out of control. The quality of your product depends on how well the audit is planned, the type of training provided to auditors, the level of engagement of top management and the way auditors behave during the audit, among other factors. You must conduct the audit with the same level of professionalism and diplomacy as if you were being paid by an outside party.

Little things that indicate the auditors have forgotten who the customer is include:

*** Treating the audit as a “Gotcha!” exercise. Auditors should never express satisfaction or glee when nonconformities are found. Focus on the facts and keep things as impersonal as possible. Failures revealed by the audit are opportunities for the future.

*** Failing to adjust the audit plan to meet the needs of the auditee. Unexpected events can occur during an audit: accidents happen, lines shut down, rush orders must be processed. The audit plan should be flexible enough to handle changes on the fly.

*** Holding surprises until the very end. The auditee should be apprised of audit results throughout the entire process. Don’t make the mistake of holding a “bombshell” until the closing meeting for maximum impact. Auditor should communicate their concern, along with all supporting evidence, when they think they might have found a problem.

*** Focusing on insignificant details at the expense of critical issues. Auditing is a detailed activity, but don’t forget to examine the effectiveness of the overall system. When faced with an issue, auditors should ask themselves, “What effect does this really have on the organization and its customers?” The answer will usually indicate if the issue is one worth delving into in great detail.


Principle 2: Planning is the key to success

Audits don’t create improvements by accident. It takes a great deal of planning and coordination. I’ve often said that a well planned audit almost runs itself. On the other hand, a poorly planned audit runs itself into the ground, and planning often gets shortchanged in the rush to get audits done.

Audit planning involves a significant amount of dialogue between the auditors and auditees. It’s a dynamic process that begins well in advance of the audit itself. Planning typically provides details around the following issues:
*** Date: When will the audit take place?
*** Location: What’s the audit’s location?
*** Scope: What are the official boundaries of the audit?
*** Objective: What is the point of performing the audit?
*** Auditors: Who will perform the audit?
*** Areas to be audited: What functions, departments or processes will be evaluated during the audit? Sometimes this is clear from the scope, but often not.
*** Topics to be audited: What subjects will be audited in the given departments? Should the auditee expect questions about document control or management commitment? This not only gives the auditee a heads-up, but it also helps guide the auditors.
*** Timing of the audit: When exactly will each department be audited? When will the opening and closing meetings take place?

The audit plan may also address other issues, but the ones mentioned above are the most common. The purpose of the audit plan is two-fold: To help the auditors understand exactly what they’ll be doing during the audit and to allow the auditees to know what to expect. It isn’t uncommon for the auditee to propose changes to the audit plan, usually minor alterations in the timing (“Instead of auditing sales at 9 a.m., can you come at 10 a.m.? We already have something scheduled for 9 a.m.”). Changes of this sort are entirely reasonable and should be accommodated to the extent possible. Remember, the customer of the audit is the auditee.

The audit plan is documented as concisely and clearly as possible. The exact format is usually dictated by the magnitude of the audit. A plan for an audit of an hour or two could take the form of an e-mail. A plan for a full day or multiday audit will often take the form of a matrix, indicating hour-by-hour blocks of activities. Whatever the format, the plan should be communicated far enough in advance of the audit for all parties to digest it and understand its effect on operations.

Principle 3: Opinions never constitute nonconformities

Everybody has opinions. As people become wiser and more experienced, they tend to develop even more opinions. Many auditors consider themselves to be wise and experienced, meaning they have loads of opinions. Sometimes these opinions become the basis for nonconformities, which is a huge mistake. Facts are the only legitimate basis for nonconformities. Opinions have no role in the process.
A child could write a good nonconformity. The problem is that children don’t write them, wise and experienced auditors do. Consider the following:

- The company committed itself to doing XYZ. The commitment is a fact, evidenced by its presence in a procedure, plan, policy, specification, contract, work instruction, standard or statement.

- The company failed to do XYZ. The failure is a fact, based on evidence such as records, observations, documents or interviews.

No opinions are present in the nonconformity, just cold, hard facts. It’s hard to argue with facts. It also makes the audit go much smoother. Sure, facts may remove a degree of creativity that auditors exercised, but creativity is better expressed in other ways.

Nonconformities aren’t the only kind of audit findings. Because the audit is a balanced process, positives are also highlighted. These may be recorded individually, summarized in an audit report, or presented orally during the closing meeting. Every organization will have at least one or two positives that can be recognized. The auditors just have to remember to look for these in the course of their auditing.

Some organizations also include another category of finding called observations, remarks, comments, opportunities, recommendations, or any number of other names. These fall into a gray area that doesn’t quite constitute nonconformity, but is still an issue worthy of investigation. Sometimes these will include specific recommendations for taking action based on past experience, established best practices or regulatory requirements. These types of findings give auditors a chance to express opinions. Audits are a great place for benchmarking and sharing best practices, as long as all parties to the audit understand and agree to how this will happen.

Principle #4: Don’t burn out your auditors

It’s human nature to utilize your best resources. If you have a patch of ground that produces great tomatoes, it’s tempting to keep planting tomatoes there, year after year. The only problem is that the soil eventually becomes exhausted. This is the case with an internal audit program, too. Experienced and well-trained auditors produce effective results, so they frequently get called on to perform audits. As a result, the organization fails to develop new auditors, and they end up with no auditors when the experienced auditors run out of gas and scream, “No more!”

One of the best strategies is to make each auditor’s ‘tour of duty’ a year and a half. Annually, train a new group of auditors, and then use the remaining six months for the experienced auditors to mentor the new group. Schedule auditors in teams of two, one new auditor and one experienced. That way, the new auditors get the benefit of observing the seasoned auditors in action, and the seasoned auditors can learn from the new perspectives and unburdened approaches employed by the new auditors.

After a few years of rotating in new groups of auditors, you’ll have utilized a significant chunk of your employees. The benefits of doing this are clear:
 Broad exposure of personnel to other functions in the organizations
 Deeper understanding of the management system and its processes
 Stronger communication and analysis skills, as a result of auditing experience
 Varied perspectives and viewpoints that come from using a wide range of personnel as auditors
 Less likelihood that the audit process will fall victim to groupthink, which happens when the same people always involved
 You will an informal ‘alumni association’ of ex-auditors who can be called on to perform audits periodically if you get into a pinch

Take the time and effort to develop new teams of auditors on a regular basis. You’ll find that the overall effectiveness of your audit process will increase significantly over time.

Principle #5: Audits should focus on critical success factors

Critical success factors are the things that keep you in business and help build customer loyalty. They will be different from industry to industry and from company to company, but there are certain critical success factors that apply to nearly all organizations. The internal audit should probe these areas in detail and dedicate significant effort to analyzing their effectiveness:

 Management review: This is the top management reviewing performance, analyzing data, making decisions, and initiating actions for improvement. It is one of the most important functions in your organization, so auditors will need to apply some effort to examining it.

 Corrective and preventive actions: Few issues have as much bearing on an organization’s success as problem solving and problem prevention. These processes should be scrutinized nearly every time an internal audit is performed.

 Customer satisfaction: The primary reason that organizations exist is to satisfy their customers. How well an organization listens to its customers and takes action on what it learns is definitely a critical success factor.

 Internal audits: If internal auditing wasn’t important, you wouldn’t be reading these words right now. Auditing is one of your primary processes for evaluating process effectiveness and driving improvements, so you will definitely want to audit it on a regular basis. Some people find the notion of auditing the audit process unusual, but it must be examined just like any other key function.

 Product realization: Whether your organization produces a good or service, it certainly has product realization. It could involve manufacturing activities (e.g., cutting, stamping, welding, and assembly) or service activities (e.g., scheduling, repair, trouble shooting, instruction). No matter what kind of activities are in place, this is your core transformation process that has the most direct impact on your customers.

Decide which processes in your organization are the most important to its success, and make sure these processes are audited in-depth and often. The results of the audit will speak for themselves.

Don't Survey Your Customers!

Scaled customer surveys are among the most widely used tools in business. Unfortunately, they're also some of the worst. There's nothing evil about surveys, but they can turn an inherently simple task, such as gathering customer feedback, into something complex and unwieldy. When that happens, there's a good chance it won't satisfy its original purpose, which in this case is making improvements. Why exactly are surveys the wrong tool for most organizations? Let's explore the reasons and then consider an alternative approach that's far more appropriate.
Surveys don't produce timely data

Most traditional customer surveys are sent out periodically to a sampling of an organization's customers, typically once or twice a year. This is a manageable frequency from an administrative standpoint because implementing a survey requires a significant dedication of time and effort. The downside is that by the time the organization receives the feedback, the information is at least six months old and usually much older. The information is almost worthless because customers have already acted on their perceptions before the organization has had a chance to do so. Customers don't wait around to tell you what they're going to do before they do it. If you're not tuned into your customers on a regular basis, you'll never know what hit you.

It makes more sense to gather customer feedback continually. Make the customer feedback process something that's always happening, not a grand event that occurs once or twice a year. This consumes far fewer resources, and it also ensures that the information is current. If you can't take action on customer perceptions within a few weeks of the perceptions being formed, there's a strong chance that you will lose your window of opportunity.

Too many questions

Another downfall of most surveys is that they try to do too much. They probe the customer experience from every imaginable angle. Although admirable, this approach results in long, unwieldy surveys that most customers run away from as fast as they can. I have personally gotten into the habit of scrawling "TOO LONG" in huge block letters on these kinds of surveys. I'm providing feedback, but not exactly the kind expected. Most people don't even bother to provide this much; they simply toss the long surveys into the trash.

The key to successful customer feedback is to ask about the few aspects of the customer experience that matter the most. By asking about everything under the sun, you're establishing the expectation that you'll take action on everything, which is impossible. You're also telling your customer, "Your time isn't very valuable, so the imposition of this long and boring survey should be no problem for you." Focus on a few vital issues, and these obstacles go away. The dilemma is that most organizations don't know what the few vital issues are--thus the need for long surveys. Your organization must back up and get its arms around the things that really matter to your customers.

Difficult to design

If you like defusing explosives, you'll love creating surveys. They include so many failure modes that they're nearly impossible to design correctly unless you do it for a living. Why are they so hard? Let's examine two of their more challenging aspects: questions and scales.

Most surveys comprise a series of questions or statements, followed by a response scale. The response scale usually represents degrees of satisfaction (e.g., "very satisfied," "satisfied," "neutral," etc.) or degrees of agreement (e.g., "strongly agree," "agree," "neutral," etc.). Both of these scales present huge challenges. Most people don't have the writing skills to craft clear, unambiguous survey questions. The result is that the questions don't accurately reflect the attribute that's being queried. In the spirit of getting the job done, customers will often take a guess at what the questions really mean. Like all guessing games, sometimes they'll be right and other times wrong. At best, your data will be 50-percent valid--not a very good percentage.

In the unlikely event that the survey questions are clear, there's still the obstacle of designing a logical response scale. This would seem to be an easy task, but it's extremely complex. Typical errors I've observed are scales that aren't balanced, scales that are biased, scales that don't have equal intervals between the points, scales that don't match the question or statement, and scales that have too many degrees of resolution. If the scale is flawed, then the data that come from it are also flawed. Junk in, junk out, as the saying goes.

No direction for improvement

The fourth downfall of traditional surveys is that they don't provide much guidance for improvement. Sure, they provide data, but what actions are you going to take based on those data? For example, let's imagine that you've asked customers to rate the technical knowledge of your sales force. The average response is 3.4 on a five-point scale, roughly halfway between "neither good nor bad" and "good." What does this number really mean? Even more important, what are you going to do about it? The data help you produce fancy charts, but they probably won't steer you toward specific improvements.

If you aren't able to take action on survey data, their value as improvement tools is zero. Had you asked customers an open-ended question such as, "What do you think is our biggest customer service weakness?" you might have received some feedback that provided a clear path for improvement. In the world of customer perceptions, data don't always rule. Perceptions are by their nature qualitative and subjective, and the attempt to produce data from such a fuzzy source can be misleading. It's better simply to get actionable information than to attempt to turn human beings into precise measuring instruments. If you capture customer perceptions, analyze the trends and take action, you've accomplished a great deal.

So, what should you do?

I've made the case that a traditional customer survey using scaled responses probably isn't the best way for most organizations to capture feedback. If not a survey, what should you do? Here's what I recommend:

Step 1: Examine your existing customer interactions. Your interactions with your customers are limitless. These contacts are conducted via telephone, e-mail, mail, fax and in person. Because you already have numerous contacts with your customers, there's no reason to invent a new contact for the sake of collecting customer feedback. Make use of the connections you already have, and all parties will generally be much happier.

Step 2: Choose an interaction suited to collecting feedback. Not all customer contacts are created equally. Certain conditions should be met when you decide which contacts will be leveraged for collecting customer feedback. In general, the contact should be neutral, routine and candid. Here's what each of these mean:

• Neutral: The contact isn't related to an existing problem or complaint. Attempting to collect feedback when a customer already has a problem is obnoxious and counterproductive. Use an interaction that's neutral in tone and purpose, such as a query or order placement.

• Routine: The contact happens on a regular, routine basis. Feedback collected from routine interactions of this sort is likely to be fresher.

• Candid: The contact occurs between parties that trust one another and are willing to communicate freely. A candid relationship is key to collecting accurate and representative perceptions.

Step 3: Develop a tool that's matched to the customer interaction. Choosing the right tool for the job is critical in every endeavor, and that goes for collecting feedback, too. Once the organization has selected an appropriate customer interaction for collecting feedback, it must develop a tool that works in that context. This is a subjective task, and certain guidelines can assist in knowing what tools work best in different situations:

• Telephone contact: An unobtrusive tool that's conducted at the end of a routine telephone call. Brevity is critical with this kind of tool because most people are anxious to get off the phone once their business is completed.

• In-person visit: A tool that enables the company to see its product in use, just as the customer experiences the product or service. The tool should also enable different people to be queried, depending on the nature of the feedback sought.

• After service or consumption: A tool that enables the customer to conveniently provide "flash feedback." Make the return of this feedback seamless. If the customer
has to expend any effort to return the feedback, it probably won't be returned.

• E-mail: A live link within the body of the message that takes the customer to a simple and visually appealing evaluation of their experience. Make sure the link works fast and is compatible with a variety of Internet browsers and computer monitors.

Benchmark customer feedback tools with other organizations. There's no virtue in being original. Borrow good ideas and approaches as you see them. Hundreds, if not thousands, of examples exist for each of the tools described above. See what other people are doing and adapt the methods to your own needs.

Step 4: Focus on open-ended questions. If you want to grab the attention of your customers, ask them what they like and don't like. It's that simple. Asking simple, open-ended questions of this sort enables the customer to dictate the content of their feedback. What's important to them is what you'll hear about. This is exactly the sort of feedback you want. Trends in open-ended feedback will inform you on the issues that customers care most about, something that many organizations don't understand.

Open-ended feedback also provides a clear path to improvement. Numerical ratings can help you prioritize issues, but they don't tell you exactly what to do. Open-ended feedback can. When 75 percent of your customers answer the question, "What makes you most frustrated about being our customer?" in the same way, you know exactly what you need to do to improve. There's no ambiguity.

Open-ended feedback doesn't help you make fancy charts. But do you really need more fancy charts to cover the walls of the conference room? No, you need improved customer satisfaction and loyalty. Open-ended feedback will reveal exactly what actions lead to long-term success, which is much more important than fancy charts.
Here are my favorite open-ended questions. Three are about all you need. They will quickly point the way to improvements that matter to your customers.
• Do you have any problems with our products that you haven't told us about?
• Is there anything you think we do particularly well?
• What could we do in the future that would make your job easier?

Step 5: Act on your opportunities. Action is the most critical step of the entire process. It starts with identifying trouble areas. Problems that are revealed through feedback must be addressed immediately. This is the business equivalent of triage: Stop the bleeding and stabilize the patient. Let's hope you won't discover too many issues that require triage, but it's better to learn of these proactively while the customer is still your customer, and not a former customer.
After addressing the trouble areas, the organization must analyze the trends. Open-ended feedback follows the same rules as most traditional numerical data: It tends to clump into categories. Group the feedback into categories and apply Pareto analysis to the results. Your opportunity areas will quickly emerge. Input these opportunities into your corrective/preventive action system and track them to completion. Treat every improvement action as a mini project, with assigned tasks, responsibilities, time frames, resources and reviews. The more sunlight shines on your improvement action, the better it will be. In other words, communicate widely. The final communication about your improvement will be to your customer: "Here's what we've done based on your feedback." These may be the most important words you ever say--and you don't have to use a traditional customer survey to say them.

The 10 Biggest Quality Mistakes

One of the most effective ways to improve is to learn from other people’s mistakes. Experience has taught me that there are plenty of mistakes out there to learn from. The trick is to recognize them and understand what to do instead. Unfortunately, I keep seeing the same mistakes over and over. They aren’t mistakes because they violate a standard such as ISO 9001; they’re mistakes because they violate good sense. Let’s examine the 10 most common quality mistakes and see how they can be corrected.

1. Limiting quality objectives to traditional quality topics

The term “quality objective” is an unfortunate one. It introduces subjectivity (i.e., quality) into a subject that should be quite clear (objectives). A much better term would simply be “measurable objectives” because that requires less interpretation. The word “quality” clouds the issue and makes many people want to narrow the focus of what a quality objective can be.

The truth is that quality is reflected in everything an organization does, and a quality objective can be anything measurable that relates to the organization’s success. A quality objective might relate to finances, customer feedback, safety, efficiency, speed or innovation. All these attributes relate to quality in one way or another. When selecting quality objectives, organizations should examine what matters most to their success. Whether the resulting measure is tied to traditional quality control or quality assurance is irrelevant.

2. Holding infrequent management reviews

Management review is the process used by top management to analyze data, make decisions and take action. Ideally, it’s a preventive process because data should indicate threats before they blossom into full-blown problems. If top managers are unable to analyze data proactively and prevent problems, then they’re not doing their jobs. Holding management reviews once or twice a year ensures that actions taken won’t be preventive. Only through timely and frequent data review can actions be preventive. Once or twice a year won’t cut it.
Many people argue that their organizations already review data on a weekly and monthly basis. This means that management review is still after the fact; the decisions have already been made. Management review must be included in existing meetings. Instead of a twice-yearly dog-and-pony show, cover the inputs and outputs of management review as they occur naturally during existing meetings. After a month or two, you’ll have addressed all the required inputs and outputs. Using this approach, you’ll have information that’s timely and resulting actions that are preventive. Another advantage is that you dispense with the need for long, laborious management reviews. They happen in a smooth and effective manner that’s much more likely to drive improvements.

3. Sending out long, complex customer surveys

The days of the long and complicated customer survey are over. People don’t have time to complete them. Even when organizations design shorter surveys, the questions are often confusing and fraught with interpretation problems. The scales that accompany the questions are often unbalanced and illogical. As a result, organizations end up with a small amount of valid data. Better to have no data at all than data that could lead you in the wrong direction.

Instead of a survey, why not simply ask your customers what they like and dislike? Don’t limit their responses to survey topics. Let your customers dictate the content of their feedback in response to open-ended questions. Few are more powerful than the following: What do you like? What don’t you like? What would you like to see different in the future? Open-ended feedback is also much easier to understand and take action on. A customer- satisfaction index of 3.8 is hard to interpret. On the other hand, seven out of 10 customers telling you that your Web site is confusing is very easy to interpret.

4. Assuming everyone knows what “nonconforming” looks like

When I visit organizations, one of my favorite questions is, “Where do you put the nonconforming products?” Control of nonconforming products is one of the most basic kinds of controls, and it speaks volumes about the rest of the controls embraced by the organization. Unfortunately, where I often find nonconforming products is wherever someone decided to leave them. They’re not uniquely identified, either. In other words, nonconforming products are treated no differently than any other products. When I ask why this is happening, the most common answer I get is, “Because everyone knows that’s nonconforming.” No, they don’t. No matter how nonconforming a product is, there will be someone who won’t recognize it as such. The stories of bent, broken and blown-up products that somehow got used anyway aren’t urban legends. They’re true.

Smart organizations positively identify all nonconforming products, and really smart organizations segregate them to remove all chance of accidental use. Error-proof your control of the nonconforming product process so that nobody has to assume anything.

5. Failing to use the corrective action process

Corrective action is the systematic way to investigate problems, identify their causes and keep the problems from happening again. Nobody wants problems, but it’s essential to have a way of dealing with them when they come up. The more the corrective action process is used, the better the organization gets at addressing its risks and problems. That’s why I’m astounded when I hear about organizations that avoid using their corrective action processes. Of course, I always ask why they’re doing this, and I often get one of these answers:
• Corrective action isn’t effective for large problems.
• Corrective action isn’t effective for small problems.
• Nobody understands root cause.
• Our problem-solving tools are confusing.
• Our procedure requires too much paperwork.
• Corrective action takes too long.
• I hate our corrective action form.
• Top management frowns on corrective action because it means that someone screwed up.

These aren’t corrective action problems but problems with the organization’s approach to corrective action. An effective corrective action process is typically seamless, simple and intuitive. The whole point is to add a little structure to problem solving, not to create additional bureaucracy.

Here are some hints to make your corrective action process more user friendly and effective:
• Strip it down to the essentials. A corrective action must clearly describe the problem, how it’s caused, actions taken to remove the causes, results of actions taken and how the actions were effective. Only include additional elements when you can prove that they add value.
• Remove all jargon from the process. Strange words only discourage people from using the process.
• Don’t insist on a raft of signatures . It’s not necessary for half the management staff to sign off on every corrective action.
• Remove paper from the process as much as possible. Use electronic media to track corrective actions.
• Communicate corrective actions widely. When people see that corrective actions accomplish something, they’re much more likely to participate in the process.
• Provide problem-solving tools, but give people some discretion in their use . If your procedure requires a failure mode and effects analysis to be completed for every corrective action, it will probably discourage people from starting corrective actions.
• Use teams for corrective actions whenever possible. This gives people experience in the process and also increases the effectiveness of most solutions.

6. Applying document control only to official documents

Most organizations do a decent job of controlling “official” documents, the procedures and work instructions that form the core of the quality management system (QMS). These are often written, approved and issued according to very specific guidelines. What organizations don’t do very well is control unofficial documents, many of which are more important than the official ones. What am I talking about? Here are some examples that are often found in production areas:
• Post-it notes with special requirements written on them
• Memos that include procedural steps
• E-mails with customer specifications
• Photographs showing what a product should look like
• Drawings indicating how product components fit together
• Samples of product showing defect limits

These informal resources become documents when they’re shared for people to
use, and they’re some of the most important documents within an organization. They’re distributed and posted in a hurry--usually without control--because the information they communicate is considered critical. Nobody can quibble with the speed of distribution, but the lack of control guarantees problems later. I’ve seen 10-year-old memos posted in organizations that exhorted personnel to perform obsolete job steps. When documents aren’t controlled, mistakes and nonconformities are inevitable. Apply document control to all documents, and scrutinize your document control process to keep it streamlined and effective.

7. Focusing audits on petty, nonstrategic details

Auditing is the process of comparing actual operations against commitments the organization has made. It’s a simple, fact-driven process that can generate huge improvements. However, these can occur only if auditors focus on the right things. Too often, internal auditors become preoccupied with petty details and neglect the big issues. They’re uncomfortable examining the big, strategic issues. It’s much easier just to nitpick. Organizations rarely provide enough training and skill-building to their internal auditors, so it’s no wonder that they aren’t prepared to carry out their duties to the fullest.

A robust internal auditing process examines make-or-break issues. Here are just a few of the items that internal auditors should probe in detail:
• Customer satisfaction. Is the organization capturing and analyzing customer satisfaction data? How is it acting on the data? Do trends show improvements in customer satisfaction?
• Management review. Does management review happen as planned? Does the necessary information and data get reviewed? What actions result?
• Corrective action. Is corrective action applied to existing nonconformities? Is it timely? Does evidence indicate that causes are removed to prevent recurrence?
• Preventive action. Does the organization take preventive action based on data and information? Is it effective?
• Internal audits. Are audits scheduled and carried out based on status, importance and prior audit results? Do audit nonconformities become corrective actions? Is the entire scope of the management system audited?
• Objectives. Are objectives established and communicated? Do employees understand them?
• Control of nonconforming products. Are all nonconformities positively identified? Are dispositions carried out as required? Are trends analyzed for improvement?

There are, of course, many other important issues an audit process could examine. The point is that internal auditors should go after the items that really affect the organization’s success. Focusing on petty details serves no purpose but to confuse everyone about the purpose of audits.

8. Training some personnel, but not all

Most organizations provide significant training to hourly production personnel. Salaried and managerial personnel are often neglected, however. Why? Because there’s a perception that salaried workers don’t affect product conformity. This is a serious error.

All personnel must be included in the training process. Salaried and managerial personnel need more--not less--training because their decisions and actions have more lasting effects. When an hourly employee makes a mistake, it could cost money. When a top manager makes a mistake, it could put you out of business. Doesn’t it make sense to train these people? Do it early and often.

9. Doing anything just because an external auditor told you to

External auditors wield great influence. Their statements and judgments can have a lasting effect on the way an organization conducts business. This can be good or bad. Usually, it’s bad. Most external auditors working for a registrar are removed from the realities of running a business. They travel from organization to organization, gradually collecting paradigms about the way a QMS should be implemented, maintained and improved. These paradigms are sometimes reflected back to the organization in the form of recommendations or nonconformities.

In my travels to companies, I often ask people why they’re carrying out a process the way they are. I always raise this question when the process seems unwieldy or illogical. In a surprising number of cases, the answer will be, “Because the external auditor said we should do it that way.” What a waste. Do a reality check on the auditor’s recommendations. Never do anything just because an auditor would like it done that way. A certificate on the wall isn’t worth it.

10. Employing someone who only oversees the QMS

Having a person who does nothing but oversee the ISO 9001 (or any other) QMS is one of the worst ideas in the history of quality. Why? Because it guarantees two things:
First, the QMS coordinator will become isolated from the rest of the organization. Because the person does nothing but serve the QMS, he or she loses touch with why the organization exists in the first place. The system becomes paramount over the organization’s business concerns. Second, the QMS will become bloated and bureaucratic because it must expand to completely fill someone’s time. Procedures become more complicated, methods more cumbersome and the benefits more ambiguous.

A QMS is nothing more than a guiding structure of methods, and it shouldn’t take a huge dedication of time and effort to maintain. Yes, someone should keep the system on track, but that person should have other responsibilities as well. Pair the ISO 9001 coordinator job with other responsibilities that focus on understanding what the organization does, especially responsibilities related to the product, customers and improvement. If the QMS is so bureaucratic that it requires the time of an entire person (or, heaven forbid, an entire staff), then the system needs to be streamlined. An effective QMS should make an organization more competitive, not weigh it down.

ISO 9001:2008 – How it will Impact your Auditing

By Craig Cochran
(Originally published in The Auditor, Jan-Feb 2008)

Auditing is all about comparing evidence to requirements. That’s why changing requirements always grab the attention of auditors. As most of you know, ISO 9001 is undergoing a revision that is scheduled to be published in 2008. The new standard, ISO 9001:2008, includes mainly editorial amendments: re-arranged words but few significant changes. There are a few things that auditors should take note of, though. Let’s take a look at the Draft International Standard of ISO 9001:2008 and discuss how it might impact your auditing.

Statutory requirements

The word statutory will be inserted in a couple of places within the introductory part of the standard. This is primarily to make the language consistent with section 7.2.1—determination of requirements related to the product—which has always required that the organization determine statutory and regulatory requirements for its products. Even though the concept is not new, the requirement reminds all auditors that they need to understand the legal requirements of the products they’re auditing. Statutory and regulatory requirements are often taken for granted by auditors, with the assumption that the organization knows what applies to its products. This is not always the case. During the planning phase, auditors must research the statutory and regulatory requirements on the product produced by the organization, and make sure they able to effectively evaluate this.

Technically, there is a difference between statutory and regulatory requirements. It’s a very fine difference, though. We’ll cover it briefly for the sake of completeness. Statutes are laws. They say what you can and can’t do in broad terms. Regulations are usually specific guidelines published and enforced by regulatory bodies. The bottom line is that statutes and regulations are both enforced by authorities that can make your life difficult. Understand what statutes and regulations apply to your products, and make sure you’re able to meet them. Satisfaction of this clause is typically achieved in a two part manner:

• Developing a process for understanding and staying up-to-date with statutes and regulations
• Compiling an index or listing of statutes and regulations applicable to your products
  

These processes are not specifically required by ISO 9001, but they would represent an effective way of meeting the requirement.

It’s worth noting that statutes and regulations can come from the country in which you are based, and they can originate from countries in which you’re selling your products. Multinational organizations have to consider statutory and regulatory requirements everywhere they operate in the world. Understanding and staying current with statutes and regulations can become somebody’s full time job for companies that operate around the globe.

This clause is also very significant for companies that produce highly-regulated products. Examples of highly regulated products include:
* Drugs and pharmaceuticals
* Medical devices
* Food
* Aircrafts and aircraft parts
* Explosives and firearms

Management representative

ISO 9001:2008 will clarify who can act as the management representative. That role must be held by a member of the organization’s management. This makes 2 things clear:
The management representative is an employee of the organization (not a consultant)
The management representative is someone with the responsibility and authority to make decisions, assign resources, and get things done.

In the past, the decision of who was assigned as management representative was usually unchallenged by auditors. This will need to receive some new scrutiny by auditors to ensure that the assignment meets the full intent of ISO 9001:2008. Besides being a member of the organization’s management, there are 3 responsibilities that must be carried out by the management representative and which auditors must verify:

1. The management representative will ensure that the processes of the quality management system are established, implemented, and maintained. This is the project management aspect of being a management representative. Establishing, implementing, and maintaining a QMS requires that the management representative must coordinate many different efforts and continually sell the benefits of the system. It bears repeating that the management representative does not own the system, though. Everybody owns the management system, led by top management.

2. The management representative will report to top management on the effectiveness of the management system. This happens during management review, possibly the most important process of the entire standard. The management rep does not need to personally collect and present the data on effectiveness, but they make sure it happens. The most effective management reviews involve a wide range of organizational managers and influencers, with the management representative coordinating their input.

3. This basically means that the management representative must help promote a customer focus throughout the organization. There is nothing more important to the organization’s success than the customer, and the management representative must continually remind everyone of this fact. Promotion of awareness can be accomplished in many ways, and here are a few simple ways that come to mind:
* Posting data on customer feedback trends
* Publishing product specifications
* Holding meetings that address customer issues
* Serving as a liaison between the organization and the customer
* Distributing memos and emails that clarify customer requirements

Competence

The competence and training requirements of ISO 9001 have long been some of the most confusing. The reason is that the standard specifies some broad requirements and gives the organization total discretion for how they will be applied. This discretion has been tightened a bit through the requirement that the organization “ensure the necessary competence has been achieved.” This replaces the previous requirements for evaluating the effectiveness of training.

Ensuring that competence has been achieved can take place in a number of manners, but the most obvious is a demonstration of the newly developed skills or abilities. This works especially well for competency building aimed at skills and training. “Okay, we’ve talked about the task, and we’ve demonstrated how it should be performed. Now you give it a try.” If the trainee is able to effectively perform the task over the period of observation, then they could be reasonably considered competent. Keep in mind that the period of observation could be an hour, day, week, or month. It all depends on the complexity of the skill being demonstrated. Most “on-the-job training” programs focus on this kind of evaluation. The trainee starts out as an apprentice and then gradually begins performing many of the tasks themselves. The training culminates in the trainee being able to demonstrate the full range of skills involved with the job.

The inspection of an employee's work or product can verify that competence has been achieved. For employees who produce a tangible good or deliver a service, this is often a reasonable indicator of whether training has had the desired effect. Many organizations already have existing systems for inspecting their products, and these systems can be channeled into the training program. But this will only work if the product's inspection is traceable back to individual employees.

Tests and examinations can be used to ensure that competence has been achieved, especially when the competence is related to knowledge and facts. Be aware that many individuals simply don't perform well on formal tests or examinations, regardless of the quality of the instruction and training materials, so this may not be an ideal gauge of effectiveness. Another drawback is that tests are heavy on administration, requiring someone to spend a great deal of time creating the tests, making sure that all learning objectives are addressed, creating answer keys, creating a grading scale, taking time to grade the tests, dealing with test anxiety and disappointment, and so on. Tests and examinations do have the advantage of resulting in a numerical score, which is easy to quantify and track over time.

Finally, some organizations use performance reviews to draw judgments on whether employees have achieved competency. Most organizations already use performance reviews of some sort. As long as a logical connection can be made between the training and the job performance, the system will work. One caution, however: Make sure to separate the record of performance review from the record of training effectiveness evaluation, as every organization seeking to keep or gain ISO 9001 registration will be required to provide evidence of the evaluation to its third-party auditor. Showing performance review records to outside parties will create ethical (as well as legal) problems, so you're far better off maintaining separate files.

Auditors will need to probe the issue of competence deeper than they have in the past. What was once accepted as meeting requirements may not meet the requirements of ISO 9001:2008.

Work environment

ISO 9001:2000 will add a note in order to clarify the scope of work environment. Notes are not auditable, of course, but they provide insights on how to interpret the requirements that are auditable. The note states that work environment includes physical, environmental, and other factors needed to produce the products in question. Consider the following examples:

Candy manufacturer: Raw materials are received into the facility and are immediately moved into a climate controlled storage area. The cleanliness of the storage area is immaculate, far different than most other warehouses. A weekly inspection is conducted to look for any evidence of pests. The raw materials are transported into manufacturing by personnel wearing white gloves and smocks, and all manufacturing is tightly controlled under Good Manufacturing Practices. All outside doors and windows are kept closed and the housekeeping is very strict; even the garbage cans are clean and spotless. Nobody with any kind of illness is permitted inside the facility, and no jewelry is permitted to be worn. Once manufacturing is complete, the finished product is stored in an area that is maintained at 40 degrees F, plus or minus 4 degrees, and the gauge used to monitor the temperature is calibrated. A weekly audit is conduced to evaluate the condition of finished product in inventory.

Insurance company: People are stationed at desks and perform work on computers and telephones. The office temperature is maintained at “typical office conditions,” which is usually a compromise between the women who like the office warmer and the men who like it cooler. Dress codes are enforced so personnel are not distracted in their work and to maintain a professional environment in the event customers visit. Personnel are not allowed to play music from their radios or computers, as the sound disturbs people in their work, even when played at low volume. Hot food items are restricted to the break rooms, as some employees were offended by the smells of certain foods that were consumed at desks. Personnel photographs are decorations are permitted in cubicles, but nothing that could constitute a threatening work environment are allowed. Everything about the office is maintained is a pleasant yet bland manner because this is the environment that was found to result in the highest productivity, lowest service defects, and fewest personnel problems.

Paper mill: The inside of the plant is very damp, and a half inch of water is on most of the floors. Additionally, the nature of the production process is very hot in some areas, and the ambient temperature in the summer can reach over 110 degrees. During winter months, temperatures in the warehouses are just a few degrees above freezing. For many years, the harsh environmental conditions were simply accepted as a given. The conditions didn’t negatively affect the product, so management felt no need to change anything. Recently it became clear that employees were becoming ill at a higher than normal rate, however. The increased illnesses affected the mill’s attendance rate, which in turn impacted its ability to produce paper on schedule. Work conditions are being improved now that the link between the environment and product conformity was recognized.

In all these cases, the work environment is focused on what is needed for the product in question. Sometimes organizations discover connections between the work environment and product conformity that they didn’t know existed, as in the paper mill example. ISO 9001 simply says that you will determine the environmental conditions that you require. Whatever you require is what you will be expected to provide and maintain.

Here are some typical controls related to specific work environment variables:

* Temperature and humidity: Gauges for monitoring, records of conditions, records of gauge calibration, investigation of affected product when conditions fail to meet environmental requirements

* Safety hazards: Identification of hazards, prioritization of risks, procedures for job safety, monitoring of compliance, records of monitoring, corrective action on accidents and near misses, regular meetings to discuss safety issues

* Lighting, noise, vibration: Specifications for characteristic, procedures for maintaining specifications, ongoing measurement of characteristic, records of measurement, calibration of gauges, records of calibration

* Housekeeping: Procedures for housekeeping, defined responsibilities, training of personnel, periodic audits of housekeeping, corrective action on nonconformities, signage to remind personnel of guidelines

* Personal hygiene & behavior: Documented policies for personnel, recurring training, monitoring by supervision, counseling for employees

ISO 9001 does not require documented procedures or records related to work environment, though it often makes sense to have such things. In cases where the organization establishes requirements for work environment, then the only way to verify that the environmental conditions were met would be through records. Documentation would also be required to consistently communicate the work environment requirements and controls.