Quality Control in Mediation
Introduction
In the work we do here at NMC, measuring the output of what we do is somewhat complex. We live in a world where quantifiable results are demanded at every turn to rank us so that: funding will flow; contracts will be continued; celebrations can happen; and course corrections can be applied. Over the years we have understood this need for measurement of our output well and have deployed a number of well-regarded tools to do just that. They have worked and are described in the balance of this quality control document.
Mediation programs, and mediators in particular, have the responsibility to ensure they are providing high quality dispute resolution services. Quality control efforts are thought not only to protect consumers but also to maintain the legitimacy of and public confidence in both the mediation process and the courts. Commentators argue that assuring quality mediation services is particularly important when mediation is funded by public funds, involves mandatory referral, or does not permit party choice of the program, the process, or the mediator, or when the parties would not have the knowledge or the resources to make informed choices.1
Much of the discussion about and efforts to assure mediator competence have focused on the process of selecting mediators. That initial determination of qualifications, however, does not ensure continuing quality mediation. Some commentators maintain that monitoring mediators' on-going performance "may be equal in importance to the initial selection process" and is one of the "most critical components of any system that seeks to ensure mediator and program competence." Beyond having minimal continuing education or training requirements, few programs engage in on-going, systematic assessment of their mediators.1
The periodic assessment of mediators can provide information that can help to maintain quality mediation in several ways. By identifying the mediators' strengths and weaknesses, it could give mediators feedback they would like to have and could use for their own skill-enhancement and professional development. The program administrator could use this information to determine which skill areas should be the focus of additional training or which mediators could benefit from additional supervision or co-mediation, as well as to assign cases to mediators according to their strengths.1
1 (2004 Assessing Mediator Performance: The Usefulness of Participant Questionnaires Roselle L. Wissler; Robert W. Rack Jr.)
The following tools and processes can be applied to some or all of our current services which include: Mediation; Training; Coaching. There is a mixture of evaluating an individual’s skills and evaluating the results that the individual and the process delivers. The underlying theory being that if we can control the process so it is consistent, and the skill level of the practitioner, we will be able to figure out where to apply changes to increase what we consider to be desirable outcomes.
Phases of Assessment and Control
Pre-Assessment of Practitioners
· Education and Training
· Experience
· Written Exams
· Performance Based Assessments (recommendations; role play evals)
In-Practice Assessment
· User Feedback Assessment Results
· Assessment by Other Practitioner (Mentor/Mentee)
· Self-Assessments by Practitioner
· Settlement Rates
· User Complaints
Future Results/Impact Assessment (TBD – not in practice at this time)
Pre-Assessment of Practitioners
(Target Goal = 100% of new mediators go through the Pre-Assessment process)
Before someone comes to work under the banner of NMC, there are some basic things they need to know and understand so we all deliver what is expected. The Executive Director and/or the Program Manager make the final evaluation of where and how to start someone in delivering services for NMC, including being accepted into the Mentoring program.
Pre-Assessment of New Practitioners Checklist Form: The ED and/or Program Manager will complete the interview, evaluation and checklist process with a positive result before anyone providing key services can start working. Best practice is that both NMC staff members support entry into the program and sign-off on the applicant.
Education and Training:
Beginner mediators need at a minimum to have completed a 40 hour basic mediation course by a reputable source. The reputation of the source is determined by the ED, the Program Manager, or both.
The value of other training related to any of the services NMC provides is determined by the ED, the Program Manager, or both.
New mediators to NMC may be admitted into service with these minimums, and will always enter the mentoring program for further education and/or assessment.
Experience:
Having had time in practicing the services NMC provides is very valuable. Assessing the amount and relevancy of experience is determined by the ED, the Program Manager, or both.
The actual number of services provided along with types/similarity of venues, and/or previous assessments at another organization are all to be considered when pre-assessing a practitioner.
Written Exams:
NMC does not currently have a written exam that covers the services it provides.
Certifications of accomplishment by accredited organizations are used to understand the qualifications of practitioners.
Related degrees by accredited organizations are used to understand the qualifications of practitioners.
Performance Based Assessments (recommendations; role play evals)
Positive recommendations from sources the candidate has worked with previously are to be evaluated by the ED, the Program Manager, or both.
If a role play is used as part of the pre-assessment the results of the session is determined by the ED, the Program Manager, or both.
In-Practice Assessment of Practitioners and NMC Services
NMC uses a variety of methods to assess the services being delivered. All of them together add up to a picture of what the organization is delivering.
User Feedback Assessment Results
(Target Goal = yearly average of at least 80% approval in all categories)
Getting input from users is frequently suggested as one of the ways to evaluate on-going mediator performance for quality. NMC uses Feedback Surveys as a means of collecting and aggregating this data. We currently have a Mediation Feedback Survey and a Training Feedback Survey.
NMC confidential surveys still allow a correlation to who the mediator(s) was and thus acts as a practitioner assessment.
Feedback Surveys also offer the opportunity to aggregate data over the course of time and can point to trends in service delivery.
Feedback surveys will be built into as many situations and processes as possible and the data will be reviewed and compiled on a regular basis.
The feedback results will help to inform individual intervention with practitioners, topics for ongoing trainings and continuing education, and overall process changes.
Assessment by Experienced Practitioner (See Mentor/Mentee Program)
(Target Goal = 100% of new to NMC mediators go through the mentoring process)
Prior to leading a process all new to NMC practitioners go through a process of observing, being mentored, co-practicing and then solo mediating.
Being raised to the level of leading or solo-practicing a process involves passing an assessment by a NMC designated practitioner of equal or superior experience level.
Mentee Evaluation Form - The candidate must pass the evaluation by the mentor(s). Ideally the evaluation will be done by several different mentor practitioners over the course of time, but may be completed by a single designated practitioner.
A “designated practitioner” is defined as someone who both the Executive Director and Program Manager of NMC agree is qualified to carry out the task of mentoring.
Self-Assessment by Practitioner
(Target Goals = mediators debrief at least 50% of their cases. All mediators use the NMC Self-Evaluation process at least once per year)
NMC uses the methodology of reflective debriefing. This process involves at least two practitioners, one acting as the mirror for the other practitioner who just provided the service. The objective is for the mediator to find new ways to look at their performance and to evaluate themselves based on this information and to use that information to assess any need for change in the future. All sessions should be designed to include a reflective debriefing.
Mediation Specific - NMC uses a Mediator Self-Evaluation Form for mediators to take a close look at their practice based on industry and NMC standard. This encourages organized and directed self-reflection and guides change when the assessment shows room for improvement. All practitioners should take this self-assessment at least yearly.
Settlement Rates
(Target Goal = Overall Settlement rate is 50% or greater for all mediations combined, yearly)
Settlement rates are often proposed as a means of monitoring mediator and service quality, but the consensus appears to be that this measure should be used with caution and never as the sole measure of performance for several reasons. First, settlement is only one possible outcome of mediation and is not necessarily the most important goal that the parties or the program want to achieve. Second, non-settlement does not mean that the mediator did not make a substantial contribution to the case, perhaps even helping the parties to resolve many of the issues, and settlement does not guarantee that all underlying issues have been addressed and permanently resolved. Third, the nature of the process by which the settlement was achieved, such as whether it permitted party self-determination, is also considered important in determining mediator/process quality. Fourth, heavy reliance on settlement as an assessment tool could lead mediators to coerce settlements. Finally, making comparisons of settlement rates among mediators within a program would be misleading if some mediators were routinely assigned more difficult cases.
NMC keeps records of all mediation outcomes in regards to whether or not an agreement was reached by the parties during mediation. NMC keeps track of this information in the Case Load Manager program. The information can be looked at as an overall rate for the organization, by case type, by individual mediator, etc…
User Complaints
(Target Goal = less than or equal to 2 complaints per 500 mediations performed)
One recommended source of user input includes the review of complaints. Because complaints tend to be rare, and their absence does not necessarily indicate the absence of problems, complaints are not likely to provide an adequate source of information for on-going quality monitoring.
NMC does not currently have a stand-alone complaint mechanism beyond the Feedback Surveys.
User complaints that do not come in the form of Feedback Surveys will be documented on a Feedback Survey and entered into the system of user feedback.