EVALUATING GROUP COUNSELING
Chapter 12

Purpose of the chapter:
To examine aspects of group evaluation, discuss rationale and provide resources.

 Importance of group process and outcome evaluations:
(Also addressed in chapter 8 and in ASGW's Best Practices Guidelines (Appendix C, page 542).

        •  Determine effectiveness of group counseling

        •  Enhance quality of the group experience

        •  Justify viability of the group as a helping modality.

□  BASIC ASSUMPTIONS: (p367-368)

        •  Main interest of counselors is the impact of group counseling in their setting.

        •  Most counselors do not have the resources available to conduct group research.

        •  Group counseling helps clients solve problems; evaluations denote its specific impact.

        •   Scientific evaluation by counselors enhances data base and future research projects.

        •   Scientific evaluation procedures contribute to best practice information in the field.

□  METHOD OF EVALUATION: (p368)

      Evaluation procedures are customized to address characteristics of the practice area.
      Need to decrease error factors and increase specificity to create effective evaluations.

        A. Process Evaluation: (p368)

                •  Increases understanding of group dynamics and improves effectiveness.

                •  Three components of group process: leader, members, and group interaction.

               •  Information is gathered through observation, analysis, and feedback.

               •  Primary goals are to evaluate:

                       °  Effectiveness of leader's methods, attitudes, and characteristics.

                       °  Effects member attitudes, reactions, and characteristics of individuals.

                       °  Group and members goals with the focus/themes of group interaction.

                1.  Expert Evaluation (p369)

                       a.   Supervision provides consistent feedback.

                       b.  Consult with professional group counselor when the group is at an impasse
                            or not going well. (See Ch 6 if a colleague is used.)

                       c.   Minimize distractions of observation: one way mirror, closed circuit TV.

                       d.  Feedback of observations provides opportunity for professional growth.

Trotzer, J.P. (1999). The Counselor and the Group: Integrating Theory, Training, and Practice. Philadelphia, PA:

              2.  Member Process Observer Evaluation (p369)

                      a.   Rotate responsibility of process observer; provides new experience for group
                            member.

                      b.  Some training in observation is required.

                      c.   Process observer input (during, closing, on request) simulates the group.

              3.  Member Evaluation (p369-370)

                      a.   3x5 cards evaluation method: members write impressions/appraisals of session
                            and suggestions for improvement.

                      b.  Questionnaire: rate specific process on a scale (boring to extremely interesting;
                            from 1-7). Anonymous information is used to assess ongoing development of group.

                      c.   Members write reactions to sessions and submit the following week.

                             i.     Advantage: creates comprehensive picture

                             ii.   Disadvantage: counselor is always a week behind.

                      d.  Members submit journals at the end of the entire group process.

                             i.    Advantage: Gain material for future learning.

                      e.   Leader dictates process summaries after each session and distributes
                            to members.

                             i.     Increases flow between sessions, deepens group interactions, and allows
                                    members to present their opinions of the group.

              4.  Evaluation of Audio and Video Replays (p370)

                     a.   Get permission if intention is to use in supervision or staffing.

                     b.  Avoid surprises/distraction from recording by consistent use over time.

                             i.     Advantage: Most comprehensive method; review a crisis, therapeutic
                                    moment, vital information, significant disclosure, or areas needing improvement.

                             ii.   Disadvantage: Time consuming

              5.  Interaction Evaluation (p3 70-3 71)
                   For formal instruments: see DeLucia-Waack (1997 b)

                      a.   Interaction Analysis Scale (Bales, 1950)

                      b.  Hill Interaction Matrix (Hill. 1966)

                      c.   Sequential Analysis of Verbal Interaction (SAVI) (Simon and Agazarian, 1974)

                             i.     Advantages: able to quantify interaction patterns and evaluate qualitative
                                    content

               6.  Group Themes (p371)
                   The focus of interaction between group members-from beginning to end.

                       a.   Identify the themes chosen by the group

                       b.  Evaluate if these themes move the group toward its goals?

                       c.   If not, the leader can re-focus the group as needed.

               7.  Process Evaluation Scales (p371-37 3)

                      a.  Leadership Process Evaluation Scale (Figure 12.1, p3 72-3 73) from leadership
                           skills in Chapter 6: reaction, interaction, and action skills.

                              i.     Determine effectiveness of specific skill categories: leadership roles,
                                     functions, and styles (5-point scale).

                              ii.   Mean score of each skill is compared with overall performance.

                              iii.  Allows counselor to target specific behaviors that need improvement.

                      b.  Member Process Evaluation Scales (Figure 12.2, p3 74-3 75) from
                           member behaviors in Chapter 7: resisting, manipulating, helping, and
                           emotional behaviors.

                              i.     Evaluate observed behaviors of individual members on prescribed scale

                              ii.   Describes characteristics of group interaction at point in time

                              iii.  Allows counselor to track individual and collective group process

                              iv.  Indicates degree of therapeutic v. resistant or manipulative

       B. Outcome Evaluation (p373-381)

               •  Looks at group goals, purposes, and objectives observed as they are related to the
                  change or impact in feelings, thoughts and behaviors.

               •  The degree of change indicates success or failure of the group process.

               •  Disadvantage: members and group are difficult to control and evaluation criteria are
                  difficult to define.

               1.   Importance of Goals (p3 76-3 78)
                    Goals are the root of effective outcome evaluation

                      a.   They should be specific, clearly defined, measurable or observable outcomes,
                            based on individual, unique goals (can compile group data)

                      b.  Techniques for generating goals:

                              i.     Initial goals can be created in screening interview and shared with
                                     the group during introductions. Create structured work oriented
                                     environment.

                              ii.   Individuals create a detailed description of a problem they want to solve.
                                   The group helps individual generate goals

                        c.   Goal setting sessions: individuals select and discuss goals.

                              i.     Behavioral Contract as an evaluation devise

                              ii.   Members write down the actions they will take to achieve goals

                       d.  Members are accountable to counselor and group members

                              i.     Goal Attainment Scaling (GAS) (see Figure 12.3, p3 78)

                              ii.   Transforms individual goals into a measurable 5-point scale,
                                    from -2 (much worse than expected) to +2 (much better than expected).

                              iii.  Individuals identify specific, concrete, measurable, problem solving goals
                                   they want to attain.

                                      •  Goal is represented by a '0' (expected outcome/target)

                                      •  Reasonable units of variance are determined in increments of 1 unit
                                          per integer above and below the target.

                              iv.  Periodic evaluations are completed to demonstrate progress toward goals.

                              v.  A combine GAS score of all members indicates overall effectives of group
                                   process.

               2.  Self and Other Reports (p37'8-380)
                   Others include: parents, other family members, teachers, coworkers, colleagues,
                   supervisors, and selected peers (inside and outside of group).
                   ** Use appropriate release of information before gathering reports from others.

                       a.   Able to test validity or invalidity of report.

                              i.     Reported changes must be verified by self and others who are familiar
                                     with the group member. Is the group member receptive to the influence
                                     of others?

                              ii.   Observation of changes caused by counselor and important others are
                                    usually a result of extensive work completed.
                                    Therefore, it exists.

                              iii.  Changes caused by counseling process can be freeing for one person and
                                   disruptive for another.

                              iv.  If no change is reported, then counselor should reevaluate the criteria.

                       b.  Important evaluation tool even though the subjective nature adds bias to
                            the report.

                              i.     Our thoughts and perception influence our behavior, therefore a self-based
                                     evaluation of client's experience.

                              ii.   Investigate cause of differing opinions/meanings associated with the
                                   change to produce additional changes in the individual, group and/or
                                   the process.

                              iii.  Balance of self and other perceptions counters bias and creates a
                                   comprehensive, balanced, and accurate assessment of change.

                       c.   Types of self-reports:

                              i.     Direct feedback, Q-sorts, rating scales, semantic differentials, or
                                     personality inventories.

                                      •  Informal: create a brief rating sheet focused on a specific problem
                                         administer it periodically to client/ important others, compare raters'
                                         perceptions of changes.

                                      •  Formal: Q-sorts, rating scales, semantic differentials, or open-ended
                                         questionnaires, and structured face to face interviews.

               3.  Outcome Criteria (p380-381)

                      a.   Criteria must be:

                              i.     Adaptable to individual goals of the client.

                              ii.   Specific and observable/measurable so it can be validated.

                              iii.  In schools, limit variables to those that directly affect education.

                              iv.  Directly impact client's performance in group and living environment.

                      b.  Types of criteria:

                              i.     Behavioral and affective (attitudes, feelings, and values).

                              ii.   In schools, use grades in specific classes, GPS, other standardized tests
                                   (achievement and aptitude).

                              iii.  Specific tests: acceptance of self and others, self-concept, dogmatism, other
                                   personality measures.

                      c.   Important questions assist choice of criteria and increases creditability of
                            counselor's claims of effectiveness of group process:

                              i.     Does it measure a relevant variable? Not just convenient?

                              ii.   Is it related to the client's goals?

                              iii.  Will the observed changes solve problems?

                              iv.  Able to use simple terms to describe results clearly to client?

                              v.  Will results portray the influence of the group counseling process?

               4.  Follow-up Procedures (p381)

                      a.   Main questions:

                             i.     Will the change last?

                             ii.   Does the latent effect create the positive change after completing treatment?

                             iii.  Immediate criteria measures increases superficial short group counseling
                                  outcomes (p210). Therefore, follow-up is more important than immediate
                                  results. Delayed evaluation is preferred.

                      b.  Purpose of follow-up:

                             i.     Maintain contact with former group members over extended period to
                                   determine if actual or professed changes are integrated into client's life
                                   style.

                      c.   Methods:

                             i.     How: Personally or with questionnaire, then tabulate data for future reference.

                             ii.   At three and six-months; casually (no schedule), or reconvene entire group
                                   periodically.

□  SUMMARY (p38l-382)

       •  Effective evaluation of group process relates process and outcome.

       •  Multiple assessments increase reliability.

       •  Multiple sources of feedback reveals focus of group process and its effectiveness.

       •  Three principle concepts:

               1.   Make evaluations part of routine.

               2.  Create general goals to assess needs of organizing groups and specific individualized
                    goals.

               3.  Use evaluation results to:

                      a.   Stimulate the group process.

                      b.  Increase expertise of group leader.

                      c.   Establish credibility of effectiveness of group process.

□  ILLUSTRATIVE GROUP COUNSELING EXAMPLE (p382-384)

       A. An Interpersonal Problem Solving Group in a Mental Health Center (p382)

               1.   Membership: 8 adults (5 women, 3 men)

              2.  Leadership: male-female co-leadership

               3.  Referral source: individual therapist

               4.  Therapeutic focus: interpersonal skills

               5.  Total commitment: 4 months/30 hours (intake, exit interview, 14 (2-hours session).

       B. Overview of the Group Process (p382-384)

               1.  Orientation/Intake Interview (p382- 383)

                       a.   Purpose:

                              i.    Gain permission / contact referring therapist for additional info.

                              ii.   Identify interpersonal problems.

                              iii.  Select issues to resolve.

                              iv.  Explain group ground rules.

                              v.  Answer questions.

                              vi.  Review Goal Attainment Scaling process (Figure 12.3).

                              vii. Commit to group process.

                       b.  Approach based on content of intakes:

                              i.     Semi-structured:

                              ii.   Specific activities related to communication skills, social skills,
                                   assertiveness skills, and conflict resolution.

                              iii.  Open ended, problem focused interaction.

               2.  Session 1 and 2 (p383)
                    Forming and security:

                       a.   Get acquainted, start self-disclosure, and give feedback using  structured
                             activities.

                       b.  Discuss/clarify ground rules.

                       c.   Share reason for joining, target behaviors, current status.

               3.  Session 3 and 4 (p383)
                    Cohesiveness and acceptance:

                       a.   Practice active listening and other interpersonal communication.

                       b.  Identify specific interpersonal problem areas.

               4.  Session 5 and 6 (p383)
                    Emphasize social skills:

                       a.   Role playing and behavior rehearsal with 'out of group' task and assignments.
                             Established in group and 'out of group' connectedness.

                      b.  Explore responsibility for self and individuation/differentiation in the

                          context of the group.

               5.  Session 7 and 8 (p383)
                   Assertiveness principles and techniques:

                       a.   Practice assertiveness skills in the group.

                       b.  Explore opportunities to use skills outside of group.

               6.  Session 9 and 10 (p384)
                   Conflict resolution/working stage:

                       a.   Identify tendencies in conflict situations.

                       b.  Identify and analyze current conflicts in terms of effective conflict resolution.

                       c.   Create conflict contracts between members and outside relationship.

                       ** Session 1-10 = 1 to 1.5 hours structure and .5 to 1 process.
                           Session 11-14 = mostly process

               7.  Session 11—14 (p384)
                   Members direct content of sessions:

                       a.   Check in system to update group on progress and request time.

                       b.  Construct goal board to display individual goals and progress.

                       c.  Final session - review individual and group progress, make commitments to
                            continue work, say goodbye.

               8.  Exit Interview (p384 

                       a.   Individual meeting with co-leaders:

                       b.  Focus on Goal Attainment Scale procedure.

                       c.   Help client assess and accept responsibility for positive changes.

                       d.  Determine the overall effectiveness of group process.

                9.  Follow-up (p384)
                     3-month later:

                       a.   Send questionnaire and survey.

                       b.  Part 1: client's goals/progress and current status.

                       c.   Part 2: Solicit feedback regarding group process experience, benefits of
                             group counseling for interpersonal problem solving.

□  LEARNING ACTIVITIES (p385-386)
      Prepare counselor for orientation aspects of organizing a group program.
      Develop leadership skill and expand counselor's repertoire of group activities.

        A. Class Presentation (p385)
              Prepare and deliver 15-20 minute talk.

                1.   Clarify potential clients' understanding of the group process

                2.  Inspire interest to participate

               3.  Conduct a Q & A, client actors ask questions

               4.  Request feedback on presentation: strengths, weaknesses, and possible omissions
                    and revisions.

       B. Staff Orientation (p385)

           Prepare and deliver 15-20 minute talk.

               1.   Classmates act as teachers, referral sources, administrators, or parents.

               2.  Discuss distinctions between different types of groups.

               3.  Demonstration group experience with all or some of classmates.

               4.  Conduct a Q & A, classmates asks questions while staying in role.

               5.  Critique presentation.

       C. Demonstration Videotape (p385)

           Develop videotape to demonstrate group process.

               1.   Stimulate class discussion about group process and the leadership role.

               2.  Get reactions from counselors in the field, referral sources, teachers, administrators,
                    and families.

               3.  Duplicate for use on the job.

       D. Demonstration Activities Project (p386)

               1.  Direct classmates to write down 3-5 group activities for a classroom situation,
                    with prospective clients, or in a staff orientation session.

               2.  Describe goals, materials needed, procedures, instructions, and the target  population
                    for each activity.

               3.  Ask each class member to demonstrate one activity.

               4.  Duplicate papers for classmates.