INTENSIVE FAMILY THERAPY WITH AT-RISK YOUTH: A PRELIMINARY CRITICAL INCIDENT STUDY by GISELLE A. TRANQUILLA A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS in THE FACULTY OF GRADUATE STUDIES GRADUATE COUNSELLING PSYCHOLOGY PROGRAM We accept this thesis as conforming to the required standard ………………………………………………………………… Robert Lees, Ed.D., Thesis Supervisor ………………………………………………………………… Marvin McDonald, Ph.D., Second Reader & Thesis Coordinator ………………………………………………………………… Faith Auton-Cuff, Ph.D., External Examiner TRINITY WESTERN UNIVERSITY © 2013, Giselle A. Tranquilla ii INTENSIVE FAMILY THERAPY WITH AT-RISK YOUTH: A PRELIMINARY CRITICAL INCIDENT STUDY ABSTRACT Multisystemic Therapy has been established as an effective treatment alternative to incarcerating or institutionalizing at-risk youth and children. This therapeutic approach has been utilized in several countries to address behavioral and psychological issues such as delinquency, conduct disorder, and suicidality (Henggeler, Rowland, Pickrel, Miller, Cunningham, Santos, Schoenwald, Randall & Edwards, 1997; Scherer & Brondino, 1994; Schoenwald, Henggeler, Brondino & Rowland, 2000). The Intensive Family Therapy Project followed the basic tenets of Multisystemic Therapy and adapted them to a rural community setting in British Columbia. The Project was a treatment program designed to work with young offenders and their families in order to address delinquent behavior from a holistic perspective. This study used the Critical Incident Technique to examine what clients found helpful and unhelpful about the treatment program and its various components, from the clients’ perspectives. Nine interviews (five parent/guardian, four youth) were conducted involving six families. Interviews were then analyzed, and the data was classified into two themes (seven categories, 26 subcategories). Results indicate participants found that involvement in the project was more helpful than hindering, as indicated by the higher rate of positive incidents identified across all categories. Clients’ voices identified Intensive Family Therapy as a valuable treatment approach to working with at-risk youth and their families. Existing research tends to support the notion that Multisystemic Therapy is a treatment approach where one must adhere rather strictly to a iii particular set of tenets. However, results of the current study indicate the potential for adapted forms of MST to be applicable, relevant and effective in working with at-risk youth and their families. Implications for future study are also discussed. Keywords: Multisystemic therapy; at-risk youth; family therapy; critical incident technique iv TABLE OF CONTENTS ABSTRACT .................................................................................................................................... ii TABLE OF CONTENTS............................................................................................................... iv LIST OF TABLES ........................................................................................................................ vii ACKNOWLEDGMENTS ........................................................................................................... viii CHAPTER 1: INTRODUCTION & REVIEW OF LITERATURE................................................1 Introduction ......................................................................................................................................1 Purpose.............................................................................................................................................1 Rationale ..........................................................................................................................................2 Approach ..........................................................................................................................................2 Review of Literature ........................................................................................................................3 Current Approaches .........................................................................................................................4 Cognitive-Behavioral Skills Training ..................................................................................6 Problem-Solving Skills Training .........................................................................................6 Parent Management Training ...............................................................................................7 Case Management ................................................................................................................7 Multisystemic Therapy ....................................................................................................................9 Components of MST ..........................................................................................................10 Effectiveness of MST ........................................................................................................13 MST in Canada and Beyond ..............................................................................................16 The Intensive Family Therapy Project, Chilliwack BC ...................................................169 Qualitative Research ......................................................................................................................20 Use of Qualitative Methods ...............................................................................................21 v Concerns Regarding Use of Qualitative Methods .............................................................22 Qualitative Research and Family Therapy.........................................................................24 Summary of Current Study ............................................................................................................25 CHAPTER 2: METHOD ..............................................................................................................26 Critical Incident Technique............................................................................................................26 Flanagan’s Procedure .........................................................................................................27 Trustworthiness & Applicability......................................................................................310 Participants .....................................................................................................................................32 Primary Researcher ........................................................................................................................33 Procedure .......................................................................................................................................35 Interviews .......................................................................................................................................35 Data Retention ...............................................................................................................................36 Data Analysis .................................................................................................................................37 Trustworthiness & Applicability....................................................................................................38 Cross-Validation ................................................................................................................41 Saturation ...........................................................................................................................42 CHAPTER 3: RESULTS ..............................................................................................................44 Description of Categories ..............................................................................................................45 Summary ........................................................................................................................................60 CHAPTER 4: DISCUSSION........................................................................................................61 Limitations .....................................................................................................................................69 Trustworthiness & Applicability........................................................................................71 Implications....................................................................................................................................74 vi Improvements In Therapy ..................................................................................................76 Future Qualitative Research ...............................................................................................77 Conclusion .....................................................................................................................................80 REFERENCES ..............................................................................................................................82 APPENDIX A: INTERVIEW PROTOCOL ................................................................................89 APPENDIX B: INITIAL CONTACT LETTER TO PARTICIPANTS .......................................91 APPENDIX C: INFORMED CONSENT FORM - CAREGIVER(S) .........................................93 APPENDIX D: INFORMED CONSENT FORM - YOUTH .......................................................95 APPENDIX E: IFT ASSESSMENT OUTLINE ..........................................................................97 APPENDIX D: SAMPLE ASSESSMENT ..................................................................................98 APPENDIX D: IFT WEEKLY SUPERVISION OUTLINE........................................................99 vii LIST OF TABLES Table 1: The % of Agreement between Raters’ and Researcher’s Categorization of Incidents ...40 Table 2: Participation Rates for Subcategories .............................................................................46 viii ACKNOWLEDGMENTS There are so many people I really should thank for their role, big or small, in helping me through this process. If I forget to name someone, please know that I am more grateful than you will ever know. First, I need to thank Rob Lees for being so faithful and supportive. I couldn’t have asked for a better supervisor or mentor through this process. Thank you for your constant encouragement, even when I lost my footing. You never gave up on my ability to complete this work, and that allowed me to pick it back up and keep going. Thank you so much. Mac, thank you for being willing to come on board half way through to make sure we all stayed on track. Your understanding and gentle support made it safe for me to come back to the table. Thank you for brainstorming with me and inspiring me to continually widen my scope to see a broader picture. Jamie, my wonderful friend and gracious editor, thank you so much for your patience with me as we walked through the painful process of getting this document to make sense. The hours you gave of your time and energy were an incredible blessing to me, and I can’t thank you enough. I dedicate all of this work to my mother, who has undoubtedly been my biggest fan and an incredible source of inspiration and support. I love you so much, and I am so grateful that God has blessed me with you as a role model. Father, to You be all the glory and honour for all that You have done in and through my life, and for all that lies ahead. 1 CHAPTER 1: INTRODUCTION & REVIEW OF LITERATURE Introduction Although many approaches to dealing with young offenders have proven to be mildly effective at best, there is at least one approach that has shown signs of being a “promising treatment” for working with problem children and adolescents (Henggeler, Melton, & Smith, 1992, p. 953). Multisystemic Therapy (MST) began establishing itself as an effective treatment alternative to incarcerating or institutionalizing at-risk youth and children, and follow-up studies conducted in a number of areas on the same topic have also shown the same types of positive results, and have now expanded in terms of targeting a wider variety of presenting issues, as well as long-term effectiveness (Henggeler, Melton, Brondino, Scherer, & Hanley, 1997; Sawyer & Borduin, 2011; Schoenwald, Henggeler, Brondino, & Rowland, 2000). Though these results are encouraging, studies investigating the effectiveness of MST have mostly been completed in the United States. However, further studies using this therapy are now being conducted on research projects in parts of Europe as well as Canada (Jaffe & Baker, 1999; Sundelin & Hansson, 1999). Depending on the location, results from these studies have varied somewhat from results generated in the US. The use of slight variations of this treatment approach, one in particular known as Intensive Family Therapy (IFT), can also be found in Canada as well as parts of Europe (Jaffe & Baker, 1999; Sundelin & Hansson, 1999). This Intensive Family Therapy approach follows the same basic principles and tenets that are foundational to MST. Purpose. The interest of the current study is to examine the perspectives of clients who have participated in an Intensive Family Therapy Project. The study will focus on what about 2 this program has been helpful to them in gaining insight and growth both personally and as family units, as well as what has hindered them in this same process. Rationale. Most of the studies which have been conducted on MST approaches and its variants have been quantitative in nature and have proven to significantly reduce recidivism rates, antisocial behavior, and increase positive family functioning, among other things (Henggeler et al., 1997; Sawyer & Borduin, 2011). However, not much work has been done looking at the effectiveness of MST from a qualitative standpoint. The current study will examine a number of Canadian families who have completed the Intensive Family Therapy (IFT) Project in British Columbia. The goal of the study is to investigate participant perceptions of what were helpful or positive experiences of the program and what were hindering or negative experiences of the program. Quantitative data has already been collected in order to ascertain to some extent the effectiveness of the use of this type of therapeutic approach, and this study will explore the components of IFT and the relative contribution of various activities to helping or hindering the process of improvement. Approach. Semi-structured interviews were conducted with members of six families who have completed treatment in the Intensive Family Therapy Project, with the aim being to target specifically what has been helpful and what has not been helpful about this specific therapeutic approach. The interviews were recorded and transcribed. Protocols were analyzed and coded using the Critical Incident Technique so as to be able to identify key themes and categories (Flanagan, 1954). Incidents can be defined as important events that occurred during involvement in the program. They become critical when they have a defined impact on a participant’s outcome measures following treatment. This impact is defined by the participant as being either positive (helpful) or negative (hindering) in nature. Incidents were placed into 3 categories based on similarities across all interviews. These categories may help to identify gaps that exist in services being provided, as well as to help support existing reports regarding what is helpful in the therapeutic process in general. Perhaps of more value may be a discussion regarding the relevance and applicability of adapted forms of MST and the potential for positive impact in the lives of at-risk youth as a treatment approach. Review of Literature In the past, many treatment plans have been put into effect in an attempt to reduce juvenile delinquency. There is a great burden placed on society by these young offenders. As the conduct disorder disrupts almost every aspect of functioning, society is increasingly recognizing the burden of suffering experienced by the young people themselves (Moretti et al., 1997). It is yet to be determined exactly how to best help young offenders move towards more prosocial positive living, as well as to help the larger community feel safe and secure. Often it would appear that the latter of these two goals has received more attention, when in fact it may not occur until the youth have received the help they need in order to function “normally”. Previous studies have indicated the results of treatment programs for young offenders have been relatively inconclusive (Moretti et al., 1997). An argument has been made for the notion that the young offender population is best dealt with through the justice system and that institutionalizing the youth will correct existing conditions and deter further delinquent behavior (Carter, Blood, & Campbell, 2001; Moretti et al., 1997). This approach has placed high value on keeping the wider community safe. Others have stated that the missing factor is stronger emphasis on strict discipline, which has led some professionals to refer youth to boot camp style treatment programs (Moretti et al., 1997). In deciphering whether or not these programs are more effective in reducing recidivism rates, results from boot camp style treatment programs 4 were compared with youth who were incarcerated, and comparisons showed that the boot camps did not prove to be more effective in lowering the likelihood of recidivism (Carter, Blood, & Campbell, 2001; Moretti et al., 1997). In a study examining various treatment approaches across Canada for youth with conduct disorder, Moretti and her associates (1997) also note some interventions which offer the potential of being valuable in treating this disorder, which has been linked to delinquency, without institutionalizing the youth. Current Approaches Treatment outcome studies have been conducted examining a number of programs and approaches, some of which have been more effective than others. Kashani, Jones, Bumby, and Thomas (1999) discuss a variety of approaches to reducing violent and high-risk behaviour in adolescents, both at the individual and family levels. Functional Family Therapy was identified as being more effective than implementing more cognitive-behavioural approaches focusing on skill building (i.e. communication skill building, problem-solving skill building, etc.) at both the individual and family levels when looking at reductions in aggressive behaviour, rates of reoffending, and improved family functioning post-treatment. Moretti and her colleagues (1997) discuss a few therapeutic approaches in particular which have demonstrated some level of efficacy in a study looking specifically at the treatment of conduct disorder in youth. Among these approaches are parent management training and problem-solving skills training. Roberts and Schervish (1988) also identify four key elements which may reflect the success of a treatment program no matter what the specific approach: behavior management of the offender while in treatment, effectiveness in terms of rates of recidivism, change in attitude, and the cost of putting one offender through the program. In the past, treatment has followed one of three basic ideas: treat the individual, treat the 5 individual within the family, treat the individual within a group (Tate, Reppucci, & Mulvey, 1995). Approaches have been administered at each of these levels. Many approaches with solely the individual in mind involve either teaching the offenders something they were never (or at best poorly) taught, or correcting negative behaviors they either picked up on their own or imitated. A link has been made between a deficit in social/cognitive/behavioral skills and juvenile delinquency, which has led to the use of several techniques which fall into the socialcognitive-behavioral approach to working with young offenders (Offord & Bennett, 1994). Howell and Enns (1995) believe that “approaches based on behavioral and social cognitive principles are most effective in reducing recidivism” (p. 150; see also Larson, Calamari, West & Frevert, 1998). Other programs have begun to consider it almost necessary to address these adolescents within the contexts of their lives, stepping outside the box of individual treatment (Kashani, Jones, Bumby, & Thomas, 1999). Moretti et al. (1997) state that the “integration of multiple interventions within an ecological context is without question the most promising new strategy to emerge in the treatment of adolescent antisocial behaviour” (p. 639). It is important to treat young offenders on an individual basis, but until other factors that play a large part in the lives of these offenders are incorporated into the individual’s treatment, programs may remain unsuccessful (Middleton & Cartledge, 1995). Not only are researchers and psychologists finding it beneficial to address the school setting and peer groups in relation to young offenders, but perhaps more importantly to also address the family unit (Grant, 1994; Pratt & Moreland, 1996). Regardless of whether or not significant results can be identified in existing research regarding effectiveness of various treatment options for this population, it continues to be said that more studies in this area are needed to effectively address the issue of young offenders. 6 Following is a brief review of some of the treatment programs and approaches being used with young offenders, focusing on treatment efficacy and where gaps in programming may still exist. A more detailed review of one approach in particular is presented as background for the current study. Cognitive-behavioral skills training. Cognitive-behavioral skills training is comprised of a number of techniques including anger management and social skills training. This approach focuses on the notion that “antisocial behaviour in adolescents stems from a lack of cognitive and interpersonal skills for managing relationships with peers, family and school” (Carter, Blood, & Campbell, 2001, p. 41). Methods such as modeling and behavioral rehearsal are used to teach youth and to improve cognitive-behavioral skills and functioning. Barkley, Guevremont, Anastopoulos, and Fletcher (1992) conducted a study on three cognitive-behavioral techniques from a systems perspective. They compared the effectiveness of three family therapy programs in dealing with a family member who has ADHD and/or ODD. The three programs they compared were child behavior management training (BMT) with parents, structural family therapy (SFT), and problem solving and communication training (PSCT). The aim of BMT is to reduce conflicts between parent and child through working on behavioral skills such as problem-solving and communication. The aim of SFT is to deal with family systems, coalitions, and issues of enmeshment and disengagement (Barkley et al., 1992). The third program, PSCT, combines the major tenets of both of the previous programs and adds components of cognitive therapy which restructure irrational beliefs. Overall, these programs have proven to show success in treating the youth and their families even beyond the length of the treatment period (Barkley et al., 1992). Problem-solving skills training. Also under the cognitive-behavioral umbrella is 7 problem-solving skills training (PSST). This approach focuses more on aspects of individual functioning such as self-control. Adolescents are taught how to work through targeted problems step by step and to explore effective solutions to those problems (Carter, Blood, & Campbell, 2001). Direct positive reinforcement is used to help foster prosocial behaviors. It has been found that cognitively based treatments such as PSST have shown to be effective in reducing antisocial behaviour, and efficacy increases when PSST is combined with other treatment approaches (Carter, Blood, & Campbell, 2001). Parent management training. Another program that has been used within the family is parent management training (PMT), which has shown promise in the area of treating young offenders (Offord & Bennett, 1994). This approach is aimed at negative parent-child interaction patterns and focuses on training parents to teach and foster more prosocial behaviour by changing the way they interact with their children (Carter, Blood & Campbell, 2001). Kazdin, Siegel, and Bass (1992) found that PMT in combination with problem-solving skills training (PSST) generally reduced problems with “child aggression, antisocial behavior, and delinquency, and parental stress, depression, and other symptoms of parent dysfunction” (p. 744). Each of these treatments used separately showed improvements in overall child functioning, but used together they were particularly effective (Kazdin et al., 1992). Again, these are programs that require the involvement and motivation of the entire family, especially the parents, in order to be more than moderately successful in treating the young offender. At this point, treatment that is family-oriented is promising and has proven to show some positive results, which in the past has placed it in the top few optimal program models used in dealing with juvenile delinquents (Roberts & Schervish, 1988). Case management. Case management is a therapeutic intervention aimed at 8 “coordinating care, brokering services, and developing community resources, among other traditional social work activities” (Halfon & Berkowitz, 1993, p. 379). Not only has it been recognized that the problems of youth today are far more complex and varied in nature for any single type of treatment to be effective, but also that it is crucial for at-risk families to somehow become attached or reattached to the greater community in which they find themselves (Halfon & Berkowitz, 1993; Tate, Reppucci, & Mulvey, 1995). Case management places a professional in the position of managing any given family’s care plan. This case manager assists in the initial assessments done with clients. Following the assessments, a care plan is designed to suit the specific needs and issues of the family and the young offender (Halfon & Berkowitz, 1993; Netting, 1992). The case manager monitors the progress and effectiveness of the care plan, and adjusts it when necessary. This professional acts as an advocate for the family and provides access to community resources, and professional assistance in terms of counseling or therapy on both individual and family levels. Halfon and Berkowitz (1993) state that “a case manager serves as a bridge between the child and family with complex needs and the service delivery system” (p. 387). Case management is a more multidimensional or multisystemic approach to working with at-risk youth and their families. With these objectives set in place, case management has attempted to bring together many treatment programs under one umbrella, as well as provide the families with connections outside the treatment program so as to allow growth to continue once treatment has come to an end. Every case is different and requires very individualized plans that are targeted to the needs of the youth and the family (Halfon & Berkowitz, 1993; Netting, 1992). Another objective of case management, which may seem contradictory to the others, is to reduce the cost of treating the individuals and families in need of assistance (Halfon & Berkowitz, 1993). Another element 9 that has also been addressed within the case management treatment plans, under the supervision of the case manager, is the lack of follow-up and outcome study in several other treatment programs (Netting, 1992). Although it is not the perfect solution, case management has proven to meet the varied needs of high-risk young offenders and their families. Not only have multisystemic approaches proven to be among the only programs to “demonstrate short- and long-term efficacy with chronic, serious, and violent juvenile offenders,” (Netting, 1992, p. 162), but case management has also been viewed as “an essential component in managing care,” making the step toward treatment in this direction a positive one (Tate, Reppucci, & Mulvey, 1995, p. 779). Multisystemic Therapy Some of the treatment programs discussed have proven to have at least partially positive results. However, the overall success of current treatments has been quite poor (Henggeler et al., 1997; Kashani, Jones, Bumby, & Thomas, 1999; Schoenwald, Henggeler, Brondino, & Rowland, 2000; Shamsie, 1981). This led Tate, Repucci, and Mulvey (1995) to state that “clearly no single, proven effective approach to working with these adolescents exists” (p. 779). In an attempt to improve the findings in this statement, the notion has been raised as to whether it is the individual who needs to be treated, or the specific behavior which has led the individual to be brought to court or referred to a treatment program. It has been suggested that interventions have not been entirely successful in working with young offenders because various treatments tend to look at only a portion of the problem and attempt to treat only a part of the individual (Kashani, Jones, Bumby, & Thomas, 1999). Henggeler, Melton, and Smith (1992) suggest that “the interventions used in [previous] studies have almost always addressed only a small portion of the factors that might contribute to a particular youth’s antisocial behavior” (p. 954). 10 Multisystemic Therapy (MST) targets five determinants or factors: the individual, family, peers, school, and community (Henggeler, Melton, Brondino, Scherer, & Hanley, 1997; Jaffe & Baker, 1999; Tate, Repucci, & Mulvey, 1995). This allows MST to view “individuals as being nested within a complex of interconnected systems” (Henggeler, Melton, & Smith, 1992, p. 955). This approach was formulated by Scott Henggeler and his associates in the early 1990’s, and it has since been the subject of several studies and a promising treatment approach to working with at-risk youth with a variety of presenting issues (Scherer & Brondino, 1994; Henggeler et al., 1997; Schoenwald, Henggeler, Brondino, & Rowland, 2000). Kashani, Jones, Bumby, and Thomas (1999) even go so far as to state that MST is “recognized as the most promising treatment approach to date for chronically delinquent and violent youth, from both a cost-effective and recidivism perspective” (p. 209). Components of MST. The assumptions underlying the development of MST and its use are identified by Cunningham (2002): 1. Youthful criminal behaviour is influenced by many factors in [the youth’s] social world (family, peers, school, etc.) so treating the youth in isolation of these factors will be ineffective. 2. The social world, or ecology, should be the “identified client” rather than the individual. 3. Most youthful criminal behaviour is transitory and will desist with the passage of time so intensive interventions should be targeted at those who are most likely to continue offending into adulthood. 4. Most justice interventions are ineffective in reducing criminal behaviour in youths most likely to continue offending into adulthood. 5. Most residential treatment programs are ineffective in addressing needs that prompted 11 admission. 6. Most justice interventions place youths in close contact with other criminally involved youth which can increase their likelihood of re-offending. 7. Every youth is unique so an intervention should be tailored to individual needs and the circumstances of their social world that create criminal behaviour and serve as barriers to its reduction. 8. Traditional focus on problem areas should be augmented with attention to the identification and amplification of strengths in both youths and their ecology. 9. Social service intervention is always episodic so the parent or parent surrogate is the key agent of change and should be empowered to make and sustain the gains. (p. 4) MST differs from many other therapeutic approaches in that, though it may not necessarily consist of brand new techniques and tools, it combines numerous resources and techniques in order to assist the client and the family in the most effective way possible, one that is naturally suited to the family (Henggeler, Melton, Brondino, Scherer, & Hanley, 1997). It is a “comprehensive and individualized treatment approach that addresses the multiple determinants of identified youth and family problems” (Henggeler et al., 1997, p. 228). Therapists using this type of approach utilize a range of therapeutic techniques when working with families (Borduin, 1994; Kashani, Jones, Bumby, & Thomas, 1999). Unlike many manualized or technique-based treatments, MST is principle-based, and Leschied (2000) lays out these basic principles as follows: 1. The primary purpose of assessment is to understand the “fit” between the identified problems and their broader context. 2. Therapeutic contacts should emphasize the positive, and should use systemic strengths as 12 levers for change. 3. Interventions should be designed to promote responsible behaviours and decrease irresponsible behaviour among family members. 4. Interventions should be present-focused and action-oriented, targeting specific and welldefined problems. 5. Interventions should target sequences of behaviours within or between multiple systems that maintain the identified problems. 6. Interventions should be developmentally appropriate and fit the developmental needs of the youth. 7. Interventions should be designed to require daily or weekly effort by family members. 8. Intervention efficacy is evaluated continuously from multiple perspectives with providers assuming accountability for overcoming barriers to successful outcomes. 9. Interventions should be designed to promote treatment generalizations and long-term maintenance of therapeutic change by empowering care givers to address family members’ needs across multiple systemic contexts. (p. 38) These principles are used to work with youth and their families not only within the home setting, but also in the school and larger community settings. Huey, Henggeler, Brondino, and Pickrel (2000) conducted a study examining the importance of therapist adherence to these principles and the effect this key element of MST may have on individual and family functioning. It was concluded that the level to which therapists adhere to these principles has a significantly positive effect on improved family functioning, as well as a negative impact on delinquent behaviour and peer relations. Usually, therapists work with the family directly in the home for an intensive period of 13 time, basically functioning as the case manager for the family in working with an individualized care plan that is set up by both the therapist and the family, including the adolescent targeted as the identified client (Kashani, Jones, Bumby, & Thomas, 1999). Sondheimer, Shoenwald, and Rowland (1994) state that “MST is childfocused, familycentered, and intensive, with treatment goals and intervention strategies tailored to the needs, strengths, and goals of the youth and family” (p. 10). Though the therapist is facilitating therapy and execution of the care plan, the youth and the family have key roles in establishing that plan. Effectiveness of MST. Henggeler and his associates (1992) began implementing MST in a university research setting, which demonstrated positive results when working with at-risk youth. In an initial study conducted in Simpsonville, South Carolina, Henggeler and his associates examined MST “as an alternative to the incarceration of violent and/or chronic juvenile offenders” (Multisystemic Therapy Services, 2004, p. 2). The results of the study showed that those who received the MST model of services had significantly lower rates of criminal activity and were incarcerated fewer times or for shorter periods upon completion of therapy as compared to those who received the regular services offered by the Department of Juvenile Justice (Multisystemic Therapy Services, 2004). The results also showed that positive family functioning actually increased following treatment, and that aggression with peers significantly decreased. In a project conducted in Columbia, Missouri, a more extensive and comprehensive study was completed looking at the effectiveness of MST as compared to individual therapy. Results here showed that at the four-year follow-up mark, those who received MST were “significantly less likely to be rearrested than youth who received individual therapy” (Multisystemic Therapy Services, 2004, p. 3). In addition, youth who were treated under the MST model also had fewer 14 arrests related to substance use/abuse and violent crime. From there, the results of this therapeutic approach have shown more positive and promising results with a fairly wide variety of presenting issues (Henggeler, Melton, Brondino, Scherer, & Hanley, 1997; Henggeler et al., 1997; Multisystemic Therapy Services, 2004; Schoenwald, Henggeler, Brondino, & Rowland, 2000). Borduin et al. (1995) note a number of studies where MST was used with adolescents with serious antisocial behavior, substance abuse issues, as well as with adolescent sexual and chronic juvenile offenders, particularly when looking at long-term effectiveness. Other studies have examined the use of MST in similar areas (Borduin, 1994; Henggeler et al., 1997; Henggeler, Melton, & Smith, 1992; Henggeler, Schoenwald, & Pickrel, 1995; Sherer & Brondino, 1994; Sondheimer, Shoenwald. & Rowland, 1994). Borduin (1994) states that “MST is effective because it directly addresses the multiple determinants of antisocial behavior in youth’s naturally occurring systems” (p. 23). Sondheimer, Shoenwald, and Rowland (1994) also anticipate the effectiveness of MST in treating youth with conduct disorder within the community instead of hospitalizing them. In addition, this approach is believed to be effective with youth who have serious emotional disturbances other than conduct disorder (Henggeler et al., 1997). In a study examining treatment effectiveness of using MST with suicidal youth, it was found that those who received MST had significantly decreased rates of suicide attempts one year following treatment as compared to those who received emergency hospitalization (Huey et al., 2004). Originally, one of the goals behind implementing MST with young offenders was simply trying to find a more cost-effective approach to reducing the number of youth being incarcerated or otherwise institutionalized while still addressing their antisocial behavior (Henggeler, Melton, & Smith, 1992). Leschied (2002) states that MST has “distinguished itself in being most cost 15 effective when compared to other interventions” (p. 49). In a study examining the cost-benefit of MST as compared to individual therapy, Klietz, Borduin, and Schaeffer (2010) indicate a significant cost advantage to using MST with young offenders in regards to the impact on taxpayers as well as victims of crime. MST is identified as being not only clinically effective, but also cost-effective (Klietz et al., 2010). A study conducted in 1992 by Henggeler, Melton, and Smith compared the use of MST with the regular services provided to young offenders through the Department of Youth Services in South Carolina, and the researchers found that recidivism rates were significantly lower for those who were assigned to the MST treatment group, as were the number of arrests and the presence of antisocial and aggressive behavior following treatment. Additionally, this particular study implemented MST with youths and families from a wider variety of cultural backgrounds to see whether the treatment was equally effective across varied cultural backgrounds. Results in this area were not statistically significant; nevertheless, the authors were at least encouraged by the results (Henggeler, Melton, & Smith, 1992). As MST was trying to gain some recognition and response from the wider community for the positive results that were being gathered through its use, Henggeler and his associates moved from a university-based treatment program to one that was community-based (Henggeler, Schoenwald, & Pickrel, 1995). There was potential for the effectiveness of the treatment to be negatively impacted by the shift as an element of experimental control could have been lost, but the results of trial studies indicated that MST was just as effective as a community-based treatment as it had been as a university-based treatment, particularly with regards to long-term outcome success (Henggeler et al., 1995). Borduin, Mann, Cone, Henggeler, Fucci, Blaske, and Williams (1995) also looked at the 16 long-term effectiveness of MST, this time comparing it to individual therapy. A number of different factors were analyzed, but of particular interest were the significant differences found between treatment groups in the areas of number of arrests and the seriousness of the crimes committed. Those who were in the MST treatment group had significantly lower scores in both of these areas than those in the individual therapy group (Borduin et al., 1995). This supports results gathered from previous studies, and encourages continued study of MST. More recently, Sawyer and Borduin (2011) conducted a study on the long-term effects of MST. The study followed participants who were initially part of a clinical trial by the Missouri Delinquency Project headed by Charles Borduin in 1995. Sawyer and Borduin (2011) completed their follow-up study at an average of 21.9 years post-treatment, which indicated that MST had a lasting positive impact in reducing recidivism. MST in Canada and beyond. Most of the studies investigating the efficacy and effectiveness of MST as a therapeutic approach to working with adolescents have been completed in the United States (Borduin et al., 1995; Henggeler, Melton, & Smith, 1992; Henggeler, Melton, Brondino, Scherer, & Hanley, 1997). Though they are rare, there are examples of studies that have been conducted beyond US borders using MST or variants of the model. In one example of a study conducted with a sample of Swedish families using Intensive Family Therapy (a variant of MST), it was found that for at least half of the families there were marked improvements in family functioning following treatment (Sundelin & Hansson, 1999). The study was preliminary in nature as reports of research using a therapeutic approach of this type on an international level are only just beginning to surface (Sundelin & Hansson, 1999). In the Netherlands, a study was conducted looking specifically at what mechanisms produced change during MST (Dekovic, Asscher, Manders, Prins, & van der Laan, 2012). 17 Incorporated in this study was a segment exploring the effectiveness of MST in a Dutch setting, and though the results varied from those in the US, Dekovic et al. (2012) indicated that “MST resulted in a decrease of adolescent externalizing problems” (p. 582). In addition, MST increased parental sense of competence and positive parenting. Tighe, Pistrang, Casdagli, Barush, and Butler (2012) explored families’ experiences of MST in a study conducted in England. Again, results indicated that MST was effective in reducing antisocial behavior. Though re-offending rates in several studies examining MST transportability remained high, results still indicated MST was more effective in reducing reoffending rates than other forms of treatment (Tighe et al., 2012). Studies have also been done in recent years in Canada using this approach, mostly in Ontario. The results of these studies have shown slightly varying results from those found in the States, though still positive in nature (Jaffe & Baker, 1999; Leschied & Cunningham, 1999). Jaffe and Baker (1999) note that since legislation was put in place in 1986 addressing a need for more severe responses to youth criminal activity, the incarceration rate for these youth increased 26%. However, this did not necessarily translate into greater efforts to prevent youth crime, or the implementation of more community-based interventions. These researchers believe that “the focus on legislative solutions has been misguided and has placed the emphasis on increasing the severity of consequences to crime rather than on solutions aimed at preventing the development of criminal behaviour and changing persistent offending patterns” (Jaffe & Baker, 1999, p. 23). As a result, a number of mental health professionals in Ontario moved toward implementing a program that would ideally address this issue of high incarceration rates among adolescent offenders. Dr. Alan Leschied and his associates established a clinical trial using MST at four locations in Ontario with three main goals in mind: a) to seek interventions that can 18 reduce reliance on custody, b) to promote community-based interventions that are at least as effective than more costly traditional residential/custody services, and c) to test the ability to transport an already verified empirically-based intervention to the Ontario context (Leschied, 2000; Leschied, 2003b, p. 1). The trial began in 1997 and was completed in 2001 with follow-up data reported up to three years following completion of the program (Cunningham, 2002). Over the four sites where the study was conducted, a total of four-hundred-nine youths and their families participated. Identified clients were randomly assigned to receive either MST or the regular existing services. A question asked upon implementation of MST in this clinical trial was whether the success of MST in the States would translate into similar results when implemented in Canada, where there are generally higher levels of resources available to youth and their families to begin with (Leschied, 2003a). Preliminary results from the trial indicated that there were no statistically significant differences between the control group (recipients of usual services) and the experimental group (recipients of MST) upon completion of the program and up to three years following treatment (Cunningham, 2002). Though these results are not ideal, Leschied (2003a) states three conclusions based on the research conducted in Ontario: 1. Multisystemic Therapy can be more effective in achieving community safety than existing services already available in Ontario. 2. Implementing Multisystemic Therapy is extremely challenging compared with more traditional services requiring enormous commitment and hard work. 3. Efforts to Implement Multisystemic Therapy must be matched to fidelity to the model or promised outcomes will not be achieved. (p. 7) There was a strong emphasis placed on evaluation throughout the trial study in Ontario 19 including treatment outcomes, incarceration rates over time, cost-benefit analysis as well as other considerations (Leschied, 2000; Leschied & Cunningham, 1999). Another concern stated in reports of the trial study identifies the importance of strict adherence to the principles of MST by the therapists providing services as well as supervisors overseeing those therapists (Leschied, 2003a; Cunningham, 2002). This concern had been previously examined by Huey, Henggeler, Brondino, and Pickrel (2000) where results identified significant effects of therapist adherence to MST principles upon family functioning, deviant behavior and peer relations. Leschied and his associates were not discouraged by the results of the trial study in Ontario, but rather they underlined the importance of continuing research in order to identify treatment options that are more effective for young offenders in the Canadian context, that reduce time in custody for these youth, and that are more cost-effective (Cunningham, 2002; Leschied, 2003a). The Intensive Family Therapy Project, Chilliwack BC The Intensive Family Therapy Project was developed when an opportunity arose to develop a new model in place of office based one-to-one psychological counselling which had been standard procedure for clients involved with Youth Forensics. A literature review conducted by MCFD and Youth Forensics indicated the effective use of Multisystemic Therapy (MST) with this population. As previously noted, MST is a principle based therapy and among its purported strengths was the opportunity to adapt the treatment model to unique client circumstances. Intensive Family Therapy was a local Canadian adaptation of an MST-informed systems approach for working with at-risk youth. The mental health consultant for the region reviewed the literature, gained support from senior management and hired an experienced therapist to begin the program. Aside from seeking a mature therapist, the hiring focused on 20 evidence of the therapist’s flexibility, creativity, and experience with collaborative and outreachbased counselling models. Because MST draws from therapeutic origins as diverse as behavioural and narrative approaches based on “the best of the empirically validated treatment approaches”, it was important that the therapists be eclectic in approach (Henggeler & Swenson, 2002). Prior to commencing the program the mental health consultant, who was to supervise the program and the therapist, reviewed current MST studies as well as the MST handbook. The program was based on the nine principles with assessment and weekly supervision following patterns suggested in the handbook. An assessment outline, sample assessment and weekly supervision outline can be found in Appendix E, F and G respectively. The emphasis was on a short term, intensive model that was fully collaborative and began with verbal contracts with family members about the nature of and expectations within the treatment. The therapist was to be available 24/7 and the expectation was that the family and surrounding support networks should be the agents of therapeutic change and would carry forward generalized learning when treatment was withdrawn. Clients who participated in the Intensive Family Therapy Project had committed a variety of offences including sexual offences and acts of aggression. Some clients had committed multiple offences and there was no attrition from the Project. Qualitative Research Most of the research done in past years in the field of psychology has been quantitative in nature. However, that trend has been gradually changing over recent years, and qualitative methodologies have become more widely recognized and used by researchers and professionals in the field (Rennie, 2002). The reasoning behind this shift is connected to wanting to gain better 21 understanding and knowledge of an individual client’s perspective of any given experience. Gilgun (2005) states that “[such] approaches are particularly useful for understanding meanings that human beings attribute to events in their lives” (p. 40). Though it would be nearly impossible to gain information from clients in large numbers as is the case when dealing with quantitative data, the depth of qualitative information gained can extend far beyond that of the raw numbers (Deacon & Piercy, 2001; Fontes & Piercy, 2000; Gilgun, 2005). Qualitative methodologies will not identify effect size or statistical relationships between variables, however they can “provide the model to be tested, the hypotheses that compose the model, and the items of instruments that represent the hypotheses” (Gilgun, 2005, p. 40). Use of Qualitative Methods. Though the use of qualitative methods is only recently becoming more widely used in the psychology disciplines, other professional fields have been using qualitative approaches to research more readily, and for some time. The field of nursing has strongly relied on qualitative methodology for a number of years (Mitchell, 2001). Another area of study where qualitative methods have been used is that of education and the learning process (Butler, 2002; Winchip, 1996). There has, however, been a push in some institutions to teach in a more in-depth manner about qualitative methods. Some universities offer separate courses on the subject, whereas others still simply address the issue as it remains embedded in a general subject such as research methodologies (Fontes & Piercy, 2000). Despite the catch-up that may be required in teaching qualitative research methods, the information gathered using qualitative methods adds to the raw numbers generated quantitatively and is in fact relied upon as valid data by itself (Fontes & Piercy, 2000). Tighe, Pistrang, Casdagli, Baruch, and Butler (2012) indicate that the use qualitative approaches may help identify potential problems or limitations in any treatment model, as well as improve overall implementation. 22 Looking specifically at recent trends in Canadian psychology, “[at] present, qualitative research in Canadian psychology is beginning to take root” both within training institutions as well as in practice and research (Rennie, 2002, p. 139). One area that has made perhaps the most use of qualitative methodologies, in Canada at least, is that of cultural psychology. Kral, Burkhardt, and Kidd (2002) even state that these methods are the “investigative tools of choice” for this particular field (p. 154). Concerns regarding use of qualitative methods. There are a number of issues which cause researchers to hesitate to use qualitative methodologies when conducting research. Gilgun (2005) identifies two of these issues in particular, the first of which is the notion of generalizability. When conducting qualitative research, sample sizes can look drastically different than those of quantitative research studies. The question that is raised is how can results from a study comprised of a sample size of three participants be generalizable to the broader population; can it be assumed that the findings are relevant or applicable to others with similar experiences. However, Gilgun (2005) reframes this question by asking what it is that these three participants have identified that has not been identified before when studying a particular issue or event. The question then becomes an issue of testing the findings of qualitative research rather than assuming these findings can be applied across a larger group. The second concern identified by Gilgun (2005) is subjectivity. When conducting qualitative studies, it is the goal of the researcher to work to understand as much as possible a participant’s experience or story. In order to be effective in collecting this data, and in order to ensure the data is valid, researchers must establish a sense of trust and relationship with participants. The concern arises when the researcher loses his or her sense of separateness from the participant and may become emotionally involved with the data or story being collected. 23 During an interview, this may not be a problem, however when analyzing the data, emotional connection may cause a researcher to identify or even create significance where there is none (Gilgun, 2005). This concern can be addressed by implementing some simple strategies before, during, and after data collection. Morrow (2005) discusses the notion of bracketing, which is described as “the process of becoming aware of one’s implicit assumptions and predispositions and setting them aside to avoid having them unduly influence the research” (p. 254). Before or even during data collection, a researcher may discuss or address in writing the conclusions that have already come to mind regarding the research topic which may be based on previous personal experience, review of literature, or even discussions with other researchers. The importance of bracketing is so that these preconceived notions will affect data collection and analysis as little as possible. Bracketing involves self-reflection and requires that the researcher be capable of a great deal of self-awareness (Morrow, 2005). One method of managing the researcher’s own biases and assumptions and emotional reactions may be to consult with a research team comprised of knowledgeable professionals and colleagues, as suggested by Gilgun (2005) and Morrow (2005). This team can also assist during data analysis to keep the researcher from creating significance or jumping to conclusions too quickly. This is done by having team members offer different perspectives or interpretations of the data than those of the researcher, playing “devil’s advocate” simply to ensure all points of view are being considered (Morrow, 2005, p. 254). The goal is to have the research and findings reflect “the perspectives of informants more than they reflect the biases of researchers” (Gilgun, 2005, p. 47). The goal of any research endeavor is to add information or knowledge that may not have been considered before in any given field of study. Using one specific method of gathering that 24 information may not uncover all there is to know about a particular phenomenon, and perhaps what should be remembered and considered when conducting any type of research within the realm of social sciences, whether quantitative or qualitative, is “whether the findings can make contributions to theory, policies, programs, [or] interventions” (Gilgun, 2005, p. 46). Qualitative research and family therapy. There are a number of different areas of psychology that have found qualitative methods of gathering information quite useful, only a small number of which have been addressed in this paper. Due to the fact that the nature of this current study incorporates work with families as opposed to solely individuals, not all of these areas have been mentioned, and the focus will naturally fall on the use of qualitative methods in family therapy. Deacon and Piercy (2001) note ten unique advantages to using qualitative methods of assessment in family therapy: a) active self-reflection; b) assessment and therapy directly complement one another; c) qualitative assessment can be chosen to “fit” therapists’ theory; d) qualitative assessment is a shared assessment; e) qualitative assessment empowers; f) qualitative assessment increases the family’s commitment to the assessment and therapy process; g) qualitative assessment supports family communication and understanding; h) qualitative assessment provides a holistic, contextually rich sense of the family; i) qualitative assessment is flexible for use with diverse families; j) qualitative assessment uses the family’s own personal constructs. Due to the nature of the current study, some emphasis was placed in particular on the notions of self-reflection and empowering families. Qualitative methods of research give voice to the numbers generated by quantitative studies, therefore helping to create a more detailed and likely more accurate picture of any given individual’s experience (Gilgun, 2005). Not only can researchers gain a better understanding of that experience, but they can also gain insight into the effectiveness of therapy and the 25 therapeutic relationship. Bischoff and McBride (1996) state that “[understanding] client perceptions of treatment may lead to determining the most important aspects of treatment” (p. 117). As is also the case when working with individual clients, determining what components of family therapy have been effective and helpful is a key part of molding therapy into an experience that individually fits each family. It has really only been in the last fifteen years that research on this subject has opened up to include client perceptions and perspectives (Bischoff & McBride, 1996). Researchers are now finding that involving clients very directly in therapy assessment as well as therapist assessment is very helpful in engaging clients (Deacon & Piercy, 2001). Summary of Current Study The current study used a qualitative approach to explore what was helpful and what was not helpful about the Intensive Family Therapy Project which took place in Chilliwack, BC. This project was built on the basic tenets of MST and adapted the model to a local setting. The researcher used the Critical Incident Technique to examine client perspectives of their involvement in the project and will discuss this process in light of current research regarding the value of client perspectives in assessment and treatment planning. 26 CHAPTER 2: METHOD Critical Incident Technique In 1954, Flanagan devised a method for collecting and analyzing qualitative data called the Critical Incident Technique (CIT). The idea behind this technique is to gather qualitative information from observers of a specified activity regarding the effectiveness or ineffectiveness of that activity. This method of research “focuses on obtaining a comprehensive description of the activity under study” (Koehn, 1995, p. 21). Twelker (2003) notes that this technique can be aimed at identifying the “gaps” in a program or process (p. 1). An “incident” may be defined as “any observable human activity that is sufficiently complete in itself to permit inferences and predictions to be made about the person performing the act” (Flanagan, 1954, p. 327). An incident becomes critical when it has an overt impact on the outcome which is obvious to the observer, whether that impact is positive or negative. This technique is not a formal set of procedures to be followed precisely when gathering information, but rather “it is best thought of as a flexible set of guidelines which must be modified and adapted to meet a specific situation one is currently addressing” (Twelker, 2003, p. 1; see also Fly, van Bark, Weinman, Strohm, & Lang, 1997). It is a flexible interview approach containing “a set of procedures for collecting direct observations of human behavior in such a way as to facilitate their potential usefulness in solving practical problems and developing broad psychological principles” (Flanagan, 1954, p. 327). When he began his work, Flanagan (1954) used this technique to examine pilot performance as part of the Aviation Psychology Program of the United States Air Forces in World War II. He later continued to develop the technique by looking at job analysis, though it has been adapted to suit studies in a number of different areas. Wong (2000) mentions a few of 27 these areas, including but not limited to work motivation, professional performance of nurses and other health care workers, job requirements for various positions in the workplace, as well as job analysis in a variety of settings (Mitchell, 2001; Mullins & Kimbrough, 1988; Ross & Altmaier, 1990). Borgen, Amundson, and McVicar (2002) looked at the experiences of unemployed fishery workers in Newfoundland and Labrador. Amundson, Borgen, Jordan, and Erlebach (2004) use the technique in their study of organizational downsizing. In his study working with First Nations people in British Columbia, McCormick (1997) examines critical incidents to explore the facilitation of healing. In the area of ethics and psychology, Tryon (2000) studied the ethical practices of graduate students. Another key element to the technique is the provision of a judgment on the part of the individual being interviewed with regards to the effectiveness of the identified behavior or activity, referring to the critical nature of the incident. Andersson and Nilsson (1964) state that the technique identifies and analyzes incidents where “the holder of a position in a certain occupation has acted in a way which, according to some criterion, has been of decisive significance for his success or failure in a task” (p. 398). These judgments required of the observers are arguably very subjective in nature; however, the researchers see observers taking part in any particular study as experts in the area of the behavior or activity being studied, or they would not be counted as participants (Flanagan, 1954). These judgments can also generally be confirmed as being objective when several subjects agree on whether a behavior was effective or ineffective (Wong, 2000). Flanagan’s procedure. Flanagan (1954) specifies five basic steps in the procedure for conducting research using CIT: General Aims, Plans and Specifications, Collecting the Data, Analyzing the Data, and Interpreting and Reporting. Twelker (2003) outlines the same basic 28 steps, with some minor variations from those stated by Flanagan. In the following example, Flanagan (1954) illustrates the first procedural step which is the importance of clearly identifying the general aim of an activity: A supervisor’s action in releasing a key worker for half a day to participate in a recreational activity might be evaluated as very effective if the general aim of the foreman was to get along well with the employees under him. On the other hand, this same action might be evaluated as ineffective if the primary general aim is the immediate production of materials or services. (p. 336) It is important that the first step in the procedure be that of providing definitions of and identifying the point of the activity or behavior being studied. The second step is that of providing observers of the specified activity or behavior with instructions regarding what they are observing and how to report those observations to the researcher(s). Flanagan (1954) states that “[o]ne of the primary aims of scientific techniques is to insure objectivity for the observations being made and reported. Such agreement by independent observers can only be attained if they are all following the same set of rules” (p. 338). The next two steps outlined in the procedure are collecting and analyzing the data. The interview approach to collecting data is generally seen as more effective, though there are other approaches which can be used (Flanagan, 1954). In conducting the interviews, there are a number of issues which need to be addressed. Wong (2000) quite clearly identifies five criteria as follows: 1. An actual and complete behaviour was reported; 2. The behaviour was directly observed by the participant; 29 3. Relevant factors were given regarding the situation where the behaviour occurred; 4. The participant made a definite judgment regarding the behaviour; 5. The participant made it clear just why the behaviour was critical (p. 47-48). Following criteria such as these helps to ensure the data being collected is valid and relevant to the study. Where CIT is used, sample size is not actually based on how many people are interviewed. Rather, sample size refers to the number of incidents that are reported. When a participant begins repeating information, it can be assumed that enough incidents have been collected from that participant (Wong, 2000). Participants will eventually begin repeating each other, and it is at this time that the saturation point will have been reached. Once this point is reached, no more participants are needed (Wong, 2000). The process of analyzing data using this technique is also arguably subjective in nature. Once interviews have been completed, they are studied in detail so as to identify key incidents or events, as described by the observer, which have a significant impact on the outcome of the activity or behavior being studied. These incidents are taken from the observer’s own words, and once all the interviews have been studied in depth, incidents are grouped by similarity across all of the interviews. Category formulation then begins: Flanagan (1954) states, “[it] is a task requiring insight, experience, and judgment” (p. 344). Andersson and Nilsson (1964) note that “one can always refer to the source material. The essential thing seems therefore to be that the category system chosen is an obvious one, and with as small a degree of arbitrariness and chance as possible” (p. 400). No real set of formal guidelines has been established describing how this process is to take place, though researchers have agreed that one way to decrease the level of subjectivity is to “submit tentative categories to others for review” (Flanagan, 1954, p. 344; see 30 also Andersson & Nilsson, 1964; Koehn, 1995; MacKnee & Mervyn, 2002; Wong, 2000). Other researchers emphasize the notion that categories are basically fuzzy in nature, where no clear boundaries can really be defined between category members and non-members, which seems to underline the importance of reviewing the process with objective judges or raters who can review the process (McCloskey & Glucksberg, 1978; Rosch, 1978). The final step in Flanagan’s (1954) process is to interpret and report the findings of the study. The goal is to have collected sufficient descriptions of events or incidents so that the activity or behavior being studied has been thoroughly depicted. Once categories of effective and ineffective aspects of that activity or behavior have been established, one can ideally identify the “gaps” that remain, for example, in service or treatment (Twelker, 2003). The technique was intended to be very practical so that problems may be solved as a result of the findings garnered through this type of research (Flanagan, 1954). An important aspect of this final step is that any limitations that exist which may be connected to any part of the procedure for any specific study must be stated in order to “avoid faulty inferences and generalizations” that may be made based on the results (Flanagan, 1954, p. 345). Equally as important, the researcher must also make a judgment on the value of the results which were generated. Trustworthiness and applicability. In recent years, terminology used in CIT studies has changed. Andersson and Nilsson (1964) discuss issues of reliability and validity or generalizability, whereas current studies emphasize trustworthiness and applicability. In some cases these terms are used interchangeably, whereas in others they are differentiated. For the current study, these terms will be used interchangeably. These issues of trustworthiness and applicability are also expounded upon by Butterfield, Borgen, Amundson, and Maglio (2005) in their work on the Enhanced Critical Incident Technique. The data collection and analysis of the 31 current research were done prior to the development of this ECIT and the implications of this timeline will be discussed further in Chapter 4. Andersson and Nilsson (1964) did a foundational study on the reliability and validity of CIT. These are two key issues in looking at this methodology, as with many qualitative methods of research, because of the subjective nature of the data analysis process. In looking at the reliability or trustworthiness of qualitative data analysis, the use of several independent judges or raters is necessary in order to be sure that the categories identified and incidents placed in those categories could be replicated by another person. In order to attain an acceptable level of trustworthiness, there should generally be an agreement level of above 75% between judges (Andersson & Nilsson, 1964; Flanagan, 1954; Wong, 2000). If this level is reached, then the trustworthiness of the analysis process is acceptable, but if this level is not reached then the process may need to be refined. The validity or applicability of CIT can be tested in a number of ways. Andersson & Nilsson (1964) mention comprehensiveness and saturation when discussing applicability. Comprehensiveness refers to whether or not the data collected sufficiently describes or depicts the activity or behaviour being studied. If the data is comprehensive, then a saturation point has been reached and no more data needs to be collected. In their study on validity, Andersson and Nilsson (1964) took 5% of the incidents from each group of interviews they conducted and asked raters to sort the incidents based on their existing category system. The result was that by the time two-thirds of the incidents had been sorted, 95% of the categories and subcategories had been used. Wong (2000) refers to validity or applicability in qualitative research as “whether the results of content analysis are believable or trustworthy, whether they resonate with people who 32 have had some experience with the phenomenon under investigation” (p. 66). Five ways of testing the applicability of results are identified: a) confirmation by the participants, b) crossvalidation by other participants, c) cross-validation by an independent judge, d) cross-validation by other researchers, and e) saturation and comprehensiveness. The final method of testing varies slightly from that which was proposed by Andersson and Nilsson (1964) in that of the twenty-five interviews which were conducted, incidents from the last three interviews were withheld until the category system had been established. Incidents from the final three interviews were then sorted. If all the remaining incidents could be sorted into the existing categories, then the system would be deemed comprehensive, covering all the major aspects of the activity or behavior. Whether the results are tested through review by participants or other professionals, or through research previously conducted in the field, testing the results of any study using a subjective research method such as CIT is important (Flanagan, 1954; Wong, 2000). Andersson and Nilsson (1964) stated that “[a]ccording to the results of the studies reported here on the reliability and validity aspects of CIT, it would appear justifiable to conclude that information collected by this method is both reliable and valid” (p. 402). Participants All participants had to have been referred to the Intensive Family Therapy Project through the Ministry of Children and Family Development in British Columbia and Youth Forensic Psychiatric Services. Each client unit was provided three months or intensive individual and family sessions, after which therapy was considered complete. All participants in this study had completed the three months of treatment and all participants had completed therapy prior to the interview process for the current study. The therapy provided through the 33 project was completed after a minimum of three months of intensive sessions with the individual youth as well as the family. The number of sessions per week varied between families. Clients were youth between the ages of 13 and 18 (at the time of entry into the Project) who had been identified as young offenders and were at risk of re-offending. As previously noted, the youth had committed a variety of offences including sexual offences and acts of aggression, and some of the youth had committed multiple offences. The term client was also extended to those family members who participated in the Intensive Family Therapy Project. Family members could include biological or adoptive parents, grandparents or other adult relatives (ie. aunts or uncles), as well as anyone who was identified as a legal guardian of any of the youth. The clients were primarily Caucasian and some were Aboriginal. Though there was no attrition from the Intensive Family Therapy Project, there were a number of families who did not respond to the first letter inviting them to participate in the current study. Seven parent components and six youth responded to the letter of invitation and initially indicated they were willing to participate in the study. Interviews were scheduled from those numbers. In this study, nine interviews were conducted between September 2004 and July 2005; five parent/guardian interviews, and four youth interviews. Of the five parent/guardian interviews, all but one involved only one parent (the mother). The other interview involved a couple, parents of one of the youth. All of the youth interviewed were male, and between the ages of 15 and 22. Primary Researcher The primary researcher of the current study is a 34-year old female who was 24 at the time interviews were conducted. She had been involved in work with at-risk youth and their families for at least five years prior to conducting interviews for the study. This involvement 34 included both paid staff positions in agencies in New Brunswick and British Columbia, as well as practicum and internship experience in both provinces. In addition, the researcher had engaged in post-secondary and graduate studies where a focus of study was often placed on issues impacting or directly related to at-risk youth and their families, including undergraduate thesis work. The researcher had done extensive review of literature in preparation for and as part of the current study. A particular focus for this study, naturally, has been MST and its adaptations. As a result of these years of experience and study, the researcher had come to believe that MST had the potential to positively impact the lives of at-risk youth and their families, both in the short and long term. In addition, the researcher is a Christian and this identity has also informed her interest in programs that can help families as well as her belief in people’s capacity for change. If families are given enough support from varying angles as well as an opportunity to process their individual and family experiences such as MST might allow, the researcher believes they may be motivated to implement new skills and strategies in order to live more fulfilling and healthy lives. The researcher was interested in exploring clients’ perspectives of engaging in the MST process. In the course of the researcher’s graduate studies, the researcher was introduced to the Intensive Family Therapy Project which is defined as an MST-informed approach to working with young offenders. In addition, an opportunity arose to be involved in a study group focusing on the use of the Critical Incident Technique in psychological research. This exposure led the researcher to the decision to use the CIT method to examine participant experiences of the Intensive Family Therapy Project in Chilliwack, BC. This study group also served as independent raters for the current study. 35 Procedure Due to the nature of the study, in order to proceed with the interview process, agency permission was sought from both the Ministry of Children and Family Development, British Columbia, and Youth Forensic Psychiatric Services. Once permission was granted, initial contact was made with potential participants in September 2003 by the researcher’s supervisor by means of a letter sent to them in the mail. The letter contained preliminary details regarding the purpose of the study, and what was to be expected of them if they chose to participate. A sample letter can be seen in Appendix B. Approximately two weeks after sending the letter, the supervisor contacted potential participants by telephone to determine whether or not they were interested and willing to take part in the study. Once confirmation was received, names and phone numbers of those who had agreed to participate based on the initial contact by the supervisor were given to the researcher, with the permission of the participants. Interview times were scheduled by telephone at the convenience of each individual participant. Interviews The researcher conducted interviews between September 2004 and July 2005, following a semi-structured outline using a variation of Flanagan’s (1954) Critical Incidents Technique. The primary questions asked were, “What has been helpful to you about this therapy experience” and “What has not been helpful to you about this therapy experience”. Following these two questions, others were included so as to help gain a better understanding of each of the participants’ experiences. Other questions may have included, “Tell me about a time when ...”, “What was that like for you”, and “How has that affected your family”. For further listing, see interview protocol in Appendix A. Interviews were conducted with the youth separate from their respective family member(s). Interviews lasted between forty-five minutes and two-and-a-half 36 hours in length. The researcher continued as long as was necessary in order to reach the saturation point for critical incidents, or until the participant began simply repeating information already given. All interviews were recorded by audio tape. At the beginning of the interview, each participant was further informed as to the purpose of the study, as well as confidentiality concerning the information gathered and any analysis forthwith of that information. An Informed Consent Form was given to each participant who was then provided with the opportunity to read the form and ask any questions they might have about any aspect of the study. The two versions of Informed Consent Forms used (guardian and youth versions) can be found in Appendix C and Appendix D. It was also made clear that at any point, participants had the right to not answer anything that made them uncomfortable, or to stop the interview all together. The length of the interviews included clarification of purpose and informed consent as well as a debriefing period following data collection. All participants were again given the opportunity to ask any questions they had regarding the study or the outcome of the interview, as well as a chance to voice any personal thoughts or concerns that came up during the course of the interview. Time was allowed to ensure that each participant was stabilized again following the interview as some of the recollections shared caused a number of participants to become quite emotional. No further resources were needed in order to stabilize the participants (ie. outside counselling services). Finally, participants were told that if they were interested, a brief summary of results of the study could be provided in written form. Data Retention All of the audiotaped interviews will be destroyed after presentation and defense of this study. The anonymized interview protocols will be saved by the research team for future 37 research. This may allow for copies to be held in the archives of the Counselling Psychology Department of Trinity Western University, the Ministry of Children and Family Development, British Columbia, or Youth Forensic Psychiatric Services. Participants were given the option of having a follow-up interview and debriefing time where results of the study in general could be shared, though no copies of the protocols or the study will be offered to participants. Participants opted to have a brief summary of results sent to them by mail. Data Analysis The overall goal of the analysis was to determine the incidents in the Intensive Family Therapy Project that were helpful in the therapeutic process, and those that were not. Both positive and negative experiences were gathered from each interview participant. Each interview was transcribed verbatim, and positive and negative meaning units were initially identified. These units would be grouped together to mark incidents that occurred during sessions. Each protocol was read over several times so as to clearly identify significant incidents within each interview, highlighting them in the transcripts. Incidents were then taken from the transcripts and written on individual cards. On the reverse side of each card, incidents were marked with either a plus (+) or a negative (–) sign in the top left corner as well as an identification code (e.g. I2p8i3). The first two symbols identify the interview number, the middle two identify the page number, and the last two identify the incident number. The incidents were then analyzed and grouped together based on similarities. Following this, categories were formed based on the groupings, and each of the incidents was placed into those categories based on common descriptors used to code the incidents. Within each category, subcategories were also identified to further distinguish individual categories and incidents from each other. Once the categories were established and clearly defined, main themes of the 38 categories emerged. Trustworthiness & Applicability Because of the arguably subjective nature of this type of qualitative research, it was important that the researcher establish a certain level of trustworthiness and applicability for the current study. A procedure outlined by Andersson and Nilsson (1964) and adapted by Wong (2000) was used for the current study’s data analysis in testing both areas. The data for the current study was collected and analyzed prior to the development of the Enhanced Critical Incident Technique which includes nine credibility checks (Butterfield, Borgen, Maglio, & Amundson, 2009). The current study used five of these nine checks to assess trustworthiness and applicability. During the categorization process, all of the incidents from each interview, except for the last one, were sorted into categories. The last batch of incidents was withheld until the categories had all been clearly defined and divided into subcategories where needed. The idea with this type of test is that if the researcher had reached a point of saturation and comprehensiveness with the categories, the remaining incidents would be able to be sorted into the existing categories without the need to create more categories. In order to be able to state that the categorization process for this study is reliable, Flanagan (1954) suggests that a level of agreement between independent raters and the researcher must be above 75%. Six members of a research team acted as independent raters for this study. Five of the raters were Masters Psychology students who were all in the process of completing their own research projects using the Critical Incident methodology. The sixth rater currently holds a Ph.D. and is working in the field of counseling as a registered psychologist and as a professor. Results from the independent rater reliability check of the current study can be seen in 39 Table 1. The first rater agreed with the researcher on 90% of the categories. Agreement between the second rater and the researcher was 100% on the categories. The third rater had an agreement level of 80% on card placement into the categories. The fourth rater had an agreement level of 90%, and the final two raters each had an agreement level of 80% on categorization. The average agreement level between each of the six raters and the researcher was approximately 87%. These allow for the researcher to state that the categorization system for this study is sufficiently trustworthy. The trustworthiness of the study was enhanced by the fact that there was only one researcher who conducted interviews with each of the participants, and foundational interview questions were the same for each participant. The data was analyzed and categorized by the same researcher so that the level of consistency would remain high and there would be less allowance for misinterpretation of the data. 40 Table 1 The % of Agreement between Raters’ and Researcher’s Categorization of Incidents # of Incidents % of Agreement Rater A 10 90% Rater B 10 100% Rater C 10 80% Rater D 10 90% Rater E 10 80% Rater F 10 80% 41 A two-hour training session was held where the raters were briefed on the data analysis process as well as the categorization system for the current study. Sample incidents were drawn and used to walk the raters through the sorting process. This allowed for some definitions of terms critical to the study to be clarified for the raters. Following this session, the categories and subcategories were further clarified by the researcher. During a second session, the raters were given brief descriptions of the edited categories, as well as the opportunity to refer back to the definitions written on the reverse side of the category heading cards. They were also given subcategory headings for each category. Raters were instructed to place the incident cards under the category heading they thought best fit each of the cards. No instruction was given regarding sorting incidents into subcategories. Each rater was given 10% of the incidents identified by the researcher (10 cards/rater). They sorted their incidents independent of one another, and none of the individual incidents were sorted by more than one rater. Card placement was then checked by the researcher and compared to the original categorization results. Following analysis and categorization, another judge was consulted to further establish validity following testing methods identified by Wong (2000). Cross-validation by an independent judge was attained through consultation with one of the therapists who implemented the family therapy sessions with participants of the current study. Two points are detailed with regards to the validity tests performed on the data collected. The first looks at cross-validating the results of the analysis with previous research studies. The second point looks at saturation and the comprehensive nature of the data. Cross-validation. In looking at research done in the area of evaluation of the therapeutic process, comparisons were made between key aspects raised by the participants of those previous 42 studies and the participants of the current study. However, because very little qualitative research has been done in the area of Multisystemic Therapy and approaches similar to it, comparisons cannot be made directly regarding the current study; therefore, cross-validation of the categorization system for the current study based on prior research remains positive, but tentative. In cross-validating the categories established for the current study with those in related studies, similar categories were identified in each. In a study conducted by Paulson, Truscott, and Stuart (1999), clients’ perceptions of what was helpful in the counseling process were studied. Nine clusters or categories were identified, and of the nine, three categories match categories in the current study directly, and three others are similar though not exact matches. Another study done by Bachelor (1995) more specifically identified client perceptions of the therapeutic alliance, and again similar subcategories of characteristics are found in this and the current study. In their study of client perceptions of couples and family therapy, Bischoff and McBride (1996) found that three categories emerged, two of which match directly with categories identified in the current study. Of greater note is a study conducted by Tighe et al. (2012) on families’ experiences of MST. In their study, researchers recognized ten themes or categories. Each of the categories identified in the current study fit within the themes listed by Tighe et al. (2012). All of these previous studies appear to support the current system of categorization, though not conclusively due to the lack of qualitative research in the area of Multisystemic Therapy as has been previously mentioned. Saturation. During the process of categorization, the researcher sorted the incident cards into categories as they were formulating. Incidents from the last interview were withheld until the process had been completed and the categories had been identified. The researcher then 43 sorted the remaining incidents which had been withheld to ascertain whether or not the identified categories were sufficient for the collected data. All of the incidents from the final interview were successfully placed in existing categories, which suggests that the categories are comprehensive enough and cover the major tenets of the experiences of participants of the Intensive Family Therapy Project. 44 CHAPTER 3: RESULTS Upon completion and analysis of nine interviews, a total of 100 incidents were identified. Of those incidents, 78 were identified as being helpful and 12 were identified as being a hindrance. A third classification of incidents was deemed necessary as 10 of the 100 incidents were identified as being both a help and a hindrance. The following are two examples from this group of incidents: Example A: There was one time the therapist came and we were already fighting, and that didn’t really work. That was a crazy day. Neither me or my mom were willing to work probably. I think that was it, because I know I didn’t really want to be in the same room as her. I was really angry. I think I must have been having a bad day, because everything was getting on my nerves. I was just kind of ignoring the therapist. That was the only time it didn’t work, nothing really got resolved. We fought a lot. I think we talked about it in the next session, though. It’s like he gave us the time we needed to cool down a bit, and then we were able to talk about it. Example B: We didn’t really have a closing session. Like, we never really did say goodbye. I don’t really know how I would have taken it if we would have had one, because I didn’t really want to let go, and it was great having the therapist around. I kind of wish I had had that last session. But, sometimes I’m happy I didn’t because I’m not the best at saying goodbye. The 10 incidents in this cluster identified qualities which were both helpful as well as a hindrance. 45 In analyzing the incidents and sorting them into categories, it also became clear that the incidents did not fit neatly into main themes of ‘Helpful’ and ‘Hindering’. Most of the categories that had been identified contained incidents of both types. Once the analysis had been completed, seven categories had been identified. Those groups were further divided into 26 subcategories in total. The major themes of the identified categories were ‘Features of the Intensive Family Therapy Project’ and ‘Results of the Project’. Five of the categories were placed under the first theme (Family Sessions, Teams, Resources, Therapeutic Traits, Technical Aspects of the Program). The last two categories (Skills Learned, Post-Program Growth) were placed under the second theme. Table 2 lists the categories within the major themes, the subcategories within each category, as well as the number of incidents and participation rates for each subcategory. Description of Categories The following will present descriptions of the categories compiled during the analysis of the interviews. Each category will be defined and subcategories listed with frequencies of incidents identified in brackets. One example of an incident will be provided for each subcategory and each example will be identified as being helpful or hindering. The client group from which the incident was gathered (parent/guardian interview or youth interview) will also be identified. The categories are presented randomly as no significance has been placed on order. Unless otherwise indicated, the frequencies contain solely helpful incidents within that subcategory. 1. Resources (17 incidents) This category has been defined as resources used either to become part of the team, or by various members of the team. This included workers from the various agencies that were part of 46 Table 2 Participation Rates for Subcategories Rate of Pos. Inc. Rate (per category) Subcategory Resources Youth Forensics 5 56% The Ministry 2 22%* Tools Used 3 33% Specialists 3 33% Total # Inc.: 17 Support Workers 4 44% Teams Team Effectiveness/Cohesion 4 44% Total # Inc.: 6 Team Conflicts 2 22%* Therapeutic Therapist Qualities 6 67% Traits Therapist Action 5 56% Total # Inc.: 15 Follow-Through 4 44% Post-Program Courses Completed 2 22%* Growth Personal Growth 7 78% Total # Inc.: 13 Family Growth 4 44% Skills Learned Anger Management 3 33% Saying “No” 2 22%* Total # Inc.: 15 Communication/Expressing Emotions 10 111% Technical Accessibility 3 33% Aspects Flexibility 4 44% Duration 2 22%* Closure/Ending 5 56% Follow-up 3 33% Total # Inc.: 17 # of Inc. Participation Category 82% 67% 93% 100% 100% 82% 47 Rate of Pos. Inc. Rate (per category) Subcategory Family Goals 1 11%* Sessions Involvement 3 33% Role Plays 2 22%* Family Conflicts & Resolution 7 78% Perspective 4 44% Total # Inc.: 17 # of Inc. Participation Category *Rates are not high enough to justify inclusion of category (see Butterfield et al., 2005) 88% 48 the Intensive Family Therapy Project as well as other professionals from the field of psychology. Team members also made use of tools as part of their work with the families which took the form of books, homework sheets and other information. The category was divided into five subcategories. Youth Forensics (five incidents; four helpful, one hindering): Example Most of my meetings with Forensics were in group format, though I did have oneon-ones, too. We had one-on-one sessions and the counselor sort of picked my brain about things like, ‘Well, why did you say this in group, and why did you say this in group.’ And ‘Oh, this is really good that you did in group, and this is really good. How are things at home’ and stuff like that. ‘Cause how things are at home is really none of everybody else’s business. So, we had the things that were contributing to my problems in group, what sort of problems were caused by my home life kind of thing. And then she could sort of talk about that in group without having to mention names and stuff like that. And if I want to go back to see the counselor even now, I can do that. (Youth interview, helpful incident) The Ministry (two incidents; one helpful, one hindering): Example We had a session at the Ministry office, and we got a lot of input from other professionals. There were a few people behind windows, and they would kind of watch us. And then they would actually tell us what they found about our family. Like, how caring we were. I was really amazed at how they saw us. We didn’t see ourselves that way. It was really, very helpful. I think we should think about what they said more often. And then we went behind the window and they talked 49 about us, and I couldn’t believe what they were saying about us. I was amazed with what kind of things they came up with that we didn’t even see ourselves. At first, even my children were kind of thinking, ‘Oh no, that’s not it.’ We were kind of pessimists about it. These people came with a positive outlook. We are the negative, and they were the positive. And, I think it was good for the children to hear, too. It was really good, I think. (Parent/guardian interview, helpful incident) Tools Used (three incidents): Example Something that stuck really well, I can’t even remember exactly what it was, but I remember this one work sheet. I lived by that for awhile. It worked. I can’t even remember what it was. I don’t think I still have it. But, I know there was something on there that just worked for me. I kept it to remind myself, but I don’t even remember what that was now. I think it had to do with something about coping with my problems. And not taking stuff from the past to get me angry in the present. (Youth interview, helpful incident) Specialists (three incidents): Example We had one session with a specialist (EMDR/OEI) where we worked on some of our past personal issues, that was part of the program. Things from that session have never been an issue for me, have never been that gut-wrenching stuff that they were at that moment that we did that session. And the fact that you don’t even have to say what it is, but he still can help you with it is amazing. (Parent/guardian interview, helpful incident) 50 Support Workers (four incidents; three helpful, one hindering): Example I had a couple of youth workers on my team. I only went out with them a couple of times. It was pretty useless for me. The thing is at the time I had my own car, so I had my own things to do. A lot of these guys that were in these kind of youth groups and stuff like that didn’t have that access, they were either in jail or didn’t have cars or whatever. So they needed some form of entertainment. And then when I was out with the support workers, it just sort of felt really uncomfortable. It didn’t work. (Youth interview, hindering incident) 2. Teams (six incidents) This category dealt with elements of working as part of a multidisciplinary team. As previously presented, the teams that were the foundation for the Project were often composed of several professionals from different settings, though not every team was composed of members from each of the organizations. The category has two subcategories. Team Effectiveness/Cohesion (four incidents): Example We had a team-based meeting where we would just run down everything and make sure that everything is still on track. The meetings were consistent, and our main focus was my daughter, and that’s where it should have been. The therapist was able to engage everybody who was there. Everybody had a say and everybody said what they had seen, what was going on, and then we all agree on certain things, if she needed more counseling, or what the next step was, and our goals, and so on. I thought the meetings were great. I thought that they were 51 important, and we all could talk freely. I mean, if I was having a problem with my daughter, or she had taken off, or whatever, everything was straight forward. Everything was out on the table. There was honesty. Sometimes we didn’t want her probation officer knowing certain things because we didn’t want her going back to jail, so we would deal with it as a team. If it was something major, of course we would have involved the probation officer. But everybody screws up once in a while, and the team was pretty good with that. (Parent/guardian interview, helpful incident) Team Conflicts (two incidents; two hindering): Example I was just annoyed with the workers I think. That’s pretty much what is was, because there were just so many at that time. I didn’t even know half their names. I had so much stuff going on then. Every day I had DARE workers, ISP workers, some other worker, and I had people from the Maples involved then still. I think I was in Youth Futures. I was just constantly busy. (Youth interview, hindering incident) 3. Therapeutic Traits (15 incidents) This category looked at elements of therapy that either facilitated or hindered the therapeutic process itself. Much of what was described in the incidents revolved around what the therapist did, or how the therapist displayed certain characteristics. There were three subcategories. Therapist Qualities (six incidents): Examples The therapist was a good listener. If we ever had a problem, we’d sit down and chat about it in sessions. He was good at listening to both sides sort of thing, and 52 not just the one. So, that’s something that I enjoyed about him. The therapist never took their side over mine, or my side over theirs. He was good at that. And I think getting us all together with the program was key. (Youth interview, helpful incident) Therapist Action (five incidents): Example Just the way the therapist was made me feel comfortable enough to even open up. It was her period; her gentle tone, her understanding of everything. She never once condoned or condemned. She came in whenever I asked, basically. Whenever she could, and I asked and I needed someone to talk to, she was there. Just the fact that she showed up and had the kindness in her heart to not condemn me for my reactions that may have not been correct, or my actions. My oldest daughter was on speed at times. So when I was losing it, the therapist would show up and help me by taking us down to the Ministry, whether we sat outside and just watched my daughter come off her high. She just kept me calm, or kept me seeing reality instead of going off into blame or anger, or whatever. That lady went even beyond the call of duty to help me and my family. (Parent/guardian interview, helpful incident) Follow-Through (four incidents; three helpful, one hindering). Example One thing that happened was because it was right at the very beginning of the program when we started. I believe we were the first family. And as other families came into it, some of our appointments got canceled. There was even a couple of times where there was no phone call, which I found I was hurt. I actually felt abandoned. The first month was great, but then, you know, the 53 therapist had other families, and then they wanted him to write a paper. And then the Ministry assigned him to do something else, and then they assigned him families out of town. And that’s what happened, and it wasn’t the therapist’s fault. But, it affected us. (Parent/guardian interview, hindering incident) 4. Post-Program Growth (13 incidents) This category was defined as growth that occurred upon completion of the program, not specifically as part of the program. This included extra work that was taken on by family members after therapy had come to an end as well as areas where they felt they had really experienced change, either as individuals or as a family. All of the incidents placed in this category were positive, or helpful, in nature. There were three subcategories under this heading. Courses Completed (two incidents): Example As a result of the work I’ve done with the therapist, I was able to take on something extra. I just finished Anger Management class. I did graduate from it, the first women’s Anger Management class. This was the first time I’ve been able to handle anything extra. (Parent/guardian interview, helpful incident) Personal Growth (seven incidents): Example Getting involved in the community, and how that can help me was something I learned. Now, I can phone up my friend at any time and I can talk to him for a good 20 minutes. Before, I wouldn’t. I didn’t really like talking on the phone. I didn’t know what to say to anybody. Now, I go to the leisure center every 54 Sunday. I’m starting to get lots of friends. That definitely wouldn’t have happened if it wasn’t for this whole experience. It was the therapist who made me promise that I would get a hold of my friends. (Youth interview, helpful incident) Family Growth (four incidents): Example We’re able to laugh and joke now as a family, and not take each other’s word so to heart. We call each other weirdoes and freaks. We’re not saying that because we’re feeling that those are real, those are just words we use. We don’t take it weird anymore, or bad. To the heart. It’s a joke, and we can handle it. (Parent/guardian interview, helpful incident) 5. Skills Learned (15 incidents) This category refers to the skills that were picked up during and as a result of the program. Some of the skills mentioned during the interviews were individualized, but the majority of skills had a more direct impact on family functioning. As with the previous category, all of the incidents classified here were helpful in nature. The category was divided into three subcategories. Anger Management (three incidents): Example I had a problem with my anger. I’d lose my cool quite easily. Not always. And the therapist helped me bring that down quite a bit. He helped me come up with new solutions, what might help, other things I can do instead of blowing up. (Youth interview, helpful incident) 55 Saying “No” (two incidents): Example I think the therapist made me stronger to tough love instead of just being my daughter’s friend. She started taking off doing crack cocaine, coming in on her withdraws, taking stuff from me. Bringing over her boyfriend, eating everything out of house and home when that’s what I have for the week. She was coming home every couple of weeks high, or coming down, and in the beginning it was ok. I’d break myself trying to keep her home. Make sure that we had pizza night, movie night, costing me a fortune to keep her at home and entertained. It got to the point where, because of what I learned from the therapist, I was able to say, ‘No more’. I learned how to be able to use tough love with my daughter. (Parent/guardian interview, helpful incident) Communication/Expressing Emotions (10 incidents): Example I learned how to talk to my family, and it really helped me a lot. How to talk to them when I’m having problems. How to get around arguments really helped. We used to have so many arguments, and now we’re lucky if we have one argument a month. It helped a lot. I learned that I don’t need to shut down when I don’t like what’s going on. It’s best to talk about it. (Youth interview, helpful incident) 6. Technical Aspects of the Program (17 incidents) This category looks at technical elements that affect the therapeutic process, but are not an overt part of therapy itself or a result of the process. The category covered everything from timing concerns and issues of setting, to how easy or hard it was to get a hold of the therapist. 56 This may be seen as a category into which all of the mis-fitting elements were placed, but the elements are all technical aspects of the program. There were five subcategories in this category. Accessibility (three incidents): Example The accessibility. If there was a problem, we could get a hold of the therapist. Being able to phone and talk through things. That was really important, too, cause that’s another thing that you don’t get with a lot of other programs. And even in the evenings. It wasn’t limited to the workday. That was really important. Now, it wasn’t something that I took advantage of, but it was something that I valued, when a day came, or a night came that I needed that, it was very, very important in order to feel comfortable. (Parent/guardian interview, helpful incident) Flexibility (four incidents): Example The fact that the therapist came to my home and I was in my own space really made things a lot easier. I could never feel comfortable enough, we could never have done what we did in an office setting. (Parent/guardian interview, helpful incident) Duration (two incidents; one helpful, one hindering): Example The program wasn’t long enough. I believe it should be minimum two years. I just really do, because that might have helped me keep my oldest daughter, too. But when the therapist left, my daughter left. And I can’t blame the therapist, and I’m not saying it’s anyone’s fault. I’m just saying that might have been a better situation in my case. I might have been able to hold on to my daughter. She might have been able to see a little more light, because she was still cloudy when 57 it ended. She hadn’t seen the progress in me yet, to feel it. And if she could have felt it, it might have made her grounded a little better. That’s all. (Parent/guardian interview, hindering incident) Closure/Ending (five incidents; three helpful, two hindering): Example Another issue was when the Ministry said “ok, that’s it”. We were going to have a phasing out period. That didn’t happen. The Ministry just said the government pulled the bucks, and away it went. So, the way it ended wasn’t good for me. I felt that there was no closure. And I felt like we needed something more, that the end should have been a review of everything we learned. And some reinforcement the therapist that said “This is where you guys were, this is where you guys are now. Look at what you’ve done, look at how far you’ve come. This is what I see, and way to go guys. You worked really hard”. Ending, just ending was not good for me. It wasn’t a healthy way to end it. I have abandonment issues. My son has really big abandonment issues, and we were kind of left with an abandonment. We didn’t have that last, that closure, that last meeting, you know. It didn’t come to fruition that way. We haven’t ended it. (Parent/guardian interview, hindering incident) Follow-Up (three incidents; two helpful, one hindering): Example I feel that they’ve left us now in the cold. I think they should have more followup. It’s very important. They helped us for a time, and all of a sudden that seemed to just have stopped because the court session was finished. I think that shouldn’t stop so abruptly. I think that follow-up is very important. Where do we go after this counseling? The ending was too abrupt. I wasn’t happy with it. I 58 wasn’t even finished, actually. I needed actually a couple more sessions to end, to round it off, but we didn’t have that. The wrap-up was not good. We didn’t have enough time, I find that’s really actually a big cut in the family again, such an abrupt finish. (Parent/guardian interview, hindering incident) 7. Family Sessions (17 incidents) The category is defined as aspects of sessions held with the identified client (the young offender) and at least one other family member in working with various team members. The category was divided into five subcategories. Goals (one incident): Example At the time, I was in the midst of being booted from my house. The one goal that wasn’t met, which I didn’t really mind, was getting me back into my house. But the other goal was to be able to sit down and chat with my mom and my step-dad and not be at each other’s throats during the conversation. That was met. The situation with my anger, finding other solutions for that, that goal was met too. So pretty much the only one really that wasn’t met, which wasn’t a major concern, was getting me back in my house. But it was agreed upon by all three parties that it would not be the best situation for me to be put back into. (Youth interview, helpful incident) Involvement (three incidents): Example At times I couldn’t even concentrate on the meetings, so I would just look like I was tuned in, but I wasn’t really grasping what was being said. And as the 59 meetings went on, that changed. I got more and more involved, which was a good thing. It really helped. Before, when somebody new came in I’d just sit on the couch and just listen to what they had to say. Now, I’ll just join in the conversation. (Youth interview, helpful incident) Role Plays (two incidents): Example Actually, the one thing I did see was when the therapist did that one session with us, and we had to move all the furniture, and more or less he would place us in a certain position in the family, and there was a line. And one of our sons was outside that line, and another was kind of outside that line. And, it was a real eye opener. Actually, it cut my heart. I didn’t realize our situation was so serious. I was able to visualize. The visual aspect was really helpful, I think. I think it really had a big impact on me. (Parent/guardian interview, helpful incident) Family Conflicts & Resolution (seven incidents): Example There were some sessions where the therapist came in on some hot situations. It was just everybody was arguing sort of together, and nothing got resolved during the session. I think it was too many personal attacks against individuals, people getting defensive. The therapist just tried to calm everybody down and it didn’t work and then he said, ‘You know what, we’ll just save this for another day,’ and talk about it. Everybody was too riled up. When we came back to it in the next session or whatever, the therapist sort of got everybody’s feelings, how they felt about it, and thinking what was the smart thing to do about it. He would sort of 60 see what our views on it were, and then sort of gave his opinion. (Youth interview, helpful incident) Perspective (four incidents): Example One of the good things that happened was the therapist identifying for me issues that I didn’t realize I had, and identifying for me where my son was coming from when I wasn’t aware of that. That was really good. Helping me realize where I was coming from. Another thing I remember was one day we were sitting here talking about privacy, and I was looking at it that I’m the parent, and if I want to walk in your room I will. And not looking at it from my son’s side as a 15-year old boy. And the therapist helped me, helped us look at that and what would you do to change it. (Parent/guardian interview, helpful incident) Summary Results of the data analysis indicate participants were able to identify more helpful incidents than hindering. The two main themes which emerged were ‘Features of the Intensive Family Therapy Project’ and ‘Results of the Project’. There were seven categories into which the incidents, 100 in total, were placed. ‘Family Sessions’, ‘Teams’, ‘Resources’, ‘Therapeutic Traits’, and ‘Technical Aspects’ were categorized together under the first theme. ‘Post-Program Growth’ and ‘Skills Learned’ were placed in the second category. 61 CHAPTER 4: DISCUSSION The current study was conducted in order to examine from families’ perspectives what was helpful or hindering about their experience of the Intensive Family Therapy Project. The Critical Incident Technique was used to analyze data gathered through interviews with members of six families who participated in the project. Analysis of the data revealed seven categories or themes into which 100 incidents were placed. These results are discussed below in light of current research on the therapeutic process, as well as trends in family therapy and treatment of at-risk youth. The results generated from this study appear to be consistent with existing concepts of what is helpful in the therapeutic process. In their study, Paulson, Truscott, and Stuart (1999) identified nine clusters or categories of helpful characteristics that are part of the counselling experience. Tighe et al. (2012) generated 10 themes described by family members in a study exploring their experiences of MST. The current study identifies seven categories, several of which match directly with previously identified clusters or themes, and some which are similar in composition. Following is a list of categories derived from the data of the current study: 1. Resources 2. Teams 3. Therapeutic Traits 4. Post-Program Growth 5. Skills Learned 6. Technical Aspects of the Program 7. Family Sessions Each category will be presented in the same order in which they were presented in Chapter 3, as 62 well as in the list provided above. Each will be discussed in light of previous research, followed by a statement of some of the limitations to the current study and implications for future research and practice. The first category is ‘Resources’ into which 17 incidents, both helpful and hindering in nature, were placed. This was one of the largest categories, noting that a high number of critical incidents were recalled regarding what or who was brought into the therapy experience in order to assist the participants in the process. This category matches that of Paulson, Truscott, and Stuart (1999) labeled ‘Generating Client Resources’. Participants found it particularly helpful when added resources were brought in to help support them. In general, they appreciated that the program aimed to work with them on whatever level they found themselves at, whether it was dealing with trauma, everyday conflicts within the home, or the consequences of an offense that has been committed. In describing the theme ‘Holistic Approach’, Tighe et al. (2012) state that participants “spoke of therapists proactively linking up with extended family and other professionals to put in place a wider network of support for the parent and to open up new possibilities for the young person” (p. 191). The subcategories within the first category of the current study were, for the most part, particular to the Intensive Family Therapy Project and may not be generalizable to other specific therapeutic approaches. However, the notion is the same that participants had positive experiences with added resources brought into therapy. In this study, 82% of the incidents were clearly identified as positive in nature, and those that were negative were not necessarily due to extreme events that occurred which drastically altered therapy. In one family, the parent component found a session held with the Ministry to be very helpful whereas the youth component of that same family found the family session to be incredibly uncomfortable and not 63 helpful. One participant was assigned youth support workers to help during the process, as most of the youth are, but the youth found that the support workers were actually not needed, and therefore they became more of a hindrance. This particular youth had a number of other team members involved, and it was felt that there were too many professionals working with the team. The support workers were not necessary in this particular case. The second category, ‘Teams’, contained the least amount of incidents. In looking at the distribution of helpful and unhelpful incidents, the highest reported ratio of helpful incidents to unhelpful incidents was found within this category (4:2). This translates into approximately 33% of the incidents being unhelpful. This high percentage can be mostly attributed to the relatively small sample size in terms of incidents in the category as compared to other categories. However, this may also indicate that the items identified in this category may be key elements to positive experiences in therapy or counselling using this particular approach. Previous studies do not account for a category of this type as it is also very specific to the therapeutic approach used in this project. Creating a new category was warranted as six of the incidents provided by participants dealt directly with the team-based method (not team members themselves) of working with the youth and their families. The team-based focus of the project is a foundational concept of this particular approach to therapy, as noted within the tenets of Multisystemic Therapy itself. Each team may consist of various members, and rarely will a team have exactly the same make-up from family to family. Therefore, the creation of a new category based on the importance of this element of the program seemed appropriate. Paulson, Truscott, and Stuart (1999) identify two clusters in their study labeled ‘Counsellor Facilitative Interpersonal Style’ and ‘Counselor Interventions’. In the current study, these two clusters are combined into one category labeled ‘Therapeutic Traits’. Tighe et al. 64 (2012) identified a theme as ‘Strong Therapeutic Relationship’ where participants discussed various therapeutic characteristics such as “empathy, understanding, and genuine care” (p. 191). In each of the studies, these clusters or categories describe specific qualities of the therapist and what action was taken by the therapist. With regards to the current study, all of the incidents in this category were helpful in nature with the exception of the subcategory ‘Follow-Through’. The results here are also supported by work done by Bischoff and McBride (1996) where they examined what was helpful and not helpful in couples and family counselling. Two of the three categories identified were “perceived helpfulness of various therapeutic techniques” and “therapist empathy and other ingredients of good therapy” (p. 117, 119). In addition, Bachelor (1995) looked at client perceptions of the therapeutic alliance, and several therapist qualities were also identified by participants such as “nonjudgmental”, “listens”, and “competent” (p. 329). These same qualities were highlighted in many of the incidents from the current study that were grouped into this category. The next two categories were the only ones comprised solely of helpful incidents. The first of the two is ‘Post-Program Growth’. Some of the participants indicated that once they had completed the program, they were able to move forward and engage in other programs not associated with the project, and complete the requirements successfully. They attribute these events to their involvement in and growth through the project. There were also marked improvements in family functioning for a number of participants, as well as individual functioning. Tighe et al. (2012) identify a theme labeled ‘Increased parental confidence and skills’, which relates specifically to the subcategories ‘Personal Growth’ and ‘Family Growth’ within the current study. Some participants noted that the changes they had seen following their involvement in the Intensive Family Therapy project may have become less apparent over time, 65 but they still state that the changes were positive and could illicit positive growth even now if they were more careful and intentional about applying what was learned. Perhaps interesting to note was the fact that some of the interviews conducted incorporated both the youth component and the parent component of the same family. In this category, some of the incidents depicting positive growth were stated by both components, which would indicate a strength regarding outcomes of the project. The other category which contained solely helpful incidents was ‘Skills Learned’, into which 15 incidents were placed. As with the previous category, the theme of ‘Increased parental confidence and skills’ identified by Tighe et al. (2012) mirrors elements of this category. In addition, their themes labeled ‘Solution focused practical approach, providing observable benefits’ and ‘Relationship improves’ identify elements which improved over the course of treatment such as better communication, less anger and conflict, consistent discipline, and closer relationships within the family unit. These themes match one of the clusters identified by Paulson, Truscott, and Stuart (1999) labeled ‘Gaining Knowledge’. Some of the items presented in this cluster included talking things out before action is taken, and learning how to deal with anger. Tighe et al. (2012) state parents felt they had a greater sense of control, consistency in parenting, and improved mental health. Three subcategories were identified within the group in the current study, the largest of which was ‘Communication/Expressing Emotions’. Participants indicated that not only did they learn how to communicate with others, specifically family members, but they were also able to observe the effects of open communication and the benefits this skill presented to the unit as a whole when applied. Again, marked improvements in functioning were noted throughout a number of the interviews with regards to elements of this category. In fact, of the nine interviews conducted, seven identify incidents grouped in this 66 category, six more specifically within the Communication/Expressing Emotions subcategory itself. Participants appeared to have higher incidents of negative experience regarding items contained in the category titled ‘Technical Aspects of the Program’, as indicated by the higher frequency of hindering incidents placed in the category. Of the 17 incidents placed in the category, four of them were unhelpful in nature. This translates into approximately 24% of incidents for the category being unhelpful. As a result, this group provides perhaps the most insight into where gaps may exist in programming and therapy. Elements such as the duration of therapy in general, accessibility of the therapist (also noted by Paulson et al., 1999; Tighe et al., 2012), and wrap-up are all components of this category. It was felt that there were almost always acceptable levels of access and flexibility regarding the occurrence of and settings for sessions. However, participants also recalled incidents where follow-up to therapy was never conducted, and where a positive ending to therapy, including a sense of closure, never occurred. This sense of lack of closure and positive ending was also identified by Tighe et al. (2012) where participants stated the ending was “too abrupt or too early” and that they “would have preferred a more tapered approach to ending” (p. 194). Though there is always the potential for extenuating circumstances to hinder a real sense of closure or a solid ending to therapy, these are important elements which were lacking for some of the participants of the current study. More attention and perhaps even alterations may be warranted in the future regarding this technical aspect of the program. The final category, ‘Family Sessions’, could encompasses several aspects of a number of other categories, but because of the nature of the Intensive Family Therapy Project and the fact that family sessions were only one component of the program, it was felt that a separate category 67 was warranted. Elements of the clusters put forth by Paulson, Truscott, and Stuart (1999) can also be traced within this category, including one labeled ‘New Perspectives’. Tighe et al. (2012) identified a subtheme within the theme ‘Holistic approach’ where participants recognized the effectiveness of involving family, specifically parents, in the therapeutic process. Participants in the current study were able to identify elements of various techniques used by the primary therapist, such as the use of role plays, which had differing impacts on the families involved in therapy. Perhaps the biggest area impacted by the sessions was that of conflicts and resolving disputes in more helpful ways within the family unit. In addition to these findings, a number of incidents placed in the subcategory ‘Family Conflicts & Resolution’ were deemed to be both helpful and unhelpful in nature. An explanation for this may be that in several instances, the therapist came into a setting where family members had already been arguing, sometimes quite heatedly. The way in which the therapist handled these particular sessions was not helpful at the time. However, when the therapist came back for subsequent sessions, the issue was almost always addressed and resolved in a very helpful manner. Participants seemed to appreciate that the therapist would give them the space and time they needed in order to be able to cool down and think and behave more rationally. Overall, in this category, approximately 88% of the incidents were helpful in nature. It is important to make note of the hindering incidents that were identified by participants, particularly with regards to the categories into which they were placed. It has been stated that an overall result of using the Critical Incident Technique may be to help identify gaps that may exist in a particular program or service being offered (Twelker, 2003). At this preliminary stage, gaps in programming for the Intensive Family Therapy Project may be identified by areas in which participants noted hindering incidents occurring. Two categories 68 worth highlighting again are ‘Therapeutic Traits’, more specifically the subcategory ‘FollowThrough’, and ‘Technical Aspects of the Program’. A goal of the current study was that the results might help to corroborate or enhance results gathered from quantitative research done with the same Intensive Family Therapy Project. The researcher expected to find a higher emphasis on what has been helpful through the therapeutic process, and this expectation was supported. Table 2 lists the rate of positive incidents found in each category. In fact, two of the categories identified contained solely helpful incidents; ‘Post-Program Growth’ and ‘Skills Learned’. These appear to be significant categories for the participants regarding positive outcomes, and they may also suggest that results of the Intensive Family Therapy Project in general are positive. These categories are comprised of what the participants learned through the process as well as how they have grown since completion of the program, which suggests that elements of the program, if not the program as a whole, are effective in bringing about change toward more positive functioning. This distinctly positive leaning in the results of the current study may also speak to the issue of the relevance and applicability of adapted forms of MST to any setting. Intensive Family Therapy is still relevant to MST in that it supports the notion the tenets of MST can be adapted and still be effective. The argument can be made that this theory is also supported by Leschied’s (2003a) findings in Ontario which indicate a positive leaning in results of their trial study. The emphasis on positive results was somewhat expected as the data that has been collected quantitatively for the same Intensive Family Therapy Project has shown positive trends with regards to improvements in both the identified clients as well as the family units. Seeing as the current study was intended to partner these quantitative studies, there was reason to expect an 69 emphasis on the positive. The consistency of the results is supported by a few factors. All of the families that took part in this study had been counselled by one of two primary therapists. It was the primary therapist who oversaw each element of the Intensive Family Therapy Project and who was to have the most consistent contact with the identified client or youth. The second factor which supported the consistency of the results was that the interviews and data analysis were all conducted by one researcher. Though the interviews were semi-structured in nature, only one interview protocol was used regarding foundational questions to be asked with each participant. Limitations The use of qualitative methodologies in research has both its strengths and limitations when looking at the larger field of study in any discipline. Rather than discuss the nature of a qualitative study regarding its generalizability across a larger population, the discussion turns more toward relevance to both research and application. To briefly address the issue of generalizability, there are a number of components to the current study which present limitations to the overarching generalizability of the findings to all participants of the Intensive Family Therapy Project. Because of the complexity of the Project in regards to the many components of the program, comprehensively covering every aspect of a study of this particular size would be almost impossible. The transient nature of the target population makes the families difficult to reach, and those who had a negative experience through the process may be dis-inclined to give any more time to the project. Also, the data collected was presented in the voices of the participants themselves from their own perspectives and therefore will not necessarily be generalizable to all those who were part of the Intensive Family Therapy Project. Having said that, inferences can still be made regarding the overall positive impact of Intensive Family 70 Therapy. In addressing the issue of the relevance of the results from the current study, a number of points may be made. Because the Intensive Family Therapy Project was based on the tenets of MST, some question may be raised regarding relevancy to MST itself. This particular therapeutic approach places a strong emphasis on following the principles outlined in Chapter 1 and on close, direct supervision of the therapist trained and certified by Henggeler and associates. There was no fidelity check built into the Intensive Family Therapy Project in Chilliwack in relation to MST principles and practices other than regular supervision. The question has been raised in the field whether the principles could be adapted and implemented in new contexts without the stringent controls of those trained by Henggeler. The preponderance of positive critical incidents in the current study seem to provide some evidence, at least from the clients’ perspectives, that an adaptation to local conditions through an MST informed approach can succeed. When considering the preponderance of positive critical incidents, it is worth noting that this is from a population that is often multi-barriered and does not feel positive about government services. The relevance of the current study can also be discussed in light of the broader subject of knowledge acquisition and exchange. Conducting research with the aim of gaining an increased understanding of an individual’s experience of any event will result in acquired knowledge. What remains to be determined is whether that knowledge is relevant to a specific therapeutic setting, group of people, or whether it leads to the possibility of further study. Whatever the case, the knowledge gained should be open to both interpretation and adaptability. Applying this notion to the current study, an argument may be made that knowledge was acquired through the processes of interviewing participants and analyzing those interviews. This knowledge can then 71 be used to enhance discussions regarding the relevance and adaptability of MST, as well as the therapeutic alliance or the counselling process in general. Trustworthiness and applicability. As mentioned in the Method section, a number of credibility checks were performed during the data analysis process of the current study to determine the trustworthiness and applicability of the results. These checks mostly followed a procedure outlined by Andersson and Nilsson (1964) and adapted by Wong (2000). The current study used three of the five checks identified by Wong (2000), which were cross-validation by an independent judge, cross-validation by other researchers, and saturation and comprehensiveness. The researcher adapted another validity check in that instead of cross-validating the results with any of the participants of the study, the researcher consulted with one of the primary therapists of the Intensive Family Therapy Project to determine if the categories established by the researcher were valid. Research conducted using the CIT method prior to 2004 varied significantly specifically with regards to the number of reliability and validity checks performed and which checks were chosen and used by various researchers (Butterfield, Borgen, Amundson, & Maglio, 2005). In their research on the CIT and its use over the previous 50 years, Butterfield et al. (2005) discussed nine credibility checks being performed consistently on CIT research coming out of the University of British Columbia. The team expanded on the application of these nine checks, and they are identified as follows: a) audiotaping interviews, b) interview fidelity, c) independent extraction of critical incidents, d) exhaustiveness, e) participation rates, f) placing incidents into categories by an independent judge, g) cross-checking by participants, h) expert opinions, and i) theoretical agreement (Butterfield, Borgen, Maglio, & Amundson, 2009). In comparing the trustworthiness and applicability of the current study with the nine 72 credibility checks identified by Butterfield, Borgen, Maglio, and Amundson (2009) as part of the Enhanced Critical Incident Technique, it must first be clarified that the data analysis process for the current study was completed before the development of these nine credibility checks. Having said that, the current study did incorporate five of the nine checks, which were audiotaping interviews, checking participation rates (see Table 2), use of independent judges/raters (see Table 1), seeking expert opinion, and researching theoretical agreement. Flanagan (1954) mentions saturation and comprehensiveness in his study. Andersson and Nilsson (1964) report acceptable levels of validity regarding comprehensiveness of gathered data. In the current study, for a number of reasons, the researcher may have difficulty stating that a saturation point was truly reached. The researcher had access to a limited number of potential participants since a relatively small number of families had actually completed the program by the time the study began. However, Flanagan (1954) states that it is not the number of participants that is important, but rather the number of incidents collected (i.e., the more complex the activity or behavior being studied, the more incidents are required to reach a point of saturation). The Intensive Family Therapy Project is a multifaceted program that touches on many different aspects of the identified client’s life and functioning. These different areas may also vary slightly from one client to the next. Therefore, with regards to the current study, more incidents would need to be collected in order to truly state that a point of saturation has been reached and that the data collected is comprehensive in nature. Reaching a saturation point in this study was inhibited by certain factors. Every family that had completed the program was initially contacted regarding participation in the study; however, only seven families responded to the initial call for participation and from those seven families only nine interviews were conducted. Accessing a larger pool of participants for the 73 current research was made difficult by factors such as the highly transient nature of the families who participated in the Project. Ultimately, once the interviews were conducted, all of the participants appeared to exhaust recollection of incidents within an average of 45 minutes. In essence, each interview seemed to reach its own point of saturation as incidents began to be repeated by the participant. Finally, for many of the participants, the requirements of the program had been completed as much as 2 years prior to the interview; therefore, recalling information from that long ago may decrease the reliable nature of the data being collected. Flanagan (1954) suggests that when working with recollections instead of current events, the recollections be as recent as possible in order to increase the likelihood of accuracy and completeness of the incidents themselves. However, Flanagan (1954) also states that “[o]n the whole, it seems reasonable to assume that, if suitable precautions are taken, recalled incidents can be relied on to provide adequate data for a fairly satisfactory first approximation to a statement of the requirements of the activity” (p. 340). As mentioned in the Method section, the consistency of the current study may be enhanced due to service being delivered by only two primary therapists. However, because no additional therapists were involved, positive outcomes could perhaps be attributed more to qualities of the therapists rather than program effectiveness. It is noteworthy that the incidents related to the subcategory “Therapist Qualities” all identify qualities that might be expected of a graduate-level counsellor. As previously discussed, the results of the current study lean toward a focus on what was helpful. This may also be considered a limitation because the participants are aware they are part of a study, which may naturally skew their reports, either positively or negatively. Participants were aware their comments could influence changes or improvements being made to the project 74 which could have potentially indicated a demand for more unhelpful incidents. However, it could be possible that some participants would wish to avoid repercussions with the therapist and/or the Ministry and may feel inhibited from being completely honest about what was not helpful for them throughout the therapeutic process. It is noteworthy, though, that all participants had completed the program and so were not at risk of repercussions within treatment. It is important to keep in mind that the balance between reported helpful incidents versus reported hindering incidents may be affected by some of the above noted factors. Implications More recent qualitative research has gained an increasing focus on client perspective and feedback (Butterfield & Borgen, 2005). Findings of the current study seem to indicate the potential value in gathering information through the lens of the participants’ own words and perspectives, as to the effectiveness of a program such as the Intensive Family Therapy Project. These findings are supported by Tighe et al. (2012). The information gathered from participants of the current study may suggest that a multisystemic or ecological approach to working with the young at-risk population have the potential to be as effective, or even more effective than individual approaches. This notion was introduced in Chapter 1 which discusses various treatment approaches being used with this population. The data of the current study appears to support this notion. A significant implication of the current study relates to the use of adapted forms of MST in varied settings. The current study suggests the notion that MST-based treatment approaches have the potential to be effective in working with at-risk youth and their families, even if they do not adhere strictly to each of the tenets outlined in Chapter 1. The families interviewed stated in their own voices and from their own perspectives that Intensive Family Therapy was largely a 75 positive experience. Absolute fidelity to the MST principles may not be necessary in order to effect positive change in clients’ lives. Another implication of the current study may be that some of the benefits of a program such as the Intensive Family Therapy Project may not become concrete and lasting for clients until well after program completion. The result of this implication is that outcome measures gathered at termination of the program may not be complete and perhaps even more prone to a ‘thank you effect’. Further evaluation of participant experiences may be recommenced in the future. As noted, this presents the limitations of memory, but greater opportunity for clients to reflect on longer-term gains. There are two particularly noteworthy issues raised by the current study which have implications for the field of counselling psychology. The first is related to the issue of treatment termination which was addressed in a number of the interviews by participants. The nature of the Intensive Family Therapy Project calls for an immediate and intense connection with the family on the part of the primary therapist. The therapist becomes almost a part of the family, and this closeness helps to facilitate efforts toward change by the members of the family. However, the loss of that closeness may also be felt more intensely once therapy has concluded. It then becomes even more important to include the termination of therapy in the process itself. Families such as those involved in the Intensive Family Therapy Project have often experienced traumatic events in their past, both as a family as well as individuals within the family. As a result, the therapeutic process is made more susceptible to breakdown and/or potentially more intense in nature. The question then becomes how much of this issue of abandonment can actually be addressed as part of therapy when the intense nature of the therapy itself may leave participants vulnerable to a feeling of abandonment and loss no matter what their experience. 76 The second noteworthy issue addresses the potential to incorporate the use of the Critical Incident Technique as an assessment tool to be used throughout therapy. The very nature of the technique draws attention to what could be improved upon, no matter what therapeutic approach is being used. Less involved versions of the technique could be used regularly as a treatment evaluation for individual as well as family therapy. The results of implementing this type of approach to treatment evaluation could include increased insight and potential for change in individuals and families involved in therapy, more effective treatment on the part of therapists, and more effective treatment programming in general. This qualitative study based on the Intensive Family Therapy Project can lead research in two future directions. The first direction is work toward improvements in the IFT Project and therapy itself. The second direction is further development and use of qualitative research in related areas of psychology. Improvements in therapy. Based on the findings of this study, an area which may provide opportunity for improvements, or at least warrant some attention, is specifically the technical side of this project. Though perhaps not as clearly identifiable through data analysis, a number of interviews with participants contained dialogue regarding a lack of closure to the program and no real ending to therapy. There was also significant dialogue on the issue of follow-up to the program. These families have gone through significant ordeals, and a feeling of abandonment following completion of the program was a common notion among many of them. Professionals entered their worlds for a relatively short period of time with the intention of helping, and when family members felt they were beginning to see things change in a positive way, they felt the professionals terminated their involvement without warning. A suggested solution may be to have a more detailed and comprehensive follow-up 77 protocol in place for when completion of the program has been achieved. Prior to entrance into the Intensive Family Therapy Project, families are told that the duration of therapy will be a certain approximate length. In future studies, families could be given more concrete plans of action regarding resources they can access following completion. In the current study, some of the participants mentioned connections that were made either for them or by themselves once they were no longer receiving services from the project, but it did not appear that this was the general case for all of the families. Some families need a little more assistance in making those connections, which may depend on the level of functioning of each individual family. The follow-up process could be more easily facilitated with the use of a protocol outlining the whole procedure more clearly. Future qualitative research. At this point, no qualitative research has been reported with regards to the Intensive Family Therapy Project which was conducted in British Columbia. Also, no known qualitative studies have been reported on the research done in Ontario using Multisystemic Therapy (the tenets of which are the foundation of the Intensive Family Therapy Project). Moreover, the work being done in South Carolina with Scott Henggeler and his associates has not reported any qualitative research on their studies with Multisystemic Therapy. One study has been reported in England by Tighe et al. (2012) examining the effectiveness of MST from families’ perspectives. This lack of reported studies seems to indicate that there are plenty of opportunities for qualitative research in this particular area of therapy. Studies have been done from a qualitative perspective looking at what has or has not been helpful about several different approaches to therapy (Bachelor, 1995; Bischoff & McBride, 1996; Paulson, Truscott & Stuart, 1999). However, Tighe et al. (2012) provide the only published qualitative research on MST. Qualitative data collected on the Intensive Family Therapy Project or any 78 similar approach to therapy with at-risk youth would complement and further enhance the existing quantitative studies. In the current study many participants identified a general sense of struggling with termination of therapy and a feeling of abandonment. Further research could explore whether these struggles may be a common experience for participants engaged in Multisystemic Therapy or one of its variants, which could in turn lead to improvements in service delivery. This information could be gathered using either quantitative or qualitative methodologies, though the nature of the current study may encourage further exploration of clients’ perspectives in their own words. Adding an interview component upon completion of the Intensive Family Therapy Project as a precursor to any follow-up may help to provide more exhaustive data for research purposes. The interviews would be scheduled during final meetings with each family for up to three months following completion of the program. This may ensure that not only will data be collected from more participating families, but recollections may be more reliable as they would be more recent. In efforts to increase the exhaustive nature of studies conducted on this approach to therapy, it may also be useful to extend the interview process to other individuals who have been directly involved in the program. This may include interviews with primary therapists, whole families, and other professionals acting as team members. Conducting interviews with primary therapists involved in working directly with clients and their families could be helpful on several levels. It would provide another first-hand perspective on the therapeutic process itself. Moreover, the interviews could be helpful in further assessing what needs to be changed or enhanced in order to make the program more 79 effective for clients as well as professionals. For example, primary therapists may feel that participating organizations involved in the Project are compelling them to use a limited approach to therapy. They may not feel that they have enough room in which to conduct therapy using an approach, such as Multisystemic Therapy, that is meant to be flexible and adaptable to certain client situations. On the other hand, primary therapists may feel that they are not provided with enough support from team-member organizations. They may feel that they are alone in supporting the family and providing every aspect of assistance that may be required. These are points which may be addressed in supervisor evaluations or similar forms of assessment, though making the interviews part of a more formal study may be more useful in the process of making adjustments as needed to the structure of the program. It may also be useful to broaden the scope of the interviews by including whole families in the process, not just the identified clients and their parents/guardians. In the current study, for example, a number of the families interviewed had siblings who could have taken part and who would likely have provided useful information from yet another perspective. This may have also been helpful to the other family members in processing the whole experience of family therapy and what may have been going on within their family unit during the time leading up to and including entrance into the program. This notion is supported by first-hand accounts of parents who were involved in the current study who mentioned that they felt it would have been helpful for their other children to be involved in the interview process. Similar to the idea of conducting interviews with primary therapists is that of interviewing other professionals who were part of the team that supported each family. These professionals may include probation officers, youth support workers, and psychiatrists, among others. Data collected from these interviews may add more depth to the study of this type of 80 program, perhaps more specifically helping to identify any existing gaps in service provision. Conclusion As there has been no qualitative research done on this specific Intensive Family Therapy Project, the current study is, in a sense, breaking some new ground. Some preliminary data has been gathered on this project, but more information could be garnered and more studies could be conducted in this area in order to potentially confirm and support the results that have already begun to be generated. The current study provided an opportunity for clients of the Intensive Family Therapy Project to voice their opinions about what they found helpful about the services they received, and what they found to be hindrances. Several incidents were identified in the category of ‘Resources’ as being particularly beneficial to participants of the study. This category spoke specifically of the key elements of the program that were pulled in to help support and assist the families, and how critical these components became in the lives of these families. The category of ‘Skills Learned’ is also noteworthy as it contained only helpful incidents. Participants were able to identify skills they had taken with them from the program, and they were able to use and benefit from those skills even months and years following completion. Findings also indicated that there may be room for improvement to the program in the areas of therapy termination and follow-up. Several participants identified incidents where they felt they had been left with a feeling of abandonment upon completion, or like therapy was not adequately terminated. This may be useful in identifying some of the areas of improvement that may exist in the current program for the Intensive Family Therapy Project. Perhaps most noteworthy is the notion that adapted forms of MST, such as Intensive Family Therapy, may have the potential to be effective in working with young offenders and 81 their families in a variety of settings. The preponderance of positive incidents identified by clients is significant (88%), even considering clients had completed the program as much as two years prior to being interviewed for the current study. The overall evidence provides some promise that an MST-informed model might make a difference in the lives of vulnerable and challenging young people and their families. The at-risk youth population is in need of continued improvements in treatment options if the aim is to keep them from becoming adult offenders. The therapeutic approach of this Intensive Family Therapy Project has the potential to provide families with skills and strategies to keep moving toward becoming healthier and more positively functioning units in society. 82 REFERENCES Amundson, N. E., Borgen, W. A., Jordan, S. & Erlebach, A. C. (2004). Survivors of downsizing: Helpful and hindering experiences. Career Development, 52(3), 256-271. Andersson, B.-E., & Nilsson, S.-O. (1964). Studies in the reliability and validity of the critical incident technique. Journal of Applied Psychology, 48(6), 398-403. Bachelor, A. (1995). Client’s perception of the therapeutic alliance: A qualitative analysis. Journal of Counselling Psychology, 42(3), 323-337. Barkley, R. A., Guevremont, D. C., Anastopoulos, A. D., & Fletcher, K. E. (1992). A comparison of three family therapy programs for treating family conflicts in adolescents with Attention-Deficit Hyperactivity Disorder. Journal of Consulting and Clinical Psychology, 60(3), 450-462. Bischoff, R. J., & McBride, A. (1996). Client perceptions of couples and family therapy. The American Journal of Family Therapy, 24(2), 117-121. Borduin, C. M. (1994). Innovative models of treatment and service delivery in the juvenile justice system. Journal of Clinical Child Psychology, 23(Suppl.), 19-25. Borduin, C. M., Mann, B. J., Cone, L. T., Henggeler, S. W., Fucci, B. R., Blaske, D. M., & Williams, R. A. (1995). Multisystemic treatment of serious juvenile offenders: Longterm prevention of criminality and violence. Journal of Consulting and Clinical Psychology, 63(4), 569-578. Borgen, W. A., Amundson, N. E., & McVicar, J. (2002). The experience of unemployment for fishery workers in Newfoundland: What helps and hinders. Journal of Employment Counseling, 39, 117-126. Butler, D. L. (2002). Qualitative approaches to investigating self-regulated learning: Contributions and challenges. Educational Psychologist, 37(1), 59-63. Butterfield, L. D., & Borgen, W. A. (2005). Outplacement counseling from the client’s perspective. The Career Development Quarterly, 53, 306-316. 83 Butterfield, L. D., Borgen, W. A., Amundson, N. E. & Maglio, A.-S.T. (2005). Fifty years of the critical incident technique: 1954-2004 and beyond. Qualitative Research, 5(4), 475-497. Butterfield, L. D., Borgen, W. A., Maglio, A.-S.T., & Amundson, N. E. (2009). Using the enhanced critical incident technique in counselling psychology research. Canadian Journal of Counselling/Revue canadienne de counseling, 43(4), 265-282. Carter, R. A., Blood, L., & Campbell, M. A. (2001, June). Youth justice feasibility study: A proposal for an integrated assessment and treatment service for Conduct Disorder/Antisocial youth in Nova Scotia. Children and Youth Action Committee of Nova Scotia and Justice Canada (File No. 6133-5-23), Halifax, Nova Scotia, Canada. Cunningham, A. (2002). One step forward: Lessons learned from a randomized study of multisystemic therapy in Canada. London, ON: Center for Children & Families in the Justice System. Retrieved from http://www.lfcc.on.ca/One_Step_Forward.pdf Deacon, S. A. & Piercy, F. P. (2001). Qualitative methods in family evaluation: Creative assessment techniques. The American Journal of Family Therapy, 29, 355-373. Dekovic, M., Asscher, J. J., Manders, W. A., Prins, P. J. M., & van der Laan, P. (2012). Withinintervention change: Mediators of intervention effects during Multisystemic Therapy. Journal of Consulting and Clinical Psychology, 80(4), 574-587. Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51, 327-359. Fly, B. J., van Bark, W. P., Weinman, L., Strohm, K., & Lang, P. R. (1997). Ethical transgressions of psychology graduate students: Critical incidents with implications for training. Professional Psychology: Research and Practice, 28(5), 492-495. Fontes, L. A., & Piercy, F. P. (2000). Engaging students in qualitative research through experiential class activities. Teaching of Psychology, 27(3), 174-180. Gilgun, J. F. (2005). Qualitative research and family psychology. Journal of Family Psychology, 19(1), 40-50. Grant, N. I. R. (1994). Preventive interventions for children and adolescents: Where are we now and how far have we come? Canadian Journal of Community Mental Health, 13(2), 17- 84 35. Halfon, N., & Berkowitz, G. (1993). Development of an integrated case management program for vulnerable children. Child Welfare, 72(4), 379-397. Henggeler, S. W. (2002). Multisystemic Therapy Program. Retrieved from http://www.strengtheningfamilies.org/html/programs_1999/04_MST.html Henggeler, S. W., Melton, G. B., Brondino, M. J., Scherer, D. G., & Hanley, J. H. (1997). Multisystemic therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting and Clinical Psychology, 65(5), 821-833. Henggeler, S. W., Melton, G. B., & Smith, L. A. (1992). Family preservation using Multisystemic Therapy: An effective alternative to incarcerating serious juvenile offenders. Journal of Consulting and Clinical Psychology, 60(6), 953-961. Henggeler, S. W., Rowland, M. D., Pickrel, S. G., Miller, S. L., Cunningham, P. B., Santos, A. B., . . . Edwards, J. E. (1997). Investigating family-based alternatives to institution-based mental health services for youth: Lessons learned from the pilot study of a randomized field trial. Journal of Clinical Child Psychology, 26(3), 226-233. Henggeler, S. W., Schoenwald, S. K., & Pickrel, S. G. (1995). Multisystemic therapy: Bridging the gap between university- and community-based treatment. Journal of Consulting and Clinical Psychology, 63(5), 707-717. Howell, A. J., & Enns, R. A. (1995). High risk recognition program for adolescents in conflict with the law. Canadian Psychology, 36(2), 149-161. Huey, S. J., Henggeler, S. W., Brondino, M. J., & Pickrel, S. G. (2000). Mechanisms of change in Multisystemic Therapy: Reducing delinquent behavior through therapist adherence and improved family and peer functioning. Journal of Consulting and Clinical Psychology, 68(3), 451-467. Huey, S. J., Henggeler, S. W., Rowland, M. D., Halliday-Boykins, C. A., Cunningham, P. B., Pickrel, S. G., & Edwards, J. (2004). Multisystemic Therapy effects on attempted suicide 85 by youths presenting psychiatric emergencies. Journal of the American Academy of Child & Adolescent Psychiatry, 43(2), 183-190. Jaffe, P. G., & Baker, L. L. (1999). Why changing the YOA does not impact youth crime: Developing effective prevention programs for children and adolescents. Canadian Psychology, 40(1), 22-38. Kashani, J. H., Jones, M. R., Bumby, K. M., & Thomas, L. A. (1999). Youth violence: Psychosocial risk factors, treatment, prevention, and recommendations. Journal of Emotional & Behavioral Disorders, 7(4), 200-211. Kazdin, A. E., Siegel, T. C., & Bass, D. (1992). Cognitive problem-solving skills training and parent management training in the treatment of antisocial behavior in children. Journal of Consulting and Clinical Psychology, 60(5), 733-747. Klietz, S. J., Borduin, C. M., & Schaeffer, C. M. (2010). Cost-benefit analysis of Multisystemic Therapy with serious and violent juvenile offenders. Journal of Family Psychology, 24(5), 657-666. Koehn, C. (1995). Sexual abuse survivors’ perceptions of helpful and hindering counsellor behaviours. Unpublished doctoral dissertation, University of Victoria, Victoria, British Columbia, Canada. Kral, M. J., Burkhardt, K. J., & Kidd, S. (2002). The new research agenda for a cultural psychology. Canadian Psychology, 43(3), 154-162. Larson, J. D., Calamari, J. E., West, J. G., & Frevert, T. A. (1998). Aggression management with disruptive adolescents in the residential setting: Integration of a cognitivebehavioral component. Residential Treatment for Children & Youth, 15(4), 1-9. Leschied, A. (2000). Informing young offender policy in current research: What the future holds. CSC Forum, 12(2), 36-40. Leschied, A. (2002). Cognitive behavioural treatment for young offenders. In K. Sakai (Ed.) Annual Report and Resource Guide. Fuchu, JA: United Nations and Far East Institute for the Prevention of Crime and Treatment of Offenders. 86 Leschied, A. (2003a). Why Multisystemic Therapy? Answering questions from the “What Works” literature in youth justice research. Promising Strategies in Adolescent Mental Health (PowerPoint presentation), Vancouver, BC. Leschied, A. (2003b). Reviewing the Ontario clinical trial with Multisystemic Therapy. Adolescent Mental Health: Evidence-Based Perspectives and Programs (PowerPoint presentation), Vancouver, BC. Leschied, A., & Cunningham, P. B. (1999). A community-based alternative for high risk young offenders. Youth & Corrections, 11(2), 25-29. MacKnee, C., & Mervyn, J. (2002). Critical incidents that facilitate homeless people's transition off the streets. Journal of Social Distress and the Homeless, 11(4), 293-306. McCloskey, M. E., & Glucksberg, S. (1978). Natural categories: Well defined or fuzzy sets? Memory & Cognition, 6(4), 462-472. McCormick, R. M. (1997). Healing through interdependence: The role of connecting in First Nations healing practices. Canadian Journal of Counselling/Revue canadienne de counseling, 31(3), 172-184. Middleton, M. B., & Cartledge, G. (1995). Effects of social skills instruction and parental involvement on the aggressive behaviors of African American males. Behavior Modification, 19(2):192-210. Mitchell, G. J. (2001). A qualitative study exploring how qualified mental health nurses deal with incidents that conflict with their accountability. Journal of Psychiatric & Mental Health Nursing, 8(3), 241-249. Moretti, M. M., Emmrys, C., Grizenko, N., Holland, R., Moore, K., Shamsie, J., & Hamilton, H. (1997). The treatment of conduct disorder: Perspectives from across Canada. Canadian Journal of Psychiatry/Revue Canadienne de Psychiatrie, 42(6), 637-648. Morrow, S. L. (2005). Quality and trustworthiness in qualitative research in counseling psychology. Journal of Counseling Psychology, 52(2), 250-260. Mullins, W. C., & Kimbrough, W. W. (1988). Group composition as a determinant of job 87 analysis outcomes. Journal of Applied Psychology, 73(4), 657-664. Multisystemic Therapy Services (2000). Multisystemic Therapy: Research On Effectiveness [On-line]. Retrieved from http://www.mstservices.com Netting, F. E. (1992). Case management: Service or symptom? Social Work, 37(2), 160-165. Offord, D. R., & Bennett, K. J. (1994). Conduct Disorder: Long-term outcomes and intervention effectiveness. Journal of the American Academy of Child and Adolescent Psychiatry, 33(8), 1069-1078. Paulson, B.L., Truscott, D., & Stuart, J. (1999). Client’s perceptions of helpful experiences in counseling. Journal of Counseling Psychology, 46(3), 317-324. Pratt, S. I., & Moreland, K. L. (1996). Introduction to treatment outcome: Historical perspectives and current issues. Residential Treatment for Children & Youth, 13(4), 127. Rennie, D. L. (2002). Making a clearing: Qualitative research in Anglophone Canadian psychology. Canadian Psychology, 43(3), 139-140. Roberts, A. R., & Schervish, P. (1988). A strategy for making decisions and evaluating alternative juvenile offender treatment programs: Compensating for missing information. Evaluation and Program Planning, 11, 115-122. Rosch, E. (1978). Principles of categorization. In E. Rosch & B. B. Lloyd (Eds.), Cognition and Categorization (pp. 27-48). Hillsdale, NJ: Lawrence Erlbaum. Ross, R. R., & Altmaier, E. M. (1990). Job analysis of psychology internships in counseling center settings. Journal of Counseling Psychology, 37(4), 459-464. Sawyer, A. M., & Borduin, C. M. (2011). Effects of Multisystemic Therapy through midlife: A 21.9-year follow-up to a randomized clinical trial with serious and violent juvenile offenders. Journal of Consulting and Clinical Psychology, 79(5), 643-652. Scherer, D. G., & Brondino, M. J. (1994). Multisystemic family preservation therapy: Preliminary findings from a study of rural and minority serious adolescent offenders. Journal of Emotional & Behavioral Disorders, 2(4), 198-207. 88 Schoenwald, S. K., Henggeler, S. W., Brondino, M. J., & Rowland, M. D. (2000). Multisystemic therapy: Monitoring treatment fidelity. Family Process, 39(1), 83-104. Shamsie, S. J. (1981). Antisocial adolescents: Our treatments do not work -- where do we go from here? Canadian Journal of Psychiatry, 26(5), 357-364. Sondheimer, D. L., Schoenwald, S. K., & Rowland, M. D. (1994). Alternatives to the hospitalization of youth with a serious emotional disturbance. Journal of Clinical Child Psychology, 23(Suppl.), 7-12. Sundelin, J., & Hansson, K. (1999). Intensive family therapy: A way to change family functioning in multi-problem families. Journal of Family Therapy, 21, 419-432. Tate, D. C., Reppucci, N. D., & Mulvey, E. P. (1995). Violent juvenile delinquents: Treatment effectiveness and implications for future action. American Psychologist, 50(9), 777-781. Tighe, A., Pistrang, N., Casdagli, L., Baruch, G., & Butler, S. (2012). Multisystemic Therapy for young offenders: Families’ experiences of therapeutic processes and outcomes. Journal of Family Psychology, 26(2), 187-197. Tryon, G. S. (2000). Ethical transgressions of school psychology graduate students: A critical incidents survey. Ethics & Behavior, 10(3), 271-279. Twelker, P. A. (2003). The Critical Incident Technique: A manual for its planning and implementation. Retrieved from http://www.tiu.edu/psychology/Twelker/critical_incident_technique.htm Winchip, S. (1996). Qualitative methodologies in the learning process: A reflective response. Journal of Instructional Psychology, 23(2), 174-178. Wong, L. (2000). What helps and what hinders in cross-cultural clinical supervision: A critical incident study. Unpublished doctoral dissertation, University of British Columbia, Vancouver, British Columbia, Canada. 89 Appendix A: Interview Protocol This protocol is designed for youth. Some wording may vary when using it with adults. INTERVIEW PROTOCOL(s) Introduction: When I talked to you on the phone, I told you what this project is about. I want you to try to remember the program you and your family went through. I want to find out what you think worked for you and what didn’t work for you about that program. I want to know what you think about what you did and went through in that program. Starting Question(s) Can you think of something that happened or something that someone did that worked for you, was helpful? Please tell me about that. Can you think of something that happened or something that someone did that got in the way for you, or was a problem? Please tell me about that. Follow-up & Probing Questions Tell me about a time when ... 1. what happened? 2. what went before?, after? 3. how did it turn out? What was that like for you? How has that affected your family? Key areas to be discussed in follow-up: - perceived therapeutic skills of therapist, - frequency and duration of therapy, - attitudes of clients towards treatment, - perceived overall efficacy of treatment Without having a more clearly defined structure in terms of areas to cover, it seems quite likely that responses may reflect verbal skills, comfort level, interpersonal skills, etc..., of respondents rather than of treatment. Debriefing I want to give you the opportunity at this point to ask any questions or raise any concerns you may have about your involvement in this study. I also want to be certain that you have had a 90 chance to express yourself adequately. (The debriefing will be informal and will be done so as to ensure that participants are comfortable as they leave the interview setting, should anything upsetting have occurred during the session.) If you would like to be informed of any of the results of this study, arrangements can be made to meet again with me, or to engage in a discussion over the phone following completion of transcription data analysis for debriefing. You can also contact MCFD or Youth Forensics for an executive summary or abstract of the study. A formal copy of the study may be accessed through the Counselling Psychology Department at Trinity Western University, or through the Norma Marion Alloway Library at Trinity Western University. 91 Appendix B: Initial Contact Letter to Participants “Intensive Family Therapy (MST) Project Study” Name Address Address March 14, 2004 Dear _____________________ This letter is an invitation to participate in a study of the Intensive Family therapy project offered by Martin Bartel or Gillian Feenstra. Your participation can help us to improve this service and hopefully may also contribute to the program continuing and serving other families too. The purpose of the study is to obtain the perspectives of parents and young people who have taken part in the service so that we can more fully understand what actions of the therapists help and what may hinder achieving the goals you had set out. The researcher, Giselle Tranquilla would need to meet with you for about an hour. She would also need to meet with the young person who was initially referred to the program for about an hour. Young people will be offered an honorarium of $ 20 for their participation. Parents will be offered an opportunity to win dinner for two. Since the sample size is small, six to eight families, the odds are reasonably good! The researcher, Giselle Tranquilla, is undertaking this project towards completion of her thesis for a Masters Degree in Counselling Psychology at Trinity Western University. The information you would provide will help Giselle complete her degree but it will also be important to the Ministry of Children and Family Development as we attempt to understand what works and why. Many people who have taken part in this kind of research find that they gain a stronger understanding of the treatment by having the chance to discuss it with the researcher who has not been a part of it. Some participants may appreciate having the opportunity to address something they didn’t like about the service or to be able to contribute to seeing that what they did like will continue. The interview will be very open and flexible. There will be two basic questions; What helped? And what hindered? your progress in the program. Giselle is willing to conduct interviews wherever you think best. This can be done in an office near you or at your home. She will need to audio tape the interview so that she can carefully think through everything that you say. Audio tapes will be kept in a locked secure space available only to the researcher. They will be erased when the study is complete. Identifying information about your comments will be deleted so that the providers of the program or others will not know which comments were made by which person. So you can be completely free to say whatever it is that you think about what helped and hindered you in the Intensive Family Therapy Project. Only the researcher will know 92 “who said what”. The researcher may check back with you to insure she has fully understood your comments. I will call you within the next few weeks to answer any questions you may have. I will also want to know if you are willing to take part in the study. If you are, then and only then would I provide your contact information to the researcher who would then contact you. You would be free to withdraw participation at any time you wished. Although we do not expect any negative consequences for individuals participating in the study, if there was anything that did prove upsetting, I or the researcher would be quite willing to meet with you or the young person to provide counselling support on the matter. Please give this matter serious consideration. The work that Martin and Gillian have been doing is a new project. The more we know about it, the more helpful we can make out services for young people. Your perspective is important for a solid understanding of the program. Thank you for taking the time to read and consider this. Sincerely, Dr. Robert Lees, R.Psych. Regional Mental Health Consultant Upper Fraser Region 1-604 -870-5888 93 Appendix C: Informed Consent Form – Caregiver(s) January 10, 2005 Title: Intensive Family Therapy with At-risk Youth: A Critical Incident Study Researcher: Giselle Tranquilla Telephone: (604) 526-4772 Faculty Supervisor: Dr. Rob Lees Telephone: (604) 870-5888 Name of Participant:________________________________ Phone:_______________________(for purpose of the draw) I understand that this study in which I have agreed to participate will involve being interviewed by the primary researcher, a graduate student in Counselling Psychology at Trinity Western University, for approximately one hour with regards to my experiences in the Intensive Family Therapy project. I understand that my participation in this study is voluntary and that I may withdraw from the study at any time and for any reason without penalty. I understand that as incentive for participation in this study, my name will be entered into a draw for dinner for two. I understand that there is no obligation to answer any question or participate in any aspect of this project that I consider invasive. I understand that all personal data will be kept strictly confidential, except when required by law (ie. disclosures of child abuse), and that all information will be coded so my name will not be associated with my answers. I understand that only the researcher named above will have access to the data. I understand that the data will be stored in a locked filing cabinet for the duration of the project. I realize that data from the study, other than audio tapes, will be archived for 5 years, at which point, paper forms will be shredded and computer forms will be erased. All audio tapes will be erased upon transcription of the interviews. I further understand that information obtained in this study will be used for the purposes of a masters thesis project with the possibility of it also being presented in conferences, journal articles, or other professional publications and venues. I understand that there may be minimal risk involved in participating in this project. I understand that support will be provided by the interviewer, and a referral for service can be made if I need it. I understand that it may be beneficial for me personally to recount my experiences in the Intensive Family Therapy project. It may also help to know that my story has been heard and may help other families or individuals who may be experiencing the same issues. 94 I have read and understand the description of the study and I willingly consent to participate in this study. Participant Signature:_____________________________ Date:________________________ If you have any questions or concerns about your participation in this study, please contact either the above named researcher or Dr. Marvin MacDonald, Thesis Coordinator at Trinity Western University, at (604) 888-7511. If you have any questions about ethical issues involved in this project you may contact Ms. Sue Funk in the Office of Research at (604) 513-2142. Thank you for your help! I have fully explained the procedure of the study to the above volunteer. Researcher’s Signature:_________________________________ Date:__________________________ 95 Appendix D: Informed Consent Form - Youth November 11, 2004 Title: Intensive Family Therapy with At-risk Youth: A Critical Incident Study Researcher: Giselle Tranquilla Telephone: (604) 526-4772 Faculty Supervisor: Dr. Rob Lees Telephone: (604) 870-5888 Name of Participant:________________________________ Phone:_______________________ I understand that this study in which I have agreed to participate will involve being interviewed by the primary researcher, a graduate student in Counselling Psychology at Trinity Western University, for approximately one hour with regards to my experiences in the Intensive Family Therapy project. I understand that my participation in this study is voluntary and that I may withdraw from the study at any time and for any reason without penalty. I understand that I only have to disclose what I am comfortable with sharing. I understand that as incentive for participation in this study, I will receive payment of $20. I understand that there is no obligation to answer any question or participate in any aspect of this project that I consider invasive. I understand that all personal data will be kept strictly confidential, except when required by law (e.g. disclosures of child abuse), and that all information will be coded so my name will not be associated with my answers. I understand that only the researcher named above will have access to the data. I understand that the data will be stored in a locked filing cabinet for the duration of the project. I realize that data from the study, other than audio tapes, will be archived for 5 years, at which point, paper forms will be shredded and computer forms will be erased. All audio tapes will be erased upon transcription of the interviews. I further understand that information obtained in this study will be used for the purposes of a masters thesis project with the possibility of it also being presented in conferences, journal articles, or other professional publications and venues. I understand that there may be minimal risk involved in participating in this project. I understand that support will be provided by the interviewer, and a referral for service can be made if I need it. I understand that it may be beneficial for me personally to recount my experiences in the Intensive Family Therapy project. It may also help to know that my story has been heard and may help other families or individuals who may be experiencing the same issues. 96 I have read and understand the description of the study and I willingly consent to participate in this study. Participant Signature:_____________________________ Date:________________________ If you have any questions or concerns about your participation in this study, please contact either the above named researcher or Dr. Marvin MacDonald, Thesis Coordinator at Trinity Western University, at (604) 888-7511. If you have any questions about ethical issues involved in this project you may contact Ms. Sue Funk in the Office of Research at (604) 513-2142. Thank you for your help! I have fully explained the procedure of the study to the above volunteer. Researcher’s Signature:___________________________________ Date:__________________________ 97 Appendix E: IFT Assessment Outline SYSTEMIC STRENTGHS SYSTEMIC WEAKNESSES/NEEDS Individual Individual Family Family Peers Peers School School Community Community 98 Appendix F: Sample IFT Assessment Intensive Family Therapy Project Strengths & Weaknesses/Needs Assessment Family: Therapist: Date: SYSTEMIC STRENTGHS SYSTEMIC WEAKNESSES/NEEDS Individual Individual Is a caring individual; likes to have connection with others including professionals; tries to be pleasing to adults whom he is connected to; is helpful/thoughtful; is sociable and outgoing; has a desire to do better. Stays out late; is a follower of others; has struggles with truancy; has a history of aggressiveness toward mom and stepdad; has been physically violent; has difficulty with appropriate sexual fantasies and acts; has FAE and related issues. Family Family Mother has worked in the past; mother has significant number of supportive friends; home environment is stable; mom is newly married providing her with ability to enforce rules and consequences; mom wants a stable home. Peers Some pro-social peers both with and without employment; is willing to make new friends; is sociable. School He has re-applied a number of times in his attempts to gain success in school; likes to be helpful to younger students; has been known to work hard in some subjects. Community Used to attend church youth group; mom still attends regularly; is involved with Community Services program Nites Alive; has had several jobs in community in the past; has likable presentation in community at times. Conflictual relationship with mom and stepdad; stepdad has little tolerance for behaviours and or mistakes made; new relationship with stepdad is very conflictual; history of unsuccessful discipline techniques. Peers Many of his closest friends support or are involved in criminal activity; is running drugs with and is using with peers; finds acceptance and support from peers with anti-social activities. School Schoolwork needs to be a priority; school attendance to become priority; cognitive abilities at a low-average range; problems with student interactions. Community Lives in small community with few job opportunities; limited opportunities to make new friends and choose pro-social peers; several close neighbours/peers who support criminal activity. 99 Appendix G: IFT Weekly Supervision Outline WEEKLY REVIEW 1. Overarching/Primary MST goal 2. Previous intermediary goals Met Partially Not 3. Barriers to intermediary goals 4. Advances in treatment 5. How does the assessment of “fit” change with this new information? 6. New intermediary goals for next week