National Research Institute for Child and Family Studies - Selected Presentations

Thompson, R. W. & Chmelka, M. B. (2005, March). Placement stability for children in treatment foster care. Poster session presented at the Public and Private Foster Care Agency Benchmarking Session at the Child Welfare League of America National Convention, Washington, DC.

Oats, R., & Shaw, T. (2004, November). Assessing the impact of a model fidelity instrument on the consultation process: Can a fidelity instrument serve multiple purposes? Poster session presented at the American Evaluation Association Conference, Atlanta, GA.

Assessing treatment fidelity can provide a wealth of information for various stakeholders and ultimately help improve outcomes for the target population. It is a challenging endeavor to create a treatment fidelity instrument that accurately captures a model and yet is flexible and user-friendly. Given the limited resources available to non-profit agencies, the most cost-effective and advantageous approach is to develop a model fidelity instrument that is multi-functional. Over several years, Boys Town has developed a comprehensive model fidelity instrument that operationalizes the hallmarks of the residential Boys Town Teaching-Family Model: Teaching, Relationships, Motivation System, Family-Style Living, Self-Government, and Moral and Spiritual Values. The primary purpose of the instrument is to evaluate, using direct observation, the degree to which staff implement the Model. The instrument is designed to be multi-functional in that it also can be used as a staff consultation and training tool. Initial efforts in evaluating the utility of the instrument are focused on staff consultation. Consultants at Boys Town assist direct-care staff in improving their model implementation skills and thus have a strong potential for impacting youth outcomes.

Thompson, R. W. (2004, November). The long-term benefits of residential care: A 16-year study. Paper presented at the Florida Coalition for Children Conference, Orlando, FL.

This is a presentation that was made to residential care providers in Florida. Methods and results from a long-term follow-up study of residential care are presented. The results indicate that youth who are placed in quality family-style residential care can have positive adult outcomes in several important areas, including educational attainment and intimate partner violence. These findings also continue to support the hypothesis that a longer length of stay in residential care is associated with better long-term outcomes. Implications for policy and practice are also discussed.

Handwerk, M., Huefner, J., Ringle, J., Almquist, J., Smitham, S., Soper, S., Howard, B., & Chmelka, B. (2004, October). The role of therapeutic alliance in therapy outcomes for youth in residential care. Paper presented at the 4th Bi-Annual Kansas Conference in Clinical Child and Adolescent Psychology, Lawrence, KS.

This presentation describes outcomes over a two-year period for 118 youth residing in a large residential-care facility in the Midwest who had been receiving adjunctive therapeutic services for behavioral and emotional problems. Outcome measures include: the Daily Incident Report, a measure of critical behavioral incidents completed for all youth at the residential facility; a Symptom Screener completed on a weekly basis by the youth and their therapist and monthly by a clinical supervisor; and a therapeutic alliance Working Relationship Scale (Doucette & Bickman, 2000) also completed weekly by the youth and their therapist. Results suggest that youth in therapy improved over the course of treatment, but improvement varied based on informant and outcome measure. Data from the Working Relationship Scale indicated low associations between therapeutic alliance and outcome. These results should not be interpreted as trivializing the importance of the therapeutic relationship, though they do suggest that other factors may be more relevant and salient in producing change in child and adolescent therapy, at least in this particular setting.

Huefner, J. C., Ringle, J. L., Daly, D. L., Ingram, S. D., & Chmelka, M. B. (2004, October). Breaking the abuse cycle: A long-term follow-up report. Poster session presented at the American Academy of Child and Adolescent Psychiatry 51st Annual Meeting, Washington, DC.

While many residential care programs have demonstrated positive short-term outcomes, this study examines long-term outcomes of residential care youth 16 years after departure. Program outcomes of 211 former residential care youth are assessed via a social functioning and quality-of-life survey. In addition, clinical file reviews identify those participants with a history of domestic abuse, substance abuse, or criminality. It is hypothesized that participants who had longer residential-care stays will realize greater treatment effects and will not differ from national normative data. For domestic abuse, there is no difference from the national norm, regardless of social history or time in the program, but the rates are encouragingly low. Arrest rates decrease as length of stay increases, regardless of criminal social history. Alcohol use is essentially the same as the general population, whereas drug usage for all groups is significantly higher than the national norm, regardless of social history. Although not all outcomes are positive, results suggest that a family-style intervention can have lasting positive effects. The most positive long-term outcomes are for youth with longer lengths of stay, regardless of abuse, substance use, or criminality histories.

Silverstein, A., Thompson, R., Little, M., & Kohm, A. (2004, August). The prevalence of bullying in a residential context. Paper presented at the Annual Convention of the American Psychological Association, Honolulu, HI.

Very little research has been reported about bullying behavior in residential settings. In this study, youth placed in a residential care and education setting were interviewed about bullying behaviors and victimization using the Olweus Scale. The presentation covers descriptive data about bullying in this setting, differences and similarities with bullying in day-school settings, and implications for policy and practice.

Castrianno, L., Chmelka, B., & Thompson, R. (2004, July). Boys Town Treatment Foster Family Services. Poster session presented at the Treatment Foster Care Agency Benchmarking Roundtable at the Foster Family Treatment Association 18th Annual Conference, Nashville, TN.

This poster presents a description of children, provided services, and selected outcome data in a multi-site treatment foster care program. Outcome variables are presented in the following domains: well-being, permanency, and safety. The emphasis of the session was sharing of outcome information across agencies in order to provide opportunities to benchmark this data nationally.

Huefner, J. C., Davis, J. L., Ringle, J. L., Ingram, S. D., & Chmelka, M. B. (2004, June). Adult outcomes of Boys Town youth: Implications for policy and practice. Paper presented at the 2004 Child Welfare League of America Mid-West Regional Training Conference and National Juvenile Justice Summit, Indianapolis, IN.

In this study, adults who had been in residential care are compared to other adults who had been at risk and also to national normative populations. Adults who had been in residential care were similar to the national population, and a longer length of stay was related to more positive long-term outcomes.

Huefner, J. C., & Xia, Y. R. (2004, June). Secure detention programs for female juvenile offenders: A follow-up report. Paper presented at the 2004 CWLA Mid-West Regional Training Conference and National Juvenile Justice Summit, Indianapolis, IN.

The need for effective juvenile female services has never been greater. Recidivism over four years was examined for 155 girls, who departed from a detention program in Pennsylvania. Re-arrest recidivism was 12% at one year and 38% at four years. These rates are significantly better than those for other girls’ programs.

Thompson, R. W. (2004, June). Applied research at Boys Town. Invited paper presented at the University of York Symposium on Children’s Services, York, England, United Kingdom.

Boys Town has been actively engaged in applied research for the past 15 years. This presentation is a summary of the types of research, selected results, and recommendations for applied research in the areas of child and family services. Residential care and education are emphasized. The types of studies, the methodology, and some of the results are discussed and compared to the work of applied researchers from Great Britain who attended the seminar.

Davis, J. L., & Huefner, J. C. (2004, May). Validating best practices using long-term outcomes. Paper presented at the 2004 Child Welfare League of America: Finding Better Ways Conference, Atlanta, GA.

Boys Town is a program with 30 years experience as an authoritative community. A 16-year follow-up study showed that the ten characteristics of authoritative communities are related to positive long-term outcomes and validated the program’s effectiveness.

Huefner, J. C., Ringle, J. L., Ingram, S. D., & Chmelka, M. B. (2004, May). What courts need to know about the long-term effects of residential care. Paper presented at the 23rd Annual Juvenile Probation and Justice Management Conference, Nashville, TN.

Children who come into residential care are at a crossroads in their lives. They can cost society in terms of lost talent and productivity or they can contribute as productive citizens. A 16-year follow-up study of former residential care youth are compared to national normative data. Results support the notion that quality residential care and education can help these children become productive citizens. Implications for pre-sentence decisions about residential care are discussed.

Kessler, C. (2004, May). Common Sense Parenting: An overview. Presented at the New York Project Site for the Center for Mental Health Services. [Contact Ron Thompson for more information regarding this presentation].

In order to gain a better understanding of Common Sense Parenting^® training, this presentation gives an overview of the proven child-care methods developed at Boys Town and how these same hallmarks and elements form the definitions and context for Common Sense Parenting training. Numerous Common Sense Parenting studies are also reviewed.

Handwerk, M. L., Huefner, J. C., Ringle, J. L., Almquist, J., & Chmelka, B. (2004, March). The role of therapeutic alliance in therapy outcomes for youth in residential care. Paper presented at the 2004 Florida Mental Health Institute Conference, Tampa, FL.

This study describes outcomes for 79 youth residing in a large residential-care facility in the Midwest who had been receiving adjunctive therapeutic services for behavioral and emotional problems. Outcome measures included: a Daily Incident Report, a Symptom Screener, and a therapeutic alliance Working Relationship Scale. In general, youth rated their symptoms as improving during therapy. Measures of actual behavior indicated improvements in all domains, but gains were generally not maintained at follow-up. Finally, therapeutic alliance as measured by the Working Relationship Scale was relatively stable based on youth and clinician ratings, but was only minimally related to outcome. Limitations of this study are also discussed.

Thompson, R. W., Daly, D. L., Huefner, J., & Ringle, J. (2004, March). Adult outcomes of Boys Town youth: A follow-up report. Paper presented at the Florida Mental Health Institute 17th Annual Conference, Tampa, FL.

The purpose of this study is to measure adult outcomes for youth served in a long-term residential care and education program. The results indicate significant treatment effects and other positive adult outcomes for youth in several important areas, including educational attainment and intimate partner violence, but concerns remain in the areas of post-secondary education, employment, and illicit drug use. Current findings also continue to support the hypothesis that a longer length of stay in residential care is associated with better long-term outcomes.

Duppong-Hurley, K., Oats, R., Shaw, T. (2003, November). The struggle to assess treatment implementation in non-profit agencies: The obstacles and triumphs of a child and family service agency. Paper presented at the American Evaluation Association Conference, Reno, NV.

The collection of treatment implementation data in non-profit agencies is often a challenging endeavor due to lack of agency interest, time, and resources. While most agencies understand the importance of collecting client outcome data, far fewer routinely obtain program implementation data. Boys Town recently began to systematically assess implementation of a variety of programs, including residential homes for youth, emergency shelters, and parenting classes. It has proven to be difficult to balance providing enough detailed implementation information to address training needs, yet keep the process brief and manageable. Current efforts to assess program implementation have been met with both enthusiasm and resistance. Using examples from the implementation assessment process, the barriers that were encountered during the project and the methods that were used to encourage agency support for measuring treatment fidelity are presented. Also, preliminary implementation data and methods for distributing implementation data with supervisors and direct-care staff are discussed.

Ingram, S., Kessler, C., Duppong Hurley, K., Czyz, D., Taylor, G., & Juliano, N. (2003, October). An efficient approach for evaluating an agency-wide intervention. Paper presented at the Alliance for Children and Families 2003 Biennial Conference, Kohler, WI.

Youth-care agencies frequently implement new initiatives, but often believe they do not have the resources to monitor the effectiveness of the new programs. However, there are numerous ways to efficiently assess the effectiveness of new interventions by utilizing existing data or incorporating economical new measures. Utilization of incident reports, exit interviews, and surveys to evaluate a new training initiative at a short-term emergency care facility are discussed. Each of these three data collection methods is highlighted with examples of how the data was analyzed, graphed, and interpreted.

Kessler, C., Ingram, S., Castrianno, L., Duppong Hurley, K., & Davis, J. (2003, October). Using outcome data efficiently: Developing a multi-purpose outcome system for the entire agency. Paper presented at the Alliance for Children and Families 2003 Biennial Conference, Kohler, WI.

This paper describes a comprehensive program evaluation system in a large private agency, which operates multiple interventions at many geographical locations. Interventions range from residential care to prevention programs, including parenting classes and in-home family-based services. Utilization of outcome data for treatment planning and monitoring, quality improvement, and strategic planning are also emphasized.

Duppong Hurley, K., Nelson, C., Shaw, T., & Oats, R. (2002, November). Assessing treatment implementation in human service agencies: Identifying training needs and improving outcomes. Paper presented at the American Evaluation Association Conference, Washington, DC.

While many human service agencies collect outcome data, far fewer collect treatment implementation data due to time and resource constraints. Treatment implementation issues are often addressed through ongoing consultation; however, this format usually consists of general observation notes and is seldom examined in a systematic manner to provide detailed information to both trainers and evaluators. As it is often difficult and time consuming to collect implementation information, this paper discusses an approach to assessing treatment implementation that is relatively easy to use and provides a variety of information to various stakeholders. The observation form that was developed included descriptive notes on the observation, ratings on specific components of the Model, overall ratings of treatment implementation, and a feedback form for staff. This paper discusses an approach used to assess treatment implementation, share preliminary data, and describe efforts to increase agency support for assessing treatment implementation.

Duppong Hurley, K., Shaw, T., Oats, R., & Burke, R. (2002, November). Evaluating systemic change in schools: Efficient methods for measuring the impact of a school-wide classroom management intervention. Paper presented at the American Evaluation Association Conference, Washington, DC.

It can often be challenging to design methods to assess systemic change in schools that provide comprehensive data, yet are easy to administer. To meet these competing needs, a school survey and an office referral database were created to evaluate a school-wide intervention, the Boys Town Education Model, which emphasizes classroom management, discipline practices, and social skills instruction. The survey was designed to be an efficient measure of student, teacher, and parent perceptions of school change across a variety of content areas. The database was created in order to allow program staff and schools to easily organize and analyze student discipline data. For example, schools using the database were able to generate reports that illustrated patterns and trends in discipline data on individual and school-wide levels. This presentation discusses the development of the surveys and database, preliminary results, and ways to encourage school-level support for assessment activities.

Thompson, R. W., Lepley, W. L., & Meyers, A. E. (2002, October). Indicators of organizational success: Development of a living document. Paper presented at the Alliance for Children and Families Annual Conference, Pennsylvania, PA.

Organizations that provide child and family services infrequently address staff and leadership incentives to improve quality of services and outcomes for consumers. This presentation covers the design and implementation of a balanced scorecard outlining both organizational processes and youth outcomes, along with an incentive package for staff and leaders that ties compensation to organizational and client success.

Castrianno, L. M., Temple-Plotz, L., Kessler, C., & Rose, V. J. (2002, July). Boys Town Treatment Foster Care: Predictors of success in treatment foster care for maltreated youth. Paper presented at Foster Family Treatment Association Conference, Chicago, IL. [Contact Ron Thompson for more information regarding this presentation.]

This paper describes a study designed to identify predictors of positive outcomes for children in treatment foster care provided by an agency with multiple sites across the country. The most significant predictor of positive outcomes for these children was the number of presenting problems at intake. The results underscore the importance of assessing the severity of risk for children in treatment foster care at intake. Other implications for practitioners are also discussed.

Thompson, R. W., Castrianno, L., & Brylske, P. (2002, July). Outcomes Roundtable. Roundtable Discussion at the Foster Family Treatment Association Conference, Chicago, IL.

Agencies that serve children and families are more and more frequently called upon to demonstrate their effectiveness with clients. This requires a system to identify and measure client outcomes, collect data, and report to consumers. This roundtable was designed to allow for discussion about barriers and other issues related to setting up and maintaining an effective system. Small discussion groups of participants interested in specific issues were given the opportunity to problem solve with each other.

Castrianno, L. (2001, November). Using outcomes to improve and promote child and family services: Using incident data for outcomes. Paper presented at the Heartland Conference on Families, Omaha, NE. [Contact Ron Thompson for more information regarding this presentation.]

This workshop includes practical instruction for agencies to design and implement an outcomes evaluation system. Involvement of staff and consumers in the design and utilization of outcome data for multiple purposes in agency-based decision-making and planning is emphasized.

Duppong Hurley, K., Shaw, T., Spenceri, M., & Burke, R. (2001, November). Monitoring implementation of a school-wide classroom management program: Making the process of measuring treatment fidelity appealing to clients. Paper presented at the American Evaluation Association Conference, St. Louis, MO.

In an applied evaluation setting, it is easier to convince clients to collect outcome data than to monitor treatment implementation. Even if there is interest in collecting implementation data, it is often a difficult and time-consuming process. Boys Town’s approach to improving the support from school administrators and internal program staff for the implementation assessment of the Boys Town education intervention was to make the process more practical by (1) tying the assessment of implementation with a consultation component and (2) utilizing an efficient classroom observation-rating technique that requires no coding and provides quick turnaround of information. This method was economical in both time and cost, while providing valuable feedback to program, school, and research staff. Highlights of the presentation include the development of the treatment fidelity instrument, success in identifying high and low implementation, inter-rater reliability analyses, and comparisons of degree of implementation with student outcome measures.

Thompson, R. W. & Davis, J. (2001, September). Using outcomes comparison analysis to improve and promote child and family service programs. Paper presented at the Alliance for Children and Families Annual Conference, Denver, CO.

In order for outcome data to be useful, it is often helpful to have some type of comparison group. This presentation consists of several examples of comparison samples that were used in the analysis of outcome data from programs serving children and their families. Examples include comparison of a single youth over time to other youth in a home, center, or larger aggregate groups. It also includes aggregate comparisons of a home or program to groups of programs or geographical locations. Finally, it includes comparison to national samples of similar youth. Applications of the data to individual treatment planning, program evaluation, and research are also included.

Mason, M., Thompson, R. W., Pecora, P. J., & Schmaltz, S. (2001, February). The 10,000 Kids Project: Benchmarking outcomes, service and cost in foster care. Paper presented at the Florida Mental Health Institute 14 th Annual Conference, Tampa, FL.

This presentation is about the creation of a collaborative made up of four agencies that provide foster care. The goal of the collaborative is to benchmark data about clients, services provided, costs, and outcomes. Methods used to create the collaborative (e.g., Memorandum of Understanding) and samples of aggregate data are presented. Future goals of the collaborative, including the participation of more agencies, are also discussed.

Swan, W. W. & Thompson, R. W. (2000). Building an evaluation trail to "scale up" from model demonstration to dissemination of proven practices. Presented at the OSEP Research Project Directors Conference.

This presentation is about a multi-site evaluation of the Boys Town Education Model in the Georgia Psychoeducational Network. The Network is a statewide system of special education programs for children with serious emotional and behavioral problems. Presenters describe an overview of the entire process of replicating and adapting the Specialized Classroom Management Program to this very unique and challenging population and establishing systems to collect, analyze, and report descriptive information and results. Suggestions for other large-scale program replications are discussed.

Thompson, R. W., & Diederich, K. (2000, April). Implementing an agency outcomes system using web-based technology. Workshop presented at the California Alliance of Child and Family Services Key Strategies 2000 Conference, Anaheim, CA.

This workshop covers the design and implementation of an agency-based outcomes evaluation system, which emphasizes utilization of objective data on client characteristics, services, and outcomes for treatment planning and monitoring, program evaluation, and service contracting. It is targeted to an audience of program administrators, and practical suggestions are offered for data-based decision-making.

Thompson, R. W., Sinisterra, D., North, T., & Castrianno, L. (2000, March). An evaluation system for community-based family-centered services. Poster session presented at the Florida Mental Health Institute 13th Annual Conference, Clearwater Beach, FL.

This poster describes an outcome evaluation of a school-based prevention and intervention program for children with school adjustment problems and their families. The emphasis of the program was on providing support and parenting instruction to parents, while consulting with teachers about classroom behavior. The results indicated that participating families reported improvements in parenting stress and child-behavior problems at home and school.

Larzelere, R., Jones, M., Smith, G., & Chmelka, B. (2000, February). Daily symptom data: Multiple uses for clinical decisions and outcome evaluations. Poster session presented at the National Association of State Mental Health Program Directors Research Institute, Washington, DC.

A description of the Treatment Progress Note, a system of tracking daily symptoms for treatment planning, utilization reviews, and outcome evaluation in the Boys Town Behavioral Health programs. The Treatment Progress Note is an expanded version of the Parent Daily Report (Chamberlain & Reid, 1987). Developed by Boys Town, it is an easy-to-use symptom checklist that tracks specific and total symptoms. Detailed descriptions of a variety of uses of the Treatment Progress Notes are included in the poster.