Implementation Challenge – New Jersey Department of Children and Families

How do we build staff capacity and enthusiasm for using data to manage and make meaningful practice change?

New Jersey’s Department of Children and Families has undertaken a massive reform effort begun in 2006. The agency worked hard to implement a case practice model emphasizing their focus on family engagement, increase training for all workers, reduce the number of children in NJDCF care, and reduce caseloads. In addition, recognizing that managing by data was important to track progress and increase accountability, the state invested substantial resources to increase the capacity of their case management data system. Being responsible for the safety, permanency, and well-being of children throughout the state requires the approximately 7,000 employees to be working collaboratively with the same goals in mind. Managers, however – from executive leadership through casework supervisors – tended to be uncomfortable using and talking about data.

The Approach

NCIC worked with NJDCF and their partners to develop an 18-month program, known as DCF Fellows, with a curriculum that included project work analyzing live administrative data, conducting quality reviews of actual cases, and developing solutions to specific challenges identified by NJDCF leadership. Acceptance in the program was competitive, with a final class of 100 Data Fellows selected from mid-level management applicants from across the agency. Executive leadership participation was a critical component throughout the program design and implementation.

NCIC’s Role

Assisted agency in understanding technical and adaptive challenges and designing their intervention through a survey of best practices in other successful states around managing with data.

Brought the agency “up on the balcony” at key points in the work to see current organizational patterns. This was most critical during the design phase, halfway through the implementation, and at its completion.

Focused on sustainability through targeted plans for evaluation, dissemination, and future funding.

Brought in experts from across the country at key decision points to provide information and ideas.

Provided financial resources to allow the agency to contract with the experts who were a right fit with New Jersey in providing the intervention.

Results

Improved staff capacity. 81% of Fellows reported “tremendous” or “substantial” improvement in their data analysis skills and 64% reported the same levels of improvement in public speaking and presentation skills. Six month follow-up survey findings reveal that the vast majority of Fellows find the skills taught are useful in their day to day work.

A better candidate pool for promotion. 40 of the 93 Fellows (43%) were promoted during the program. A combination of the program recruiting talented staff, managers valuing the skills those staff developed, and the program’s strong reputation appear to have contributed to promotions.

A model established for data-driven management. Fellows used the Manage by Data model consistently in each of 19 small group projects, several of which were implemented locally and associated with measurable improvements in outcomes for children and families.

Improved child protective investigations. The Fellows analyzed investigation completion timeliness in several local offices. To their surprise, they found the data did not support the common belief that offices taking more time produced better results. With that knowledge, they used the data to identify which offices were both faster and produced better results. They discovered while it was counter-intuitive to some, offices that took the time to have supervisors conference investigations with staff saved time – and produced better outcomes. Using that information, they changed practice in a struggling office – and doubled the rate of investigation timeliness.

Improved Child Abuse hotline performance. The Fellows learned that hotline screeners can improve investigation practice in the field if they can reduce the rate of anonymous referrals. The Fellows looked at the data and discovered that some screeners had very low rates (3-8%) of anonymous referrals while others had very high rates (almost 50%). The Fellows listened to the recorded calls of both groups of screeners. They discovered the low rate screeners persuaded more callers to give their names by using strong engagement skills. The Fellows developed a revised script for screeners and anonymous referrals went down significantly, a decline of more than 2,000 over 18 months.

Improved service delivery. Using census data to analyze the demographics of the populations served in several local offices, the Fellows found many areas had changed significantly over the previous decade. More families were speaking Spanish and more were being referred from towns that previously had few referrals. The fellows recommended changes to service contracts to increase accessibility through changing the location of service delivery and increasing the numbers of Spanish-speaking staff.