![]() ![]() |
| RESULTS
Themes Agencies are enhancing their case review and foster/adoptive parent recruitment systems to reflect findings emerging from the Child and Family Services Reviews (CFSR) The 2002 phone poll found that child welfare agencies were in the midst of preparing for or responding to the Child and Family Services Reviews (CFSR). The results of the 2003 poll show that agencies continue to feel the impact of the CFSR and are implementing changes in systems based on the CFSR findings. Specifically, 27 (73%) of the agencies responding to our poll, have modified their case review process during the past year in response to ASFA and the CFSR. Several agencies mentioned that their case review system is now based on the CFSR model. Additionally, 23 respondents, (62%) have changed their foster/adoptive parent recruitment process during the last year. Mentioned changes include: involving the business community in supporting and participating in recruitment efforts, hiring a marketing firm to design a campaign to attract families interested in special needs adoption, focusing on recruiting minority foster homes through ads in Spanish on Spanish radio, working in urban centers with black churches and partnering with Indian tribes to reach those populations and using a foster care mascot, ‘Buddy, the Bear’, who attends community-based activities and helps make the public aware of the need for foster parents. Agencies view the Adoption and Safe Families Act (ASFA) from a ‘good practice’ perspective Both the 2002 and 2003 ASFA phone poll analyses indicate that child welfare agencies continue to actively implement the provisions of the Adoption and Safe Families Act (ASFA). Agencies tend to view ASFA requirements as ‘good practice’ and are taking steps, such as joint training, starting interagency workgroups, communicating the shared responsibility for meeting outcomes for children in care and setting up informational websites, to implement this approach throughout the agency and the child protective network. One respondent’s comment sums up what many others stated ‘…case planning and caseworker contact appear to be key to positive outcomes in implementing requirements of both ASFA and CFSR’. Agencies continue to deliver ASFA training and to integrate ASFA related topics into on-going training, meetings and forums Of the 37 child welfare agencies
surveyed in 2003, 33 (89%) reported that their agency continued to provide
ASFA training to their staff during the last year. The most frequently
mentioned training topics were ASFA requirements, time frames for reunification,
concurrent planning and permanency issues. Agency training unit staff
continue to be mentioned most frequently (n =25 or 68% of states responding)
as delivering the training. The percent of respondents reporting that
ASFA training was incorporated into other training increased from 59%
in 2002 to 76% in 2003. There was also an increase in the percent reporting
ASFA training was incorporated into new worker training, from 51% in 2002
to 65% in 2003. The managerial and supervisory skills needed to implement ASFA continue to change while the workers’ needed skill set remains steady Interestingly in the 2001 poll,
‘casework’ was the key skill respondents said was needed by managers,
supervisors and workers to implement ASFA. The results of the 2002 poll
show that while ‘casework’ remains the 1st ranked skill for workers, ‘understanding
the requirements of ASFA’ is ranked number one for managers and ‘communication’
is ranked first for supervisors. In the 2003 phone poll, ‘using data effectively’
was number one for managers, ‘collaboration’ and ‘understanding how to
implement ASFA’ tied for 1st for supervisors and ‘casework’ remained number
one for workers. Barriers to the agency's support for staff as they implement the requirements of ASFA were similar in 2002 and 2003. In the 2003 poll, over three-quarters of the state respondents mentioned lack of resources, staff turnover and heavy workload as barriers to staff support as they implement ASFA. In contrast, in 2001 agencies mentioned lack of resources, difficult relationship with the courts, staff turnover and limitations or lack of computer systems most frequently.
Child Welfare Agency Poll Results Understanding of ASFA Respondents were asked to rate their agency's staff in terms of their understanding of ASFA requirements by ranking the level of understanding of managers, supervisors and workers on a scale of 1 to 5 with 1 being ‘poor’ and 5 being ‘comprehensive’. The average ratings appear below:
In terms of understanding of ASFA, managers ranked ahead of supervisors and workers in polls for all three years. All three groups showed an increase in understanding from 2001 to 2002 and 2003 with both supervisors and workers showing a statistically significant increase (p<.01) over the three years. The most frequent comment regarding worker understanding was that high staff turnover affects understanding, reported by 14% (n=5) of the respondents. Eight percent (n=3) reported that workers had good understanding of some elements, but not others. In both the 2002 and the 2003 polls we compared how respondents rated ‘Understanding of ASFA’ by managers, supervisors and workers in states in the initial phases of the CFSR (planning or assessment) to states in the later phases (review completed or working to complete PIP activities). In 2002 there were statistically significant differences between the two groups, with states in the later phases of the CFSR process rating the ‘Understanding of ASFA’ by managers, supervisors and workers significantly higher than did states in the initial phases. In 2003 there were no statistically significant differences between states in the early and later stages of CFSR. It appears that due to early retirement programs and high turnover, some states gave lower scores to supervisors’ and managers’ understanding of ASFA, even states in the later phases of the CFSR process. Selected Participant Remarks Regarding Understanding of ASFA We had a huge retirement incentive recently and that had an impact on staff throughout the agency. We have many new county directors who need training on many areas, including ASFA. Many of our supervisors are fairly new and thus weren’t part of our early ASFA training and implementation efforts. 50% of our workers have been with us 5 years or less. (AL) We’ve had higher turnover on the worker level than with supervisors or managers so some workers know ASFA well while others would ask what ASFA is all about. (CO) Most child welfare staff who remain in their job for longer periods of time are very knowledgeable. The lower rating is due to high levels of turnover that effect continuity of application of ASFA-related practices. (VA) Skills Respondents were asked to list the skills that managers, supervisors
and workers need to have to implement ASFA. The average rank is based
on scores between 1 (Do not need it) and 5 (Extremely important). The
top skills for each are listed below. Manager's Skills
While ‘casework’
was listed as the number one skill needed by managers in the first poll
(2001), it was ranked last in both 2002 and 2003. In 2003 ‘using data
effectively’ was ranked as the most important skill for managers to have,
up from second place in 2002, and ‘understanding how to implement ASFA’
was ranked number 2. However, if rounded to the nearest whole number,
all of these skill areas except ‘casework’ average a ranking of ‘5’ –
Extremely Important. These differences of less than .1 among these skills
indicate that all of these skills are considered very or extremely important
for managers to have. Supervisor's Skills
For supervisors,
‘using data effectively’ continued to be ranked as the least important
skill needed to implement ASFA, while ‘collaboration’ and ‘understanding
how to implement ASFA’ were ranked first or second by most respondents. Workers' Skills
In the 2003 poll, ‘casework’’ continues to be the skill most often mentioned as important to workers’ ability to implement ASFA. The rankings of the remaining areas are the same as in 2002, with a new category for workers, ‘using data effectively,’ ranked least important. Administrative Activities Respondents were asked if, as part of the ASFA implementation, the agency had undertaken any activities during the past year in certain administrative areas. A summary of the responses appears below. Selected remarks by participants, which appear below as well, are very informative regarding the changes made.
Selected Participant Remarks Regarding Changes in Administrative Activities We are just starting to see generic statements about supporting ASFA principles in job descriptions. (AL) Our agency has incorporated the various aspects of ASFA into the normal day- to -day casework practice, case management and our caseworker training curricula. We view the various requirements of ASFA i.e., concurrent planning, termination of parental rights filings, Establishing Permanency within timeframes etc., as good casework practice and management that if incorporated, would help us better serve the needs of vulnerable children of the state and their families as opposed to viewing them as activities separate and distinct from the case work. (AR) We have increased family involvement in case planning and now relatives must meet the same licensing standards as other care givers. (CA) We updated our case review instrument to include the ASFA elements. We have increased our foster/adoption services staff by more than 10. (CT) We were provided funds by our legislature to staff and implement statewide centralized intake and statewide Title IV-E eligibility determination. (HI) The case plan has been redesigned as a primary tool to assist workers in documenting compliance with ASFA. In addition, a cover letter for court reports prompts the worker and the judge to target ASFA requirements in court determinations. The court improvement project is developing model court orders that will also assist in triggering determinations in compliance with ASFA. (IA) Our agency has launched a more diligent recruitment campaign which has resulted in more analysis of the pool of foster/adoptive parents. We are using more data in this area. (KY) We increased the amount of graduate student stipends to allow more staff to go to graduate school and are using IV-E funds for recruitment on campus. (LA) The Director of Recruitment has expanded the involvement of the business community in supporting and participating in our recruitment efforts. Despite budget cuts, the Department has been able to use television and public events to do recruiting that is focused on the needs of the children in care. This would not have been possible without the collaboration and support of our corporate partners. (MA) We have taken the case management requirements and embedded them in performance appraisals for managers, supervisors and caseworkers. For example, meeting ASFA deadlines would become part of performance appraisal. (MD) We have undertaken various efforts involving the media, including radio announcements, use of billboards, transit posters and have begun using private contractors to recruit foster/adoptive parents. (MD) We have strengthened case reviews; now they are based on the CFSR model. (MN) We have initiated a new review process where cases are selected at random and reviewed at quarterly supervisory meetings. (MT) We have expanded the foster care health program. There are now a total of 8 field nurses who serve foster children. (NH) We have places a greater emphasis on hiring bi-lingual staff. (NC) ‘Buddy, the Bear’ is the Division’s foster care mascot. Buddy attends community-based activities and helps make the public aware of the need for foster parents. Buddy often gets media attention at events he attends. (NJ) The department has a marketing contract with a vendor who provides extensive media coverage via print, radio and television to recruit foster and adoptive parents; use of the Ohio Adoption Photo Listing that features children who are available for adoption; Ohio adoption web page; produced and disseminated a smaller more user friendly adoption magazine that features about 40 children; Adoption and Foster Care Conference that is attended by public and private agency staff, foster and adoptive parents. (OH) Our CQI group continues to perform the internal CFSR process. We are almost ready to release new case review screens to our SACWIS. These screens will mimic CFSR tools and will be available to everyone who has security access. (OK) An outcomes management function has been established which has resulted in an increase in cases that are being reviewed. Requirements of CFSR and PIP have strengthened case review practice in the field. (SD) We recently updated the statewide Recruitment Plan to include Faith Based recruitment. (TX) We have begun to use the CFSR process to review cases. (UT) The agency received an Adoption Opportunity Grant to hire a marketing firm to design a campaign to attract families interested in special needs adoption. We have started using campaign materials, i.e., print, web page. The agency is also participating in a Casey breakthrough program for recruitment of foster families. (VT) Our focus has been on recruiting minority foster homes. We put ads in Spanish on Spanish radio, worked in urban centers with black churches and partnered with Indian tribes to reach those populations. (WA) Training for Agency Staff Of the 37 states surveyed
in 2003, 89% (n=33) of the respondents reported that their agency continued
to provide ASFA training to their staff during the last year. The most
frequently mentioned training topics were ASFA requirements (32%), time
frames for reunification (27%), concurrent planning (24%), and permanency
issues (22%). In terms of methods of training, conducting statewide training
sessions on ASFA was mentioned most often for the third year in a row.
In 2003, 65% of respondents reported that they conduct statewide training
sessions on ASFA. Including ASFA topics as part of in-service training,
which rose from 36.2% of the responses in 2001 to 49% in 2002, remained
at the 49% response level in 2003. Pre-service training, ranked third,
was mentioned by 46% of respondents in 2003. Training Methods
Agency training unit staff continue to be mentioned most frequently (68% of states responding) as delivering the training. Others who were mentioned as providing training included outside consultants (38%), central office staff (38%), state university staff (35%), regional staff (22%), and court staff (19%). These numbers reflect an increase in the use of state university staff from 2003, when 15% of respondents reported using university staff for training. The percent of respondents reporting that ASFA training was incorporated into other training increased from 59% in 2002 to 76% in 2003. There was also an increase in the percent reporting ASFA training was incorporated into new worker training, from 51% in 2002 to 65% in 2003. Only 11% indicated ASFA training was not incorporated into on-going training, compared to 20% in 2002. Selected Participant Remarks Regarding Training for Agency Staff If best practice is followed, ASFA is attended to. Thus, we’ve done quite a bit of practice training but nothing specifically on ASFA. (AL) We trained on how case planning leads to outcomes related to ASFA implementation. (KY) The concepts of ASFA have been incorporated into the training that every social worker hired by the Department receives. (MA) All new workers receive training on ASFA topics, but the topics are embedded in the curriculum and are part of the training--they are not identified specifically as requirements of ASFA. (MN) Supervisory staff struggle with the guidelines for TPR under ASFA so we continue to provide training in this area. (MS) We are totally redoing our training to develop a menu of options depending on the competency needs of the position. The types of training we have offered to date include: Engaging clients, Documentation, Family assessment, Concurrent planning and Risk assessment. The pre-service core training has also been modified to be inclusive of ASFA requirements in case work with clients. (NM) Our training system is great. It has been knowledge based and now we are adding a focus on transfer of learning so that is what the managers and supervisors are learning. (PA) Training with or for Courts Most agencies (65%) continue
to do ASFA related training with or for the courts on topics such as permanency
issues (24%), legal issues (22%), ASFA (22%), timeframes for reunification
(19%) and reasonable efforts (14%). The most frequently mentioned method
of training in 2003 was ‘Court Improvement Project events are attended
by agency staff’, reported by 14 of the 37 states (38%). The percent of
states using other training methods ranged from 0% (agency published newsletter,
ranked 6th) to 35% (agency and court staff working together).
As well as doing training with and for the courts, agencies report doing ASFA related training for tribes (27%), mental health (19%) and health care (14%) providers, schools (8%), foster care providers (5.4%), community stakeholders (11%) judges (5.4%), parents (5.4%), court services (5.4%) and staff in schools of higher education or graduate school (5.4%). One state reported providing training to each of the following groups: child welfare board members, legislators and substance abuse providers. In 2003, agencies were asked
what the number one training need was for their agency and community partners
(not just on ASFA). Their primary responses are as follows
We are developing foster/adopt training to help reduce the length of time in care and reduce multiple placements. In DE 65-68% of adoptions are foster parent adoptions. (DE) Foster parents have received training in how to recognize the role they play in meeting ASFA requirements. (ME) The tribal workers were invited to agency trainings and new worker trainings. Tribes have also arranged for training on IV-E and ASFA given by NICWA. (NE) Providers are limited by a lack of knowledge about kids involved with the agency; they could benefit from training on the behavioral health needs of kids in care. (WV) Information Systems Respondents were asked to rate on a scale of 1 to 5, with 1 being ‘poor’ and 5 being ‘outstanding’, the support provided by the state information system for the work of managers, supervisors and workers in implementing ASFA. The average rating was 3.14 in 2001, rose to 3.43 in 2002, and to 3.51 in 2003. In both the 2002 and 2003 polls, county administered states gave their information system a lower rating (average 3.14 in 2003) than state administered states (average 3.69 in 2003). This was a statistically significant difference (p<.05). When the responses of agencies in the later phases of the Child and Family Services Review (review completed and working under a Program Improvement Plan) were compared to those agencies in the initial (planning and assessment) phases, states in the later phases gave their systems a higher rating (average 3.75) than states which are in the earlier phases (average 3.25). These results were similar to findings in 2002, but were not statistically significant. Selected Participant Remarks Regarding Information Systems The system doesn’t allow us to extract the information we need to help us know how workers are doing. Not all areas of the state are on the system. There is a high rate of worker error. (AK) As additional management reports continue to roll out, our SACWIS (TRAILS) will become more and more useful. The data is in there; we’re just now figuring out how to get it out in a meaningful way. (CO) Our information system has made tremendous progress in recent years. We could not begin to address CFSR issues without TWIST (KY's SACWIS system) (KY) The Division has developed a web-based system, the ‘Permanency Tracking System,’ to assist supervisory and management staff in their case review responsibility. The system captures important information and alerts supervisors to upcoming due dates and other significant events. (NJ) We now have people who put the information that needs to be there into the system; at least for child welfare people entering data is second nature. The system gives you good data. I wish the system were child based not family based so information about the nature of the agency’s status with a particular child would be more accurate. (RI) Staff Support In previous years, respondents
were asked what activities worked well in supporting staff as they implement
ASFA requirements. In 2003, state respondents were asked to rate the activities
mentioned in years 1 and 2 of the study from 1 (‘not important’) to 5
(‘crucial’). ‘Provide management support’ was ranked number 1 in 2003,
as it was in 2002, and ‘view ASFA as good casework practice’ was still
ranked second, with ‘training’ still ranked third. However, all of these
items were ranked close to the ‘crucial’ category; indeed, if they were
rounded to single digits, they would all be equal to the ‘5’ crucial designation. Most Effective Practices for Staff Support
Barriers to the agency's support
for staff as they implement the requirements of ASFA were similar in 2002
and 2003. In the 2003 poll, over three-quarters of the state respondents
mentioned lack of resources, staff turnover, and heavy workload as barriers
to staff support as they implement ASFA. Over half the states also mentioned
large numbers of cases and lack of services as barriers. Two barriers
mentioned by one state each were backlog of cases in court and a disproportional
number of African American children in care.
Of the 37 states where the poll was conducted, 8 were state supervised, county-administered states. These states were asked if any of a list of items impacts their ASFA implementation. Among the 8 states, the following number of states checked these items:
Child and Family Services Review Process The poll asked agencies to identify where they were in the Child and Family Services Review (CFSR) process.
We are through the CFSR, received our final report, submitted our PIP and had it approved. Involving everyone in the process increased familiarity with ASFA in persons throughout the agency. (AZ) A significant lesson for Kentucky was the importance of analytical skills. We are able to produce lots of reports and staff are using the reports. We still need more depth in understanding the implications of the data. (KY) Case planning and caseworker contact appear to be key to positive outcomes in implementing requirements of both ASFA and CFSR. (TX) The CFSR taught us the need for greater consistency in child welfare practice--the need to improve focus on family needs and not only the target child, the need to adopt child welfare practice standards for all populations served by the Department including delinquent youth, the need for a good quality assurance program, the need for greater availability of services and the need for more worker contact and improved quality of that contact. (WY) General Remarks Respondents were asked if there was anything else they wanted to tell others regarding the best way to implement the requirements of ASFA and the CFSR. Five people (14%) mentioned that they were able to use lessons from ASFA and the CFSR to improve practice; four (11%) recommended that agencies understand staff needs and four that they regard ASFA as best practice. Selected Participant General Remarks Present, expect and normalize ASFA as best practice. (AL) To support staff on the requirements of ASFA and the CFSR
The accuracy of data in your SACWIS and the investment of stakeholders in service delivery are so important. (CO) Need to involve non-custodial parent and get the judicial system on board. (CT) We have seen the value of bringing information to the first line staff, especially in peer case review process. Staff liked doing the review. Plan to continue after the CFSR review is complete. (LA) ASFA is the foundation for the work we do. It's not the language we are after, it's the outcomes we want to use in a productive way. (NE) Prioritizing how states can best address the staff resource/turnover issue is integral to any further improvement. (VA)
Court Project Poll Results Understanding of ASFA by Agency Staff Court improvement project (CIP) personnel were asked to rate the child welfare agency staff in terms of their understanding of ASFA requirements on a scale of 1 to 5 with 1 being ‘poor’ and 5 being ‘comprehensive’. The average rating given in the 2001 poll was 3.25, which rose to 4.0 in the 2002 and 2003 polls. Participants noted that, while mangers understood ASFA requirements, some ‘field staff didn’t seem to understand the requirements as well,’ although they have improved from previous years. One interviewee found that state staff understood ASFA, but some local staff ‘appear to never have heard of some of the provisions’; another responded that ‘change in leadership and a move to prioritize child welfare services may impede progress’. Skills Court improvement project respondents were asked to rate the skills (on a scale of 1 to 5 with 1 being ‘don’t need it’ and 5 being ‘extremely important) that child welfare supervisors and workers need to have to implement ASFA. The skills mentioned most often are listed below. Supervisor's
Skills
Administrative Activities Court improvement project respondents were asked if, as part of the state’s ASFA implementation activities, the court and the child welfare agency had undertaken or changed day-to-day joint activities and communication. Respondents reported that changes had occurred in both areas. Following are selected comments regarding these changes. Day to day joint activities Some courts and agencies are ensuring that the next court date is scheduled during each hearing. In some jurisdictions, court reports have been modified to designate a certain part of the court report that addresses reasonable efforts. (MD) About a year ago, the agency
and its attorneys started submitting an ASFA 12 month permanency report
as a way of improving the 12 month hearing. We are starting to see the
positive impact when we receive that report in a timely way. The agency
and court worked very hard to implement the Subsidized Legal Custodian
option now available under our law. We needed to work out many issues
and develop a complete policy to support this new option. There was a major training for Guardians Ad Litem. It was part of their required training. Consisted of a two-day session on ASFA. Attended by GALs and social workers, judges also. (WI) Communication A change in administration has curtailed communication at the state level. (FL) Some jurisdictions are having regular multi-disciplinary meetings with all of its child welfare stakeholders to address ASFA-related issues. (MD) The court and agency have a positive, productive relationship at all levels. We have a special courtroom and a committee that focuses on large families (4+ kids) and their unique service needs, which often include finances and housing. We have 10,000 court involved cases and 25% of them involve large families. Also we have a special aging out program and courtroom designed to help move kids into Independent Living. The court and agency meet in the CIP committees and subcommittees monthly to address problems and issues. (PA) Communication has progressed since last year due to the various trainings that involve court and agency staff and because of preparation for the CFSR. Also, there is a good relationship between the director of the court improvement project and agency staff. (WI) Sharing Information The project team asked court improvement project respondents if they were getting information they needed regarding cases from the agency. Three of the respondents (75%) said they were getting the information, while one (25%) noted that ‘the agency does not file the required court reports or they are not filed timely’. Training Respondents were asked how they handled training on ASFA for court and agency personnel. All four respondents indicated that during the past year there has been ASFA training for court personnel, for the child welfare agency, and for attorneys. Permanency issues were the number one training topic for court personnel (4 respondents) and for attorneys (2 respondents), while training for child welfare workers covered myriad topics such as Title IVE , performance contracting, use of forms and the CFSR process. These topics were also covered in training for attorneys and court personnel; in addition, court personnel received training in guardianship, timelines, orders and findings, concurrent planning, terminology and family conferencing. Training was generally provided jointly, and/or at conferences, meeting and seminars. Three of the four respondents reported that training in ASFA was incorporated into other training. Court personnel were trained by CIP personnel (n=3), court staff (n=2), attorneys (n=1) or consultants (n=1). When asked what the number one training need was for judges, responses were: ASFA and permanency planning hearing requirements (2 responses), including completion of court orders; expediting petitions for termination of parental rights; and extraordinary findings for continuing case plans. For court personnel, the number one training need was felt to be: case management to assist judges in moving cases; the interplay between agency and court personnel, identification of roles and responsibilities; use of the computer system; and court orders. For child welfare staff, the number one training need was considered to be: Basics of ASFA; responsibility for attending court hearings and reporting on ASFA requirements; understanding of time frames; and concurrent planning (2 responses).
In the 2002 poll, court improvement project respondents were asked to rate (on a scale of 1 to 5 with 1 being ‘not important’ and 5 being ‘crucial’) the importance of specific activities in terms of working with the child welfare agency as it implements ASFA requirements. The most important activity, according to these respondents, was maintaining a good working relationship between agency and court staff. This was rated ‘crucial’ by all four respondents. Barriers As for the state interviewees, the court improvement respondents saw lack of resources, cited by three of the four persons interviewed (75%) as the primary barrier to implementing ASFA. Other barriers cited were: ‘lack of understanding of the key stakeholders’ responsibilities and how they are structured’ (3 responses, 75%), ‘timelines for TPR are too short, unable to make informed decisions in every case’ (2 responses), ‘lack of understanding of the law’ (2 responses), ‘federal and state law not clear on what a permanency hearing is’ (1 response), ‘some of the provisions of ASFA are nor clear’ (1 response); and ‘lack of information on how the statewide roll out of privatization will ensure adequate training of new providers on ASFA’ (1 response). General Remarks Any encouragement from the Children’s Bureau on their expectation that the courts and executive branches continue to jointly strive for standards of excellence in achieving safe and permanent homes for children and their families (would help in the implementation of ASFA). (FL) Provide resources so that the jurisdictions can receive one on one training on ASFA. Having persons who are available to travel throughout the State to provide 2 to 4 hour training sessions periodically would appear to be beneficial. The smaller, shorter seminars, discussions work with the local stakeholders seem to foster collaboration and a better understanding of the law and issues surrounding ASFA. (MD) Training is critical. Need to have an atmosphere where people in the state can talk about their mistakes and what can be learned from them. A supportive environment makes it possible for stakeholders to work together more easily. The result is the development of understanding and respect among stakeholders that has allowed us to move forward. (WI)
>> Continue to the Next Section
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||