Final Performance Report Choice Enhancement and Empowerment Project Prepared for: Center for Independent Living 2539 Telegraph Avenue Berkeley, CA 94704 Prepared by: Lita Jans, Ph.D. Stuart Hanson Susan Stoddard, Ph.D. InfoUse 2560 Ninth Street, Suite 216 Berkeley, CA 94710 510-549-6520 December, 1999 Final Performance Report Choice Enhancement and Empowerment Project Abstract Choice Enhancement and Empowerment Project (CEEP) Grant Number: H235D30024 Center for Independent Living 2539 Telegraph Avenue Berkeley, CA 94704 510-841-4776 The Choice Enhancement and Empowerment Project (CEEP) targeted the approximately 60,000 persons with severe disabilities residing in Alameda and San Francisco Counties, particularly those from ethnic minority communities and people with mental disabilities. CEEP was one of seven Choice Demonstration Projects funded by the Rehabilitation Services Administration (RSA). CEEP was designed to empower people with disabilities to make vocational choices and to expand the scope of vocational rehabilitation opportunities available to people with disabilities. To initiate services, CEEP participants attended a group orientation session, and met with a counselor to complete the intake process and begin work on a written rehabilitation plan. Upon completion of an approved plan, participants made purchases of training, equipment, job placement, and other services. During its six-year duration, the project served 494 participants. Overall, 60% of CEEP consumers achieved or made substantial progress toward their vocational goals. Of successful consumers, 32% reported self-employment, 25% reported employment in a job, and 43% reported that they were empowered and had made substantial progress toward their vocational goals. Consistent with its goals, CEEP served a high percentage of ethnic and cultural minorities: 51% of consumers were non-white. Among the ethnic minority groups, Native Americans showed an especially high success rate- 71% achieved or made substantial progress toward their vocational goals. People with a range of disabilities participated in CEEP. A striking 32% of consumers identified a mental disorder as their primary disability, and the success rate of this group of participants was comparable to that of other participants. For independent living centers and other organizations considering implementation of an informed choice approach to vocational rehabilitation, CEEP has demonstrated a model with many strengths, particularly in serving minority populations and people with mental disabilities. The CEEP model stressed (1) group intake and orientation, (2) empowerment workshops provided by professional consultants, (3) a focus on creative self-employment, (4) an open (non-vendorized) provider system, and (5) consumer control and a high level of flexibility to meet individual needs. Three major products resulting from the demonstration project are (1) Group Orientation and Intake, (2) Empowerment Workshops, and (3) Open Provider System Model. Interested parties may obtain these products by accessing InfoUse's Promising practices from the Choice Demonstration Projects: An operations manual (Stoddard, Hanson, & Temkin, 1999b) on the Internet at http://www.infouse.com/choice. Comparison with other Choice projects revealed that CEEP had somewhat higher levels of reported choice and satisfaction than the levels reported across all the Choice projects. CEEP spent a larger than average percentage of resources on purchasing goods and services for participants, compared to other projects. In contrast to other Choice projects, CEEP purchases were primarily business and vocational training, college and university training, and computers for work or self-employment. Table of Contents Abstract i Table of Contents iii Summary of Project Accomplishments 1 Goals and objectives 1 Evaluation methodology 1 Highlights of major achievements 2 Challenges and difficulties encountered 4 Program Description: Services and Participants 7 How consumers were served 7 Project staff 7 Purchasing goods and services 8 Consumer participants 8 Products available and contact person for further information 9 Will the project be sustained after Federal funding ceases? 9 Will the project maintain the same level of services as federal funding decreases? 10 Major Evaluation Findings: Outcome Measures and Their Relationship to Consumer and Service Characteristics 10 CEEP success rates 10 Purchased services costs 10 Length of service 12 Underserved groups 12 Level of education 17 Major Evaluation Findings: Program Strengths and Weaknesses 17 Orientation 17 Empowerment training 19 Open provider system and vendor relationships 20 Innovative approaches to self-employment 22 Counseling and consumer choice 26 Payment systems 27 Criteria for success 28 Project isolation and tensions in the case management role 28 Dissemination and replication of project results 29 Major Evaluation Findings: Comparing CEEP with other Choice Demonstration Projects 30 Overall Choice and Satisfaction 30 Patterns of Purchased Goods and Services 32 Average Annual Costs Per Participant 35 Lessons learned: Recommendations for Other Organizations Considering Implementation of This Model 38 References 41 Appendix: Project Materials 43 Orientation Guidelines 43 Eligibility Criteria 45 "I Hate to Budget" 47 Workshop Evaluation 50 Make Informed Choices When Hiring Contractors! 51 Final Performance Report Project Title: Choice Enhancement and Empowerment Project Project Number: 84.234K Project Performance Period: From October 1, 1993 to September 30, 1999 Grantee: Center for Independent Living 2539 Telegraph Avenue Berkeley, CA 94704 510-841-4776 Summary of Project Accomplishments Goals and objectives The Choice Enhancement and Empowerment Project (CEEP) was sponsored by the Center for Independent Living in Berkeley, California (BCIL), a pioneer in the Independent Living movement. BCIL is a non-profit organization with offices in Berkeley and Oakland; a majority of board and staff members are people with disabilities. During six years of operation, the project targeted the approximately 60,000 persons with severe disabilities residing in Alameda and San Francisco Counties, particularly ethnic minorities and people with mental disabilities. Consistent with independent living philosophy, CEEP aimed to: 1. Empower persons with disabilities to make vocational choices, 2. Enhance the scope of vocational rehabilitation opportunities traditionally available to persons with disabilities, and 3. Target minority populations, particularly Latinos and Asian Americans, and persons who had a prior unsuccessful experience with the state Vocational Rehabilitation (VR) system. Evaluation methodology This evaluation report is based on interviews with key personnel, board members, consumers, and vendors. We reviewed consumer case files, linked and analyzed the consumer and accounting databases, and reviewed previous evaluation reports and relevant project documents. The evaluators reviewed the 494 case files from the project and conducted in-depth interviews with 34 consumers randomly selected from four outcome categories used in the project: 1. Successful consumers who reported self-employment (95 case files, 11 interviews), 2. Successful consumers who reported jobs (73 case files, 9 interviews), 3. Successful consumers who reported that they were empowered and had made substantial progress toward vocational goals (127 case files, 10 interviews), and 4. Consumers who reported that they were not successful or did not complete their plans (199 case files, 4 interviews). This evaluation focused primarily on interviewing participants in the successful categories and we planned to interview fewer participants who were not successful or did not complete. Not surprisingly, participants in this category were harder to locate, particularly those who had ended unsuccessfully a number of years ago. We had access to a number of interviews with unsuccessful participants, completed in 1996 as part of the Year Three local evaluation (Jones & Jans, 1996b). We used those interviews to fill out the picture of consumers who were not successful. The InfoUse evaluation team also interviewed past and present project directors, current staff members, board members, and vendors. The accounting department supplied relevant variables from the accounting database and we linked those variables to the consumer database in order to calculate purchased service costs per individual. Previous evaluation reports that we reviewed are cited in the References section, and project documents are included in the Appendix section. Highlights of major achievements Overall, 60% of CEEP consumers were successful in achieving or making substantial progress toward their vocational goals. Of successful consumers, 32% reported self-employment, 25% reported employment in a job, and 43% reported that they were empowered and had made substantial progress toward their vocational goals. The counselor, on the basis of a final telephone or in-person interview with the consumer, made the programmatic determination of whether a participant had been successful. Self-employment success. Creative self-employment was a particular strength of the project, described in more detail in the section on self-employment outcomes. Success in self-employment encompassed a wide range of different employment activities, from part-time dog grooming to full-time computer consulting. Some successful participants attained self-employment that was completely self-sustaining. Other consumers who reported success in self-employment chose specifically not to generate a level of income that would cause them to lose their disability benefits. These consumers were grateful that CEEP respected this goal as a valid choice. Job success. Participants who successfully completed in the job category reported a variety of jobs, from very part-time (a few hours a week) to full-time employment. Teachers were the largest category of professions in this category, followed by people engaged in computer work (word processing, programming, sales, manufacture, and graphics) and social services work (counselor, advocate, social worker, and nurse). Success through empowerment and progress toward goals. People who reported empowerment and substantial progress toward attaining vocational goals were often still engaged in working toward their goals by the end of the project. A few had experienced health problems that forced them to postpone the completion of their goals, but felt that they had made solid progress. Some were attending school when the project ended. Others reported that their plans and/or the project had run out of money before they had fully achieved their goals. Almost without exception, the empowered consumers felt that their counselors supported them and encouraged them to believe in their ability to make choices within the constraints of their individual situations. They felt that CEEP project staff placed value on their improved personal well-being, even if they had not attained employment. Project staff and consumers also saw goals clarification itself as a valid product of participation in CEEP. Purchased services. CEEP provided education, training, and equipment resources to many consumers who might not have been able to access such resources through other avenues. The cost of purchased services averaged $3,736 per participant who purchased any services. The average purchased services cost of successful cases was $4,207 per participant, ranging from an average of $3,641 for consumers who made substantial progress on their goals to an average of $5,093 for consumers who reported achieving their self-employment goals. Ethnic minorities. Consistent with program goals, CEEP served a high percentage of ethnic and cultural minorities: 51% of consumers were non-white. Among the ethnic minority groups, Native Americans showed an especially high success rate -71% achieved their vocational goals. People with mental disabilities. A striking 32% of consumers identified a mental disorder as their primary disability, and the success rate of this group of consumers was comparable to that of consumers whose primary disability was not a mental disability. Philosophy of choice and consumer control. When consumers, project staff and others we interviewed were asked what was unique about CEEP, they talked with great excitement and enthusiasm about a philosophy of choice and an emphasis on consumer control. Although philosophy of choice and consumer control are hard to precisely define and quantify, they were seen by many as the "hallmark" of the CEEP model. Compared to other Choice projects nationwide, CEEP participants reported somewhat higher levels of choice and satisfaction than the levels reported across all the Choice projects. Counseling relationship. Consumers frequently identified the encouragement and support of the counselors as critical to their success in attaining goals. The majority of consumers we interviewed felt personally supported and "cheered on" by the counselors. Flexible approach. Flexibility in working with individual needs was another frequently cited strength of CEEP. For example, counselors sometimes conducted home visit intakes with people who had great difficulty leaving the house, and some of these people were able to attain their vocational goals. We identified a number of elements of the model that seem to embody the CEEP philosophy and could potentially be replicated. These include group orientations, empowerment seminars, and an open market approach that allowed people to "use the marketplace" to find services and products. A creative approach to self-employment was another strength of the project that could be emulated. Finally we discuss elements of the project and the counseling relationship that seem to promote consumer control, empowerment, and a flexible approach to individual needs. Challenges and difficulties encountered Sponsored by and housed at a center for independent living, the CEEP created a unique model of choice, but also faced particular challenges in undertaking such a project. There are lessons to be learned from aspects of the project that did not work optimally, as well as from the successes of the project. Difficulties encountered in the implementation of CEEP can be instructive for other organizations, especially independent living centers, that might attempt to build on the CEEP choice model. Turnover. CEEP experienced a great deal of turnover during the project. According to the Research Triangle Institute (RTI) evaluation, project staff turned over 100% during the first two years, causing numerous other problems and delays in implementation (MSPD Evaluation Support Center, Research Triangle Institute, 1995). During the duration of the CEEP, there were three different BCIL executive directors who served as project directors for the CEEP. In addition, there were three different sets of personnel in the BCIL accounting department, which was critical to the disbursement of monies to consumers. However, the project manager stayed with the project from Year 2 until the end, and there was only one change in the counselor positions during the same period. Difficulty defining success. Throughout the project, the criteria for success remained ill-defined and subjective. The absence of a clear definition of success in the CEEP was noted in the Research Triangle Institute's national evaluation (MSPD Evaluation Support Center, RTI, 1995) as well as in the local evaluations for Years Two and Three (Jones & Jans, 1996a; 1996b). Rejecting the idea that "choice" could be measured simply in terms of vocational outcomes, project staff wrestled with how to define success in a way that did justice to the alternative goals of the project. Participant empowerment was viewed as essential, and a participant's enhanced independence and personal well-being were seen as important project outcomes. However, the assessment of success came down to the counselor, on the basis of a final interview, deciding whether the participant felt empowered and had made substantial progress toward goals. Sometimes there were data to substantiate success, but other times the assessment was more subjective. This may have inadvertently created a pressure on participants to say that they had been successful, absent more objective ways of evaluating progress. A measurable set of criteria would have enhanced the credibility and replicability of the project. Problems with the process of approving and making payments. CEEP complied with all federal guidelines for financial accountability, developing a complex system of approval and payment for consumer goods and services. The accounting department reported that 95% of vendors were paid within 60 days, an exceptional record by industry standards. Nevertheless, the system for approving and making payments created communication problems and was perceived by many consumers and counselors as rigid, cumbersome, and far too slow. The project had an extreme impact on the accounting department at BCIL. One former executive director estimated that handling payments related to CEEP comprised 75% of the workload of the accounting department. The system of payment approval created strains between CEEP staff and the accounting department and resulted in numerous delays in payment. Participants and project staff complained bitterly about how long it took for providers of goods and services to receive payment. Consumers worried that payment delays made them look embarrassingly unprofessional and that service providers would see them as "second class citizens." We believe that many of these difficulties arose from unresolved conflicts between the need for accountability and the desire to provide maximum levels of funding and services. These issues are also discussed below, in the section on case management and the dual roles of advocate and services broker. Gaps in business expertise. In previous evaluations, consumers with self-employment goals complained that project staff did not have enough expertise in reviewing business plans and guiding entrepreneurial activities. During interviews in Year Two and Year Three of the project, project staff affirmed that they had a need for more training in these areas (Jones & Jans, 1996a, 1996b). Over the course of the project, staff increasingly developed expertise in guiding small businesses, as well as resources for referring participants for consultation with other experts. Nevertheless, it seems important for future projects with a self-employment focus to create these systems of expertise and resources before the project even begins. Networks should include people with expertise in business start-up, business plans, marketing, financial management, and other related topics. Project isolation. Many of the CEEP and BCIL staff whom we interviewed mentioned that CEEP was isolated from the rest of the BCIL. It is not unusual for such a project to become isolated when a relatively large amount of grant money for a specific project comes to a grass-roots advocacy organization. Nevertheless, CEEP's status as a project relatively separate from the rest of the CIL's activities took its toll in terms of strained relationships within the organization. This indirectly impacted consumers. For example, one project director thought that CEEP participants could have made better use of the other services of the BCIL, such as IL skills training, benefits counseling, housing, and advocacy. If CEEP had been wholly integrated within the organization as originally contemplated in the initial grant proposal, much of BCIL's strengths could have been brought to bear on the consumer's issues. Case management and the dual roles of advocate and resources broker. Whenever a grass-roots advocacy organization takes on a case management role, tensions emerge between the desire to provide consumers with resources and the need for fiscal accountability. In CEEP, for example, conflicts erupted between the project staff who tended to favor a system with maximal flexibility for consumers, and the accounting staff and executive directors, who tended to press for more rigorous accountability and control over funds. No system of disbursing funds can completely eliminate conflicts between the "gatekeeper" and "service provider/advocate" functions that are part of brokering services. In the section on lessons learned, we make some recommendations on ways in which organizations seeking to replicate the CEEP model might work with the tensions between gatekeeper and service provider/advocate functions. Program Description: Services and Participants How consumers were served Participants learned about CEEP from friends, contacts in social agencies, community organizations, and the California Department of Rehabilitation. After attending an orientation session with other applicants, an individual met with a counselor to complete the application process and begin working on the design of a written rehabilitation program. The length of time it took to put a rehabilitation program into writing varied, depending on the participant's goals, the barriers to achieving those goals, and the participant's level of motivation. When a participant and counselor agreed that the plan was complete, it was given to the project director for approval. Upon approval of the plan and funding, the participant could make the purchases of training, equipment, job placement services, and other assistance. Many CEEP participants expressed interest in starting small businesses. Those with self-employment goals were required to complete business plans as part of their vocational plans. During the process of designing and/or completing vocational plans, consumers were offered the opportunity to attend workshops that were open to all participants. Topics ranged from "Dressing for Success" to "Starting Your Own Business." Because of high participant interest in starting small businesses, a number of workshops focused on issues related to self-employment. Project counselors functioned as technical advisors, using their experience with many different plans and purchases to provide feedback to participants about the basic elements of solid workable plans. Staff encouraged participants to shop around, find the best combination of product and service features available to them, and negotiate the price. During most of the project, there was no pre-established list of vendors although one had been proposed as part of a voucher system in the original proposal. Project staff During the last three years, CEEP employed a part-time project director (the executive director of BCIL), a full-time project manager, two counselors, an accounting assistant, and a part-time secretary. (The original position of intake coordinator was reconfigured to hire two counselors, in an effort to serve more participants [MSPD Evaluation Support Center, RTI, 1995].) Project staff arranged orientation and empowerment seminars for project participants. The project manager and counselors provided counseling, guidance, placement services, and all types of case management services. Purchasing goods and services The majority of goods and services purchased under CEEP were for business and vocational training, computer equipment, college and university education, and self-employment inventory/set-up. Most goods and services were purchased from educational institutions and the open market (commercial merchandise retailers) (Stoddard, Hanson & Temkin, 1999a). Participants were able to use vouchers if they were acceptable to vendors. A participant could also obtain a quote or calculate the actual price in advance and request a check made out to the vendor. In some situations, the participant paid for a service or product out-of-pocket and brought in the receipt for reimbursement. When a check request was received, the accounting department compared the request to the approved amount and disbursed the money, as long as the amount requested did not exceed the amount approved by more than ten percent. Consumer participants The CEEP project served 494 consumers, with 295 completing as successful (60% success rate). The average length of time in the program was 22 months. CEEP served a high percentage of ethnic minorities: 51% of participants were in the non-white category. These included African-Americans (29%), Asian-Americans (7%), Latino/Hispanic-Americans (5%), Native Americans (1%), and multi-racial or other (9%). About 49% were white. People with a wide range of disabilities were represented. About one-third of participants reported a mental disability as their primary disability. As a group, CEEP participants had relatively high levels of education. About 9% had less than a high school degree. Seventeen percent (17%) had obtained a high school degree or equivalent but not gone on to college. Seventy three percent (73%) had at least some college; 33% had an associate degree or some college; 27% had a college degree; and 13% had a graduate degree. More detailed information on the characteristics of consumers and their relationship to project outcomes are presented the Major Evaluation Findings section. Products available and contact person for further information The CEEP project produced several documents and materials. Some of these products are included in InfoUse's Promising practices in the Choice Demonstration Projects: An operations manual (Stoddard, Hanson, & Temkin, 1999b). This document is available on the Internet at http://www.infouse.com/choice. Copies of CEEP products are also attached to this report. Other documents may be obtained by contacting: Jan Garrett Executive Director Berkeley Center for Independent Living 2539 Telegraph Avenue Berkeley, CA 94704 (510) 841-4776 (Voice) (510) 848-3101 (TDD) (510) 841-6168 (FAX) Will the project be sustained after Federal funding ceases? Federal funding for CEEP ended on September 30, 1999. In preparation for the end of federal funds, open cases were referred to the California Department of Rehabilitation. On September 30, 1999, all project staff were laid off or transferred to other jobs within CIL, and all project activities ceased. However, CEEP continues to inspire new efforts to promote employment outcomes among working-aged people with disabilities in the Bay Area. For example, BCIL, in collaboration with the World Institute on Disability and the community-based Computer Technologies Program, has launched a series of seminars, entitled the "Maximizing Independence Project." These seminars bring together consumers, business professionals, and human services representatives at "Town Hall" meetings to discuss choice in employment. The Maximizing Independence Project uses empowerment principles and training formats developed in CEEP to address work incentive and health care issues in relation to employment and self-employment opportunities for people with disabilities. The seminars will take place in early 2000 with support from the California Department of Rehabilitation and the California Employment Development Department. Will the project maintain the same level of services as federal funding decreases? No. See above. Major Evaluation Findings: Outcome Measures and Their Relationship to Consumer and Service Characteristics CEEP success rates When a participant completed a plan, project staff interviewed the participant to determine whether he or she reported having a job or self-employment, or felt empowered and had made substantial progress in attaining vocational goals. All of these were considered successful. For purposes of this report, all other cases were grouped in the category "Not successful/Did not complete." The following table shows the number and percentage of people in each category: Table 1: Success Rates Group Number Percent Successful 295 60% Self-employed 95 19% Job 73 15% Progress 127 26% Not successful/Did not complete 199 40% TOTAL 494 100% Purchased services costs To better understand the costs and benefits of CEEP services, the evaluators looked at the cost of purchased consumer services, using information supplied by the BCIL accounting department. For the purposes of this report, only monies that could be directly linked to a specific individual consumer were included. Other direct costs, such as bus and BART passes for transportation, and the costs of orientations, empowerment training, job fairs and assistive technology events were not included. The sum total of purchased services costs included in these calculations was $1,490,476. The table below shows the average purchased services cost per participant in each of the groups. Table 2: Purchased Service Costs Group Purchased service cost per participant (average) Successful (n = 295) $4,207 Self-employed (n = 95) $5,093 Job (n = 73) $4,014 Progress (n = 127) $3,641 Not successful/Did not complete (n = 199) Began plan, received some services (n = 110) $2,496 Did not complete plan (n = 89) $0 Any purchased service (n = 405) $3,736 Technical note: In all of the tables that show average purchased service costs, the average is the total amount for the group divided by the number of people in the group who had purchased services. People who did not have purchased services are not included in the calculations. It is not surprising that successful cases received more purchased services, on average, than cases that were not successful. Nevertheless, a substantial amount per case (an average of approximately $2,500) was spent on consumers who began a plan, received some services, but did not complete the plan. Interviews with staff, the program manager, and program directors indicated that many of the people who were not successful needed more intensive "pre-employment counseling" and independent living skills training than the project was able to provide, given its employment focus. Some non-successful consumers we interviewed supported this viewpoint. As one participant put it, "My personal problems (finding suitable housing) interfered with my education." However, other consumers who were not successful expressed bitter anger at CEEP and did not believe that more IL skills training or pre-employment counseling would have helped. One participant felt "sort of seduced" by the enthusiasm and encouragement the project gave her, but ended up feeling ultimately betrayed when her goals were not attained. These findings indicate that people with self-employment goals received more purchased services. There are a number of possible explanations for this. Self-employed people were probably more likely to need computers as part of their plan. They may also have had higher costs associated with self-employment inventory and set up. Business training costs (such as business plan and marketing consultation) probably also contributed to higher purchased services costs. Length of service We also examined length of service for each of the groups, as shown in Table 3. Table 3: Average Length of Service Group Average length of service Successful (n = 295) 24 months Self-employed (n = 95) 24 months Job (n = 73) 23 months Progress (n = 127) 25 months Not successful/Did not complete (n = 199) 18 months Began plan, received some services (n = 110) 23 months Did not complete plan (n = 89) 12 months TOTAL (n = 494) 22 months The average length of participation in CEEP was 22 months, or just under two years. As might be expected, the successful cases were open for a longer average period of time (24 months) than the cases that were not successful or did not complete services (18 months). However, this difference was mainly attributable to the 89 people who did not complete a plan and whose average time in the program was 12 months. There were no notable differences in length of service among the three successful groups (self-employed, job, progress). Underserved groups One of the primary goals of the CEEP project was to reach groups that are traditionally underserved by vocational rehabilitation services. At the inception of the project, underserved populations were defined as ethnic minority groups (particularly Latino/Hispanic and Asian American people). This is consistent with the underserved populations defined in the Rehabilitation Act (Rehabilitation Services Administration, 1998). People who had not been rehabilitated with the Department of Rehabilitation were also targeted by CEEP as an underserved population. During the course of the project, it became clear that CEEP was also reaching a large number of people with mental disabilities. Although the Rehabilitation Act specifically mentions only ethnic and racial groups as underserved, many researchers have identified people with mental disabilities as underserved (Cook & Picket, 1995; Milazzo-Sayre, Henderson, & Manderscheid, 1997; Snyder, O'Neil, Temple, & Comwell, 1996; Yelin & Cisternas, 1997). The CEEP definition of mental disabilities included people with mental illness, substance abuse disorders, traumatic brain injury and mental retardation. This broadly defined group of people with mental disorders was added as another target population for the project, with the goal of identifying and reducing barriers to employment for people with mental disabilities. We discuss results for each of these underserved groups, below. Ethnic minorities. The following table shows the ethnic distribution of CEEP consumers, including the number and percentage in each group. Additional columns show the average purchased service cost per ethnic group, and the percentage of each group that was successful (total), reported self-employment (S/E), reported finding a job, or made substantial progress toward employment goals. Table 4: Average purchased services costs and success rates, by ethnic group Ethnic group n % Average cost Percent Success Percent S/E Percent Job Percent Progress White 240 49% $3,994 70% 26% 15% 28% African-American 145 29% $3,375 47% 12% 9% 26% Asian-American 35 7% $4,061 51% 1% 23% 20% Latino/Hispanic 23 5% $3,695 57% 17% 26% 13% Native American 7 1% $3,106 71% 14% 43% 14% Multi-racial 29 6% $2,822 48% 14% 14% 21% Other 15 3% $4,111 67% 20% 20% 27% TOTAL 494 100% $3,736 60% 19% 15% 26% Technical note: The evaluators recoded several individuals with more than one designated ethnicity into the multiracial category, and recoded one ethnic category with small numbers ("Middle Eastern") into the "other" category. The CEEP project was highly successful in attracting minority consumers to the project; more than half of consumers were non-white. To better understand the extent to which CEEP served an ethnic minority population, we compared CEEP consumers with the overall population of people with severe disabilities in the United States. We used findings from the Survey of Income and Program Participation (SIPP), a major national survey with a broad-based definition of disability (McNeil, 1997). The SIPP figures from 1994-95 estimate the population of people with a severe disability. We also compared CEEP consumers to consumers served by centers for independent living (CILs) nationwide (Jans & Stoddard, in press). The CIL population included consumers who received direct services from federally funded centers for independent living in 1997. Table 5 below compares the CEEP, SIPP and CIL populations in terms of ethnic distribution. Table 5: Ethnic distributions of CEEP participants, estimates of Americans with a severe disability (SIPP) and people served by federally-funded centers for independent living (CILs) Ethnic group CEEP SIPP CIL White 53% 73% 64% African-American 32% 16% 18% Hispanic 5% 8% 13% Asian or Pacific Islander 8% 2% 2% Native American 2% 1% 2% TOTAL 100% 100% 99% Technical note: For these analyses we excluded people in the categories of "multi-ethnic" and "other," which are not used on the SIPP. As Table 5 indicates, the percentage of people served by CEEP who were African-Americans was particularly high (32%), compared to 16% of the population of people with a severe disability and 18% of consumers served by centers for independent living nationwide. CEEP also served a relatively high percentage of Asian-Americans (8%), compared to only 2% of the SIPP and the CIL populations. The percentage of Native Americans (2%) served was comparable to the SIPP (1%) and CIL (2%) populations. However, CEEP served a smaller percentage of Hispanic consumers (5%), compared to the SIPP (8%) or CIL (13%) populations. Overall, CEEP served a higher percentage of non-white consumers, compared to either SIPP or CIL populations. Looking at Table 4, three groups (white, Native American and other) had success rates that were higher than the overall average for CEEP consumers. Other groups (African-Americans, Latinos, Asian-Americans, and multi-racial) had lower success rates than the group as a whole. The fact that the project manager was Native American could be one factor in the high success rate of Native Americans. CEEP staff noted that language barriers often prevented outreach and communication with non-English-speaking ethnic minorities. One recommendation to future projects would be to prioritize the hiring of ethnic minority and bilingual staff. It is noteworthy that Asian-Americans had an extremely low rate of successful self-employment (1%), compared to 19% for the group as a whole and 26% for whites, who had the highest rate of successful self-employment. In trying to understand this finding, project staff observed that many Asian-Americans involved their families in decisions about employment. Staff speculated that some individuals may have been reluctant to undertake self-employment goals that they perceived as going outside of their traditional familial networks. In working with people who have strong ties to family networks, future projects might focus on ways to encourage self-employment that takes advantage of these extended family and community connections. In terms of reporting success in obtaining a job, Native Americans had the highest success rate (43%), and Latinos (26%) and Asian-Americans (23%) were also above the average rate for the entire group (15%). African-Americans had the lowest rate of reporting success in obtaining a job (9%). People with mental disabilities. People with mental disabilities comprised another target group of the CEEP project. As CEEP defined it, this category included people with many different mental disorders, including psychotic and neurotic disorders, substance abuse disorders, learning disabilities, traumatic brain injury, and developmental disabilities such as mental retardation and autism. This classification system is based on the Diagnostic and statistical manual of mental disorders (DSM-IV) (American Psychiatric Association, 1994), but it encompasses a more diverse group of people than are sometimes included in definitions of mental disability, including the definition used by the Rehabilitation Services Administration (RSA). CEEP tended to classify consumers as having a primary mental disability if the staff believed that mental illness created the major barrier to employment. Table 6 below presents the outcomes for people whose primary diagnosis was a mental disability, compared to people whose primary disability was not a mental disability (although they may have had a secondary mental disability). Table 6: Average purchased services costs and success rates, by whether or not mental disability is primary disability Primary disability n % Average cost Percent success Percent S/E Percent job Percent progress Mental disability 156 32% $3,106 58% 14% 15% 28% Mental not primary 338 68% $4,041 61% 22% 15% 25% TOTAL 494 100% $3,736 60% 19% 15% 26% The success rate for people with a primary mental disability was comparable to the rest of the CEEP population, despite the fact that their purchased services costs were lower. People with a primary mental disability found jobs at the same rate as other CEEP participants; their rate of self-employment, however, was lower. Given the particular vulnerabilities and barriers to employment faced by people with mental disabilities, the program seems to have been highly effective in working with this population. Project staff noted that people with mental disabilities sometimes thought that they would not qualify for the project because they did not have physical disabilities. Staff explained that people with mental disabilities also face barriers to employment and that staff had to provide a great deal of support to overcome those barriers. Staff who were seen by their peers as particularly successful in working with people with mental disabilities were those who were exceptionally patient and warm. Project staff and directors noted that people with mental disabilities were often more fragile, easily discouraged, and full of self-doubt but often responded positively when counselors reiterated their faith that the consumers "could do it." People with mental disabilities, as a group, were seen as requiring a great deal of "pre-employment counseling" but staff also stressed that they could achieve positive outcomes with support and encouragement. Previous Vocational Rehabilitation status. We also examined success rates for consumers, according to their previous status with vocational rehabilitation (VR) agencies. People who had previously not been rehabilitated with VR had been targeted as a population of interest for the CEEP. The results of the data analysis are shown in Table 7 below. Table 7: Average purchased services cost and CEEP success rate, by previous Vocational Rehabilitation (VR) status Previous VR status # % Average cost Percent success Percent S/E Percent job Percent progress No previous closures 225 46% $3,821 60% 20% 16% 24% Not rehabilitated 181 37% $3,602 55% 17% 14% 24% Rehabilitated 40 8% $3,245 63% 18% 10% 35% Ineligible 31 6% $3,900 81% 19% 19% 42% Missing 17 3% $2,961 65% 24% 6% 35% TOTAL 494 100% $3,736 60% 19% 15% 26% The target population, people who were not rehabilitated in their previous closure with VR, had a slightly lower success rate in CEEP than people with another previous VR status. Nevertheless, 55% of people who were not rehabilitated in VR reported success or substantial progress toward goals at CEEP. It is important to remember that the Vocational Rehabilitation definition of rehabilitation is different from the CEEP definition of success, making it impossible to meaningfully compare outcomes. Differences in definitions and approach were reflected in interviews with CEEP consumers who were not rehabilitated in VR but did consider themselves successful with CEEP. They saw CEEP as more willing to work with self-employment goals, more flexible in the definition of success, and more willing to provide what the consumer wanted. Level of education We also looked at the outcomes for people with different levels of education. These findings are presented in Table 8. Table 8: Average purchased services costs and success rates, by education level Level of education # % Average cost Percent success Percent S/E Percent job Percent progress No formal education 2 >1% $3,613 100% N/A 100% N/A Grade school 10 2% $3,449 60% 20% N/A 40% Some high school 32 7% $3,005 38% 9% 9% 19% H.S. degree 73 15% $2,889 45% 15% 8% 22% GED 10 2% $6,043 70% 20% 20% 30% Some college 163 33% $3,625 59% 17% 15% 26% College degree 134 27% $3,896 69% 22% 17% 30% Masters degree 56 11% $4,472 61% 25% 18% 18% Ph.D. 12 2% $3,933 92% 50% 17% 25% TOTAL 494 100% $3,736 60% 19% 15% 26% As a group, CEEP consumers had relatively high levels of education - 73% had at least some college. It is interesting that people with the least and the most education had the highest success rates; those with no formal education (n = 2) achieved 100% success and those with a Ph.D. (n = 12) had a 92% success rate. With a few exceptions, however, participants with more education (at least a college degree) had higher success rates and higher purchased services costs than those with less education. Consumers with no formal education (100% success rate) and those with a GED (70% success rate) were the exceptions to that general trend. Participants with higher education levels also had more success in self-employment and in finding a job. Major Evaluation Findings: Program Strengths and Weaknesses Orientation One of the unique features of the CEEP was its group orientation process. When an applicant contacted the program, he or she was invited to a group orientation meeting. Group orientations were conducted approximately every other month during the first three years of the project, and about once per month during the final years of the project. Up to 35 participants were permitted to attend these meetings, which lasted two hours. Orientation attendance was required of all those wishing to become project participants. The intake evaluation was done with all prospective participants to determine their eligibility for the project under the RSA rules: (1) Was the applicant disabled? (2) Was the applicant currently receiving VR services? and (3) Was the applicant from within the eligible service area? Prospective participants were also provided with information on the Client Assistance Program (CAP) and the project's internal appeals procedure. The project accepted recent medical records or evidence of eligibility for other disability benefit programs as proof of eligibility. Additional orientation topics included: * Philosophy of consumer control: 1. Informed consumers will find what best suits them; 2. Give price boundaries; 3. Empowerment and advocacy are priorities; and, 4. Allow room for choices to go wrong. Choices can be evaluated by their real-world consequences; • Overview of eligibility requirements; • How the project works; • Available services; • What is expected of participants; and, • Examples of how consumers have used the program. Before the end of the orientation, project staff distributed application forms and made appointments with all those who were eligible and interested in entering the project. At these individual meetings, staff members completed intakes and began to work with these participants on the development of their Individualized Written Rehabilitation Programs (IWRPs). Project staff reported that 80% of group orientation attendees opted to continue in the project. According to the InfoUse evaluation report (Stoddard, Hanson, and Temkin, 1999a), CEEP staff regarded the consistency afforded by the group orientation process to be essential to its effectiveness. They stressed the importance of imparting the same information in the same format at the same time to ensure that participants receive consistent information. Initially, CEEP advertised the group orientation sessions on a first-come, first-served basis. However, early in the third year of the demonstration project, the project was so popular that large numbers of individuals congregated outside BCIL's doors, and some people had to be turned away. The waiting list was closed at the end of the project's third year. Occasionally CEEP has had to modify its group orientation approach to accommodate the needs of participants who have environmental illness, severe emotional disabilities, or other disabilities that limit their ability to take part in a group setting. In such cases, CEEP has conducted one-on-one orientations. Project staff also emphasized the need to use multiple formats (written, oral, and graphic) to accommodate participants with learning disabilities or dyslexia. Project staff believe group orientation facilitated the establishment of the participant/staff relationship, since participants were linked to staff counselors just after the orientation session. Empowerment training A second unique feature of the CEEP model was empowerment training. CEEP organized and presented a series of empowerment seminars that provided participants with information on a range of vocational options, job-search skills, and other job-related issues. One goal of these seminars was to empower participants to advocate for quality in the goods and services they choose. These seminars provided participants with information and tools that helped them to make meaningful choices. The most popular seminars were on the subject of small business self-employment. One of these was "Women in Small Business," established and led by a former participant; another was presented by the Service Corps of Retired Executives (SCORE). An employment panel that included representatives of three of the Bay Area's largest employers-Bank of America, Nordstrom, and Silicon Graphics, Inc.-was particularly well attended, with over 40 participants. Other topics included training on federal and state laws relating to employment, employee rights and responsibilities, the Americans with Disabilities Act (ADA), "How to dress for success," "I hate to budget," "Finding your niche in the workplace," "Personal finance for small business and self," "Stress management," and "Self-advocacy." In the beginning of the project, staff and the project advisory committee chose topics, but increasingly, over the final years of the project, project participants selected topics. Presenters included private-sector image consultants, career counselors, non-profit community organizations, operators of small businesses, and others. Participation in the seminars was voluntary and at no cost to project participants. Payment to presenters was made as an indirect cost to CEEP rather than from participants' allocations for goods and services. As the program developed, CEEP made increasing use of paid rather than volunteer presenters to ensure quality. Staff interviewed potential presenters to make sure they were sensitive to disability-related issues, had a firm knowledge of the content area, and had conducted similar workshops in the course of their professional careers. As an additional quality assurance measure, participants were asked to evaluate presenters at the end of each seminar. This was meant to serve as an assessment of the usefulness of the seminar topics as well as the quality of the presenters. Open provider system and vendor relationships Another unique feature of CEEP was the open provider system designed to help consumers choose and access vendors of goods and services identified in the vocational plans. It is interesting that the project originally started with the idea of qualifying vendors and allowing consumers to purchase services from those vendors using vouchers. During the first year, meetings were held to familiarize potential vendors with CEEP and to cultivate positive relationships between the project and vendors. Over the course of the first few years, the project completely shifted to a non-vendorized approach to purchased services, where the consumer has complete choice and responsibility for selection of vendors. CEEP participants generally researched the costs of goods and services, and the attractiveness of vendors and service providers, before approval of their IWRPs. Cost estimates and preferred vendors were identified in those plans. Based on each plan's budget, each participant received an allocation of funds (average amount: $3,800) that constituted the individual's budget for payment of identified goods and services. Participants were not limited to a pre-selected or approved list of vendors, but rather used an open provider payment system. CEEP staff encouraged participants to search on the open market to find the best deal on products, services, added value, and other factors. Information on some providers' services and costs was available in project files, but staff preferred that each participant did his or her own research. Staff avoided making recommendations to participants. Typically, participants used the Yellow Pages, family members, and other sources to identify potential vendors, whom they then contacted and interviewed for additional information. However, project staff often required participants to document the vendors' legitimacy, particularly when vendors were very small in scale. In such cases, participants submitted copies of the vendor's business license and, if applicable, proof of professional licensure. Consumers could choose to make purchases using vouchers or letters of authorization. The merchant or vendor then billed the project for the purchase, and was reimbursed directly. Alternatively, participants could provide information to the project in advance of the purchase by obtaining a quoted price. In this case, the consumer requested a check, and gave the check directly to the vendor to make the purchase. Check requests that were consistent with the IWRP were forwarded to the project accountant, who cut checks on a regular weekly schedule. With an instituted agency-wide financial management system, turnaround time for check disbursement ideally took about two weeks. However, participants and staff, including accounting staff, concurred that the process often took much longer. Project staff reported that vendors were generally receptive to the use of the open vendor system, although some initially expressed a preference for a vendorized system in which they would be on a preferred provider list. One factor that made the open provider system effective, according to project staff, was that there was no "vendorization" system in place that had to be dismantled. Because CEEP operated in a non-governmental agency, it was not bound by state procurement rules for purchases of goods and services, as are state rehabilitation agencies in general. Since the agency within which CEEP operated has no recent history of using a vendorization system, the project did not have to contend with possible vendor resistance to the dismantling of an entrenched system. Given that there was no vendorization system, some vendors who were interviewed for the evaluation were not even aware that they had provided services as part of the CEEP. Staff people viewed that as a success of the project: service providers saw CEEP participants as no different from any other customers. Although we identify the open system as an important strength of CEEP, there was also evidence that the project did cultivate some specialized relationships with service providers, and that this was, in fact, beneficial to CEEP participants. For example, one outside vendor assisted about twenty participants in writing business plans. Her approach was tailored to the individual participant. She noted that some people needed to be guided, step by step, while others moved rapidly through the process of developing a plan. Project staff and consumers saw those services as useful and cost-effective for participants. In fact, knowing that others in the project had positive (or negative) experiences with a provider could empower consumers with the knowledge to make their own choices. Even within a non-vendorized system, it might be advantageous to develop a "recommended resources" list. To maximize consumer control and choice, the resource list could include ratings by previous participants. It would also be important to make it clear that consumers were free to pursue other resources as well. Innovative approaches to self-employment Self-employment was a unique focus of BCIL's Choice project from an early stage of the project (MSPD Evaluation Support Center, RTI, 1995). Project staff, as well as many consumers, clearly viewed self-employment as one of the important contributions and successes of the CEEP choice model. We reviewed the case files of 94 consumers who were successful in achieving self-employment goals and conducted in-depth interviews with 11 of these consumers. A listing of the types of successful self-employment demonstrates the wide variety of businesses that consumers created. The list also conveys something of the "flavor" of the CEEP choice model and the creative activities that it encouraged and supported. Types of successful self-employment included: Business, office and career support services: * writing and editing services (5) * clerical/word processing services (4) * transcription services (3) * free-lance web page designer (3) * business and career development coach (2) * free-lance computer programmer (2) * resume consultant * disability consultant to companies * computer systems engineer * business product supplier * mediation services * small business administrator * multi-media consultant * marketing consultant * financial broker Arts: * musician/singer (6) * photographer (5) * other artist (4) * graphic designer (2) * video artist * self-employed composer (creating music for video games) * guitar teacher * free-lance movie critic * furniture maker * sewing business * clothes designer Alternative health, counseling and related services: * massage therapist (5) * non-profit peer support organizations (2) _ support for people who have experienced violence or trauma _ support for people with traumatic brain injury (TBI) * marriage, family and child counselor (MFCC) (2) * health-related workshop leader (menopause, pain management) (2) * psychologist * music therapist * attendant referral service * motivation speaker on health * fitness consultant * stress management consultant Other: * college instructor; adult educator (3) * tutor/teacher (for children) (2) * small retail store owner (2) * dog-grooming and training (2) * marketing products to pet owners * priest * minister * tool inventor * recorder for the blind * licensed contractor * landscape designer * vintage toy business * child-care provider Business and office support services were among the most popular choices of the successfully self-employed. These small businesses offered writing, editing, word processing, transcription, and a wide range of computer-related services to companies, other organizations, and individuals. Some of these businesses built directly on the disability experiences of the participant. For example, one consumer offers consulting to companies on their disability policies and employee accommodations. In other cases, specialized equipment that was provided by CEEP enabled people to perform transcription and word processing services that were not possible without the assistive technology. A unique feature of CEEP was a willingness to work with consumers who had self-employment goals in the creative arts and alternative health professions. In the health arena, businesses ranged from more traditional careers such as psychologist or marriage and family counselor, to more "alternative" healing professions such as massage therapist, music therapist, stress consultant, and health-related motivational speaker. In the arts, participants set up a wide variety of small businesses, from furniture making, sewing, and clothes design to visual arts such as photography and graphic design. Musical pursuits were highly popular, with a number of participants playing music and singing at events such as weddings, and making compact disks to promote their businesses. A miscellaneous category of other self-employment included a wide variety of activities. People were successful as part-time college instructors as well as teachers, tutors, and child-care workers with youth and children. Small businesses that involved pets were popular, including dog grooming, dog training and a pet products business. A priest and a minister were among those who attained self-employment goals. This category also included a landscape designer, licensed contractor, and a number of retail businesses. In the interviews, some participants made it clear that they had limited their self-employment goals to an income level that would not interfere with continuing to receive disability benefits, particularly Social Security Disability Insurance (SSDI). They saw the CEEP project as supportive of goals that did not require them to stop receiving disability benefits. One of the project directors that we interviewed believed that programs modeled after CEEP should make the disability benefits issue even more explicit. He thought that consumers should receive benefits counseling before vocational goals are developed, so that consumers can make a rational decision to work at a level of income that will actually benefit them, overall. Most public benefits contain disincentives for people with disabilities to work. People may be able to keep their benefits if they make small amounts of money. They may be able to pay for services if they make very large salaries with excellent benefits. However, many people with disabilities would lose personal assistance or home health services, medical coverage, transportation, housing, and income if they become employed. Until work disincentives are completely eliminated for people with disabilities, he asserted, it is important to "maximize employability within the constraints of the person's real life situation." Benefits counseling could help the consumer to make a realistic appraisal of the risks and benefits of working at different income levels, with a net result of increased economic advantage through participation in the project. Interviews with project staff and consumers also made it clear that most people who were successful in self-employment had high levels of motivation. Project staff mentioned that they had gotten a better sense of who was suitable for self-employment by the later years of the project-it was not necessarily people who were more educated, but people who had an entrepreneurial spirit. In some cases participants had very clear goals before they came to CEEP. One participant put it bluntly; "I knew what I wanted and I did it on my own." This person saw CEEP as a provider of financial resources and not much else. Other consumers, however, genuinely appreciated the process of working on a business plan with a CEEP counselor who helped them clarify their business goals to create a workable plan. In earlier evaluations of the project, both consumers and staff mentioned that staff did not have enough training or expertise in how to set up businesses (Jones & Jans, 1996a, 1996b). This opinion did not come up frequently in the interviews we conducted for this final evaluation, suggesting that project staff had gained more expertise over the course of the project. A number of staff members as well as project directors mentioned that it was important for organizations setting up a project modeled after CEEP to include people with strong business experience in the project from the very start. Finally, a detailed description of the process of one consumer who achieved self-employment with CEEP illustrates some of the strengths of the project in serving people with self-employment goals. The vignette demonstrates how the CEEP approach was particularly appropriate with people who were clear on what they wanted to do but needed flexibility and specific resources to accomplish their goals. Case Vignette on Self-employment When he applied to the Choice Empowerment and Enhancement Project at the Berkeley Center for Independent Living, Paul (not his real name) was a divorced man in his late fifties. He had a high school diploma and shared an apartment in San Francisco with a roommate. His work income was irregular and was less than $15,000 in the previous year. At the time he applied to the CEEP project, he was in immediate danger of having his telephone service terminated and gas/electricity turned off. Paul's condition-muscular dystrophy-was first noticed when he was in junior high school, and diagnosed in 1971. By 1989, he couldn't walk. His employment history included work in the accounting department of a radio station, marketing and advertising, ownership of several ad agencies, and consulting work. The amount of consulting he was doing declined to about one job per month. Before coming to CEEP, Paul had developed a business plan-to provide professional business services such as word processing, high level editing, advertising, publicity, and promotions-that suited his interests and abilities. He lacked a computer and related equipment that would allow him to do this work from his home. He contacted many government officials and elected representatives, trying to find a program that would assist him to enhance his employment. He had no success until referred to CEEP by the Americans with Disabilities Act (ADA) Coordinator in the Office of the Mayor of San Francisco. After Paul contacted CEEP, project staff conducted a face-to-face orientation at his bedside. Paul quickly completed his intake and Rehabilitation Plan. Because he already had a clear vocational goal, the basics of a business plan, and knowledge of what goods and services he would need, the time required to get his plan approved was minimal. Paul took advantage of training opportunities to work on time management and to speak to others to promote his business. But he felt that he did not need to attend empowerment workshops and that they were not worth the inconvenience of a lengthy trip across the Bay. In the six weeks after obtaining his computer system, his business generated income of $5,000 and he had more work lined up. The marketing section of his business plan predicted this sort of income potential, and Paul was confident he could average an income of between $400 and $800 each week. He attributes much of his confidence and success to participating in CEEP. Counseling and consumer choice This report has already described many of the qualities of the counseling relationship that contributed to success. These include flexibility in meeting the individual needs of consumers and a commitment to consumer control of the process of developing, clarifying, and achieving vocational goals. As a business consultant who assisted consumers in developing business plans described it, CEEP "really helped to level the playing field for this group of individuals." A former program director commented, "people weren't pigeonholed. They were allowed to take responsibility and ownership for their choices." And as consumers put it, "They listened and cared...(and) paid attention to me as an individual." "They firmly believed in me." "They allowed me to decide what I wanted to do and how to get there." Payment systems In the first year of CEEP, a few consumers took unfair advantage of the flexibility of the project. As one project staff member described it, "there were essentially no limits on spending and the project was ripped off by a few bad apples." Partly in response to that problem, a $3,800 guideline was developed as the average cost of a plan. It was not an absolute limit; in particular, consumers could obtain additional funding for accessibility accommodations. Apart from that exception, spending over the $3,800 guideline was rarely allowed. A complex system of checks and balances also evolved to ensure accountability in the payment of vendors. The system had several steps. First, an Individual Written Rehabilitation Plan (IWRP) was prepared that defined the goods and services required to achieve the consumer's employment objectives. The consumer then researched the availability, price, and quality of the specific goods and services identified in the IWRP and discussed the results with the CEEP counselor and program manager. Both the counselor and the consumer had to sign the expenditure plan, which then had to be approved by the project manager. Next, the expenditure plan and the IWRP were submitted to the project director (who was also the executive director of BCIL). To ensure that a particular expenditure did not exceed the consumer's spending plan, an up-to-date financial report for the consumer that reported total dollars spent and total dollars remaining had to accompany the other documents. All of these documents were reviewed at a weekly meeting between the project director and project manager. If all the paperwork was in order, the expenditure plan was approved and submitted to the accounting department to cut a check. If the consumer had already purchased the item, accounting required an original receipt before writing the check. Regardless of the amount, BCIL policy required that all checks have two signatures from senior BCIL staff. Normally, payment reached the vendor two weeks after the accounting department received an invoice with all appropriate signatures. However, there were often lengthy delays in the process of getting the appropriately approved documentation to the accounting department. These resulted from simple breakdowns such as a missing signature from a consumer, the lack of an up-to-date report on what the consumer had already spent, the cancellation of the weekly meeting due to illness and other factors, and the non-availability of senior BCIL staff to sign checks. Although CEEP staff and some consumers expressed great frustration with this system, only a handful of consumers actually experienced serious consequences such as being unable to enroll in a training or educational program. Some vendors who were more familiar with consumers paying cash at the time of purchase (as opposed to the accounts payable method used in this project) also reported frustration with payment delays. Criteria for success Defining the criteria for success proved to be a challenge for the CEEP project. This can be especially difficult for a project that decides not to use traditional vocational rehabilitation criteria and instead seeks to expand the notion of what constitutes success. For example, CEEP seemed to implicitly recognize as a valid choice the idea that someone would set out to make a relatively small amount of money that would not jeopardize his or her disability benefits. For a person on public benefits who receives many services that are necessary for independent living, a goal of maximizing employment without losing needed services makes practical, economic sense. But the project did not make this idea explicit in the plans, making it hard to know whether the goals had been achieved. In general, the project had a difficult time defining objective, measurable criteria for success. The solution was simply for the consumer and counselor to sit down together at the end of services and decide whether the consumer felt empowered and had either achieved or made substantial progress toward goals. Developing more objective and measurable criteria for success that are based on early assessment of an individual's economic situation would be an important early step for any organization that undertakes a project modeled after CEEP. Project isolation and tensions in the case management role The consensus of project directors and other CIL staff was that CEEP was relatively isolated from the rest of the CIL. The staff was seen as more "professional" and "formal" than the rest of the CIL. Part of this might be due to the fact that the grant required a separate intake and record keeping procedure, more formal and detailed than the procedures of the rest of the organization. We also believe that some of this isolation may be attributed to tensions inherent in the case management role. Frequently, when a grass-roots advocacy organization takes on a case management role, tensions emerge between the desire to provide consumers with resources and the need for fiscal accountability. The CEEP project was responsible, on the one hand, for the equitable distribution of limited funds to consumers (the "gatekeeper" function). Given the demand for CEEP services, this necessitated rationing of funds. On the other hand, the project, as part of a CIL, also had a strong commitment to advocacy on behalf of the consumer. The role of an advocate or services provider is to assist the consumer in obtaining as many resources as possible. Conflicts between these two roles showed up throughout the project. During the first year, for example, participants were allowed to purchase expensive goods and services (more than $10,000 per person, in some cases). This is consistent with the services provider role, providing funds and services that people want. When a few participants then disappeared or failed to follow through with their plans, the project essentially rationed services to an average of $3,800 per participant. Once this "gatekeeping" guideline was in place, project staff were quite comfortable with the limit and, in fact, saw it as helpful to participants. Project staff also noted that consumers sometimes expected them to "act like a battering ram" in advocating for participants' disability rights. These consumers thought that BCIL was supposed to be the "heavy hand" of enforcement of consumers' rights under the Americans with Disabilities Act. In contrast, project staff saw themselves more as teachers who could assist consumers in advocating for themselves. In this case, consumers were pushing for CEEP staff to take more of an advocacy role, while CEEP staff viewed themselves more as case managers. In this situation, they might have been more likely than other departments within the CIL to insist that the consumers take responsibility and do the advocacy for themselves. Conflicts also erupted between the project staff who tended to favor a system with maximal flexibility for consumers (the service provider/advocate role), and the accounting staff and executive directors, who tended to press for more rigorous accountability and control over funds (the gatekeeper role). The isolation of CEEP from the rest of BCIL was also a function of general overload and high demand on the resources of all BCIL staff, including CEEP staff. As is typical in grass-roots advocacy organizations, each department was bombarded with demands on its time and other resources. All the CEEP and BCIL staff we interviewed acknowledged how hard it was to work together across departments when there was such high demand on all of the departments. CEEP's status as a project separate from the rest of BCIL's activities took its toll in terms of strained relationships within the organization. Project isolation also indirectly impacted consumers. For example, one project director thought that CEEP participants could have made better use of the other services of BCIL, such as IL skills training and advocacy, if CEEP had been better integrated within the organization. Dissemination and replication of project results Project staff were very disappointed that features of the CEEP model were not adopted by other centers for independent living and the state VR agencies. According to InfoUse's evaluation report (Stoddard, Hanson, and Temkin 1999a), the California Department of Rehabilitation (DR) initiated group orientation sessions in its larger offices in early 1997 to ensure consistency of information. DR started this procedure as one of its own Choice strategies after consulting with independent living centers and other constituent groups. One district office that serves the same geographical area as the CEEP reports that it discontinued group orientations after finding that participants disliked being placed on a waiting list and attendance at orientation sessions dropped off. However, group orientation is still used at some other DR offices. Materials that were developed in the project are included in the Appendix of this report. BCIL also developed a video, "Making Choices, Empowering Lives," which is included along with this report. This lively and energetic video captures the spirit and philosophy of CEEP, presenting in consumers' own words what the project has meant to them. Major Evaluation Findings: Comparing CEEP with other Choice Demonstration Projects CEEP was one of seven RSA-sponsored projects funded to demonstrate different approaches to implementing informed choice. The other six Choice grantees were Arkansas Rehabilitation Services (AR); Vermont Division of Vocational Rehabilitation (VT); Washington Division of Vocational Rehabilitation (WA); Southwest Business, Industry and Rehabilitation Association (SWBIRA); The Development Team, Incorporated (TDTI); and United Cerebral Palsy Associations, Inc. (UCPA). InfoUse conducted the federally-funded evaluation of the seven projects. In this section, we present and discuss data from InfoUse's final report (Stoddard, Hanson, and Temkin, 1999a) in order to illustrate CEEP's model in the context of the other projects. We show data on participants' reported choice and satisfaction, patterns of purchased services, and service costs. Overall Choice and Satisfaction To assess participants' perceptions of the degree of choice and satisfaction experienced in each project, InfoUse conducted a telephone survey of a random sample of Choice project consumers. Table 9 indicates that 71.8% of all demonstration participants surveyed in the InfoUse study said they had had "total choice" (40.4%) or "a lot of choice" (31.4%) in their projects. In comparison, 86.8% CEEP participants reported "total choice" (40.9%) or "a lot of choice" (45.5%), indicating a level of satisfaction somewhat higher than the level across all the Choice projects. Table 9: Participants' Report of Degree of Overall Choice BCIL AR VT WA SWBIR TDTI UCPA Totals N = 261 132 305 226 302 109 109 1,444 No choice 0.0 10.0 0.0 8.3 22.2 0.0 4.5 7.2 Not much choice 4.5 16.7 0.0 8.3 5.6 11.1 18.2 7.0 Some choice 9.1 23.3 10.0 0.0 27.8 16.7 13.6 14.0 A lot of choice 45.5 33.3 20.0 50.0 11.1 66.7 9.1 31.4 Total choice 40.9 16.7 70.0 33.3 33.3 5.6 54.5 40.4 Total 100.0 100.0 100.0 99.9 100.0 100.1 99.9 100.0 Note: All response values are expressed in percentages. Percentages may not sum to 100% due to rounding. Table 10 shows data on the extent to which the Choice projects met participants' expectations. Across all of the Choice participants surveyed, 73.9% reported that the projects met or exceeded their expectations, compared to 81.8% at BCIL CEEP. Table 10 also indicates that about four of five (81.4%) of all project participants reported that they would recommend the project to a friend, and 51.5% reported doing so. For CEEP participants, 100% would recommend the project to a friend, with 72.2% reporting that they actually did so. Table 10: Participant Overall Satisfaction BCIL AR VT WA SWBIR TDTI UCPA Total N = 261 132 305 226 302 109 109 1,444 Project exceeded expectations 40.9 16.7 36.4 66.7 22.2 27.8 21.7 35.4 Project lived up to expectations 40.9 40.0 54.5 16.7 38.9 33.3 34.8 38.5 Project fell below expectations 18.2 43.3 9.1 16.7 38.9 39.8 43.5 26.2 Question not answered 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 Total 100 100 100 100 100 100 100 100.1 Participant would recommend project to a friend 100 90.0 90.9 83.3 56.3 77.8 69.6 81.4 Participant has recommended project to a friend 72.2 56.7 54.5 58.3 27.8 55.6 34.8 51.5 Note: All response values are expressed in percentages. Percentages may not sum to 100% due to rounding. Patterns of Purchased Goods and Services Tables 11 and 12 are derived from the purchased services dataset developed in InfoUse's case review process. For each separate service purchase during a specific 18 month study period, InfoUse identified the item purchased, the type of source of purchase, and the dollar cost of the purchase. InfoUse used data from actual financial transaction records derived from case file data (Stoddard, Hanson & Temkin, 1999a). Each of the seven Choice projects implemented unique models for helping participants choose goods and services. Thus, the services available to a Choice participant, and consequently the participant's selection of services in each project, were largely determined by the particular service model used. In some projects, certain services were mandatory or integral to the service model used. In the CEEP model, participants were not required to purchase specific types of services. Instead, they were encouraged to shop in the open market where purchases are concentrated in retail stores and computer stores, as well as educational programs. Table 11 shows how purchase service dollars were used by participants enrolled in the Choice projects during our sample period. Percentages over 5% are highlighted in bold for ease of reading. For the demonstration program as a whole, 14% of purchase service dollars was spent on business or vocational training, another 8.1% on college or university training, almost 11% on assessment, 9.4% on job placement services, 9.3% on computers and peripherals, and 8.5% on job-finding services. In CEEP, patterns of participant purchase of services varied from this overall pattern. CEEP's participants emphasized business/vocation (28%) and college/university training (18.9%) and obtaining computers for work and self-employment (25.3%). Table 12 shows where participants made purchases as a percentage of total purchases. Each column identifies the types of providers where program-purchased service dollars were spent. Overall, Table 12 shows about 30% of purchases going to employment service organizations, and another 11% going to Community Rehabilitation Facilities. Educational institution expenditures accounted for nearly 16%. These are more traditional purchasing sources for vocational rehabilitation. The second highest source is commercial merchandise retailers (17%). In contrast, the majority of CEEP participant purchases were from either educational institutions (44.7%) or from retailers (21.5%). Table 11: What the Purchased Service Dollars Buy/Percent of Service Funds by Program by Item SERVICES BCIL AR VT WA SWBIR TDTI UCPA Totals weights 7.91 3.88 10.17 7.53 9.74 3.52 3.41 N = 261 132 305 226 302 109 109 1,444 n = 33 34 30 30 31 31 32 221 Business/Vocational Training 28.0 30.3 9.0 4.0 * * 0.5 14.0 Assessment 0.2 7.5 1.6 2.8 38.6 0.1 13.5 10.8 Job Placement * * 2.6 * 19.3 * 50.2 9.4 Computers and Peripherals 25.3 6.0 2.7 5.5 * 38.9 * 9.3 Job-Finding Services 0.4 * 0.3 14.8 18.7 0.5 25.2 8.5 College/University Training 18.9 13.4 8.3 3.3 * 1.8 * 8.1 Self-Employment Inventory/Set-up 13.1 3.7 11.0 7.3 * 3.7 * 5.6 Other Services (excluding any of above) 0.4 6.6 0.5 22.4 2.9 2.9 0.0 5.5 Maintenance 0.8 17.5 8.9 0.1 * * * 4.9 Assistive Technology Devices * 7.3 21.3 0.6 5.2 7.8 * 4.9 Transportation 4.9 2.8 2.0 2.9 1.3 1.0 1.5 2.7 Grooming/Clothes 2.1 0.6 2.5 6.8 2.3 4.9 * 2.4 Miscellaneous Training 4.6 0.4 6.4 2.2 0.7 0.9 * 2.0 Motor Vehicle Purchase * * 3.7 2.2 * 20.0 3.2 1.9 Restoration 0.7 0.0 4.3 3.2 0.2 4.2 1.0 1.3 Counseling and Guidance * * 3.5 0.7 * 4.0 4.1 1.1 Motor Vehicle Repair * * 8.6 0.6 * 4.0 * 1.0 On-the-job Training * 1.3 * 3.5 * * 0.3 0.8 Assistive Technology Services * 1.9 * 0.5 * 0.6 * 0.5 Adjustment Training * 0.6 2.9 0.6 * * * 0.5 Office-Related Equipment and Supplies 0.1 0.1 * 1.1 * 4.6 * 0.4 Personal Assistance Services * * * * * * * * Case Management * * * * * * * * Cannot Determine 0.4 * * 14.9 10.7 0.2 0.3 4.2 TOTAL 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 Note: Unless otherwise indicated, numbers are percentages. * = zero value 0.0 = less than .05 value Table 12: Where Purchases are Made/Percent of Total Purchases by Project by Source of Purchase SOURCES BCIL AR VT WA SWBIRA TDTI UCPA Totals weights 7.91 3.88 10.17 7.53 9.74 3.52 3.41 N = 261 132 305 226 302 109 109 1,444 n = 33 34 30 30 31 31 32 221 Employment Services Organization * 3.7 * 30.0 90.1 1.8 76.0 30.3 Commercial Merchandise Retailer 21.5 12.0 37.2 28.8 2.2 49.8 * 17.0 Educational Institution 44.7 17.8 14.9 7.4 * 7.0 * 15.7 Community Rehabilitation Facility * 36.4 13.4 * * * 14.8 11.2 Consumer 0.2 8.2 7.0 3.3 0.3 1.5 * 3.1 Individuals (Private) 0.4 2.4 2.7 1.3 * 24.1 7.1 3.0 Commercial Services Retailer 0.8 6.5 5.7 1.9 0.2 6.1 0.8 2.8 Vocational Assessment Services Organization * 5.2 * * * * 1.3 1.3 Other Medical/Allied Health Services Provider 0.6 1.6 2.7 2.1 * 2.3 * 1.1 Transportation Agency 1.8 * 0.8 2.9 0.9 * * 1.0 Other Sources 2.3 1.0 0.6 * * 0.2 * 0.7 Other Public Agency 1.1 * 0.5 0.5 0.1 0.3 * 0.3 Employer * 1.3 * * * * * 0.3 Adaptive Equipment Suppliers * 0.6 * * 0.7 * * 0.3 Other Private Agency 0.2 * * 0.5 * 0.2 * 0.1 Member of Consumer's Family * 0.5 * * * * * 0.1 In-House Purchase (internal to project) * 0.0 * 0.0 * * * 0.0 Hospital * * * * * 0.1 * 0.0 Independent Living Center * * * * * * * * Welfare Agency * * * * * * * * Cannot Determine 26.4 2.7 14.6 21.5 5.5 6.6 0.0 11.5 TOTAL 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 Note: Unless otherwise indicated, numbers are percentages. * = zero value 0.0 = less than .05 value Average Annual Costs Per Participant InfoUse's estimates of costs per participant focused on average annual purchased services costs and average annual total costs per participant, based on a sample of consumers served during one year of each demonstration project. This method underestimates total costs because some cases may still be active. (Note that these annual figures are different from the purchased services costs reported in another section of the current report, based on total purchased services per participant, across the six years of CEEP.) Nevertheless, InfoUse's cost findings allow for comparisons across projects. Table 13 indicates that average annual purchased services costs ranged from $654 to $3,306 per participant across the seven Choice projects. For all projects, average annual purchase service costs per participant was $1,700. In contrast, VR's average annual cost per consumer was $997. BCIL CEEP's average annual purchased services costs of $3,353 are higher than all other projects. As a percentage (60%) of all annual costs, CEEP is exceeded only by the Arkansas Choice project. This indicates that CEEP spent a higher percentage of their total resources on purchasing goods and services for consumers, compared to most other Choice projects. Table 13 also shows that total average annual cost per participant in the seven projects was $3,670, and ranged from $2,030 to $6,967 across the seven Choice projects. RSA's VR program spent an average of $1,910 per consumer per year. Across the projects, average per participant costs varied according to type of participant served by the projects, program practices, limitations on participant budgets, and the amount of services provided directly by project staff. Added costs frequently were incurred in providing the additional services and supports useful in expanding choice (e.g., helpers, empowerment training, extensive consumer plan-making processes). In CEEP, average annual total costs per participants were $5,552, an amount that falls in the middle of the range of all projects. To summarize, this comparison of CEEP with the other six Choice projects shows that: * CEEP participants reported "total choice (40.9%) or "a lot of choice" (45.9%), indicating a level of satisfaction somewhat higher than the levels reported across all Choice projects. * 81.8% of CEEP participants reported that the project met or exceed expectations. All of the surveyed participants (100%) reported that they would recommend the project to a friend with 72.2% reporting that they actually did so. These results are higher than reported across all Choice projects. * In CEEP, patterns of participant purchase of services varied from the overall pattern exhibited across all projects. CEEP's participants emphasized business/vocational training (28%) and college/university training (18.9%) and obtaining computers for work and self-employment (25.3%). * Almost all CEEP participant purchases were from educational institutions (44.7%) or retailers (21.5%). * CEEP's average annual purchased services costs per participants of $3,353 are higher than all other projects. As a percentage (60%) of all annual costs, CEEP is exceeded only by the Arkansas Choice project. This indicates that CEEP spent a larger percentage of resources on purchasing goods and services for consumers than most other Choice projects. * Average annual total costs per participant were $5,552, an amount that falls in the middle of the range of all projects. * Table 13: Choice Demonstration One Year Cost Estimates BCIL AR VT WA SWBIR TDTI UCPA Total Reporting From: Jul-95 Jul-95 Jan-96 Oct-95 Oct-95 Oct-95 Oct-95 Period To: Jun-96 Jun-96 Dec-96 Sep-96 Sep-96 Sep-96 Sep-96 Total Participants 104 122 215 140 152 55 18 806 Total Annual Project Replication Costs $577,417 $459,654 $436,393 $436,088 $596,147 $326,618 $125,407 $2,957,724 Average Annual Purchased Service Cost/Participant $3,353 $2,454 $654 $957 $1,941 $1,681 $3,306 $1,700 Average Annual Internal Cost/Participant $2,199 $1,314 $1,376 $2,158 $1,981 $4,258 $3,661 $1,970 Total Average Annual Cost/Participant $5,552 $3,768 $2,030 $3,115 $3,922 $5,939 $6,967 $3,670 Purchased Service Cost as a Percent-age of Total Cost 60% 65% 32% 31% 49% 28% 47% 46% Lessons learned: Recommendations for Other Organizations Considering Implementation of This Model It was unusual for an independent living center to take on the functions of an employment organization. CEEP demonstrated a number of alternative strategies for developing employment options for people with disabilities. We hope that other organizations will use the lessons learned from the CEEP model to develop new programs to enhance opportunities for people with disabilities. In summary, we present a number of recommendations that emerge from this evaluation and that might be helpful for other organizations that are considering implementation of the CEEP model of choice. 1. Remember that the core of the model is to foster consumer choice and control. As one counselor put it, "We are not handing out fish, but rather teaching people how to fish." 2. Early on, the project must develop objective and measurable criteria for success. These might be based on the individual plan, so that each consumer has a particular set of outcomes that constitute success. But the outcomes must be specified in advance, and be objective and measurable. This will ensure that the project can demonstrate to others that it accomplished what it set out to accomplish. 3. The need for integration of the project within the overall organization is an important "lesson learned" from the CEEP experience. This report discusses some of the factors that seemed to lead to isolation of the project from the rest of organization. 4. All project staff agreed that it is vital to hire bilingual-bicultural staff and set up outreach networks early in the project. 5. Group orientation was a strong feature of the project. Future projects might carry the model of peer support even further. A number of people we interviewed mentioned the idea of peer mentors or support groups, in which consumers who had been successful could share their experience, support and guidance with others. 6. Empowerment workshops were an integral part of the CEEP from its inception, and proved to be a powerful feature for many participants. On the basis of the CEEP experience, paid professional presenters rather than volunteers are recommended to ensure the quality of the empowerment training provided to participants. Staff interviewed potential presenters to make sure they were sensitive to disability-related issues, had a firm knowledge of the content area, and had conducted similar workshops in the course of their professional careers. 7. The devendorized, open system was seen as a promising practice that other organizations could readily adopt. 8. Even in a system where the consumer has the choice of going to anyone to purchase goods or services, there may be advantages to cultivating some special project relationships with people who have particular skills or goods to offer participants. Sometimes consumers wanted more guidance. The project could develop a "recommended resources" list, including reviews or ratings by other participants. It would be important that participants still have a choice about whether or not to use those services. 9. The experience of CEEP clearly indicates that there must be clear guidelines and expectations for consumers seeking self-employment. Even participants who originally complained about having to do a business plans generally found the process to be useful in clarifying and setting realistic goals for their small businesses. 10. Project staff, directors, and consumers saw the need for developing business and entrepreneurial expertise early in the project. This might include initial benefits counseling; formal training for counselors in business plans and business analysis skills; a strong network of business people; and early development of resources for long-term capital, lines of credit, and other financial resources. 11. There were relatively high costs associated with people who began a plan and received some services but did not complete their plans. This suggests that programs initiating a choice project modeled after CEEP should devote concerted attention to pre-employment issues, so that consumers who begin plans are ready to follow through with their plans. This would include IL skills training, a particular strength of centers for independent living. 12. An organization that undertakes a project modeled after CEEP must think very carefully about money issues before the project is set up. Inherent tensions between the desire to provide consumers with resources and the need for fiscal accountability will inevitably exist and can disrupt the organization unless the project is carefully designed and carried out. Early on, it is important to make these role tensions explicit, to recognize that the project must serve both gatekeeper and advocate/service provider functions, and to forge compromises between these roles. Accounting staff must be integrally involved in the planning and implementation of the project. It may also be important to separate the roles so that counselors are not making decisions about how much money to disburse to consumers while at the same time providing advocacy and job counseling services. 13. The complicated system of keeping track of payments to consumers created tensions and strains within the organization. On the basis of interviews with accounting and other staff, we recommend that future projects use "QuickBooks" or a comparable software with payroll features that the accounting staff already uses. References American Psychiatric Association (1994) Diagnostic and statistical manual of mental disorders (DSM-IV). Washington DC: American Psychiatric Association. Berkeley Planning Associates (1995, April). Evaluation report, Choice Enhancement and Empowerment Project, Year one. Prepared for the Berkeley Center for Independent Living. Cook, J.A., & Picket, S.A. (1995). Recent trends in vocational rehabilitation for persons with psychiatric disabilities. American Rehabilitation, 20(4), 2-12. Gilmartin, D., Stoddard, S., & Armstrong, C.R. (1996, April). An evaluation of Choice Demonstration Projects: Working paper on promising practices. An InfoUse report. Prepared for Rehabilitation Services Administration, Contract HR95034001. Jans, L., & Stoddard, S. (In press). Chartbook on centers for independent living in the United States. An InfoUse report. Washington DC: National Institute on Disability and Rehabilitation Research. Jones, E. & Jans, L. (1996a, September). Program evaluation report, Year two, Choice Enhancement and Empowerment Project. Prepared for the Berkeley Center for Independent Living. Jones, E. & Jans, L. (1996b, December). Program evaluation report, Year three, Choice Enhancement and Empowerment Project. Prepared for the Berkeley Center for Independent Living. McNeil, J.M. (1997). Americans with disabilities: 1994-95. U.S. Bureau of the Census, Current Population Reports, P70-61. Washington, DC: U.S. Department of Commerce. Milazzo-Sayre, L., Henderson, M., & Manderscheid, R. (1997). Serious and severe mental illness and work. In R.J. Bonnie & J. Monahan (eds). Mental disorder, work disability and the law. Chicago: The University of Chicago Press. MSPD Evaluation Support Center, Research Triangle Institute (1995, October). Evaluation of first year Choice Demonstration Projects. Prepared for the U.S. Department of Education Planning and Evaluation Service. Rehabilitation Services Administration (1998). The Rehabilitation Act of 1973, as amended. Washington DC: Rehabilitation Services Administration. Snyder, J., O'Neil, T., Temple, L., & Cormwell, R. (1996). Psychiatric disabilities: Concerns, problems, and solutions in independent living for an underserved population. Lawrence, KS: University of Kansas, Research and Training Center on Independent Living. Stoddard, S., Hanson, S., & Temkin, T. (1999a, February). An evaluation of Choice Demonstration Projects, Final report. An InfoUse Report (Contract HR95034001). Washington DC: Rehabilitation Services Administration. Stoddard, S., Hanson, S., & Temkin, T. (1999b, February). Promising practices in the Choice Demonstration Projects: An operations manual. An InfoUse Report (Contract HR95034001). Washington DC: Rehabilitation Services Administration. Yelin, E.H., & Cisternas, M.G. (1997). Employment patterns among persons with and without mental conditions. In R.J. Bonnie & J. Monahan (eds). Mental disorder, work disability and the law. Chicago: The University of Chicago Press. Appendix: Project Materials Orientation Guidelines 1. Starts at about 10:08 AM 2. Introduce yourself and staff introduce themselves, brief backgrounds of education and experience and why they like working at client choice. 3. History of Client Choice --First initiated from the 1992 amendments to 1973 Rehabilitation Act; in '92 it was re-amended to add choice! Here we are. There are seven choice projects nationwide --we are the only independent living center --Florida, Vermont, United CP Association, Arkansas Southwest Arizona, and Seattle 4. Follow thru the brochure re: Criteria of Client Choice #1 #1 Explain why! #2 Explain why! #3 Explain why! #4 Explain Why or accommodation needs and what the employer is responsible for #5 Read the "Employment Outcome" (Consumers can read the rest themselves) And what does it measure up to be "J O B" 5. Consumers introduce themselves and what services they hope to receive 6. Empowerment projects throughout the month check your mail!!! Fun stuff coming up 7. Explain the process A) Attend an Orientation B) Sign up with client choice staff for an intake interview and bring documents. C) At that time we talked on an action plan with you so you can make all your choices in your vocational avenues. Then we will write an individual written plan-which is an agreement of what client will pay for. D) Plan can be budgeted up to $3,800.00 each. E) Please inform us on any current job activity so we can document this to the federal government - showing that choice really helps people with disabilities make a difference in the rehabilitation model. F) Then you're on your way! Any questions???????????? Sign up with counselors!!! Thanks for coming and ask for more referrals Eligibility Criteria 1. Applicants must reside in either Alameda or San Francisco counties. 2. Applicants can not have an open file with Department of Rehabilitation. 3. Applicants must meet the criteria of severe disability as specified under the Department of Rehabilitation regulations. 4. Applicants must be unemployed, or working part time, twenty hours a week or less. 5. Applicants who work part time must need further training, or assistive technology in order to achieve their employment outcome. Attachment from (Rehabilitation Act-Title One 1992) The term of "employment outcome" means, with respect to an individual, entering or retaining full-time or, if appropriate part-time competitive employment in the integrated labor market (including satisfying the vocational outcome of supported employment) or satisfying any other vocational outcome including self-employment. 6. Applicants who work full time and request services to retain employment must demonstrate that they have requested reasonable accommodation from their employers, and these accommodations have been determined to be an undue hardship for the employer. 7. Applicants who work full time and request services to retain employment must provide medical evidence that their disabilities will deteriorate in the near future, to the point that they can no longer work without adaptable aids or re-training. 8. Applicants must demonstrate the desire and willingness to achieve their vocational goal with a focus on EMPOWERMENT. 9. Applicant's vocational services plans must meet the grant's guidelines of financial support plan on the average of $4,000 per person in expenditures for vocational training, and or adaptive equipment. Some plans may need more or less financial assistance to be successful i.e. achieving their vocational goals. Each case will be evaluated by the services manager and approved by the project director. Clients will be assisted in pursuing all avenues of alternative funding to achieve their vocational goals. 10. Self-Employment/Small Business Enthusiast must present a small business plan for consideration. Everyone who wishes to pursue this avenue must submit a business plan regardless of education or experience. 11. Clients with Masters degree seeking advance degrees or another masters degree do not qualify for services of Client Choice and Empowerment Project unless career goals proven to be unrealistic due to disability only. "I Hate to Budget" A workshop provided by Consumer Credit Counseling Service of the East Bay, Inc. Presenter: Tina Powis-Dow, MBA, Marketing & Education Director at Consumer Credit Counseling service, spent 5 years as an auditor in the banking industry and has over 10 years education experience. Impulsive Spending What Motivates Us To Overspend Money is a measuring stick; it represents value and enables buyers and sellers to agree on a price. But money also symbolizes status and power. This allows emotions to distort the measurement of value. Once emotions are involved, money is easily misused. Whether we admit it or not, adults play money games with each other every day. We use money as a lure, a weapon, or a reward with friends, spouses, and our children. Because of money, some people treat us well, while others act as if we didn't matter. Because of money, we can develop self-respect or decide that we are failures. For some people, self-image is so closely tied to their possessions that they buy things in hopes of feeling better. They become compulsive spenders, showering themselves with things they neither need nor want. When they feel needy or lacking, they splurge on something they cannot afford. They spend impulsively, incur debt, feel guilty, promise never to do it again, only to repeat the same cycle the next time the feeling of "not enough" comes up. An awareness of irrational motives to overspend is the first step in gaining control over a spending problem. Say "No" To Overspending Some people overspend because they don't know how to say "no". Have you ever purchased something that you didn't want because you were unable to refuse the salesperson? Have you ever bought an unnecessary item for someone you care about because you didn't know how to turn them down? Have you ever gone shopping with a friend even though you couldn't afford to buy because you didn't know how to decline? If so, you need to learn to say "no". Feeling hesitant, anxious, or nervous is an indication that you want to say no. Assert your right to ask for more information before making a decision. Give yourself time to evaluate the request. Then practice saying "no"; it will get easier the more you do it. These are two ways of refusing . One is to say "no" with an explanation (but not an excuse). It lets the other person know the reason behind your decision. The second method is to say "no" with no explanation. This is especially suitable for use with strangers or people you don't care about. Remember, you are not obligated to explain yourself to others, although you may choose to do so. Also, learn to refuse without saying "I'm sorry". Apologies may tempt others to play on your guilt feelings. Others may not always understand or accept your refusal the first time you offer it. You may have to repeat your message over and over again. This broken record technique is particularly helpful when the other person tries to pressure or manipulate you into changing your mind or deliberately evades giving you a direct response. Learning to say "no" in an appropriate, assertive way to salespeople, family and friends can help you avoid spending money in an unplanned, unwanted way. It can help prevent you from becoming an overspender. Ways To Control Impulse Buying * Formulate a budget allowing a certain amount of discretionary spending for each family member. Writing down spending limits can strengthen the commitment not to overspend. * Don't make your budget too tight-try to allow some "fun" money. We're more likely to buy impulsively when we're feeling deprived. * Record your spending habits and look for patterns. * Avoid sales of luxury items. Sales make it easier to justify unnecessary purchases. * Be aware of give-away gimmicks that induce you to buy something you don't need. * Leave credit cards or checkbooks at home. A trip back home to pick up the cards or checkbook will give you time to think about the purchase. * Don't shop for recreation. * Stop and think what you will have to give up if you make the purchase. * Sleep on the decision - the bigger the purchase, the more nights. * Make a shopping list and stick to it. * Ask someone to support you in self-control and discipline if impulse buying is a habit. * Limit the number of credit cards you own and keep the credit limit on each card low. * Develop a desire for experiences rather than things. Seek activities and diversions that do not cost money. Should You Buy It? 1. Do you really need this item? Yes No 2. Is the price reasonable? Yes No. 3. Is this the best time to buy the item? Yes No 4. If this is a bargain, is it a current model? Yes No 5. If "on sale", is the price a true sale price? Yes No. 6. Are you sure no less expensive item can be substituted? Yes No 7. Are you sure there are no major disadvantages? Yes No 8. If excessive in price, will it truly satisfy an inner need? Yes No. 9. Have you checked and researched the item? Yes No 10. Have you budgeted for this item? Yes No 11. Do you know the retailer's reputation? Yes No. 12. Does this retailer offer any special services with the item? Yes No Score your answers as follows: 9 - 11 yes -- consider buying the product. 6 - 8 yes --think again. Fewer than 6 yes -- forget it. Workshop Evaluation Name of workshop: ______________________________Date: __________ Please circle the number that best represents your impressions about the seminar and the speaker. Your input helps us get a better sense of your likes & dislikes. 1=Poor 2=Fair 3=Good 4=Very good 5=Excellent Was speaker knowledgeable? 1 2 3 4 5 Was seminar informative? 1 2 3 4 5 Was speaker receptive to your needs? 1 2 3 4 5 How helpful was Client Choice Staff? 1 2 3 4 5 Was topic essential to your career goal? 1 2 3 4 5 Would you recommend the speaker? 1 2 3 4 5 Would you attend similar workshops in the future? 1 2 3 4 5 How do you rate the speaker overall? 1 2 3 4 5 How do you rate the seminar overall? 1 2 3 4 5 Comments: ____________________________________________________________ ____________________________________________________________ ____________________________________________________________ Make Informed Choices When Hiring Contractors! Independent Contractors by Tara Kelly As you all know, the Client Choice Enhancement and Empowerment Project is dedicated to bringing you quality service with the greatest flexibility in choice possible. As this project grows older and we work out the kinks, we have become more aware of which guidelines you, as a consumer of services or a small business owner, need to follow in order to make responsible choices while maintaining the greatest amount of independence in your plan. So as you work on your plan, you will still have the freedom to choose which agency or individual you want to use when purchasing services, equipment, training etc. however, if you are choosing an individual who is not acting as an employee of a larger company etc., you will need to insure that the person you are choosing is an independent contractor. What is an independent contractor? As the name implies, an independent contractor is an individual who offers her/his services to the public for a fee. This person is operating on her/his own and not as an employee of a company or agency. Why do I need to know? Knowing whether or not the person you plan to hire is an independent contractor is significant to you whether you are starting a small business or simply hiring someone to provide you a service as an individual. There are many state and federal guidelines governing individual contractors and for tax purposes, you must be able to make this determination in order to protect yourself and follow the law. How do I Know? Your counselor has available to you a list of questions to ask yourself when hiring an independent contractor. This list of questions were copied directly from literature created by the state regarding independent contractors. As the organization dispersing the money you will use for your plan, we are also required to follow these guidelines. Therefore, you will need to provide the following documents to us when hiring any individual to provide services to you before our accounting department can process a check: * Proof that they are an established business offering services to the general public (i.e. a business card, flyer, letterhead, advertisement etc.) * A quote or invoice for services which lists their tax ID #(or social security #) * A copy of their license (if the type of work requires this) So, keep creating innovative plans and making wise decisions as you use the resources of the project to achieve your vocational goals. Center for Independent Living (510) 841-4776 Step 4 - Putting your plan into action Once you have been approved for funds through the IWRP, you have the responsibility to contact the "vendors" (schools, training programs, stores etc.) to arrange for payment. There are a couple of aspects to consider in doing this: You can work with any "vendor" you choose provided they are licensed or qualified to perform the work you are hiring them to do. (i.e., they do this for a business and can provide a Tax ID # or a Soc. Sec. #). Ask your counselor for more information about how to determine this. Only items outlined in your IWRP will be paid for. If you need additional monies for your plan, contact your counselor to do an amended IWRP prior to making any purchases. You will need documentation for the accounting department before any checks will be issued. Documentation means an invoice or quote from the "vendor" for the equipment service, etc., or a brochure outlining prices for classes. Other documentation may be accepted. Check with your counselor if you have questions. Once you have documentation of the prices, you should submit them to your counselor who will then write a "check request" to get payment from the accounting department. However, some "vendors" already have accounts set up with us, or will accept a "letter of authorization" from our program in lieu of immediate payment. Explore these options with the "vendor" and your counselor since any requests for payment take approximately 2-3 weeks to be processed by our program. When your check is ready you can pick it up directly from our accounting department during regular CIL business hours or at your request the check will be mailed directly to the "vendor". In any event, you are expected to get a receipt from the "vendor" for our accounting department's records. If you have any difficulties in dealing with the "vendors" during this process, please feel free to contact your counselor for assistance. Most "vendors" you will use are not familiar with our program and will often need some sort of explanation or assurance from someone in our agency. Step 5 - Success!! Whether you are starting a small business, getting some training, or simply looking for help with job placement the ultimate goal is to become financially independent. We need your help in "measuring" your success. So, let us now when you get that job, finish that training or start getting paid for your independent work. Send us copies of payments made to you for your work as a small business or a pay stub from your first pay period at work. Something that we can then use to say "this person made it." The success of this program and the idea of "choice" in vocational goals for people with disabilities will be determined by your success. So stay in touch and let us know when it works! v 53