|
(DISCUSSION) PrIDE report on Morrisania West’s YERP.
L. Jackson-Simpson commended D. Mauroff and Comm. Richard on the work of putting on the gang conference.
L. Jackson-Simpson said the program served kids 14-19 yrs of age, with goals of supplying employment and job placement services. She said there were 15 intake forms making up the dataset, with 8 discharge records. These are the old AJI forms. She mentioned off the cuff some anecdotal information about the clients.
Sheila Arcelona (monitor) introduced the services of the program.
Comm Richard said that the CBO should have been here. He asked the staff to report back to the organization his feelings.
Andrew Robinson, LFA, said that this report basically is a profile of participants. encompassing youth who are 17-18 yrs of age, mainly male, African American, in low income families. He gave a summary of the demographics of the sample population (copy of the report available from the Community Programs Division). He said that the old forms and system was not set up well. There are no items in the surveys that ask what specific things worked for them in the program. Just items asking for personal feelings. Sheila said that there are different assessments. The intention is to go through the entire program, giving pre and post questions to see if any of that program worked.
Comm Richard asked if CARC collaborates with PrIDE in gathering data. L. Jackson-Simpson said that CARC is not a dept contractor so isn’t required to work with it.
Ques: how many times has LFA been in touch with Morrisania for collecting data. LFA reported that Morrisania has been very pro active in wanting to get data to them. Gathering data for them has been difficult.
Data collection has been only on a sporadic basis (mainly during intake and discharge). What LFA would like is an ongoing data collection process, so they can gather periodic progress data.
Comm Bonilla asked what were the outcomes being looked for: recidivism? Achieving educational goals, employment goals? And if they were any of those, what was it that made that happen? That is the data that he is more interested in. He thinks that’s the more important information to know; what is the thing that keeps youth out of the system.
A. Robinson said that the old system didn’t have anything in those forms, which asked what interventions led to positive outcomes. Participants’ are asked at intake what they want out of the program, and in the end they are asked again did they get what they wanted out of the program.
S. Arcelona said that the intention is that the participant would get the full offering of a program, and then testing whether or not those things worked (in outcomes).
Comm Richard brought the discussion back to the YERP report.
There were no more questions regarding that particular report.
(public comments)
Jacqueline Lingad Ricci, speaking as an individual and citizen, asked the Committee to look at how much has been spent on the PrIDE project to date, how many CBOs have benefited from the project. She also encouraged the Committee to review contracts and performances to see what direct positive benefits have arisen from the CBOs themselves. The survey that CBOs are required to complete and return to LFA didn’t appear to her to have been all that extensive and so the knowledge gained from them may not be equal to the costs for getting them. Maybe the funds (in these hard times) could be better used directly by those CBOs who are suffering cuts in funding. Perhaps using part of it for direct services to the youth served by the Dept. Perhaps look at using Dept. staff to do these surveys. She emphasized that in these hard economic times, the Commission and Dept should look closely at its priorities.
(ACTION) Review and possible recommendation to Finance Committee, on renewal of contract for LaFrance Associates for FY 2003-2004.
L. Jackson-Simpson reviewed that last spring LaFrance got the contract to continue the PrIDE program. They have been using the original documents from AJI, and revised some to come up with more timely responses. She said that the money for this comes from TANF and for the purpose of evaluation, and was approved by the JJCC, which is the body that oversees and sets the spending priorities for TANF monies. LFA has revised the instruments to make them more user friendly. They use a logic model, which will enable them to understand the relationship between the services youth receive and the types of outcomes expected. She said the contract for this year is $30K less than last year’s. Last year was $255K. This year, it is $225K.
Sarah said that LFA is also trying to develop reports that summarize information from providers that will be given back to them so they see how they’re doing. In this next year, she referred to the report given to the Committee regarding their planned direction, they hope to further build the program (integrate it) into the Dept’s operations.
Comm Bonilla asked what that means. Ans: the goal would be to have the Dept run PrIDE, with possible assistance in writing program evaluation reports.
Comm Bonilla asked why it should take another year before it can be integrated into the Dept’s operations (expecting that in 2004-05 this will be fully a Dept initiative)? He asked if this is the best way to spend dollars when there are other direct services are in great need? He asked whether there was a way to shorten the delay the time till integration is done. L. Jackson-Simpson said that LFA has had to re do the tools used in the PrIDE program, and is just now starting to collect the data, educating those needing to know how to use them.
S. Arcelona said that there was a solicitation (for the PrIDE contract), which could be extended for three years, with the phase in plan. First they have been concentrating on the community programs. Next they wanted to do the probation services to see how effective that is, but this has been slow in coming. There are no links between the data collection and probation services. The Dept’s IT division doesn’t have the capacity to take this over. So, as the Dept takes over more responsibility (eg. IT is taking over programming responsibilities that were previously contracted out), the plan was for a 3 year transition.
The Dept also needs to have new positions to take on this added work.
Comm Richard asked where we are now compared to a year ago.
L. Jackson-Simpson said it was a question of capacity. Evaluations can be costly, but she felt the Dept was getting “big bang for our bucks.” The tools are revised because the documents developed by AJI kept reports from being done in a timely manner. The tools are streamlined and more periodic reviews are being done. Their operations are now on the premises, side by side with IT. There are more responses from the youth (satisfaction surveys).
S. Arcelona said the Dept has a responsibility to evaluate the effectiveness of the programs. Approximately $4 mil is given to programs, and about $250K is used to evaluate them.
Quest: how is this knowledge benefiting the Probation Officers? Ans: they are given summary sheets of program evaluation, available to them for referral of their wards. A. Robinson said that they are working with POs to see what information they want that will be helpful to them.
Quest: which CBOs have been successful. Ans: eg. Occupational Therapy Program used their evaluation to help leverage other funding, and to expand their services. S. Arcelona also gave Girls 2000 as a small CBO who used the PrIDE information to assist them in funding searches.
Sarah said there was a great resistance from CBOs in doing data collection. After the report on Girls 2000, they were asking for how to get more data back from PrIDE regarding their own services. Now they want to cooperate with the PrIDE process.
Comm. Bonilla asked how long PrIDE has been in the Dept. S. Arcelona said since 1999, but they didn’t start data collection until 2001.
Ques: what has the total cost been? Ans. $150K budgeted, $100K spent. The rest carried forth to the next year. Arcelona didn’t have all the figures on hand, but offered to get it together for the committee.
Comm. Bonilla expressed concerns and questions regarding the costs and time spent on this project (eg. why original contractor was not kept if things were going well). He acknowledged the importance of data, but he continued to wonder what we would have once it was over, 7 yrs and more than a million dollars.
Gwen Tucker answered: “ we started from ground zero. It was the total development. The first two years was total development. That means writing all the programs and all that kind of thing, and bringing, it was the start up funds. If you start up a program, if you were starting, and I’m sure you, if you worked for the public health you’d know what it takes for start up funds. Any kind of program that starts up, it takes at least a year to get it off the ground. And what we’re talking about here is information technology. We’re talking about also, evaluation techniques and we’re talking about crafting it in such a way that it meets San Francisco’s needs. We also ran into the situation where we had to look at the various kinds of community programs that we had. Some programs were more sophisticated, and some were less sophisticated. So the information that was gathered had to be, if you take, ( ) it, so that each kind, each program could be involved. This took quite a while, and if you’ve written a program or run a program from the beginning, from ground up.. and I’m sure Commissioner Richard would understand this, when you start a business, the start up is the hardest. We’re beyond start up.. We struggled, we went backwards and forwards trying to get the system in place because of the importance of getting some evaluation tools for the community programs, because it would in fact enhance the services they are able to deliver. Yes, we made the investment. Now we’re at the point of delivering and that’s what LaFrance is doing. We learned by our mistakes, but now we are doing that, and we are at the point now where benefits is what we’re getting in return. It’s just like an investment in business, a capital investment. We made the capital investment. Now we’re at the maintenance point. We spent also a great deal of time with information technology thinking this, and integrating this system in our current JJIS system, which took time from the information technology staff to work with LaFrance, so that this technology will read on the JJIS system, and we will be able to then run it ourselves. So we have crossed the hurdle getting to the point where we can deliver. Delivery might have taken longer than we expected, but its got to do with the peculiarity and the differences and the various programs, because the programs we have span from the very smallest to the very largest. So any kind of evaluation system we’ve got to meet all of those needs. And that’s where we are now where we can get the studies, the reports out. When it was done in Philadelphia, and done around the country in other places, it has taken as much as four years to get where we are now, so we , yes, you do the capital investment, as you do in any business, but you reap the results, and that’s where we are now. And LaFrance is delivering that.
Comm Richard asked if training the staff was written into the program?
L. Jackson-Simpson said that over the years, the Dept has had to reduce staff. This has taken time to get to this level, where CBOs are getting used to doing data collection.
She mentioned that other city agencies are now looking at adding evaluations to their contracts, and are talking to the Dept about utilizing PrIDE for their process.
G. Tucker said that before TANF monies were used for this, they used general fund monies. Federal grantors want to know the data. And now foundations are asking for data driven outcomes. “It is imperative that we have evaluations of individuals we are contracting with. And that’s one of the reasons why we have spent so much time working with the community, to get them onboard, to get into the mindset, to buy into it, because it enables them better to serve and they get feedback about what they’re doing and what works and what doesn’t. So then they are in a better position to ( ) federal dollars or should these TANF dollars disappear, to say, these area the results that we had with the programming that we did, and they can go to foundations and get money because they have results, they have demonstrated results, they have real time information about the kind of work that they did. So, it’s not a time economically, and we’ve been moving in this direction for a while, where we can do feel good and say ‘okay, well, that’s my homey, and I know they do good work.’ But then you say what is the demonstration of the outcome, where is your data that says you got results. That date is far past. People do not give money, and with the money that we put out with this dept, when you look at the amount of contracts that we have, it’s almost, and you include the MOCJ money, we’re putting out about $5.5 million dollars in contracts. You cannot do that without having some demonstrated results.”
Comm Bonilla questioned the validity, the credibility, the objectiveness of the surveys used for compiling data. What’s the control against the “homey factor”?
A. Robinson said that in research design, this is a common issue. It represents a small margin of error, generally. The questions generally give options for answering or not, and for the most part they do not ask for personal feelings, but just quantitative matters.
He said that much more data from other sources has to be brought in to help in assessing these surveys. The sample size has been disappointing (12-15), but partly the difficulty is in getting consent from parents for those who are not wards of the court. They are trying to resolve this issue because it is holding up their using completed sets of surveys (they have 65 from one program they are waiting to use).
Quest: is consent a legal requirement or choice?
A. Robinson said it was in place when they began the contract.
S. Arcelona said that previously Judge Hitchens made them get consents from those who were not formal wards of the court, because of the way AJI had put the forms together, but LaFrance is revising them so they can separate programmatic information from personal identifiers. They will ask Judge Hitchens if this would be allowed without the need for consent (presented 2 weeks ago).
G. Tucker said, that “it is as valid as any self reported information can be. There are several kinds of researches done. There are self reported, which is the kind we’re doing here, or the validated, and if its validated, we would go to JJIS to look at some key points. But we’re not validating information, we’re using self reported information, so what you say about the fact that information may be skewed. That is a possibility any time you’re doing a survey, but once you get the aggregate information, you can almost cross check with some of the questions, the design of the questionnaire, there’s almost cross checking on it, so if you say something adversely different, somewhere down later in the survey that is cross checked and if it’s adversely different, it’ll come up. And then there’ll be a significant difference of information that we get, when you’re talking about evaluation. So the research design does address that and I know we’re all kind of naïve about research, however that’s the reason why we have research associates, we have individuals who do that, ..I’ve studied research and I’ve done statistics and I’ve done this kind of stuff for quite a while, but I choose not to. But I do understand the process because it is very difficult, but the design itself does some cross validation of information. And the fact that the staff administers it, and is trained to administer it. And that’s the reason why you have closed end questions so the kids can less get around the questions.”
Quest: have there been times when information of a negative nature had been given back to the CBO and they were able to improve themselves? Ans: yes.
S. said that they are still struggling to get a complete set of data forms so they can do an evaluation of the program. They said that in all the reports done to date, there have been “snippits” of this, but not complete as they would like to do. They’re hoping that they will be able to do some soon.
G. Tucker said that “it is our responsibility as funders to provide technical assistance to those individuals that we fund. This information, such as the information cited about Morrisania West, when we begin to do technical assistance and we would do technical assistance and training with the CBOs, which we do on a regular basis, it is this information that we can help them by giving them the technical information and helping them do things better that they weren’t doing, so, any information that is negative isn’t used as a hammer, it’s used as constructive feedback so they can develop better programming, or be better at what they do. We’re not using this to determine whether they’re funded or not, this is an instrument for technical assistance as well as constructive growth, because what we’re about here is empowering the community to serve their own. But if they don’t have the information to serve as they need to, because they don’t have the information, then they have no way of knowing whether they’re good or bad. And we have no way of knowing what kind of technical assistance they need because very often the CBOs don’t know what they need, but we’re getting information here to improve our technical assistance to the community based organizations, which in turn will empower them to do better jobs, and then they would be more capable of serving the community, so it’s a circle.”
There was a discussion about having adequate data in order to do evaluative reports that would help in improving the CBOs interventions. They are not there yet.
Comm. Bonilla summarized that his biggest issue is that when the transition is finalized, there’s going to be more than $2 mil. spent on a system, which will still not be totally operative. His preference would be to spend that money on direct services.
G. Tucker asked, was the contract already signed?
L. Jackson-Simpson said that the JJCC approved the expenditure of the funds, $225K to be expended for evaluation. The Dept is here to recommend the continuation of the LaFrance contract into ’03-’04.
Comm Bonilla asked what would happen if the Commission said no.
Ans: they would have to go back to the JJCC.
G. Tucker said that she thinks “the protocol is that the Committee either send the recommendation or the questions for further discussion. Also, one of the concerns, which she feels is a serious concern, is the investment that has been made in the programming, in the evaluation system, the, as far as we have gone and the impact that it’s had so far in terms of the outcomes and the results that we are now reaping. A pre mature break in this would mean, the community based organizations would lose because they would not be able to get the information that they are now getting, which is real time information, which gives them feedback about how they do business, and therefore, they would in fact they would be able to do business better and better enable to get their services done. Also when it comes to the issue of TANF funding and its availability, if the TANF funding is not completed, that, if it’s not continued by the Federal government, the CBOs are going to need to have information to get private funding so the information and feedback they get here, will enable them to get funding from other sources, in fact the information that they get now, which is the evaluative information resources, will enable them to augment the funding that they have, so while you were saying that you would prefer to direct funds to another CBO, for instance, in a sense, not funding this program to continue, you’re disabling them because we have an uncertainty about when TANF is going to, if TANF is going to be continued. We need to be enabling the community based organizations in giving them as much help and technical information in this next year as possible so that they will be positioned to have the information to move forward to get alternative funding, and we see this evaluative information and the technical assistance that we will be giving as a result of that putting the CBOs in a better place to move forward, than they are now. But next year they’ll be in a better position. In terms of the transition, nobody can guess what’s going to happen transition. No one can guess what’s going to happen with the next fiscal year and the condition that we’re in. We have to maximize what we do for our CBOs, and what we’re asking is that you maximize the opportunity for the CBOs to get as much information as possible, to improve themselves as much as possible, and thereby position themselves to get additional funding. They’ll be in a better position because they will have outcomes about what they’ve been doing. There are many community based organizations that have been doing this for years, but have nothing to present to foundations, etc. about outcomes and what they’ve been doing, so we are providing that kind of technical assistance, that kind of technical information to the CBOs.”
The discussion continued around the benefits to be gotten from the PrIDE system vs. the continuing problems of developing it to the level of credibility and completeness where the data does help CBOs improve their services (not there yet) as well as gives us concrete information regarding the outcomes of their services on the Dept population.
LFA admitted that data sets are not complete and they look forward to the day they can do analysis of program services for their efficacy, while hopefully the CBOs can be convinced of the usefulness of this program and cooperate in gathering complete data.
It was repeated that the Committee was concerned with the cost benefits associated with getting this project fully operational, vs. directly helping CBOs in tight financial times.
Comm Bonilla moved to recommend this project to Finance Committee for further discussion on the financial aspects of the project.
G. Tucker said that procedurally, any decisions that come from this committee have to go to the full committee. Parliamentrally (sic), recommendations don’t go from one committee to the other. She said that “you could make a recommendation, and if the City Attorney were here, I’m sure she’d explain it to you. Committees don’t recommend to another committee. It has to go back to the full Commission with the information, and then it comes from the full Commission back out.” That’s her understanding of what the requirements are. One committee cannot recommend to another committee. It would have to go to the full Commission, and then the Commission would send it back to the Finance, if that was what was so desired by the full Commission.
Comm. Bonilla differed with that understanding.
L. Jackson-Simpson said that these dollars are governed by the Juvenile Justice Coordinating Council and not the Dept. so if you decide not to contract with LaFrance, then the Council could do what their pleasure is with that resource. It doesn’t necessarily mean it will go to programs or CBOs.
LJS repeated that the monies are a carve out of the federal dollars, and the governing body on the expenditure of those funds is the JJCC. The Commission can say yea or nay on individual contracts, but that body decides how the dollars are spent.
S. Arcelona said that they will be asking the MOCJ to buy into using this system for their programs, to help defray the costs. They will be presenting to the MOC in June.
Comm. Bonilla asked for background information on the PrIDE program; costs to date, and comparisons between what was projected in the contracts with what was actually accomplished, projected costs until the final transition, why AJI was not chosen to continue, and a copy of the LaFrance contract.
A. Robinson thanked the Committee and said they were happy to provide what the Committee wanted, if they haven’t already been able to answer all the questions.
(public comments)
L. Holmes said that while LFA has done a commendable job, there have been problems with the PrIDE program since its inception. He estimated that to date the Dept has spent $1. 6 mil on getting this system in place. There are outstanding problems around the information gathered. Some of it is sensitive and can affect the youth. POs are required to report things that might be violations of the law. This will have to be addressed if a more complete data gathering process is to be successful.
(DISCUSSION) Update on the Gang Free Communities Initiative (D. Mauroff)
D. Mauroff reported that in April they completed their implementation plan for the Bayview Hunters Point, and Mission district. This will be the masterplan for what they will be doing in this next year. Also, the collaborative just finished a successful one day conference on gangs, May 8, attended by about 350 people. Now it’s a matter of oversight to see the plan through. The Grand Jury will be issuing a report on Truancy soon. The schools are trying to deal with the issue, and the Gang Free Initiative will be a partner in that.
Comm. Richard asked if the project is on time. D. Mauroff said that there aren’t any hard deadlines. SF is on the higher level of completion rather than lower end.
Comm Bonilla asked if there are other depts. that can help budget the work of this initiative in their budgets. Mauroff said that the whole point is re allocation of resources from within different depts. to assign staff to this initiative, rather than adding new costs. Each partner is tapping existing resources, with the over arching goal to reduce violence. The most serious risk offenders will be worked with, and hopefully with that will be able to affect the incidence of violence in the neighborhoods.
Comm Bonilla asked if there were actions intended to work with the upper echelons of the gangs. Mauroff said there are criteria (eg. 24 yrs of age), but for the older youth, Operation Ceasefire will probably impact that.
Comm Richard asked which orgs were involved. In the Mission: Arriba Juntos, Mission Neighborhood Centers, Boys and Girls Club, Horizons, Instituto Familiar de La Raza, the Community Bridges Beacon. In Bayview Hunters Point; Girls 2000, Brothers Against Guns, the Peacekeepers. The goal is to identify other partners to deal with other related issues; substance abuse, employment, along with Rec and Park.
(public comments)
L. Holmes commented that the focus should be more at-risk kids and not just those already involved. That’s what is needed in the Bayview Hunters Point.
Public comment on any matter within the subject matter jurisdiction of the Juvenile Probation Commission
Comm. Richard again congratulated L. Jackson-Simpson and all those who successfully put on that one day seminar on gangs and gang violence.
|
