SFGov

back to Juvenile Probation Department home page

Programs_Committee

2009 2008 2007 2006 2005 2004 2003 2002 2001 2000

MINUTES OF AUGUST 21, 2001 PROGRAM COMMITTEE MEETING
Held at City Hall rm 408 1 Dr. Carlton B. Goodlett Place SF 94102

The minutes of this meeting set forth all actions taken by the Commission on the matters stated, but not necessarily the chronological sequence in which the matters were taken up

1. (ACTION) Roll Call
The meeting was called to order at 4:45 pm by Comm. Aramburo. Comms. Grossman and Dupre were present at the gavel.
2. (ACTION) Approval of minutes of July 24, 2001, and August 16, 2001 meetings.
Action was tabled until the next meeting.
3. (ACTION) Possible recommendation to full Commission contract for continuation of contract from July 1, 2001 to June 30, 2002 with the American Justice Institute for PRIDE system, in the amount of $375,000.
Comm. Arámburo reviewed the process that led to today’s meeting. She asked for any comments from the Committee members. Comm. Grossman said she was disappointed that she could not attend the last Committee meeting at the AJI office. She was further disappointed to hear that there was no demonstration of the database and/or some form of analysis of the 400 cases already inputted into the system. Her understanding from the 7/24 meeting was that the purpose of holding the meeting at the AJI office was to get a demonstration on their computers of the database management.
She commented that the tapes of the 8/16 committee meeting were unintelligible (audible quality left a lot to be desired) and that people do not identify themselves when speaking, so it is difficult to make out who is talking. One interesting observation she made from a speaker was that it (the PrIDE program) allows us to have a leg up on the competition and that the policy makers will give us more credibility because we’ll have empirical driven arguments to give to them. If this is true, then she felt that would be very important. However, she said she was not surprised that they are not providing relevant data. In the beginning, they stated that they wouldn’t have relevant data for 3-4 yrs. But she is still disappointed that no demonstration was given this last meeting, with the data on hand.
Comm. Grossman asked the Chief what else is out there. She said other outcome management systems were referred to in different reports (eg. J. Bell’s report on Juvenile Hall detention). Do we have any dialog with those other places to see if there are other systems that can serve our purposes. Comm. Grossman continued to emphasize her concern for the expense of this program, and her awareness of the fiduciary responsibility she has to the Dept, the tax payers, and the kids we serve. She doesn’t want to see us wasting our money. She gave an example from business where a mistake is made and then rather than rectifying it, money is just kept funneling into it (called, putting good money after bad).
She agrees that the type of information sought would be beneficial to the Dept, but why are the people being measured affecting input of the data. Basically "the measured" are determining the output. This is a problem.
Even some of our biggest critics concede that we have a great database, and our CBO staffs know what trends are happening, what works. When you work day in and day out with kids, you "tweak" things until they do work. She repeated that it is a big responsibility to spend the Dept’s money on the children and families they serve. In conclusion, she is not sure spending up to the million dollars for this project, and not being able to expect anything for 3-4 yrs, is a good use of the Dept’s funds, vis a vis the big issues eg. LCRS, expiration of TANF funding.
Comm. Arámburo said this is a question of cost benefit analysis of the program. Even if this proposed model was subsumed in-house, would we then have the trend analysis and data reports that ProDES has in Philadelphia, to be able to further access funding through the Mayor’s office, or independently through foundations. The costs in Philadelphia are so high and we have no information that they have been able to use their data to access additional funding. So ultimately a decision is needed on whether an in-house model at $400-450K annual costs is something that is of significant use apart from the outcome measures.
There are differences between the outcome measures in Philadelphia and SF. Our contractors are relatively short term, unlike Philadelphia, so that the outcome measures reflect a window of time that is too short to produce meaningful results. Additionally, the trend results shown in Philadelphia have been toward greater institutionalization. Despite the costs and information generated by ProDES, it is not leading to the policy directions that this Commission would be interested in pursuing.
The Chief responded: the Dept is equally concerned with the financial responsibility it shoulders, and mentioned a couple of occasions in the past where it raised concerns about the performance of contractors and had recommendations for how those concerns could be addressed, including defunding.
He touched on a few areas: scope and purpose of PrIDE. PrIDE vs JJIS, data collection and CBOs as data collectors, costs concerns, how data used beyond program monitoring, leveraging additional $$ is a real use for this information. That’s the experience in Philadelphia. Good data on program performance puts you in a position to effectively argue for additional dollars.
Scope/purpose: produce outcome measurements for specific program objectives for every youth in every kind of program. Based on the Philadelphia model but specifically for SF. Begin with the contracted CBOs but to expand to every aspect of the operation of the Dept.
In its final stage, PrIDE will be able to produce case specific information, program specific information, system-wide, and topical information on specific areas. We will be able to tell who does well in what kind of services and be able to track changes over time, in clients’ responses and adjustments in their homes, schools, communities, and self esteem.
The JJIS is a data rich system (20+ yrs of work). It can produce aggregate statistical data, some retrospective trend information, and possibly some prospective trends based on various configurations of the data and projections. It can do some individual client information by event. But event driven client information is not the same as what PrIDE will be able to provide.
Point: POs taking data vs. CBOs doing it. Deficits for POs; their role as peace officers, their time to be able to take the information.
This program is different because it involves the CBOs in the designing of it. The issues that have come up among the CBOs were elicited by the program on purpose. This process had to be designed to convince people that the program was not going to be a hammer used to beat up community programs, to give them buy-in to the process.
Even if people at sometime provide false, incomplete, or manufactured information, that would become readily apparent over a relatively short period of time when you start looking at comparing individual program results and comparing similar currents going into similar services and you get major discrepancies and then you ask why is this going on. The system is designed to do just that. If some agencies wanted to supply inaccurate data, they would raise "red flags" about why is this, that and the other program doing reasonably well with these kinds of people, but these kids over "here" only stayed 3 months, none of them have any parents and they’re all graduating from Stanford (obviously some exaggerations, but) the data would begin to produce a picture that raises questions about why your results are so different.
Question: what period of time will this be? Answer: guess within the ensuing year after we get some follow up information on the set of 400, and then additional clients who are in the system, then we have a complete data set. Then we’re in the position to start doing some comparative analysis between programs. The Chief said possibly 6 mos from now, before we have that kind of data before we can start to raise red flags, not 6 mos before we are able to generate some kind of information or data from the system.
Question: that would just be a red flag on the data entry, not necessarily on the quality of the services.
Answer: it would be a red flag on the data input, and on what’s going on in that program.
Question: it could be that the data was corrupted for some time before the red flag would be triggered.
Answer: could be. But even if 20% is corrupted, there’s still 80% good solid information.
Comm. Grossman: the bottom line is the dollar. Is the contractor providing information that will do what it is supposed to do, at the best possible price. We still haven’t seen a demonstration of that, after spending $654K.
The Chief: if you provide child specific data in a public forum, then you risk violating confidentiality. There should be some way to provide that information in some form that was responsive to the requests of the Commission. It can happen, it will happen. In terms of deliverables, probably by mid-Sept we’ll be able to have the first report from this system of those clients in the system right now, and subsequent reports at additional integrals beyond that. We can provide examples of the kinds of reports this system will be capable of doing when it is fully implemented. In terms of other systems (eg. Santa Cruz, Portland), they’re not doing this. We got some positive comments from J. Bell in his report about the data we provided to him. We do have good solid data and good people who manage it, but that is not intended, designed, or capable of producing these kinds of things. At some point there would be a convergence between what we’re doing now and what this system is designed to do, in that integration process, and in fact we would be able to use our data in this process to do lots of different kinds of things. Right now, it’s two separate realities, and what’s happening in Santa Cruz, Portland, Sacramento, is not this. This is one of a few of this kind of system in the country, that we’re aware of, and the kind of growing pains we’re going through right now are just that, from our perspective, growing pains.
Comm. Arámburo: that’s one red flag. Why is there only 2 jurisdictions (Philadelphia and SF) that feel this is a good use of funds. Especially in light of the previously stated fact that in Philadelphia, this hasn’t helped them in establishing a trend toward less institutionalization. The 2 questions arising are: is this the best model, and is this the best vendor. One million out of a ten million dollar TANF budget is significant, and there is not a single report.
She further stated that she was uncomfortable with this request and that the Dept was recommending an extension, without some sort of accounting. The reports to date have been static data, but for the most part they are items from the Philadelphia model, with some tweaking for SF particulars, but bottom line, how significant is it.
The Chief said that SF is not the only place that thinks this is a good model. He cited it as a finalist in Harvard U and The Ford Foundation’s competition for Innovations in American Government award. He said that "many jurisdictions have heard a presentation on it, and haven’t taken it up because they are unwilling to take hold of it and do what needs to be done, to give it time to take root and grow and be productive…. So, we are not the only ones." The vendor was selected because they participated with CJRI in Philadelphia. They were involved in that implementation.
The Chief requested the Committee approve the contract provisionally with requests for periodic reports.
He reiterated that he wanted the Committee to approve the contract, and that they be allowed to go forward, and they would be required to report back Sept, Oct. Nov, as often as the Program Committee wants, when there are specific questions you want to raise. And if for whatever reasons over that period of time if they don’t demonstrate the utility of this process, and its results, then the Commission wouldn’t have to ask him to terminate the contract. They’ve done it before.
Comm. Arámburo said she asked Dr. Bennett at the July 24 meeting, which the Chief attended; how much did it cost Philadelphia to get it up and running ($4 million, and an annual maintenance cost, of which 45% is indirect costs) and he couldn’t answer. She asked the Chief to tell her now what’s it going to cost to get this program "live?" Is it going to cost another million dollars?
The Chief said that by converting the positions in PrIDE to equivalent civil service positions, they obtained a figure of around $475K as ongoing staffing to operate it. There is no way to estimate the integration costs because it was going to be a part of the 2000-2003 budget formulation process.
Comm. Grossman recounted that in April 99 when asking for $150K, at that time the Chief said that he would be putting it into the annual Dept. budget, and thought that DCYF would also be able to support ProDES. In Jan. 26, 2000, the request was for the $197K grant, where projected testing of the evaluation system would be done in July, including TANF projects. In Sept of 2000 when they came to the Commission for $307K, this was going to totally come from TANF funds, and the Chief noted that he was looking to put this project into the Dept’s budget, but that would happen in 2-3 years.
She said that if the project could demonstrate this system by the end of Sept, she’d be inclined to go with the provisional approval.
The Chief repeated his request: "Program Committee recommend to the full Commission that the contract be approved conditionally and that it go forward and be allowed to continue to operate and the conditions would be: by the Sept meeting we answer these questions, by the Oct. meeting we answer these questions, and if at any point in time we are unable to provide satisfactory responses to those questions, then I’m going to have a concern, along with the program committee, and then we’ll have to make a decision at that point. This contract, like any contract is a commitment to spend up to the authorized dollar amount, and this contract as in any contract, there are opportunities for us to get out of it if we feel we’re not getting the money’s worth, and not getting the kind of performance that’s expected."
Comm. Dupré offered as a condition to the approval, that a first report and demonstration of the deliverables be presented in mid Sept, and that PrIDE report every month for the first 6 mos, to the Program Committee, and a breakdown of how dollars are to be utilized.
Comm. Arámburo said that it might be time better spent if the Chief met with the AJI and figured out what it’s going to cost to really get this project going, and that if it took a week, then they (the Committee) should wait. She suggested the item be tabled for two weeks until a full estimate of the expected costs for the full project could be given to the Committee.
Sheila Arcelona said that all the conditions could be made a part of the scope of work in the contract.
Comm. Grossman wanted an estimate of the total costs by a specific date.
The Chief said that by the Sept meeting they could provide a proposal for some options for integrating the system into the Dept and its continued maintenance, assumptions that accompany those options and related costs.
Comm. Grossman said that if all of that was included, she could second the motion. She asked, what happens if they don’t meet the provisions. Is the contract cancelled? The Chief said that the Dept has the authority to do that.
There was no further discussion. The question was called and on row call vote, the motion passed 2-1, with Comm. Arámburo voting no.
4. (DISCUSSION) Public Comment on any matter within the subject matter jurisdiction of the Juvenile Probation Commission.
Comm. Dupré said that this was a very arduous task, getting to the approval stage of this contract. He noted the attention to the fiduciary responsibility of the Commission, and said that he still believed in staff that we are doing something no other county in Calif is doing with TANF dollars, that this project would help leverage funds.
5. (ACTION) Adjournment
There being no further business, the meeting was adjourned at 5:45pm.

Meeting Information

Programs_Committee