Reflections on Managing Client Requests for Evaluation Data Before Contract Ends

Here is what happened:

I heard from one colleague on the EVALTALK listserv who thoughtfully recommended that I revise the contract, if it did not contain language clarifying access to data by the client. I thought this was a good idea going forward anyway. Then I made a list of the benefits to the client to have access, the negatives of granting access, and the benefits and negatives of granting access for my professional obligation to my client. I drafted a memoranda in which I proposed some language that would clarify my expectations of the use of the data by my client. Here is what I said:

a. The external evaluation contract provides that analyses and reports based on findings from data collected during the program year require that the evaluation company must conduct the analyses and reporting on company computers.

b. The external evaluator could not verify that the integrity of the data and any analyses conducted by the client, and, therefore, could not use any data set prepared by the client in the external evaluation.

c. The consent form signed by providers and parents prohibits the external evaluator from providing individually identifiable information to anyone outside of the evaluators and evaluation company.

d. The client understands and agrees to the condition that the external evaluator may not use data files derived from an analyses of the (information provided) for the external evaluation, unless joint analytical plan is authorized.

e. The client will use the data provided only for internal purposes, and not submit reports derived from the analyses outside of the internal evaluation staff.

e. By agreeing to these conditions, the client will make no changes to the budget based on providing the information requested.

At a subsequent meeting with the client, after reviewing my draft proposal, the client agreed that sharing anonymous data would not help them, and that the conditions outlined by my draft proposal are appropriate. They withdrew the request, and we proceeded to move forward on the evaluation plan, amicably and on a sound relationship.

During the meeting, I also learned that the request was made to see if there was some way to assist me in making the tight deadlines that were required for the grant, and not for any other reason that may have been imagined. Because of my efforts to remain as transparent as possible - usually expressed to them through details about the status of the data collection, and my transformation of my evaluation infrastructure to comply with the evaluation requirements - they were becoming anxious about my capacity to meet the contract obligations. While they valued my transparency and honesty, I had made the client managers more anxious over time.

The lesson for me is to consider the client's capacity for reception of information (transparency) about the process of an external evaluation - while transparency models the way the evaluator wants the client to communicate, the side effect may by an erosion of confidence in the capacity to deliver on the contract.

This experience is also a confirmation of the principle that challenges are really opportunities to improve and learn to be better.