Moving on…

The consultations on the ISEAL Impacts Code have moved.

Please join us at http://community.isealalliance.org/impacts-code for further updates and discussions!

Impacts Code Newsletter August 2009

During July Patrick and I have been able to undertake a major rewrite of the Draft Impacts Code based on input from the Steering Committee. I have been in contact with most ISEAL members regarding consultation on the Impacts Code and helping them articulate their theory of change. I took a week of holiday as well.

Yours sincerely,

Paddy Doherty, Credibility Tools Manager

Impacts Code Newsletter July 2009

The Impacts Code Steering Committee met on 18-19 June and provided detailed comments and suggestions for improving the draft before it is posted for the first formal comment period in September. Patrick and I presented two workshop sessions at the ISEAL AGM conference at Royal Holloway University (22-24 June) and received good feedback from these sessions.

Yours sincerely

Paddy Doherty

Impacts Code Newsletter June 2009

Since my last newsletter the Draft Impacts Code has undergone many changes; getting it ready for final review by the Impacts Code Steering Committee on 18-19 June. Both the Impacts Code Issues Committee and the Methodologies Committee had meetings to review and discuss various aspects of the draft Impacts Code. I had direct conversations with many members of the Impacts Code Committees, and a continuous stream of comments on various sections of the draft document. The latest draft is posted on the Impacts Code Wiki for public review and comment.

Yours,

Paddy Doherty, ISEAL Credibility Tools Manager

Impacts Code Newsletter May 2009

There’s been steady progress on the draft ISEAL Impacts Code in the last month. Amazingly, the Impacts Code committees do not seem to tire of this work, and have been both critical and encouraging. Here is a copy of short note from Francisco Bustamante of UTZ Certified, which is typical of the committee support I receive:

It is very interesting to see the evolution in the Code. You will find my comments enclosed directly in the text. I think we need to pay more attention in the methodological issues in data analysis, because the statistical methods have their own procedures; so I suggest not to be very descriptive and let the standards define the quantitative and qualitative data handling according with the selected indicators and all the other considerations in the Evaluation design (with a good advisory sources). Congratulations for your hard work.

In this newsletter, you’ll find more good suggestions for the ISEAL Impacts Code, a theory of change for standards systems and – of course – an overview of the steps ahead.

Enjoy reading!

Paddy Doherty
Credibility Tools Manager, ISEAL Alliance

Impacts Code Newsletter April 2009

Since the last newsletter Sasha and I attended the “Growing a 21st Century Agricultural Revolution” in Lansdowne, Virginia March 18th and 19th.

The presentations and subsequent discussions were excellent, and I took away some good feedback on the Impacts Code. One important point was that the economic theme was not as well developed as the social and environmental themes. This is true, and has prompted me to spend more time on the economic issues.

Since returning from Lansdowne, I have been working fulltime on the draft Impacts Code. This involves lots of research, thinking, and writing, deleting and writing again. I have spoken or exchanged emails with many of you during the past month and I appreciate all the help I am getting—the resulting Impacts Code will be better for it.

Highlights

The Impacts Code wiki has been very useful as a way to enable everyone to see the latest thinking on the draft Code and to take part in its development. It allows me to upload current drafts and to manage them online. Use by others is growing slowly, and I realise not everyone will want to contribute to the wiki, but most people are happy to view the latest draft online. The Impacts Code Blog is growing slowly, and requires constant input on my part—to make it interesting for people to want to go there.

I have been considering ways to make the theory behind the Impacts Code more understandable and relevant for standards systems. Concepts like the ‘theory of change’ are developed and discussed by social scientists, who have interest and background in this area. However, I am trying to construct a document that can be readily understood and implemented by management and staff of standards systems, who generally do not have this background.

I am writing lots of guidance, to try to explain the theory in plain language. This is not easy—there is a reason why specialists speak in their own vernacular—but I am persevering and I believe, getting somewhere.  I have just completed a draft of section 7 (Developing the Assessment Framework), which explains to Standards Systems how to describe a theory of change for their work, and subsequently how to use that theory to determine what to monitor.

This is tough going for me, but as I said above, things are becoming clearer. I have rearranged the draft code a bit (this was expected, as the draft I put up in the wiki was quite rough) and developed the introductory section titled “The Context for Impact Assessment”. This new section explains the theoretical basis for the Impacts Code in what I hope is plain language (though any comments you want to provide in this area are very much appreciated).

Next Steps

The drafting procedure is going as planned, with a draft ready for review by the Impacts Code Steering Committee in June. Once the Steering Committee is satisfied with the draft, it will be posted for formal review. I’ll be posting new sections of the Impacts Code on the wiki today. So please review the draft, make some changes to improve the document, or leave a comment.

Who should do impact assessment?

While attending the Growing a 21st Century Agricultural Revolution conference in Lansdowne Virginia last week, I had the opportunity to speak with a researcher (on impacts) from the Netherlands.  He was admant that organisations should not do their own impact assessments.  His reasoning was that the crediblity of the assessment would be threatened by the perception of a conflict of interest (if you do it yourself, you are more likely to make your organisation look good).  He recommended that impact assessments should be done by outside organisations, unconnected to the subject of the impact assessment.

This suggestion is similar to what has been proposed by the researcher ISEAL contracted (Aimee Russillo) to write the two impact research papers that inform the theory behind the draft impacts code.  Aimee suggested that standards systems should develop monitoring and evaluation programs, then use the data from ongoing monitoring and evaluation to develop stand-alone impact studies.  The impact studies would be carried out by independent researchers (acedemics, consultants, NGOs, perhaps other standards systems) on specific aspects of the standards program.

The suggestion (that standards systems should not do their own impact assessment) is reasonable from the viewpoint of credibility, but it may make impact assessment (already a large task) even more daunting for standards organisations. We are developing a tool that will provide credible results; yet is understanable, and reasonable to implement.  Is there a middle ground? If a standard system should wish to undertake impact assessment within their monitoring and evaluation programs, there may be ways to ensure the credibility of the results.  But what about perception–will those results ever be viewed as credible? All of us involved in this project are having to consider the implications–eventually we’ll have to make a decision.  I’d appreciate your point of view.

Paddy

Impacts Code Newsletter March 2009

This is the second newsletter on the Impacts Code Project. If you wish to receive these updates via email, please sign-up to the mailing list in the sidebar.

Paddy Doherty, Credibility Tools Manager

Update

Since my last newsletter, we’ve had a meeting with the Impacts Code Steering Committee (teleconference, Feb 26). All three committees (Steering committee, Issues committee, Methodology committee) have been working hard on the Issues (what to measure) section of the Impacts Code in preparation for posting on the Impacts Code wiki today. I have been communicating with many of you in the development of the draft code and I can happily say that I am pleased with the results of our efforts. Until I began work on this project, I thought that sheep shearing was the hardest thing I had ever done.

Highlights

Perhaps the biggest news is the Common Impact Hypothesis for Standard Systems. This idea originated during the Impacts Code Methodology Committee meeting in Boston on January 28^th. The (draft) common hypothesis:

“Standard systems are an effective tool to deliver on healthy ecosystems, social well-being, and economic stability.”

Though it’s a simple concept, agreement on a common impact hypothesis allows for a collective perspective on standard systems as an entire movement. The common hypothesis brings the voluntary standards movement beyond compliance with standards (our usual focus); to allow for the progress of the movement to be evaluated as whole; against other types of interventions.

Coupled with this notion was the idea that the effectiveness of standard systems should then be assessed along with their contribution to environmental, social, and economic goals.  Thus, the Impacts Code Committees have come up with four meta-themes (what to measure):

1.    Effectiveness of the standard system

2.    Environmental

3.    Social

4.    Economic

The committees have developed draft (level 1) meta-themes, (level 2) themes, and (level 3) sub-themes. Our intention (so far) is to develop example indicators for all the level three sub-themes. How prescriptive the Impacts Code will be regarding what standard systems will be required to measure is a matter for you to ponder and comment about. At present, I believe the Impacts Code committees are thinking that standard systems will need to consider their contribution to impact across all three meta-themes (‘effectiveness of the standard system’ will be required) and if they do not monitor and report on all three, then they will need to explain why not.

Consultation

As promised, ISEAL has developed an Impacts Code Blog and an Impacts Code wiki. Please visit both sites; sign in to the wiki site so that you can modify the draft text and leave comments. Using these tools is a new experience for me and I am enjoying learning about ways we can work more effectively and reduce the need to travel to meetings.

The draft themes document (what to measure) is now posted on the Impacts Code wiki for review and comment. The draft themes will remain on the wiki, and I will post other draft sections of the code as they become available. I’ll let you know in subsequent newsletters when new sections are available for review.

The steering committee will meet in June to review the draft Impacts Code before it is posted for formal public comment.

Next Steps

Sasha and I are attending the “Growing a 21^st Century Agricultural Revolution” in Lansdowne Virginia on March 18-20. We’ll be presenting about the Impacts Code project and I expect there will be opportunity to discuss some of the details of the draft Impacts Code.  I am currently working on section six (Defining the Monitoring and Evaluation Program) of the Impacts Code and getting sections one through four ready for posting on the wiki.  I’ll be working on section seven (Theoretical Models) throughout March and April; and I will really need your help with this.

Impacts Code Newsletter – February 2009

This is the first of what will be monthly updates on the ISEAL Impacts Code development process. If you want to receive these updates via email, please sign-up to the mailing list in the sidebar.

Paddy Doherty
Credibility Tools Manager, ISEAL Alliance

Overview

Purpose

The purpose of this project is to develop a Code of Good Practice for Measuring the Impact of Social and Environmental Standard Systems (the Impacts Code). The process (for developing the Impacts Code) follows the ISEAL Code of Good Practice for Setting Social and Environmental Standards; especially concerning stakeholder involvement and managing the consultation process.

Impacts Code Committees

The process to develop the Impacts Code began in September 2008 when ISEAL announced its intention to develop the Code; and called for nominations to the steering and two technical committees (Issues Committee & Methodologies Committee – see below).

Stakeholder Mapping

While the Impacts Code committees were being assembled, the stakeholder mapping process was being undertaken. This involves identifying any persons who might have an interest in the Impacts Code and creating a little database that shows their organisational interest (NGO, CB, Gov., industry and others) and the region of the world where they are located. The process is inexact, but it does provide us with an idea of what types of interests are being represented, and where they are located. Then (throughout the process) we can work to ensure there are more persons involved from the less represented groups and regions. As would be expected, the list is heavily weighted towards Europe and North America. Organisationally, the list is more evenly balanced between government, ISEAL members, researchers, other standards bodies, industry, donors, and consultants. There is an obvious lack of the voice of the producer and the consumer (though it is often pointed out that we are all consumers). Stakeholder mapping will continue throughout the Impacts Code project.

Research

Concurrent with the development of the committees and stakeholder mapping ISEAL commissioned Aimee Russillo to research the state of the art in impact measurement within standards systems. Aimee’s two reports “R082 ISEAL Key Impact Issues WHAT” and “R083 ISEAL Methodology HOW” are now available of the ISEAL impacts webpage.

What we’ve learned

Now that we have had an in-person meeting of all the committees (Steering Committee Dec 8 2008, Technical Committees Jan 27, 28 2009) we are getting an idea of stakeholders expectations, and how we might be able to meet those expectations within the Impacts Code. Here are some of the more salient observations:

• traditional impact assessment is not a useful model for standard systems, which are ongoing intereventions that change over time—therefore we are focusing on measuring our contribution to long-term impact;
• impact assessment must be integrated into an overall monitoring and evaluation system within a standard system;
• we need to develop common language and metrics for impact assessment;
• clarity on the goals of the standard systems is the necessary first step towards developing an ‘impact pathway’ model for impact assessment;
• the impact pathway will illustrate the causal chain of assumptions that lead from inputs to activities to outputs to outcomes, and finally towards our contribution to impact;

Stakeholders have provided us with some notions of what they want to see in the actual document:

• The Impacts Code should be affordable to implement;
• We need to explain the concepts in the code in very clear language;
• The code should be a prescriptive as possible but no more detail than needed;
• The code should include some guidance on the data management process.