Step 7: Successful advocacy is reflective and agile

Step 7 of the 7-Step advocacy method concerns Evaluation.

The first 6 steps of the 7-Step advocacy method have been concerned with preparation and action: How do you prepare to engage and then how do you best engage. In step 1 we discussed the importance of priorities and KPIs and in step 6 we looked at how you best manage all of your advocacy workstreams. The final step of our method is all about ensuring you have a structured approach and commitment towards evaluating and reviewing your advocacy campaigns at regular milestones and upon completion. You should always evaluate your work across the first 5 steps of the 7-step advocacy method. Successful advocacy is reflective and agile, and must monitor, evaluate and review constantly.

Step 7 of the 7-step advocacy method is without doubt the most often neglected in advocacy practice and certainly one of the most difficult to consistently apply. Whilst the essence of evaluation is quite straightforward – you need to evaluate in-depth what you did, how you did it, what happened, what impact you had and how successful you were. Actually, evaluating your advocacy often fails to happen in a regular, rigorous and systematic fashion. In practice, public affairs strategies tend to change iteratively without ever formally being subjected to rigorous and in-depth audit and evaluation. Indeed, formal full public affairs reviews, audits or evaluations are quite rare in the private sector and far from systematic amongst NGOs. The main reasons for this are time and money.

Reviews, audits or evaluations do not have to last weeks and cost significant amounts of money. At its most simple, it is about taking a day with some key internal stakeholders and taking stock of all internal processes, preparations, engagement and actions to critically evaluate and identify successes and failures. From an evaluation perspective there is a difference between preparation, engagement and action. Advocacy preparation lends itself to being audited. You can examine each step in the process and evaluate what is in place, what exists, what has been created and used. You should always strive for incremental improvements in your preparations. It is something you can control. For example:

• Improving the objective setting process;
• Tendering for monitoring and intelligence gathering to get better coverage or better value;
• Assessing the stock of assets, identifying gaps and filling them one by one.

Evaluating engagement and actions is more difficult because not everything is in your control and so the evaluation is more subjective. As we discussed in our blog on Step 1: Prioritization, (e.g. objective and priority setting) public affairs is an activity whose impact is difficult to both quantify and attribute to one action or organization. This is why SMART Objectives and KPIs are so valuable. They establish a sound basis for your ongoing monitoring and punctual review and evaluation.

It is crucial to have an open culture towards evaluation. It is not about blame or highlighting inadequacies for the sake of it. It is about learning, improving and delivering. Public affairs is a profession that faces continuously changing internal and external variables and where the recipe for success is rarely the same twice. The focus should be just as much internal to the organization (inadequate processes, acting too late or slowly, unclear priorities, inadequate data, lack of senior management engagement or buy-in) as external (political environment, ability to access key stakeholders, social media trends and reactions). The continuous improvement of your public affairs work is essential for your own personal development as much as it is for the success of your organization.

Evaluate, the final step of the 7-step advocacy method, can be broken into two parts:

1. Audit of Advocacy Method’s Preparatory Steps (steps 1 to 4)
2. Evaluation of Engagement (step 5)

We will look at each of them in turn.

Audit of Advocacy Method’s Preparatory Steps

The preparation phase (steps 1 to 4) of the advocacy process are easily audited because each step is distinct and can be subjected to effective check-listing of what needs to be in place and what needs to happen to be effective. This is much more objective than the engagement and action stage of the advocacy process. Each element can be approached as follows:

Monitoring and Intelligence Gathering (Step 2): An audit should ensure that you have the right tools to acquire the right information, at the right time, to inform your priorities, shape your strategy and deliver your objectives. Thus, for example, evaluate the following:

• Deadlines: Do you know all the deadlines in your process (to be able to respond)?
• Stakeholders: Do you know what all the key stakeholders were saying and doing (in order to know who to target and what to say)?
• Milestones: Did you miss any key moment, information, deadline, opportunity, conference or event?

Focus on the issue you are monitoring and gathering information on. Is the data and analysis correct and accurate? Are your intelligence needs met? Then evaluate your sources. Is the range sufficient, are they reliable, timely and easily translated into your intelligence analysis? Additionally, assess whether you are getting value for money from the resources you have put into this exercise.

Stakeholder Mapping (step 3): Focus on ensuring you have the right key stakeholder overviews in place to know who your key audiences are. Are they updated? Do you have a clear view of all the key people for your files? How up to date is your data?

Information Management (step 4): Take stock of all your advocacy assets. What you have used? How effectively have you used it? What feedback have your received? Do you have the right assets for your strategy? Were you missing anything? Did you include enough data to support your advocacy? Did you maintain a steady drumbeat of communications?

You should perform this type of evaluation audit at the least every 6 months as well as upon completion of your campaign.

Evaluation of Engagement

The engagement and action phase of your campaign (step 5) is more subjective when it comes to evaluating impact. You have less control over outcomes and policy change than over your actions and outputs. That said, you can evaluate three elements:

• Impact: This is a very direct measure and usually the most common one used. At its most obvious, did I get my proposed text into the final legislative text? Was my opinion reflected in the official position? Was the budget allocated as requested? Your evaluation seeks to evaluate direct cause and effect. You wanted A. Was A in the final outcome? Examine the milestone or final legislative text for your amendment wording or requirements. Compare initial texts with the final text. Warning: you cannot, and should not, attribute outcomes and impact solely to your actions. Only in rare instances can you demonstrate sole responsibility for an outcome. In the event that a policy reflects an unfavorable compromise (or limits damage), then evaluate your role in this outcome. There are often very good reasons why the outcome landed as it did, despite your best efforts.

• Perception: Evaluate how political decision-makers perceive your organization and your advocacy efforts. Questions to ask include: Did we clearly identify ourselves? Do key stakeholders know who we are? Who did we mobilize for an amendment? Were we better known (for the right reasons) at the end of the period than at the beginning of the period? Evaluation data can be based on your observations and any supporting documents (attendance at events, access to key people, statements etc.). However, budget allowing, stronger data can be acquired by conducting focus groups or a survey among key stakeholders. Explore how well your organization and your ideas were received. Ensure you have a broad range of indicators for perception. For example, how many calls did you receive from key stakeholders? How many invitations did you receive to participate in events or panels or meetings? How many meeting requests were accepted by your key stakeholders? Surveying perception may result in key stakeholder perception improving as a result of your openness to feedback.

• Relevance: The third key element of evaluation is to assess the relevance of your arguments and positions in the policy debate. Ask yourself: Were my arguments credible? Did I have the right data? Were my arguments understood by my contacts? Which arguments were taken up and why? Did aspects of my position find their way into the public debate or public view? Such an evaluation verifies whether your engagement was relevant and meaningful to the debate.

Beyond evaluating impact, perception and relevance you should evaluate individual elements of your engagement and actions. Check out our free Indicators and Measures Template (PDF) with a framework for measuring not only your Baseline, milestones and impact, but also examples of KPI for advocacy activities. Use your KPIs to evaluate:

• Direct engagement (i.e. access/uptake of positions)
• Media and social media campaigns (i.e. coverage/uptake and spread of positions)
• Events (i.e. key audience attendees/follow up actions)
• Trade Associations (i.e. objectives and KPIs met)
• Reports commissioned (i.e. coverage/uptake/contacts made)
• Consultants (i.e. objectives achieved/value-added)

Every individual engagement action you undertook represents an opportunity-cost moment. Evaluate how it worked and how you could have improved it, or replaced it with another activity. Opportunity-cost evaluation raises the question of when do you walk away from an activity or campaign? And how do you do walk away? This is a tough choice. For example, if you feel a trade association membership is not giving you what you need (i.e. directly contributing towards you achieving your objectives) you need to address that. Start by reiterating your needs. If this doesn’t help, step away. Every membership you hold and every event you run should add value, in a coordinated and non-duplicating way.  If it doesn’t, then you are wasting resource.

Another important area of evaluation learning is benchmarking yourself against organizations working in your space. Observe what others are doing. See what is and is not working. This is an excellent opportunity to adopt other people’s ideas, strategies and incorporate them into your own work. Learn from your peers through horizontal platforms such as the Public Affairs Council. They offer specific issue and horizontal public affairs webinars, discussions, conferences and roundtables. Take some time to sit back for a short period and learn.

We cannot stress enough just how important evaluation is to your advocacy work. Evaluation is not just a process. It is a way of thinking and acting. It is a culture that is open to successes and failures. Feedback works best when it is ongoing and when it motivates your team and demonstrates value to your organization. To ensure continual improvement our top recommendations for evaluation are:

1. Systematic: Take a systematic approach to evaluation. Use robust methods and define evaluation criteria (i.e. evaluate for relevance, effectiveness, efficiency, sustainability and impact). Be transparent and share the evaluation data and report. Check out our free Indicators and Measures Template (PDF) with a framework for measuring not only your baseline, milestones and impact, but also examples of KPI for advocacy activities.
2. 6-Monthly: Ensure you do a 6-monthly evaluation of your preparation steps as well as a review upon completion. Take out at least half a day with your team to do this.
3. Team: Incorporate evaluation-reflection into your monthly team meetings (or 1-1 discussions) to embed this quality culture thinking within your practice.
4. Scoreboard: Make sure you capture KPIs and evaluation data throughout the year in a place-form (i.e. a scoreboard) that you can access-use at a later stage for your evaluation.
5. Independent Evaluator: Consider an independent evaluator. If you need help getting set up or implementing your evaluation reach out to professionals who specialize in advocacy evaluation.

This is the final blog in the series of our 7-Step Method. We hope, through these blogs, to have outlined the benefits of breaking down advocacy into such a clearly defined method. We also hope to have identified the challenges, opportunities, key knowledge, skills, tools, and deliverables associated with each step. To further explore our 7-steps of advocacy method as well as our free advocacy tools, including our free advocacy assessment 7-step snapshot report, please visit: advocacystrategy.com