2. Why We Need Systemic Thinking
• My journey from small farmers to microcredit
to inclusive market systems…and back again
• Practitioners, evaluators and donors need
systemic thinking to meet new challenges:
– Measuring participation under facilitation
– Maintaining accountability with flexibility
– Evaluating sustainability in evolving systems
3. Beneficiaries and Boundaries
• Identifying participants
– Currently inconsistent
• Understanding spillover
– Good for impact
– Bad for evaluators?
• Relates to boundaries
of system
• 3 types of participants
– direct, indirect, imitators Source: Outreach, Outcomes and Sustainability
in Value Chain Projects by Creevey et al., Sept.
2011, USAID AMAP microREPORT #171.
4. Credibility and Accountability
• Issue: Systems change
and projects must adapt
• Accountability to donors
– Not going away
– But targets can constrain
project effectiveness
• Credibility of evidence
– Baselines, control groups
and attribution
• Causal modeling
– Useful and essential
– Should be flexible
5. Sustainability as Systemic Change
• Markets as systems
– Value chain as network of
firms/actors relationships
• Sustainability as systemic
change
– New, better relationships
– Learning and adaptation
– Broadening of benefits
• Sustainability as an
emergent property
6. Evaluation Challenges
1. Adopt consistent and comprehensive
measures of participation under facilitation.
2. Agree with donors on ways to demonstrate
accountability under flexible interventions.
3. Adapt causal modeling to guide practice and
to evaluate evolving systems.
4. Advance knowledge of sustainability through
meaningful indicators of systemic change.
7. Measuring Impacts
in Market Systems:
Rethinking the Current Paradigm
Dr Shamim Bodhanya
Academic Leader: Higher Degrees and Research
, Graduate School of Business and Leadership, University of KwaZulu-Natal
Chairperson – Institute of Natural Resources
South Africa
Tel: +27 31 260 1493
Email: bodhanyas1@ukzn.ac.za
http://www.linkedin.com/in/shamimbodhanya
8. Bounded Rationality
“The capacity of the human mind for
formulating and solving complex problems is
very small compared with the size of the
problem whose solution is required for
objectively, rational behaviour in the real world
or even for a reasonable approximation to such
objective rationality.”
(Simon, 1957, p 198)
10. Wicked problems (mess)
• Unbounded
• Ill-defined
• Multiple, conflicting goals
• Goals may also be ill-defined
• Multiple perspectives, values
11. Complex Adaptive System
“A complex adaptive system (CAS) is a
system comprised of heterogeneous
agents that interact locally with each
other based on local schema, such that
the behavior of the system arises as a
result of feedback relationships
between the agents, and the system
evolves as the schemata of the agents
adapt based on the feedback.”
Bodhanya, 2008
12. Complex Adaptive System
“A complex adaptive system (CAS) is a
system comprised of heterogeneous
agents that interact locally with each
other based on local schema, such that
the behavior of the system arises as a
result of feedback relationships
between the agents, and the system
evolves as the schemata of the agents
adapt based on the feedback.”
Bodhanya, 2008
17. Rethinking the paradigm
• Social systems are complex adaptive systems
• Radically challenges our worldviews
• Flat earth versus spherical earth
• Ability to predict and control
• What does this mean for human agency and
volition?
• Changes our entire conception of planning
• Societal change
• Markets
This does not in anyway imply that
human actors must be fatalistic.
24. Towards a way forward
• We are on a journey
• Context – Contingent on local conditions
• Theoretical frameworks <-> Social Reality
• Learning systems
• Models –
– Soft models
– Narratives / Metaphor
• Language is world constituting
• Strategic conversations
• Facilitation
• What is measurement?
• Agency in measurement tools
• Artifacts
• Generative relationships
26. Simplifying complexity
through systems thinking
Panel: Measuring impact in market
systems (November 7th 2012)
Richard Hummelbrunner
ÖAR Regionalberatung
Graz, Austria
27. Systemic or Systematic?
Systemic
focus on the whole
and the parts
Three core dimensions:
Interrelationships
Perspectives
Boundaries
Systematic
focus on the parts,
step-by-step
=
+
29. Consequences for monitoring
• Regard interventions as social systems
Unit of observation: intervention and context
Observe relevant contextual factors (scanning) during
implementation, in particular relevant actions of others
Look beyond intended routes and effects, avoid tunnel
view, capture broader range of effects (irrespective of
intentions)
• Different approach towards deviations from plan
Do not per se regard as negative (‘correction reflex’)
Do not treat as isolated phenomena, but connect with
intervention logic
Information to understand the internal dynamics and self-
organising forces at work within target social system
30. Linear or ‚circular‘ logic models ?
Inputs Outputs Results
Mechanisms
Context
Needs /
Problems
Issues
Impact
32. Measuring Impacts
in Market Systems:
Rethinking the Current Paradigm
Dr Shamim Bodhanya
Academic Leader: Higher Degrees and Research
, Graduate School of Business and Leadership, University of KwaZulu-Natal
Chairperson – Institute of Natural Resources
South Africa
Tel: +27 31 260 1493
Email: bodhanyas1@ukzn.ac.za
http://www.linkedin.com/in/shamimbodhanya
33. 2nd economy
MANY FARMERS
•Simple technology
•Small markets (mainly family
& neighbours)
•Unprocessed goods
Communal
land tenure
1st economy $$
FEW FARMERS
•Highly mechanised
•Involves agro-processing
•Large markets (diverse
consumers and products)
entrepreneurs
corporates
Freehold
tenure
NB. Markets:
•Land
•Finance
•Physical
•Labour
Communal
ownership L
Land – ownership
entity
Keeping land
productive
80/20
- 20% of people
produce 80% of
the food
Infrastructure
Private sector:
•Primary production
•Transport
•Processing
Organised
agric. /
National government
Policy & programmes
DEA, DWA, DAFF
NAFU
NERPO
Commodity groups
(SASA, FSA, Grain
SA, etc)
Provincial
& local
govt
Policy
Farmers
Financial orgs:
•Landbank
•Private banks
•Donors
•Microfinance
WATER
GLOBAL WARMING Increasing prices:
•Fuel (oil) – supply too
•Fertilizer
MAFISA
•Landbank
•DAFF
•DFIs
Cheap imports
Tariff protection
WTO
Education & training
Research
34. Issues
• Practitioners have a good sense of real world
complexity
• Overly formalistic tools
• Attempt to straightjacket that experience into
inappropriate tools
• Unintended consequences
• Multiple perspectives
• Policy resistance
• Boundary judgments
• Bounded rationality
35. Systemic change
• Events – Pattern - Structure
• Structure
• Relationships between actors
• Mental models
• Incentives
• Information flows
• Changing feedback loops
• Strength of the loops
36. Considerations
• Systemic M&E is conceptually challenging
• ...yet we must stay rooted to practice
• Rigour and credibility without overly
mechanistic, formalistic approaches to M&E
• Donor centric – How to get accountability
while shifting the centre of gravity to change
on the ground and systemic change
42. Laws of the Fifth Discipline (Senge)
• Today’s problems come from yesterday’s solutions
• The harder you push the harder the system pushes back (policy resistance)
• Behaviour grows better before it grows worse
• The easy way out usually leads back in
• The cure can be worse than the disease
• Faster is slower
• Cause and effect not closely related in time and space
• Small changes can produce big results – but the areas of highest leverage
are often the least obvious
• You can have your cake and eat it too – but not at once
• Dividing an elephant in half does not produce two small elephants
• There is no blame
43. Characteristics of Complex Adaptive
Systems
• Agents with schemata
• Emergence
• Self-organisation
• Sensitive dependence on initial conditions
• History
• Path dependence
• Far-from-equilibrium
• Co-evolution
• Fitness Landscapes
• Edge of chaos
• Artifacts
• Persistence
• Egalitarianism
44. Simplifying complexity
through systems thinking
Panel: Measuring impact in market
systems (November 7th 2012)
Richard Hummelbrunner
ÖAR Regionalberatung
Graz, Austria
45. • Replace impacts chains with configurations/networks, permitting to
Link elements at the same level
Connect different levels (e.g. Outputs - Results)
Capture reciprocal or feed-back relationships
Allocate activities or assumptions with effects
Show different strategy options, impact pathways
Identify leverage points for interventions
• More refined modelling (if appropriate/useful)
E.g. Influence or Multiple Cause Diagrammes (all or partly)
Causal Loop Diagrammes (identify Feedback Loops)
Represent qualitative features (z.B. intensity, duration, delays)
Consequences of a systemic view of effects
46. Example of outcome configuration
(Source: New GIZ Impact Model)
Outcome
Outcome
Outcome
Outcome
Outcome
Outcome
Outcome
Outcome
Outcome
Outcome
Outcome
47. Example of outcome configuration
(Source: New GIZ Impact Model)
Outcome
Outcome
Outcome
Outcome
Outcome
Outcome
Outcome
Objective
Outcome
Outcome
Outcome
Outcome
48. Example of outcome configuration
(Source: New GIZ Impact Model)
Outcome
Outcome
Outcome
Outcome
Outcome
Outcome
Outcome
Outcome
Objective
Outcome
Outcome
Outcome
49. Challenges for capturing effects
in complicated and complex situations
• Main challenges
Contribution to objectives through many factors / outputs
Difficult to establish clear causalities and relationships
between outputs and further effects (results, impacts)
Temptation to attribute effects irrespective of contribution
Impact not appropriate to hold program actors accountable
• Limited utility of many monitoring systems
Predominant focus on inputs or outputs (easy to capture)
Predominant use of quantitative indicators (easy to
measure, capture only narrow part of reality)
Information on result / impact indicators comes often (too)
late to change course during implementation
50. Lines of influence in a program
(example: EU Structural Fund - Programs)
High
Low
INPUTS OUTPUTS RESULTS IMPACTS
Project owners,
partners, External
factors
Programme Actors
Funding conditions
Project development
Project selection
Observe implementation
of projects
Influence actions of others
Logical Framework RBM
Outcome
Mapping
51. The ‚Process Monitoring of Impacts‘ approach
Theory-based monitoring approach making use of logic models
Focus on processes, which should lead to results / impacts
Logic models are considered as hypotheses (to be modified
during implementation), not as ‘blue-prints
Perspectives and observations of various stakeholders are
captured and reflected, applying use of systemic methods
Core rationale of the approach
Provide information for programme actors as early as
possible on the likeliness of achieving results/impacts
Particular emphasis on domains that can be influenced by
them or for which they are responsible.
52. Monitoring of change processes
Basic assumptions for change (along a result chain)
Inputs are used to
produce outputs
(= projects)
Outputs are used
(by someone, in a
specific manner)
to reach results
Results will lead to
(expected) impacts
in a plausible
manner
Inputs Outputs Results Impacts
53. External expertise,
Process consulting
Services to sensitize for
innovations
Enterprises (esp. SME)
collaborate in networks
(also with large
enterprises)
Enterprises (esp. SME)
carry out reorganisation
processes
Enterprises (esp. SME)
carry out product finding
processes
Enterprises (esp. SME)
introduce new
technologies
Enterprises (esp. SME)
gain new markets
Enterprises (esp. SME)
conceive innovation-/
investment projects
Increased linkages between
enterprises at regional level/
scale
Sustainable stabilisation of
enterprises
New/ improved services,
products and production
processes
Adaption to international
competition
Increase in employment/
new jobs
New contacts with clients,
new orders
External expertise,
Process consulting
Services to sensitize for
innovations
Enterprises (esp. SME)
carry out reorganisation
processes
Enterprises (esp. SME)
carry out product finding
processes
Enterprises (esp. SME)
introduce new
technologies
Enterprises (esp. SME)
gain new markets
Enterprises (esp. SME)
prepare innovation-/
investment projects
Sustainable stabilisation of
enterprises
New/ improved services,
products and production
processes
Adaption to international
competition
Increase in employment/
new jobs
New contacts with clients,
new orders
Advise for co-operation
I
I
I
OUTPUT (TYPES) USE of OUTPUTS RESULT IMPACT
I Quantifiable indicator
Example of Logic model (Enterprise support Scheme)
54. Result Based Management (RBM)
Emerging tendencies
• Shift focus of performance information
from outputs (goods and services produced) to outcomes (benefits)
• Set performance expectations for outcomes
Clarify conceptional issues (function, purpose and location of targets)
• Different approach to accountability
Influencing outcomes (not achieving them)
• Assess contributions to outcomes (instead of claiming attribution)
Take account of other contributing factors, gestation period of outputs
• Beware of straightforward links between performance and budgets/costs
Managing for outcomes requires authority for managers to do so,
i.e. more flexibility on activities, resources and outputs
55. Adapting Logframes to deal with complexity:
Differentiate effects in line with situation
• Categorize outputs via portfolio matrix:
locate in one of the three domains: simple, complicated, complex
has implications for completing other elements of logframe
• If outputs predominantly lie in the ‘complicated’ domain:
carefully identify indicators and assumptions to enable
monitoring of unfolding practice, relevant factors and context
conditions
• If many (or majority of) outputs are considered to be ‘complex:
Identify indicators that allow documenting initial conditions and -
in combination with assumptions - capturing emerging
phenomena
Not all interventions (or parts thereof) are treated as ‘simple’!
56. Output Portfolio
DEGREE OF CERTAINTY ( About what to do)
DEGREE
OF
AGREEMENT
(Between
Stakeholders)
High agreement Some disagreement High disagreement
High
agreement
Some
disagreement
High
disagreement
57. Types of indicators
Output
Type of
Indicators
Lagging
Coincident
Leading
Time
Example
Services delivered by Centre
Service Centre established
Building permits obtained,
Work contracted