Mapping for Scientific Impact Assessments
At the end of 2018, CartONG was selected by the French Development Agency (AFD) to participate in the scientific impact evaluation (see the table below for more details) of the Program for the Rehabilitation and Integration of Housing Districts (PRIQH2 in French – see at the bottom of this article for more details on this program which is being implemented by the Tunisian State from 2019 to 2023), with the aim of adding a mapping dimension to the program’s evaluation methodology.
This initiative is a new approach not only for AFD but also for CartONG, who neither had had the opportunity to participate in a scientific impact evaluation nor in the evaluation of an urban development project.
What is a Scientific Impact Evaluation?
The scientific impact evaluation method comes from the field of medical sciences where the effects of taking a drug are compared to a “placebo” or “control” scenario. Applied to a project, the method consists of comparing, before and after the project, a group that benefited from the project and a “control” group that did not benefit from any intervention. The objective is therefore to identify measurable changes in both groups, and to determine whether the changes observed in the group that benefited from the project can be attributed to the project being evaluated – making it possible to establish whether there is a causal relationship between a project and one or more outcomes. This scientific method will make it possible to understand the possible areas for improvement in future programs for both AFD and the Tunisian Urban Rehabilitation and Renovation Agency (ARRU in French)n who implements the project for the Tunisian state.
The main component of scientific impact evaluations is the econometric impact assessment designed to quantify the impact of interventions on socio-economic outcomes at the household and individual level. A team of researchers from the United Nations University and the Tunisian firm BJKA Consulting is in charge of such aspect. They will conduct surveys to determine whether or not individuals have experienced changes in their living conditions, perceptions, confidence levels, aspirations, employment status, etc., after benefiting from the project.
As part of this specific scientific impact evaluation, AFD also wished to add a second component by conducting a spatial and territorial impact evaluation of the PRIQH2 project at the neighborhood level as well as the mapping of the evaluation results. CartONG is in charge of this additional component that should make it possible to identify global trends that cannot be observed at the level of individual respondents.
Find out how a scientific impact evaluation is carried out at AFD by visiting their website. Please also note that in the case of the PRIQH2 project, the “control neighborhoods” used as a reference during the scientific impact evaluation will benefit in a second phase of the project from the same interventions as those in the “rehabilitated districts” (see the rest of the article below for all details).
The benefits of a cartographic approach in the evaluation of an urban rehabilitation project
Participating in this impact evaluation is therefore a great challenge for our team, whose roadmap consists of:
- Consolidating a geographical database and developing mapping tools to analyze the impact of the program, as well as
- Strengthening the capacities of the local partner in GIS and promoting the appropriation of the tools developed within the framework of the evaluation in order to go beyond their primary goal – of being monitoring and evaluation tools – so they can also be used to facilitate decision-making and institutional dialogue.
In addition, CartONG has decided to go beyond the scope of this initial roadmap to mobilize as many open source resources as possible – particularly OpenStreetMap (OSM) – and to also to raise awareness among the various stakeholders regarding the challenges and opportunities linked to Open Data and ICT (New Information and Communications Technologies).
It has now been a little over 6 months since the exchanges with the team of researchers, the Tunisian Urban Rehabilitation and Renovation Agency (ARRU in French) and AFD began. We have already carried out two on-site missions – in January and June 2019 – which enabled us to reflect together on the impact evaluation protocol, i.e. in particular which results to measure, which indicators to use so as to compare the initial and the final situations, which districts to consider, etc.
Following these initial discussions, CartONG’s work will be structured around 4 main axes:
- Observing whether PRIQH2 promotes urban sprawl and the emergence of informal housing areas adjacent to the 20 selected neighborhoods (“10 rehabilitated neighborhoods” – beneficiaries of the interventions – and 10 “control neighborhoods”)
- Observing whether the improvement of basic infrastructures as well as the construction of community social facilities and business premises promotes, or not, the consolidation of the urban fabric within the 10 rehabilitated neighborhoods and if it improves, or not, their territorial integration.
- Analyzing the various survey results in relation to the position of households within the neighborhoods in order to identify whether the results are homogeneous or whether there are disparities within the housing blocks – this analysis taking into consideration the whole 20 neighborhoods.
- Analyzing the results of the surveys in relation to the distance of households from the different types of interventions in order to observe whether there are correlations between improved living conditions and the immediate proximity of these interventions within each of the 10 rehabilitated neighborhoods.
In concrete terms, how does CartONG contribute to this evaluation project?
CartONG brings its cartographic expertise around 2 main lines of action detailed below.
1- Consolidation of a geographical database and development of mapping tools to analyze the impact of the program
After having carried out a diagnosis of the data available at the disposal of ARRU and its partners in the first quarter of 2019, we began to consolidate a GIS database containing cartographic and statistical data – unfortunately quite limited in number and therefore difficult to use – that could be used to feed impact indicators.
We thus decided to complete this GIS database with vectorialized data that we will retrieve from or create directly into OSM based on the sample of the 20 observed neighborhoods. From satellite imagery, we will be able to map roads, buildings, land uses and the vegetative cover. If possible, we will then try to add points of interest (neighborhood facilities, streetlights, shops, etc.).
Following this step, we should be able to produce detailed maps of these 20 neighborhoods illustrating and characterizing the situation before the rehabilitation work begins; then 5 years later, after the program has been implemented, we will produce updated maps to illustrate and make the changes visible. These before/after maps will allow us to measure changes, for example in terms of density, footprint coefficient, orchard and field surface, etc. The statistical processing of these data and the large number of possible crossovers between these variables will allow for a diversity of analysis.
As this cartographic approach to impact evaluation remains relatively experimental, CartONG is unable to confirm yet that it will be possible to observe significant spatial changes from satellite imagery over the course of only 5 years. To anticipate these possible limitations of photo-imaging interpretation and maximize our analytical capabilities at the end of the project, we intend to encourage ARRU to increase its capacity to produce and collect geo-localized data in the coming years. It will also be important to ensure that they provide as much information as possible from the field (such as the height of the buildings, the presence of shops or the location of facilities, etc.).
2- Strengthening the GIS capacities of the local partner and facilitating the appropriation of the tools developed within the framework of the impact evaluation
The improvement of their GIS tool and the definition of a data model for the integration of the information necessary for the design of impact indicators has begun, and is being carried out in close collaboration with ARRU. Our work within the framework of this impact evaluation is also an opportunity to strengthen the GIS skills of the ARRU teams, and to illustrate how much GIS can be beneficial to the agency.
To this end, CartONG will offer a series of training courses over the next three years to define the strategic objective of their geographic information system and the procedures necessary for its proper functioning (i.e. including all phases: i) neighborhood identification phase, ii) analysis and planning phase iii) monitoring phase, and iv) phase of valorization of the programs).
This first project is therefore a very interesting opportunity for CartONG to explore the use of mapping for impact evaluation studies and to work with a local institutional partner over the long run. This will help us gain experience in supporting capacity building outside of the humanitarian sector. We hope that the priority given to taking into account the partner’s capacities and needs in the choice of tools and methods for this project (user-friendly and open source tools, open data & data sharing, etc.) will facilitate the sustainability of the implemented approach, and that it can be replicated elsewhere!
*****
Some background information on PRIQH2
Over the past three decades, Tunisia has experienced strong urban growth, making it one of the most urbanized countries in the Middle East and North Africa region. This phenomenon of rapid urbanization has led to the development of many precarious neighborhoods that face major urban problems such as the lack of basic services and facilities, and the multiplication of informal housing areas, which results in significant socio-economic disparities.
Following the Tunisian revolution of 2011 (during the Arab Spring), the State decided to step up its intervention in favor of such districts in order to improve the living conditions of inhabitants, to support the levers of local socio-economic development and to prevent the development of new informal districts. The government launched its first Program for the Rehabilitation and Integration of Housing Districts (PRIQH) in 2012, which benefited more than 860,000 inhabitants in 155 precarious neighborhoods throughout the country. A second phase of the PRIQH was launched in 2019 and is being financed by the Tunisian government, the Agence Française de Développement (AFD), the European Union and the European Investment Bank and is executed by the Agence de Réhabilitation et de Rénovation Urbaine (ARRU), a public operator under the supervision of the Tunisian Ministry of Equipment, Housing and Spatial Planning.
PRIQH2 is structured around four complementary components: 1) the development of basic infrastructures, 2) housing improvement, 3) the development of shared social facilities and business premises and 4) social housing developments near rehabilitated neighborhoods.
We encourage you to visit the AFD website to learn more: project page & article about the project.
