There is broad consensus that—from a global perspective—the excessive use of natural resources by human beings (in particular the resource consumption of the richer countries) has manifold consequences [1, 2]. The anthropogenic overexploitation of natural resources results, for example, in (i) shortages of resources, such as oil, coal, sand, or rock phosphate [3], (ii) climate change [4], or (iii) decrease of soil fertility [5]. By various means, we are crossing the planetary boundaries [6]. With respect to biogeochemical flows and genetic diversity, the planetary boundaries are stressed to such an extent that we risk non-reversible changes of the environment and even the habitability of the earth. Agriculture, in particular—respectively the production and use of synthetic fertilizers, pesticides, or seeds—contributes to this risk [7]. How, therefore, can we change the situation? Or, more specifically: How can we meet the (human) demand for food and energy without overexploiting our planet?

Recognized recommendations for a sustainable use of resources include, for example [2, 8, 9, 10]: (i) establishing circular economies; (ii) using effective, efficient, adapted, and affordable technologies; and (iii) utilizing locally available resources for soil fertility management. Measures to establish circular economies comprise inter alia (i) the concept of bioenergy, hence the provision and use of energy from biomass [11], respectively renewable resources or organic waste materials, and (ii) the implementation of ecological sanitation (EcoSan). With the latter, various material flows are collected separately, treated specifically, and, if possible, ultimately recycled to agricultural land [12]. Both topics, bioenergy and EcoSan, are well-studied and have already been brought into practical applications.

Against this backdrop, and given that over 50% of global food products are produced by small-scale farmers [13], the sustainability potentials of smallholder communities in the Global South is of major relevance. In the Northwest of Tanzania (TZ), a representative region for Sub-Saharan Africa (SSA), the main energy carrier is firewood, mostly used for cooking on a three-stone fire, whilst pit latrines are the most common sanitation approach [14]. This pattern, alongside a steady population growth, increasingly stresses natural resources such as arable land, aquifers or other water sources, and forests [15, 16].

To counteract local deforestation and soil nutrient depletion, two Tanzanian farmers initiatives have started three projects located in Karagwe, Kagera region, together with German partners and donors [17]. The main objective of the technological development cooperation was to contribute to sustainable community development by adapting and introducing small-scale and locally adequate cooking and sanitation technologies [18]. At the same time, a circular resource economy shall be implemented to improve local soils and food production. Therefore, residues from cooking and sanitation are used to increase the recycling of nutrients and the recovery of carbon for restoring soil humus.

Previous work covers how these locally adapted technologies in smallholder households contribute to environmental health by reducing deforestation, climate gas emissions, and nutrient leaching to aquifers [19]. We further showed that using residues from cooking and sanitation could stimulate the productivity of smallholder farming by improving the acidic and nutrient-depleted local soil [20, 21]. Subsequently, we widened our perspective to evaluate the technologies studied according to multiple sustainability dimensions.

The challenge of any sustainability assessment [22] is to translate defined and basically agreed upon principles into an operational model [23]. However, even though measurable criteria can be applied to operationalize sustainability, it is individuals or societies who evaluate the plausibility and credibility of relevant assessments. If the process involves multiple actors, differing opinions may also cause trade-offs between stakeholders. Multi-Criteria Decision Analysis (MCDA) attempts to address this challenge [24, 25, 26]. In the first instance, any assessment of alternatives against multiple criteria, irrespective of who participates in the analysis, can generally be called a Multi-Criteria Analysis (MCA) [27, 28]. Moreover, MCDA indicates a specific MCA, addressing participatory decision-making processes [29]. Both approaches, MCA and MCDA, are applied in planning, decision, and evaluation processes and to assess likely consequences of decisions [30]. In the following, we use the abbreviation MC(D)A when referring to characteristics of both approaches.

According to [31], the MC(D)A is generally suitable for decisions related to sustainable development because the method is largely transparent, participatory, and inter-disciplinary. Furthermore, the MC(D)A is often applied in environmental decision making [32, 33] and strategic planning for natural resources management [34]. While the number of MC(D)A applications has clearly grown over the last two decades, the method was mainly applied in scientific communities in the Global North [35]. Among the view examples of MC(D)A-studies from SSA are (i) comparisons of different sanitation approaches in Uganda [36] and in Burkina Faso [35], (ii) an assessment of cook stoves in TZ [37], and (iii) an evaluation of appropriate farming techniques in TZ [38].

With this study, we want to contribute to advancing the applicability of MC(D)A in the regional context of SSA as well as in the topical context of technology-driven sustainable community development in the Global South and Global North alike.

Based on the MC(D)A methodology, our aim is (i) to develop and propose an integrated method for a participatory sustainability assessment, (ii) to translate into an applicable tool, and (iii) to test the tool with selected stakeholders (pre-testing). Our specific focus addresses small-scale cooking and sanitation technologies alongside the use of residues in smallholder farming, and on the example of Tanzanian case studies.

The specific research objectives are: (i) to identify a set of criteria and applicable methods within the framework of MC(D)A; (ii) to develop and propose a handy tool for an ex-ante, participatory, and multi-perspective technology assessment; (iii) to conduct a pre-testing of the tool and assess the sustainability of selected technologies.

The research questions are as follows: (RQ1) What are the most relevant criteria and applicable methods for assessing sustainability of small-scale cooking and sanitation technologies? (RQ2) How did the stakeholders involved in pre-testing rated the technologies at hand with regard to relevant criteria and to overall sustainability? (RQ3) How proved specific stakeholder groups perceptions? The RQ1 thereby refers specifically to the methodological development of the tool whilst RQ2 and RQ3 refer to pre-testing of the tool.

The study area is the Karagwe district, in the Kagera region (lat. 01 ∘ 33’ S; long. 31 ∘ 07’ E; alt. 1500-1600 m.a.s.l.). Kagera forms a part of the Lake Victoria basin and is located about 1,500 km away from the Tanzanian capital Dar Es Salaam. The regional economy is dominated by smallholder agriculture with about 90% of households trading agricultural products grown on their farms [14].

Three Karagwe projects serve as our case studies: (i) Biogas Support for Tanzania (BiogaST), which aims to implement small-scale biogas digesters to use bioenergy from harvest and kitchen residues for cooking; (ii) Efficient Cooking in Tanzania (EfCoiTa), which disseminates advanced designs of improved cook stoves (ICS), such as microgasifier cook stoves; and (iii) Carbonization and Sanitation (CaSa), which works with EcoSan, including a urine-diverting dry toilet (UDDT), thermal sanitation of faeces, and composting of human excreta mixed with other organic wastes including biochar. The latter material is a residue from cooking with microgasifiers. Cooking and sanitation are connected through the use of residues for soil fertility management, which makes the energy-sanitation-agriculture nexus even more interesting to study. In prior publications, we introduced the case studies [17] and described the technologies [19]. All technologies shall be implemented and used in both, households and institutions (e.g. in schools or hospitals). The local non-governmental organisation (NGO) Mavuno Project - Improvement for Community Relief and Services (MAVUNO) facilitates the BiogaST- and CaSa-projects. The Programme for Community Habitat Environmental Management (CHEMA) runs the EfCoiTa-project. German project partners are Engineers Without Borders (EWB) Berlin/Germany and Technische Universität (TU) Berlin. Financing of the projects is provided by German foundations BayWa (BiogaST) and Heidehof (CaSa and EfCoiTa). After completing pilot studies, a strategy and a plan for implementing technologies in the smallholder community needs to be developed. Hence, the respective technologies are the subject of decisions at hand. Decision makers are either the NGOs, implementing the technologies on a project-basis, or single households, purchasing them, for example, from local manufacturers or at local markets.

The method shall: (i) enable a systematic analysis; (ii) consider multiple perspectives related to sustainability; (iii) involve local communities and authorities in the assessment; (iv) integrate available data; and (iv) be conducted ahead of the technologies implementation in the community. Against this backdrop, we chose to follow the systematic concept of MC(D)A.

According to [34], MC(D)A consists of three pillars, which are: (i) the presence of multiple criteria; (ii) the formal approach with a set of analytical methods; and (iii) the involvement of individuals or groups of individuals in the assessment (for MCA) or in the decision process (for MCDA). The MC(D)A further combines a range of methodologies to use quantitative and qualitative data for (i) measuring sets of criteria, (ii) considering consequences of decisions, and for (iii) sequencing alternatives [29]. Applied MC(D)As often combine (i) qualitative methods for problem structuring, (ii) quantitative methods for problem analysis, and (iii) soft methods for stakeholder participation [34]. Aggregation of an assessment is possible from the level of different criteria into an overall performance of alternatives and, likewise, from the level of individual preferences into a common, average preference [29] (For the interested reader, a summary of the fundamental terms used in MC(D)A can be found in Table A.1). Based on a comprehensive literature review, especially [23, 33, 39, 40, 41, 42, 43] inspired the design of our MC(D)A approach. The dynamic process of developing the specific design of our method and tool will be described in the following paragraphs (with more details provided in Appendix Section A.2).

(A) Framing the context: Formulating the decision problem involved learning about the context of the decision, such as livelihood of the community, specific approaches of the case studies, and ways of using technologies in daily life. The main activities, therefore, included: participating in projects; short- and long-term stays in Karagwe; teamwork with project workers; reading project reports, governmental reports, and non-governmental reports; and communicating with scientists and practitioners in the region. Based on this, we described the environment of the decision including driving forces (Table 1) and motivations (Table 2) that represent the objectives of the projects initiators to develop and implement ‘new’ technologies in the Karagwe smallholder community.

In addition, two Process Flow Diagrams (PFD) foster a better understanding of the technologies and possible nutrient recycling approaches when interacting with people. Both PFDs indicate how the technologies analysed are integrated in on-farm resource management and how they interact with the natural environment in terms of resource consumption and emissions. One PFD is a more systematic illustration (Figure 1); the other PFD is a more pictorial illustration (Figure A.1).

(B) Creating alternatives: Alternatives to be analysed are either (i) discrete technologies or (ii) a mix of technologies [39]. In the present study, we decided to analyse discrete technologies that are defined by the case studies (Table 3; cf. Tables A.3–A.5 for pictures and descriptions). Currently, the most common combination of technologies for cooking and sanitation found in Karagwe smallholdings is a three-stone fire and pit latrine [14]. The alternatives assessed are further technologies that are locally available (i.e. technically and commercially) for implementation. We compared (i) technologies represented in the case study projects, and (ii) other widespread alternatives. Regarding the first point, cooking alternatives include: a system composed of a biogas digester and a biogas burner, developed and adapted to Karagwe conditions by the BiogaST-project; and two types of ICSs promoted by the EfCoiTa-project, such as the rocket stove and the microgasifier. MAVUNO endorses implementation of biogas systems at households through development cooperation projects whilst CHEMA produces and disseminates rocket stoves and microgasifiers at their workshop. Sanitation alternatives include to ecological approaches, namely, “EcoSan” that refers to using an UDDT and “CaSa” that refers to using the UDDT in combination with a sanitation oven for thermal treatment of excreta pursuant to the practices of the CaSa-project. MAVUNO is engaged in testing and promoting both approaches. Regarding the second point, charcoal stoves (also called “charcoal burners”) are commonly available in Karagwe on markets and from local stove sellers and a combination of a flush toilet and septic tank can be accessed through local plumbers. The charcoal and biogas alternative also include the charcoal production and biogas digester. These processes provide the energy carrier used in households. Rocket or microgasifier stoves, in comparison, make direct use of firewood or sawdust. The latter is an available waste resource in many anthropogenic settings.

(C) Selecting criteria: In order to operationalize sustainability, relevant and feasible, thus appropriate, criteria need to be identified to describe the multiple dimensions. According to MC(D)A methodology, main-criteria describe the objectives, or highest-level criteria, and sub-criteria describe the attributes, or lowest-level criteria [29, 32]. To identify such criteria, we performed a comprehensive literature review. Our focus was specifically on comparable work conducted (i) for topics such as bioenergy, EcoSan, and soil fertility or natural resource management, and (ii) in a regional context of TZ, East-Africa, or SSA. We further collaborated with practitioners, academic professionals, and other experts in bioenergy and/or sanitation in an East-African-context. Activities and methods applied include: (i) semi-structured interviews conducted in TZ and Uganda; (ii) group discussions based on the world café method [45] in TZ and Germany; and (iii) discussion with the coordinator of BiogaST and CaSa projects (A. Bitakwate) to conclude a final set of criteria.

(D) Collecting data: To describe how the alternatives performed against the criteria we needed to collect relevant data. Most data used derive from prior studies related to the case studies [17, 19, 20, 21]. Additional data originate from project documents of the pilot studies, judgements of local expert, and literature as well as internet reviews. Descriptions provided to participants also include information about data sources and the estimated quality of data (cf. Table A.9).

(E) Analysing stakeholders and selecting participants: Participants of a MC(D)A can be split into two groups: (i) the planner; and (ii) stakeholder representatives [40]. The planner facilitates the whole MC(D)A and, therefore, pre-selects criteria, prepares the tool and descriptions of the alternatives beforehand, and moderates the assessment process. Stakeholder representatives participate, in particular, in conducting the weighting of criteria and the scoring of alternatives. According to [23], relevant stakeholders that should be considered when conducting a participatory MC(D)A include: (i) funding agencies; (ii) governmental authorities; (iii) specialists (e.g. stove technicians, EcoSan-experts, or strategic advisors); (iv) (future) staff for operation and maintenance; (v) end-users of the technologies and/or of the by-products; (vi) legislator and enforcement agencies; (vii) research institutions; (viii) local, national and international NGOs; and (ix) site residents (if any). In our case, local smallholders are probably the stakeholders being most affected by the technologies assessed.

(F) Preparing the assessment tool: The following requirements had been defined for the assessment tool: (i) free of charge; (ii) intuitive to use; (iii) transparent (e.g. that it is possible to see and understand also the computational model); and (iv) possible to use off-line for participants because internet access is not always reliable in Karagwe. Unfortunately, most available commercial or free software was not appropriate for several reasons. Most of them were either (i) too cost-intensive, such as V ⋅ I ⋅ S ⋅ A, (ii) too complex, such as PROMETHEE I because visualization is too scientific, or (iii) working only online, such as Decisionarium or Web-HIPRE.

As a consequence, we developed a hand-made and Excel-based assessment tool. The tool is tailored to our specific application and based on a template of [23]. In total, the designed tool comprises sheets for (i) providing information to participants, (ii) collecting judgments of participants during weighting and scoring, and (iii) calculating, summarizing, and visualizing results. The latter include so-called Performance Matrices (PM) that summarize weights, scores, and results for each participant and all alternatives. To elicit individual preferences from participants, we selected applicable methods for participatory weighting and scoring from [29, 32, 39]. The weighting further comprises ranking and rating of main- and sub-criteria. Ranking is the ordering of criteria according to individually perceived importance. Rating means assigning individual numeric weights as points ranging from 0 to 100 to each criterion in order to differentiate between the criteria.

In order to test the developed method, we performed a pre-testing of the tool with a selected group of participants. Subjects of the pre-testing are the alternatives as described above. However, the technologies currently most used are not assessed in order to keep the total amount of alternatives manageable.

Against the backdrop in which there is a significant language gap between Tanzanian farmers and German participants, who mostly do not speak Swahili, we considered direct participation of farmers as not feasible. Furthermore, and pursuant to [46], farmers may hesitate to freely express their thoughts if white researchers and/or donors are present during the assessment. Overall, stakeholders participating in pre-testing the MC(D)A tool include: (i) the facilitating NGOs, represented by the management, project coordinators, and other staff members; (ii) the cooperation partners, represented by researchers, volunteers, and donor agents. In total, 10 people committed their participation and the number of participants in each group is, respectively, six (four of MAVUNO and two of CHEMA) and four (four scientists from TU; three members of EWB; and one member of the advisory board from Heidehof foundation; including double affiliations). Planner and moderator was A. Krause. In the pre-testing, participating staff members of local NGOs also act as representatives of the local smallholders. (The tool itself should, however, support farmers in decision-making within the smallholder community.)

Pre-testing followed a “9-steps-approach”, where the actual assessment of alternatives is conducted in a stepwise and participatory procedure, including the following nine steps:

Unfortunately, due to time constraints and because participants had not been in one place, rather located in both countries, Tanzania and Germany, discussion and adjustments of scoring as part of step 8, was not possible. Planner/facilitator and participants communicated in English and mainly via e-mail or by using Skype or phone calls on demand, for example to clarify questions. Presentations were shared as PDF-documents and via file hosting service. (More details provided in Appendix Section A.3).

In order to systematically assess the sustainability of small-scale cooking and sanitation technologies, we propose a tool called the Multi-Criteria Technology Assessment (MCTA). We designed the MCTA by adapting MC(D)A and developing a tool, which is adequate for the sustainability assessment of small-scale cooking and sanitation technologies. The process included activities as described in Section 2.2. Results with respect to the identification of most relevant criteria and applicable methods for assessing the sustainability of small-scale cooking and sanitation technologies (RQ1) are presented and discussed in the following paragraph.

Selection of criteria: [-10]To define the main-criteria considered in the MCTA, we merged sustainability categories of [47] with aspects from the Integrated Sustainable Waste Management approach of [48] that describes “the enabling environment of sustainability”. Finally, we chose the following main-criteria as being most relevant: technological-operational, environmental, health/hygiene, socio-cultural, socio-economic/financial, and political/legal criteria. Pursuant to [23], we use a six-pointed so-called “sustainability star” (Figure 2) to visualise how we define sustainability in the MCTA. Each of the six dimensions represents one of the six main-criteria and thus reflects an objective of sustainability that is addressed in the analysis of cooking and sanitation technologies. Furthermore, the categories “acceptability”, “affordability”, and “reliability” are assigned to the main-criteria. Derived from Sustainable Development Goal no. 7 of the United Nations, these indicate characteristics, which feature a sustainable use of energy technologies [49].

The final selection of sub-criteria (Table A.8) constitutes a synthesis of criteria from [23, 36, 37, 50] (cf. Table A.6). From a total of 84 sub-criteria considered in the MCTA, 80 and 75 are applied for assessing cooking and sanitation technologies, respectively (Table A.7).

Overall, we consider the chosen criteria as significant to systematically identify strengths and weaknesses of technologies and to assess their sustainability. However, criteria are not thoroughly independent from each other. Some criteria describe different objectives even though they are related to each other. For example, two of the technological-operational criteria are “preferably high use of locally available resources” and “preferably low use of industrial resources”. Both sub-criteria are related to the socio-economic/financial criterion “preferably low cost for implementation” because the material use influences production costs. Another reason for preferring a substantial use of locally available resources is, however, a higher degree of independence from international markets (and main providers of industrial materials). To sum up, most of the sub-criteria applied are intermediate or so-called “means-to-an-end” objectives rather than fundamental objectives [39]. Nevertheless, according to [32] this approach is not contradicting the MC(D)A concept.

Collection of data: Overall, our study integrates quantitative data of recent scientific findings and explorative practitioners’ experiences with qualitative data collected from literature, oral explanations by farmers, learning from participating in the projects, and expert judgements. Quantitative data of prior studies, where we employed Material Flow Analysis (MFA) [19] also in combination with Soil Nutrient Balancing (SNB) [21], was particularly important to describe environmental attributes. The MFA identifies and quantifies sources, sinks, directions, magnitudes, connections, dependencies, and shifts of material flows between the anthroposphere and the environment [51]. The SNB specifically measures, calculates, and balances various input and output flows of nutrients to and from agricultural land [52].

Against the backdrop that reaching environmental sustainability requires a systematic reduction of the physical degradation of nature [53], we consider MFA and SNB as highly suitable methods to be integrated in a multi-criteria sustainability assessment. By combining qualitative and quantitative methods for data collection, the MCTA realises an integrated approach as recommended by [24].

Data collection, however, was challenging. Available and accessible data in the given context was not sufficient to estimate and to describe performances of most technologies. In order to generate a profound basis for the MCTA, we needed significant additional efforts and approximately two years for conducting performance tests of the technologies, elaborating a field trial and MFA and SNB modelling. Hence, describing the performance of the alternatives was challenging and time-consuming, which resulted in certain shortcomings of the pre-test.

Calculations: The computational model applied in the MCTA is set up through combining the Multi-Attribute Value Theory (MAVT) and Simple Additive Weighting (SAW) out of a wide range of existing MC(D)A methods [29, 34]. The MAVT basically uses multiple attributes, for comparing alternatives according to their value. SAW describes the mathematical approach to determine such values through systematic aggregation.

Computations pursue the following process: Firstly, ranking and rating of main-criteria is done by applying the SWING-method [29]; sub-criteria are then simply rated. From the numeric weights (i.e. points ranging from 0 to 100) given by participants during rating, the tool supports the planner in determining the relative weights of sub- and main-criteria (Equations A.1–A.5). Subsequently, scoring is conducted for each sub-criterion and each alternative separately. Participants therefore assign numeric scores in points ranging from - 10 to +10 (Table 4) that should reflect their individually perceived performance of an alternative and the associated value. In the MCTA, participants also have the option to waive scoring, by indicating a * instead of assigning points, if they consider available information as insufficient and/or feel unsure about scoring. In offering this *-option, we attempt to not force participants into an arbitrary judgement.

Pursuant to the SAW, the assigned scores are firstly weighted on the lowest level of single attributes. The tool, therefore, determines the weighted score for each sub-criterion as the product of (calculated) relative weight and (assigned) score (Equation A.6). Computations are adapted for the *-case as described in the Appendix (Equations A.4 and A.6). Subsequently, plain addition determines the weighted scores for each main-criterion (Equation A.7). To deduce the overall value of an alternative, weighted scores are then further aggregated to result in a final Sustainability Index (SI). The SI is determined for each person (Equation A.8) along with an average SI of all participants for each alternative (Equation A.9). To evaluate significant differences between alternatives, value ranges characterised by the mean value ± standard error of the mean (SEM) are compared pursuant to [54].

Presentation of results: To understand the variety of assessments of individual participants, the visualization of results is an essential element of the MCTA. In total, the tool provides the following visualizations, which are inspired by [23]: (i) a colour-coded scatter plot showing the relative weights assigned to main-criteria; (ii) bar diagrams ranking the alternatives with respect to the average SI; (iii) colour-coded scatter plots showing the average SI for all alternatives in one graph; (iv) colour-coded scatter plots showing assessment results on the level of main-criteria; (v) colour-coded bar diagrams showing the distribution of the SI as the balance of positive and negative weighted scores on the level of main-criteria; and (vi) a summary of scoring and weighting results on the level of main-criteria. The latter three types of visualizations summarize the voting of all participants in one graph for each alternative. The colour-coded scatter plots indicate assessment results of different stakeholder groups alongside the mean of all participants in different colours. The first and fourth graphs are designed by using a net diagram and by arranging the axes that represent main-criteria according to the sustainability star. The sixth graph visualises connections between the perceived importance of a certain dimension of sustainability and the assigned performance of an alternative. The former is indicated by numeric weights assigned to respective main-criteria plotted on the x-axis and the latter by numeric scores that an alternative receives for certain main-criteria plotted on the y-axis. The area, where certain main-criteria score negatively (numeric score < 0) while, at the same time, these main-criteria are considered important (numeric weight > 50), is marked as “red area”. Results are presented for all main-criteria (colour-coded symbols) and for all participants (number of signs) in one graph for each alternative.

We acknowledge that the computational approach of the MCTA is rather simple, however, in accordance with basic principles of MC(D)A modelling [29]. For example, on the one hand, the final single index makes alternatives directly commensurable [ibid.]. On the other hand, the applied methods, MAVT and SAW, belong to compensatory MC(D)A techniques [ibid.]. This means that a very bad scoring for one attribute can be compensated by a very good scoring for another and vice versa. Furthermore, instead of determining specific value functions for each participant, we assumed linearity. The main reason was to avoid overstraining participants. Furthermore, the statistical approach is rather simple but viable. As featured by [34], who demand a “more transparent, simple, and easily accessible participatory modelling paradigm and process”, we thus consider the MCTA as a valid methodological extension of the MC(D)A. Finally, visualisations support communication of results to and between participants. Graphs are designed so that they can effectively help (i) to reveal issues that need to be addressed before considering a technology suitable for sustainable community development, (ii) to identify the specific fields where further improvements of the technology are needed, and, finally, (iii) to avoid potential project ruptures in the future.

In order to perceive the feasibility of the developed method and tool, we performed a pre-testing of the MCTA. [-10]In this section we firstly show how participants weighted the different main-criteria describing the overall sustainability. Subsequently, we present selected results of the sustainability assessment of cooking and sanitation alternatives. We provide additional graphical visualizations alongside plot data in the Supplements.

Weighting of main-criteria: In the course of MCTA’s pre-test, the choice group of participants assigned, on average, the highest weights to the environmental main-criterion; the political-legal objective is perceived as least important (Figure 3). Second in level of importance are the remaining four main-criteria with no significant differences (Figure S.2). We further find a high variance in the distribution of weights of all participants. Weightings among participants are most divergent for socio-cultural criteria ( Δ m ⁢ i ⁢ n m ⁢ a ⁢ x ≈ 20%) and least divergent for political/legal criteria ( Δ m ⁢ i ⁢ n m ⁢ a ⁢ x ≈ 11%).

Comparing weights assigned by participants, there is a tendency that Tanzanians consider health/hygiene and environmental criteria more important in comparison to Germans. German representatives, in contrast, perceive technological-operational and socio-cultural criteria as more important compared to Tanzanians.

We explain the finding that stakeholders consider the environment to be the most important dimension of sustainability because the environmental main-criterion include many sub-criteria related to agriculture, and is consequently perceived as highly important for smallholders. On the contrary, according to [55], residents in the vicinity of Arusha/TZ do not consider environment as an important objective when choosing renewable energy technologies for electrification. Only one interviewee out of about 40 respondents, who were mainly farmers, addressed an environmental criterion as an influencing factor to purchase a solar home system [ibid.]. Respondents instead mentioned health and hygiene as the most important objectives for their families [ibid.].

Additionally, [56] state that environmental criteria are less relevant for residents when assessing sanitation systems in a peri-urban area in Burkina Faso. In order to perceive a certain sanitation system as appropriate, factors such as costs and service quality are most relevant for participants [ibid.]. Nevertheless, we understand our finding that environmental criteria are most relevant, by the fact that (i) our study area is a rural area farther away from urban settlements compared to the study areas of [55] or [56] and (ii) that many participants in our study are closely related to environmental settings (e.g. farmers with off-farm income, agricultural technicians, environmental scientists, and environmental activists).

In other comparative studies of small-scale bioenergy technologies from SSA, however, highest weights are assigned to the technological-operational criterion. For example, in an assessment of ICSs available on the Tanzanian market, criteria related to the construction or operation of stoves ranked highest, such as manufacturability, durability, or portability [37]. Furthermore, stakeholders participating in an assessment of a biogas system in Bahir Dar, a city in northwest Ethiopia, consider technological-operational criteria as most important dimension of sustainability [23].

In our study, lowest priority is assigned to the political/legal main-criterion. We reason that there is little governmental support in TZ and especially in the study region, which is recognized by stakeholders, and thus influences their judgements. National programs for promoting renewable energies and supporting access to such technologies especially in rural areas of TZ include the Rural Energy Fund, the Rural Energy Agency, and the Tanzanian Domestic Biogas Program (TDBP) [37, 57]. However, to our knowledge, these programs have not yet reached Karagwe. For example, TBDP has not yet chosen or appointed implementation partners in the region. According to [37], Tanzanian standards for cooking technologies are only available for charcoal stoves whilst existing standards are not enforced. Furthermore, governmental initiatives, intended to support bioenergy implementation, are not coordinated and, thus, largely inefficient [ibid.]. With respect to sanitation, [58] also criticizes a strong lack of legislation in the sector of non-sewer sanitation service in TZ.

The finding that only average importance is given to socio-cultural aspects is comparable to studies of [23] and [37] and in the study of [56] only minor importance is assigned to socio-cultural criteria.

Sustainability assessment of cooking alternatives: When pre-testing the MCTA, all analysed alternatives reach average SI-values between 0 and + 1 (Figure 4). Hence, the choice group of participants perceives cooking alternatives overall as “acceptable” and attributes a “good, ordinary” performance to respective technologies (cf. Table 4). Mean values of the four alternatives, however, do not significantly differ from each other, as overlapping error bars in Figure 4 indicate [54].

Regarding the six dimensions considered to assess sustainability, all stoves receive positive (weighted score > 0) assessment results mainly for health/hygiene, socio-cultural or technological-operational (except biogas system) functions (Figure 5). Negative (weighted score < 0) assessment results refer to socio-economic/financial (for charcoal and biogas alternatives) and environmental (for charcoal only) criteria.

When crosschecking the assessment results with literature, we consider our findings for cooking alternatives as highly comparable to [37]. In both studies, microgasifiers and rocket stoves receive positive scores because of their potential to reduce fuel consumption and greenhouse gas (GHG) emissions.

Interestingly, Tanzanian participants in particular assigned positive scores to the biogas system and for sub-criteria on GHG-emissions despite the fact that the description clearly stated that GHG-emissions would increase compared to cooking on a three-stone fire. We reason that Tanzanian stakeholders are aware of the minor influence of TZ on global GHG-emissions, compared to Germany, other European countries, or the United States. We further argue that there is a strong and positive pre-conception of the biogas system as an “environmental-friendly technology”. Regarding the usability of cooking technologies, all stakeholders participating in the MCTA agreed that the biogas system performs the worst of all alternatives analysed. Major problems identified are thereby (i) water requirements for a sound operation of the digester and (ii) lack of robustness of the system towards changes in climate conditions and user abuse. Also with respect to costs (including costs for implementation, operation, and maintenance), the biogas technology is assessed as the most disadvantageous cooking alternative. To underline scorings of costs, investment costs of locally available alternatives in Karagwe are as follows: costs for implementing a biogas system are approximately €1,200 , which makes possessing a biogas digester a substantial investment for smallholders. In comparison, acquisition costs of charcoal stoves or ICSs range from 2 to about €25 (cf. Table A.5). Furthermore, [57] identify high installation and maintenance costs as key barriers for biogas implementation in TZ, despite the fact that households are willing to adopt the technology. Hence, [57] conclude that implementing biogas in rural areas requires financial support by national programs through subsidies and/or loans.

Considering the perceptions of stakeholder groups, we find a very high variance in individual valuations of participants. On the example of the rocket stove, weighted scores of main criteria range from - 5 (“unfavourable”) to + 7 (“clearly favourable”) between participants (Figure 5). With respect to the SI, most Tanzanian representatives assess charcoal burners, rocket stoves, and microgasifiers negatively and below the average SI (Figure S.3). German participants, however, assess these technologies with positive and above-average valuations. A reverse tendency, however, is depicted for the biogas systems: judgements of German participants are rather negative and below the average SI, whilst Tanzanians assess the technology positively and above the average SI. Most prominently, staff members of CHEMA ascribe the highest positive points to rocket and microgasifier stoves, which are the cooking technologies that CHEMA disseminates. Likewise, staff members of MAVUNO rate ICSs with rather negative scores compared to the biogas system, which is the technology MAVUNO promotes in Karagwe.

Sustainability assessment of sanitation alternatives: Pre-testing the MCTA, all alternatives reach average SI-values in the range of about - 2 to + 2 (Figure 6). Both ecological approaches to sanitation are rated significantly better compared to the septic alternative. There is, however, no significant difference in the assessment of EcoSan and CaSa because respective error bars in Figure 6 overlap [54]. Overall, the performance of EcoSan and CaSa approaches is assessed as “good or ordinary”, which means that the choice group of participants values both approaches as “acceptable” to “slightly favourable” alternatives. On the contrary, participants assess the septic alternative as “slightly unfavourable” alternative.

Regarding the six sustainability dimensions, EcoSan and CaSa receive positive assessment results mainly for technological-operational, environmental, and socio-economic/financial functions (Figure 7). Negative assessment results refer, in particular, to the political/legal criterion. Shortcomings of the septic alternative mainly refer to the environmental, health/hygiene, and socio-economic/financial criteria.

Crosschecking our results with literature, the observation that a clearly better socio-economic/financial performance is assigned to the ecological approaches compared to the septic alternatives, is in line with [56]. For the case of Burkina Faso, [56] report that in particular low- and middle-income households perceive the UDDT as more financially appropriate compared to septic tanks. Strategic thinking, which includes considering operational costs as an important factor, influences households’ judgements [56]. In contrast, high-income households consider the septic system as more appropriate compared to the UDDT, because for this social group factors such as service quality are more important than costs [ibid.]. Farmers in Karagwe belong, however, mainly to low- and middle-income household groups [14]. (Table A.5 summarizes costs of sanitation alternatives.)

The pre-test further indicated certain doubts about the approaches, in particular on the sub-criteria “safe working conditions” and “safety in operation through avoiding risks of infection to users”. In contrast, literature commonly promotes EcoSan as a safe sanitation alternative because of the multi-barrier approach applied [12, 59]. To our knowledge, there were, however, no instances of sickness during the course of the CaSa pilot project that ran more than four years. Further investigations might follow-up this significant observation (cf. Section 4.2.). In total, the performance of ecological alternatives against the health/hygiene main-criterion is neutral. [-10]Our finding ultimately supports integrating health/hygiene as criterion when assessing sanitation technologies. This helps to identify areas of conflict, as shown in our case. Health/hygiene criteria should, therefore, not be excluded from analyses as it was, for example, the case in studies by [56] or [60].

Furthermore, we find that socio-cultural performances assessed for EcoSan and CaSa alternatives are comparable to assessment results for the septic system. We attribute this finding to the practical demonstration of UDDT-technology in the region. On the contrary, [56] identify UDDT as less socio-culturally accepted compared to a septic system and, therefore, request more alternatives and their advantages and disadvantages before assessing the technologies. This was exactly the case in the CaSa pilot project’s course.

Perceptions of stakeholder groups with respect to the average SI assigned to sanitation alternatives (Figure S.4) reveals as well high variations in assessment results. The EcoSan alternative, for example, weighted scores of main criteria range from - 7.5 (“clearly unfavourable”) to + 6.5 (“clearly favourable”) between participants (Figure 5). Nonetheless, the characteristic patterns of stakeholders’ valuations differ here. For both ecological alternatives, Tanzanian stakeholders assess negatively and below the average SI, whilst German participants rate the alternatives clearly positive and above the average SI. In contrast, most German participants assess the septic alternative as clearly more negative and below the average SI compared to their Tanzanian fellows. Only for the septic system, the assessments of all participants are rather homogenous and less differentiated compared to other sanitation technologies. In addition, the septic alternative receives an average negative SI from all except one participant. Most prominent is, however, that one German participant, who has been strongly involved in designing, developing, and testing the CaSa-approach, assigns the highest positive scores to that sanitation alternative.

In this section we firstly summarize experiences, insights, and lessons learned, that derived from the participatory pre-test of the MCTA regarding the assessment process and tool. Next, we discuss certain shortcomings of the methodology. Specific adjustments we recommend for future applications of the MCTA are summarized in Section 4.2.

In the first instance, the participatory design of the method we developed is a direct response to [34], who stress the need to develop more participatory elements at all levels of MC(D)A to make the method applicable in real-world practices. The relevant problem addressed is the ex-ante sustainability assessment of small-scale cooking and sanitation technologies in the exemplary context of smallholder communities in Karagwe, TZ.

We further learned that the visualisations of results, as integrated part of the MCTA-tool, are highly supportive to shed light on individual perspectives and valuations of stakeholders for the assessed alternatives. The visualisations help to make subjectivity in scoring results transparent. Here, we agree with [40] who emphasises that showing stakeholders each other’s preferences is rather one of the most relevant results of conducting an MC(D)A. The visualisations further allow for a target-oriented improvement to overcome shortcomings of the technologies. [15]Due to the low sample size and the high variance we observed in the tool’s pre-test, some visualization are not discussed thoroughly in this article. They are, however, presented in the Supplements (cf. Figures S.5–S.8).

As other benefits of the MCTA, we concluded that the assessment process promotes collaborative learning and a better understanding of the technologies assessed. Final feedback revealed that five out of six participants articulated that the MCTA supported them in forming their own opinion about the technologies (Figure S.9). Hence, our method reacts on [61] who promote “embedded learning”, that is integrating recent findings and sustainability assessment. This integration also constitutes an important part of the analytical tool we developed. Information that is provided to participants during the assessment process and describe various impacts of the technologies assessed, originate inter alia from recent action research conducted in Karagwe.

Despite these benefits, we further experienced the challenge of “balancing three tensions: (i) maintaining scientific credibility, (ii) assuring practical saliency, and (iii) legitimising the process to multiple participants” [62]. In particular, we identify certain limitations referring to (I) the choice of alternatives and participants, (II) the realised means of participation, (III) time consumption, communication effort and complexity, and (IV) gender balance.

(I) Firstly, we acknowledge that technologies mostly applied for the time being, which are three-stone fire and pit latrine, are not among the alternatives assessed. One could further criticise the fact that farmers were not involved in the assessment even though they represent the party most concerned and that the technologies assessed were developed to improve livelihoods of smallholders. [10]Reasons for conducting a pre-test of the MCTA-tool with local representatives limited to NGOs’ staff members are disclosed in Section 2.3. Our proceeding is supported by [63] who also recommend establishing intermediaries between research institutions and farmers when evaluating agricultural projects. Culture and social standing of research and/or NGOs can inhibit reciprocal communication between researchers and farmers.

(II) We acknowledge that the MCTA process is not participatory throughout as intended. This is a consequence of additional empirical data generated and that we faced a conflict between the scope of the work and the available time budget when actually starting the pre-test. Hence, we had to adapt the 9-steps-approach. Initially we wanted to include more feedback loops during the process, for example, in order to discuss experiences after scoring, and, thereafter, offering participants the opportunity to adapt their valuations. Moreover, the assessment was conducted separately, meaning that Tanzanian participants were in TZ and German participants in Germany. This fact results, in limited insights into the individual perceptions and, thus, inhibits joint understanding and learning of participants.

(III) We understand that high time consumption is often reported as a problem when conducting a participatory MC(D)A [38, 40]. Nonetheless, in the final feedback, most participants judged the scope of the MCTA, defined by time budget, details, number of criteria, clarity of the task, amount of information, etc., as good and to cope with (Figure S.9). Moreover, extensive communication (via e-mail and Skype) between the planner and individual participants was required to identify and solve possible misunderstandings. For example, participants scored alternatives significantly different although the performance of these alternatives was described as strongly comparable. Differing backgrounds of participants might explain this, as well as the problems they faced in understanding all (scientific and non-scientific) terms used in the descriptions. Participants might also simply not agree with the description provided, due to contrasting personal experiences and knowledge. However, because of the decentralized process we applied, it was not possible to directly address these observations, for example in individual or group discussions.

(IV) Lastly, we acknowledge that we pre-tested the MCTA-tool with in total two females and eight males; both women white and with academic background. Hence, we failed to reach a gender balance and, in particular, to include female representatives of Tanzanian partners. We attribute this shortcoming (i) to choosing English as working language, and (ii) to a general dominance of men at the level of project coordination and/or management in the participating NGO communities.

With regards to differences in stakeholders’ perceptions, our findings are further in line with [29], who state that applying a MC(D)A-approach does not create an objective, unbiased, and value-free analysis. In a nutshell: individual perspectives and subjective judgements are “normal”. [-20]It is hence a methodological challenge to make these differences visible instead of neglecting them. For example, we found that assessments of participants in the pre-test seem substantially biased towards the technology they promote. Visualizing these differences expediently is only the first step. The lack of group discussion (as part of the step 8; cf. Section 2.3.) is a clear shortcoming. For example, we have no specific insights into why differences exist and which arguments underlie the variations observed. Interpretation of results (by participants and the authors) is thus largely based on guesswork.

According to the final feedback, two thirds of participants believe that the MCTA is also applicable within their community if further adapted (cf. Section 4.3). In particular, the tool can help (i) to clarify existent doubts and expected benefits, (ii) to stimulate discussions, and (iii) to compare varying assessments of projects facilitating NGOs and local communities.

With respect to the objective to develop an integrated method for a participatory sustainability assessment, we conclude the following strengths and benefits of the MCTA method and tool:

From a methodological perspective, combining the MC(D)A with analytical methods is a viable integrated approach to sustainability assessment. On one hand, MC(D)A helps to structure the assessment. On the other, the integration of results from analytical studies, such as MFA, SNB, or laboratory analysis, allows for illustrating the performance of alternatives assessed through semi-quantitative objectives of sustainability, and particularly the environmental dimension.

Finally, the developed method provides hands-on support through a decent set of criteria and a basic assessment tool. Basic skills in Excel are the main requirement for using the MCTA-tool. Nevertheless, conducting an MCTA is an extensive process that demands considerable time budgets and a strong commitment from all participants.

To summarize, we conclude that the MCTA is an appropriate and applicable method to analyse potential effects and interrelations of different cooking and sanitation technologies on household levels. The method also indicates contradictions with respect to different sustainability goals. In addition, the MCTA enhances transparency about individual preferences, values, doubts, and objectives among all participants, respectively stakeholders.

Even though the number of individuals participating in the pre-test of the MCTA was rather small, we conclude:

With respect to the latter, representatives of the German project partners generally tend to be more enthusiastic about the analysed technologies while representatives of the Tanzanian partners are more sceptical. This observation holds true especially for the ICSs technologies, such as microgasifier and rocket stoves, and the EcoSan and CaSa sanitation alternatives. Technologies that are more common and established in both countries, such as biogas or septic systems, are assessed more similarly among participants. Another example for varying assessments is that representatives of the implementing NGOs and of the technology-developers or scientific partners tend to favour the alternative that they have supported and accompanied, and, likewise, rate competing technologies below average.

From our hands-on experience in facilitating the MCTA pre-test, we finally hypothesise that the higher the assessors diversity with respect to education, language, class, and scientific or practical experiences and knowledge, (i) the more difficult the process because of uncertainties about the common understanding and (ii) the more important the process and the more interesting the results. Testing this hypothesis could be subject of future scientific MC(D)A work.

Pre-testing the MCTA with a choice group of participants can serve as a basic proof-of-concept, with the method as a simple but viable sustainability assessment tool for the multi-perspective evaluation of cooking and sanitation alternatives. Hence, we conclude that, by providing a framework and a tool, the MCTA can effectively support case studies and relevant strategic planning and decision-making. Moreover, the MCTA is also practicable for other real-world applications related to small-scale cooking and sanitation technologies, such as in other regions and/or for other technologies.

If the MCTA is applied in a different setting, necessary adjustments include re-phrasing the descriptions of alternatives and adapting the set of criteria. To make the MCTA further applicable for its use in communities, methodological challenges and problems, as described in the preceding sections, need to be addressed. We thus recommend the following with respect to the choice of participants:

In addition, we recommend the following with respect to the assessment process and method:

Reducing the number of sub-criteria could follow a participatory approach comprising, firstly, a moderated discussion about the existing catalogue of sub-criteria and, thereafter, a selection of those criteria considered as the most relevant. Methodologically, the “silent negotiation” could be applied in a moderated workshop, which helps to cope with power-imbalances in the consortium [24]. If practitioners do not aim at running through a whole assessment process, the existing catalogue of criteria could also serve to structure focus group discussions or community assemblies. Moderated discussions between participants can help to enhance the joint understanding of arguments that underlie and explain differences in the quantitative assessments. For example, a group discussion after reading the descriptions can help to ease a common understanding about the information provided before scoring the alternatives. At the same time, such a discussion could help to alleviate the time budget of the facilitator for clarifying open questions from participants. With more time available, another option could be to firstly conduct scoring of alternatives without description, secondly with description prepared by the planner, and finally to compare and to discuss results together. Finally, and with respect to the alternatives, we recommend the inclusion of technologies that are most commonly applied so far to support a comparison of alternatives to the actual real-world situation.