Setting research priorities on multiple micronutrient supplementation in pregnancy

Prenatal micronutrient deficiencies are associated with negative maternal and birth outcomes. Multiple micronutrient supplementation (MMS) during pregnancy is a cost-effective intervention to reduce these adverse outcomes. However, important knowledge gaps remain in the implementation of MMS interventions. The Child Health and Nutrition Research Initiative (CHNRI) methodology was applied to inform the direction of research and investments needed to support the implementation of MMS interventions for pregnant women in low- and middle-income countries (LMIC). Following CHNRI methodology guidelines, a group of international experts in nutrition and maternal health provided and ranked the research questions that most urgently need to be resolved for prenatal MMS interventions to be successfully implemented. Seventy-three research questions were received, analyzed, and reorganized, resulting in 35 consolidated research questions. These were scored against four criteria, yielding a priority ranking where the top 10 research options focused on strategies to increase antenatal care attendance and MMS adherence, methods needed to identify populations more likely to benefit from MMS interventions and some discovery issues (e.g., potential benefit of extending MMS through lactation). This exercise prioritized 35 discrete research questions that merit serious consideration for the potential of MMS during pregnancy to be optimized in LMIC.


Introduction
Adequate nutrition is important throughout the life cycle but is particularly important during preg-nancy, to support both maternal health and fetal development. Many micronutrients have critical roles during this life stage (especially vitamins A, B 6 , B 9 , B 12 , C, D, and E, and minerals iron, zinc, iodine, copper, and selenium 1 ) for which the recommended intakes may increase by up to 50% to accommodate the higher maternal, placental, and fetal demands. These increased nutritional demands of pregnancy, in combination with the preexisting nutritional deficiencies among undernourished (and/or the even higher nutritional demands for adolescent) pregnant women, may put their health and that of their offspring at risk. 2 Maternal micronutrient malnutrition is associated with low birth weight (LBW) (<2500 g), preterm birth (<37 weeks), being born small-for-gestational-age (SGA), and perinatal and maternal mortality, among other pregnancy-related adverse outcomes. 1 Prenatal multiple micronutrient supplementation (MMS) provides a good solution to supply those essential nutrients. A series of publications recently put forward by the New York Academy of Sciences (NYAS) presented evidence of the benefits of MMS on maternal and perinatal outcomes (i.e., significant risk reduction of LBW, SGA, preterm birth, and stillbirth 3 ), in addition to those provided by iron and folic acid. 2,[4][5][6] These publications identify populations in low-and middle-income countries (LMIC), where a switch to MMS would be justified, consistent with the WHO Antenatal Care Guidelines, 7 and would be highly cost-effective. 4 However, important gaps in knowledge remain in the implementation of MMS in prenatal care programs, which affect the ability of this intervention to achieve optimal performance.
To clarify research needs in solving knowledge gaps in MMS implementation, NYAS, acting on behalf of the recently assembled MMS Technical Advisory Group (MMS-TAG), conducted a research prioritization exercise using the wellestablished Child Health and Nutrition Research Initiative (CHNRI) methodology. 8 CHNRI provides a systematic and transparent method for setting priorities in health research 8 using a rationale, conceptual framework, application guidelines, and strategies to address the needs of various stakeholders. 9 It has been used mostly for setting research priorities to address global child health issues (e.g., zinc deficiency 9 ), as initially designed, but has also been applied outside this field (e.g., dementia 10 ), as this method is replicable, simple, democratic, flexible, and adjustable to many different contexts and needs. 11 The specific aim of this research prioritization exercise was to inform the direction of research and investments needed to support the implementation of MMS interventions for pregnant women in LMIC. The ultimate aim is to reduce the burden associated with micronutrient deficiencies in this vulnerable population group, particularly the adverse pregnancy and birth outcomes.
A first step in the CHNRI method consists of defining the context (population of interest, disease burden, timescale, etc.) and risk preferences associated with this research priority setting exercise (Table 1).

Methods
As recommended by the guidelines for the implementation of the CHNRI method, 8,11 the following steps were followed: 1. Selected the project managers for this exercise (F.G., G.B., and M.W.B., from the MMS-TAG secretariat at the NYAS) and specified the context and risk management preferences (Table 1). 2. Selected and asked a small group of specialists in nutrition and maternal health (mostly from the MMS-TAG) to contribute research questions believed to be important for the implementation of prenatal MMS interventions in LMIC. 3. Organized a large number of the proposed research questions into four fundamental instruments of health research (i.e., the four domains: "description," "delivery," "development," and "discovery"); removed or combined redundant questions to obtain a consolidated final list of questions. The four domains were defined as: -Description: What is the burden of the problem and its determinants, for example, feasible methods to determine if a population has multiple micronutrient deficiencies? -Delivery: How can we improve the delivery of existing interventions, for example, how to improve adherence and compliance? -Development: How do we improve the efficiency of existing interventions, for Research will be diversified across countries that show a high prevalence of micronutrient deficiencies among pregnant women and/or high rates of adverse pregnancy and birth outcomes, which will have different levels of risk and feasibility CHRNI, Child Health and Nutrition Research Initiative; LBW, low birth weight; LMIC, low-and middle-income countries; SGA, small-for-gestational-age; WRA, women of reproductive age. example, by modifying the product formulation, reducing costs, and so on? -Discovery: Does the research lead to innovation, for example, by demonstrating new benefits in trials or mechanisms of action of MMS?
To facilitate classification, each domain was, in turn, divided into subdomains, that is, "description" questions were classified under "prevalence" or "assessment"; "delivery" questions were classified under "adherence," "coverage," "packaging," or "training"; "development" questions were classified under "dosage" or "implementation"; and "discovery" questions were classified under "formulation" or "impact." 4. Developed and discussed the criteria for setting health research priorities. Four proposed criteria were later used to rate each research question based on whether the research question was: -Answerable: Is it feasible to answer this research question within 5-10 years? -Impactful: Are the results from this research likely to inform future practice and/or policy on MMS during pregnancy? -Effective: Will the research effectively improve maternal and birth outcomes? -Equitable: Will the results of the research help enable the benefits of MMS to reach the poorest and currently underserved women?
5. Constructed an evaluation template and sent it to a broader group of specialists (including antenatal care (ANC) managers and nongovernmental organizations operating in this space, identified by the MMS-TAG) to score the final list of research questions against each priority-setting criterion, and to attribute weights to each of these criteria. Attributing weights to each criterion (from 0% to 100%) allowed each participant to give some criteria more importance over the others.
The Likert scale used to answer each of the four criteria questions and the score attributed to each answer was: -Highly likely (score: 1) -Somewhat likely (score: 0.75) -Neutral, neither likely, or unlikely (score: 0.5) -Somewhat unlikely (score: 0.25) -Highly unlikely (score: 0) -Do not know (no score; coded as a missing answer) 6. Compiled the data, calculating the final unweighted and weighted research priority scores (RPSs) for each research question, and assigned ranks. To calculate the final unweighted RPS, the average of the scores for each of the four criteria (from all the participants) was computed, followed by the calculation of the mean of these four criteria scores (i.e., all four criteria were given equal weight, equivalent to 25%). For the calculation of the weighted RPS, the average of the priority scores for each of the four criteria (from all the participants) was computed, taking into account the group average weight attributed to each criterion. The weighted RPS was calculated using the following formula (Eq. 1): where C a is the average criterion score for answerable, C i is the average criterion score for impactful, C e is the average criterion score for effective, C q is the average criterion score for equitable, W a is the average weight for answerable, W i is the average weight score for impactful, W e is the average weight score for effective, and W q is the average weight score for equitable. In addition, the average expert agreement (AEA) score, defined as the level of agreement among scorers, was calculated for each of the 35 research questions. The AEA score is the average proportion of scorers who chose the mode (most common score) across the four criteria. The AEA score was calculated as follows (Eq. 2): where q is a question that experts are being asked to evaluate, ranging from 1 to 35.

Results
This prioritization exercise was conducted between April and June 2019 with a group of international experts in nutrition and maternal health, who provided and ranked the most urgent gaps in knowledge for the successful implementation of prenatal MMS interventions. This group included participants based on a variety of countries, including Bangladesh, Canada, Denmark, Ghana, India, Italy, Kenya, Mozambique, Switzerland, Thailand, the United Kingdom, and the United States of America.
The initial small group of 25 experts submitted a total of 73 research questions, which were analyzed and reorganized (e.g., by eliminating or combining redundant questions), resulting in a final number of 35 research questions. The evaluation exercise was then sent to a larger group of experts (n = 87), who were invited to score each of the 35 research questions against the four evaluation criteria. Thirty-five participants completed the evaluation exercise and the summary of the results is presented in Tables 2-4. These 35 participants work in a wide range of organizations, as exemplified in Figure 1, including academic or research, nongovernmental, the United Nations or multilateral,   Note: The questions are color coded by the type of domain: yellow for "description," green for "delivery," orange for "development," and blue for "discovery." AEA, average expert agreement. and nonprofit organizations, donors, and also work as consultants or are self-employed. Table 2 shows the consolidated list of 35 research questions ranked according to the final unweighted RPS, with the respective domain and subdomain for each question. The questions are color coded by type of domain: yellow for "description," green for "delivery," orange for "development," and blue for "discovery." The top 10 research options include questions that focused on strategies to increase ANC attendance and MMS adherence, as well as on the parameters and methods needed to identify the populations that are more likely to benefit from prenatal MMS interventions, and some discovery issues (e.g., the potential benefit of extending MMS interventions beyond pregnancy, and during the period of lactation). The AEA ranged from 0.32 to 0.53, out of a possible 1.00. Table 3 describes the number of research questions allocated to each domain and subdomain, as well as the mean, minimum, and maximum RPSs for each subdomain. The fourth domain "discovery" received a higher number of questions, while the lower number of questions were allocated to the third domain "development." The higher RPS was observed in the subdomain "assessment" and the lower score was attributed to the subdomain "dosage." As described above, participants were also asked to attribute weights to each of the four evaluation criteria. The group average weight attributed to answerable was 32.2%, to impactful was 25.5%, to effective was 25%, and to equitable was 17.1%.  Table 2, and the top 10 research options include similar topics between the ranks established by the unweighted and weighted RPSs. Table 4 lists the ranked final number of research questions and allows the comparison between the average unweighted RPS (in black) and the weighted RPS (in red), per each of the four evaluation criteria. This table helps to understand, for instance, that the question ranked overall in the third place (regarding the benefits of MMS supplementation continued through lactation) was considered the most answerable, and the question ranked overall in the first place (covering strategies that best increase ANC attendance and adherence to MMS) was considered the most impactful, independently of the use of weighted or unweighted criteria.

Discussion
This paper summarizes the research prioritization exercise conducted on the topic of MMS in pregnancy-an exercise that has never been conducted. The NYAS invited a group of international specialists to provide and rank the most urgent gaps in knowledge, focusing particularly on aspects that would improve the delivery and effectiveness of this intervention in LMIC populations. Following the well-established CHNRI methodology, this process prioritized 35 nonredundant research questions that merit serious considerations   The questions are color coded by the type of domain: yellow for "description," green for "delivery," orange for "development," and blue for "discovery." if the potential of MMS is to be fully realized in LMIC.
The RPS varied between 52.7% and 83.2%, which is in line with the range observed in previous research priority exercises that used the CHNRI methodology. 9,10 The AEA showed that from 32% to 53% of the scorers shared their views on the proposed 35 research questions. This range is lower than the AEA ranges reported in other articles that used the CHNRI methodology; for example, the AEA scores ranged from 54% to 86% on the topic of integrated community case management, 13 and from 53% to 78% on the topic of dementia. 10 However, this difference is likely to be caused by a greater number of possible answers to the scoring criteria in the present study. For instance, most of those research priority exercises used a simple ordinal scale ("yes," "no," and "undecided") or a dichotomous statement ("yes," "no," and "no answer"), while we used a Likert scale, which has a progressive rating ("highly likely," "somewhat likely," "neutral," "somewhat unlikely," and "highly unlikely").
The questions that received the highest priority in this exercise include the use of behavioral change and counseling strategies, and community workers who can increase ANC attendance and adherence to MMS, including in hard-to-reach populations. This is not surprising given that low adherence to prenatal micronutrient supplementation is a major barrier to achieve the full potential benefits of this intervention, even when the coverage of the supplementation program is satisfactory. A study across 22 LMIC found that 83% of all pregnant women had at least one ANC visit and a similar proportion (81%) received IFA tablets, but only 8% consumed at least 180 IFA tablets. 14 In addition, among the highest ranked topics are questions about the best (field-friendly and costeffective) indicators and methods needed to identify the populations that are more likely to benefit from prenatal MMS interventions in maternal nutrition programs. This may be justified by the fact that countries interested in adopting MMS interventions may feel a lack of clear guidance regarding the interpretation of the statement: "Countries with a high prevalence of nutritional deficiencies might consider the benefits of MMS on maternal health to outweigh the disadvantages and may choose to give MMS that include iron and folic    7 Governments may want to better identify higher risk groups and regions where the effectiveness of prevention is likely to be the highest and thus offer the greatest public health return on investment.
The research question that received the lowest score pertains to the marginal cost and benefits of adding each vitamin or mineral to MMS. The group of specialists may have attributed less importance to this question in light of the widely used and wellestablished UNICEF/WHO/United Nations University International Multiple Micronutrient Preparation "UNIMMAP" formulation, 15 which has been administered in many of the trials that demonstrated the additional benefits of MMS over IFA 3 and proved to be cost-effective. 4,16 While the addition of a variety of nutrients known to be important during pregnancy (i.e., calcium, magnesium, choline, and n-3 long-chain polyunsaturated fatty acids) was proposed in several research questions listed here, it is possible that a consideration of the time and resources that would be required to answer these questions contributed to their relatively lower RPS.
Overall, it was relatively straightforward to apply the CHNRI methodology to the implementation of MMS interventions for pregnant women in LMCI. Nonetheless, a few challenges were encountered. For instance, the allocation of each research question to the four domains "description," "delivery," "development," and "discovery" proposed by the CHRNI methodology 11 was not always clear, as some questions did not fit clearly into any domain, while others could belong to more than one domain. In fact, other authors who used the CHNRI method to reduce children's disease burden 9 have used other domains, namely "effectiveness," "affordability, deliverability, and sustainability," and "new innovations." Reducing the number of research questions into a consolidated, smaller, and more easily manageable final list of questions also resulted in excessive aggregation of several topics. For example, the question that received the highest RPS ("What strategies (e.g., cash transfers, easier ANC access, free MMS, pharmacy vouchers, quality service delivery, mass media, social and behavior change communication interventions, and SMS text messages) can best increase ANC attendance and adherence to MMS, including in hard-to-reach populations?") combined a wide variety of different strategies, which may not be easily comparable. Furthermore, the answer to this question may vary from country to country, or from region to region, depending on the service delivery bottlenecks associated with the two proposed outcomes.
A limitation of the current exercise is that respondents were not asked to take into account cost considerations to answer the research questions, except indirectly by the first scoring criterion (where the participant is asked whether it is feasible to answer the research question within 5-10 years). Thus, it is possible that participants judged the cost associated with some of the more highly valued research questions to exceed available donor resources. Furthermore, despite our attempt to reach (via email) a large group of stakeholders with a wide range of backgrounds and roles, the participants of this exercise were skewed toward academics, which may bias results; however, those who did respond had substantial experience providing technical support to programs.
In summary, the study team found it possible to apply the CHNRI methodology to  develop a list of research questions required to improve the implementation of this important and cost-effective nutritional intervention in LMIC populations. These research gaps need to be urgently addressed. Given the simple nature and relatively low cost of this exercise, this research prioritization exercise could be repeated periodically as new information becomes available.
Disclaimer N.D. is a UNICEF staff member. The opinions expressed in the documents included in this article are those of the authors and do not necessarily reflect the policies or views of UNICEF.