### Epidemiological background

The World Health Organization reported 9.4 million incident TB cases and 1.7 million TB deaths in 2009 and estimated that only 63% of annual incident TB cases were detected and reported; of these, 86% were successfully treated [10, 11]. Given the disease burden, the United Nations Millennium Development Goals include targets and indicators related to TB control. The targets include decreasing TB incidence by 2015, halving TB prevalence and mortality by 2015 (compared with 1990), and diagnosing 70% of new smear-positive cases and curing 85% of these cases by 2015. However, despite current efforts, many countries will not achieve these targets [10–14].

The HIV-AIDS pandemic is the major worldwide challenge to TB control [11, 13, 15, 16]. HIV creates a situation of serious uncertainty for public health interventions based on pre-HIV era models [10, 11, 13]. This is reflected in population distribution, spread, control, and recurrence. Latently and actively infected individuals contribute differently to spread of disease. It is necessary to consider infectivity, rapidity of progression, re-infection, individuals with higher susceptibility for infection and reinfection resulting from HIV coinfection, etc. in order to produce refined models of diagnosis and treatment.

Many different epidemiological models have been used to evaluate treatment strategies. Deterministic compartment models are the most common, and we use a slightly modified version of the widely used Murray-Salomon model [17–19] to describe the evolution of TB/HIV epidemics under various scenarios. The details of the model appear in Appendix “The Murray-Salomon model” section.

### Info-gap theory

The robustness function is the basic decision-support tool in an info-gap analysis. If our dynamic model were accurate we could evaluate any proposed intervention in terms of the outcome of that intervention that is predicted by the model. An intervention with low predicted TB prevalence is preferred over an intervention with higher predicted prevalence.

The problem is great model uncertainty. This means that predicted outcomes are unreliable and it is unrealistic to prioritize interventions in terms of their predicted outcomes. Using the model to find the intervention whose predicted outcome is best, is not suited to planning with highly uncertain models.

Model-based predictions are useful, but when deciding which public health intervention to implement, we should also ask: how wrong could the model be, and an acceptable outcome is still guaranteed? For any specified intervention we ask: what is the largest error in the model, up to which all realizations of the model would yield acceptable outcomes? Equivalently, what outcomes can reliably be anticipated from this intervention, given the unknown disparity between the model and reality? Answers to these questions lie in the robustness function, specified in Appendix “Definition of robustness” section. The robustness is dimensionless, and equals the greatest fractional error in the model parameters that is consistent with a specified outcome requirement. We use the robustness function to prioritize the interventions in terms of their robustness against uncertainty for achieving the required outcome.

Knight [20] recognized that probability distributions are sometimes unknown and that severe uncertainty may be non-probabilistic. Wald [21], Ben-Tal and Nemirovski [22] and others developed tools for robustly managing non-probabilistic uncertainty by minimizing the worst outcome on a set of possibilities. Info-gap theory is non-probabilistic and handles situations where worst cases are unknown.

We summarize here the main attributes of the info-gap robustness function: a plot of robustness-to-uncertainty versus required performance. This is the basic info-gap tool for prioritizing available options.

#### Robustness trades off against performance [23, 24]

More demanding performance requirements are less robust against uncertainty than less demanding requirements. This trade off is quantified and expressed graphically by the monotonic robustness curve.

#### Model predictions have zero robustness against uncertainty [25]

When models are highly uncertain, it is unrealistic to prioritize one’s options based on predicted outcomes of those options, because those predictions have no robustness to errors in the underlying models. Options must be evaluated in terms of the level of performance that can be reliably achieved; this is expressed by robustness.

Combining the trade off and zeroing properties yields realistic prioritization of options.

#### Prioritization of options depends on performance requirements

Prioritization of options may change as requirements change. This is called “preference reversal” and is expressed by the intersection of the robustness curves of different options. Preference reversal provides insight to anomalous behavior such as the Ellsberg and Allais paradoxes in human decision making [8], the equity premium puzzle in economics [8], and animal foraging [26]. We will show that preference reversal occurs when selecting public health interventions because priorities are time- and context-dependent.

#### Info-gap models of uncertainty are non-probabilistic

Info-gap robustness analysis is implementable even when probability distributions are unknown, and thus is suited to severe uncertainty. In contrast, Monte Carlo simulation, Bayesian analysis, or probabilistic risk assessment require knowledge of probabilities. Other non-probabilistic tools include interval analysis, fuzzy set theory [27], possibility theory [28] and Robust Decision Making (RDM). A comparison of info-gap and RDM has recently been published [29].

#### Info-gap is operationally distinct from the min-max or worst-case decision strategy [9]

Info-gap robustness does not require knowledge of a worst case. When even typical scenarios are poorly characterized, it is usually impractical to characterize worst cases, which is required by the min-max strategy. Info-gap theory does require specifying acceptable outcomes. Thus it is well suited to policy making, because preferences on outcomes are the driving force.

#### Info-gap robustness may proxy for the probability of satisfying the performance requirement [8, 30, 31]

A more robust option is often more likely to achieve the required outcome. By prioritizing the options using info-gap robustness, one maximizes the probability of satisfying the requirement, without knowing probability distributions. The proxy property is central to understanding survival in economic [8], biological [26] and other competitive environments [31].

### Info-gap implementation

Info-gap methodology requires three main elements: a *system model,* a *performance measure* and a *model of uncertainty*. The system model is a mathematical representation of a system and its influence on the variables of interest, for which management aspirations (performance criteria) are set. A performance measure assesses value or utility of outcomes. The model of uncertainty is a non-probabilistic representation of the degree to which the value of parameters, the form of a function, or the structure of a model may deviate from nominal estimates.

The system model in our example is summarized in two functions. *C*(*t*) is the variation over time of the total number of TB cases, untreated and treated, HIV-positive and HIV-negative, as a fraction of the initial population. *R*(*t*) is the total number of relapses, fast and slow, HIV-positive and HIV-negative, as a fraction of the initial population. (See eqs.(23) and (24) in Appendix “The Murray-Salomon model” section.)

The public health practitioner wishes to control the total number of TB cases: the fewer the better. However, trying to minimize this prevalence depends on model predictions that are highly uncertain. The performance requirement is to keep the total fraction of TB cases at a specified time, *t*
_{m}, below a critical value, *C*
_{m}, eq.(25) in Appendix “Performance requirements” section.

Grassly *et al*[32] note, in discussing epidemiology of HIV/AIDS, that “not all sources of error are amenable to statistical analysis” (p.i37), due to biased, inaccurate or unavailable data. The basic idea of info-gap model uncertainty is that we do not know how wrong our estimates are, we have no reliable knowledge of worst cases, and we do not know probability distributions for the estimates. The info-gap model uncertainty model is a non-probabilistic quantification of uncertainties.

A dominant uncertainty in TB dynamics with HIV prevalence is in model parameter values, though HIV causes significant uncertainties in model structure. Structural uncertainty refers to missing terms in the equations, missing equations, or unknown nonlinearities. Structural uncertainty is dealt with much less frequently than parameter uncertainty because of technical challenges. We focus on parameter uncertainty in this paper because of its importance and to facilitate the presentation of this first application of info-gap theory to public health.

We use info-gap theory [8] to model and manage uncertainties in the following parameters: slow and fast relapse rates for HIV positives and negatives, TB infection rates for HIV positives and negatives, and the HIV infection rate. Much literature suggests these parameters for their impact on the course of epidemics and the difficulty in measuring them [10, 11, 16, 33–36]. Other uncertainties could also be investigated, depending on the purpose of the analysis. We use estimated values for each uncertain parameter, and estimated errors typically chosen as half of an interval estimate of the parameter. The info-gap model of uncertainty is specified in Appendix “Uncertainty” section.

We aim to achieve the performance requirement by judicious choice of control variables, defined in Appendix “Control variables” section. Eligible control variables are any coefficients of the dynamic model that can be influenced by public health or related medical intervention. We use the diagnosis rate, cure rate, relapse rate, and HIV infection rate. We define an intervention in terms of the values of these variables [15, 34, 37–40].