Achieving Efficiency in Simulation of Distribution Tails with Black-Box Importance Sampling MSME, NGO, and MNC Organization

Journal: Operations Research 

The ability to estimate and control tail risks, besides being an integral part of quantitative risk management, is central to running operations requiring high service levels and cyber-physical systems requiring high reliability. Despite its importance, scalable algorithmic approaches for measuring and controlling tail risks have been challenging to develop, even for problems that employ common modeling tools like linear programs. This difficulty arises mainly from the infrequent occurrence of relevant risky events in the samples and the need for a need to engineer variance reduction techniques on a case-by-case basis. An important question to ask therefore is the following: Can instance-agnostic algorithms overcome these challenges and provide similar levels of variance reduction?

Addressing this question in the article, ‘Achieving efficiency in black-box simulation of distribution tails with self-structuring samplers’, the authors identify an elementary transformation of the samples that achieves a near-optimal variance reduction when applied alike across a wide range of models. This transformation seeks to replicate the behavior of data in a target rare region of interest from samples that are representative, but less rare region that is more frequently observed in empirical samples.

The authors demonstrate the utility of their scheme in the evaluation of probabilities of (a) large losses in a portfolio credit risk setting, and (b) large delays in contextual routing. The proposed sampler for portfolio credit risk also serves as an entirely novel addition that extends the scope of applicability of this line of research pursued over the past two decades. The authors also demonstrate that their scheme easily adapts to solving the more complicated problem of efficient decision making (optimization) in the presence of distribution tails.

In addition to providing a widely applicable tool for estimation/optimization of tail risks, the authors develop a general mathematical theory to derive the structure of rare events for the aforementioned class of problems. This theory has independent applications in providing managerial insights into how to handle tail events.