{% raw %} Title: Create a Markdown Blog Post Integrating Research Details and a Featured Paper ==================================================================================== This task involves generating a Markdown file (ready for a GitHub-served Jekyll site) that integrates our research details with a featured research paper. The output must follow the exact format and conventions described below. ==================================================================================== Output Format (Markdown): ------------------------------------------------------------------------------------ --- layout: post title: "Nested sampling for frequentist computation: fast estimation of small $p$-values" date: 2021-05-27 categories: papers --- ![AI generated image](/assets/images/posts/2021-05-27-2105.13923.png) Will Handley Content generated by [gemini-2.5-pro](https://deepmind.google/technologies/gemini/) using [this prompt](/prompts/content/2021-05-27-2105.13923.txt). Image generated by [imagen-3.0-generate-002](https://deepmind.google/technologies/gemini/) using [this prompt](/prompts/images/2021-05-27-2105.13923.txt). ------------------------------------------------------------------------------------ ==================================================================================== Please adhere strictly to the following instructions: ==================================================================================== Section 1: Content Creation Instructions ==================================================================================== 1. **Generate the Page Body:** - Write a well-composed, engaging narrative that is suitable for a scholarly audience interested in advanced AI and astrophysics. - Ensure the narrative is original and reflective of the tone and style and content in the "Homepage Content" block (provided below), but do not reuse its content. - Use bullet points, subheadings, or other formatting to enhance readability. 2. **Highlight Key Research Details:** - Emphasize the contributions and impact of the paper, focusing on its methodology, significance, and context within current research. - Specifically highlight the lead author ({'name': 'Andrew Fowlie'}). When referencing any author, use Markdown links from the Author Information block (choose academic or GitHub links over social media). 3. **Integrate Data from Multiple Sources:** - Seamlessly weave information from the following: - **Paper Metadata (YAML):** Essential details including the title and authors. - **Paper Source (TeX):** Technical content from the paper. - **Bibliographic Information (bbl):** Extract bibliographic references. - **Author Information (YAML):** Profile details for constructing Markdown links. - Merge insights from the Paper Metadata, TeX source, Bibliographic Information, and Author Information blocks into a coherent narrative—do not treat these as separate or isolated pieces. - Insert the generated narrative between the HTML comments: and 4. **Generate Bibliographic References:** - Review the Bibliographic Information block carefully. - For each reference that includes a DOI or arXiv identifier: - For DOIs, generate a link formatted as: [10.1234/xyz](https://doi.org/10.1234/xyz) - For arXiv entries, generate a link formatted as: [2103.12345](https://arxiv.org/abs/2103.12345) - **Important:** Do not use any LaTeX citation commands (e.g., `\cite{...}`). Every reference must be rendered directly as a Markdown link. For example, instead of `\cite{mycitation}`, output `[mycitation](https://doi.org/mycitation)` - **Incorrect:** `\cite{10.1234/xyz}` - **Correct:** `[10.1234/xyz](https://doi.org/10.1234/xyz)` - Ensure that at least three (3) of the most relevant references are naturally integrated into the narrative. - Ensure that the link to the Featured paper [2105.13923](https://arxiv.org/abs/2105.13923) is included in the first sentence. 5. **Final Formatting Requirements:** - The output must be plain Markdown; do not wrap it in Markdown code fences. - Preserve the YAML front matter exactly as provided. ==================================================================================== Section 2: Provided Data for Integration ==================================================================================== 1. **Homepage Content (Tone and Style Reference):** ```markdown --- layout: home --- ![AI generated image](/assets/images/index.png) The Handley Research Group stands at the forefront of cosmological exploration, pioneering novel approaches that fuse fundamental physics with the transformative power of artificial intelligence. We are a dynamic team of researchers, including PhD students, postdoctoral fellows, and project students, based at the University of Cambridge. Our mission is to unravel the mysteries of the Universe, from its earliest moments to its present-day structure and ultimate fate. We tackle fundamental questions in cosmology and astrophysics, with a particular focus on leveraging advanced Bayesian statistical methods and AI to push the frontiers of scientific discovery. Our research spans a wide array of topics, including the [primordial Universe](https://arxiv.org/abs/1907.08524), [inflation](https://arxiv.org/abs/1807.06211), the nature of [dark energy](https://arxiv.org/abs/2503.08658) and [dark matter](https://arxiv.org/abs/2405.17548), [21-cm cosmology](https://arxiv.org/abs/2210.07409), the [Cosmic Microwave Background (CMB)](https://arxiv.org/abs/1807.06209), and [gravitational wave astrophysics](https://arxiv.org/abs/2411.17663). ### Our Research Approach: Innovation at the Intersection of Physics and AI At The Handley Research Group, we develop and apply cutting-edge computational techniques to analyze complex astronomical datasets. Our work is characterized by a deep commitment to principled [Bayesian inference](https://arxiv.org/abs/2205.15570) and the innovative application of [artificial intelligence (AI) and machine learning (ML)](https://arxiv.org/abs/2504.10230). **Key Research Themes:** * **Cosmology:** We investigate the early Universe, including [quantum initial conditions for inflation](https://arxiv.org/abs/2002.07042) and the generation of [primordial power spectra](https://arxiv.org/abs/2112.07547). We explore the enigmatic nature of [dark energy, using methods like non-parametric reconstructions](https://arxiv.org/abs/2503.08658), and search for new insights into [dark matter](https://arxiv.org/abs/2405.17548). A significant portion of our efforts is dedicated to [21-cm cosmology](https://arxiv.org/abs/2104.04336), aiming to detect faint signals from the Cosmic Dawn and the Epoch of Reionization. * **Gravitational Wave Astrophysics:** We develop methods for [analyzing gravitational wave signals](https://arxiv.org/abs/2411.17663), extracting information about extreme astrophysical events and fundamental physics. * **Bayesian Methods & AI for Physical Sciences:** A core component of our research is the development of novel statistical and AI-driven methodologies. This includes advancing [nested sampling techniques](https://arxiv.org/abs/1506.00171) (e.g., [PolyChord](https://arxiv.org/abs/1506.00171), [dynamic nested sampling](https://arxiv.org/abs/1704.03459), and [accelerated nested sampling with $\beta$-flows](https://arxiv.org/abs/2411.17663)), creating powerful [simulation-based inference (SBI) frameworks](https://arxiv.org/abs/2504.10230), and employing [machine learning for tasks such as radiometer calibration](https://arxiv.org/abs/2504.16791), [cosmological emulation](https://arxiv.org/abs/2503.13263), and [mitigating radio frequency interference](https://arxiv.org/abs/2211.15448). We also explore the potential of [foundation models for scientific discovery](https://arxiv.org/abs/2401.00096). **Technical Contributions:** Our group has a strong track record of developing widely-used scientific software. Notable examples include: * [**PolyChord**](https://arxiv.org/abs/1506.00171): A next-generation nested sampling algorithm for Bayesian computation. * [**anesthetic**](https://arxiv.org/abs/1905.04768): A Python package for processing and visualizing nested sampling runs. * [**GLOBALEMU**](https://arxiv.org/abs/2104.04336): An emulator for the sky-averaged 21-cm signal. * [**maxsmooth**](https://arxiv.org/abs/2007.14970): A tool for rapid maximally smooth function fitting. * [**margarine**](https://arxiv.org/abs/2205.12841): For marginal Bayesian statistics using normalizing flows and KDEs. * [**fgivenx**](https://arxiv.org/abs/1908.01711): A package for functional posterior plotting. * [**nestcheck**](https://arxiv.org/abs/1804.06406): Diagnostic tests for nested sampling calculations. ### Impact and Discoveries Our research has led to significant advancements in cosmological data analysis and yielded new insights into the Universe. Key achievements include: * Pioneering the development and application of advanced Bayesian inference tools, such as [PolyChord](https://arxiv.org/abs/1506.00171), which has become a cornerstone for cosmological parameter estimation and model comparison globally. * Making significant contributions to the analysis of major cosmological datasets, including the [Planck mission](https://arxiv.org/abs/1807.06209), providing some of the tightest constraints on cosmological parameters and models of [inflation](https://arxiv.org/abs/1807.06211). * Developing novel AI-driven approaches for astrophysical challenges, such as using [machine learning for radiometer calibration in 21-cm experiments](https://arxiv.org/abs/2504.16791) and [simulation-based inference for extracting cosmological information from galaxy clusters](https://arxiv.org/abs/2504.10230). * Probing the nature of dark energy through innovative [non-parametric reconstructions of its equation of state](https://arxiv.org/abs/2503.08658) from combined datasets. * Advancing our understanding of the early Universe through detailed studies of [21-cm signals from the Cosmic Dawn and Epoch of Reionization](https://arxiv.org/abs/2301.03298), including the development of sophisticated foreground modelling techniques and emulators like [GLOBALEMU](https://arxiv.org/abs/2104.04336). * Developing new statistical methods for quantifying tensions between cosmological datasets ([Quantifying tensions in cosmological parameters: Interpreting the DES evidence ratio](https://arxiv.org/abs/1902.04029)) and for robust Bayesian model selection ([Bayesian model selection without evidences: application to the dark energy equation-of-state](https://arxiv.org/abs/1506.09024)). * Exploring fundamental physics questions such as potential [parity violation in the Large-Scale Structure using machine learning](https://arxiv.org/abs/2410.16030). ### Charting the Future: AI-Powered Cosmological Discovery The Handley Research Group is poised to lead a new era of cosmological analysis, driven by the explosive growth in data from next-generation observatories and transformative advances in artificial intelligence. Our future ambitions are centred on harnessing these capabilities to address the most pressing questions in fundamental physics. **Strategic Research Pillars:** * **Next-Generation Simulation-Based Inference (SBI):** We are developing advanced SBI frameworks to move beyond traditional likelihood-based analyses. This involves creating sophisticated codes for simulating [Cosmic Microwave Background (CMB)](https://arxiv.org/abs/1908.00906) and [Baryon Acoustic Oscillation (BAO)](https://arxiv.org/abs/1607.00270) datasets from surveys like DESI and 4MOST, incorporating realistic astrophysical effects and systematic uncertainties. Our AI initiatives in this area focus on developing and implementing cutting-edge SBI algorithms, particularly [neural ratio estimation (NRE) methods](https://arxiv.org/abs/2407.15478), to enable robust and scalable inference from these complex simulations. * **Probing Fundamental Physics:** Our enhanced analytical toolkit will be deployed to test the standard cosmological model ($\Lambda$CDM) with unprecedented precision and to explore [extensions to Einstein's General Relativity](https://arxiv.org/abs/2006.03581). We aim to constrain a wide range of theoretical models, from modified gravity to the nature of [dark matter](https://arxiv.org/abs/2106.02056) and [dark energy](https://arxiv.org/abs/1701.08165). This includes leveraging data from upcoming [gravitational wave observatories](https://arxiv.org/abs/1803.10210) like LISA, alongside CMB and large-scale structure surveys from facilities such as Euclid and JWST. * **Synergies with Particle Physics:** We will continue to strengthen the connection between cosmology and particle physics by expanding the [GAMBIT framework](https://arxiv.org/abs/2009.03286) to interface with our new SBI tools. This will facilitate joint analyses of cosmological and particle physics data, providing a holistic approach to understanding the Universe's fundamental constituents. * **AI-Driven Theoretical Exploration:** We are pioneering the use of AI, including [large language models and symbolic computation](https://arxiv.org/abs/2401.00096), to automate and accelerate the process of theoretical model building and testing. This innovative approach will allow us to explore a broader landscape of physical theories and derive new constraints from diverse astrophysical datasets, such as those from GAIA. Our overarching goal is to remain at the forefront of scientific discovery by integrating the latest AI advancements into every stage of our research, from theoretical modeling to data analysis and interpretation. We are excited by the prospect of using these powerful new tools to unlock the secrets of the cosmos. Content generated by [gemini-2.5-pro-preview-05-06](https://deepmind.google/technologies/gemini/) using [this prompt](/prompts/content/index.txt). Image generated by [imagen-3.0-generate-002](https://deepmind.google/technologies/gemini/) using [this prompt](/prompts/images/index.txt). ``` 2. **Paper Metadata:** ```yaml !!python/object/new:feedparser.util.FeedParserDict dictitems: id: http://arxiv.org/abs/2105.13923v2 guidislink: true link: http://arxiv.org/abs/2105.13923v2 updated: '2022-01-14T02:30:30Z' updated_parsed: !!python/object/apply:time.struct_time - !!python/tuple - 2022 - 1 - 14 - 2 - 30 - 30 - 4 - 14 - 0 - tm_zone: null tm_gmtoff: null published: '2021-05-27T15:06:03Z' published_parsed: !!python/object/apply:time.struct_time - !!python/tuple - 2021 - 5 - 27 - 15 - 6 - 3 - 3 - 147 - 0 - tm_zone: null tm_gmtoff: null title: "Nested sampling for frequentist computation: fast estimation of small\n\ \ $p$-values" title_detail: !!python/object/new:feedparser.util.FeedParserDict dictitems: type: text/plain language: null base: '' value: "Nested sampling for frequentist computation: fast estimation of small\n\ \ $p$-values" summary: 'We propose a novel method for computing $p$-values based on nested sampling (NS) applied to the sampling space rather than the parameter space of the problem, in contrast to its usage in Bayesian computation. The computational cost of NS scales as $\log^2{1/p}$, which compares favorably to the $1/p$ scaling for Monte Carlo (MC) simulations. For significances greater than about $4\sigma$ in both a toy problem and a simplified resonance search, we show that NS requires orders of magnitude fewer simulations than ordinary MC estimates. This is particularly relevant for high-energy physics, which adopts a $5\sigma$ gold standard for discovery. We conclude with remarks on new connections between Bayesian and frequentist computation and possibilities for tuning NS implementations for still better performance in this setting.' summary_detail: !!python/object/new:feedparser.util.FeedParserDict dictitems: type: text/plain language: null base: '' value: 'We propose a novel method for computing $p$-values based on nested sampling (NS) applied to the sampling space rather than the parameter space of the problem, in contrast to its usage in Bayesian computation. The computational cost of NS scales as $\log^2{1/p}$, which compares favorably to the $1/p$ scaling for Monte Carlo (MC) simulations. For significances greater than about $4\sigma$ in both a toy problem and a simplified resonance search, we show that NS requires orders of magnitude fewer simulations than ordinary MC estimates. This is particularly relevant for high-energy physics, which adopts a $5\sigma$ gold standard for discovery. We conclude with remarks on new connections between Bayesian and frequentist computation and possibilities for tuning NS implementations for still better performance in this setting.' authors: - !!python/object/new:feedparser.util.FeedParserDict dictitems: name: Andrew Fowlie - !!python/object/new:feedparser.util.FeedParserDict dictitems: name: Sebastian Hoof - !!python/object/new:feedparser.util.FeedParserDict dictitems: name: Will Handley author_detail: !!python/object/new:feedparser.util.FeedParserDict dictitems: name: Will Handley author: Will Handley arxiv_doi: 10.1103/PhysRevLett.128.021801 links: - !!python/object/new:feedparser.util.FeedParserDict dictitems: title: doi href: http://dx.doi.org/10.1103/PhysRevLett.128.021801 rel: related type: text/html - !!python/object/new:feedparser.util.FeedParserDict dictitems: href: http://arxiv.org/abs/2105.13923v2 rel: alternate type: text/html - !!python/object/new:feedparser.util.FeedParserDict dictitems: title: pdf href: http://arxiv.org/pdf/2105.13923v2 rel: related type: application/pdf arxiv_comment: "5 pages + refs, 3 figures. added refs + schematic of algorithm.\n\ \ closely matches published version" arxiv_journal_ref: Phys.Rev.Lett. 128 (2022) 2, 021801 arxiv_primary_category: term: physics.data-an scheme: http://arxiv.org/schemas/atom tags: - !!python/object/new:feedparser.util.FeedParserDict dictitems: term: physics.data-an scheme: http://arxiv.org/schemas/atom label: null - !!python/object/new:feedparser.util.FeedParserDict dictitems: term: astro-ph.IM scheme: http://arxiv.org/schemas/atom label: null - !!python/object/new:feedparser.util.FeedParserDict dictitems: term: hep-ex scheme: http://arxiv.org/schemas/atom label: null - !!python/object/new:feedparser.util.FeedParserDict dictitems: term: hep-ph scheme: http://arxiv.org/schemas/atom label: null - !!python/object/new:feedparser.util.FeedParserDict dictitems: term: stat.CO scheme: http://arxiv.org/schemas/atom label: null ``` 3. **Paper Source (TeX):** ```tex % nested sampling macros \newcommand{\nlive}{n_\text{live}} \newcommand{\niter}{n_\text{iter}} \newcommand{\ndynamic}{n_\text{dynamic}} \newcommand{\pvalue}{\text{\textit{p-}value}\xspace} \newcommand{\pvalues}{\text{\pvalue{}s}\xspace} \newcommand{\Pvalue}{\text{\textit{P-}value}\xspace} \newcommand{\Pvalues}{\text{\Pvalue{}s}\xspace} \newcommand{\Z}{\mathcal{Z}} \newcommand{\logZ}{\ensuremath{\log\Z}\xspace} \newcommand{\like}{\mathcal{L}} \newcommand{\threshold}{\like^\star} \newcommand{\pg}[2]{p\mathopen{}\left(#1\,\rvert\, #2\right)\mathclose{}} \newcommand{\Pg}[2]{P\mathopen{}\left(#1\,\rvert\, #2\right)\mathclose{}} \newcommand{\p}[1]{p\mathopen{}\left(#1\right)\mathclose{}} \newcommand{\intd}{\text{d}} \newcommand{\sampleParams}{\mathbf{x}} \newcommand{\modelParams}{\boldsymbol{\theta}} \newcommand{\param}{x} \newcommand{\stoppingtol}{\epsilon} \newcommand{\efr}{\ensuremath{\code{efr}}\xspace} \newcommand{\nr}{\ensuremath{n_r}\xspace} \newcommand{\expectation}[1]{\langle #1 \rangle} \newcommand{\MN}{\textsc{MultiNest}\xspace} \newcommand{\PC}{\textsc{PolyChord}\xspace} \newcommand{\ee}{\mathrm{e}}\documentclass[% reprint, %superscriptaddress, %groupedaddress, %unsortedaddress, %runinaddress, %frontmatterverbose, %preprint, %preprintnumbers, nofootinbib, %nobibnotes, %bibnotes, amsmath,amssymb, aps, %pra, %prb, %rmp, %prstab, %prstper, floatfix, prl, ]{revtex4-2} \bibliographystyle{apsrev4-2} % algorithm - before cleverref \usepackage[ruled,vlined,linesnumbered,noalgohanging]{algorithm2e} \newcommand{\myalcaptitle}[1]{\small\MakeUppercase{#1}} \SetAlCapSty{myalcaptitle} \newcommand{\myalcap}[1]{\small #1} \SetAlCapNameSty{myalcap} \SetAlCapHSkip{0pt} \SetAlgoCaptionSeparator{.} \setlength{\interspacetitleruled}{0.25em} \SetAlgoCaptionLayout{raggedright} \SetAlgoHangIndent{0pt} \usepackage{graphicx} \usepackage{bm} \usepackage{hyperref} \usepackage{xspace} \usepackage[capitalise]{cleveref} \usepackage{lmodern} \usepackage{microtype} \usepackage{fontawesome} \usepackage[T1]{fontenc} \usepackage{siunitx} \hypersetup{ colorlinks=true, allcolors=[rgb]{0.26,0.41,0.88}, } \sisetup{ range-phrase =\text{--} } \input{macros} % comments \usepackage[usenames]{xcolor} %TC:macro \AF [ignore] \newcommand{\AF}[1]{{\color{blue}\textbf{TODO AF:} \textit{#1}}} %TC:macro \SH [ignore] \newcommand{\SH}[1]{{\color{orange}[\textbf{SH:} #1]}} %TC:macro \WH [ignore] \newcommand{\WH}[1]{{\color{green}[\textbf{WH:} #1]}} % add back in section numbers \setcounter{secnumdepth}{3} % styling of treating cite as a noun \newcommand{\refcite}{\cite} % word limit %TC:macro \wordlimit [ignore] \newcommand{\wordlimit}[1]{} %{{\color{red}[\textit{#1}]}} \begin{document} \title{Nested sampling for frequentist computation: fast estimation of small \pvalues} \author{Andrew Fowlie} \affiliation{Department of Physics and Institute of Theoretical Physics, Nanjing Normal University, Nanjing, Jiangsu 210023, China} \email{andrew.j.fowlie@njnu.edu.cn} \author{Sebastian Hoof} \affiliation{Institut f\"{u}r Astrophysik, Georg-August-Universit\"{a}t {G\"{o}ttingen},\\Friedrich-Hund-Platz~1, 37077~{G\"{o}ttingen}, Germany} \author{Will Handley} \affiliation{Cavendish Laboratory \& Kavli Institute for Cosmology, University of Cambridge,\\ JJ Thomson Avenue, Cambridge, CB3~0HE, United Kingdom} % \date{\today} \begin{abstract} We propose a novel method for computing \pvalues based on nested sampling (NS) applied to the sampling space rather than the parameter space of the problem, in contrast to its usage in Bayesian computation. The computational cost of NS scales as $\log^2{1/p}$, which compares favorably to the $1/p$~scaling for Monte Carlo (MC) simulations. For significances greater than about $4\sigma$ in both a toy problem and a simplified resonance search, we show that NS requires orders of magnitude fewer simulations than ordinary MC estimates. This is particularly relevant for high-energy physics, which adopts a $5\sigma$ gold standard for discovery. We conclude with remarks on new connections between Bayesian and frequentist computation and possibilities for tuning NS implementations for still better performance in this setting. \href{https://github.com/andrewfowlie/ns_for_p_values}{\faGithub} \end{abstract} \maketitle \section{Introduction}\label{sec:intro} For decades, \pvalues have played a vital role in the discovery of new phenomena in high-energy physics~(HEP)~\citep{Lyons:1986em,Cowan:2013pha,Cranmer:2015nia,Cousins:2018tiz,Zyla:2020zbs,junk} as well as many other fields and disciplines. A~\pvalue is the probability under a null hypothesis of observing results that are at least as ``extreme'' as the observed data. When the observed \pvalue is smaller than a pre-specified threshold $\alpha$ we reject the null hypothesis and claim a discovery. As \wordlimit{brought to the attention of the public} popularized by the Higgs boson discovery~\citep{Aad:2012tfa,Chatrchyan:2012ufa}, HEP usually requires $\alpha \sim 10^{-7}$, also known as the ``$5\sigma$ rule''~\citep{Lyons:2013yja}.\footnote{In HEP, it is conventional to convert \pvalues into significances by the rule $Z = \Phi^{-1}(1 - p)$, where $\Phi$ is the standard normal cumulative distribution function.} Establishing a discovery thus requires the computation of $p \lesssim 10^{-7}$. This exposes difficulties in standard approaches to computing \pvalues, including the look-elsewhere effect~\cite[see e.g.,][]{Algeri:2016gtj}, broken assumptions in popular asymptotic results~\citep[see e.g.,][]{Cowan:2010js} %such as Wilks'\ theorem~\citep{Wilks:1938dza}, and the computational cost of computing \pvalues through Monte Carlo (MC) simulations. %The latter requires $\gtrsim 1 / p$ simulations. %, which would be extremely computationally demanding for the tiny \pvalues required for discovery in HEP. To overcome these problems, semi-analytic asymptotic formulae were developed alongside the Higgs searches~\cite[e.g., Gross--Vitells][]{Gross:2010qma}. However, as discussed in the reviews \refcite{Cranmer:2015nia,Cousins:2018tiz,Zyla:2020zbs}, MC simulations are often unavoidable, as the asymptotic formulae make assumptions that often do not hold or that are difficult to check\wordlimit{, as discussed in detail in}~\refcite{2020NatRP...2..245A}. Straightforward examples include small sample sizes that cannot justify taking the asymptotic limit or when the hypotheses under consideration are not nested. \wordlimit{Since it is impossible to select and summarize prominent examples from the diverse range of disciplines using \pvalues, we only mention some recent examples in physics.} For example, MC simulations were used by ATLAS and CMS at the LHC in e.g., searches for deviations from the Standard Model (SM) of particle physics~\cite{1807.07447,2010.02984}, searches for supersymmetry~\cite{1405.3961} and measurements of the Higgs boson's properties~\cite{1212.6639,1405.3961}. Outside collider physics, they were used in searches for dark matter by the XENON collaboration~\cite{2006.09721,2105.00599} and in astronomy by Fermi-LAT~\cite{1611.03184} and IceCube~\cite{2107.08159}. Lastly, MC simulations are used in global fits in particle physics~e.g., \refcite{1508.05951,2007.05517,2104.05631,1311.1822,1506.07685,2008.06083}, including \textsc{Gfitter} fits of the SM~\cite{1407.3792}. In this \textit{Letter} we present a novel technique for computing global or local \pvalues based on nested sampling (NS) that in some regimes performs exponentially better than MC simulations. \wordlimit{In \cref{sec:pvalues} we describe our technique and compare it with MC simulations. We then assess the usability of our algorithm for benchmark cases in \cref{sec:examples}. Finally, we conclude in \cref{sec:conc}, including a brief discussion of limitations of our approach and an explanation of its success by rephrasing the problem in terms of \emph{compression}.} A basic implementation of our algorithm, including the examples provided in this \textit{Letter}, is available on Github~\cite{code}. \section{\pvalues}\label{sec:pvalues} Mathematically, \pvalues can be defined as the probability that a test-statistic (TS) $\lambda$ is at least as great as the \emph{observed} TS $\lambda^\star$ assuming that the null hypothesis $H_0$ is true, \begin{equation}\label{eq:pval_def} p =\Pg{\lambda \ge \lambda^\star}{H_0} \, . \end{equation} The task at hand is therefore to determine a tail probability from the sampling distribution of~$\lambda$ under~$H_0$. \wordlimit{Typically this involves generating simulated sets of data $\sampleParams$ from the \emph{sampling space} of the null hypothesis, and using these to estimate the fraction for which $\lambda(\sampleParams)\ge\lambda^\star$ holds true.} \subsection{\Pvalues from Monte Carlo simulations}\label{sec:mc} Using MC simulations to \wordlimit{correctly} determine the sampling distribution requires no asymptotic assumptions and \wordlimit{generally} provides a robust and reliable \pvalue estimate~\refcite{Algeri:2016gtj}. These estimates obey binomial statistics since for each simulation either $\lambda \geq \lambda^\star$ or not. The \pvalue can then be estimated by $\hat{p} = m/n$ for $m$ occurrences of $\lambda \geq \lambda^\star$ from $n$ simulations. Evidently, a meaningful estimate requires at least $1/p$ simulations as the fractional error on~$\hat{p}$ is of order of the Wald estimate, \begin{equation}\label{eq:mc_pval_err} \frac{\Delta p}{p} = \sqrt{\frac{1 / p}{n}} \, . \end{equation} See \refcite{10.1214/ss/1009213286} for further discussion of errors for binomial parameters. \subsection{\Pvalues from nested sampling}\label{sec:ns} \begin{figure*} \centering \includegraphics[width=7.0in]{ill.pdf} \caption{Schematic illustration of MC~(\textit{left}) and NS~(\textit{right}) methods for computing \pvalues. The blue curve represents the probability distribution of the test statistic~$\lambda$. The observed, critical value of~$\lambda$ is indicated by a thick, red, vertical line and the area beyond it~(red shaded region) is the \pvalue. The shaded yellow region in the right panel illustrates the compression of the area under the curve at each step of the NS algorithm~(progressively darker shading).} \label{fig:ill} \end{figure*} The nested sampling~(NS) algorithm~\citep{2004AIPC..735..395S,Skilling:2006gxv} has primarily enjoyed success as a tool for computing the Bayesian evidence~(see e.g.~\refcite{Kass:1995loi}), which is estimated through an identity involving the volume variable \begin{equation}\label{eq:X} X(\like^\star) = \int_{\like(\modelParams) > \like^\star} \pi(\modelParams) \, \intd\modelParams \, , \end{equation} where $\pi$ is the prior distribution, $\like$ is the likelihood and $\modelParams$ are the model parameters. The threshold $\like^\star$ increases at each iteration of the algorithm. A scheme to estimate $X$ % from the samples drawn from the sequence of constrained priors lies at the heart of NS. See \refcite{2021arXiv210109675B} for a recent review. To understand why this is useful for \pvalue calculations, let us re-interpret \cref{eq:X} in a frequentist context. Rather than thinking of it as an integral over the parameter space, consider it \wordlimit{as} an integral over the sampling space. To give a concrete example, if we wish to simulate five Gaussian measurements, the sampling space would be five-dimensional and $\modelParams$ would consist of the five simulated Gaussian measurements $\sampleParams$ rather than model parameters. We thus re-write \cref{eq:X} as \begin{equation}\label{eq:p_value_X} p = \Pg{\lambda > \lambda^\star}{H_0} = \int_{\lambda(\sampleParams) > \lambda^\star} \pg{\sampleParams}{H_0} \, \intd\sampleParams \, , \end{equation} which allows us to use NS to estimate \pvalues. A comparison of \cref{eq:X,eq:p_value_X} reveals that the pseudo-data~$\sampleParams$ play the role of the model parameters~$\modelParams$, the sampling distribution of the pseudo-data $\pg{\sampleParams}{H_0}$ plays the role of the prior and, lastly, the TS $\lambda(\sampleParams)$ plays the role of the likelihood. Thus in the Bayesian setting, the volume variable in \cref{eq:X} is the fraction of the prior in which the likelihood exceeds a threshold, whereas in the frequentist one in \cref{eq:p_value_X}, it is the fraction of the sampling distribution in which the TS exceeds a threshold. % \footnote{Note that we don't use the important NS identity, % \begin{equation*} % \mathcal{Z} = \int \like(\sampleParams) \pi(\sampleParams) \intd\sampleParams = \int \like(X) dX\, , % \end{equation*} % and that the algorithm is invariant under monotonic transformations of the likelihood. % } The NS scheme for a statistical estimate of the volume variable works as follows. First, we draw $\nlive \sim \mathcal{O}(\numrange{100}{1000})$ ``live points'' from the sampling space. The parameter $\nlive$ acts as a resolution parameter, such that the runtime scales approximately as $\nlive$, the uncertainty as $1 / \sqrt{\nlive}$ and the chances of missing modes in multi-modal problems as $1/\nlive$. At each subsequent iteration, the live point with the least extreme TS is replaced by one with a more extreme TS drawn from the sampling space (i.e.\ a draw from the constrained prior). This continues until iteration $\niter$, when the least extreme TS among the live points exceeds the observed TS. \wordlimit{We denote the iteration at which that occurs by $\niter$.} We estimate the \pvalue using the usual NS estimate of the volume,\footnote{This follows from $\ee^{\langle\log X\rangle}$, i.e., from an unbiased estimator of $\log X$. % There exists an unbiased estimator of $X$, but the relative difference is of order $1 / \nlive$. % Alternatively, one could estimate % \begin{equation*} % \langle X(\like^\star) \rangle = \left(\frac{\nlive}{\nlive + 1}\right)^{\niter}\, . % \end{equation*} } \begin{equation}\label{eq:p_NS} p = X(\lambda^\star) \simeq \prod_{i=1}^{\niter} \ee^{-1/\nlive} = \ee^{-\niter / \nlive} \, . \end{equation} Thus NS decomposes a tiny \pvalue into a product of $\niter$ moderate factors. This algorithm is shown schematically in \cref{algo:ns_pvalues}. \begin{algorithm} \caption{Schematic nested sampling algorithm for estimating \pvalues.\label{algo:ns_pvalues}} \SetAlgoLined Draw $\nlive$ sets of pseudo-data from the sampling distribution --- the live points\; Initialize $\niter = 0$\; \Repeat{ $\lambda_\mathrm{min} \ge \lambda^\star$ }{ $\niter = \niter + 1$\; Find the minimum TS $\lambda_\mathrm{min}$ amongst the live points\; Replace live point corresponding to $\lambda_\mathrm{min}$ by one drawn from the sampling distribution subject to $\lambda > \lambda_\mathrm{min}$\; } \KwRet{Estimate of $p = \ee^{-\niter / \nlive}$}\; \end{algorithm} \wordlimit{In more practical terms, we initialize NS with the TS in place of the likelihood and the sampling distribution in place of the prior. As the NS algorithm runs, the threshold in \cref{eq:p_value_X} increases monotonically. We stop once we reach a TS of~$\lambda^\star$, where we may choose~$\lambda^\star$ as the observed TS in an experiment or some arbitrary value for the purpose of calibrating the distribution of the TS before performing an experiment.} In the NS algorithm, $\niter$ is approximately Poisson-distributed with expectation value and variance both equal to $\nlive \log 1 / p$. From \cref{eq:p_NS} we see that $\log p$ follows a normal distribution with error \begin{equation}\label{eq:ns_pval_err} \frac{\Delta p}{p} \approx \Delta \log p = \frac{\Delta \niter}{\nlive} = \sqrt{\frac{\log 1/p}{\nlive}} \, , \end{equation} with the approximation holding when $\Delta p / p \lesssim 1$. However, the number of TS evaluations required in an NS run $n_\text{eval}$ must be proportional to the number of iterations before stopping. In fact, from the known properties of NS, \begin{equation} n_\text{eval} = \niter / \epsilon = (\nlive \log 1 / p) / \epsilon \, , \end{equation} where $\epsilon$ denotes a problem-specific efficiency factor.\footnote{This neglects the (generally negligible) $\nlive$ evaluations required for the initial live points.} Thus we actually have \begin{equation} \frac{\Delta p}{p} \approx \Delta \log p = \sqrt{\frac{\log^21/p}{\epsilon \, n_\text{eval}}} \, . \end{equation} \wordlimit{The theoretical foundations for NS in this setting are further strengthened by the connection to a form of subset simulation~\cite{Au_2001,beck2015rare}, a technique from rare event sampling that achieves similar $\log^r{1/p}$ scaling with $r = \numrange{2}{3}$.} % Also relevant here \cite{Bect_2017}. To compare the theoretical performances of MC and NS, we consider the ratio of the number of evaluations required to achieve a fixed relative error on the \pvalue. Note that the TS could be based on the profiled likelihood such that each evaluation could involve a complicated minimization over the actual model parameters. Comparing the error estimates from \cref{eq:mc_pval_err,eq:ns_pval_err}, we find a ratio of \begin{equation}\label{eq:pval_err_ratio} \frac{\text{Evaluations for NS}}{\text{Evaluations for MC}} = \frac{(\log^21/p) / \epsilon}{1 / p}\, . \end{equation} % The relative number of evaluations $r \rightarrow 0$ when $p\rightarrow 0$, i.e.\ For small enough \pvalues, NS has the potential to beat MC by an arbitrarily large margin. The efficiency factor, however, could spoil NS performance in realistic applications % NS wins for $\epsilon \ge p \log^2 1/ p$.\footnote{ % $p \le \epsilon / (4 W^2_{-1}(-\sqrt{\epsilon} / 2)) \approx \epsilon / \log^2 \epsilon$ a % Note that $p \log^2 1/ p$ reaches a maximum at $4 / e^2$ such that NS always wins if $\epsilon \ge 4 / e^2 \simeq 0.54$.} and depends the details of the NS implementation and the problem at hand, including the dimensionality. Because of these complications, we will consider the computational performance for two benchmark cases in \cref{sec:examples}. We summarize and illustrate the idea of ``nested \pvalue computation'' in comparison to MC in \cref{fig:ill}. We want to emphasize that, in the context of this work, \emph{NS is merely a mathematical tool}. \wordlimit{We do not require the evidence estimate or any aspects of Bayesian model comparison, and we do not assume or exploit any connections between the Bayesian evidence and \pvalue. However, there are clearly underappreciated connections between Bayesian and frequentist computation, on which we comment briefly in \cref{sec:conc}.} \section{Applications and Performance}\label{sec:examples} To illustrate how to use our algorithm and to assess its performance, we conduct two case studies. First, we consider the \wordlimit{well-studied} example of multi-dimensional Gaussians. %, for which asymptotic results hold. Second\wordlimit{, as a more realistic HEP example,} we apply our algorithm to a simplified version of a resonance search. %, including a goodness-of-fit test for which asymptotic results do not hold. \subsection{Gaussian measurements}\label{sec:gausian_measurements} \begin{figure} \centering \includegraphics[width=3.375in]{performance.pdf} \caption{Performance of NS versus MC for the $\chi^2_d$ example. The performance of perfect NS and MC are independent of $d$. Performance was measured by the TS evaluations required to compute the \pvalue to within $10\%$ uncertainty. \wordlimit{NS is orders of magnitude faster for significances greater than about $4\sigma$ and moderate dimension.}} \label{fig:performance} \end{figure} Consider the \pvalue associated with $d$ independent Gaussian measurements. We define the TS as the sum of the $d$ associated chi-squared variates. Thus $\lambda \sim \chi^2_d$ and the \pvalue may be computed analytically. We construct the problem in NS by mapping from draws from a $d$ dimensional unit hypercube with coordinates $\param_i \sim \mathcal{U}(0, 1)$, to $d$ draws from a chi-squared distribution via $z_i = F_{\chi^2_1}^{-1}(\param_i)$ for $i = 1, \ldots, d$ such that $z_i \sim \chi^2_1$ and $\lambda = \sum z_i \sim \chi^2_d$. We show the performance of NS and MC on this problem in \cref{fig:performance} as a function of the significance. The performance of perfect NS (in which we suppose that one could sample directly from the constrained prior) and MC are independent of $d$. The performance of real implementations of NS however depends on $d$, as they must utilize numerical schemes for sampling from the $d$-dimensional constrained prior. We demonstrate the performance of \MN~\cite{Feroz:2007kg,Feroz:2013hea} and \PC~\cite{Handley:2015fda}, which utilize ellipsoidal and slice sampling, respectively.\footnote{We changed the codes to stop once the required TS threshold was reached. \wordlimit{The number of live points was dictated by the desired uncertainty. In general, we suggest at least about $\nlive = 100$ or at least $\nlive > d$ as the live points guide the sampling from the constrained prior.} While we chose $\nlive$ to achieve the desired uncertainty, we generally suggest to choose $\nlive \geq \mathrm{max}(100,d)$. We set $\texttt{efr} = 0.3$ and $\texttt{num\_repeats} = 5d$ and \texttt{do\_clustering = False}. See the associated codes for our complete settings~\cite{code}.} We use the number of TS evaluations as a measure of performance, which is sensible when the computational cost is dominated by TS evaluations. While this is usually a good assumption in practice, we note that \MN appears to suffer from a greater computational overhead than \PC in our example, possibly due to the linear algebra operations required for clustering live points and constructing ellipsoids, especially for $d = 30$ over $5\sigma$. For this case, \MN was about 2,000~times slower than \PC~(with sizeable variability in run times), which significantly exceeds the ratio of TS evaluations needed. \subsection{Resonance search}\label{sec:higgs} \wordlimit{Our second example is a simplified version of} The original Higgs discovery by the ATLAS Collaboration~\cite{Aad:2012tfa} \wordlimit{in the diphoton channel. This} is a typical example of a resonance search (testing a spectrum for the presence of a signal above a smooth background), which includes the common complications of parameters on the boundary of the parameter space and the look-elsewhere effect. % Here, the null hypothesis~$H_0$ corresponds to the Standard Model~(SM) \wordlimit{background-only} hypothesis, with a known shape and an unknown nuisance parameter~$b$, the total number of background events. The alternative hypothesis $H_1$ is that of the SM background plus a Higgs boson with a Gaussian signal, with a known width but an unknown mass $m_h$ and an unknown positive signal strength $s \ge 0$.\footnote{In the original analysis, the signal is described by a Crystal~Ball function and the background model is more complex~\cite{Aad:2012tfa}. \wordlimit{We ignore these details to provide an accessible description.}} The null hypothesis lies at the $s = 0$ boundary. The data $\sampleParams = \{n_i\}$ consists of the Poisson-distributed observed counts~$n_i$ in the 30~bins shown in Figure~4 of~\cite{Aad:2012tfa}. The TS is the log-likelihood ratio \begin{equation}\label{eq:ts_llr_higgs} \lambda(\sampleParams) = 2\log\left(\frac{\max P(\sampleParams \, |\, b, s, m_h) }{\max P(\sampleParams \, |\, b, s = 0)}\right) \, , \end{equation} where we maximize over all parameters for a given (pseudo-)data set~$\sampleParams$. In summary, the data space~$\sampleParams$, and hence the pseudo-data sampling space, is 30-dimensional while the null and alternative models have only one and three model parameters respectively. % \footnote{Note that, in this problem, the TS has a plateau at zero, as the signal strength must be positive. For the local significance, that plateau occupies half of the sampling space, and so an implementation of NS that supports plateaus must be used or a correction of $\log(0.5) - 0.5$ must be added by hand to $\log X$~\cite{Fowlie:2020gfd}.} To compute \pvalues in the presence of a nuisance parameter~\cite{Demortier:1099967}, we plug in the best-fit value of the unknown nuisance parameter $b$ in \cref{eq:pval_def}. We perform brute-force MC simulations and NS with \MN and \PC to calibrate the TS~$\lambda$. The results are shown in \cref{fig:resonance_search}. \wordlimit{, where we use the conservative Clopper--Pearson intervals~\cite{Clopper1934,10.1214/ss/1009213286} to estimate the MC uncertainty instead of \cref{eq:mc_pval_err}.} \begin{figure} \centering \includegraphics[width=3.375in]{higgs_nested_vs_mc} \caption{Calibration of the resonance search TS. We show the estimated \pvalue with shaded uncertainties for MC simulations~(gray; \num{2e9} samples), \PC~(red; weighted average of four runs), and \MN~(blue; single run). } \label{fig:resonance_search} \end{figure} To reach a TS of $\lambda = 50$~($6.5\sigma$) \PC requires about \num{3e6} TS evaluations. \MN typically requires around \num{3.5e7} TS evaluations, i.e.\ an order of magnitude more. \wordlimit{However, this number can sometimes be a factor of a few larger or smaller, due to a less stable performance for \MN, which is not unexpected in higher dimensions.} To achieve a similar level of uncertainty as a single NS run for $\lambda = 50$, where $\Delta\log_{10} p \approx 0.2$ for both \MN and \PC, we would require about \num{e11}~MC~simulations. Since this would be computationally fairly expensive, we only simulated \num{2e9}~samples for \cref{fig:resonance_search}. The improved performance for NS vs MC in terms of TS evaluations is in line with \cref{fig:performance} and also with CPU run times required: a typical \PC run took about 300\,CPUh~(equivalent hours on a single computing core) while the required \num{e11}~MC~simulations would take around \num{9.2e6}\,CPUh, such that we achieved a speed-up of about 31,000. Note that the gradient of the linear slope in \cref{fig:performance} also agrees with the asymptotic Gross--Vitells method~\cite{Gross:2010qma}, $\log p \approx c_0 - \frac12 \lambda$. \wordlimit{, where the unknown constant~$c_0$ must be computed through simulations.} Whilst this method may be used to efficiently compute \pvalues in the resonance search example, it is not applicable in other cases of interest\wordlimit{. When, e.g., the hypotheses under consideration are not nested, one usually has to resort to MC simulations}~\cite{2020NatRP...2..245A}. As a concrete example, consider the goodness-of-fit test of the SM plus Higgs hypothesis, which can be performed using the TS~\cite{Baker:1983tu} \begin{equation} \lambda(\sampleParams) = -2\log\left(\frac{\max P(\sampleParams \, |\, b, s, m_h) }{ P(\sampleParams \, |\, \mu_i = n_i )}\right) \, , \label{eq:higgs_gof} \end{equation} with $b$, $s$, and $m_h$ are as in \cref{eq:ts_llr_higgs}. \wordlimit{The difference now is that we normalize the TS by the Asimov data set, i.e.} We set the $\mu_i$ parameters in the Poisson distribution for each bin equal to the pseudo-counts data~$n_i$ in that bin. Since the $\mu_i$ are different from the model parameters of the Higgs hypothesis, these two models are not nested. The na\"ive expectation for the goodness-of-fit test is that it follows a $\chi_d^2$ distribution with $d = 30 - 3 = 27$ degrees of freedom \wordlimit{~(DOF), subtracting the ``parameter DOF'' from the ``data DOF.''} The implied critical value of the TS for $5\sigma$~($p \approx \num{28.7e-8}$) is then $\lambda^\star \approx 80.8$. However, with direct MC simulations and \PC, we estimate the corresponding \pvalues to be $p = (8.4 \pm 1.0) \times 10^{-8}$ and $p = (6.6 \pm 1.1) \times 10^{-8}$, respectively, or around $5.3\sigma$. This discrepancy from the asymptotic result should be anticipated, as the SM plus Higgs is not a linear model~\cite{andrae2010dos}. \wordlimit{Our goodness-of-fit study also illustrates the benefits of a proper TS calibration as a $5\sigma$ rejection could have already been claimed for $\lambda \gtrsim 76$ as opposed to $\lambda \gtrsim 80.8$.} The six \PC runs that we used for this estimate took on average about \num{1.3e6} function calls and \num{220}\,CPUh. The CPU time per TS function call is consistent with our MC analysis, where we simulated \num{8.2e8} samples in total. \wordlimit{This indicates again that the TS evaluation time dominates over potential computational overhead of \PC, as expected for a TS in realistic applications.} The achieved speed-up of about~\num{100} is consistent with \cref{fig:performance}. \section{Discussion and conclusions}\label{sec:conc} In this \textit{Letter}, we propose the use of nested sampling~(NS) for computing \pvalues. Our proposal is compatible with readily available, established implementations of the NS~algorithm. \wordlimit{We demonstrated the validity of our algorithm and estimated its speed and uncertainty. The code to reproduce our results is available at \refcite{code}.} The most notable advantage of using NS is the order-of-magnitudes speed-up \wordlimit{and increase in accuracy} that can be achieved for small \pvalues of scientific importance in comparison with Monte~Carlo~(MC) simulations. This advantage can be traced to the fact that, despite their conceptual differences, \pvalues and the Bayesian evidence have one thing in common that makes them difficult to compute: \emph{compression}. For the Bayesian evidence, the compression occurs between prior and posterior distributions, % \begin{equation} % H = \int \pg{\sampleParams}{D} \log \left(\frac{\pg{\sampleParams}{D}}{\p{\sampleParams}}\right) \intd\sampleParams % \end{equation} and is typically enormous if the data favor a narrow region of the parameter space. % \SH{This is often the case when prior distribution are chosen to be broad and ``uninformative'' while the data prefers a much narrower region of parameter space.} % It means that simulation from the prior is inefficient. For \pvalues, the compression is from the whole sampling space to the region where the test statistic~(TS) is greater than its observed value. % (see \cref{eq:pval_def}). By definition, the compression is enormous for small \pvalues. % In terms of the entropy, the compression is % \begin{equation} % H = \log 1/p % \end{equation} % The large compression means that simulating from the sample space and finding the \pvalue by brute force can be inefficient. We can understand the applicability of NS to compression problems in both Bayesian and frequentist settings as follows: simulating from the entire sampling distribution (or the entire prior in the Bayesian setting) is inefficient when the interesting region of sampling space is tiny. Conversely, simulating only from the region of interest (or from the posterior in the Bayesian setting) does not allow reliable inference about its relative size. Heuristically, it is not surprising that successful strategies simulate from a series of distributions that form a path between e.g.\ the prior and posterior % (NS, Sequential Monte Carlo, annealed importance sampling and thermodynamic integration; (see \refcite{10.1214/13-STS465,10.1214/ss/1028905934} for further discussion). % By doing so, they manage to compute a tiny probability as the product of many moderate ones. % NS is unique, however, in that the bridging distributions are in fact the constrained prior. % (and we don't bridge to the posterior). Since the constrained prior can be related to the \pvalue by \cref{eq:p_value_X}, the sequence of constrained priors in NS naturally forms a path between the entire sampling space and the tiny tail area of interest corresponding to the \pvalue, which makes it particularly well-suited for frequentist computation. Path sampling in this manner is a generalization of importance sampling~\cite{10.1214/ss/1028905934}. See \refcite{2013CoPhC.184.2438W} for related work on computing \pvalues through importance sampling and \refcite{Feroz:2013hea} for an importance sampling algorithm that uses NS draws. % It would be possible to design annealing schedules so that other algorithms for Bayesian computation could make similar bridging distributions, though the athermal nature of NS may be an advantage. We see three potential drawbacks when using NS in this setting. First, the data space in a realistic problem can be very high-dimensional. For example, a search for new particles at the LHC could look for evidence in a histogram containing 100 bins or more. While some implementations of the NS scheme potentially suffer from poor efficiency in higher dimensions, using e.g.\ NS with slice sampling achieves $\epsilon \propto 1 / d$ behavior and has been applied successfully to problems with hundreds of dimensions~\cite{2020arXiv200412211J}. %More generally, our idea naturally inherits other possible drawbacks of NS. Second, while an NS run with $\nlive$ live points can be efficiently parallelized into in principle as many as $\nlive$ runs with a single live point, those individual runs \wordlimit{cannot be further parallelized and} must proceed linearly.\footnote{The uncertainty of the estimate, however, may \wordlimit{always} be reduced by \wordlimit{simultaneously} running the algorithm multiple times \wordlimit{on more processing units}.} By using exponentially large amounts of computational resources, it is therefore possible to make brute-force MC compute \pvalues faster than any realistic NS algorithm, albeit at significantly greater overall computational expense. % In practice, however, this may only be relevant for NS algorithms that do not scale well with the number of parameter dimensions. Lastly, there is a subtlety concerning substantial plateaus in the TS, that is, regions of sampling space that lead to an identical TS. In some NS implementations plateaus lead to faulty estimates of the volume variable~\cite{Fowlie:2020gfd}, and thus would lead to faulty estimates of the \pvalue. In such cases, an implementation of NS that supports plateaus must be used or the \pvalue must be corrected using e.g., \textsc{anesthetic}~\cite{Handley:2019mfs}. % Furthermore, the sampling space in some problems may be trans-dimensional i.e.\ the number of dimensions itself is a random variable~(see discussion around Eq.~(40.11) in \refcite{Zyla:2020zbs}). % Consider, for example, an unbinned search for a resonance in the invariant mass distribution of pairs of photons. The total number of observed events is a random Poisson variable and each event has an invariant mass associated with it~(see discussion around Eq.~(40.11) in \refcite{Zyla:2020zbs}). % Whilst there are strategies for trans-dimensional NS~\cite{2015MNRAS.448.3206B}, and also be cause this problem could be avoided by re-binning the data, we leave their consideration to future work. Despite its drawbacks, the in-principle exponential improvement makes our method a valuable tool---in particular when considering small \pvalues. We demonstrated in practice that our algorithm can significantly reduce the computational burden of \pvalue~calculations with the popular \MN and \PC algorithms. This allows for a straightforward adoption of our algorithm, encouraging more rigorous \pvalue calculations and potentially opening up problems that were previously computationally unfeasible. \wordlimit{On a more conceptual note, our algorithm links Bayesian and frequentist statistics through the common computational challenge of compression, and their common solution of utilizing a bridging sequence of distributions. The NS algorithm is particularly appropriate for this task due to a connection between the constrained prior and the \pvalue. %In the Bayesian setting, it operates on the parameter space of the model using the likelihood function. In the frequentist one, it operates on the sampling space using the TS. While NS is merely a mathematical tool for our purposes, the existence of such a useful connection between Bayesian and frequentist methods hidden in plain sight may inspire further research and potentially uncover more connections between computational methods in the two rivaling statistical interpretations.} \section*{Acknowledgments} AF was supported by an NSFC Research Fund for International Young Scientists grant 11950410509. SH was funded by the Alexander von Humboldt Foundation and the German Federal Ministry of Education and Research. WH was supported by a Royal Society University Research Fellowship. We used the Scientific Computing Cluster at GWDG, the joint data centre of Max Planck Society for the Advancement of Science~(MPG) and the University of G\"ottingen. This work was also performed using the Cambridge Service for Data Driven Discovery (CSD3), part of which is operated by the University of Cambridge Research Computing on behalf of the STFC DiRAC HPC Facility (www.dirac.ac.uk). The DiRAC component of CSD3 was funded by BEIS capital funding via STFC capital grants ST/P002307/1 and ST/R002452/1 and STFC operations grant ST/R00689X/1. DiRAC is part of the National e-Infrastructure \bibliography{references} \end{document} ``` 4. **Bibliographic Information:** ```bbl %apsrev4-2.bst 2019-01-14 (MD) hand-edited version of apsrev4-1.bst %Control: key (0) %Control: author (72) initials jnrlst %Control: editor formatted (1) identically to author %Control: production of article title (-1) disabled %Control: page (0) single %Control: year (1) truncated %Control: production of eprint (0) enabled \begin{thebibliography}{46}% \makeatletter \providecommand \@ifxundefined [1]{% \@ifx{#1\undefined} }% \providecommand \@ifnum [1]{% \ifnum #1\expandafter \@firstoftwo \else \expandafter \@secondoftwo \fi }% \providecommand \@ifx [1]{% \ifx #1\expandafter \@firstoftwo \else \expandafter \@secondoftwo \fi }% \providecommand \natexlab [1]{#1}% \providecommand \enquote [1]{``#1''}% \providecommand \bibnamefont [1]{#1}% \providecommand \bibfnamefont [1]{#1}% \providecommand \citenamefont [1]{#1}% \providecommand \href@noop [0]{\@secondoftwo}% \providecommand \href [0]{\begingroup \@sanitize@url \@href}% \providecommand \@href[1]{\@@startlink{#1}\@@href}% \providecommand \@@href[1]{\endgroup#1\@@endlink}% \providecommand \@sanitize@url [0]{\catcode `\\12\catcode `\$12\catcode `\&12\catcode `\#12\catcode `\^12\catcode `\_12\catcode `\%12\relax}% \providecommand \@@startlink[1]{}% \providecommand \@@endlink[0]{}% \providecommand \url [0]{\begingroup\@sanitize@url \@url }% \providecommand \@url [1]{\endgroup\@href {#1}{\urlprefix }}% \providecommand \urlprefix [0]{URL }% \providecommand \Eprint [0]{\href }% \providecommand \doibase [0]{https://doi.org/}% \providecommand \selectlanguage [0]{\@gobble}% \providecommand \bibinfo [0]{\@secondoftwo}% \providecommand \bibfield [0]{\@secondoftwo}% \providecommand \translation [1]{[#1]}% \providecommand \BibitemOpen [0]{}% \providecommand \bibitemStop [0]{}% \providecommand \bibitemNoStop [0]{.\EOS\space}% \providecommand \EOS [0]{\spacefactor3000\relax}% \providecommand \BibitemShut [1]{\csname bibitem#1\endcsname}% \let\auto@bib@innerbib\@empty % \bibitem [{\citenamefont {Lyons}(1986)}]{Lyons:1986em}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {L.}~\bibnamefont {Lyons}},\ }\href@noop {} {\emph {\bibinfo {title} {{Statistics for Nuclear and Particle Physicists}}}}\ (\bibinfo {publisher} {Cambridge University Press},\ \bibinfo {year} {1986})\BibitemShut {NoStop}% \bibitem [{\citenamefont {Cowan}(2013)}]{Cowan:2013pha}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {G.}~\bibnamefont {Cowan}},\ }in\ \href {https://doi.org/10.1007/978-3-319-05362-2_9} {\emph {\bibinfo {booktitle} {{69th Scottish Universities Summer School in Physics}: {LHC Physics}}}}\ (\bibinfo {year} {2013})\ pp.\ \bibinfo {pages} {321--355},\ \Eprint {https://arxiv.org/abs/1307.2487} {arXiv:1307.2487 [hep-ex]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Cranmer}(2014)}]{Cranmer:2015nia}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {K.}~\bibnamefont {Cranmer}},\ }in\ \href {https://doi.org/10.5170/CERN-2014-003.267} {\emph {\bibinfo {booktitle} {{2011 European School of High-Energy Physics}}}}\ (\bibinfo {year} {2014})\ pp.\ \bibinfo {pages} {267--308},\ \Eprint {https://arxiv.org/abs/1503.07622} {arXiv:1503.07622 [physics.data-an]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Cousins}(2018)}]{Cousins:2018tiz}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {R.~D.}\ \bibnamefont {Cousins}},\ }\href@noop {} {\bibfield {journal} {\bibinfo {journal} {arXiv e-prints}\ } (\bibinfo {year} {2018})},\ \Eprint {https://arxiv.org/abs/1807.05996} {arXiv:1807.05996 [physics.data-an]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Zyla}\ \emph {et~al.}(2020)\citenamefont {Zyla} \emph {et~al.}}]{Zyla:2020zbs}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {P.}~\bibnamefont {Zyla}} \emph {et~al.} (\bibinfo {collaboration} {Particle Data Group}),\ }\href {https://doi.org/10.1093/ptep/ptaa104} {\bibfield {journal} {\bibinfo {journal} {PTEP}\ }\textbf {\bibinfo {volume} {2020}},\ \bibinfo {pages} {083C01} (\bibinfo {year} {2020})},\ \bibinfo {note} {"Chap. 40. Statistics by Glen Cowan"}\BibitemShut {NoStop}% \bibitem [{\citenamefont {Junk}\ and\ \citenamefont {Lyons}(2020)}]{junk}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {T.~R.}\ \bibnamefont {Junk}}\ and\ \bibinfo {author} {\bibfnamefont {L.}~\bibnamefont {Lyons}},\ }\href {https://doi.org/10.1162/99608f92.250f995b} {\bibfield {journal} {\bibinfo {journal} {Harvard Data Science Review}\ }\textbf {\bibinfo {volume} {2}},\ \bibinfo {pages} {4} (\bibinfo {year} {2020})},\ \Eprint {https://arxiv.org/abs/2009.06864} {arXiv:2009.06864 [physics.data-an]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Aad}\ \emph {et~al.}(2012)\citenamefont {Aad} \emph {et~al.}}]{Aad:2012tfa}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {G.}~\bibnamefont {Aad}} \emph {et~al.} (\bibinfo {collaboration} {ATLAS}),\ }\href {https://doi.org/10.1016/j.physletb.2012.08.020} {\bibfield {journal} {\bibinfo {journal} {Phys. Lett. B}\ }\textbf {\bibinfo {volume} {716}},\ \bibinfo {pages} {1} (\bibinfo {year} {2012})},\ \Eprint {https://arxiv.org/abs/1207.7214} {arXiv:1207.7214 [hep-ex]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Chatrchyan}\ \emph {et~al.}(2012)\citenamefont {Chatrchyan} \emph {et~al.}}]{Chatrchyan:2012ufa}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {S.}~\bibnamefont {Chatrchyan}} \emph {et~al.} (\bibinfo {collaboration} {CMS}),\ }\href {https://doi.org/10.1016/j.physletb.2012.08.021} {\bibfield {journal} {\bibinfo {journal} {Phys. Lett. B}\ }\textbf {\bibinfo {volume} {716}},\ \bibinfo {pages} {30} (\bibinfo {year} {2012})},\ \Eprint {https://arxiv.org/abs/1207.7235} {arXiv:1207.7235 [hep-ex]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {{Lyons}}(2013)}]{Lyons:2013yja}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {L.}~\bibnamefont {{Lyons}}},\ }\href@noop {} {\bibfield {journal} {\bibinfo {journal} {arXiv e-prints}\ } (\bibinfo {year} {2013})},\ \Eprint {https://arxiv.org/abs/1310.1284} {arXiv:1310.1284 [physics.data-an]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Algeri}\ \emph {et~al.}(2016)\citenamefont {Algeri}, \citenamefont {Conrad}, \citenamefont {van Dyk},\ and\ \citenamefont {Anderson}}]{Algeri:2016gtj}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {S.}~\bibnamefont {Algeri}}, \bibinfo {author} {\bibfnamefont {J.}~\bibnamefont {Conrad}}, \bibinfo {author} {\bibfnamefont {D.~A.}\ \bibnamefont {van Dyk}},\ and\ \bibinfo {author} {\bibfnamefont {B.}~\bibnamefont {Anderson}},\ }\href {https://doi.org/10.1088/1748-0221/11/12/P12010} {\bibfield {journal} {\bibinfo {journal} {JINST}\ }\textbf {\bibinfo {volume} {11}}\bibfield {number} {\bibinfo {number} { (12)},\ \bibinfo {pages} {P12010}},\ }\Eprint {https://arxiv.org/abs/1602.03765} {arXiv:1602.03765 [physics.data-an]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Cowan}\ \emph {et~al.}(2011)\citenamefont {Cowan}, \citenamefont {Cranmer}, \citenamefont {Gross},\ and\ \citenamefont {Vitells}}]{Cowan:2010js}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {G.}~\bibnamefont {Cowan}}, \bibinfo {author} {\bibfnamefont {K.}~\bibnamefont {Cranmer}}, \bibinfo {author} {\bibfnamefont {E.}~\bibnamefont {Gross}},\ and\ \bibinfo {author} {\bibfnamefont {O.}~\bibnamefont {Vitells}},\ }\href {https://doi.org/10.1140/epjc/s10052-011-1554-0} {\bibfield {journal} {\bibinfo {journal} {Eur. Phys. J. C}\ }\textbf {\bibinfo {volume} {71}},\ \bibinfo {pages} {1554} (\bibinfo {year} {2011})},\ \bibinfo {note} {[Erratum: \href{10.1140/epjc/s10052-013-2501-z}{Eur.~Phys.~J.~C \textbf{73}, 2501 (2013)}]},\ \Eprint {https://arxiv.org/abs/1007.1727} {arXiv:1007.1727 [physics.data-an]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Gross}\ and\ \citenamefont {Vitells}(2010)}]{Gross:2010qma}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {E.}~\bibnamefont {Gross}}\ and\ \bibinfo {author} {\bibfnamefont {O.}~\bibnamefont {Vitells}},\ }\href {https://doi.org/10.1140/epjc/s10052-010-1470-8} {\bibfield {journal} {\bibinfo {journal} {Eur. Phys. J. C}\ }\textbf {\bibinfo {volume} {70}},\ \bibinfo {pages} {525} (\bibinfo {year} {2010})},\ \Eprint {https://arxiv.org/abs/1005.1891} {arXiv:1005.1891 [physics.data-an]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {{Algeri}}\ \emph {et~al.}(2020)\citenamefont {{Algeri}}, \citenamefont {{Aalbers}}, \citenamefont {{Mor{\^a}}},\ and\ \citenamefont {{Conrad}}}]{2020NatRP...2..245A}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {S.}~\bibnamefont {{Algeri}}}, \bibinfo {author} {\bibfnamefont {J.}~\bibnamefont {{Aalbers}}}, \bibinfo {author} {\bibfnamefont {K.~D.}\ \bibnamefont {{Mor{\^a}}}},\ and\ \bibinfo {author} {\bibfnamefont {J.}~\bibnamefont {{Conrad}}},\ }\href {https://doi.org/10.1038/s42254-020-0169-5} {\bibfield {journal} {\bibinfo {journal} {Nature Reviews Physics}\ }\textbf {\bibinfo {volume} {2}},\ \bibinfo {pages} {245} (\bibinfo {year} {2020})},\ \Eprint {https://arxiv.org/abs/1911.10237} {arXiv:1911.10237 [physics.data-an]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {{Aaboud}}\ \emph {et~al.}(2019)\citenamefont {{Aaboud}} \emph {et~al.}}]{1807.07447}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {M.}~\bibnamefont {{Aaboud}}} \emph {et~al.} (\bibinfo {collaboration} {ATLAS}),\ }\href {https://doi.org/10.1140/epjc/s10052-019-6540-y} {\bibfield {journal} {\bibinfo {journal} {Eur. Phys. J.}\ }\textbf {\bibinfo {volume} {C79}},\ \bibinfo {eid} {120} (\bibinfo {year} {2019})},\ \Eprint {https://arxiv.org/abs/1807.07447} {arXiv:1807.07447 [hep-ex]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Sirunyan}\ \emph {et~al.}(2021)\citenamefont {Sirunyan} \emph {et~al.}}]{2010.02984}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {A.~M.}\ \bibnamefont {Sirunyan}} \emph {et~al.} (\bibinfo {collaboration} {CMS}),\ }\href {https://doi.org/10.1140/epjc/s10052-021-09236-z} {\bibfield {journal} {\bibinfo {journal} {Eur. Phys. J. C}\ }\textbf {\bibinfo {volume} {81}},\ \bibinfo {pages} {629} (\bibinfo {year} {2021})},\ \Eprint {https://arxiv.org/abs/2010.02984} {arXiv:2010.02984 [hep-ex]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Chatrchyan}\ \emph {et~al.}(2014)\citenamefont {Chatrchyan} \emph {et~al.}}]{1405.3961}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {S.}~\bibnamefont {Chatrchyan}} \emph {et~al.} (\bibinfo {collaboration} {CMS}),\ }\href {https://doi.org/10.1103/PhysRevD.90.112001} {\bibfield {journal} {\bibinfo {journal} {Phys. Rev. D}\ }\textbf {\bibinfo {volume} {90}},\ \bibinfo {pages} {112001} (\bibinfo {year} {2014})},\ \Eprint {https://arxiv.org/abs/1405.3961} {arXiv:1405.3961 [hep-ex]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {{Chatrchyan}}\ \emph {et~al.}(2013)\citenamefont {{Chatrchyan}} \emph {et~al.}}]{1212.6639}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {S.}~\bibnamefont {{Chatrchyan}}} \emph {et~al.},\ }\href {https://doi.org/10.1103/PhysRevLett.110.081803} {\bibfield {journal} {\bibinfo {journal} {\prl}\ }\textbf {\bibinfo {volume} {110}},\ \bibinfo {eid} {081803} (\bibinfo {year} {2013})},\ \Eprint {https://arxiv.org/abs/1212.6639} {arXiv:1212.6639 [hep-ex]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {{Aprile}}\ \emph {et~al.}(2020)\citenamefont {{Aprile}} \emph {et~al.}}]{2006.09721}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {E.}~\bibnamefont {{Aprile}}} \emph {et~al.},\ }\href {https://doi.org/10.1103/PhysRevD.102.072004} {\bibfield {journal} {\bibinfo {journal} {\prd}\ }\textbf {\bibinfo {volume} {102}},\ \bibinfo {eid} {072004} (\bibinfo {year} {2020})},\ \Eprint {https://arxiv.org/abs/2006.09721} {arXiv:2006.09721 [hep-ex]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {{Baxter}}\ \emph {et~al.}(2021)\citenamefont {{Baxter}} \emph {et~al.}}]{2105.00599}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {D.}~\bibnamefont {{Baxter}}} \emph {et~al.},\ }\href@noop {} {\bibfield {journal} {\bibinfo {journal} {arXiv e-prints}\ } (\bibinfo {year} {2021})},\ \Eprint {https://arxiv.org/abs/2105.00599} {arXiv:2105.00599 [hep-ex]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {{Albert}}\ \emph {et~al.}(2017)\citenamefont {{Albert}} \emph {et~al.}}]{1611.03184}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {A.}~\bibnamefont {{Albert}}} \emph {et~al.} (\bibinfo {collaboration} {Fermi-LAT and DES Collaborations}),\ }\href {https://doi.org/10.3847/1538-4357/834/2/110} {\bibfield {journal} {\bibinfo {journal} {\apj}\ }\textbf {\bibinfo {volume} {834}},\ \bibinfo {eid} {110} (\bibinfo {year} {2017})},\ \Eprint {https://arxiv.org/abs/1611.03184} {arXiv:1611.03184 [astro-ph.HE]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Sharma}\ and\ \citenamefont {O'Sullivan}(2021)}]{2107.08159}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {A.}~\bibnamefont {Sharma}}\ and\ \bibinfo {author} {\bibfnamefont {E.}~\bibnamefont {O'Sullivan}} (\bibinfo {collaboration} {IceCube}),\ }\href {https://doi.org/10.22323/1.395.0971} {\bibfield {journal} {\bibinfo {journal} {PoS}\ }\textbf {\bibinfo {volume} {ICRC2021}},\ \bibinfo {pages} {971} (\bibinfo {year} {2021})},\ \Eprint {https://arxiv.org/abs/2107.08159} {arXiv:2107.08159 [astro-ph.HE]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {{Bechtle}}\ \emph {et~al.}(2016)\citenamefont {{Bechtle}}, \citenamefont {{Camargo-Molina}}, \citenamefont {{Desch}}, \citenamefont {{Dreiner}}, \citenamefont {{Hamer}}, \citenamefont {{Kr{\"a}mer}}, \citenamefont {{O'Leary}}, \citenamefont {{Porod}}, \citenamefont {{Sarrazin}}, \citenamefont {{Stefaniak}}, \citenamefont {{Uhlenbrock}},\ and\ \citenamefont {{Wienemann}}}]{1508.05951}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {P.}~\bibnamefont {{Bechtle}}}, \bibinfo {author} {\bibfnamefont {J.~E.}\ \bibnamefont {{Camargo-Molina}}}, \bibinfo {author} {\bibfnamefont {K.}~\bibnamefont {{Desch}}}, \bibinfo {author} {\bibfnamefont {H.~K.}\ \bibnamefont {{Dreiner}}}, \bibinfo {author} {\bibfnamefont {M.}~\bibnamefont {{Hamer}}}, \bibinfo {author} {\bibfnamefont {M.}~\bibnamefont {{Kr{\"a}mer}}}, \bibinfo {author} {\bibfnamefont {B.}~\bibnamefont {{O'Leary}}}, \bibinfo {author} {\bibfnamefont {W.}~\bibnamefont {{Porod}}}, \bibinfo {author} {\bibfnamefont {B.}~\bibnamefont {{Sarrazin}}}, \bibinfo {author} {\bibfnamefont {T.}~\bibnamefont {{Stefaniak}}}, \bibinfo {author} {\bibfnamefont {M.}~\bibnamefont {{Uhlenbrock}}},\ and\ \bibinfo {author} {\bibfnamefont {P.}~\bibnamefont {{Wienemann}}},\ }\href {https://doi.org/10.1140/epjc/s10052-015-3864-0} {\bibfield {journal} {\bibinfo {journal} {Eur. Phys. J.}\ }\textbf {\bibinfo {volume} {C76}},\ \bibinfo {eid} {96} (\bibinfo {year} {2016})},\ \Eprint {https://arxiv.org/abs/1508.05951} {arXiv:1508.05951 [hep-ph]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {{Athron}}\ \emph {et~al.}(2021)\citenamefont {{Athron}}, \citenamefont {{Bal{\'a}zs}}, \citenamefont {{Beniwal}}, \citenamefont {{Camargo-Molina}}, \citenamefont {{Fowlie}}, \citenamefont {{Gonzalo}}, \citenamefont {{Hoof}}, \citenamefont {{Kahlhoefer}}, \citenamefont {{Marsh}}, \citenamefont {{Prim}}, \citenamefont {{Scaffidi}}, \citenamefont {{Scott}}, \citenamefont {{Su}}, \citenamefont {{White}}, \citenamefont {{Wu}},\ and\ \citenamefont {{Zhang}}}]{2007.05517}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {P.}~\bibnamefont {{Athron}}}, \bibinfo {author} {\bibfnamefont {C.}~\bibnamefont {{Bal{\'a}zs}}}, \bibinfo {author} {\bibfnamefont {A.}~\bibnamefont {{Beniwal}}}, \bibinfo {author} {\bibfnamefont {J.~E.}\ \bibnamefont {{Camargo-Molina}}}, \bibinfo {author} {\bibfnamefont {A.}~\bibnamefont {{Fowlie}}}, \bibinfo {author} {\bibfnamefont {T.~E.}\ \bibnamefont {{Gonzalo}}}, \bibinfo {author} {\bibfnamefont {S.}~\bibnamefont {{Hoof}}}, \bibinfo {author} {\bibfnamefont {F.}~\bibnamefont {{Kahlhoefer}}}, \bibinfo {author} {\bibfnamefont {D.~J.~E.}\ \bibnamefont {{Marsh}}}, \bibinfo {author} {\bibfnamefont {M.~T.}\ \bibnamefont {{Prim}}}, \bibinfo {author} {\bibfnamefont {A.}~\bibnamefont {{Scaffidi}}}, \bibinfo {author} {\bibfnamefont {P.}~\bibnamefont {{Scott}}}, \bibinfo {author} {\bibfnamefont {W.}~\bibnamefont {{Su}}}, \bibinfo {author} {\bibfnamefont {M.}~\bibnamefont {{White}}}, \bibinfo {author} {\bibfnamefont {L.}~\bibnamefont {{Wu}}},\ and\ \bibinfo {author} {\bibfnamefont {Y.}~\bibnamefont {{Zhang}}},\ }\href {https://doi.org/10.1007/JHEP05(2021)159} {\bibfield {journal} {\bibinfo {journal} {JHEP}\ }\textbf {\bibinfo {volume} {2021}}\bibfield {number} {\bibinfo {number} { (5)},\ \bibinfo {eid} {159}},\ }\Eprint {https://arxiv.org/abs/2007.05517} {arXiv:2007.05517 [astro-ph.CO]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Isidori}\ \emph {et~al.}(2021)\citenamefont {Isidori}, \citenamefont {Lancierini}, \citenamefont {Owen},\ and\ \citenamefont {Serra}}]{2104.05631}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {G.}~\bibnamefont {Isidori}}, \bibinfo {author} {\bibfnamefont {D.}~\bibnamefont {Lancierini}}, \bibinfo {author} {\bibfnamefont {P.}~\bibnamefont {Owen}},\ and\ \bibinfo {author} {\bibfnamefont {N.}~\bibnamefont {Serra}},\ }\href {https://doi.org/10.1016/j.physletb.2021.136644} {\bibfield {journal} {\bibinfo {journal} {Phys. Lett. B}\ }\textbf {\bibinfo {volume} {822}},\ \bibinfo {pages} {136644} (\bibinfo {year} {2021})},\ \Eprint {https://arxiv.org/abs/2104.05631} {arXiv:2104.05631 [hep-ph]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {{Blennow}}\ \emph {et~al.}(2014)\citenamefont {{Blennow}}, \citenamefont {{Coloma}}, \citenamefont {{Huber}},\ and\ \citenamefont {{Schwetz}}}]{1311.1822}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {M.}~\bibnamefont {{Blennow}}}, \bibinfo {author} {\bibfnamefont {P.}~\bibnamefont {{Coloma}}}, \bibinfo {author} {\bibfnamefont {P.}~\bibnamefont {{Huber}}},\ and\ \bibinfo {author} {\bibfnamefont {T.}~\bibnamefont {{Schwetz}}},\ }\href {https://doi.org/10.1007/JHEP03(2014)028} {\bibfield {journal} {\bibinfo {journal} {JHEP}\ }\textbf {\bibinfo {volume} {2014}},\ \bibinfo {eid} {28}},\ \Eprint {https://arxiv.org/abs/1311.1822} {arXiv:1311.1822 [hep-ph]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Elevant}\ and\ \citenamefont {Schwetz}(2015)}]{1506.07685}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {J.}~\bibnamefont {Elevant}}\ and\ \bibinfo {author} {\bibfnamefont {T.}~\bibnamefont {Schwetz}},\ }\href {https://doi.org/10.1007/JHEP09(2015)016} {\bibfield {journal} {\bibinfo {journal} {JHEP}\ }\textbf {\bibinfo {volume} {09}},\ \bibinfo {pages} {016}},\ \Eprint {https://arxiv.org/abs/1506.07685} {arXiv:1506.07685 [hep-ph]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {{Coloma}}\ \emph {et~al.}(2021)\citenamefont {{Coloma}}, \citenamefont {{Huber}},\ and\ \citenamefont {{Schwetz}}}]{2008.06083}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {P.}~\bibnamefont {{Coloma}}}, \bibinfo {author} {\bibfnamefont {P.}~\bibnamefont {{Huber}}},\ and\ \bibinfo {author} {\bibfnamefont {T.}~\bibnamefont {{Schwetz}}},\ }\href {https://doi.org/10.1140/epjc/s10052-020-08774-2} {\bibfield {journal} {\bibinfo {journal} {Eur. Phys. J.}\ }\textbf {\bibinfo {volume} {C81}},\ \bibinfo {eid} {2} (\bibinfo {year} {2021})},\ \Eprint {https://arxiv.org/abs/2008.06083} {arXiv:2008.06083 [hep-ph]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {{Baak}}\ \emph {et~al.}(2014)\citenamefont {{Baak}}, \citenamefont {{C{\'u}th}}, \citenamefont {{Haller}}, \citenamefont {{Hoecker}}, \citenamefont {{Kogler}}, \citenamefont {{M{\"o}nig}}, \citenamefont {{Schott}},\ and\ \citenamefont {{Stelzer}}}]{1407.3792}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {M.}~\bibnamefont {{Baak}}}, \bibinfo {author} {\bibfnamefont {J.}~\bibnamefont {{C{\'u}th}}}, \bibinfo {author} {\bibfnamefont {J.}~\bibnamefont {{Haller}}}, \bibinfo {author} {\bibfnamefont {A.}~\bibnamefont {{Hoecker}}}, \bibinfo {author} {\bibfnamefont {R.}~\bibnamefont {{Kogler}}}, \bibinfo {author} {\bibfnamefont {K.}~\bibnamefont {{M{\"o}nig}}}, \bibinfo {author} {\bibfnamefont {M.}~\bibnamefont {{Schott}}},\ and\ \bibinfo {author} {\bibfnamefont {J.}~\bibnamefont {{Stelzer}}},\ }\href {https://doi.org/10.1140/epjc/s10052-014-3046-5} {\bibfield {journal} {\bibinfo {journal} {Eur. Phys. J.}\ }\textbf {\bibinfo {volume} {C74}},\ \bibinfo {eid} {3046} (\bibinfo {year} {2014})},\ \Eprint {https://arxiv.org/abs/1407.3792} {arXiv:1407.3792 [hep-ph]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Fowlie}\ \emph {et~al.}(2021{\natexlab{a}})\citenamefont {Fowlie}, \citenamefont {Hoof},\ and\ \citenamefont {Handley}}]{code}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {A.}~\bibnamefont {Fowlie}}, \bibinfo {author} {\bibfnamefont {S.}~\bibnamefont {Hoof}},\ and\ \bibinfo {author} {\bibfnamefont {W.}~\bibnamefont {Handley}},\ }\href@noop {} {\bibinfo {title} {Code and data for ``nested sampling for frequentist computation''}},\ \bibinfo {howpublished} {\url{https://github.com/andrewfowlie/ns_for_p_values}} (\bibinfo {year} {2021}{\natexlab{a}})\BibitemShut {NoStop}% \bibitem [{\citenamefont {Brown}\ \emph {et~al.}(2001)\citenamefont {Brown}, \citenamefont {Cai},\ and\ \citenamefont {DasGupta}}]{10.1214/ss/1009213286}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {L.~D.}\ \bibnamefont {Brown}}, \bibinfo {author} {\bibfnamefont {T.~T.}\ \bibnamefont {Cai}},\ and\ \bibinfo {author} {\bibfnamefont {A.}~\bibnamefont {DasGupta}},\ }\href {https://doi.org/10.1214/ss/1009213286} {\bibfield {journal} {\bibinfo {journal} {Statistical Science}\ }\textbf {\bibinfo {volume} {16}},\ \bibinfo {pages} {101} (\bibinfo {year} {2001})}\BibitemShut {NoStop}% \bibitem [{\citenamefont {Skilling}(2004)}]{2004AIPC..735..395S}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {J.}~\bibnamefont {Skilling}},\ }in\ \href {https://doi.org/10.1063/1.1835238} {\emph {\bibinfo {booktitle} {American Institute of Physics Conference Series}}},\ \bibinfo {series} {American Institute of Physics Conference Series}, Vol.\ \bibinfo {volume} {735},\ \bibinfo {editor} {edited by\ \bibinfo {editor} {\bibfnamefont {R.}~\bibnamefont {{Fischer}}}, \bibinfo {editor} {\bibfnamefont {R.}~\bibnamefont {{Preuss}}},\ and\ \bibinfo {editor} {\bibfnamefont {U.~V.}\ \bibnamefont {{Toussaint}}}}\ (\bibinfo {year} {2004})\ pp.\ \bibinfo {pages} {395--405}\BibitemShut {NoStop}% \bibitem [{\citenamefont {Skilling}(2006)}]{Skilling:2006gxv}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {J.}~\bibnamefont {Skilling}},\ }\href {https://doi.org/10.1214/06-BA127} {\bibfield {journal} {\bibinfo {journal} {Bayesian Analysis}\ }\textbf {\bibinfo {volume} {1}},\ \bibinfo {pages} {833} (\bibinfo {year} {2006})}\BibitemShut {NoStop}% %%CITATION = INSPIRE-1670681;%% \bibitem [{\citenamefont {Kass}\ and\ \citenamefont {Raftery}(1995)}]{Kass:1995loi}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {R.~E.}\ \bibnamefont {Kass}}\ and\ \bibinfo {author} {\bibfnamefont {A.~E.}\ \bibnamefont {Raftery}},\ }\href {https://doi.org/10.1080/01621459.1995.10476572} {\bibfield {journal} {\bibinfo {journal} {J. Am. Statist. Assoc.}\ }\textbf {\bibinfo {volume} {90}},\ \bibinfo {pages} {773} (\bibinfo {year} {1995})}\BibitemShut {NoStop}% %%CITATION = JSTNA,90,773;%% \bibitem [{\citenamefont {{Buchner}}(2021)}]{2021arXiv210109675B}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {J.}~\bibnamefont {{Buchner}}},\ }\href@noop {} {\bibfield {journal} {\bibinfo {journal} {arXiv e-prints}\ } (\bibinfo {year} {2021})},\ \Eprint {https://arxiv.org/abs/2101.09675} {arXiv:2101.09675 [stat.CO]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Feroz}\ and\ \citenamefont {Hobson}(2008)}]{Feroz:2007kg}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {F.}~\bibnamefont {Feroz}}\ and\ \bibinfo {author} {\bibfnamefont {M.~P.}\ \bibnamefont {Hobson}},\ }\href {https://doi.org/10.1111/j.1365-2966.2007.12353.x} {\bibfield {journal} {\bibinfo {journal} {Mon. Not. Roy. Astron. Soc.}\ }\textbf {\bibinfo {volume} {384}},\ \bibinfo {pages} {449} (\bibinfo {year} {2008})},\ \Eprint {https://arxiv.org/abs/0704.3704} {arXiv:0704.3704 [astro-ph]} \BibitemShut {NoStop}% %%CITATION = ARXIV:0704.3704;%% \bibitem [{\citenamefont {Feroz}\ \emph {et~al.}(2013)\citenamefont {Feroz}, \citenamefont {Hobson}, \citenamefont {Cameron},\ and\ \citenamefont {Pettitt}}]{Feroz:2013hea}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {F.}~\bibnamefont {Feroz}}, \bibinfo {author} {\bibfnamefont {M.~P.}\ \bibnamefont {Hobson}}, \bibinfo {author} {\bibfnamefont {E.}~\bibnamefont {Cameron}},\ and\ \bibinfo {author} {\bibfnamefont {A.~N.}\ \bibnamefont {Pettitt}},\ }\href {https://doi.org/10.21105/astro.1306.2144} {\bibfield {journal} {\bibinfo {journal} {The Open Journal of Astrophysics}\ } (\bibinfo {year} {2013})},\ \Eprint {https://arxiv.org/abs/1306.2144} {arXiv:1306.2144 [astro-ph.IM]} \BibitemShut {NoStop}% %%CITATION = ARXIV:1306.2144;%% \bibitem [{\citenamefont {Handley}\ \emph {et~al.}(2015)\citenamefont {Handley}, \citenamefont {Hobson},\ and\ \citenamefont {Lasenby}}]{Handley:2015fda}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {W.~J.}\ \bibnamefont {Handley}}, \bibinfo {author} {\bibfnamefont {M.~P.}\ \bibnamefont {Hobson}},\ and\ \bibinfo {author} {\bibfnamefont {A.~N.}\ \bibnamefont {Lasenby}},\ }\href {https://doi.org/10.1093/mnrasl/slv047} {\bibfield {journal} {\bibinfo {journal} {Mon. Not. Roy. Astron. Soc.}\ }\textbf {\bibinfo {volume} {450}},\ \bibinfo {pages} {L61} (\bibinfo {year} {2015})},\ \Eprint {https://arxiv.org/abs/1502.01856} {arXiv:1502.01856 [astro-ph.CO]} \BibitemShut {NoStop}% %%CITATION = ARXIV:1502.01856;%% \bibitem [{\citenamefont {Demortier}(2008)}]{Demortier:1099967}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {L.}~\bibnamefont {Demortier}},\ }in\ \href {https://doi.org/10.5170/CERN-2008-001.23} {\emph {\bibinfo {booktitle} {PHYSTAT-LHC Workshop on Statistical Issues for LHC Physics}}}\ (\bibinfo {year} {2008})\ pp.\ \bibinfo {pages} {23--33}\BibitemShut {NoStop}% \bibitem [{\citenamefont {Baker}\ and\ \citenamefont {Cousins}(1984)}]{Baker:1983tu}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {S.}~\bibnamefont {Baker}}\ and\ \bibinfo {author} {\bibfnamefont {R.~D.}\ \bibnamefont {Cousins}},\ }\href {https://doi.org/10.1016/0167-5087(84)90016-4} {\bibfield {journal} {\bibinfo {journal} {Nucl. Instrum. Meth.}\ }\textbf {\bibinfo {volume} {221}},\ \bibinfo {pages} {437} (\bibinfo {year} {1984})}\BibitemShut {NoStop}% \bibitem [{\citenamefont {{Andrae}}\ \emph {et~al.}(2010)\citenamefont {{Andrae}}, \citenamefont {{Schulze-Hartung}},\ and\ \citenamefont {{Melchior}}}]{andrae2010dos}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {R.}~\bibnamefont {{Andrae}}}, \bibinfo {author} {\bibfnamefont {T.}~\bibnamefont {{Schulze-Hartung}}},\ and\ \bibinfo {author} {\bibfnamefont {P.}~\bibnamefont {{Melchior}}},\ }\href@noop {} {\bibfield {journal} {\bibinfo {journal} {arXiv e-prints}\ } (\bibinfo {year} {2010})},\ \Eprint {https://arxiv.org/abs/1012.3754} {arXiv:1012.3754 [astro-ph.IM]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Cameron}\ and\ \citenamefont {Pettitt}(2014)}]{10.1214/13-STS465}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {E.}~\bibnamefont {Cameron}}\ and\ \bibinfo {author} {\bibfnamefont {A.~N.}\ \bibnamefont {Pettitt}},\ }\href {https://doi.org/10.1214/13-STS465} {\bibfield {journal} {\bibinfo {journal} {Statistical Science}\ }\textbf {\bibinfo {volume} {29}},\ \bibinfo {pages} {397 } (\bibinfo {year} {2014})},\ \Eprint {https://arxiv.org/abs/1301.6450} {arXiv:1301.6450 [stat.ME]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Gelman}\ and\ \citenamefont {Meng}(1998)}]{10.1214/ss/1028905934}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {A.}~\bibnamefont {Gelman}}\ and\ \bibinfo {author} {\bibfnamefont {X.-L.}\ \bibnamefont {Meng}},\ }\href {https://doi.org/10.1214/ss/1028905934} {\bibfield {journal} {\bibinfo {journal} {Statistical Science}\ }\textbf {\bibinfo {volume} {13}},\ \bibinfo {pages} {163 } (\bibinfo {year} {1998})}\BibitemShut {NoStop}% \bibitem [{\citenamefont {{Wiebusch}}(2013)}]{2013CoPhC.184.2438W}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {M.}~\bibnamefont {{Wiebusch}}},\ }\href {https://doi.org/10.1016/j.cpc.2013.06.008} {\bibfield {journal} {\bibinfo {journal} {Computer Physics Communications}\ }\textbf {\bibinfo {volume} {184}},\ \bibinfo {pages} {2438} (\bibinfo {year} {2013})},\ \Eprint {https://arxiv.org/abs/1207.1446} {arXiv:1207.1446 [hep-ph]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {{Javid}}\ \emph {et~al.}(2020)\citenamefont {{Javid}}, \citenamefont {{Handley}}, \citenamefont {{Hobson}},\ and\ \citenamefont {{Lasenby}}}]{2020arXiv200412211J}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {K.}~\bibnamefont {{Javid}}}, \bibinfo {author} {\bibfnamefont {W.}~\bibnamefont {{Handley}}}, \bibinfo {author} {\bibfnamefont {M.}~\bibnamefont {{Hobson}}},\ and\ \bibinfo {author} {\bibfnamefont {A.}~\bibnamefont {{Lasenby}}},\ }\href@noop {} {\bibfield {journal} {\bibinfo {journal} {arXiv e-prints}\ } (\bibinfo {year} {2020})},\ \Eprint {https://arxiv.org/abs/2004.12211} {arXiv:2004.12211 [stat.ML]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Fowlie}\ \emph {et~al.}(2021{\natexlab{b}})\citenamefont {Fowlie}, \citenamefont {Handley},\ and\ \citenamefont {Su}}]{Fowlie:2020gfd}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {A.}~\bibnamefont {Fowlie}}, \bibinfo {author} {\bibfnamefont {W.}~\bibnamefont {Handley}},\ and\ \bibinfo {author} {\bibfnamefont {L.}~\bibnamefont {Su}},\ }\href {https://doi.org/10.1093/mnras/stab590} {\bibfield {journal} {\bibinfo {journal} {Mon. Not. Roy. Astron. Soc.}\ }\textbf {\bibinfo {volume} {503}},\ \bibinfo {pages} {1199} (\bibinfo {year} {2021}{\natexlab{b}})},\ \Eprint {https://arxiv.org/abs/2010.13884} {arXiv:2010.13884 [stat.CO]} \BibitemShut {NoStop}% \bibitem [{\citenamefont {Handley}(2019)}]{Handley:2019mfs}% \BibitemOpen \bibfield {author} {\bibinfo {author} {\bibfnamefont {W.}~\bibnamefont {Handley}},\ }\href {https://doi.org/10.21105/joss.01414} {\bibfield {journal} {\bibinfo {journal} {J. Open Source Softw.}\ }\textbf {\bibinfo {volume} {4}},\ \bibinfo {pages} {1414} (\bibinfo {year} {2019})},\ \Eprint {https://arxiv.org/abs/1905.04768} {arXiv:1905.04768 [astro-ph.IM]} \BibitemShut {NoStop}% %%CITATION = ARXIV:1905.04768;%% \end{thebibliography}% ``` 5. **Author Information:** - Lead Author: {'name': 'Andrew Fowlie'} - Full Authors List: ```yaml Andrew Fowlie: {} Sebastian Hoof: {} Will Handley: pi: start: 2020-10-01 thesis: null postdoc: start: 2016-10-01 end: 2020-10-01 thesis: null phd: start: 2012-10-01 end: 2016-09-30 supervisors: - Anthony Lasenby - Mike Hobson thesis: 'Kinetic initial conditions for inflation: theory, observation and methods' original_image: images/originals/will_handley.jpeg image: /assets/group/images/will_handley.jpg links: Webpage: https://willhandley.co.uk ``` This YAML file provides a concise snapshot of an academic research group. It lists members by name along with their academic roles—ranging from Part III and summer projects to MPhil, PhD, and postdoctoral positions—with corresponding dates, thesis topics, and supervisor details. Supplementary metadata includes image paths and links to personal or departmental webpages. A dedicated "coi" section profiles senior researchers, highlighting the group’s collaborative mentoring network and career trajectories in cosmology, astrophysics, and Bayesian data analysis. ==================================================================================== Final Output Instructions ==================================================================================== - Combine all data sources to create a seamless, engaging narrative. - Follow the exact Markdown output format provided at the top. - Do not include any extra explanation, commentary, or wrapping beyond the specified Markdown. - Validate that every bibliographic reference with a DOI or arXiv identifier is converted into a Markdown link as per the examples. - Validate that every Markdown author link corresponds to a link in the author information block. - Before finalizing, confirm that no LaTeX citation commands or other undesired formatting remain. - Before finalizing, confirm that the link to the paper itself [2105.13923](https://arxiv.org/abs/2105.13923) is featured in the first sentence. Generate only the final Markdown output that meets all these requirements. {% endraw %}