Empirikos.jl

Consider $n$ independent samples $Z_i$ drawn from the following hierarchical model

\[\mu_i \sim G, \ \ Z_i \sim p_i(\cdot \mid \mu).\]

Here $G$ is the unknown prior (effect size distribution) and $p_i(\cdot \mid \mu),i=1,\dotsc,n$ are known likelihood functions.

This package provides a unified framework for estimation and inference under the above setting, which is known as the empirical Bayes problem [Herbert Robbins (1956)].

Installation

The package is available from the Julia registry. It may be installed on Julia version 1.6 as follows:

using Pkg
Pkg.add("Empirikos")

For some of its functionality, this package requires a convex programming solver. The requirement for such a solver is that it can solve second order conic programs (SOCP), that it returns the dual variables associated with the SOCP constraints and that it is supported by JuMP.jl. We recommend using the MOSEK solver through the MosekTools.jl package and we used MOSEK for all simulations and empirical examples in [Nikolaos Ignatiadis, Stefan Wager (2019+)]. MOSEK is a commercial solver, but provides free academic licenses. An open-source alternative is Hypatia.jl.

Getting started

Below are some vignettes using this package for empirical Bayes tasks. There are also available as Pluto.jl notebooks at the following directory.

Nonparametric estimation using the Nonparametric Maximum Likelihood estimator (NPMLE):
- A vignette that partially reproduces the vignette of the REBayes package [Roger Koenker, Jiaying Gu (2017)].
Nonparametric confidence intervals for empirical Bayes estimands as developed in [Nikolaos Ignatiadis, Stefan Wager (2019+)]:

Modularity

This package has been designed with the goal of modularity. Specialized code (using Julia's multiple dispatch) can be easily added to more efficiently handle different combinations of estimation targets, statistical algorithms, classes of priors and likelihoods. Please open an issue if there is a combination thereof that you would like to use (and which does not work currently or is slow).

In R:

REBayes [Roger Koenker, Jiaying Gu (2017)].
Ashr [Matthew Stephens (2016)]
DeconvolveR [Balasubramanian Narasimhan, Bradley Efron (2020)]
EbayesThresh [Iain M Johnstone, Bernard W Silverman (2005)]

In Julia:

Aurora.jl

References

koenker2017rebayes: Roger Koenker, Jiaying Gu, REBayes: Empirical Bayes mixture methods in R, Journal of Statistical Software, 82(8), 1–26, 2017.
stephens2016false: Matthew Stephens, False discovery rates: a new deal, Biostatistics, 18(2), 275–294, 2016.
efron2016empirical: Bradley Efron, Empirical Bayes deconvolution estimates, Biometrika, 103(1), 1–20, 2016.
narasimhan2020deconvolver: Balasubramanian Narasimhan, Bradley Efron, deconvolveR: A G-Modeling Program for Deconvolution and Empirical Bayes Estimation, Journal of Statistical Software, 94(1), 1–20, 2020.
johnstone2005ebayesthresh: Iain M Johnstone, Bernard W Silverman, EbayesThresh: R and S-Plus programs for Empirical Bayes thresholding, Journal of Statistical Software, 12, 1–38, 2005.
robbins1956empirical: Herbert Robbins, An Empirical Bayes Approach to Statistics, Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, 1956.
ignatiadis2019bias: Nikolaos Ignatiadis, Stefan Wager, Confidence intervals for nonparametric empirical Bayes analysis, arXiv preprint arXiv:1902.02774, 2019+.