Group A1 Limits Project
Summary of project
At the July 2006 BIRS workshop Statistical Inference Problems in
High Energy Physics and Astronomy, after listing the main methods
that have been proposed to solve the
A1 problem, an attempt was made
to collectively construct a matrix that listed their properties. This
resulted in considerable discussion, and the addition of a few more
methods. It became clear that the matrix was only reasonably complete
in the column marked coverage, and that only for a single channel. As
all the methods had reasonable coverage properties, more information
was needed for a prospective user to decide which to use.
A collaborative project to supply the necessary information about each
method was therefore initiated. Proponents of each method have agreed
to supply their resulting intervals for a common set of test cases.
This will permit direct comparison of frequentist coverage, interval
lengths, and Bayesian credibility. It was decided that 1 channel and
10 channel cases would be investigated. Those doing the work would
have until the end of October 2006 to complete the task. Joel Heinrich
agreed to generate the test cases and process the results.
A brief specification of the task
is available.
single channel data sets
Here are two files in ASCII text format. Each line of each file has an
(n,y,z,t,u) instance for which the
participants should provide two upper limits: at the 90% and 99%
level. (Some methods may insist on providing 2 sided limits for some
(n,y,z,t,u) combinations.)
The first set's nuisance parameters have uncertainties of about
10%, while in the second set this is increased to about 30%.
I.e., participants should provide two (or in some cases four)
extra columns for each file.
10 channel data set
Here is a single file in ASCII text format. Each line of each file has
the (n,y,z,t,u) for each of the 10
channels (for a total of 50 numbers per line). Upper limits to be
provided as specified above.
Update February 1 2007: The following directories contain the
contributions recieved so far. The .coverage files tabulate
the frequentist coverage (2nd column) as a function of the parameter
of interest (1st column), for a single set of true values
of the nuisance parameters. The 3rd column, when present shows the
uncertainty of the coverage due to Monte Carlo integration. In some
cases, both 1-sided (upper limits) and 2-sided intervals have been
provided.
Additional Links
Joel Heinrich
Last modified: September 15, 2007 12:48AM
validate this page