sae_checkup

Standard Analysis Environment Checkup 1

Tuesday, October 12, 2004 - Friday, October 29, 2004 (Monday, October 11, 2004 is a holiday)

Goal

To assess the current state of the SAE and make suggestions as to how to improve the SAE. Specifically, regarding current tools:

Are the tools qualitatively correct, that is, do they facilitate the relevant analytical steps reasonably well?
Do the tools have sensible interfaces? If not, what are changes which would improve the interfaces?
Do the tools perform the tasks they claim to perform?
Are the tools ready for DC2? If not, what changes should have highest priority?

Regarding the SAE as a whole:

Are there tools or functionalities which are not currently scheduled for completion prior to DC2 which will be needed for DC2?
Is there functionality which will be necessary for analysis, which is altogether missing in the SAE as it is currently constituted?

Approach

In the SAE design, the tools are broken into groups according roughly to their roles in analysis. Anyone is welcome to participate in this exercise, and to evaluate any tool. However, to make sure all the tools are covered at least somewhat, it is proposed that eight reviewers be designated to cover the specified tools as indicated:

Analysis Track	Tool Name(s)	Reviewers
General Tools	dataSubselector, map_tools	1 & 2
Observation Simulation	obsSim, orbSim	1 & 2
Likelihood Analysis	Likelihood	3 & 4
Pulsar Analysis	glbary, pulsePhase, stpsearch	5 & 6
GRB Analysis	evtbin, rspgen, Xspec (external tool)	7 & 8

Since all the reviewers will potentially have to run dataSubselector, obsSim and orbSim, they could send any comments to the designated reviewers of those tools. Of course, if the comments are too extensive, they would be welcome to perform a separate evaluation.

There doesn't need to be a hard-and-fast rule about this, but ideally each designated reviewer would work completely independently of the other one working on the same tools to maximize the number of independent perspectives.

Evaluation Forms

A standard evaluation form is being developed, which reviewers could use. Please send any suggestions for this form to James Peachey.

Specific Suggestions for Tool Evaluations

Jim Chiang proposed that the following specific analytical steps would be useful in this evaluation:

1. Fitting multiple point sources with Xspec 12 (obsSim, rspgen,
   evtbin, Xspec12):

   Multiple and overlapping point sources within a given region of
   interest would be fit using the new capabilities of Xspec 12.

2. Fitting GRBs with Xspec (11 or 12) (obsSim, rspgen, evtbin, Xspec):

   Very short-timescale transients with power-law spectra would be
   simulated using the TransientTemplate or GRB sources. evtbin would
   be used to generate light curves; pha files would be created for
   each time epoch. Xspec would be used for time-resolved spectral
   analysis.

3. Unbinned GRB analysis (obsSim, Likelihood):

   A low counts burst (< 100 events) is simulated using obsSim and
   analyzed using Likelihood.

4. Pulsar light curve generation (obsSim, glbary, pulsePhase,
   dataSubselector, stpsearch, D4 database):

   A pulsar with barycentering "de-corrections" applied would be
   simulated. These data would have corrected arrival times computed
   using glbary, pulse phases computed using pulsePhase, the light
   curves would be generated, and significances of the pulsed emission
   would be evaluated.

5. Pulsar period search (obsSim, glbary, pulsePhase, dataSubselector,
   stpsearch):

   A pulsar period search would be performed.

6. Likelihood analysis of diffuse sources (obsSim, Likelihood):

   Generate a discrete diffuse source (i.e., not Galactic or
   extragalactic diffuse) and fit with likelihood using a FITS
   template file.

7. Exercising the plotting interface (obsSim, Likelihood, st_graph):

   Data and model counts spectra resulting from a likelihood analysis
   are plotted to demonstrate the relative contribution of different
   source model components to fitted data.

8. Source detection and localization (obsSim, Likelihood):

   Generate a small (~5--10) sample of unidentified point sources at
   high Galactic latitude for a ~1 week long simulation time. Use
   TsMap and any other methods to find the sources, generate spectral
   parameters, and report standard positional error contours.

9. AGN monitoring observations (obsSim, Likelihood, evtbin?)

   Perform a daily monitoring analysis of a few known flaring AGN
   sources over a month of observations. The simulations would
   include nearby sources that may also have flux variations.

4 and 5 will not be entirely doable since we apparently don't have a
pulsar source that has the barycenter "de-corrections" applied
available for simulation, but we could still proceed with these using
the existing pulsar sources simply by skipping the glbary part. Some
tasks may require a small amount of coding (e.g., integrating st_graph
into existing tools in 8, perhaps preparing a suitable interface for
3), but this checkout period is also meant to test drive APIs.

Standard Analysis Environment Checkup 1

Goal

Approach

Analysis Track

Tool Name(s)

Reviewers

Evaluation Forms

Specific Suggestions for Tool Evaluations