Standard Analysis Environment Checkup 1

Tuesday, October 12, 2004 - Friday, October 29, 2004 (Monday, October 11, 2004 is a holiday)

Goal

To assess the current state of the SAE and make suggestions as to how to improve the SAE. Specifically, regarding current tools:
Regarding the SAE as a whole:

Approach

In the SAE design, the tools are broken into groups according roughly to their roles in analysis. Anyone is welcome to participate in this exercise, and to evaluate any tool. However, to make sure all the tools are covered at least somewhat, it is proposed that eight reviewers be designated to cover the specified tools as indicated:

Analysis Track

Tool Name(s)

Reviewers

General Tools
dataSubselector, map_tools
1 & 2
Observation Simulation
obsSim, orbSim
1 & 2
Likelihood Analysis
Likelihood
3 & 4
Pulsar Analysis
glbary, pulsePhase, stpsearch
5 & 6
GRB Analysis
evtbin, rspgen, Xspec (external tool)
7 & 8

 Since all the reviewers will potentially have to run dataSubselector, obsSim and orbSim, they could send any comments to the designated reviewers of those tools. Of course, if the comments are too extensive, they would be welcome to perform a separate evaluation.

There doesn't need to be a hard-and-fast rule about this, but ideally each designated reviewer would work completely independently of the other one working on the same tools to maximize the number of independent perspectives.

Evaluation Forms

A standard evaluation form is being developed, which reviewers could use. Please send any suggestions for this form to James Peachey.

Specific Suggestions for Tool Evaluations

Jim Chiang proposed that the following specific analytical steps would be useful in this evaluation:

1. Fitting multiple point sources with Xspec 12 (obsSim, rspgen,               
   evtbin, Xspec12):                                                           
                                                                               
   Multiple and overlapping point sources within a given region of             
   interest would be fit using the new capabilities of Xspec 12.               
                                                                               
2. Fitting GRBs with Xspec (11 or 12) (obsSim, rspgen, evtbin, Xspec):         
                                                                               
   Very short-timescale transients with power-law spectra would be             
   simulated using the TransientTemplate or GRB sources.  evtbin would         
   be used to generate light curves; pha files would be created for            
   each time epoch.  Xspec would be used for time-resolved spectral            
   analysis.                                                                   
                                                                               
3. Unbinned GRB analysis (obsSim, Likelihood):                                 
                                                                               
   A low counts burst (< 100 events) is simulated using obsSim and             
   analyzed using Likelihood.                                                  
                                                                               
4. Pulsar light curve generation (obsSim, glbary, pulsePhase,                  
   dataSubselector, stpsearch, D4 database):                                   
                                                                               
   A pulsar with barycentering "de-corrections" applied would be               
   simulated.  These data would have corrected arrival times computed          
   using glbary, pulse phases computed using pulsePhase, the light             
   curves would be generated, and significances of the pulsed emission         
   would be evaluated.                                                         
                                                                               
5. Pulsar period search (obsSim, glbary, pulsePhase, dataSubselector,          
   stpsearch):                                                                 
                                                                               
   A pulsar period search would be performed.                                  
                                                                               
6. Likelihood analysis of diffuse sources (obsSim, Likelihood):                
                                                                               
   Generate a discrete diffuse source (i.e., not Galactic or                   
   extragalactic diffuse) and fit with likelihood using a FITS                 
   template file.

7. Exercising the plotting interface (obsSim, Likelihood, st_graph):           
                                                                               
   Data and model counts spectra resulting from a likelihood analysis          
   are plotted to demonstrate the relative contribution of different           
   source model components to fitted data.                                     
                                                                               
8. Source detection and localization (obsSim, Likelihood):                     
                                                                               
   Generate a small (~5--10) sample of unidentified point sources at           
   high Galactic latitude for a ~1 week long simulation time.  Use             
   TsMap and any other methods to find the sources, generate spectral          
   parameters, and report standard positional error contours.                  
                                                                               
9. AGN monitoring observations (obsSim, Likelihood, evtbin?)                   
                                                                               
   Perform a daily monitoring analysis of a few known flaring AGN              
   sources over a month of observations.  The simulations would                
   include nearby sources that may also have flux variations.                  
                                                                               
4 and 5 will not be entirely doable since we apparently don't have a           
pulsar source that has the barycenter "de-corrections" applied                 
available for simulation, but we could still proceed with these using          
the existing pulsar sources simply by skipping the glbary part.  Some          
tasks may require a small amount of coding (e.g., integrating st_graph         
into existing tools in 8, perhaps preparing a suitable interface for           
3), but this checkout period is also meant to test drive APIs.