panaSKImg¶This howto provides an introduction to the configuration file for panaSKImg, the main script for analysis provided by pysimdamicm.
The configuration file is written in JSON (stands for JavaScript Object Notation) and uses the data structure dictionary (a collection of key-value pairs).
d = {
<key>:<value>,
<key2>:<value2>
.
.
.
<keyn>:<valuen>
}
The json file has two different main sections (which in turn contain other dictionaries):
input: parameters related with the input data file. This contains at least two dictionaries:
image: mandatory for the correct interpretation of the input data file. For instance, to define the active region and the overscan and prescan regions. It also has a set of paramters that can be used to confine a specific region of the CCD to be used during execution. Data for some setups must be corrected before processing, this must also be informed in this section.
convention: the software assumes that certain information can be found in the header of the input data file: number of skips, number of pixels per axis, exposure time, readout time, as well as the binning in each axis used to read out the image, ... This section is setup dependent and mandatory to properly run panaSKImg.
process: this dictionary contains a dictionary for each process to apply to data. The process's dictionary has only those parameters allowed for tuning the process. The key of the sub-sections (or processes) is the name of the process class from any module by pysimdamicm.processes (for example, FitDarkCurrentProcess from pysimdamicm.processes.skipper_analysis). A part from the dictionaries related to processes, there is the key $\color{blue}{\text{sequece}}$, which must be used to define which list of processes (using semicolon) must be executed and in which order. If a process is not included in $\color{blue}{\text{sequece}}$, despite it is listed in the json file as a sub-suction/dictionary will not be executed!
input dictionary¶This dictionary looks like this:
{
"input":
{
"image":
{
"extensions" :0,
"axis_to_compress":1,
"correct_leach_bug":true,
"correct_polarity":false,
"id_skip_start":0,
"id_skip_end":-1,
"id_row_start":0,
"id_row_end":-1,
"id_col_start":0,
"id_col_end":-1,
"n_rows_overscan":0,
"n_rows_prescan" :3,
"n_cols_overscan":15,
"n_cols_prescan" :3,
"active_region_rows":null,
"active_region_cols":null
},
"convention":
{
"Nskips":"NDCMS",
"Ncols":"NAXIS1",
"Nrows":"NAXIS2",
"Npbin":"NPBIN",
"Nsbin":"NSBIN",
"ampl":"AMPL",
"exposure_time":"MEXP",
"read_time":"MREAD"
}
},
...
image dictionary¶The CCD image can have different defined regions:
n_rows_overscan: number of rows of overscan (including the binning)
n_rows_prescan: number of rows of prescan (including the binning)
n_cols_overscan: number of columns of overscan (including the binning)
n_cols_prescan: number of columns of prescan (including the binning)
active_regioln_rows: first and last pixel row for the active region (if not reported, it will be determined by the external regions)
active_region_cols: start and end pixel for the columns that make the cols (if not reported, it will be determined by the external regions)
These are 'real' regions of the CCD, however it may be necessary to define a region that is common to all processes but different from these. This can be done with the following parameters:
id_row_start: first row (pixel number) to consider (remember first pixel is 0)
id_row_end: last row (pixel number) to consider
id_col_start: first column to consider
id_col_end: last column to consider
The value -1 means the last pixel (in rows or columns, what is appropriate).
In case the single skip measurements should not be taking into account use:
id_skip_start: first skip (id value) to be taken
id_skip_end: first skip (id value) to be taken
The same as in the previous case, the id for the single skips starts at 0 and a negative value means last one.
Another parameter related with the single skip measurements is the direction in which these measurements have been stored: along columns or rows.
axis_to_compress: columns (1) or rows (0), by default assumed to be columns.There are a couple of corrections that can be applied on raw data:
correct_leach_bug: this is a very particular correction of the Chicago setup. The position of some columns and rows do not correspond to the real one (contact me if you need more information).
correct_polarity: this correspond to setups where the pixel charge is inversely proportional to the signal, i.e. pixels with large charge correspond to zero-charge pixel. In this case, the apply correction is
convention dictionary¶Mandatory parameters:
Nskips: string or keyword in the fits file header pointing to the number of skips (default "NDCMS")Ncols: string or keyword in the fits file header pointing to the number of columns (default "NAXIS1")Nrows: string or keyword in the fits file header pointing to the number of rows (default "NAXIS2")Only mandatory for FFTNoisePlot process:
Npbin: string or keyword in the fits file header pointing to binning size for columns (default "NPBIN")Nsbin: string or keyword in the fits file header pointing to binning size for rows (default "NSBIN")exposure_time: string or keyworkd in the fits file header pointing to the measured exposure time (default "MEXP")read_time: string or keyword in the fits file header pointing to the read out time (default "MREAD")Others:
ampl: name of the amplifier, this is mandatory for correction correct_leach_bug (default "AMPL")process dictionary¶Here how looks like this dictionary:
{
"input":
{
...
},
...
"process":
{
"sequence":"CompressSkipperProcess;PedestalSubtractionProcess;SignalPatternRecognition;ClusterFinder",
"CompressSkipperProcess":
{
"func_to_compress":["mean"]
},
"PedestalSubtractionProcess":
{
"image":"mean_compressed",
"method":"gauss_fit",
"use_mad":false,
"in_overscan":true,
"axis":"row",
"n_sigma_win_fit":3,
"n_sigma_to_mask":-1,
"show_fit":false,
"histequ":false
},
"CalibrationProcess":
{
"image":"mean_compressed_pedestal_subtracted",
"gain":5.3
},
"FitCalibrationConstant":
{
"image":"mean_compressed_pedestal_subtracted",
"n_peaks":8,
"calibration":10,
"n_sigma_win_fit":3
},
"FitDarkCurrentProcess":
{
"image":"mean_compressed_pedestal_subtracted",
"method":"root",
"do_calibration":false,
"n_peaks":3,
"n_sigma_fit":2,
"mu_gauss":0.0,
"sigma_gauss":0.2,
"lambda_poisson":0.05,
"fit_options":"QSL"
},
"ChargeLossPlot":
{
"skip_id_list":[0,1,20,30],
"skip_id_baseline":-1,
"histequ":false,
"gray_palette":false,
"pcd_nbins":1000,
"pcd_charge_range":[1000,1500]
},
"FFTNoisePlot":
{
},
"RNvsNskipsPlot":
{
"n_skips_per_block":100,
"is_blank":false
},
"SignalPatternRecognition":
{
"method" : 1,
"image" : "mean_compressed_pedestal_subtracted_e",
"isdata" : true,
"mask" : true,
"threshold" : [null,3,4],
"units" : "e"
},
"ClusterFinder":
{
"method" : 1,
"max_nearest_neighbor" : 1
}
}
}
Each sub-process corresponds to a process whose parameters will be properly explained in its howto.
Here I will explain only two parameters that can be included in any process, i.e. parameters that are common to all of them:
save-img: set (true) to record the intermediate iamge as fits (if any)
save-plot: set (true) to record any plot done in the process (if any)
If not included, panaSKImg will use the default one (normally False).
sequence param¶As mentioned at the beginning of this document, one important parameter (and mandatory) is sequece. This must be used to define which list of processes (using semicolon) must be executed and in which order. If a process is not included in sequece, despite it is listed in the json file as a sub-suction will not be executed!
image param¶Another important parameter is image that should point to the image we want to apply the process. The image is a data member of the object RawData (see RawData class). Each process during execution time (when the method execute_process is called) creates a new image as a data member of RawData with the name
<input_image_name>_<process_tag>
where
<input_image_name> stands for the name of the processed image, <process_tag> is a process dependent suffix (see next table)The following table shows the data member that each process will create and append to the RawData instance (column appended RawData attribute), assuming as an input image the one by default (third column of the table):
| process name | image param | RawData attribute |
appended RawData attribute |
|---|---|---|---|
CompressSkipperProcess |
raw | image | image_mean_compressed |
PedestalSubtractionProcess |
mean_compressed | image_mean_compressed | image_mean_compressed_pedestal_subtracted |
CalibrationProcess |
mean_compressed_pedestal_subtracted | image_mean_compressed_pedestal_subtracted | image_mean_compressed_pedestal_subtracted_e |
The second column is the strig we must use on the JSON file configuration.
The skipper data have been averaged with the mean function. This means that RawData has, a part from the image attribute created during instantiation of RawData, a new attribute known as image_mean_compressed, the averaged image.
If the image configuration parameter of the PedestalSubtractionProcess dictionary is mean_compressed means that the pedestal estimation will be done using the averaged image (RawData.image_mean_compressed) and also the subtraction. The final pedestal subtracted image
rawdata.image_mean_compressed - pedestal
will be added as an attribute of the RawData object and will be called image_mean_compressed_pedestal_subtracted. Note that the string for the image parameter does not contain the prefix *image*.
If now we want the subtracted image in units of electrons, we should apply the CalibrationProcess to image_mean_compressed_pedestal_subtracted. This process will in turn create a new attribute to the RawData instance known as mean_compressed_pedestal_subtracted_e.
So, after this process chain (CompressSkipperProcess,PedestalSubtractionProcess,CalibrationProcess) the raw data will have the following images as attributes:
attribute of RawData |
definition | type | dim | process creator |
|---|---|---|---|---|
| image | skipper image (raw data) | 3D array | (Nrow,Ncol,Nskip) | RawData instance |
| image_mean_compressed | averaged image | 2D array | (Nrow,Ncol) | CompressSkipperProcess.execute_process |
| image_mean_compressed_pedestal_subtracted | pedestal-subracted, averaged image | 2D array | (Nrow,Ncol) | PedestalSubtractionProcess.execute_process |
| image_mean_compressed_pedestal_subtracted_e | pedestal-subracted, averaged image in units of $e^{-}$ | 2D array | (Nrow,Ncol) | CalibrationProcess.execute_process |
Not all process create an attribute on the RawData instace. Here the list:
related with analysis processes:
FitCalibrationConstant
FitDarkCurrentProcess
related with reconstructiones:
SignalPatternRecognition
ClusterFinder
and related with commissionning processes:
ChargeLossPlot
FFTNoisePlot
RNvsNskipsPlot
Here an example of a full configuration JSON file:
{
"input":
{
"image":
{
"extensions" :0,
"axis_to_compress":1,
"correct_leach_bug":true,
"correct_polarity":false,
"id_skip_start":3,
"id_skip_end":-1,
"id_row_start":0,
"id_row_end":-1,
"id_col_start":0,
"id_col_end":-1,
"n_rows_overscan":0,
"n_rows_prescan" :3,
"n_cols_overscan":15,
"n_cols_prescan" :3,
"active_region_rows":null,
"active_region_cols":null
},
"convention":
{
"Nskips":"NDCMS",
"Ncols":"NAXIS1",
"Nrows":"NAXIS2",
"Npbin":"NPBIN",
"Nsbin":"NSBIN",
"ampl":"AMPL",
"exposure_time":"MEXP",
"read_time":"MREAD"
}
},
"process":
{
"sequence":"CompressSkipperProcess;PedestalSubtractionProcess;CalibrationProcess;SignalPatternRecognition;ClusterFinder",
"CompressSkipperProcess":
{
"func_to_compress":["mean"]
},
"PedestalSubtractionProcess":
{
"image":"mean_compressed",
"method":"gauss_fit",
"use_mad":false,
"in_overscan":true,
"axis":"row",
"n_sigma_win_fit":3,
"n_sigma_to_mask":-1,
"show_fit":false,
"histequ":false
},
"CalibrationProcess":
{
"image":"mean_compressed_pedestal_subtracted",
"gain":5.3
},
"FitCalibrationConstant":
{
"image":"mean_compressed_pedestal_subtracted",
"n_peaks":8,
"calibration":10,
"n_sigma_win_fit":3
},
"FitDarkCurrentProcess":
{
"image":"mean_compressed_pedestal_subtracted",
"method":"root",
"do_calibration":false,
"n_peaks":3,
"n_sigma_fit":2,
"mu_gauss":0.0,
"sigma_gauss":0.2,
"lambda_poisson":0.05,
"fit_options":"QSL"
},
"ChargeLossPlot":
{
"skip_id_list":[0,1,20,30],
"skip_id_baseline":-1,
"histequ":false,
"gray_palette":false,
"pcd_nbins":1000,
"pcd_charge_range":[1000,1500]
},
"FFTNoisePlot":
{
},
"RNvsNskipsPlot":
{
"n_skips_per_block":100,
"is_blank":false
},
"SignalPatternRecognition":
{
"method" : 1,
"image" : "mean_compressed_pedestal_subtracted_e",
"isdata" : true,
"mask" : true,
"threshold" : [null,3,4],
"units" : "e"
},
"ClusterFinder":
{
"method" : 1,
"max_nearest_neighbor" : 1
}
}