Block
Calibration
- What it does
- It cleans every raw frame with the darks and flats: it takes off the camera’s own signal, evens out the vignetting and the dust shadows, and repairs the hot and dead pixels.
- What it shows
- In Inspect, the master dark and the master flat, with the flat minimum, the share of excluded pixels and the sky noise; in the Report, the calibration frames used and these figures.
- What can be set
- Usually nothing: the defaults suit. Flat scale (about 40 px) for flats borrowed from another night, and the Hot pixel threshold if too many pixels are excluded.
The Calibration block turns the raw frames of a session into frames ready to register. It builds the master dark, bias and flat from the calibration frames, finds the hot, cold and dusty pixels, then subtracts the dark and divides by the flat for every light. The frames stay raw Bayer mosaics: nothing is debayered here. Before it, the inventory lists the frames, checks that they agree and reads their metadata.
- Calibration
- Registration
- Selection
1The session and its frames
A session is one night with one camera: the folders of its frames, set in the Session block. It takes five kinds of frames, and only the lights are required.
| Kind | What it is for | Without it |
|---|---|---|
| Lights | The frames of the target. At least one folder is needed. | The run stops: no light frame found. |
| Darks | The thermal signal and the hot pixels of the sensor, at the exposure of the lights. | The master bias, or the black level, stands for the dark. No dark removes the hot pixels; the search in the lights finds them (section 5). |
| Flats | The vignetting and the dust. | The flat is all ones: nothing is corrected. |
| Bias | The offset of the sensor. | The black level read from the frames is used. |
| Flat darks | What is taken off the flats before they are averaged. | The bias, else the black level, is taken off the flats. |
The frames are camera raw files (ARW, CR2, CR3, NEF, DNG, ORF, RW2, PEF, SRW, RAF), FITS files (FIT, FITS, FTS, also compressed as FZ or GZ), or frames already in colour (JPEG, PNG, TIFF). In a folder that holds raw or FITS frames, the JPEG, PNG and TIFF files beside them are not taken for frames. A master, a calibration frame another program already stacked, can stand in for the frames of its kind (section 3).
2The inventory
The inventory lists the frames of the five kinds and reads each one’s metadata: size, colour pattern, exposure, ISO or gain, time, camera, focal length, pixel size, black and white levels, pointing and object name. Many automatic settings further down the workflow read these values. It also computes the integration time, the sum of the exposures when the lights have several.
It checks the set before anything is computed, and stops with the reason when:
- a folder is not there (its disk may not be plugged in);
- there is no light frame, or none can be read;
- a kind mixes camera raw or FITS frames with JPEG, PNG or TIFF images: a stack takes one kind;
- darks, flats, bias or flat darks have another size than the lights;
- calibration frames are already in colour while the lights are a mosaic, or the other way round, or, for frames in colour, reach another white level (another bit depth): a dark is taken off and a flat divided by, value for value.
Two cases do not stop it. A file that cannot be read (empty or damaged) is left out and named in the log. A light whose size or colour pattern differs from the majority of the lights is ignored, with a warning. A sensor with no colour pattern is read as monochrome. For FITS frames whose header has no saturation level, a plateau of clipped pixels found in a few frames gives it: a 14-bit camera saved in 16-bit FITS saturates at 16383, not at 65535. When the header declares an overscan, only the useful area is kept.
3Master dark and bias
The master bias is the median of the bias frames. The master dark is the median of the dark frames. A master given as such (a file whose header says how many frames it stacks) is used as it is, not stacked again. Several masters of a kind are averaged, weighted by the frames each holds, and the single frames given beside a master are left out, since the master holds them already.
The lights are grouped by exposure and gain, and each group gets the dark that fits it. Darks are never averaged across exposures.
| Case | Dark used |
|---|---|
| Darks of the lights’ exposure and gain | These darks, as they are. |
| No such darks, a bias or a known black level, darks of the same gain | The darks of the nearest exposure. Their thermal signal (the dark minus the bias, or minus the black level) is multiplied by the ratio of the exposures, and the bias is put back. The scale factor is shown in the log. |
| No such darks, no bias, no black level | No dark. Without a bias to tell the thermal signal from the offset, scaling would scale the offset too. The log says so. |
| No dark at all | The master bias, else the black level. The log warns that no dark removes the hot pixels; the search in the lights finds them (section 5). |
When the master dark’s level is far under the bias (under half of it), FLOAT warns that its bias was taken off already: it then does not take the lights’ off.
4Master flat, and flats from another night
The flat darks are taken off each flat. Without flat darks it is the bias, and without a bias the black level, never a dark’s thermal signal. Several flat darks of different exposures are not averaged: those of the flats’ exposure are used, else the nearest.
Each flat is then divided by its central level (the median of its central 400 × 400 pixels). The flats are averaged without the values farther than 3 robust standard deviations (1.4826 times the median absolute deviation) from the median of that pixel. The master flat is normalised to 1 at the centre, for each position of the colour pattern separately.
Flat frames can be borrowed from another night of the same optics. They hold the vignetting of the instrument but not this night’s dust. The setting Flat scale then keeps only the scales of the master flat larger than the value: each colour plane is smoothed with a Gaussian whose standard deviation σ is that value, in sensor pixels.
A flat that has no light left at its centre once the flat dark is taken off stops the run and is named. A master flat is taken as already calibrated: nothing is taken off it, and the flat darks given beside it are not used. When the flats go under the level they are corrected with, FLOAT warns that their bias was taken off already and is taken off again.
5Defective pixels
A pixel is excluded when one of these holds. Each excluded pixel is then replaced by the mean of its same-colour neighbours that are neither excluded nor saturated.[1]
- Hot in the dark. Its dark signal at the lights’ exposure (its value in the master dark above the dark’s median) exceeds the setting Hot pixel threshold times the sky noise of one calibrated light, both in ADU.
- Cold or dusty in the flat. In its colour plane, its flat response differs from the median of its 9 × 9 neighbourhood by more than the setting Flat defect threshold (30 % by default).
- Unlit. Its flat is under a tenth of the median flat: an overscan, a dead column.
- Hot in the lights. A search in the lights themselves, with two methods (below).
Persistence
The default. It takes 40 lights spread over the session, calibrated, and keeps for each pixel the lower quartile of its values. A defect is bright in every frame; a star covers a given pixel in some frames only. A pixel is flagged when it stands out from the median of its 5 × 5 same-colour neighbours by more than 5 times the noise of one frame. A saturated defect lights its four nearest neighbours by crosstalk, so the candidates and their eight neighbours are replaced before the stars are looked for.
Stars are found as Source Extractor does,[2] through the sep library,[3] at 5 sigma of the background noise, on at least 5 pixels, and only round objects count (axis ratio under 2). A candidate under such a star in the lower quartile, where the frames hardly moved, is not judged: the stack’s rejection takes it.
Per frame
Siril’s cosmetic correction without darks, applied to each light with its default settings (hot sigma 3). It is made for a Bayer sensor.[1] The pixels it finds are replaced by the mean of their same-colour neighbours, before the saturated pixels are marked, so that a saturated defect is corrected and not taken for a star’s core.
On an X-Trans sensor, Per frame is replaced by Persistence when the session has 15 lights or more, and the search is off below that. Persistence itself needs 15 lights.
6Calibrating each light
Each light becomes (raw − dark) / flat, with the dark of its exposure group. Pixels at or above the saturation level (97.6 % of the white level by default) are marked saturated, so that the stack treats them apart and does not average clipped values. An excluded pixel is never marked saturated, even if it clips: a hot pixel taken for the core of a bright star drew the drift of the mount in the stack.
The calibrated frames are written as 16-bit floats on the scratch disk, scaled so that the brightest valid value stays inside that range. They stay raw mosaics. For each frame, the level and the noise are measured on the four positions of the colour pattern (the red, green and blue samples on an X-Trans sensor). A frame with fewer than half of its samples usable, for instance a white frame taken to mark the start of a series, has no statistics: Selection leaves it out.
The frames are calibrated in parallel, by processes, and the work resumes frame by frame after an interruption. When the masters of an earlier result were made with the same settings, only the calibrated frames are rebuilt. Masters identical to those of another session are kept on the disk once.
With Apply calibration off, the frames are taken as they are, as when they come already calibrated: only the black level is taken off, with no dark, no flat and no excluded pixel.
7Several sessions, frames in colour, an image as source
Several sessions. A project can hold several sessions. Each is inventoried and calibrated with its own masters, then the Sessions block joins them into one set of calibrated frames that Registration and Stacking read as one session’s. The sessions must have the same sensor size and colour pattern, or the run stops. The joined set uses the lowest scale of the sessions, and a pixel is saturated in it where it is saturated in any session.
Frames already in colour. JPEG, PNG and TIFF frames are not debayered: the dark and the flat are applied colour by colour, the frames are registered on their luminance and stacked with the Classic method. Hot pixels are not searched in the lights, since demosaicing has spread them. A stretched frame is not linear, which is the case of every JPEG: the dark and flat corrections are then approximate, and FLOAT says so.
An image as source. The Image block starts a workflow from an image file instead of a session: FITS, TIFF, PNG or JPEG, normally a linear stack. It is written as the result of the Stacking block, and the blocks after it read it as a stack. A monochrome image is given its one channel three times. Values in the range 0 to 1 are multiplied by 65535, to the scale of a 16-bit sensor. The image has no half-stacks, so Denoise is greyed. A file that is missing, damaged or not of its format is refused with the reason before the run. With Already stretched, the image goes to Post as it is.
8Settings
| Setting | Default | Range | What it does |
|---|---|---|---|
| Hot pixel threshold | 0.6 × sky noise | 0.1 – 10 | A pixel whose dark signal exceeds this fraction of a light’s sky noise is excluded. |
| Flat defect threshold | 0.3 | 0.05 – 0.9 | Pixels whose flat response differs from their neighbourhood by more than this fraction are excluded. |
| Hot pixels (lights) | on | on, off | Also look for hot pixels in the lights, besides the darks’. Darks taken at another temperature miss pixels that are hot only when the sensor is warmer. |
| Hot pixels | Persistence | Persistence, Per frame | How they are found in the lights (section 5). |
| Flat scale | 0 px | 0 – 500 | Keep only the scales of the master flat larger than this (the σ of a Gaussian). 0 keeps the whole flat. |
| Saturation level | 0.976 × white level | 0.5 – 1 | Raw values above this fraction of the white level are marked saturated. |
| Apply calibration | on | on, off | Off, the frames are taken as they are (section 6). |
| Parallel workers | automatic | 1 – 64 | The number of processes that calibrate frames in parallel. Automatic: the number of CPU cores minus two, at most 12. |
| Frame limit | 0 | 0 or more | Inventory. Use only N lights, evenly spaced over the session. 0 uses all. |
| Image file | none | FITS, TIFF, PNG, JPEG | Image block. The image the workflow starts from. |
9Limits, and what was tried
- Without darks, hot pixels are not removed by a dark. The log warns of it. The search in the lights is what remains, and it needs 15 lights or more with Persistence.
- Without flats, nothing is corrected. The vignetting and the dust stay, and the sky model of the Gradient block flattens the vignetting of the sky background.
- Flats from another night correct the vignetting only. The dust of that night is not this night’s: Flat scale keeps the large scales.
- Darks of another exposure are scaled only above a bias or a known black level. Otherwise the lights go without dark. Two masters of 300 s and 60 s once averaged into one of 180 s, then scaled with a black level of 0, put the calibrated sky below zero.
- Frames in colour are approximate. Stretched frames are not linear, and no hot pixel is searched in them.
- The first Persistence threshold was too low. Set in sigmas of the lower quartile’s residual, which is a third of a frame’s noise, it flagged 178,000 pixels in a Sony A7S session at ISO 4000, 110,000 of them within 2 to 5 sigmas of a frame. It is now in sigmas of one frame’s noise.
- Stars once lost flux to the search. The cores of bright stars were flagged, 10 to 26 % of their flux on a Sony A7S series, where the frames hardly moved. These pixels are now left to the stack’s rejection.
- A saturated defect was taken for a star. It lights its four nearest neighbours by crosstalk, which a lone-pixel condition read as a star. That condition was removed.
References
- Siril documentation, Cosmetic Correction. siril.readthedocs.io
- Bertin, E. & Arnouts, S. 1996, SExtractor: Software for source extraction, A&AS 117, 393. doi:10.1051/aas:1996164
- Barbary, K. 2016, SEP: Source Extractor as a library, Journal of Open Source Software 1(6), 58. doi:10.21105/joss.00058