Functions

The individual pipeline functions are introduced and explained below. All functions presented here can be called from the terminal.

pp_run([-prefix string, ][-target string, ][-filter string, ][-fixed_aprad float, ][-solar, ][-rerun_registration, ][-asteroids, ][-keep_wcs, ]images)

serves as a wrapper for all the individual pipeline processes

Parameters:

• -prefix – (optional) the prefix of all science data images if pp_run is called using images = all
• -target – (optional) the target name to be used in all images, this overwrites the OBJECT keyword in the FITS headers; note to replace blanks with underscores if the target’s name is a designation
• -filter – (optional) manual filter name override for the photometric calibration step
• -fixed_aprad – (optional) use this fixed aperture radius for all data instead of finding the aperture radius in a curve-of-growth analysis
• -source_tolerance – (optional) this parameter defines the level at which sources are rejected in the image registration process; see pp_register() for details.
• -solar – the photometric calibration (pp_calibrate) is only using stars with solar-like colors (see pp_calibrate documentation for details); if the calibration using -solar fails (too few matches with reference catalog), the calibration is automatically repeated without the -solar option
• -rerun_registration – (optional) for some data, the image registration has to be run twice; this option will rerun the registration step, if not all frames registered successfull during the first try.
• -asteroids – (optional) make use of -asteroids option of pp_distill()
• -keep_wcs – (optional) skip registration and keep wcs information
• images – images on which the pipeline is supposed to run, wildcard symbols ('*', '?') can be used; or, by using all, PP runs on all FITS files in underlying directories (the range of images can be limited by using the -prefix option)

The use of pp_run is discussed in the Quickstart reference.

This wrapper should work successfully for most data sets. If the analysis fails at some point - or provides inadequate results - every single step in the pipeline can be run manually, providing the possibility to manually tweak the process parameters.

The following functions describe the individual pipeline processes in the logical order:

pp_prepare([-ra degrees, ][-dec degrees, ][-flipx, ][-flipy, ][-rotate degrees, ][-target string, ][-keep_wcs, ]images)

prepares image files for use in the pipeline

Parameters:

• -ra – (optional) manually sets the frame center RA
• -dec – (optional) manually sets the frame center declination
• -flipx – (optional) forces the image’s x-axis to be flipped in wcs coordinates relative to the respective Telescope Setup setting
• -flipy – (optional) forces the image’s y-axis to be flipped in wcs coordinates relative to the respective Telescope Setup setting
• -rotate – (optional) rotates the image’s orientation in the sky (East-of-North) relative to the respective Telescope Setup setting
• -target – (optional) the target name to be used in all images, this overwrites the OBJECT keyword in the FITS headers; note to replace blanks with underscores if the target’s name is a designation
• keep_wcs – retain original wcs header information and use that as initial seed for the image registration process
• images – images on which pp_prepare is supposed to run

This function prepares the image data by creating necessary FITS header keywords (e.g., the observation midtime MIDTIMJD, the pixel scale SECPIX, …), and by including fake wcs information that is required by SCAMP. Existing wcs information are deleted, as they might cause confusion with those information generated by SCAMP.

The diagnostic output of this function is a list of all frames with information on observation midtime, target name, filter, airmass, integration time, and field of view. A thumbnail image of each input image is available from this table.

pp_extract([-snr float, ][-minarea integer, ][-paramfile path, ][-aprad float, ][-telescope string, ][-ignore_saturation, ][-nodeblending, ][-quiet, ]images)

wrapper for Source Extractor

Parameters:

• -snr – (optional) minimum SNR of sources to be extracted, default: 1.5
• -minarea – (optional) minimum number of connected pixels above the SNR threshold for a valid source, default: 3
• -paramfile – (optional) manual override for the Source Extractor parameter file
• -aprad – (optional) aperture photometry aperture radius in pixels; if no aperture radius is given here, the default aperture radius for this telescope/instrument combination is used (see Telescope Setup reference)
• -telescope – (optional) manual override for the telescope identifier (see Supported Observatories)
• -ignore_saturation – (optional) using this option will not flag saturated sources; as a result, they are not rejected in the registration and calibration process
• -nodeblending – (optional) deactives Source Extractor deblending
• -quiet – (optional) suppress output on the screen
• images – images to run pp_extract on

pp_extract is automatically called by pp_register() and pp_photometry(). Usually, there is no reason to call this function manually.

pp_register([-snr float, ][-minarea integer, ][-cat catalogname, ][-source_tolerance string, ][-nodeblending, ]images)

astrometric calibration of the input images using SCAMP

Parameters:

• -snr – (optional) minimum SNR of sources to be extracted for the registration, default: 3
• -minarea – (optional) minimum number of connected pixels above the SNR threshold for a valid source, default: Telescope Setup setting
• -cat – (optional) reference catalog override for astrometric calibration (a list of supported catalogs is listed here: Supported Reference Catalogs); if not specific catalog is requested, those listed in the Telescope Setup reference are tried
• -source_tolerance – (optional) this parameter defines the cumulative level at which sources are rejected in the image registration process (in the following sequence, each level includes the previous rejection scheme): none: only flawless sources are used in the registration; low: sources with bright neighbors are considered; medium: blended sources are considered; high: saturated sources are considered; the default is high; see the Source Extractor manual section on internal flags for details.
• -nodeblending – (optional) deactives Source Extractor deblending in the detection of sources in the image frames
• images – images to run pp_register on

pp_register automatically calls pp_extract() to identify all sources in the field of view of each image; the source catalogs are stored as .ldac files. The -snr and -minarea options are passed on to pp_extract()/Source Extractor in order to specify the source properties. pp_register utilizes SCAMP to match the source catalogs with astrometric catalogs as specified for this telescope/instrument combination (see Telescope Setup reference), or as provided by the user with the -cat option. Catalogs are accessed through the CDS Vizier server; the downloaded catalog is written as a .cat file into the working directory for later inspection. Among others, SCAMP outputs two diagnostic numbers: AS_CONTRAST and XY_CONTRAST. The image registration generally has succeeded if both numbers are greater than 2.5 - the higher the contrast numbers, the better the fit. Unless every image has been registered properly, each catalog is matched twice using information from the last SCAMP run. The routine ends if all images have been registered properly or all catalogs have been used twice. Before starting the registration process, this function will check the distribution of sources in all images in the plane of the sky. If there appear to be two or more fields that are non-overlapping and separted by at least 5 degrees, those fields with fewer sources will be rejected and considered not registered. The -nodeblending option will deactivate deblending that is usually performed by Source Extractor. The advantage of deactivating deblending is that bright and saturated sources are recognized as single objects instead of a group of fainter sources; this is especially useful in the registration process.

The diagnostic output of this function is a table of the SCAMP output parameters and a presentation of each image overplotted with the catalog sources used in the matching.

pp_photometry([-snr float, ][-minarea float, ][-aprad float, ][-target string, ][-background_only, ][-target_only, ]images))

curve-of-growth analysis of the input images and source extraction using a derived optimum aperture radius resulting in final instrumental magnitudes

Parameters:

• -snr – (optional) minimum SNR of sources to be accounted for in the analysis, default: 2
• -minarea – (optional) minimum number of connected pixels above the SNR threshold for a valid source, default: Telescope Setup setting
• -aprad – (optional) if this option is used, the curve-of-growth analysis is skipped and instrumental magnitudes are derived with this aperture radius
• -target – the target name to be used in all images, this overrides the OBJECT keyword in the FITS headers; note to replace blanks with underscores if the target’s name is a designation
• -background_only – only account for background sources in the curve-of-growth analysis
• -target_only – only account for the target in the curve-of-growth analysis
• -nodeblending – (optional) deactives Source Extractor deblending
• images – images to run pp_photometry on

pp_photometry calls pp_extract() with a list of 20 different aperture radii in order to establish a separate curve-of-growth for the target (if it is a moving target) and the average of all fixed sources in the images. The motivation behind this split is to identify and minimize the impact of potential trailing caused by the relative motion of the target. The optimum aperture radius is derived based on different strategies: (1) the default is to pick the smallest aperture radius at which both the target and the background fractional fluxes are greater than 70% and the difference between the target and background curves is smaller than 5% (minimizing systematic offsets in the measured fluxes); (2) the smallest aperture radius at which the average fractional background flux is greater than 70% if the option -background_only is used; (3) the smallest aperture radius at which the target flux is greater than 70% if the option -target_only is used. These strategies have been derived empirically and lead to reliable flux measurements in most cases. The -target <targetname> option allows for overriding the target name in image header’s OBJECT keyword. If the function is called with the option -aprad <aperture radius>, no curve-of-growth analysis is performed and the provided aperture radius is adopted as the optimum aperture radius. Finally, this function runs pp_extract() again over all input images using the derived optimum aperture radius resulting in a new .ldac file for each input image providing instrumental magnitudes for all sources in the field.

The diagnostic output of this function are two plots. The first plots shows the fractional combined flux and the fraction SNR of the target and the background sources as a function of aperture radius. The optimum aperture radius is indicated with a vertical line. The second plots shows the median PSF FWHM per frame as a function of time as derived by Source Extractor. The optimum aperture diameter is indicated by a horizontal line - this line should always be slightly higher than the measured FWHMs.

pp_calibrate([-minstars int/float, ][-catalog string, ][-filter string, ][-maxflag integer, ][-instrumental, ][-solar, ][-use_all_stars, ]images)

photometric calibration of each input frame in one specific filter

Parameters:

• -minstars – (optional) minimum number of reference stars used in the photometric calibration; if int, use at least this number of stars; if float use at least this fraction of the available reference stars; if this option is not used, the default is 0.5 (i.e., use at least 50% of all available reference stars)
• -catalog – (optional) manual override for the reference catalog; a list of available reference catalogs is available here: Supported Reference Catalogs) or using this routine’s help function; if this option is not used, the photometric reference catalogs list in the Telescope Setup are used
• -filter – (optional) manual override for the filter used in the observations; if this option is not used, the filter name is read from the image FITS headers
• -maxflag – (optional) the maximum flag value for sources to be still considered in the calibration process and written into the resulting photometry database; flag values as tabulated in the Source Extractor manual; default value is 3, allowing for sources to have bright neighbors and to be blended with another source; value of 7 permits sources to be (partially) saturated
• -instrumental – (optional) if this option is used, the calibration process is skipped entirely and instrumental magnitudes are written to the photometry database for each image
• -solar – only use stars with solar-like colors; use this feature for photometry of Solar System bodies. Solar-like stars are selected based on their g-i and r-i colors, hence, this feature is currently only available for photometric calibration using the PANSTARRS, APASS, and SDSS catalogs. The threshold of solar-like colors is defined by the _pp_conf.solcol parameter; the default is the actual color index +- 0.2 mag.
• use_all_stars – if used, no quality checks are performed on calibration stars and all stars are used in the calibration.
• images – images to run pp_calibrate on

Instrumental magnitudes provided by pp_photometry() are matched with photometric catalogs in order to derive the magnitude zeropoint of each input image. Photometric catalogs are accessed through CDS Vizier, as specified in the respective Telescope Setup setting, or as specified by the -catalog option. If -catalog is not used, a number of catalogs are tried; if it is used, only one catalog is tried. If no sources are available from either catalog, the function finishes using instrumental magnitudes. Filter transformations are implemented as documented in Supported Catalog Transformations. The calibration process requires a minimum number of matched sources in the field (currently 3) and uses an iterative Chi2 fitting process as documented in Mommert (2016).

This function results in a SQLite database file (.db) for each image file, holding calibrated and instrumental magnitudes for all sources found in the field of view.

The diagnostic output of this function consists of a plot of the magnitude zeropoint of all input images as a function of time, as well as a table of all input images, their zeropoints, and the number available catalog sources and the number of sources used in the calibration. Furthermore, detailed information is available on each input image: all catalog sources used in the calibration are listed with their properties, a thumbnail of the image is shown with the calibration sources overplotted, and a diagnostic plot is generated. This plot shows the magnitude zeropoint and its uncertainty as a function of the number of calibration sources used; also, it shows the magnitude residuals as a function of source brightness.

pp_distill([-target string, ][-offset float float, ][-positions string, ][-fixedtargets string, ][-variable_stars, ][-asteroids, ][-reject string, ]images)

extraction of calibrated photometry for targets

Parameters:

• -target – (optional) the target name to be used in all images, this overrides the OBJECT keyword in the FITS headers; note to replace blanks with underscores if the target’s name is a designation
• -offset – (optional) position offset to apply on target positions (e.g., Horizons position for moving targets) in arcsec; requires two floats, one for RA and one for Dec
• -positions – (optional) file that lists the position of the target as a function of time for all frames; exact format: image filename, ra (deg), dec (deg), observations midtime (JD); if this option is used, the header OBJECT keyword will not be used to identify the target
• -fixedtargets – (optional) file that list targets with fixed positions; exact format: target name, ra (deg), dec (deg); if this option is used, the header OBJECT keyword will not be used to identify the target
• -variable_stars – (optional) match source catalog with the VSX catalog to identify and extract variable stars
• -asteroids – (optional) find serendipitously observed asteroids in the image field using IMCCE’s SkyBoT service; extract objects that are bright enough and have accurate orbits
• -reject – (optional) this option enables the filtering of data based on predefined criteria before they enter the final photometry file; a single rejection schema identifier or a comma-separated list of identifiers (no whitespaces) can be provided. Rejected observations will still show up in the final photomoetry file, but will be commented out using #; no diagnostic output will be generated for rejected observations. Valid identifiers are: pos (rejects observations with positional residuals greater than 10 arcsec). Default: pos
• images – images to run pp_distill on

This function will automatically read the target name from the FITS images (or use the manually provided one), pull target positions from JPL Horizons, and extract calibrated photometry from the database catalogs created with pp_calibrate() in to a photometry_<targetname>.dat file. In addition to the primary target, this function also creates a photometry output file for one relatively bright star that is present in the first and the last image of the series - this star serves as a control star to check the consistency of the derive magnitude zeropoints. If either the -positions or -fixedtargets option is used, JPL Horizons will not be queried, same if -asteroids is used. The latter will query the target field using IMCCE’s SkyBoT service and extract asteroids from the source catalog that have positional uncertainties less than 5 pixels (un-binned) and are brighter than 90% of the sources in the field. Note that both the options -variable_stars and -asteroids will extract the source that matches the provided target position best - confusion with an unrelated source is possible.

Functions that provide additional functionality:

pp_manident([-zoom float, ]images)

manual target identification

Parameters:

• zoom – zoom factor applied to images when loaded; number greater than one will increase the size and vice versa; default zoom value is 0.5
• images – images to run pp_manident on

This function allows to manually identify a target in the images provided and creates a file with the target’s position in each image; the resulting file can be used by pp_distill() to extract target photometry. Loading all images might take a while, the loading progress is displayed. Once all images have been loaded, the first image is displayed in a window with green circles, indicating sources identified by pp_extract(). You can browse between the images with the a and d keys, or display the next frame with a right-click. Left-click on the target in at least two different images (the target circle color will turn to red) will make this function interpolate the target trajectory using second-order splines or third-order splines, if more manual positions are provided. Browsing between the images will show the interpolated (or extrapolated) target position indicated with a yellow circle. If the target is incorrectly identified in some images, click on it again to mark it manually (red circle) which will automatically update the spline interpolation. Once the target is properly identified in all images, hit q to close the window to write the positions file (positions.dat). A few notes: (1) pp_manident() uses WCS coordinates to identify the target; the images do not necessarily have to be registered, i.e., the fake WCS information provided by pp_prepare() will work perfectly fine, allowing the user you to apply this function also on un-registered images; however, be aware that the coordinates listed in the positions.dat file might not be real RA and Dec; (2) pp_manident() relies on source catalogs created by pp_extract() so either pp_extract(), or better pp_photometry() have to be run over the images previously; please refer to the Manual Target Identification walkthrough for a recipe on how to use this function.

pp_combine([-comoving, ][-targetname str, ][-manual_rates float, float, ][-method {average, median, clipped}, ][-backsub, ][-keep_files, ]images)

image combination

Parameters:

• comoving – if used, the images will be combined in the moving frame of a moving target; the target name will be taken from the OBJECT header keyword or the targetname parameter
• targetname – manual override for the target name if comoving parameter is used
• manual_rates – use manual rates instead of queried ephemerides; in units of arcsec per second in RA and Dec; RA rate includes factor of cosine Dec
• method – image combination method: [average, median, clipped] as provided by SWARP
• backsub – if used, the background will be subtracted from each frame prior to image combination
• keep_files – if used, intermediate files are not deleted
• images – images to run pp_manident on

This function allows the combination of images using different methods. The function makes use of the SWARP software. By default, images are combined in the rest frame of the background (stars are enhanced, moving objects are partially removed); the -comoving option enables the combination in the moving frame of one target. In the latter case, images are shifted based on target ephemerides; manual rates can be provided, too. For details on the combination process, please refer to the SWARP manual. Image files produced by pp_combine can be used in any other PP function.

pp_stackedphotometry([-comoving, ][-filter str, ][-method {average, median, clipped}, ][-fixed_aprad float, ][-snr float, ][-solar, ]images)

perform automated aperture photometry on stacked images

Parameters:

• comoving – if used, the images will be combined in the moving frame of a moving target; the target name will be taken from the OBJECT header keyword or the targetname parameter
• filter – manual override for the filter band
• method – image combination method: [average, median, clipped] as provided by SWARP; default: clipped
• fixed_aprad – use fixed aperture radius for aperture photometry instead of performing a curve-of-growth analysis
• snr – minimum SNR for sources to be identified
• -solar – the photometric calibration (pp_calibrate) is only using stars with solar-like colors (see pp_calibrate documentation for details)
• images – images to run pp_stackedphotometry on

This function stacks the images provided in the background frame (skycoadd.fits) using pp_combine. If the -comoving option is used, it also creates a combined image in the moving frame of the target provided in the OBJECT FITS header keyword (resulting in comove.fits). The combination method can be set with the -clipped parameter; the default is a clipped average. Note that while a median combination might produce cleaner images, it does not conserve flux; hence, you are advised not to use the median here. The magnitude zeropoint for the respective filter band (override of header filter information using the -filter option) is then derived from the skycoadd.fits image using pp_photometry and pp_calibrate. The target photometry is finally extracted using pp_distill. If the -comoving option is used, the magnitude zeropoint derived from skycoadd.fits is applied to comove.fits, from which the target’s instrumental magnitude is extracted in that case.