A Description of Cluster Format Files

Originally the catalogues from this project were supplied as ASCII files with the format described below. However, as the size of the catalogues has become larger, increasingly they are supplied as FITS files (written as "fits-basic" in Topcat) or VOTables. Hence below we first describe the contents of the file in terms of the ASCII format, and then give the column headers which are used in the VOTable and FITS versions.

The ASCII Format

The first character (excepting spaces) on the top line should be a number, indicating how any colours the catalogue contains (e.g. 2 for a catalogue of V vs B-V). The next line should be the names which describe those colours, and the third line is left for comments. The catalogue then has one line per star with the following columns.
1) Field number. Sometimes integer, sometimes real of the form 3.02 where the number after the decimal is the CCD number.
2) Star identification number.
3 to 8) Star RA and declination. A negative declination is always marked by a minus sign before the degrees, even if the latter is zero.
9 & 10) Star X and Y position in the CCD image.
11) A magnitude.
12) An uncertainty. For reasons explained in Naylor et al (2002), this is actually the bright side error bar, since in magnitude space uncertainties are technically two sided. The same paper explains how to derive the faint side error bar (should you need it), and the explanation is repeated below.
13) A flag.
Columns 11 to 13 are then repeated for each colour.

FITS Format

These files contain the same information as above, with the following field headers.
FIELD (integer or 8-byte integer) and CCD (integer).
STAR_ID (integer or 8-byte integer) - when combined with FIELD and CCD provides a unique identfier).
RA and DEC (double - in degrees).
XPOS and YPOS (float - position on the CCD or array).
Then, for each colour, or magnitude (V-I in this example) we have the following.
V_I_MAG (float).
V_I_UNCERT (float).
V_I_FLAG (string, should be two characters).
V_I_NEG_FLUX (logical - for objects flagged "OO" this is set false for objects with a negative flux, which can be calculated from the magnitude).

To make it really easy to plot data from these tables, the first few columns are the magitudes and colours of those stars which have uncertainties less than 0.1 mags and are flagged "OO", these are given the names such as V_MAG_CLEAN, V_B_MAG_CLEAN etc. If the photometry does not match these criteria, the value is set as null.

The Flags

The flags are two character strings. For colours (e.g. g-i) each character is the flag for one of the two filters which went to making up the colour. The order of the flags is the same as the order of the colour, thus for g-i the g flag comes first and the i flag second. For magnitudes (e.g. g) in the natural system, which have been constructed without using colour terms, the first character is always "O", and the second the flag for that filter. If a colour term has been used to construct a magnitude, the flag is a copy of the flag for the colour used. For example, a V magnitude whose colour term was derived from B-V, might have a flag of NB. This would mean that the instrumental B measurement had been flagged N and instrumental V as B.

Strong and weak flags

The flags are divided into strong and weak. A strong flag will always be written, overwriting any other flags, whilst a weak flag will only be written if the measurement is unflagged so far. Thus the order the flags are applied is important. Most flags are strong, a weak flag is only used if there might be reason to trust the resulting photometry, or if there is an underlying reason for it. The ill-determined background criterion is very cautious, so if inspection of the image suggests the the background is not strongly structured, such data can be useful. Similarly patching bad pixels can work very well, at which point images on flagged pixels may be worthwhile. The list of flags, in the order they are applied, is as follows.

Flag	Flag Name	Algorithm	Type	Flag Number
O	O.K.			0
S	Saturated pixel.	Pixel flagging.	Strong	4
L	Counts above pixel Linearity limit.	Pixel flagging.	Strong
F	Bad pixel	Pixel flagging.	Strong	8
R	Uncalibrated Region	Pixel flagging.	Strong
W	CroWding test	Star searching.	Strong	2
N	Non-stellar	Star shape estimator.	Strong	1
B	Background fit failed	Sky determination.	Weak	3
I	Ill-determined background	Sky determination.	Weak	5
P	Position fit failed	Flux measurement.	Weak
E	Too close to Edge	Flux measurement.	Weak	2
M	Negative (Minus) counts	Flux measurement.	Weak	9
A	Absent input data	Data input to calibration.	Strong	7
H	Poor profile correction	Profile correction.	Weak
T	AperTure photometry	Combining catalogues.	Weak
V	Variable	Combining catalogues.	Weak	6
C	Large relative transparency correction (Cloudy)	Combining catalogues.	Weak
G	Poor seeing	Combining catalogues.	Weak
Z	No photometric calibration	Combining fields.	Strong

Older versions of the software used numerical flags, whose values are given in the last column.

The negative counts flag in ASCII files

If a star is flagged as having negative counts (M), the magnitude is replaced with the uncertainty in flux converted into magnitudes. Thus magnitudes for objects flagged M should be treated as bright (i.e. lower) limits. It also means that colours which are quoted for these objects are limits. For example a flag of MO in the B-V colour, means that the B magnitude is a lower limit, so B-V is a lower limit. The uncertainty becomes meaningless, unless you want the description for real aficionados...

Upper limits and the two-sidedness of error bars

The bright end of the error bar for objects with positive fluxes must match smoothly onto the upper limit for objects with negative fluxes. For this to be true it means that as the flux for an object tends to zero, the result of subtracting the magnitude error from the magnitude (i.e. calculating the bright end of the error bar) must tend towards the value obtained by converting the flux error into a magnitude. This forces our definition of the magnitude error bar (E) to be the bright end of the error bar, subtracted from the magnitude;

E = -2.5log10(f) - (-2.5log10(f+e)) = 2.5log10((f+e)/f).

Where e is the uncertainty and f the modulus of the flux. And of course the magnitude is given as

M = -2.5log10(f).

Where again note that f is the modulus of the flux. So, from the catalogue you can regain the modulus of the flux from the above formula, and the uncertainty in flux space is given by

e= 10.0**((E-M)/2.5) - 10**(-M/2.5).

n-sigma upper limits, U(n) can then be calculated as

U(n) = M - 2.5log10(1-n+10^(0.4E)).

Of course, in reality the error bar in magnitude space is not single sided. This definition we have chosen is, of course, the error bar in the bright direction. Rather than quote two-sided errors, we simply note that the faint side error bar can be calculated from the bright-side error bar in the following way. The difference between the bright-side error bar and faint side error bar is given by,

2.5log10(f/(f-e))-2.5log10((f+e)/f)=-2.5log10(1-(10^(E/2.5) -1)^2).

Upper limits for points flagged as negative

On writing out a negative flux, the flag is set to M. The uncertainty is normally set to the modulus of the flux, converted into magnitudes, and the magnitude given is the 1 sigma upper limit to the flux, again as a magnitude. Older versions of the code then multiply this by minus one, but newer versions will not (which has happened is obvious, as these stars are faint and should have positive magnitudes). The reason we suggest you normally ignore this number, is that thereafter it will be treated as the uncertainty. So, for example, as soon as a colour is made, it becomes meaningless.