version 08 MAY 2014 by Jen Winters
version 08 MAY 2015 by Todd Henry (slight modifications)
version 24 AUG 2022 by Todd Henry (slight modifications)
This is used to update the astrometry catalog for CTIOPI.
*******************************************************************************
UPDATING THE CTIOPI ASTROMETRY CATALOG
*******************************************************************************
---------------------------------------------------------------------------
0. WHAT GOES IN THE CATALOG
---------------------------------------------------------------------------
All parallax and photometry frames.
VRI relative photometry frames --- if not in the parallax filter
code as "x" instead of "." in the filter column.
WIDEMS frames --- many will be defocused, but some might be
focused. Code as "o" instead of "." in the filter column.
GRKTRO/PLUTO frames --- even if FULL chip. Code as "xxo" instead
of "xx." in the filter column.
---------------------------------------------------------------------------
1. PREPARE HEADERS (if needed)
---------------------------------------------------------------------------
A. Make sure you have all of the "long" headers for the nights of a
run in a directory, typically named "2015.0416.long" (you will
rename files into the format 20150416).
B. If you do not have the long headers, you can generate them by using
the following command in IRAF: imhead 2015*.fits l+ > 'date'
C. If you obtained the headers via email, each one will have things
added on top that you must get rid of at this point. Delete those
rows all the way down to where the data begins. Example: if you
export the email to a directory, it will look like this:
From v12@ctioa4.ctio.noao.edu Sat Jul 15 03:59:19 2006
Date: Sat, 15 Jul 2006 03:58:58 -0400 (CST)
From: Visitor #12
To: winters@astro.gsu.edu
Subject: 20060714.09.headers.long
20060714.09.001.fits[1118,1024][ushort]: bias
No bad pixels, min=0., max=0. (old)
Line storage mode, physdim [1118,1024], length of user area 2997 s.u.
Created Sat 03:55:10 15-Jul-2006, Last modified Sat 03:55:10 15-Jul-2006
Pixel file "20060714.09.001.fits" [ok]
BSCALE = 1.0000000000E0 / REAL = TAPE*BSCALE + BZERO
and so on. You must get rid of everything including blank rows so
that the first line of the header reads:
"20060714.09.001.fits[1118,1024][ushort]: bias"
D. Check that you have the frame counts for each night.
---------------------------------------------------------------------------
2. ADD NIGHTS FOR THE RUN TO THE OLD CATALOG
---------------------------------------------------------------------------
All of the real work is going to happen in
/recons/disk4/CTIOPI/catalog.dir
Open 4 terminal windows ready to work there:
1 --- cd unprocessed ... where headers are copied
2 --- stay there ... where ./excatv4 and ./rebuildcat 0.9 are run
3 --- cd tmp ... where nightly log is edited
4 --- cd data ... where augmented catalog is checked
The file "ccdlog0.9" in the directory "data" is the fundamental
form of the catalog from frame #1 until the present.
E. In terminal 4, check that 'ccdlog0.9' is there
(/nfs/recons4/CTIOPI/catalog.dir/data/), is sorted by frame number
and that it has the correct number of old frames.
F. In terminal 1, copy the first (chronologically) night's headers t1 COPY long headers into "unprocessed"
into the directory /nfs/recons4/CTIOPI/catalog.dir/unprocessed.
Make sure the name is just 20150416, not 2015.0416.long
It is VERY IMPORTANT that you work chronologically because the
program assigns frame numbers to the nights added and these must be
in the correct sequence.
G. In terminal 2, convert the long headers into catalog format: t2 RUN ./excatv4
cd /nfs/recons4/CTIOPI/catalog.dir and
type "./excatv4" --- v4 as of May 2017
You should get a message that says:
"This command takes a bit of time--please be patient.
Splitting raw email files into single-frame-per-file format.
Processing each frame.
... (many numbers here)
Next framenumber for telescope 0.9 will be: 92131"
After a few seconds, you should get another message:
"Please open another terminal window and examine these files:
/nfs/recons4/CTIOPI/catalog.dir/tmp/ccdtmp0.9
Ready to insert this data into the CCD catalogs and append it to
the CCDLOGS for this telescope(s)? Type "y" to proceed or "n" to
exit now:"
!!!!!!!!!! DO NOT TYPE y OR n, YET !!!!!!!!!!
H. In terminal 3, check the format of catalog data for that night: t3 EMACS ccdtmp0.9 in "tmp"
cd /nfs/recons4/CTIOPI/catalog.dir/tmp
emacs "ccdtmp0.9"
1. Check the number of frames.
2. Check names and types of frames (pi vs. phot vs. other) by
comparing to short headers.
3. Change "o" filters to "ov" filters (line up v with r and i). t3 EDIT v filters, add .s
4. Next to the filter column, add a column to designate the frame
type. We generally adopt up to 10 focused frames nearest
transit per night (e.g. 25 MANTIS or RTWINS frames have 10 with
. and 15 with o):
. = good for pi
b = bad frame (a terrible frame unusable for anything)
d = dropped
o = other target (someone else's special target or a non-parallax
object --- e.g. DELTAM, GRKTRO, MANTIS, RTWINS, WIDEMS)
p = photometry
r = refraction
s = standard star/field for photometry
x = different from a bad frame --- it might be useful in the
future (ex: VRI monitoring where 2 filters are non-pi)
Note: If, in the process of adding the codes, the far right columns
get moved to the right by one space, you need to make sure you
remove this blank column. Otherwise, the program will crash.
I. In terminal 2, type "y" (once everything is looking good) and hit t2 RUN ./rebuildcat 0.9
enter. You will then get a message that says:
Update complete.
run "./rebuildcat 0.9" *** AFTER EACH NIGHT ***
J. In terminal 4, check that the new working ccdlog0.9 has the correct t4 CHECK ccdlog0.9 in "data"
number of frames. Then, you may proceed with adding additional nights.
---------------------------------------------------------------------------
3. MAKE THE FINAL CATALOG ALL SHINY AND NEW
---------------------------------------------------------------------------
K. After all of the nights have been entered for the catalog update
you are doing, you should cd into
/nfs/recons4/CTIOPI/catalog.dir/data. There is a file called
'ccdlog0.9' and this is the catalog in its raw form (i.e.
chronological order). You should open that file and look at the
nights you have added to make sure everything looks ok. You will
have already done this after processing each night, so here you are
just glancing to make sure you don't have any major errors
(e.g. nights accidentally added twice.) Make any corrections in
this file.
This file MUST end in ONLY ONE BLANK LINE. Otherwise, the
rebuildcat command will die. The ccdlog must also be sorted by the
far left column (the frame number) in order for the numbering
sequence to be correct and in order for there to be no duplicate
numbers.
L. Make any tweaks to names or frame codes. The link to the tweaks
page is found directly beneath the link to the Astrometry Catalog
on the protected page. It helps to sort the 'ccdlog0.9' text file
by object name to make these modifications.
*** RE-SORT BY FRAME NUMBER WHEN YOU ARE DONE MODIFYING. ***
M. When you are ready to rebuild the catalog, go to
/nfs/recons4/CTIOPI/catalog.dir, type "rebuildcat 0.9" (NOT the
"./rebuildcat 0.9" that the code prompted you for earlier) and hit
enter. It will take a few seconds to rebuild the catalog.
N. You should now cd into data, then cd into ccd0.9
(/nfs/recons4/CTIOPI/catalog.dir/data/ccd0.9)
O. Type in "cat ccd* >> catalog.astr.YYYY.MMDD". This will
concatenate the file and produce the final catalog for posting.
P. Copy this file into the protected directory, add the name at the
top of the file, and link it to the weblink for the RECONS group to
see.
*******************************************************************************
NOTES
*******************************************************************************
1. For objects that have both parallax and photometry frames, you need
to check to see which is the chosen parallax filter. The
astrometry observing list is a good place to find this information.
2. Check that the names for the stars are correct. Star names should
all be standardized (e.g. "LHS2112"). If you see a name like
"E2mosit", make a note of it. It will have to be changed later.
If there are several "LHS2212AB" entries and then one "LHS2212" by
itself, change it so that it says "LHS2212AB" also.
For the most part, knowledge of star names will come with time, but
in the meantime, things that may tip you off to the wrong names
are:
-- a short list of frames. The stars will all appear (unless you
sort them) with breaks in between the different stars. (see
above instructions for more on this).
-- If you suspect that a name may be wrong, refer to the protected
ctiopi page and check the astrometry and/or photometry observing
list(s).
3. In each of the individual files for the data, the second to last
column has a number that will be either negative or positive. A
negative number tells you that the frame was taken in the evening.
*******************************************************************************
***** A TIDY CATALOG IS A HAPPY CATALOG *****
*******************************************************************************