PROCEDURE for getting, checking, and sorting RECONS astrometry
(updated 2022.11.12 EHV)

NOTE: the location where you place data to process it (the "landing place") is:
  /nfs/recons2/incoming.0.9m/

0. Make a notes file for this processing session, for ex., check out this file: 
     /nfs/morgan/users/vrijmoet/public_html/SMARTS/procrun.202101
   Update its header to note the observer and date, and update the dates within. 
   Make sure the 'file locations' say 'CTIO' and the notes are cleared. 
   Add notes for any special nights (other PIs) and any issues noted in the night reports.

1. For each night observed, copy the data from ctio36 to GSU. 

   Location of data on ctio36: /u363/v12/ctiopi/yyyymmdd
   Location to put data on GSU servers: /nfs/recons2/incoming.0.9m/
   You MUST be located on joy to connect to ctio36 (not recons, not huygens).

   Example of copying night '20210130' using rsync:
     rsync -avz v12@139.229.12.14:/u363/v12/ctiopi/20210130 /nfs/recons2/incoming.0.9m/
   Be prepared to enter the ctio36 password.
   In your notes file, update file location to 'ATL'.

2. Change the permissions for every folder and file so everyone can read/write:
     chmod -R a+rw 20220411
   Do this before doing any further copying, so the raw AND processed files all
   have these permissions.

3. Copy each night's directory to raw.temporary/.

4. Check each night's calibration files to make sure they are ok. 

   Finish this step BEFORE doing any processing! This is because sometimes
   you'll need to copy raw calibs from one night into another night's folder, and
   those copied calibs must NOT be already bias-subtracted (their rawness must
   match the science frames in their new folder).

   4a. Use the header file to make a list of calibration frames: 
         cp 20210130.09.headers.short list.calibs
       Then edit list.calibs to show only filenames for zeros and flats.

   4b. Check counts of all calib frames:
         imstat @list.calibs 
       Make sure they are reasonable (20,000-30,000 counts). 
       Note anything weird in the notes file. 

   4c. View the calibs to make sure they look ok:
         displayexam list.calibs
       Note anything weird in the notes file.

   4d. If any calibs are missing/bad ('bad' includes low counts), copy some
       from an adjacent night (AFTER checking those are not bad either). 
       Note this action in the notes file. 

5. In each night's folder, process the frames:
   5a. fixnameall
   5b. gof (not gofall!)
   5c. procall

6. Check the permissions of the *.o.fits files to make sure everyone can still
   read/write them. Do chmod a+rwx *.o.fits if necessary.

7. Sort the frames to their pi folders. You MUST be on recons for this to work.
   7a. In iraf: 
         hselect *.o.fits 'ra, $I, object' yes >> sort.file
       Use *.o.fits not *.fits!!!

   BONUS: Before you continue, check the observing list and/or headers.short
          file to see if any new fields were set up this run. 
          Go create folders for those new setups in the right places in /regions/.

   7b. In linux, you must be logged in to recons. Then:
         astrometry.processor sort.file

   7c. If it can't find the folder, open a new terminal and investigate:
       (NOTE: always check the observing list to verify system names, including 'ab')
       -- Is it a new setup? (check headers.short) 
           --> if so, make a folder for it in /regions/
               then tell astrometry.processor to skip the problem frame and continue.
       -- Is there an 'ab' missing from the folder name in /regions/ ? 
           --> if so, use mv to update the folder name
               then tell astrometry.processor to skip the problem frame and continue.
       -- Is the name in the header misspelled? 
           --> if so, get the path to the folder in /regions/ (carefully!)
         (note that the path MUST be in the form /recons/disk4/regions/ not /nfs/recons4/regions)

   7d. Re-run astrometry.processor to catch any frames that were skipped initially above. 
       Check that no *.o.fits are left in the folder!

   ALTERNATIVE OPTION: if you can't get the syntax right in 7c to get the script to
                       sort some frames, you can always just move them manually with 'mv'. 
                       No big deal.

8. Copy each date's raw.temporary/ folder to /nfs/whiting1/CTIOPI/CTIOPI.raw.2022/
   (or CTIOPI.raw.2023, or whatever year it is now).
   In the notes file, note that this night is sorted and DONE.