pro nis_proc, calb_mode, dark_mode, noise_mode, rad_mode, $ indir, filename, version_id, verbose, $ dir, dark1=dark1, dark2=dark2, sourcedir=sourcedir ;============================================================================= ; ABSTRACT ; nis - calibrate nis image data from FITS format files ; Shutter open/closed, Wide/narrow slit, cal/other targets ; ; INPUTS ; calb_mode - calibration mode from niscal ; 0 = Averaged raw spectra in DN, coverted to Floating point ; 1 = (spectra - dark) in DN ; 2 = fully corrected DN ; 3 = radiance ; 4 = I over F ; (NOTE that this calb_mode 5 for niscal will cause ; it to iterate 5 times per file through nis_proc ; calibratingeach file in the file list to all 5 levels. ; dark_mode - Source of Dark for calibration ; 0 = (default) linear interpolation from darks ; surrounding data spectra (with a sum or average ; dark for the spectra at each end. ; 1 = Spectra are non-interleaved. Darks are derived ; from 2 other dark-only sequences (usually ; preceeding and following spectra sequence) ; DARKS ARE SUTTER-CLOSED ; 2 = Spectra are non-interleaved. Darks are derived ; from dark-only sequences LOOKING INTO ; SPACE. ; 3 = Dark subtraction SKIPPED (usually because it was ; done externally). This option will almost ; certainly crash on an interleaved dark sequence. ; 4 = Dark from a single reference dark file: ; Cc_300dark_mean.dat ; 5 = Darks may or may not be interleaved. Abundant space darks ; are found looking into space on either side of the ; asteroid (as in mirror scans across the asteroid). Dark ; subtraction is perfomed by fitting space background ; across the sequence and subtracting. Shutter closed ; darks are ignored ; 6 = Same as 5 but the fit is "tighter" to the dark window. No ; data is dropped to shrink the dark window. Better for ; sequences that have only one or two frames looking into ; space ata time. ; 7 = Same as mode 5, but including corrected shutter-closed darks ; 8 = same as mode 6, but including corrected shutter-closed darks ; noise_mode - how spectral noise is determined ; 0 = noise for heach nis channel is determined from ; the standard deviation of all dark ; spectra in sequence (Default) ; 1 = initial noise predetermined from st. devs. from ; a file of 300 dark spectra taken on an early ; in-space NIS calibration run ; rad_mode - select radiometric correction table to be used' ; 0 = Radiative Transfer Model applied to gold sphere, ; APL xtalk, for water/atm removal is ; the correction used for all 5.0 level data ; 1 = "Placeholder" for other user-defiend correction ; indir - directory where input files are located ; filename - input file from niscal (single filename or list of filenames) ; version_id - version ID of niscal ; sourcedir - directory for NISCAL source files. Programming directory of ; NRI is /lv2124/zen/nis_cal, operational source directory ; for NEAR Science team is on loring.jhuapl.edu: ; /project/near/ST/NIS/nis_cal_vX.Y ; verbose - toggle detailed summary of actions, including output of ; hardcoded variable such as motor voltage. ; dir - output directory. If not '', it is assumed that ; output file is to be saved in directory hosting the ; IDL process. ; dark1,2 - Optional: filenames for the bracketing darks for dark_mode=3 ; ; OUTPUTS ; Output files - several MET coded FITS files may result from processing ; xxx.dko = dark only file - dark spectra separated and saved. ; Overwritten each time any ; given sequence is run through NISCAL. ; xxx.raw = uncalibrated spectra file with darks removed. ; Spectra divided by exposure time, (DN). ; xxx.dks = spectra with darks subtracted, (DN). ; xxx.cdn = Calibrated DN (corrected for temp, xtalk, mirror ; position. If this is a cal target observation ; this includes the photometric correction. ; xxx.rad = Radiance. takes .cdn data and applies DN to radiance ; conversion ; xxx.iof = Converts radiance to I over F. ; If the input file has not yet been formatted into the Calibrated NIS FITS File ; format (CNFF), then nis_proc will do so, producing a xxx.000 file in the ; input directory ; ; USES (List of functions and procedures called not in standard IDL libraries) ; parse_1st - for list of files ***Not used yet*** ; nis_fits_format - step 0 ; readfits - step 1 nis_c_div - step 2 ; nis_c_dark or nis_c_darksep - step 3 ; nis_c_gain - step 4 nis_c_temp - step X (Not implemented) ; nis_c_cross - step 5 nis_c_mirr - step 6 ; nis_c_fovfill - step X2 (Not imploemented - placeholder for later) ; nis_c_phot - step 7 ; nis_c_rad - step 8 nis_c_ecor - step 9 ; nis_c_iof - step 10 nis_c_write - step 11 ; ; HISTORY ; Created: September 10, 1997 by N. Izenberg (APL) ; Closely to loosely modeled on msi_proc (June 27, 1997) by M. Robinson ; & E. Cisneros ; MODIFIED ; 10/01/97 (NRI) Procedure status check. Check readiness for NIS data tests. ; 10/03/97 (NRI) Check after NIS SDC meeting. Need for SPICE data determined. ; FITS header will not have extra data in it, so that info ; needs to be determined within the calibration procedures. ; 11/17/97 (NRI) Change the order of procedure. Calibration proceeds from ; detector outward, so slit correction takes place before mirror. ; 11/24/97 (NRI) Continuing work. Mirror correction almost fixed - slit ; correction needs to account for mirror pos also. ; Add error for error propagation through calibration. ; 12/15/97 (NRI) Step 1-8 sub-processes validated individually during ; radiometric calibration testing. ; 12/19/97 (NRI) v1.0 all sub-processes done. Begin full checkout of version 1 ; and documentation. ; 12/22/97 (NRI) v1.1 add verbose to all sup-processes. ; 01/16/98 (NRI) v1.2 add nis cube procedure for fisdt viewable product. ; 01/19/98 (NRI) add sourcedir, made spice call step 4 and temp cor. step X ; 01/22/98 (NRI) v1.3 adjust fisdt image cube preparation and add radiometric ; correction options ; 02/02/98 (NRI) v1.4 add gain correction (fix oversight), add slit mode ; 02/05/98 (NRI) add nis_c_darksep capability ; 02/06/98 (NRI) v1.5 add option to change sourcedir ; fix rad_mode and slit_mode to represent proper values. ; 05/13/98 (NRI) v1.6 Once and for all remove list file processing option. This ; will be done by other etaxternal program, if at all. ; 06/09/98 (NRI) v1.7 Eliminate NIS_C_CUBE in favor of nis_plot for quick ; viewing and future software for data cube visualization. ; General tightening of comments and flow. ; 07/08/98 (NRI) v2.0 Add Pre-cal format step 0 ; Eliminate nis_parse_head and nis_extract. Incorporate the ; functions into nis_proc. ; Use UNFF (uncal NIS FITS File) to create CNFF (calibrated) ; using nis_fits_format ; Revamp nis_c_write ; 10/22/98 (NRI/JB) v3.0 Complete overhaul to .conf file format and list ; processing capability ; 08/31/99 (NRI) v3.5 modify for Cal modes in header ; 02/14/00 (NRI) v3.8 Fix High DN data rollover ; Change .raw to .rfd because it's really not raw, it's ; averaged and converted to floating point ; 02/28/00 (NRI) v4.0 Move ecor to before radiance calc. ; 05/04/00 (JFB,NRI) v4.1 ; Saturation Check at .rfd step. Keyword NIS-023 changed ; in nis_fits_format to indicate number of saturated ; channels. Flag in nis_c_write. Text output ; Fix to darksep for single dark spectra. ; Use of NAXIS keyword instead of Size() where appropriate ; throughout. ; Spline dark subtraction option. ; 06/08/00 (NRI/DW) v4.2 improved motor volt cor, + MVC mod to spline dark ; (Dark mode 7 and 8) added step 2B ; Eliminate motor_volt variable, since total NIS dataset ; uses only 15 volt setting. ; 06/27/00 (NRI/DW/JFB) v5.0 MET tables for Empirical correction. ; METfind subproc for all MET-based data. ; heter event flag. ; 05/XX/01 (NRI) Actual 5.0 update for archiving. ; updated nis_fits_format ; remove all spice programs ; Emp cor step include high freq noise cor and polarization cor. ; ;***************************** IMPORTANT NOTE BELOW ************************ ; SEE VERSION VARIABLE IN NISCAL.PRO ROUTINE, which passes into nis_proc.pro ;***************************** IMPORTANT NOTE ABOVE ************************ ; ; NOTES ; - Temp correction is nil unless demonstrated need. cf. discussion ; with Jeff Warren, 11/97. Variations in temps in lab tests result in ; detector variations within noise. ;============================================================================= ; Main Program ;On_error,1 if n_elements(calb_mode) eq 0 then begin print print,"Usage is:" print,"nis_proc, calb_mode, dark_mode, noise_mode, rad_mode," print," slit_mode, indir, filename, version_id, " print," verbose, dir, dark1=dark1, " print," dark2=dark2", sourcedir=sourcedir print print,"For more detail, look in program header. print print," calb_mode: 0-4 Calibration level (raw,dks,cdn,rad,iof)" print," dark_mode: 0-8 Source of dark (interleaved or sperarate seqs)" print," noise_mode: 0-1 Spectral noise (from source or library)" print," rad_mode: 0-5 Radiometric correction (GSCor, LS, LScor,M,A,C)" print," indir: '/dir/' Input file directory, '' = current dir" print," filename: 'name.FIT' Input file " print," version_id: X.Y Version ID of niscal" print," verbose: 0-1 Toggle detailed summary of actions" print," dir: '/dir' Output directory. '' = current dir" print," dark1,2 'name.FIT' Dark_mode=3 - names for bracketing darks" print," sourcedir: '/dir/' Directory for NISCAL source files. Not" print," needed if environment varianble NIS_CALB is defined in" print," your .cshrc." return endif ;-------------------- ;INITIALIZE VARIABLES ;-------------------- ; Check for source directory of the niscal prodedures and files. ; This is a unix structure. if not (keyword_set (SOURCEDIR)) then sourcedir=getenv('NIS_CALB') exit1=0 ; This is a toggle that is flipped when the desired cal ; level is reached print, ' >>>> Calibration mode:', calb_mode print, ' >>>> (0=raw DN, 1=raw-dark DN, 2=corrected DN 3=rad, 4=I/F)' ; extract version ID vid =strmid(sstrr(version_id),0,3) fname1='fname1' ;initialize filename variable append='.non' ;append to incoming file name for output file name ; for radiance output, .iof for I/F ; for raw DN output, .rdn ; placeholder is "non" calibrated modes=[calb_mode,dark_mode, noise_mode, rad_mode] ;================================================================= ;00000000000000000000000000000000000000000000000000000000000000000 ; STEP 0: Format uncalibrated file into Calibrated NIS FITS File ; (CNFF) format. This changes the header, and adds ; empty data columns for spice and noise ; The procedure used - nis_fits_format - is standalone ; and is skipped if the filename has a '.000' suffix ; NISCAL saves the CNFF in the input directory. ; 07/12/01 NISCAL 5.0 Compliant ;================================================================= fname1=filename ;get file name to read indir1=indir ; Set up directory references for indir2=indir ; data file and bounding dark files indir3=indir ; (if used) dir1=dir dir2=dir dir3=dir sl=strlen(fname1) nis_fits_format,fname1,indir1,dir1,modes,verbose indir1=dir1 if (dark_mode eq 1 or dark_mode eq 2) and calb_mode ne 0 then begin nis_fits_format,dark1,indir,dir2,modes,verbose nis_fits_format,dark2,indir,dir3,modes,verbose indir2=dir2 indir3=dir3 endif indir2=dir ;================================================================= ;11111111111111111111111111111111111111111111111111111111111111111 ; STEP 1: Read in FITS format raw nis file ; 07/12/01 NISCAL 5.0 Compliant ;================================================================= ; Extract sequence ; seq_1 is the nis spectra sequence for fname1 ; header_data is the whole FITS header for fname1 ;------------------------------------------------ seq_1 = readfits(indir1+fname1,header_data) seq_1=float(seq_1) ; convert to floating point. ; Pre-CNISF 5.0 sequence data is 44:107 ; CNISF 5.0 Data is 67:130 for Raw ; CNISF 5.0 Data is 131:194 for Calibration ; CNISF 5.0 Data is 195:258 for Noise seq_time = double(strmid(fname1,3,9)) if ( seq_time le 50513800. and seq_time gt 0) then begin print," >>>> Spectra acquired with cover ON....." print," >>>> Only some early checkout calibrations did this." endif if ( seq_time eq 0. ) then begin print," >>>> Time of acquisition not determinable from file name" endif if verbose eq 1 then begin print,' ' print,' >>>> File name: ' print,' >>>> ',fname1 print,' STEP 1: NIS sequence extracted' print,' >>>> Begin calibrating ' print print,' NIS Sequence Commands, Description' print,' '+sxpar(header_data,'NIS-019')+sxpar(header_data,'NIS-020') print,' '+sxpar(header_data,'NIS-021')+sxpar(header_data,'NIS-022') endif ;================================================================= ;1B 1B 1B 1B 1B 1B 1B 1B 1B 1B 1B 1B 1B 1B 1B 1B 1B 1B 1B 1B 1B 1B ; STEP 1B: Gather significant MET-based data ; 1) Motor Correction table ; 2) Heater event times ;================================================================= nis_c_metfind, seq_1, sourcedir, verbose, header_data, $ voltcor, volterr ;================================================================= ;22222222222222222222222222222222222222222222222222222222222222222 ; STEP 2: Divide each spectra by exposure time to get average ; spectra through sequence ; Initial error (err) is ; per-detector standard deviation of 300 individual ; spectra taken from August 8, 1997 test run. These ; types of test are consitent, though actual stdevs ; vary somewhat with each series, esp the higher InGaAs due ; to DC drift ; Outputs xxx_# shows type (sequence, spectra) and step ; So seq_1 turns into seq_2 ;================================================================= obs_target = strmid(fname1,18,1) ; Observation target for solar dist calc ; C=CalTarget, E=Eros ; M=Moon, H=Earth, S=Space, D=Dark, S,T=Special ; 0=unknown nis_c_div,seq_1, seq_2, dark_mode, obs_target, sourcedir, verbose, header_data, $ satflag if calb_mode eq 0 then begin Print, " STOP: Avg Raw DN converted to floating point" spec_out=seq_2 append='.rfd' goto,skip endif ;================================================================= ;33333333333333333333333333333333333333333333333333333333333333333 ; STEP 3: Subtract appropriate dark interpolated from sequence ; or from other provided source. Separate and save ; dark file to protected directory. Create 2 arrays of ; equal # of rows - one of spectra, one of darks ; output seq_2, is the dark-corrected data spectra from ; the sequence ; For dark_mode 7 and 8: Dark-Motor Voltage Correction ; Start with canned correction and move to dynamic ; adjusts closed shutter darks for motor voltage, so they ; and space darks can be used the same way in the spacedark ; interpolation ;================================================================= Case dark_mode of 0 : nis_c_dark,seq_2, noise_mode, voltcor, $ volterr, header_data, filename,spec_3,vid,sourcedir,dir, $ verbose, exit1 3 : begin if verbose eq 1 then begin print, "NO DIVISION - Observations are still coadded" print, "NO DARK SUBTRACTION" endif spec_3=seq_1 end 5 : nis_c_darkspace, seq_2, 5, noise_mode, spec_3, $ header_data, verbose 6 : nis_c_darkspace, seq_2, 6, noise_mode, spec_3, $ header_data, verbose 7 : begin nis_c_dmv,seq_2,seq_2a, voltcor, volterr, header_data, $ sourcedir,verbose nis_c_darkspace, seq_2a, 7, noise_mode, spec_3, $ header_data, verbose seq_2=seq_2a end 8 : begin nis_c_DMV,seq_2,seq_2a, voltcor, volterr, header_data, $ sourcedir,verbose nis_c_darkspace, seq_2a, 8, noise_mode, spec_3, $ header_data, verbose seq_2=seq_2a end else : nis_c_darksep,seq_2, obs_target, indir2, indir3, dark1, $ dark2, noise_mode, spec_3, sourcedir,verbose, voltcor, $ volterr, dark_mode, header_data ; Dark modes 1,2, and 4 endcase if exit1 ne 0 then goto, exiter if calb_mode eq 1 then begin Print," STOP: Dark Subtracted Spectra: No Other Processing" spec_out=spec_3 append='.dks' goto,skip endif ;================================================================= ;44444444444444444444444444444444444444444444444444444444444444444 ; STEP 4: Gain correction ; Divide Ge channels by gain factor of ~10 if set to high gain ;================================================================= nis_c_gain, spec_3, spec_4,verbose ;================================================================= ;55555555555555555555555555555555555555555555555555555555555555555 ; STEP 5: Correct for Crosstalk ; Uses table from J. Bell. Only first 6 Ge ; detectors require correction. ; **** Crosstalk is gain sensititve **** ;================================================================= nis_c_cross, spec_4, spec_5, sourcedir, rad_mode, verbose ;================================================================= ;66666666666666666666666666666666666666666666666666666666666666666 ; STEP 6: Correct for Mirror Position ; Uses table from ; Coefficients available for each detector derived from ; large sphere lab data. Narrow slit must be corrected to wide ; slit immediately after this operation. ; Errors not propagated. ;================================================================= nis_c_mirror, spec_5, spec_6, flag2, sourcedir, verbose ;================================================================= ;77777777777777777777777777777777777777777777777777777777777777777 ; STEP 7: Apply Photometric model to Cal Target Measurements ; Normal ops view cal target at 5 degrees yaw. If ; yaw is not 5 degrees then correction is applied to ; this geometry. ; No error propagation ; S/C yaw can be calculated from spice data ; which could revise yaw value for each spectrum ; Since yaws for cal-target observations do not ; deviate much from 5 degrees off sun. Version 5.0 ; uses that as a default. ;================================================================= if obs_target eq "C" then begin if n_elements(yaw) ne 0 then yaw=yaw else yaw=5. nis_c_phot, spec_6, verbose, yaw=yaw endif if calb_mode eq 2 then begin Print," STOP: Calibrated DN calculated" spec_out=spec_6 append='.cdn' goto,skip endif ;================================================================= ;88888888888888888888888888888888888888888888888888888888888888888 ; STEP 8: Apply DN per Radiance Unit scaling from Cal Target ; Applies radioetric response of to detectors at cal ; target. Converts DN to radiance units; ; Conversion is Counts / mW cm^-2 sr^-1 um^-1 ;================================================================= nis_c_rad, spec_6, spec_8, rad_mode, sourcedir, verbose, flag3 if calb_mode eq 3 then begin print," STOP: Radiance calculated" spec_out=spec_8 append='.rad' goto,skip endif ;================================================================= ;99999999999999999999999999999999999999999999999999999999999999999 ; STEP 9: Convert to I/f ; output is converted from radiance to I over F ; If converting to I/F, empirical corrections (step 10) are also ; applied. ;================================================================= nis_c_iof, spec_8, spec_9, sourcedir, header_data, verbose,solflag ;================================================================= ;10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 10 ; STEP 10: Apply Empirical Corrections ; High Frequency Noise correction ; Polarization Correction ;================================================================= nis_c_ecor, spec_9, spec_10, verbose, sourcedir, header_data, polflag if calb_mode eq 4 then begin spec_out=spec_10 append='.iof' goto,skip endif skip: ; this is the spot that all the calb_mode steps have been ; kicking out to. Next step iss to save the file at whatever ; level of processing has been selected. ;================================================================= ;11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 ; STEP 11: Write out Calibrated spectra file ;================================================================= nis_c_write,spec_out,calb_mode,dark_mode,noise_mode,rad_mode,$ fname1,header_data,vid,append, sourcedir, verbose, $ satflag,flag2, flag3,polflag,solflag,dark1=dark1,dark2=dark2,dir=dir exiter: print,' ' print,' ' ;12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 12 print,'############### END PROCESSING RUN ################' return end