Description of the Arecibo Planetary Radar Doppler Spectra bundle. Michael C. Nolan Bundle generation date: 2024-01-15 Peer Review: 2023-02-08 Discipline Node: Small Bodies Node This bundle contains all of the Arecibo Planetary Radar Doppler spectra included in the publication Virkki et al., 2022, comprising Arecibo Planetary Radar Doppler spectra for 167 asteroids obtained from December 2017 to December 2019. The data were processed using the procedure described in Magri et al. (2007), except that for objects with tag nfreq = 0 or 1, that is, most near-Earth objects with short round-trip light-travel times, the "frequency hopping" described in that paper was not used. Instead, baselines are subtracted by excluding the (hand-noted) signal region from the polynomial fit described in that paper. Doppler Data Set Organization. The data are organized in a hierarchical tree. They are organized by object, then year, then day. Within each day there is a directory "cw" or similar, with the expectation that there will be parallel imaging and raw data deliveries. Directory names such as "cw_Tsyscorr" have no specific meaning but likely indicate that the data have been reprocessed after the system parameters we remeasured for the time the data were taken. In the "cw" directory, there will either be data files or further subdirectories named (for example) 1set, 2set, etc., if data were taken in separate sets that should not be combined because of differences in observing conditions (most typically using a different reference ephemeris). In some cases not all of the subdirectories are present, for example if the data in one of the sets was not able to be processed. If available, the Documentation tree will have a pdf file with a plot of the smoothed spectrum for reference. That plot may have been smoothed to the resolution noted on the plot for ease of viewing. If it is smoothed, the filename will have the letter s before the resolution in Hz, otherwise it will not. The resolution in the filename may be rounded. The smoothing function is a Gaussian with a Full Width at Half Maximum of sqrt(2 ln 2 pi) times the smoothing parameter shown in the plot. If there are multiple spectra they are summed using a variance-weighted mean as described in Magri et al. (2007). In these plots, the OC polarization is plotted as a solid line and the SC polarization is plotted as a dashed line. If present, the Polarizion Ratio is the ratio of the integral of the SC / OC power over a user-defined interval that was chosen to be as close as possible to the apparent width of the spectra. The format of the data is described in the document DopplerDataDescription.pdf. The data are stored in a single csv file with a multi-line metadata header and then the actual data. The csv file and label are formatted such that the PDS4 viewer can display each of these as separate tables. The data are scaled so that the standard deviation of the background noise is 1.0. To convert to physical units (square meters), multiply the data by the values in the sdev tag (see DopplerDataDescription.pdf) Acknowledgements: For data taken before 2010 October 1: The Arecibo Observatory is part of the National Astronomy and Ionosphere Center, which is operated by Cornell University under a cooperative agreement with the National Science Foundation. For data taken from 2010 October 1 through 2011 September 30: The Arecibo Observatory is part of the National Astronomy and Ionosphere Center, which is operated by Cornell University under a cooperative agreement with the National Science Foundation. The Arecibo Observatory planetary radar program is supported under the NASA NEOO program grant number NNX10AP64G. For data taken from 2011 October 1 through 2012 September 30: Arecibo Observatory is a facility of the National Science Foundation operated under cooperative agreement with SRI International in alliance with Universities Space Research Association (USRA) and Ana G. Méndez Universidad Metropolitana (UMET). The Arecibo Observatory planetary radar program is supported under the NASA NEOO program grant number NNX12AF24G. For data taken from 2012 October 1 through 2018 March 31: Arecibo Observatory is a facility of the National Science Foundation operated under cooperative agreement with SRI International in alliance with Universities Space Research Association (USRA) and Ana G. Méndez Universidad Metropolitana (UMET). The Arecibo Observatory planetary radar program is supported under the NASA NEOO program grant numbers NNX12AF24G and NNX13AQ46G. For data taken after 2018 March 31: The Arecibo Observatory is a facility of the National Science Foundation operated under cooperative agreement by the University of Central Florida and in alliance with Universidad Ana G. Méndez and Yang Enterprises, Inc. The Arecibo Observatory planetary radar program is funded through NASA's Near-Earth Object Observation (NEOO) program grant no. 80NSSC19K0523 awarded to the University of Central Florida. Caveats: The data are scaled to the standard deviation of the thermal noise, with calibration factors in the metadata. However, in many cases, the actual uncertainties are dominated by systematic uncertainty. See Virkki et al., 2022 for more information. References: Virkki, A.K., Marshall, S.E., Venditti, F.C.F., Zambrano-Marín, L.F., Hickson, D.C., McGilvray, A., Taylor, P.A., Rivera-Valentín, E.G., Devogèle, M., Díaz, E.F., Bhiravarasu, S.S., Hernández, B.A., Sánchez-Vahamonde, C.R., Nolan, M.C., Perillat, P., Cabrera, I., González, E., Padilla, D., Negrón, V., Marrero, J., Lebrón, J., Bagué, A., Jiménez, F., López-Oquendo, A., Repp, D., McGlasson, R.A., Presler-Marshall, B., Howell, E.S., Margot, J.-L., Desai, S.P., 2022. Arecibo Planetary Radar Observations of Near-Earth Asteroids: 2017 December-2019 December. Planet. Sci. J. 3, 222. https://doi.org/10.3847/PSJ/ac8b72 Magri, C., Nolan, M.C., Ostro, S.J., Giorgini, J.D., 2007. A radar survey of main-belt asteroids: Arecibo observations of 55 objects during 1999–2003. Icarus 186, 126–151. https://doi.org/10.1016/j.icarus.2006.08.018 The following is example python to read the data format. Please note that it is not a finished package but is intended to further document the format of the data. #!/usr/bin/env python3 # -*- coding: utf-8 -*- """ Created on Tue Mar 7 22:06:55 2023 @author: nolan """ import numpy as np import pandas as pd def readcsvspec(files,calibrate=True): """Ingest PDS Radar Spectra This routine reads in a series of PDS Radar spectra CSV files and parses the data and the metadata. Parameters ---------- files : list of csv filenames List of one or more csv files in the PDS radar spectra format. calibrate : bool (default True) By default the spectra are returned as calibrated cross-section by multiplying each spectrum by the sdev tag. if calbrate is False they will remain normalized to unit standard deviation. Returns ------- numpy array of frequency axes. dimensions are [numspec, nfreq] numpy array of spectra: dimensions are [numspec,2 pol, nfreq] array of dicts containing the PDS keywords in each csv file array of dicts containing the "tags" for each spectrum (two values each) array of dicts containing the "extra tags" for each spectrum """ first=True for i in files: d=pd.read_csv(i,header=None) #Can't search through NaNs, but removing causes other trouble ds=d.fillna('') # Gather up the markers, fail if not all found try: keysstart=d[ds[0].str.contains(r'(?i)# *Keywords')].index.tolist()[0]+1 except IndexError: print("Couldn't find keywordstart marker in file %s." % i) try: tagsstart=d[ds[0].str.contains(r'(?i)RDFTags')].index.tolist()[0]+1 except IndexError: print("Couldn't find tags start marker in file %s." % i) try: extrasstart=d[ds[0].str.contains(r'(?i)RDFExtra *Tags')].index.tolist()[0]+1 except IndexError: print("Couldn't find Extra Tags start marker in file %s." % i) try: coldefstart=d[ds[0].str.contains(r'(?i)# *Column *Definitions')].index.tolist()[0]+1 except IndexError: print("Couldn't find coldefs start marker in file %s." % i) try: datastart=d[ds[0].str.contains(r'(?i)# *Data')].index.tolist()[0]+1 except IndexError: print("Couldn't find data start marker in file %s." % i) # Most ranges go from start to end-2, avoiding marker keywords=d[:][keysstart:tagsstart-1] tags=d[:][tagsstart:extrasstart-1] extras=d[:][extrasstart:coldefstart-1] data=d[:][datastart:].to_numpy(dtype=float) freq=data[:,0] spec = data[:,1:3].transpose() kd = dict(keywords.values[:,0:2].tolist()) td = dict(zip(tags.values[:,0].tolist(),tags.values[:,1:3].tolist())) ed = dict(zip(extras.values[:,0].tolist(),extras.values[:,1].tolist())) if calibrate: if 'sdev' in td: spec[0,:] *= float(td['sdev'][0]) spec[1,:] *= float(td['sdev'][1]) else: print("Calibration requested but no sdev in file %s, quitting"%i) exit(-1) if first: freqa=np.array(freq,ndmin=2) speca=np.array(spec,ndmin=3) keysa=[kd] tagsa=[td] extsa=[ed] first=False else: freqa=np.append(freqa,[freq],0) speca=np.append(speca,[spec],0) keysa.append(kd) tagsa.append(td) extsa.append(ed) return(freqa,speca,keysa,tagsa,extsa)