0010 queued restartDP.sh using "at" to be run at 0430 MDT (1030 UTC) 0030 queued postObservationActions.sh using "at" to be run at 0615 MDT (1215 UTC) 0410 restarted DP/TBN 0430 restart DP, no TBN, by way of "at" command 0500 LR001_120522_1100_0021_B2_PS.sdf gain 7, split BW started as planned 0600 LR001_120522_1100_0021_B2_PS.sdf gain 7, split BW completed as planned, MGT underflow on beam 1 0630 postObservationActions.sh executed, by way of "at" command 0830 Temperatures seem to be making slow climb consistent with HVAC iceing, but too early to tell. 0830 Ran storage report =========== Storage report for DR1 =========== DR1 DRSUs Attached: 2 DRSU selected: 1 DRSU1: /dev/md0 Barcode: '<Not Implemented Yet>' Total Storage: 13972.59 GB Remaining Storage: 13835.87 GB Contiguous Storage: 13835.87 GB Usage: 0% File Count: 1 files DRSU2: /dev/md1 Barcode: '<Not Implemented Yet>' Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% File Count: 0 files =========== Storage report for DR2 =========== DR2 DRSUs Attached: 2 DRSU selected: 1 DRSU1: /dev/md0 Barcode: '<Not Implemented Yet>' Total Storage: 13972.59 GB Remaining Storage: 11499.51 GB Contiguous Storage: 11499.51 GB Usage: 17.00% File Count: 10 files DRSU2: /dev/md1 Barcode: '<Not Implemented Yet>' Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% File Count: 0 files =========== Storage report for DR3 =========== DR3 DRSUs Attached: 2 DRSU selected: 2 DRSU1: /dev/md0 Barcode: '<Not Implemented Yet>' Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% File Count: 0 files DRSU2: /dev/md1 Barcode: '<Not Implemented Yet>' Total Storage: 13972.59 GB Remaining Storage: 12974.83 GB Contiguous Storage: 12974.83 GB Usage: 7.00% File Count: 5 files =========== Storage report for DR4 =========== DR4 DRSUs Attached: 2 DRSU selected: 2 DRSU1: /dev/md0 Barcode: '<Not Implemented Yet>' Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% File Count: 0 files DRSU2: /dev/md1 Barcode: '<Not Implemented Yet>' Total Storage: 13972.59 GB Remaining Storage: 12974.83 GB Contiguous Storage: 12974.83 GB Usage: 7.00% File Count: 5 files =========== Storage report for DR5 =========== DR5 DRSUs Attached: 1 DRSU selected: 1 DRSU1: /dev/md0 Barcode: '<Not Implemented Yet>' Total Storage: 9315.08 GB Remaining Storage: 6199.83 GB Contiguous Storage: 6199.83 GB Usage: 33.00% File Count: 1 files 1130 Chris Wolfe, siging out, handing over station control to Jake Hartman 1200 Joe Craig takes over briefly to run some tests 1300 TBN/DP restarted 1330 JH resumes control 1500 Submitted ~/MCS/tp/120523/Planning/LR001_120523_0100_0022_B2_PS.sdf, which CW prepared. 1130 PASI images were noise; INI'ed DP and restarted TBN 2250 Queued Joe C's script to record fringes for Cyg A: "cd ~/MCS/exec; at 04:32 05/24/12 -f ./120523_CygA_0432.sh" Xpol = stand 254; Ypol = outrigger Contents of 120523_CygA_0432.sh: ./mesix ASP FIL 00000 # all channels to split-BW filter sleep 1 ./mesix ASP ATS 00015 # 30 dB atten sleep 1 ./mesix ASP AT1 00008 # 16 dB atten sleep 1 ./mesix ASP AT2 00006 # 12 dB atten ./mesix DP_ DRX "3 1 37900000 3 0 0" # Beam, Tuning, Freq, BW, Gain, Subslot ./mesix DP_ DRX "3 2 74030000 3 0 0" # Beam, Tuning, Freq, BW, Gain, Subslot ./mesix DP_ BAM "3 all_zero_delay.df fringe_254stand_258ref.gf 0" ./mesix DR3 REC "56071 38040000 3600000 DRX_FILT_3" # MJD, MPM, Len, Format 1030 PASI looks OK, but DP giving typical error: "T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected|" INI'ed DP and restarted TBN 1400 Copied fringing observation on DR3 to an external disk: ./mesix DR3 CPY \ "056071_000019603 0 14512495872 /dev/sda1 056071_000019603_DRX_B3.dat" Results at dr3:/media/kiker/056071_000019603_DRX_B3.dat 0109 Calibration is lost for no particular reason I can discern ... "T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected|" 1200 INI'ed DP and restarted TBN 1410 System still reporting calibration loss; INI'ed again, and now OK 1750 INI'ed DP; restarted TBN; scheduled Joe's obs in 120526_CygA_0651.sh: 0010 INI'ed DP; did not restart TBN, in anticipation of Joe's 8 hr obs 0920 Joe started copying 1000 INI'ed DP and restarted TBN 2120 Submitted ~/MCS/tp/120528/Planning/LR001_120528_1025_0023_B2_NS.sdf for the "Pulsar 19hrLST monitoring" campaign. Note: did not show up in mbox/mesq.dat the first time I ran tpss; showed up within seconds the second time. 0850 INI'ed DP and restarted TBN 0900 Ran "postObserationActions.sh LR001_120528_1025_0023_B2_NS" 0930 Submitted ~/MCS/tp/120529/Planning/LR001_120529_0020_0024_B2_NS.sdf for the "Pulsar 9hrLST monitoring" campaign. 1910 Installed a new SSMIF: cd MCS/tp ./tprs ~/SSMIF_120516.txt cp ssmif.dat mbox/ cp ~/SSMIF_120516.txt ~/MCS/tp/done cp ssmif.dat ~/MCS/tp/done/SSMIF_120516.dat 2100 To check that the SSMIF install worked, take fringes for Cyg A, which transits at 04:34 MT = 10:34 UT. Setup script from Joe C. "cd ~/MCS/exec; at 03:35 05/30/12 -f ./120530_CygA_0937.sh" Scheduled at TBN shutdown for before the above script runs: cd ~/MCS/sch; echo ./INIdp.sh | at 03:20 05/30/12 Output written to dr1:/LWADATA/056077_000049550_spectrometer.dat 0705 Restarted TBN. 0925 TBN was no good because of wrong ASP settings; ran setARX_split.sh. INI'ed DP, restarted TBN. Still getting persistent MGT underflow messages, even though no beamforming / TBN interaction is going on. 1020 Ran "postObserationActions.sh LR001_120529_0020_0024_B2_NS" DR storage, as of 1100: =========== Storage report for DR1 =========== DR1 DRSUs Attached: 2 DRSU selected: 1 DRSU1: /dev/md0 Barcode: '<Not Implemented Yet>' Total Storage: 13972.59 GB Remaining Storage: 13835.87 GB Contiguous Storage: 13835.87 GB Usage: 0% File Count: 1 files DRSU2: /dev/md1 Barcode: '<Not Implemented Yet>' Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% File Count: 0 files =========== Storage report for DR2 =========== DR2 DRSUs Attached: 2 DRSU selected: 1 DRSU1: /dev/md0 Barcode: '<Not Implemented Yet>' Total Storage: 13972.59 GB Remaining Storage: 10560.80 GB Contiguous Storage: 10560.80 GB Usage: 24.00% File Count: 13 files DRSU2: /dev/md1 Barcode: '<Not Implemented Yet>' Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% File Count: 0 files =========== Storage report for DR3 =========== DR3 DRSUs Attached: 2 DRSU selected: 2 DRSU1: /dev/md0 Barcode: '<Not Implemented Yet>' Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% File Count: 0 files DRSU2: /dev/md1 Barcode: '<Not Implemented Yet>' Total Storage: 13972.59 GB Remaining Storage: 12851.80 GB Contiguous Storage: 12851.80 GB Usage: 8.00% File Count: 10 files =========== Storage report for DR4 =========== DR4 DRSUs Attached: 2 DRSU selected: 2 DRSU1: /dev/md0 Barcode: '<Not Implemented Yet>' Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% File Count: 0 files DRSU2: /dev/md1 Barcode: '<Not Implemented Yet>' Total Storage: 13972.59 GB Remaining Storage: 12974.83 GB Contiguous Storage: 12974.83 GB Usage: 7.00% File Count: 5 files =========== Storage report for DR5 =========== DR5 DRSUs Attached: 1 DRSU selected: 1 DRSU1: /dev/md0 Barcode: '<Not Implemented Yet>' Total Storage: 9315.08 GB Remaining Storage: 6199.83 GB Contiguous Storage: 6199.83 GB Usage: 33.00% File Count: 1 files 1230 Submitted ~/MCS/tp/120531/Planning/LR001_120531_0013_0025_B2_NS.sdf and ~/MCS/tp/120531/Planning/LR001_120531_1013_0026_B2_NS.sdf for the Pulsar 09hrLST and 19hrLST campaigns. Scheduled DP INI prior to observation: cd ~/MCS/exec; echo ./INIdp.sh | at 18:00 05/30/12 2350 Reverted to the 120218 SSMIF: cd ~/MCS/tp ./tprs SSMIF_120218.txt cp ssmif.dat mbox/ Scheduled another Cyg A drift scan -- same as last night's: "cd ~/MCS/exec; at 03:35 05/31/12 -f ./120531_CygA_0937.sh" 0715 INI'ed DP, restarted TBN 1440 INI'ed DP and restarted TBN because I got sick of looking at MGT underflow(s) 2215 Restored 120516 SSMIF Scheduled another Cyg A drift scan -- same as last night's: cd ~/MCS/exec; at 03:35 06/01/12 -f ./120601_CygA_0937.sh Scheduled a TBN shutdown for before the above script runs: cd ~/MCS/sch; echo ./INIdp.sh | at 03:20 06/01/12 0900 INI'ed DP, restarted TBN 1920 Holy crap I accidentally ran "rm *" in ~/MCS/tp/. Did my best to restore everything, then sent an e-mail to the staff exploder notifying others. Dammit dammit dammit. 1945 Scheduled Frank's Cyg A transit observations: cd ~/MCS/sch/operatorScripts ./makeDayFolder.sh 120602 cd ~/MCS/tp cp ~/120602_CygA_Track_p.sdf \ 120602/Planning/COMM_120602_0955_0001_B1_XX.sdf cp ~/120602_CasA_Track_p.sdf \ 120602/Planning/COMM_120602_0955_0002_B2_XX.sdf ./tpss 120602/Planning/COMM_120602_0955_0001_B1_XX.sdf 5 0 mbox ./tpss 120602/Planning/COMM_120602_0955_0002_B2_XX.sdf 5 0 mbox cd ~/MCS/sch; echo ./INIdp.sh | at 03:30 06/02/12 cd ~/MCS/sch; at 03:45 06/02/12 -f ./120602_b34_0955.sh cd ~/MCS/sch; echo './INIdp.sh;./startTBN_split74.sh' | at 05:30 06/02/12 NOTE ERROR: the script queued with at needed to be run in exec/! These observations weren't useful due to this mistate. 0930 Copied previous night's Cyg A transit observations: cd ~/MCS/exec ./mesix DR1 CPY \ "056080_000065139 0 305698313472 /dev/sdf1 056080_000065139_DRX_B1.DAT" ./mesix DR2 CPY \ "056080_000065140 0 305699696352 /dev/sda1 056080_000065140_DRX_B2.DAT" 2300 Scheduled redo of Frank's Cyg A transit observations. cd ~/MCS/sch/operatorScripts ./makeDayFolder.sh 120604 cd ~/MCS/tp cp ~/120604_CygA_Track_p.sdf \ 120604/Planning/COMM_120604_0947_0001_B1_XX.sdf cp ~/120604_CasA_Track_p.sdf \ 120604/Planning/COMM_120604_0947_0002_B2_XX.sdf ./tpss 120604/Planning/COMM_120604_0947_0001_B1_XX.sdf 5 0 mbox ./tpss 120604/Planning/COMM_120604_0947_0002_B2_XX.sdf 5 0 mbox cd ~/MCS/sch; echo ./INIdp.sh | at 03:25 06/04/12 cd ~/MCS/exec; at 03:40 06/04/12 -f ./120604_b34_0947.sh cd ~/MCS/sch; echo './INIdp.sh;./startTBN_split74.sh' | at 05:30 06/04/12 1755 Submitted ~/MCS/tp/120605/Planning/LR001_120605_0000_0027_B2_NS.sdf and ~/MCS/tp/120605/Planning/LR001_120605_0954_0028_B2_NS.sdf for the Pulsar 09hrLST and 19hrLST campaigns. 1800 Noticed that ASP status was SHUTDOWN -- not sure how that happened. Restarted ASP and set filter values to 0, 15, 8, 6 as per observation; first five minutes of that obs are likely bad due to ASP reset. 0745 Restarted TBN 0800 Performed post-run actions on pulsar observations. Storage report: ================ Storage report for DR1 ================ Current DRSU: 1 of 2 DRSU1: /dev/md0 Barcode: "<Not Implemented Yet>" Total Storage: 13972.60 GB Remaining Storage: 13260.61 GB Contiguous Storage: 13260.61 GB Usage: 5.00% Files: 1 056067_000030759 DRX_FILT_5 136.69 GB 2 056080_000065139 DRX_FILT_7 284.70 GB 3 056082_000075344 DRX_FILT_7 284.45 GB DRSU2: /dev/md1 Barcode: "<Not Implemented Yet>" Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% Files: (none) ================ Storage report for DR2 ================ Current DRSU: 1 of 2 DRSU1: /dev/md0 Barcode: "<Not Implemented Yet>" Total Storage: 13972.60 GB Remaining Storage: 8778.53 GB Contiguous Storage: 8778.53 GB Usage: 37.00% Files: 1 056064_000001823 DRX_FILT_7 331.87 GB 2 056064_000004995 DRX_FILT_7 265.65 GB 3 056065_000010373 DRX_FILT_4 81.17 GB 4 056066_000017892 DRX_FILT_4 81.17 GB 5 056067_000023670 DRX_FILT_7 331.87 GB 6 056067_000025682 DRX_FILT_4 81.04 GB 7 056067_000027743 DRX_FILT_7 265.65 GB 8 056067_000030756 DRX_FILT_7 663.01 GB 9 056068_000001067 DRX_FILT_4 81.16 GB 10 056069_000008505 DRX_FILT_7 265.64 GB 11 056070_000012427 DRX_FILT_7 331.85 GB 12 056075_000039381 DRX_FILT_7 265.53 GB 13 056076_000042552 DRX_FILT_7 331.87 GB 14 056078_000052666 DRX_FILT_7 331.87 GB 15 056078_000054919 DRX_FILT_7 265.65 GB 16 056080_000065140 DRX_FILT_7 284.71 GB 17 056082_000075345 DRX_FILT_7 284.31 GB 18 056083_000079413 DRX_FILT_7 331.85 GB 19 056083_000082319 DRX_FILT_7 265.65 GB DRSU2: /dev/md1 Barcode: "<Not Implemented Yet>" Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% Files: (none) ================ Storage report for DR3 ================ Current DRSU: 2 of 2 DRSU1: /dev/md0 Barcode: "<Not Implemented Yet>" Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% Files: (none) DRSU2: /dev/md1 Barcode: "<Not Implemented Yet>" Total Storage: 13972.60 GB Remaining Storage: 12584.24 GB Contiguous Storage: 12584.24 GB Usage: 9.00% Files: 1 056065_000010374 DRX_FILT_4 81.15 GB 2 056066_000017890 DRX_FILT_4 81.17 GB 3 056067_000025683 DRX_FILT_4 81.02 GB 4 056067_000030757 DRX_FILT_7 663.01 GB 5 056068_000001068 DRX_FILT_4 81.16 GB 6 056069_000010999 DRX_FILT_3 0.04 GB 7 056069_000011089 DRX_FILT_3 0.04 GB 8 056069_000011234 DRX_FILT_3 0.04 GB 9 056071_000019603 DRX_FILT_3 13.52 GB 10 056073_000028850 DRX_FILT_3 108.13 GB 11 056082_000075325 DRX_FILT_7 264.91 GB ================ Storage report for DR4 ================ Current DRSU: 2 of 2 DRSU1: /dev/md0 Barcode: "<Not Implemented Yet>" Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% Files: (none) DRSU2: /dev/md1 Barcode: "<Not Implemented Yet>" Total Storage: 13972.60 GB Remaining Storage: 12707.27 GB Contiguous Storage: 12707.27 GB Usage: 9.00% Files: 1 056065_000010375 DRX_FILT_4 81.15 GB 2 056066_000017891 DRX_FILT_4 81.17 GB 3 056067_000025684 DRX_FILT_4 81.09 GB 4 056067_000030758 DRX_FILT_7 663.01 GB 5 056068_000001069 DRX_FILT_4 81.16 GB 6 056082_000075326 DRX_FILT_7 264.91 GB ================ Storage report for DR5 ================ Current DRSU: 1 of 1 DRSU1: /dev/md0 Barcode: "<Not Implemented Yet>" Total Storage: 9315.08 GB Remaining Storage: 6199.83 GB Contiguous Storage: 6199.83 GB Usage: 33.00% Files: 1 056067_000028900 DEFAULT_TBN 3115.22 GB 0815 JH handing control to Greg Taylor