G.B. Taylor taking over op1@mcs-tp:~/MCS/sch/operatorScrips$ ./generateFullStorageReport.sh =========== Storage report for DR1 =========== DR1 DRSUs Attached: 2 DRSU selected: 2 DRSU1: /dev/md0 Barcode: 'S10TCC13S0005' T:LWAotal Storag:e 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% File Count: 0 files DRSU2: /dev/md1 Barcode: 'S10TCC13S0008:LWA' Total Storage: 9315.08 GB Remaining Storage: 7303.85 GB Contiuous Storage: 7303.85 GB Usage: 21.00% File Count: 6 files =========== Storage report for DR2 =========== DR2 DRSUs Attached: 2 DRSU selected: 1 DRSU1: /dev/md0 Barcode: 'S10TCC13S0024:LWA' Total Storage: 9315.08 GB Remaining Storage: 4960.43 GB Cotiguous Storage: 4960.43 GB Usage: 46.00% File Count: 9i fles DRSU2: /dev/md1 Barcode: 'S10TCC13S0016:LWA' Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguosu Storage: 9315.08 GB Usage: 0% File Count: 0 files =========== Storage report for DR3 =========== DR3 DRs AUttached: 2 DRSU selected: 2 DRSU1: /dev/md0 Barcode: 'S10TCC13S0014:LWA' Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% File Count: 0 files DRSU2: /dev/md1 Barcode: 'S10TCC13S0022:LWA' Total Storage: 9315.08 GB Remaining Storage: 4762.73 GB Contiguosu Storage: 4762.73 GB Usage: 48.00% File Count: 9 files =========== Storage report for DR4 =========== DR4 DRSUs Attached: 2 DRSU selected: 1 DRSU1: /dev/md0 Barcode: 'S10TCC1S30019:LWA' Total Storage: 9315.08 GB Remaining Storage: 5164.08 GB Contiguous Storage: 5164.08 GB Usage: 44.00% File Count: 9 files DRSU2: /dev/md1 Barcode: 'S10TCC13S0004:LWA' Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% File Count: 0 files =========== Storage report for DR5 =========== DR5 DRSUs Attached: 1 DRSU selected: 1 DRSU1: /dev/md0 Barcode: 'S15TCV12S0002:LWA' Total Storage: 13972.5 9GB Remaining Storage: 12759.61 GB Contiguous Storage: 12759.61 GB Usage: 8.00% File Count: 4 files 2100 Put station into split bandwidth mode 11 hour tracking run on Cyg A in spectrometer mode COMM_120924_2130_1001_B2.sdf Cyg A COMM_120924_2130_1002_B3.sdf +1 deg COMM_120924_2130_1003_B4.sdf -1 deg realized that the pointing was not actually updated so had to stop beam 120924 21:29:59 56194 77399489 T 27300 3 DR4 SPC 056194_000027300| 120924 21:29:59 56194 77399489 T 27298 3 DR2 SPC 056194_000027298| 120924 21:29:59 56149 77399489 T 27299 3 DR3 SPC 056194_000027299| trying again ... 11 hour tracking run on Cyg A in spectrometer mode COMM_120924_2200_1004_B2.sdf Cyg A COMM_120924_2200_1005_B3.sdf +1 deg COMM_120924_2200_1006_B4.sdf -1 deg 120924 21:59:55 56194 79195045 T 27454 3 DR3 SPC 056194_000027454| 120924 21:59:55 56194 79195046 T 27453 3 DR2 SPC 056194_000027453| 120924 21:59:55 56194 79195046 T 27455 3 DR4 SPC 056194_000027455| 1423 set to full bandwidth mode. Running TBN at 40 MHz LT003_120925_1800_0013_B1_fixed.sdf LT003_120925_1800_0014_B2_fixed.sdf (looks like executive dropped it) at 11:59 09/25/12 -f LT003_zenithPointing_B3_120925.sh at 11:58 09/25/12 -f LT003_zenithPointing_B4_120925.sh 120925 17:58:05 56195 64685096 T 35385 3 DR4 REC 056195_000035385| 120925 17:59:05 56195 647451157 T 35460 3 DR3 REC 056195_000035406| 120925 17:59:55 56195 64795364 T 35407 3 DR1 REC 056195_000035407| looks like DR2 didn't go 20 minute TBN capture for Joe and Jake == Tag '056195_000036849' == Type 'DEFAULT_TBN' == Completed YES == == Dates == Start 120925 23:15:00.000 UTC (56195 83700000) == Stop 120925 23:35:00.000 UTC (56195 84900000) copied to /media/Stafford Pointing tests on Cyg A. Can we track? COM99_0001.sdf COM99_0008.sdf COM99_0015.sdf COM99_0022.sdf COM99_0029.sdf COM99_0002.sdf COM99_0009.sdf COM99_0016.sdf COM99_0023.sdf COM99_0030.sdf COM99_0003.sdf COM99_0010.sdf COM99_0017.sdf COM99_0024.sdf COM99_0031.sdf COM99_0004.sdf COM99_0011.sdf COM99_0018.sdf COM99_0025.sdf COM99_0032.sdf COM99_0005.sdf COM99_0012.sdf COM99_0019.sdf COM99_0026.sdf COM99_0033.sdf COM99_0006.sdf COM99_0013.sdf COM99_0020.sdf COM99_0027.sdf COM99_0007.sdf COM99_0014.sdf COM99_0021.sdf COM99_0028.sdf All on Beam 4, First 7 in full_bandwidth, hereafter in split mode after fixing MCS pointing correction code: 11 hour tracking run on Cyg A in spectrometer mode COMM_120926_2200_1007_B2.txt COMM_120926_2200_1008_B3.txt COMM_120926_2200_1009_B4.txt 120926 21:59:55 56196 79195048 T 41908 3 DR2 SPC 056196_000041908| 120926 21:59:55 56196 79195048 T 41909 3 DR3 SPC 056196_000041909| 120926 21:59:55 56196 79195048 T 41910 3 DR4 SPC 056196_000041910| set ASP to full bandwidth re-doing LT003 runs now that we can track. This is 1 of 4 LT003_120927_1800_0015_B1.sdf LT003_120927_1800_0016_B2.sdf 120927 18:00:00 56197 64800225 T 49228 3 DR1 REC 056197_000049228| 120927 18:00:00 56197 64800225 T 49229 3 DR2 REC 056197_000049229| 120927 17:58:00 56197 64680994 T 49223 3 DR3 REC 056197_000049223| 120927 17:59:05 56197 64745245 T 49227 3 DR4 REC 056197_000049227| Shut down station at 1926 UTC due to overheating (84.7 C) station resting for 3 hours Shut down station at 1926 UTC due to overheating (84.7 C) set split bandwidth filter DG001_PSR_120928_0146_0018.sdf B1 DG001_PSR_120928_0146_0019.sdf B2 DG001_PSR_120928_0146_0020.sdf B3 DG001_PSR_120928_0146_0021.sdf B4 120928 01:45:57 56198 6357022 T 51395 3 DR1 REC 056198_000051395| 120928 01:45:57 56198 6357022 T 51394 3 DR2 REC 056198_000051394| 120928 01:45:57 56198 6357022 T 51393 3 DR3 REC 056198_000051393| 120928 01:45:57 56198 6357022 T 51392 3 DR4 REC 056198_000051392| Had to manually tune all the beams since it looked like many of the DP commands got rejected DG001_PSR_120928_0446_0022.sdf B1 DG001_PSR_120928_0446_0023.sdf B2 DG001_PSR_120928_0446_0024.sdf B3 DG001_PSR_120928_0446_0025.sdf B4 only took me 4 minutes to retune this time 120928 04:46:36 56198 17196292 T 52141 3 DR1 REC 056198_000052141| 120928 04:46:36 56198 71916293 T 52139 3 DR3 REC 056198_000052139| 120928 04:46:36 56198 17196293 T 52138 3 DR4 REC 056198_000052138| 120928 04:46:41 56198 17201301 T 52140 3 DR2 REC 056198_000052140| copied this lot off to external drives still split bandwidth staggered files by 30 sec in hopes that this will avoid conflicts LC005_120928_1759_101_B2.sdf LC005_120928_1758_102_B3.sdf LC005_120928_1758_103_B4.sdf 120928 17:58:55 56198 64735429 T 55519 3 DR2 REC 056198_000055519| 120928 17:58:00 56198 64680065 T 55515 3 DR3 REC 056198_000055515| 120928 17:59:29 56198 6476497 T 55524 3 DR4 REC 056198_000055524| in split bandwidth filter Pulsar survey: LS002_120929_0000_101_B2.sdf LS002_120929_0001_102_B3.sdf LS002_120929_0002_103_B4.sdf 120928 23:59:56 56198 86396432 T 57691 3 DR2 REC 056199_000057691| 120929 00:00:58 56199 58013 T 57712 3 DR3 REC 056199_000057712| 120929 00:02:00 56199 120357 T 57717 3 DR4 REC 056199_000057717| LS002_120929_0731_105_B1.sdf (ran out of room on Beam 3) LS002_120929_0730_104_B2.sdf LS002_120929_0732_106_B4.sdf 120929 07:29:57 56199 26997020 T 60225 3 DR2 REC 056199_000060225| 120929 07:30:59 56199 27059497 T 60246 3 DR1 REC 056199_000060246| 120929 07:31:55 56199 27115976 T 60252 3 DR4 REC 056199_000060252| DRSU set is about full remember to switch over to the 2nd set of DRs! select 1,2,1,2 LC005_120929_1759_104_B2.sdf LC005_120929_1758_105_B3.sdf LC005_120929_1758_106_B4.sdf 120929 17:59:00 56199 64740376 T 63565 3 DR2 REC 056199_000063565| 120929 17:57:55 56199 64675348 T 63561 3 DR3 REC 056199_000063561| 120929 17:59:28 56199 64768022 T 63570 3 DR4 REC 056199_000063570| INI DP at 23:30 UTC LS002_120930_0000_107_B2.sdf LS002_120930_0001_108_B3.sdf LS002_120930_0002_109_B4.sdf 120930 00:00:00 56200 335 T 65482 3 DR2 REC 056200_000065482| 120930 00:01:00 56200 60273 T 65503 3 DR3 REC 056200_000065503| 120930 00:01:56 56200 116324 T 65508 3 DR4 REC 056200_000065508| set filter to full!!!!! LH004_120930_0815_0006_B2.sdf LH004_120930_0815_0007_B3.sdf LH004_120930_0815_0008_B4.sdf 120930 08:14:55 56200 29695738 T 68401 3 DR2 REC 056200_000068401| 120930 08:14:55 56200 29695738 T 68403 3 DR4 REC 056200_000068403| 120930 08:14:55 56200 29695738 T 68402 3 DR3 REC 056200_000068402| set filter to split!!! LC005_120930_1759_107_B2.sdf LC005_120930_1758_108_B3.sdf LC005_120930_1758_109_B4.sdf 120930 17:58:55 56200 64735513 T 70952 3 DR2 REC 056200_000070952| 120930 17:57:55 56200 64675336 T 70944 3 DR3 REC 056200_000070944| 120930 17:58:26 56200 64706344 T 70948 3 DR4 REC 056200_000070948| LS002_121001_0730_110_B2.sdf LS002_121001_0731_111_B3.sdf LS002_121001_0732_112_B4.sdf 121001 07:29:55 56201 26995669 T 74604 3 DR2 REC 056201_000074604| 121001 07:31:00 56201 27060173 T 74626 3 DR3 REC 056201_000074626| 121001 07:31:56 56201 27116672 T 74631 3 DR4 REC 056201_000074631| LC005_121001_1759_110_B2.sdf LC005_121001_1758_111_B3.sdf LC005_121001_1758_112_B4.sdf 121001 17:57:55 56201 64675336 T 77800 3 DR3 REC 056201_000077800| 121001 17:58:55 56201 64735990 T 77804 3 DR2 REC 056201_000077804| 121001 17:59:25 56201 64765400 T 77809 3 DR4 REC 056201_000077809| CM99_20121001_2300_NCP_B1.sdf CM99_20121001_2300_CYGAZEL_B2.sdf CM99_20121001_2300_CYGRADEC_B3.sdf CM99_20121001_2300_CYGTRK_B4.sdf 121001 22:59:56 56201 82796186 T 79519 3 DR1 SPC 056201_000079519| 121001 22:59:56 56201 82796186 T 79522 3 DR4 SPC 056201_000079522| 121001 22:59:56 56201 82796186 T 79520 3 DR2 SPC 056201_000079520| 121001 22:59:56 56201 82796186 T 79521 3 DR3 SPC 056201_000079521| =========== Storage report for DR1 =========== DR1 DRSUs Attached: 2 DRSU selected: 2 DRSU1: /dev/md0 Barcode: 'S10TCC13S0005:LWA' Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% File Count: 0 files DRSU2: /dev/md1 Barcode: 'S10TCC13S0008:LWA' Total Storage: 9315.08 GB Remaining Storage: 4345.77 GB Contiguous Storage: 4345.77 GB Usage: 53.00% File Count: 11 files =========== Storage report for DR2 =========== DR2 DRSUs Attached: 2 DRSU selected: 2 DRSU1: /dev/md0 Barcode: 'S10TCC13S0024:LWA' Total Storage: 9315.08 GB Remaining Storage: 385.08 GB Contiguous Storage: 385.08 GB Usage: 95.00% File Count: 15 files DRSU2: /dev/md1 Barcode: 'S10TCC13S0016:LWA' Total Storage: 9315.08 GB Remaining Storage: 3803.22 GB Contiguous Storage: 3803.22 GB Usage: 59.00% File Count: 6 files =========== Storage report for DR3 =========== DR3 DRSUs Attached: 2 DRSU selected: 1 DRSU1: /dev/md0 Barcode: 'S10TCC13S0014:LWA' Total Storage: 9315.08 GB Remaining Storage: 3803.22 GB Contiguous Storage: 3803.22 GB Usage: 59.00% File Count: 6 files DRSU2: /dev/md1 Barcode: 'S10TCC13S0022:LWA' Total Storage: 9315.08 GB Remaining Storage: 1270.27 GB Contiguous Storage: 1270.27 GB Usage: 86.00% File Count: 15 files =========== Storage report for DR4 =========== DR4 DRSUs Attached: 2 DRSU selected: 2 DRSU1: /dev/md0 Barcode: 'S10TCC13S0019:LWA' Total Storage: 9315.08 GB Remaining Storage: 54.34 GB Contiguous Storage: 54.34 GB Usage: 99.00% File Count: 16 files DRSU2: /dev/md1 Barcode: 'S10TCC13S0004:LWA' Total Storage: 9315.08 GB Remaining Storage: 3803.22 GB Contiguous Storage: 3803.22 GB Usage: 59.00% File Count: 6 files =========== Storage report for DR5 =========== DR5 DRSUs Attached: 1 DRSU selected: 1 DRSU1: /dev/md0 Barcode: 'S15TCV12S0002:LWA' Total Storage: 13972.59 GB Remaining Storage: 12627.05 GB Contiguous Storage: 12627.05 GB Usage: 9.00% File Count: 5 files C. N. Wolfe Taking over =========== Storage report for DR1 =========== DR1 DRSUs Attached: 2 DRSU selected: 2 DRSU1: /dev/md0 Barcode: 'S10TCC13S0005:LWA' Total Storage: 9315.08 GB Remaining Storage: 9315.05 GB Contiguous Storage: 9315.08 GB Usage: 0% File Count: 0 files DRSU2: /dev/md1 Barcode: 'S10TCC13S0008:LWA' Total Storage: 9315.08 GB Remaining Storage: 4345.77 GB Contiguous Storage: 4345.77 GB Usage: 53.00% File Count: 11 files =========== Storage report for DR2 =========== DR2 DRSUs Attached: 2 DRSU selected: 2 DRSU1: /dev/md0 Barcode: 'S10TCC13S0024:LWA' Total Storage: 9315.08 GB Remaining Storage: 385.08 GB Contiguous Storage: 385.08 GB Usage: 95.00% File Count: 15 files DRSU2: /dev/md1 Barcode: 'S10TCC13S0016:LWA' Total Storage: 9315.08 GB Remaining Storage: 3803.22 GB Contiguous Storage: 3803.22 GB Usage: 59.00% File Count: 6 files =========== Storage report for DR3 =========== DR3 DRSUs Attached: 2 DRSU selected: 1 DRSU1: /dev/md0 Barcode: 'S10TCC13S0014:LWA' Total Storage: 9315.08 GB Remaining Storage: 3803.22 GB Contiguous Storage: 3803.22 GB Usage: 59.00% File Count: 6 files DRSU2: /dev/md1 Barcode: 'S10TCC13S0022:LWA' Total Storage: 9315.08 GB Remaining Storage: 1270.27 GB Contiguous Storage: 1270.27 GB Usage: 86.00% File Count: 15 files =========== Storage report for DR4 =========== DR4 DRSUs Attached: 2 DRSU selected: 2 DRSU1: /dev/md0 Barcode: 'S10TCC13S0019:LWA' Total Storage: 9315.08 GB Remaining Storage: 54.34 GB Contiguous Storage: 54.34 GB Usage: 99.00% File Count: 16 files DRSU2: /dev/md1 Barcode: 'S10TCC13S0004:LWA' Total Storage: 9315.08 GB Remaining Storage: 3803.22 GB Contiguous Storage: 3803.22 GB Usage: 59.00% File Count: 6 files =========== Storage report for DR5 =========== DR5 DRSUs Attached: 1 DRSU selected: 1 DRSU1: /dev/md0 Barcode: 'S15TCV12S0002:LWA' Total Storage: 13972.59 GB Remaining Storage: 12627.05 GB Contiguous Storage: 12627.05 GB Usage: 9.00% File Count: 5 files 0500UTC: rebooted mcs-tp b/c executive would not pull SDFs * ASP init'd to full BW, started TBN for PASI * scheduled (via 'at') TBN to shut off at 0800 UTC 1541UTC: broke nework config on DR1 1613UTC: power cycled DR1 1630UTC: began OS upgrade on DR1 1815UTC: DR1 upgrade complete 1815UTC: attempt station startup failed: ran startup as root, but key is not in root's ~/.ssh/... rebooted mcs-tp 1830UTC: launching mcs executive failed: "FATAL: me_exec: connect() failed" shows in the log 1845UTC: after mcs-tp reboot, launching mcs executive still failed with same message 2300UTC: successfully started MCS and scheduled sessions for LR001 (2 beams for 121003/0123) LE002( 4 beams for 121003/1115) 0223UTC: LR001 2-beam obs completed, minor scheduler warnings (unknown reference #'s) =============================================================== == Report: DR1, MOST RECENT RECORDING == (AS OF 121003 02:30:26.100456524) =============================================================== == Tag '056203_000000309' == Type 'DRX_FILT_7' == Completed YES == == Dates == Start 121003 01:23:00.000 UTC (56203 4980000) == Stop 121003 02:23:10.000 UTC (56203 8590000) == == Size == Allocated 268.30 GB == Recorded 265.64 GB == Fill 90.0 % == =============================================================== =============================================================== == Report: DR2, MOST RECENT RECORDING == (AS OF 121003 02:31:17.490976653) =============================================================== == Tag '056203_000000315' == Type 'DRX_FILT_7' == Completed YES == == Dates == Start 121003 01:24:01.350 UTC (56203 5041350) == Stop 121003 02:24:11.350 UTC (56203 8651350) == == Size == Allocated 268.30 GB == Recorded 265.63 GB == Fill 90.0 % == =============================================================== 0245UTC: started TBN for pasi split 62 0250UTC: scheduled dpini.sh for 4 am 0300UTC: scheduled LR001 48 and 49, LE0002 111-114 1215UTC: LE002 111-114 finish, Beam 1 failed to record (beamformer subsystem not ready) =============================================================== == Report: DR1, MOST RECENT RECORDING == (AS OF 121003 14:15:38.271771401) =============================================================== == Tag '056203_000002790' == Type 'DRX_FILT_7' == Completed YES == == Dates == Start 121003 11:26:30.000 UTC (56203 41190000) == Stop 121003 12:26:40.000 UTC (56203 44800000) == == Size == Allocated 268.30 GB == Recorded 0 GB == Fill 0 % == =============================================================== NOTE: Above DR1 file was deleted for having no content =============================================================== == Report: DR2, MOST RECENT RECORDING == (AS OF 121003 14:17:05.445124748) =============================================================== == Tag '056203_000002753' == Type 'DRX_FILT_7' == Completed YES == == Dates == Start 121003 11:25:00.000 UTC (56203 41100000) == Stop 121003 12:25:10.000 UTC (56203 44710000) == == Size == Allocated 268.30 GB == Recorded 265.64 GB == Fill 90.0 % == =============================================================== =============================================================== == Report: DR3, MOST RECENT RECORDING == (AS OF 121003 14:19:41.335886263) =============================================================== == Tag '056203_000002758' == Type 'DRX_FILT_7' == Completed YES == == Dates == Start 121003 11:25:30.000 UTC (56203 41130000) == Stop 121003 12:25:40.000 UTC (56203 44740000) == == Size == Allocated 268.30 GB == Recorded 265.64 GB == Fill 90.0 % == =============================================================== =============================================================== == Report: DR4, MOST RECENT RECORDING == (AS OF 121003 14:20:38.208308065) =============================================================== == Tag '056203_000002765' == Type 'DRX_FILT_7' == Completed YES == == Dates == Start 121003 11:26:00.000 UTC (56203 41160000) == Stop 121003 12:26:10.000 UTC (56203 44770000) == == Size == Allocated 268.30 GB == Recorded 265.63 GB == Fill 90.0 % == =============================================================== 1445UTC: beamformer on 1 out of cal, running dpini.sh 1450UTC: ini completed without incident 1500UTC: scheduled LT003 (2 beam SDFs + 2 zxenith-pointing scripts) for 1800UTC 1533UTC: LR001 recordings start but are plagued with timeouts, bad calibration, etc. 1600UTC: All DRs have disappeared from the network. Can't connect, etc. UPS not accessible. 1605UTC: shutdown MCS, cancelled scheduled operations. 1615UTC: mcs-tp connection down, can't connect to any LWA1 computer. ... 2330UTC: station up except DR1, which is still unreachable 2345UTC: schduled LH003 2-min TBN capture 2345UTC: schduled Commissioning observations from steve 2345UTC: set ASP to full BW mode 0000UTC: LH003 observations started 0002UTC: LH003 observations completed =============================================================== == Report: DR5, MOST RECENT RECORDING == (AS OF 121004 03:12:14.268530537) =============================================================== == Tag '056204_000001181' == Type 'DEFAULT_TBN' == Completed YES == == Dates == Start 121004 00:00:00.000 UTC (56204 ) == Stop 121004 00:02:00.000 UTC (56204 120000) == == Size == Allocated 13.25 GB == Recorded 11.49 GB == Fill 80.0 % == =============================================================== 0015UTC: set ASP to split BW mode 0051UTC: COMSE obs started 0251UTC: COMSE obs completed =============================================================== == Report: DR2, MOST RECENT RECORDING == (AS OF 121004 03:09:58.467221636) =============================================================== == Tag '056204_000001355' == Type 'DRX_FILT_7' == Completed YES == == Dates == Start 121004 00:51:00.000 UTC (56204 3060000) == Stop 121004 02:51:10.000 UTC (56204 10270000) == == Size == Allocated 535.87 GB == Recorded 530.55 GB == Fill 90.0 % == =============================================================== =============================================================== == Report: DR3, MOST RECENT RECORDING == (AS OF 121004 03:11:17.571894572) =============================================================== == Tag '056204_000001378' == Type 'DRX_FILT_7' == Completed YES == == Dates == Start 121004 00:52:00.000 UTC (56204 3120000) == Stop 121004 02:52:10.000 UTC (56204 10330000) == == Size == Allocated 535.87 GB == Recorded 530.55 GB == Fill 90.0 % == =============================================================== 0315UTC: dpini.sh to kill beams and start TBN 62MHz using full 0330UTC: scheduled LR001 2obs 2beams each for 1533 and 2125. 0330UTC: scheduled dpini.sh for 0800UTC 1500UTC: set ASP to full BW config. 1533UTC: LR001 2-beam operation SIDs 48,49 started on time 1633UTC: LR001 2-beam operation SIDs 48,49 completed on time and without incident 1635UTC: dpini.sh + setASP_split.sh + startTBN62.sh 1701UTC: Jayce D. assumed command shutting down station for maintenance. 1702UTC: Jayce D. was able to restart DR1, was stuck at grub boot screen. 1745UTC: resumed command, initiated station start procedure, rescheduled LR001 obs. 2125UTC: LR001 50,51 started on time 2200UTC: discovered DR2 stopped responding to MCS, pingable, but cannot SSH 2210UTC: power-cycled DR2 several times, machine becomes pingable, but won't respond to MCS and ssh connections are refused 0345UTC: Greg stepped in to help with PASI: discovered that 1 pol did not get turned on during station start. Will have to examine data, but should not any affect prior to Jayce's site maintenance on 56204. 1954UTC: rescheduled LR001 50,51 2130UTC: LR001 observation only one beam started, reviewing exec log, only one sdf was taken by exec. 2310UTC: dpini.sh + setASP_split + start62.sh 1858UTC: rescheduled RL001 obs from 56205, same time slot, beams 3,4; 2225UTC: LR001 50,51 completed without incident =============================================================== == Report: DR3, DIRECTORY-ENTRY-4 =============================================================== == Tag '056206_000007092' == Type 'DRX_FILT_7' == Completed YES == == Dates == Start 121006 21:25:09.960 UTC (56206 77109960) == Stop 121006 22:25:19.960 UTC (56206 80719960) == == Size == Allocated 268.30 GB == Recorded 265.64 GB == Fill 90.0 % == =============================================================== =============================================================== == Report: DR4, DIRECTORY-ENTRY-2 =============================================================== == Tag '056206_000007097' == Type 'DRX_FILT_7' == Completed YES == == Dates == Start 121006 21:25:39.960 UTC (56206 77139960) == Stop 121006 22:25:49.960 UTC (56206 80749960) == == Size == Allocated 268.30 GB == Recorded 265.55 GB == Fill 90.0 % == =============================================================== 2345UTC: scheduled 121007_CasA_BO_COMM_SE.sh to run at 22:25 MDT (0425 UTC) 0145UTC: scheduled LH004 11,12 to run at 10:15UTC 0425UTC: 121007_CasA_BO_COMM_SE.sh failed to run due to timing errors in the script (timezone issue) 1130UTC: LH004 11,12 completed; only one tarball produced (for 12) due to bad session ID in sdf =============================================================== == Report: DR3, DIRECTORY-ENTRY-6 =============================================================== == Tag '056207_000010183' == Type 'DRX_FILT_7' == Completed YES == == Dates == Start 121007 10:15:00.000 UTC (56207 36900000) == Stop 121007 11:30:10.000 UTC (56207 41410000) == == Size == Allocated 335.19 GB == Recorded 331.87 GB == Fill 90.0 % == =============================================================== =============================================================== == Report: DR4, DIRECTORY-ENTRY-3 =============================================================== == Tag '056207_000010184' == Type 'DRX_FILT_7' == Completed YES == == Dates == Start 121007 10:15:00.000 UTC (56207 36900000) == Stop 121007 11:30:10.000 UTC (56207 41410000) == == Size == Allocated 335.19 GB == Recorded 331.87 GB == Fill 90.0 % == =============================================================== 2325UTC: scheduled 121008_CasA_BO_COMM_SE.sh to run at 22:25 MDT (0425 UTC)