2014-05-19 12:25:17 [DEBUG   ] Finished updating ant stats in 0.975 seconds
2014-05-19 12:25:37 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 12:25:56 [DEBUG   ] Finished updating board flags in 1.403 seconds
2014-05-19 12:25:57 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 12:26:17 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 12:26:37 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 12:26:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 12:26:56 [DEBUG   ] Finished updating board flags in 1.442 seconds
2014-05-19 12:26:57 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 12:27:06 [DEBUG   ] Got command RPT from MCS: ref #193084
2014-05-19 12:27:06 [DEBUG   ] summary = NORMAL
2014-05-19 12:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193084  14 56796 44826350 A NORMALNORMAL'
2014-05-19 12:27:06 [DEBUG   ] Got command RPT from MCS: ref #193085
2014-05-19 12:27:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 12:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193085  48 56796 44826351 A NORMALSystem calibrated and operating normally'
2014-05-19 12:27:17 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 12:27:37 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2014-05-19 12:27:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 12:27:56 [DEBUG   ] Finished updating board flags in 1.410 seconds
2014-05-19 12:27:57 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2014-05-19 12:28:17 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2014-05-19 12:28:37 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2014-05-19 12:28:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 12:28:57 [DEBUG   ] Finished updating ant stats in 0.430 seconds
2014-05-19 12:28:57 [DEBUG   ] Finished updating board flags in 1.953 seconds
2014-05-19 12:29:17 [DEBUG   ] Finished updating ant stats in 0.374 seconds
2014-05-19 12:29:38 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:29:56 [DEBUG   ] Finished updating board flags in 1.163 seconds
2014-05-19 12:29:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:30:18 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:30:38 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:30:56 [DEBUG   ] Finished updating board flags in 1.035 seconds
2014-05-19 12:30:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:31:18 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 12:31:38 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:31:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 12:31:56 [DEBUG   ] Finished updating board flags in 0.972 seconds
2014-05-19 12:31:58 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 12:32:06 [DEBUG   ] Got command RPT from MCS: ref #193101
2014-05-19 12:32:06 [DEBUG   ] summary = NORMAL
2014-05-19 12:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193101  14 56796 45126346 A NORMALNORMAL'
2014-05-19 12:32:06 [DEBUG   ] Got command RPT from MCS: ref #193102
2014-05-19 12:32:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 12:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193102  48 56796 45126347 A NORMALSystem calibrated and operating normally'
2014-05-19 12:32:18 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:32:38 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:32:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 12:32:57 [DEBUG   ] Finished updating board flags in 1.404 seconds
2014-05-19 12:32:58 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 12:33:18 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:33:38 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:33:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 12:33:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 12:33:57 [DEBUG   ] Finished updating board flags in 1.414 seconds
2014-05-19 12:33:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:34:18 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:34:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 12:34:57 [DEBUG   ] Finished updating board flags in 1.485 seconds
2014-05-19 12:34:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:35:18 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 12:35:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 12:35:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 12:35:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 12:35:57 [DEBUG   ] Finished updating board flags in 1.308 seconds
2014-05-19 12:35:58 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 12:36:18 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:36:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 12:36:57 [DEBUG   ] Finished updating board flags in 1.204 seconds
2014-05-19 12:36:58 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-19 12:37:06 [DEBUG   ] Got command RPT from MCS: ref #193118
2014-05-19 12:37:06 [DEBUG   ] summary = NORMAL
2014-05-19 12:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193118  14 56796 45426714 A NORMALNORMAL'
2014-05-19 12:37:06 [DEBUG   ] Got command RPT from MCS: ref #193119
2014-05-19 12:37:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 12:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193119  48 56796 45426715 A NORMALSystem calibrated and operating normally'
2014-05-19 12:37:18 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 12:37:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 12:37:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 12:37:57 [DEBUG   ] Finished updating board flags in 1.180 seconds
2014-05-19 12:37:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 12:38:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 12:38:38 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 12:38:58 [DEBUG   ] Finished updating ant stats in 1.029 seconds
2014-05-19 12:38:58 [DEBUG   ] Finished updating board flags in 2.049 seconds
2014-05-19 12:39:18 [DEBUG   ] Finished updating ant stats in 0.975 seconds
2014-05-19 12:39:38 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 12:39:58 [DEBUG   ] Finished updating ant stats in 0.828 seconds
2014-05-19 12:39:58 [DEBUG   ] Finished updating board flags in 1.984 seconds
2014-05-19 12:40:18 [DEBUG   ] Finished updating ant stats in 0.775 seconds
2014-05-19 12:40:38 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 12:40:58 [DEBUG   ] Finished updating ant stats in 0.631 seconds
2014-05-19 12:40:58 [DEBUG   ] Finished updating board flags in 1.810 seconds
2014-05-19 12:41:18 [DEBUG   ] Finished updating ant stats in 0.574 seconds
2014-05-19 12:41:38 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2014-05-19 12:41:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 12:41:57 [DEBUG   ] Finished updating board flags in 1.180 seconds
2014-05-19 12:41:58 [DEBUG   ] Finished updating ant stats in 0.399 seconds
2014-05-19 12:42:06 [DEBUG   ] Got command RPT from MCS: ref #193135
2014-05-19 12:42:06 [DEBUG   ] summary = NORMAL
2014-05-19 12:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193135  14 56796 45726706 A NORMALNORMAL'
2014-05-19 12:42:06 [DEBUG   ] Got command RPT from MCS: ref #193136
2014-05-19 12:42:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 12:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193136  48 56796 45726707 A NORMALSystem calibrated and operating normally'
2014-05-19 12:42:19 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:42:39 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:42:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 12:42:57 [DEBUG   ] Finished updating board flags in 1.024 seconds
2014-05-19 12:42:59 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:43:19 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:43:39 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:43:57 [DEBUG   ] Finished updating board flags in 0.969 seconds
2014-05-19 12:43:59 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:44:19 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:44:39 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:44:58 [DEBUG   ] Finished updating board flags in 1.422 seconds
2014-05-19 12:44:59 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 12:45:14 [DEBUG   ] Got command TBW from MCS: ref #193153
2014-05-19 12:45:14 [DEBUG   ] tbw = exited with status True, code 0x00
2014-05-19 12:45:14 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBW   193153   8 56796 45914198 A NORMAL'
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 pkill tbn
2014-05-19 12:45:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 pkill tbn
2014-05-19 12:45:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 pkill tbn
2014-05-19 12:45:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 pkill tbn
2014-05-19 12:45:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 pkill tbn
2014-05-19 12:45:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 pkill tbn
2014-05-19 12:45:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 pkill tbn
2014-05-19 12:45:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 pkill tbn
2014-05-19 12:45:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 pkill tbn
2014-05-19 12:45:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 pkill tbn
2014-05-19 12:45:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 pkill tbn
2014-05-19 12:45:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 pkill tbn
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa27', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '1', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa24', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '2', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa23', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '3', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa18', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '4', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa10', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '5', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa32', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '6', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa11', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '7', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa30', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '8', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa12', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '9', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa8', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '10', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa13', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '11', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa26', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '12', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa14', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '13', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa22', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '14', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa20', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '15', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa1', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '16', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa5', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '17', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa25', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '18', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa35', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '19', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa19', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '20', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa21', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '21', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa6', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '22', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa7', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '23', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa33', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '24', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa15', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '25', '12000000']
2014-05-19 12:45:16 [DEBUG   ] spawn_process: at 2014-05-19 12:45:16 running ['/usr/local/bin/lwa28', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '26', '12000000']
2014-05-19 12:45:16 [INFO    ] Finished spawning TBW
2014-05-19 12:45:58 [DEBUG   ] Finished updating board flags in 1.481 seconds
2014-05-19 12:46:58 [DEBUG   ] Finished updating board flags in 1.429 seconds
2014-05-19 12:47:06 [DEBUG   ] Got command RPT from MCS: ref #193158
2014-05-19 12:47:06 [DEBUG   ] summary = NORMAL
2014-05-19 12:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193158  14 56796 46026710 A NORMALNORMAL'
2014-05-19 12:47:06 [DEBUG   ] Got command RPT from MCS: ref #193159
2014-05-19 12:47:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 12:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193159  48 56796 46026711 A NORMALSystem calibrated and operating normally'
2014-05-19 12:47:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 12:47:58 [DEBUG   ] Finished updating board flags in 1.267 seconds
2014-05-19 12:48:01 [DEBUG   ] Got command INI from MCS: ref #193177
2014-05-19 12:48:01 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2014-05-19 12:48:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   193177   8 56796 46081007 ABOOTING'
2014-05-19 12:48:20 [INFO    ] -------------------
2014-05-19 12:48:20 [INFO    ]  LWA board status
2014-05-19 12:48:20 [INFO    ] -------------------
2014-05-19 12:48:20 [INFO    ] DP 1: 2 lwa27
2014-05-19 12:48:20 [INFO    ] DP 1: 3 lwa24
2014-05-19 12:48:20 [INFO    ] DP 1: 4 lwa23
2014-05-19 12:48:20 [INFO    ] DP 1: 5 lwa18
2014-05-19 12:48:20 [INFO    ] DP 1: 6 lwa10
2014-05-19 12:48:20 [INFO    ] DP 1: 7 lwa32
2014-05-19 12:48:20 [INFO    ] DP 1: 8 lwa11
2014-05-19 12:48:20 [INFO    ] DP 1: 9 lwa30
2014-05-19 12:48:20 [INFO    ] DP 1: 10 lwa12
2014-05-19 12:48:20 [INFO    ] DP 1: 11 lwa8
2014-05-19 12:48:20 [INFO    ] DP 1: 12 lwa13
2014-05-19 12:48:20 [INFO    ] DP 1: 13 lwa26
2014-05-19 12:48:20 [INFO    ] DP 1: 14 lwa14
2014-05-19 12:48:20 [INFO    ] DP 1: 16 lwa22
2014-05-19 12:48:20 [INFO    ] DP 1: 17 lwa20
2014-05-19 12:48:20 [INFO    ] DP 1: 18 lwa1
2014-05-19 12:48:20 [INFO    ] DP 1: 19 lwa5
2014-05-19 12:48:20 [INFO    ] DP 1: 20 lwa25
2014-05-19 12:48:20 [INFO    ] DP 1: 21 lwa35
2014-05-19 12:48:20 [INFO    ] DP 1: 22 lwa19
2014-05-19 12:48:20 [INFO    ] DP 1: 23 lwa21
2014-05-19 12:48:20 [INFO    ] DP 1: 24 lwa6
2014-05-19 12:48:20 [INFO    ] DP 1: 25 lwa7
2014-05-19 12:48:20 [INFO    ] DP 1: 26 lwa33
2014-05-19 12:48:20 [INFO    ] DP 1: 27 lwa15
2014-05-19 12:48:20 [INFO    ] DP 1: 28 lwa28
2014-05-19 12:48:20 [INFO    ] DP 2: 1 lwa16
2014-05-19 12:48:20 [INFO    ] DP 2: 15 lwa34
2014-05-19 12:48:20 [INFO    ] Total of 28 boards
2014-05-19 12:48:20 [INFO    ] Total of 260 stands
2014-05-19 12:48:20 [INFO    ] -------------------
2014-05-19 12:48:20 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:21 [DEBUG   ] __waitForTBW - lwa27 tbw process terminated, errcode=143
2014-05-19 12:48:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:21 [DEBUG   ] __waitForTBW - lwa24 tbw process terminated, errcode=143
2014-05-19 12:48:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:21 [DEBUG   ] __waitForTBW - lwa23 tbw process terminated, errcode=143
2014-05-19 12:48:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:21 [DEBUG   ] __waitForTBW - lwa18 tbw process terminated, errcode=143
2014-05-19 12:48:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:22 [DEBUG   ] __waitForTBW - lwa10 tbw process terminated, errcode=143
2014-05-19 12:48:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:22 [DEBUG   ] __waitForTBW - lwa32 tbw process terminated, errcode=143
2014-05-19 12:48:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:22 [DEBUG   ] __waitForTBW - lwa11 tbw process terminated, errcode=143
2014-05-19 12:48:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:22 [DEBUG   ] __waitForTBW - lwa30 tbw process terminated, errcode=143
2014-05-19 12:48:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:23 [DEBUG   ] __waitForTBW - lwa12 tbw process terminated, errcode=143
2014-05-19 12:48:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:23 [DEBUG   ] __waitForTBW - lwa8 tbw process terminated, errcode=143
2014-05-19 12:48:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:23 [DEBUG   ] __waitForTBW - lwa13 tbw process terminated, errcode=143
2014-05-19 12:48:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:24 [DEBUG   ] __waitForTBW - lwa26 tbw process terminated, errcode=143
2014-05-19 12:48:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:24 [DEBUG   ] __waitForTBW - lwa14 tbw process terminated, errcode=143
2014-05-19 12:48:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:24 [DEBUG   ] __waitForTBW - lwa22 tbw process terminated, errcode=143
2014-05-19 12:48:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:25 [DEBUG   ] __waitForTBW - lwa20 tbw process terminated, errcode=143
2014-05-19 12:48:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:25 [DEBUG   ] __waitForTBW - lwa1 tbw process terminated, errcode=143
2014-05-19 12:48:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:25 [DEBUG   ] __waitForTBW - lwa5 tbw process terminated, errcode=143
2014-05-19 12:48:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:25 [DEBUG   ] __waitForTBW - lwa25 tbw process terminated, errcode=143
2014-05-19 12:48:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:26 [DEBUG   ] __waitForTBW - lwa35 tbw process terminated, errcode=143
2014-05-19 12:48:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:26 [DEBUG   ] __waitForTBW - lwa19 tbw process terminated, errcode=143
2014-05-19 12:48:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:26 [DEBUG   ] __waitForTBW - lwa21 tbw process terminated, errcode=143
2014-05-19 12:48:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:26 [DEBUG   ] __waitForTBW - lwa6 tbw process terminated, errcode=143
2014-05-19 12:48:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:27 [DEBUG   ] __waitForTBW - lwa7 tbw process terminated, errcode=143
2014-05-19 12:48:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:27 [DEBUG   ] __waitForTBW - lwa33 tbw process terminated, errcode=143
2014-05-19 12:48:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:27 [DEBUG   ] __waitForTBW - lwa15 tbw process terminated, errcode=143
2014-05-19 12:48:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2014-05-19 12:48:27 [DEBUG   ] __waitForTBW - lwa28 tbw process terminated, errcode=143
2014-05-19 12:48:27 [INFO    ] __waitForTBW - TBW finished in 191.516 s
2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:27 [WARNING ] AntennaStats: command returned 1; error: FPGAs not programmed

2014-05-19 12:48:28 [WARNING ] AntennaStats: no access to lwa28
2014-05-19 12:48:28 [WARNING ] AntennaStats: last command was /usr/local/bin/lwa28 /lwa/software/amplitude
2014-05-19 12:48:28 [WARNING ] AntennaStats: last command output as 
2014-05-19 12:48:28 [DEBUG   ] Finished updating ant stats in 0.126 seconds
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29709
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29710
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29711
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29712
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29713
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29714
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29715
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29716
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29717
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29718
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29719
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29720
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29721
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29722
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29723
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29724
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29725
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29726
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29727
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29728
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29729
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29730
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29731
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29732
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29733
2014-05-19 12:48:28 [DEBUG   ] 2014-05-19 12:48:28 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29734
2014-05-19 12:48:28 [INFO    ] Started antennas statistics background thread
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29709 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29710 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29711 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29712 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29713 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29714 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29715 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29716 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29717 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29718 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29719 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29720 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29721 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29722 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29723 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29724 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29725 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29726 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29727 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29728 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29729 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29730 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29731 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29732 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29733 exited with status 0
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29734 exited with status 0
2014-05-19 12:48:40 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] spawn_process: at 2014-05-19 12:48:40 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 29761
2014-05-19 12:48:40 [DEBUG   ] 2014-05-19 12:48:40 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 29762
2014-05-19 12:48:46 [DEBUG   ] 2014-05-19 12:48:46 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 29761 exited with status 0
2014-05-19 12:48:46 [DEBUG   ] 2014-05-19 12:48:46 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 29762 exited with status 0
2014-05-19 12:48:46 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2014-05-19 12:48:46 [DEBUG   ] spawn_process: at 2014-05-19 12:48:46 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2014-05-19 12:48:46 [DEBUG   ] spawn_process: at 2014-05-19 12:48:46 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2014-05-19 12:48:46 [INFO    ] Setting Beam 1 xbars
2014-05-19 12:48:46 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2014-05-19 12:48:46 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2014-05-19 12:48:46 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2014-05-19 12:48:46 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2014-05-19 12:48:46 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2014-05-19 12:48:46 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2014-05-19 12:48:46 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2014-05-19 12:48:46 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2014-05-19 12:48:46 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2014-05-19 12:48:47 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2014-05-19 12:48:47 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2014-05-19 12:48:47 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2014-05-19 12:48:47 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2014-05-19 12:48:47 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2014-05-19 12:48:47 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2014-05-19 12:48:47 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2014-05-19 12:48:47 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2014-05-19 12:48:47 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2014-05-19 12:48:48 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2014-05-19 12:48:48 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2014-05-19 12:48:48 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2014-05-19 12:48:48 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2014-05-19 12:48:48 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2014-05-19 12:48:48 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2014-05-19 12:48:48 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2014-05-19 12:48:48 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2014-05-19 12:48:48 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2014-05-19 12:48:49 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2014-05-19 12:48:49 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2014-05-19 12:48:49 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2014-05-19 12:48:49 [DEBUG   ] Finished updating ant stats in 1.362 seconds
2014-05-19 12:48:49 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2014-05-19 12:48:49 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2014-05-19 12:48:49 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2014-05-19 12:48:49 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2014-05-19 12:48:49 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2014-05-19 12:48:50 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2014-05-19 12:48:50 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2014-05-19 12:48:50 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2014-05-19 12:48:50 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2014-05-19 12:48:50 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2014-05-19 12:48:50 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2014-05-19 12:48:50 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2014-05-19 12:48:50 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2014-05-19 12:48:50 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2014-05-19 12:48:51 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2014-05-19 12:48:51 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2014-05-19 12:48:51 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2014-05-19 12:48:51 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2014-05-19 12:48:51 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2014-05-19 12:48:51 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2014-05-19 12:48:51 [INFO    ] Setting Beam 2 xbars
2014-05-19 12:48:51 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2014-05-19 12:48:51 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2014-05-19 12:48:51 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2014-05-19 12:48:51 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2014-05-19 12:48:52 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2014-05-19 12:48:52 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2014-05-19 12:48:52 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2014-05-19 12:48:52 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2014-05-19 12:48:52 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2014-05-19 12:48:52 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2014-05-19 12:48:52 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2014-05-19 12:48:52 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2014-05-19 12:48:52 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2014-05-19 12:48:53 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2014-05-19 12:48:53 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2014-05-19 12:48:53 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2014-05-19 12:48:53 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2014-05-19 12:48:53 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2014-05-19 12:48:53 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2014-05-19 12:48:53 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2014-05-19 12:48:53 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2014-05-19 12:48:53 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2014-05-19 12:48:54 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2014-05-19 12:48:54 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2014-05-19 12:48:54 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2014-05-19 12:48:54 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2014-05-19 12:48:54 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2014-05-19 12:48:54 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2014-05-19 12:48:54 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2014-05-19 12:48:54 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2014-05-19 12:48:54 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2014-05-19 12:48:55 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2014-05-19 12:48:55 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2014-05-19 12:48:55 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2014-05-19 12:48:55 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2014-05-19 12:48:55 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2014-05-19 12:48:55 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2014-05-19 12:48:55 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2014-05-19 12:48:55 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2014-05-19 12:48:55 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2014-05-19 12:48:55 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2014-05-19 12:48:56 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2014-05-19 12:48:56 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2014-05-19 12:48:56 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2014-05-19 12:48:56 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2014-05-19 12:48:56 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2014-05-19 12:48:56 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2014-05-19 12:48:56 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2014-05-19 12:48:56 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2014-05-19 12:48:56 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2014-05-19 12:48:57 [INFO    ] Setting Beam 3 xbars
2014-05-19 12:48:57 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2014-05-19 12:48:57 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2014-05-19 12:48:57 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2014-05-19 12:48:57 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2014-05-19 12:48:57 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2014-05-19 12:48:57 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2014-05-19 12:48:57 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2014-05-19 12:48:57 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2014-05-19 12:48:57 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2014-05-19 12:48:58 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2014-05-19 12:48:58 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2014-05-19 12:48:58 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2014-05-19 12:48:58 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2014-05-19 12:48:58 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2014-05-19 12:48:58 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2014-05-19 12:48:58 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2014-05-19 12:48:58 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2014-05-19 12:48:58 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2014-05-19 12:48:58 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2014-05-19 12:48:59 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2014-05-19 12:48:59 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2014-05-19 12:48:59 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2014-05-19 12:48:59 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2014-05-19 12:48:59 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2014-05-19 12:48:59 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2014-05-19 12:48:59 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2014-05-19 12:48:59 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2014-05-19 12:48:59 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2014-05-19 12:49:00 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2014-05-19 12:49:00 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2014-05-19 12:49:00 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2014-05-19 12:49:00 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2014-05-19 12:49:00 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2014-05-19 12:49:00 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2014-05-19 12:49:00 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2014-05-19 12:49:00 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2014-05-19 12:49:00 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2014-05-19 12:49:01 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2014-05-19 12:49:01 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2014-05-19 12:49:01 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2014-05-19 12:49:01 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2014-05-19 12:49:01 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2014-05-19 12:49:01 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2014-05-19 12:49:01 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2014-05-19 12:49:01 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2014-05-19 12:49:01 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2014-05-19 12:49:02 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2014-05-19 12:49:02 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2014-05-19 12:49:02 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2014-05-19 12:49:02 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2014-05-19 12:49:02 [INFO    ] Setting Beam 4 xbars
2014-05-19 12:49:02 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2014-05-19 12:49:02 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 12:49:02 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 12:49:02 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 12:49:02 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 12:49:03 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2014-05-19 12:49:03 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2014-05-19 12:49:03 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2014-05-19 12:49:03 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2014-05-19 12:49:03 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2014-05-19 12:49:03 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2014-05-19 12:49:03 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2014-05-19 12:49:03 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2014-05-19 12:49:03 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2014-05-19 12:49:04 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2014-05-19 12:49:04 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2014-05-19 12:49:04 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2014-05-19 12:49:04 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2014-05-19 12:49:04 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2014-05-19 12:49:04 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2014-05-19 12:49:04 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2014-05-19 12:49:04 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2014-05-19 12:49:04 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2014-05-19 12:49:05 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2014-05-19 12:49:05 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2014-05-19 12:49:05 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2014-05-19 12:49:05 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2014-05-19 12:49:05 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2014-05-19 12:49:05 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2014-05-19 12:49:05 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2014-05-19 12:49:05 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2014-05-19 12:49:05 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2014-05-19 12:49:06 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2014-05-19 12:49:06 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2014-05-19 12:49:06 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2014-05-19 12:49:06 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2014-05-19 12:49:06 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2014-05-19 12:49:06 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2014-05-19 12:49:06 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2014-05-19 12:49:06 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2014-05-19 12:49:06 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2014-05-19 12:49:06 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2014-05-19 12:49:07 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2014-05-19 12:49:07 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2014-05-19 12:49:07 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2014-05-19 12:49:07 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2014-05-19 12:49:07 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2014-05-19 12:49:07 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2014-05-19 12:49:07 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2014-05-19 12:49:07 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2014-05-19 12:49:07 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2014-05-19 12:49:08 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2014-05-19 12:49:08 [INFO    ] All xbars set
2014-05-19 12:49:08 [INFO    ] Setting adder chain routing for all beams
2014-05-19 12:49:09 [DEBUG   ] Finished updating ant stats in 1.096 seconds
2014-05-19 12:49:13 [INFO    ] Setting beam ids
2014-05-19 12:49:23 [INFO    ] Starting beam delay calibration
2014-05-19 12:49:29 [DEBUG   ] Finished updating ant stats in 0.956 seconds
2014-05-19 12:49:35 [INFO    ] Beam 1 latency is 5711
2014-05-19 12:49:35 [INFO    ] Beam 2 latency is 5762
2014-05-19 12:49:35 [INFO    ] Beam 3 latency is 5743
2014-05-19 12:49:35 [INFO    ] Beam 4 latency is 6676
2014-05-19 12:49:35 [INFO    ] Stop calibration
2014-05-19 12:49:43 [ERROR   ] Beam delay calibration failed
2014-05-19 12:49:43 [CRITICAL] Calibration failed
2014-05-19 12:49:43 [DEBUG   ] spawn_process: at 2014-05-19 12:49:43 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2014-05-19 12:49:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2014-05-19 12:49:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2014-05-19 12:49:43 [DEBUG   ] spawn_process: at 2014-05-19 12:49:43 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2014-05-19 12:49:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2014-05-19 12:49:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2014-05-19 12:49:43 [DEBUG   ] spawn_process: at 2014-05-19 12:49:43 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2014-05-19 12:49:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2014-05-19 12:49:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2014-05-19 12:49:43 [DEBUG   ] spawn_process: at 2014-05-19 12:49:43 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2014-05-19 12:49:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2014-05-19 12:49:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2014-05-19 12:49:43 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -5 msec
2014-05-19 12:49:44 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 268 ms (1400503784.273 s) with PID 31444 and time 1400503785
2014-05-19 12:49:44 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -5 msec
2014-05-19 12:49:44 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 359 ms (1400503784.364 s) with PID 31447 and time 1400503785
2014-05-19 12:49:46 [DEBUG   ] Finished updating ant stats in 0.784 seconds
2014-05-19 12:49:46 [INFO    ] Started antennas statistics background thread
2014-05-19 12:49:46 [INFO    ] Started board flags background thread
2014-05-19 12:49:46 [INFO    ] Started BAM background thread
2014-05-19 12:49:46 [INFO    ] Finished the INI process (ref. #193177) in 105.587 s
2014-05-19 12:49:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2014-05-19 12:49:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 12:49:47 [DEBUG   ] Finished updating board flags in 1.420 seconds
2014-05-19 12:50:06 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 12:50:26 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 12:50:46 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2014-05-19 12:50:47 [DEBUG   ] Finished updating board flags in 1.415 seconds
2014-05-19 12:51:06 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2014-05-19 12:51:27 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 12:51:47 [DEBUG   ] Finished updating ant stats in 1.029 seconds
2014-05-19 12:51:48 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 12:51:48 [DEBUG   ] Finished updating board flags in 1.941 seconds
2014-05-19 12:52:06 [DEBUG   ] Got command RPT from MCS: ref #193182
2014-05-19 12:52:06 [DEBUG   ] summary = ERROR
2014-05-19 12:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193182  13 56796 46326718 A  ERRORERROR'
2014-05-19 12:52:06 [DEBUG   ] Got command RPT from MCS: ref #193183
2014-05-19 12:52:06 [DEBUG   ] info = T_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x06! Beamformer calibration failed
2014-05-19 12:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193183  72 56796 46326719 A  ERRORT_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x06! Beamformer calibration failed'
2014-05-19 12:52:07 [DEBUG   ] Finished updating ant stats in 0.972 seconds
2014-05-19 12:52:27 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 12:52:47 [DEBUG   ] Finished updating ant stats in 0.830 seconds
2014-05-19 12:52:48 [DEBUG   ] Finished updating board flags in 1.790 seconds
2014-05-19 12:53:01 [DEBUG   ] Got command RPT from MCS: ref #193199
2014-05-19 12:53:01 [DEBUG   ] summary = ERROR
2014-05-19 12:53:01 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193199  13 56796 46381002 A  ERRORERROR'
2014-05-19 12:53:06 [DEBUG   ] Got command INI from MCS: ref #193200
2014-05-19 12:53:06 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2014-05-19 12:53:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   193200   8 56796 46386011 ABOOTING'
2014-05-19 12:53:07 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 12:53:26 [INFO    ] -------------------
2014-05-19 12:53:26 [INFO    ]  LWA board status
2014-05-19 12:53:26 [INFO    ] -------------------
2014-05-19 12:53:26 [INFO    ] DP 1: 2 lwa27
2014-05-19 12:53:26 [INFO    ] DP 1: 3 lwa24
2014-05-19 12:53:26 [INFO    ] DP 1: 4 lwa23
2014-05-19 12:53:26 [INFO    ] DP 1: 5 lwa18
2014-05-19 12:53:26 [INFO    ] DP 1: 6 lwa10
2014-05-19 12:53:26 [INFO    ] DP 1: 7 lwa32
2014-05-19 12:53:26 [INFO    ] DP 1: 8 lwa11
2014-05-19 12:53:26 [INFO    ] DP 1: 9 lwa30
2014-05-19 12:53:26 [INFO    ] DP 1: 10 lwa12
2014-05-19 12:53:26 [INFO    ] DP 1: 11 lwa8
2014-05-19 12:53:26 [INFO    ] DP 1: 12 lwa13
2014-05-19 12:53:26 [INFO    ] DP 1: 13 lwa26
2014-05-19 12:53:26 [INFO    ] DP 1: 14 lwa14
2014-05-19 12:53:26 [INFO    ] DP 1: 16 lwa22
2014-05-19 12:53:26 [INFO    ] DP 1: 17 lwa20
2014-05-19 12:53:26 [INFO    ] DP 1: 18 lwa1
2014-05-19 12:53:26 [INFO    ] DP 1: 19 lwa5
2014-05-19 12:53:26 [INFO    ] DP 1: 20 lwa25
2014-05-19 12:53:26 [INFO    ] DP 1: 21 lwa35
2014-05-19 12:53:26 [INFO    ] DP 1: 22 lwa19
2014-05-19 12:53:26 [INFO    ] DP 1: 23 lwa21
2014-05-19 12:53:26 [INFO    ] DP 1: 24 lwa6
2014-05-19 12:53:26 [INFO    ] DP 1: 25 lwa7
2014-05-19 12:53:26 [INFO    ] DP 1: 26 lwa33
2014-05-19 12:53:26 [INFO    ] DP 1: 27 lwa15
2014-05-19 12:53:26 [INFO    ] DP 1: 28 lwa28
2014-05-19 12:53:26 [INFO    ] DP 2: 1 lwa16
2014-05-19 12:53:26 [INFO    ] DP 2: 15 lwa34
2014-05-19 12:53:26 [INFO    ] Total of 28 boards
2014-05-19 12:53:26 [INFO    ] Total of 260 stands
2014-05-19 12:53:26 [INFO    ] -------------------
2014-05-19 12:53:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32256
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32257
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32258
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32259
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32260
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32261
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32262
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32263
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32264
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32265
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32266
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32267
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32268
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32269
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32270
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32271
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32272
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32273
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32274
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32275
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32276
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32277
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32278
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32279
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32280
2014-05-19 12:53:33 [DEBUG   ] 2014-05-19 12:53:33 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 32281
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32256 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32257 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32258 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32259 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32260 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32261 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32262 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32263 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32264 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32265 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32266 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32267 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32268 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32269 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32270 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32271 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32272 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32273 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32274 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32275 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32276 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32277 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32278 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32279 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32280 exited with status 0
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 32281 exited with status 0
2014-05-19 12:53:45 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] spawn_process: at 2014-05-19 12:53:45 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 32308
2014-05-19 12:53:45 [DEBUG   ] 2014-05-19 12:53:45 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 32309
2014-05-19 12:53:51 [DEBUG   ] 2014-05-19 12:53:51 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 32308 exited with status 0
2014-05-19 12:53:51 [DEBUG   ] 2014-05-19 12:53:51 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 32309 exited with status 0
2014-05-19 12:53:51 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2014-05-19 12:53:51 [DEBUG   ] spawn_process: at 2014-05-19 12:53:51 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2014-05-19 12:53:51 [DEBUG   ] spawn_process: at 2014-05-19 12:53:51 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2014-05-19 12:53:51 [INFO    ] Setting Beam 1 xbars
2014-05-19 12:53:51 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2014-05-19 12:53:51 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2014-05-19 12:53:51 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2014-05-19 12:53:51 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2014-05-19 12:53:52 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2014-05-19 12:53:52 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2014-05-19 12:53:52 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2014-05-19 12:53:52 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2014-05-19 12:53:52 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2014-05-19 12:53:52 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2014-05-19 12:53:52 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2014-05-19 12:53:52 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2014-05-19 12:53:52 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2014-05-19 12:53:52 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2014-05-19 12:53:53 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2014-05-19 12:53:53 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2014-05-19 12:53:53 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2014-05-19 12:53:53 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2014-05-19 12:53:53 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2014-05-19 12:53:53 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2014-05-19 12:53:53 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2014-05-19 12:53:53 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2014-05-19 12:53:53 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2014-05-19 12:53:54 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2014-05-19 12:53:54 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2014-05-19 12:53:54 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2014-05-19 12:53:54 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2014-05-19 12:53:54 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2014-05-19 12:53:54 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2014-05-19 12:53:54 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2014-05-19 12:53:54 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2014-05-19 12:53:54 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2014-05-19 12:53:55 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2014-05-19 12:53:55 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2014-05-19 12:53:55 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2014-05-19 12:53:55 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2014-05-19 12:53:55 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2014-05-19 12:53:55 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2014-05-19 12:53:55 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2014-05-19 12:53:55 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2014-05-19 12:53:55 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2014-05-19 12:53:56 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2014-05-19 12:53:56 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2014-05-19 12:53:56 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2014-05-19 12:53:56 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2014-05-19 12:53:56 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2014-05-19 12:53:56 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2014-05-19 12:53:56 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2014-05-19 12:53:56 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2014-05-19 12:53:56 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2014-05-19 12:53:56 [INFO    ] Setting Beam 2 xbars
2014-05-19 12:53:56 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2014-05-19 12:53:57 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2014-05-19 12:53:57 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2014-05-19 12:53:57 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2014-05-19 12:53:57 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2014-05-19 12:53:57 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2014-05-19 12:53:57 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2014-05-19 12:53:57 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2014-05-19 12:53:57 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2014-05-19 12:53:57 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2014-05-19 12:53:58 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2014-05-19 12:53:58 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2014-05-19 12:53:58 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2014-05-19 12:53:58 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2014-05-19 12:53:58 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2014-05-19 12:53:58 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2014-05-19 12:53:58 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2014-05-19 12:53:58 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2014-05-19 12:53:58 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2014-05-19 12:53:59 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2014-05-19 12:53:59 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2014-05-19 12:53:59 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2014-05-19 12:53:59 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2014-05-19 12:53:59 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2014-05-19 12:53:59 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2014-05-19 12:53:59 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2014-05-19 12:53:59 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2014-05-19 12:53:59 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2014-05-19 12:54:00 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2014-05-19 12:54:00 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2014-05-19 12:54:00 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2014-05-19 12:54:00 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2014-05-19 12:54:00 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2014-05-19 12:54:00 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2014-05-19 12:54:00 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2014-05-19 12:54:00 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2014-05-19 12:54:00 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2014-05-19 12:54:00 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2014-05-19 12:54:01 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2014-05-19 12:54:01 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2014-05-19 12:54:01 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2014-05-19 12:54:01 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2014-05-19 12:54:01 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2014-05-19 12:54:01 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2014-05-19 12:54:01 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2014-05-19 12:54:01 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2014-05-19 12:54:01 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2014-05-19 12:54:02 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2014-05-19 12:54:02 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2014-05-19 12:54:02 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2014-05-19 12:54:02 [INFO    ] Setting Beam 3 xbars
2014-05-19 12:54:02 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2014-05-19 12:54:02 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2014-05-19 12:54:02 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2014-05-19 12:54:02 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2014-05-19 12:54:02 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2014-05-19 12:54:02 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2014-05-19 12:54:03 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2014-05-19 12:54:03 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2014-05-19 12:54:03 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2014-05-19 12:54:03 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2014-05-19 12:54:03 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2014-05-19 12:54:03 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2014-05-19 12:54:03 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2014-05-19 12:54:03 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2014-05-19 12:54:03 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2014-05-19 12:54:04 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2014-05-19 12:54:04 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2014-05-19 12:54:04 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2014-05-19 12:54:04 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2014-05-19 12:54:04 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2014-05-19 12:54:04 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2014-05-19 12:54:04 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2014-05-19 12:54:04 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2014-05-19 12:54:04 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2014-05-19 12:54:05 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2014-05-19 12:54:05 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2014-05-19 12:54:05 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2014-05-19 12:54:05 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2014-05-19 12:54:05 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2014-05-19 12:54:05 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2014-05-19 12:54:05 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2014-05-19 12:54:05 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2014-05-19 12:54:05 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2014-05-19 12:54:05 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2014-05-19 12:54:06 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2014-05-19 12:54:06 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2014-05-19 12:54:06 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2014-05-19 12:54:06 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2014-05-19 12:54:06 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2014-05-19 12:54:06 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2014-05-19 12:54:06 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2014-05-19 12:54:06 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2014-05-19 12:54:06 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2014-05-19 12:54:07 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2014-05-19 12:54:07 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2014-05-19 12:54:07 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2014-05-19 12:54:07 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2014-05-19 12:54:07 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2014-05-19 12:54:07 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2014-05-19 12:54:07 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2014-05-19 12:54:07 [INFO    ] Setting Beam 4 xbars
2014-05-19 12:54:07 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2014-05-19 12:54:07 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 12:54:08 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 12:54:08 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 12:54:08 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 12:54:08 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2014-05-19 12:54:08 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2014-05-19 12:54:08 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2014-05-19 12:54:08 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2014-05-19 12:54:08 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2014-05-19 12:54:08 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2014-05-19 12:54:09 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2014-05-19 12:54:09 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2014-05-19 12:54:09 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2014-05-19 12:54:09 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2014-05-19 12:54:09 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2014-05-19 12:54:09 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2014-05-19 12:54:09 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2014-05-19 12:54:09 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2014-05-19 12:54:09 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2014-05-19 12:54:10 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2014-05-19 12:54:10 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2014-05-19 12:54:10 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2014-05-19 12:54:10 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2014-05-19 12:54:10 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2014-05-19 12:54:10 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2014-05-19 12:54:10 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2014-05-19 12:54:10 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2014-05-19 12:54:10 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2014-05-19 12:54:11 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2014-05-19 12:54:11 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2014-05-19 12:54:11 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2014-05-19 12:54:11 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2014-05-19 12:54:11 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2014-05-19 12:54:11 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2014-05-19 12:54:11 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2014-05-19 12:54:11 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2014-05-19 12:54:11 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2014-05-19 12:54:12 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2014-05-19 12:54:12 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2014-05-19 12:54:12 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2014-05-19 12:54:12 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2014-05-19 12:54:12 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2014-05-19 12:54:12 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2014-05-19 12:54:12 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2014-05-19 12:54:12 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2014-05-19 12:54:12 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2014-05-19 12:54:12 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2014-05-19 12:54:13 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2014-05-19 12:54:13 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2014-05-19 12:54:13 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2014-05-19 12:54:13 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2014-05-19 12:54:13 [INFO    ] All xbars set
2014-05-19 12:54:13 [INFO    ] Setting adder chain routing for all beams
2014-05-19 12:54:18 [INFO    ] Setting beam ids
2014-05-19 12:54:28 [INFO    ] Starting beam delay calibration
2014-05-19 12:54:40 [INFO    ] Beam 1 latency is 5743
2014-05-19 12:54:40 [INFO    ] Beam 2 latency is 5763
2014-05-19 12:54:40 [INFO    ] Beam 3 latency is 5747
2014-05-19 12:54:40 [INFO    ] Beam 4 latency is 5733
2014-05-19 12:54:40 [INFO    ] Stop calibration
2014-05-19 12:54:48 [INFO    ] Beam delay calibration successful
2014-05-19 12:54:48 [INFO    ] Calibration passed
2014-05-19 12:54:48 [DEBUG   ] spawn_process: at 2014-05-19 12:54:48 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2014-05-19 12:54:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2014-05-19 12:54:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2014-05-19 12:54:48 [DEBUG   ] spawn_process: at 2014-05-19 12:54:48 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2014-05-19 12:54:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2014-05-19 12:54:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2014-05-19 12:54:48 [DEBUG   ] spawn_process: at 2014-05-19 12:54:48 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2014-05-19 12:54:48 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2014-05-19 12:54:49 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2014-05-19 12:54:49 [DEBUG   ] spawn_process: at 2014-05-19 12:54:49 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2014-05-19 12:54:49 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2014-05-19 12:54:49 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2014-05-19 12:54:49 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -5 msec
2014-05-19 12:54:49 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 288 ms (1400504089.293 s) with PID 1420 and time 1400504090
2014-05-19 12:54:49 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -5 msec
2014-05-19 12:54:49 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 390 ms (1400504089.395 s) with PID 1423 and time 1400504090
2014-05-19 12:54:50 [DEBUG   ] Finished updating ant stats in 0.856 seconds
2014-05-19 12:54:50 [INFO    ] Started antennas statistics background thread
2014-05-19 12:54:50 [INFO    ] Started board flags background thread
2014-05-19 12:54:50 [INFO    ] Started BAM background thread
2014-05-19 12:54:50 [INFO    ] Finished the INI process (ref. #193200) in 104.511 s
2014-05-19 12:54:51 [DEBUG   ] Finished updating board flags in 1.050 seconds
2014-05-19 12:55:10 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2014-05-19 12:55:30 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 12:55:50 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 12:55:51 [DEBUG   ] Finished updating board flags in 0.996 seconds
2014-05-19 12:56:10 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 12:56:30 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 12:56:50 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 12:56:51 [DEBUG   ] Finished updating board flags in 1.289 seconds
2014-05-19 12:57:06 [DEBUG   ] Got command RPT from MCS: ref #193201
2014-05-19 12:57:06 [DEBUG   ] summary = NORMAL
2014-05-19 12:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193201  14 56796 46626718 A NORMALNORMAL'
2014-05-19 12:57:06 [DEBUG   ] Got command RPT from MCS: ref #193202
2014-05-19 12:57:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 12:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193202  48 56796 46626719 A NORMALSystem calibrated and operating normally'
2014-05-19 12:57:10 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2014-05-19 12:57:30 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 12:57:50 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 12:57:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 12:57:51 [DEBUG   ] Finished updating board flags in 1.163 seconds
2014-05-19 12:58:06 [DEBUG   ] Got command RPT from MCS: ref #193222
2014-05-19 12:58:06 [DEBUG   ] summary = NORMAL
2014-05-19 12:58:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193222  14 56796 46686006 A NORMALNORMAL'
2014-05-19 12:58:10 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 12:58:30 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2014-05-19 12:58:50 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2014-05-19 12:58:52 [DEBUG   ] Finished updating board flags in 1.583 seconds
2014-05-19 12:59:10 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 12:59:30 [DEBUG   ] Finished updating ant stats in 0.396 seconds
2014-05-19 12:59:51 [DEBUG   ] Finished updating ant stats in 1.231 seconds
2014-05-19 12:59:52 [DEBUG   ] Finished updating board flags in 1.389 seconds
2014-05-19 13:00:11 [DEBUG   ] Finished updating ant stats in 1.172 seconds
2014-05-19 13:00:31 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 13:00:51 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 13:00:52 [DEBUG   ] Finished updating board flags in 1.182 seconds
2014-05-19 13:01:11 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 13:01:31 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 13:01:51 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:01:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:01:52 [DEBUG   ] Finished updating board flags in 1.011 seconds
2014-05-19 13:02:06 [DEBUG   ] Got command RPT from MCS: ref #193224
2014-05-19 13:02:06 [DEBUG   ] summary = NORMAL
2014-05-19 13:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193224  14 56796 46926714 A NORMALNORMAL'
2014-05-19 13:02:06 [DEBUG   ] Got command RPT from MCS: ref #193225
2014-05-19 13:02:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 13:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193225  48 56796 46926715 A NORMALSystem calibrated and operating normally'
2014-05-19 13:02:11 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 13:02:31 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 13:02:51 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:02:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:02:52 [DEBUG   ] Finished updating board flags in 1.011 seconds
2014-05-19 13:03:11 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:03:31 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:03:43 [DEBUG   ] Got command DRX from MCS: ref #193246
2014-05-19 13:03:43 [INFO    ] sending drx command to lwa34
2014-05-19 13:03:43 [DEBUG   ] run_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa34', '/lwa/software/drx 1 1 58000000.0 7 13 79']
2014-05-19 13:03:43 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-19 13:03:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   193246   8 56796 47023015 A NORMAL'
2014-05-19 13:03:43 [DEBUG   ] Got command DRX from MCS: ref #193247
2014-05-19 13:03:43 [INFO    ] sending drx command to lwa16
2014-05-19 13:03:43 [DEBUG   ] run_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 58000000.0 7 13 79']
2014-05-19 13:03:43 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-19 13:03:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   193247   8 56796 47023019 A NORMAL'
2014-05-19 13:03:43 [DEBUG   ] Got command DRX from MCS: ref #193248
2014-05-19 13:03:43 [INFO    ] sending drx command to lwa16
2014-05-19 13:03:43 [DEBUG   ] run_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 58000000.0 7 13 79']
2014-05-19 13:03:43 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-19 13:03:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   193248   8 56796 47023023 A NORMAL'
2014-05-19 13:03:43 [DEBUG   ] Got command DRX from MCS: ref #193245
2014-05-19 13:03:43 [INFO    ] sending drx command to lwa34
2014-05-19 13:03:43 [DEBUG   ] run_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa34', '/lwa/software/drx 5 1 58000000.0 7 13 79']
2014-05-19 13:03:43 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-19 13:03:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   193245   8 56796 47023027 A NORMAL'
2014-05-19 13:03:43 [DEBUG   ] Got command DRX from MCS: ref #193250
2014-05-19 13:03:43 [INFO    ] sending drx command to lwa34
2014-05-19 13:03:43 [DEBUG   ] run_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa34', '/lwa/software/drx 1 2 74000000.0 7 13 79']
2014-05-19 13:03:43 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-19 13:03:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   193250   8 56796 47023031 A NORMAL'
2014-05-19 13:03:43 [DEBUG   ] Got command DRX from MCS: ref #193251
2014-05-19 13:03:43 [INFO    ] sending drx command to lwa16
2014-05-19 13:03:43 [DEBUG   ] run_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 74000000.0 7 13 79']
2014-05-19 13:03:43 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-19 13:03:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   193251   8 56796 47023035 A NORMAL'
2014-05-19 13:03:43 [DEBUG   ] Got command DRX from MCS: ref #193252
2014-05-19 13:03:43 [INFO    ] sending drx command to lwa16
2014-05-19 13:03:43 [DEBUG   ] run_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 74000000.0 7 13 79']
2014-05-19 13:03:43 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-19 13:03:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   193252   8 56796 47023039 A NORMAL'
2014-05-19 13:03:43 [DEBUG   ] Got command DRX from MCS: ref #193249
2014-05-19 13:03:43 [INFO    ] sending drx command to lwa34
2014-05-19 13:03:43 [DEBUG   ] run_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa34', '/lwa/software/drx 5 2 74000000.0 7 13 79']
2014-05-19 13:03:43 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-19 13:03:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   193249   8 56796 47023043 A NORMAL'
2014-05-19 13:03:43 [INFO    ] Finished starting DRX
2014-05-19 13:03:43 [INFO    ] Finished starting DRX
2014-05-19 13:03:43 [INFO    ] Finished starting DRX
2014-05-19 13:03:43 [INFO    ] Finished starting DRX
2014-05-19 13:03:43 [INFO    ] Finished starting DRX
2014-05-19 13:03:43 [INFO    ] Finished starting DRX
2014-05-19 13:03:43 [INFO    ] Finished starting DRX
2014-05-19 13:03:43 [INFO    ] Finished starting DRX
2014-05-19 13:03:43 [DEBUG   ] Got command BAM from MCS: ref #193254
2014-05-19 13:03:43 [INFO    ] Finished processing BAM request
2014-05-19 13:03:43 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 13:03:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193254   8 56796 47023190 A NORMAL'
2014-05-19 13:03:43 [DEBUG   ] Got command BAM from MCS: ref #193255
2014-05-19 13:03:43 [INFO    ] Finished processing BAM request
2014-05-19 13:03:43 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 13:03:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193255   8 56796 47023350 A NORMAL'
2014-05-19 13:03:43 [DEBUG   ] Got command BAM from MCS: ref #193256
2014-05-19 13:03:43 [INFO    ] Finished processing BAM request
2014-05-19 13:03:43 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 13:03:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193256   8 56796 47023474 A NORMAL'
2014-05-19 13:03:43 [DEBUG   ] Got command BAM from MCS: ref #193253
2014-05-19 13:03:43 [INFO    ] Finished processing BAM request
2014-05-19 13:03:43 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 13:03:43 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193253   8 56796 47023598 A NORMAL'
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2014-05-19 13:03:43 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2014-05-19 13:03:43 [DEBUG   ] spawn_process: at 2014-05-19 13:03:43 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2014-05-19 13:03:51 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:03:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 13:03:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa24, chip 1
2014-05-19 13:03:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 1
2014-05-19 13:03:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:03:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-19 13:03:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:03:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 13:03:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:03:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2014-05-19 13:03:52 [DEBUG   ] Finished updating board flags in 1.041 seconds
2014-05-19 13:04:11 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:04:31 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:04:51 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:04:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:04:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 13:04:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 13:04:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:04:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:04:53 [DEBUG   ] Finished updating board flags in 2.008 seconds
2014-05-19 13:05:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:05:31 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:05:51 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:05:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:05:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 2
2014-05-19 13:05:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa12, chip 1
2014-05-19 13:05:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 5
2014-05-19 13:05:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 1
2014-05-19 13:05:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:05:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:05:53 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:05:53 [DEBUG   ] Finished updating board flags in 1.969 seconds
2014-05-19 13:06:11 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:06:31 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:06:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:06:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:06:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2014-05-19 13:06:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:06:52 [DEBUG   ] Finished updating board flags in 1.243 seconds
2014-05-19 13:07:06 [DEBUG   ] Got command RPT from MCS: ref #193261
2014-05-19 13:07:06 [DEBUG   ] summary = NORMAL
2014-05-19 13:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193261  14 56796 47226722 A NORMALNORMAL'
2014-05-19 13:07:06 [DEBUG   ] Got command RPT from MCS: ref #193262
2014-05-19 13:07:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 13:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193262  48 56796 47226723 A NORMALSystem calibrated and operating normally'
2014-05-19 13:07:11 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:07:31 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:07:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:07:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 1
2014-05-19 13:07:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2014-05-19 13:07:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:07:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:07:52 [DEBUG   ] Finished updating board flags in 1.216 seconds
2014-05-19 13:08:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:08:31 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:08:51 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:08:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 1
2014-05-19 13:08:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-19 13:08:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2014-05-19 13:08:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-19 13:08:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:08:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:08:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 13:08:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 1
2014-05-19 13:08:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa1, chip 1
2014-05-19 13:08:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:08:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:08:52 [DEBUG   ] Finished updating board flags in 1.164 seconds
2014-05-19 13:09:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:09:31 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:09:51 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:09:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 13:09:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:09:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:09:52 [DEBUG   ] Finished updating board flags in 1.081 seconds
2014-05-19 13:10:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:10:31 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:10:51 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 13:10:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-19 13:10:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:10:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 1
2014-05-19 13:10:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:10:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2014-05-19 13:10:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2014-05-19 13:10:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:10:53 [DEBUG   ] Finished updating board flags in 1.532 seconds
2014-05-19 13:11:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:11:31 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:11:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:11:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-19 13:11:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:11:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 13:11:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 1
2014-05-19 13:11:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa1, chip 1
2014-05-19 13:11:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2014-05-19 13:11:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:11:53 [DEBUG   ] Finished updating board flags in 1.429 seconds
2014-05-19 13:12:06 [DEBUG   ] Got command RPT from MCS: ref #193278
2014-05-19 13:12:06 [DEBUG   ] summary = NORMAL
2014-05-19 13:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193278  14 56796 47526732 A NORMALNORMAL'
2014-05-19 13:12:06 [DEBUG   ] Got command RPT from MCS: ref #193279
2014-05-19 13:12:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 13:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193279  48 56796 47526732 A NORMALSystem calibrated and operating normally'
2014-05-19 13:12:11 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2014-05-19 13:12:31 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 13:12:51 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 13:12:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:12:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 13:12:53 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:12:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:12:53 [DEBUG   ] Finished updating board flags in 1.316 seconds
2014-05-19 13:13:11 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 13:13:31 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:13:51 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 13:13:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 13:13:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2014-05-19 13:13:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:13:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2014-05-19 13:13:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:13:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:13:53 [DEBUG   ] Finished updating board flags in 1.228 seconds
2014-05-19 13:14:11 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 13:14:31 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:14:51 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:14:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa24, chip 1
2014-05-19 13:14:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-19 13:14:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:14:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:14:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 1
2014-05-19 13:14:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 13:14:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2014-05-19 13:14:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:14:53 [DEBUG   ] Finished updating board flags in 1.179 seconds
2014-05-19 13:15:11 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 13:15:31 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:15:51 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:15:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:15:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:15:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2014-05-19 13:15:53 [DEBUG   ] Finished updating board flags in 1.013 seconds
2014-05-19 13:16:11 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:16:31 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 13:16:51 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 13:16:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa12, chip 1
2014-05-19 13:16:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 5
2014-05-19 13:16:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa1, chip 1
2014-05-19 13:16:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:16:53 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:16:53 [DEBUG   ] Finished updating board flags in 1.073 seconds
2014-05-19 13:17:06 [DEBUG   ] Got command RPT from MCS: ref #193295
2014-05-19 13:17:06 [DEBUG   ] summary = NORMAL
2014-05-19 13:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193295  14 56796 47826722 A NORMALNORMAL'
2014-05-19 13:17:06 [DEBUG   ] Got command RPT from MCS: ref #193296
2014-05-19 13:17:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 13:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193296  48 56796 47826723 A NORMALSystem calibrated and operating normally'
2014-05-19 13:17:11 [DEBUG   ] Finished updating ant stats in 0.593 seconds
2014-05-19 13:17:31 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:17:51 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2014-05-19 13:17:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2014-05-19 13:17:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:17:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 1
2014-05-19 13:17:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2014-05-19 13:17:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:17:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:17:54 [DEBUG   ] Finished updating board flags in 2.009 seconds
2014-05-19 13:18:11 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:18:31 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2014-05-19 13:18:51 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:18:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-19 13:18:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 13:18:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:18:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:18:53 [DEBUG   ] Finished updating board flags in 1.357 seconds
2014-05-19 13:19:11 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2014-05-19 13:19:31 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:19:51 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:19:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 2
2014-05-19 13:19:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:19:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2014-05-19 13:19:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2014-05-19 13:19:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:19:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2014-05-19 13:19:53 [DEBUG   ] Finished updating board flags in 1.245 seconds
2014-05-19 13:20:11 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2014-05-19 13:20:31 [DEBUG   ] Finished updating ant stats in 0.397 seconds
2014-05-19 13:20:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:20:53 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 13:20:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 13:20:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2014-05-19 13:20:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 13:20:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:20:53 [DEBUG   ] Finished updating board flags in 1.151 seconds
2014-05-19 13:21:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:21:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:21:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:21:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-19 13:21:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:21:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 13:21:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:21:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 5
2014-05-19 13:21:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:21:53 [DEBUG   ] Finished updating board flags in 1.036 seconds
2014-05-19 13:22:06 [DEBUG   ] Got command RPT from MCS: ref #193312
2014-05-19 13:22:06 [DEBUG   ] summary = NORMAL
2014-05-19 13:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193312  14 56796 48126726 A NORMALNORMAL'
2014-05-19 13:22:06 [DEBUG   ] Got command RPT from MCS: ref #193313
2014-05-19 13:22:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 13:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193313  48 56796 48126727 A NORMALSystem calibrated and operating normally'
2014-05-19 13:22:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:22:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:22:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:22:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2014-05-19 13:22:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 1
2014-05-19 13:22:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 5
2014-05-19 13:22:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 13:22:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:22:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:22:53 [DEBUG   ] Finished updating board flags in 0.970 seconds
2014-05-19 13:23:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:23:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:23:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:23:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-19 13:23:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 13:23:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2014-05-19 13:23:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 13:23:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:23:54 [DEBUG   ] Finished updating board flags in 1.422 seconds
2014-05-19 13:24:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:24:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:24:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:24:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa24, chip 1
2014-05-19 13:24:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-19 13:24:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:24:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-19 13:24:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 13:24:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2014-05-19 13:24:54 [DEBUG   ] Finished updating board flags in 1.373 seconds
2014-05-19 13:25:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:25:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:25:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:25:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-19 13:25:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 13:25:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:25:54 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:25:54 [DEBUG   ] Finished updating board flags in 1.188 seconds
2014-05-19 13:26:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:26:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:26:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:26:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa24, chip 4
2014-05-19 13:26:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 5
2014-05-19 13:26:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:26:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2014-05-19 13:26:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:26:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:26:55 [DEBUG   ] Finished updating board flags in 2.000 seconds
2014-05-19 13:27:06 [DEBUG   ] Got command RPT from MCS: ref #193329
2014-05-19 13:27:06 [DEBUG   ] summary = NORMAL
2014-05-19 13:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193329  14 56796 48426726 A NORMALNORMAL'
2014-05-19 13:27:06 [DEBUG   ] Got command RPT from MCS: ref #193330
2014-05-19 13:27:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 13:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193330  48 56796 48426727 A NORMALSystem calibrated and operating normally'
2014-05-19 13:27:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:27:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:27:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:27:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 13:27:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:27:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-19 13:27:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 13:27:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:27:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:27:55 [DEBUG   ] Finished updating board flags in 1.792 seconds
2014-05-19 13:28:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:28:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:28:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:28:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:28:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 5
2014-05-19 13:28:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 13:28:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-19 13:28:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:28:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 13:28:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:28:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2014-05-19 13:28:54 [DEBUG   ] Finished updating board flags in 1.105 seconds
2014-05-19 13:29:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:29:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:29:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:29:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:29:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:29:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:29:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 5
2014-05-19 13:29:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 1
2014-05-19 13:29:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 13:29:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:29:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2014-05-19 13:29:54 [DEBUG   ] Finished updating board flags in 1.022 seconds
2014-05-19 13:30:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:30:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:30:52 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:30:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:30:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:30:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 5
2014-05-19 13:30:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2014-05-19 13:30:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:30:55 [DEBUG   ] Finished updating board flags in 1.558 seconds
2014-05-19 13:31:12 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2014-05-19 13:31:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:31:52 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 13:31:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:31:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:31:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2014-05-19 13:31:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:31:55 [DEBUG   ] Finished updating board flags in 1.324 seconds
2014-05-19 13:32:06 [DEBUG   ] Got command RPT from MCS: ref #193346
2014-05-19 13:32:06 [DEBUG   ] summary = NORMAL
2014-05-19 13:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193346  14 56796 48726726 A NORMALNORMAL'
2014-05-19 13:32:06 [DEBUG   ] Got command RPT from MCS: ref #193347
2014-05-19 13:32:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 13:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193347  48 56796 48726727 A NORMALSystem calibrated and operating normally'
2014-05-19 13:32:12 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:32:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:32:52 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:32:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 1
2014-05-19 13:32:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2014-05-19 13:32:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-19 13:32:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:32:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:32:55 [DEBUG   ] Finished updating board flags in 1.216 seconds
2014-05-19 13:33:12 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:33:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:33:52 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:33:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:33:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 13:33:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:33:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 13:33:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-19 13:33:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa1, chip 1
2014-05-19 13:33:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:33:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2014-05-19 13:33:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:33:55 [DEBUG   ] Finished updating board flags in 1.193 seconds
2014-05-19 13:34:12 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2014-05-19 13:34:32 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:34:52 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:34:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 1
2014-05-19 13:34:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2014-05-19 13:34:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa8, chip 4
2014-05-19 13:34:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 13:34:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 13:34:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:34:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2014-05-19 13:34:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2014-05-19 13:34:55 [DEBUG   ] Finished updating board flags in 1.012 seconds
2014-05-19 13:35:12 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2014-05-19 13:35:32 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:35:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:35:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 13:35:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:35:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:35:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 13:35:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:35:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-19 13:35:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:35:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2014-05-19 13:35:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:35:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2014-05-19 13:35:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:35:55 [DEBUG   ] Finished updating board flags in 1.014 seconds
2014-05-19 13:36:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:36:32 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:36:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:36:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 5
2014-05-19 13:36:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:36:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 13:36:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:36:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 3
2014-05-19 13:36:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:36:56 [DEBUG   ] Finished updating board flags in 1.997 seconds
2014-05-19 13:37:06 [DEBUG   ] Got command RPT from MCS: ref #193363
2014-05-19 13:37:06 [DEBUG   ] summary = NORMAL
2014-05-19 13:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193363  14 56796 49026722 A NORMALNORMAL'
2014-05-19 13:37:06 [DEBUG   ] Got command RPT from MCS: ref #193364
2014-05-19 13:37:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 13:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193364  48 56796 49026723 A NORMALSystem calibrated and operating normally'
2014-05-19 13:37:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:37:32 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:37:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:37:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 1
2014-05-19 13:37:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 5
2014-05-19 13:37:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-19 13:37:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:37:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:37:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:37:56 [DEBUG   ] Finished updating board flags in 1.823 seconds
2014-05-19 13:38:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:38:32 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:38:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:38:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-19 13:38:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 5
2014-05-19 13:38:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 13:38:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 13:38:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:38:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:38:55 [DEBUG   ] Finished updating board flags in 1.236 seconds
2014-05-19 13:39:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:39:32 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:39:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:39:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:39:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2014-05-19 13:39:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:39:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:39:55 [DEBUG   ] Finished updating board flags in 1.236 seconds
2014-05-19 13:40:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:40:32 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:40:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:40:55 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 13:40:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-19 13:40:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 1
2014-05-19 13:40:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 13:40:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2014-05-19 13:40:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2014-05-19 13:40:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:40:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:40:55 [DEBUG   ] Finished updating board flags in 1.150 seconds
2014-05-19 13:41:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:41:32 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:41:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:41:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 13:41:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2014-05-19 13:41:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 1
2014-05-19 13:41:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:41:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 13:41:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:41:55 [DEBUG   ] Finished updating board flags in 1.055 seconds
2014-05-19 13:42:06 [DEBUG   ] Got command RPT from MCS: ref #193380
2014-05-19 13:42:06 [DEBUG   ] summary = NORMAL
2014-05-19 13:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193380  14 56796 49326730 A NORMALNORMAL'
2014-05-19 13:42:06 [DEBUG   ] Got command RPT from MCS: ref #193381
2014-05-19 13:42:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 13:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193381  48 56796 49326731 A NORMALSystem calibrated and operating normally'
2014-05-19 13:42:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:42:32 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:42:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:42:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa24, chip 4
2014-05-19 13:42:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:42:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:42:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:42:56 [WARNING ] MonitorFlags: VCCINT 0.976 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:42:56 [DEBUG   ] Finished updating board flags in 1.537 seconds
2014-05-19 13:43:12 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2014-05-19 13:43:32 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:43:52 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:43:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-19 13:43:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:43:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 5
2014-05-19 13:43:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 1
2014-05-19 13:43:56 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:43:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:43:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2014-05-19 13:43:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:43:56 [DEBUG   ] Finished updating board flags in 1.358 seconds
2014-05-19 13:44:12 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:44:32 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:44:52 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:44:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-19 13:44:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:44:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:44:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:44:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2014-05-19 13:44:56 [DEBUG   ] Finished updating board flags in 1.234 seconds
2014-05-19 13:45:12 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:45:32 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:45:52 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:45:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 13:45:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:45:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:45:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:45:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 5
2014-05-19 13:45:56 [DEBUG   ] Finished updating board flags in 1.223 seconds
2014-05-19 13:46:12 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:46:32 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:46:52 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:46:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 5
2014-05-19 13:46:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 13:46:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-19 13:46:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:46:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 13:46:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa7, chip 2
2014-05-19 13:46:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:46:56 [DEBUG   ] Finished updating board flags in 1.158 seconds
2014-05-19 13:47:06 [DEBUG   ] Got command RPT from MCS: ref #193397
2014-05-19 13:47:06 [DEBUG   ] summary = NORMAL
2014-05-19 13:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193397  14 56796 49626730 A NORMALNORMAL'
2014-05-19 13:47:06 [DEBUG   ] Got command RPT from MCS: ref #193398
2014-05-19 13:47:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 13:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193398  48 56796 49626731 A NORMALSystem calibrated and operating normally'
2014-05-19 13:47:12 [DEBUG   ] Finished updating ant stats in 0.397 seconds
2014-05-19 13:47:33 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:47:53 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 13:47:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-19 13:47:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:47:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 13:47:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-19 13:47:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 13:47:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:47:56 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:47:56 [DEBUG   ] Finished updating board flags in 1.008 seconds
2014-05-19 13:48:13 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2014-05-19 13:48:33 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:48:53 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:48:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:48:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:48:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2014-05-19 13:48:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 13:48:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2014-05-19 13:48:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:48:56 [DEBUG   ] Finished updating board flags in 1.012 seconds
2014-05-19 13:49:13 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:49:33 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:49:53 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:49:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:49:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 1
2014-05-19 13:49:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:49:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:49:57 [DEBUG   ] Finished updating board flags in 2.028 seconds
2014-05-19 13:50:13 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:50:33 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:50:53 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 13:50:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:50:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 5
2014-05-19 13:50:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 13:50:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2014-05-19 13:50:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2014-05-19 13:50:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:50:56 [DEBUG   ] Finished updating board flags in 1.406 seconds
2014-05-19 13:51:13 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2014-05-19 13:51:33 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:51:53 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:51:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 1
2014-05-19 13:51:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:51:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2014-05-19 13:51:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:51:56 [DEBUG   ] Finished updating board flags in 1.434 seconds
2014-05-19 13:52:06 [DEBUG   ] Got command RPT from MCS: ref #193414
2014-05-19 13:52:06 [DEBUG   ] summary = NORMAL
2014-05-19 13:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193414  14 56796 49926730 A NORMALNORMAL'
2014-05-19 13:52:06 [DEBUG   ] Got command RPT from MCS: ref #193415
2014-05-19 13:52:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 13:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193415  48 56796 49926731 A NORMALSystem calibrated and operating normally'
2014-05-19 13:52:13 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:52:33 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:52:53 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:52:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 3
2014-05-19 13:52:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:52:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 1
2014-05-19 13:52:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa1, chip 2
2014-05-19 13:52:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 13:52:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:52:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:52:56 [DEBUG   ] Finished updating board flags in 1.356 seconds
2014-05-19 13:53:13 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:53:33 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:53:53 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:53:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 1
2014-05-19 13:53:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2014-05-19 13:53:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:53:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2014-05-19 13:53:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:53:57 [DEBUG   ] Finished updating board flags in 1.810 seconds
2014-05-19 13:54:13 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:54:33 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:54:53 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 13:54:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:54:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 13:54:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 13:54:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:54:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2014-05-19 13:54:57 [DEBUG   ] Finished updating board flags in 1.750 seconds
2014-05-19 13:55:13 [DEBUG   ] Finished updating ant stats in 0.793 seconds
2014-05-19 13:55:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 13:55:53 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:55:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa24, chip 4
2014-05-19 13:55:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:55:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-19 13:55:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 13:55:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2014-05-19 13:55:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:55:56 [DEBUG   ] Finished updating board flags in 1.053 seconds
2014-05-19 13:56:13 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:56:33 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:56:53 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 13:56:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2014-05-19 13:56:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2014-05-19 13:56:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:56:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 13:56:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 13:56:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:56:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2014-05-19 13:56:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:56:56 [DEBUG   ] Finished updating board flags in 1.057 seconds
2014-05-19 13:57:06 [DEBUG   ] Got command RPT from MCS: ref #193431
2014-05-19 13:57:06 [DEBUG   ] summary = NORMAL
2014-05-19 13:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193431  14 56796 50226734 A NORMALNORMAL'
2014-05-19 13:57:06 [DEBUG   ] Got command RPT from MCS: ref #193432
2014-05-19 13:57:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 13:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193432  48 56796 50226735 A NORMALSystem calibrated and operating normally'
2014-05-19 13:57:13 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2014-05-19 13:57:33 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:57:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:57:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2014-05-19 13:57:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-19 13:57:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:57:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 13:57:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2014-05-19 13:57:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:57:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 13:57:56 [DEBUG   ] Finished updating board flags in 0.939 seconds
2014-05-19 13:58:13 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:58:33 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:58:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:58:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:58:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 1
2014-05-19 13:58:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:58:57 [DEBUG   ] Finished updating board flags in 1.378 seconds
2014-05-19 13:59:13 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:59:33 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:59:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 13:59:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-19 13:59:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 13:59:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 13:59:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 5
2014-05-19 13:59:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 13:59:57 [DEBUG   ] Finished updating board flags in 1.214 seconds
2014-05-19 14:00:13 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:00:33 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:00:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:00:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-19 14:00:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 14:00:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 14:00:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 5
2014-05-19 14:00:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 1
2014-05-19 14:00:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2014-05-19 14:00:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:00:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2014-05-19 14:00:57 [DEBUG   ] Finished updating board flags in 1.176 seconds
2014-05-19 14:01:13 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:01:33 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:01:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:01:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 1
2014-05-19 14:01:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 14:01:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:01:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 5
2014-05-19 14:01:57 [DEBUG   ] Finished updating board flags in 1.014 seconds
2014-05-19 14:02:06 [DEBUG   ] Got command RPT from MCS: ref #193448
2014-05-19 14:02:06 [DEBUG   ] summary = NORMAL
2014-05-19 14:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193448  14 56796 50526727 A NORMALNORMAL'
2014-05-19 14:02:06 [DEBUG   ] Got command RPT from MCS: ref #193449
2014-05-19 14:02:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 14:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193449  48 56796 50526727 A NORMALSystem calibrated and operating normally'
2014-05-19 14:02:13 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:02:33 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:02:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:02:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-19 14:02:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa12, chip 1
2014-05-19 14:02:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 14:02:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 14:02:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:02:58 [DEBUG   ] Finished updating board flags in 2.051 seconds
2014-05-19 14:03:13 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:03:33 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:03:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:03:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 1
2014-05-19 14:03:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa18, chip 5
2014-05-19 14:03:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-19 14:03:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 5
2014-05-19 14:03:57 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 14:03:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:03:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 5
2014-05-19 14:03:57 [DEBUG   ] Finished updating board flags in 1.387 seconds
2014-05-19 14:04:13 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:04:33 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:04:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:04:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 5
2014-05-19 14:04:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 1
2014-05-19 14:04:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 14:04:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:04:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 5
2014-05-19 14:04:57 [DEBUG   ] Finished updating board flags in 1.242 seconds
2014-05-19 14:05:13 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2014-05-19 14:05:33 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:05:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:05:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 14:05:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 14:05:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 5
2014-05-19 14:05:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 14:05:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 14:05:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:05:57 [DEBUG   ] Finished updating board flags in 1.213 seconds
2014-05-19 14:06:13 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:06:33 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:06:53 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-19 14:06:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 14:06:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 14:06:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 14:06:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2014-05-19 14:06:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:06:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 5
2014-05-19 14:06:57 [DEBUG   ] Finished updating board flags in 1.153 seconds
2014-05-19 14:07:06 [DEBUG   ] Got command RPT from MCS: ref #193472
2014-05-19 14:07:06 [DEBUG   ] summary = NORMAL
2014-05-19 14:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193472  14 56796 50826730 A NORMALNORMAL'
2014-05-19 14:07:06 [DEBUG   ] Got command RPT from MCS: ref #193473
2014-05-19 14:07:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 14:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193473  48 56796 50826731 A NORMALSystem calibrated and operating normally'
2014-05-19 14:07:13 [DEBUG   ] Finished updating ant stats in 0.397 seconds
2014-05-19 14:07:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 14:07:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 14:07:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa12, chip 1
2014-05-19 14:07:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 14:07:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 5
2014-05-19 14:07:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 14:07:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 5
2014-05-19 14:07:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:07:57 [DEBUG   ] Finished updating board flags in 1.018 seconds
2014-05-19 14:08:14 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 14:08:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 14:08:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 14:08:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-19 14:08:58 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 14:08:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:08:58 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 14:08:58 [DEBUG   ] Finished updating board flags in 1.582 seconds
2014-05-19 14:09:14 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 14:09:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 14:09:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 14:09:57 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 14:09:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 14:09:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 5
2014-05-19 14:09:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa35, chip 1
2014-05-19 14:09:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:09:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 14:09:58 [DEBUG   ] Finished updating board flags in 1.362 seconds
2014-05-19 14:10:14 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2014-05-19 14:10:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:10:54 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:10:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 14:10:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 5
2014-05-19 14:10:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 5
2014-05-19 14:10:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 1
2014-05-19 14:10:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa25, chip 1
2014-05-19 14:10:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:10:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 1
2014-05-19 14:10:58 [DEBUG   ] Finished updating board flags in 1.218 seconds
2014-05-19 14:11:14 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:11:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:11:54 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:11:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 14:11:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 14:11:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:11:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2014-05-19 14:11:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2014-05-19 14:11:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 14:11:58 [DEBUG   ] Finished updating board flags in 1.173 seconds
2014-05-19 14:12:06 [DEBUG   ] Got command RPT from MCS: ref #193517
2014-05-19 14:12:06 [DEBUG   ] summary = NORMAL
2014-05-19 14:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193517  14 56796 51126738 A NORMALNORMAL'
2014-05-19 14:12:06 [DEBUG   ] Got command RPT from MCS: ref #193518
2014-05-19 14:12:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 14:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193518  48 56796 51126739 A NORMALSystem calibrated and operating normally'
2014-05-19 14:12:14 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:12:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:12:54 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:12:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 14:12:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2014-05-19 14:12:58 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 14:12:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 14:12:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:12:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 1
2014-05-19 14:12:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 14:12:58 [DEBUG   ] Finished updating board flags in 1.028 seconds
2014-05-19 14:13:14 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:13:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:13:54 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:13:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-19 14:13:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-19 14:13:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:13:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 5
2014-05-19 14:13:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2014-05-19 14:13:59 [WARNING ] MonitorFlags: VCCINT 0.976 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 14:13:59 [DEBUG   ] Finished updating board flags in 2.012 seconds
2014-05-19 14:14:14 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:14:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:14:54 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:14:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa13, chip 1
2014-05-19 14:14:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 14:14:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 1
2014-05-19 14:14:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 1
2014-05-19 14:14:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-19 14:14:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 14:14:58 [DEBUG   ] Finished updating board flags in 1.413 seconds
2014-05-19 14:15:00 [DEBUG   ] Got command INI from MCS: ref #193534
2014-05-19 14:15:00 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2014-05-19 14:15:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   193534   8 56796 51300111 ABOOTING'
2014-05-19 14:15:20 [INFO    ] -------------------
2014-05-19 14:15:20 [INFO    ]  LWA board status
2014-05-19 14:15:20 [INFO    ] -------------------
2014-05-19 14:15:20 [INFO    ] DP 1: 2 lwa27
2014-05-19 14:15:20 [INFO    ] DP 1: 3 lwa24
2014-05-19 14:15:20 [INFO    ] DP 1: 4 lwa23
2014-05-19 14:15:20 [INFO    ] DP 1: 5 lwa18
2014-05-19 14:15:20 [INFO    ] DP 1: 6 lwa10
2014-05-19 14:15:20 [INFO    ] DP 1: 7 lwa32
2014-05-19 14:15:20 [INFO    ] DP 1: 8 lwa11
2014-05-19 14:15:20 [INFO    ] DP 1: 9 lwa30
2014-05-19 14:15:20 [INFO    ] DP 1: 10 lwa12
2014-05-19 14:15:20 [INFO    ] DP 1: 11 lwa8
2014-05-19 14:15:20 [INFO    ] DP 1: 12 lwa13
2014-05-19 14:15:20 [INFO    ] DP 1: 13 lwa26
2014-05-19 14:15:20 [INFO    ] DP 1: 14 lwa14
2014-05-19 14:15:20 [INFO    ] DP 1: 16 lwa22
2014-05-19 14:15:20 [INFO    ] DP 1: 17 lwa20
2014-05-19 14:15:20 [INFO    ] DP 1: 18 lwa1
2014-05-19 14:15:20 [INFO    ] DP 1: 19 lwa5
2014-05-19 14:15:20 [INFO    ] DP 1: 20 lwa25
2014-05-19 14:15:20 [INFO    ] DP 1: 21 lwa35
2014-05-19 14:15:20 [INFO    ] DP 1: 22 lwa19
2014-05-19 14:15:20 [INFO    ] DP 1: 23 lwa21
2014-05-19 14:15:20 [INFO    ] DP 1: 24 lwa6
2014-05-19 14:15:20 [INFO    ] DP 1: 25 lwa7
2014-05-19 14:15:20 [INFO    ] DP 1: 26 lwa33
2014-05-19 14:15:20 [INFO    ] DP 1: 27 lwa15
2014-05-19 14:15:20 [INFO    ] DP 1: 28 lwa28
2014-05-19 14:15:20 [INFO    ] DP 2: 1 lwa16
2014-05-19 14:15:20 [INFO    ] DP 2: 15 lwa34
2014-05-19 14:15:20 [INFO    ] Total of 28 boards
2014-05-19 14:15:20 [INFO    ] Total of 260 stands
2014-05-19 14:15:20 [INFO    ] -------------------
2014-05-19 14:15:20 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:21 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:22 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:23 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10770
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10771
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10772
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10773
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10774
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10775
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10776
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10777
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10778
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10779
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10780
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10781
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10782
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10783
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10784
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10785
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10786
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10787
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10788
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10789
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10790
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10791
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10792
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10793
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10794
2014-05-19 14:15:27 [DEBUG   ] 2014-05-19 14:15:27 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 10795
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10770 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10771 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10772 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10773 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10774 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10775 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10776 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10777 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10778 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10779 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10780 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10781 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10782 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10783 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10784 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10785 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10786 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10787 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10788 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10789 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10790 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10791 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10792 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10793 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10794 exited with status 0
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 10795 exited with status 0
2014-05-19 14:15:39 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] spawn_process: at 2014-05-19 14:15:39 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 10822
2014-05-19 14:15:39 [DEBUG   ] 2014-05-19 14:15:39 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 10823
2014-05-19 14:15:45 [DEBUG   ] 2014-05-19 14:15:45 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 10822 exited with status 0
2014-05-19 14:15:45 [DEBUG   ] 2014-05-19 14:15:45 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 10823 exited with status 0
2014-05-19 14:15:45 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2014-05-19 14:15:45 [DEBUG   ] spawn_process: at 2014-05-19 14:15:45 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2014-05-19 14:15:45 [DEBUG   ] spawn_process: at 2014-05-19 14:15:45 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2014-05-19 14:15:45 [INFO    ] Setting Beam 1 xbars
2014-05-19 14:15:45 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2014-05-19 14:15:45 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2014-05-19 14:15:45 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2014-05-19 14:15:45 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2014-05-19 14:15:45 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2014-05-19 14:15:46 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2014-05-19 14:15:46 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2014-05-19 14:15:46 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2014-05-19 14:15:46 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2014-05-19 14:15:46 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2014-05-19 14:15:46 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2014-05-19 14:15:46 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2014-05-19 14:15:46 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2014-05-19 14:15:46 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2014-05-19 14:15:47 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2014-05-19 14:15:47 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2014-05-19 14:15:47 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2014-05-19 14:15:47 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2014-05-19 14:15:47 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2014-05-19 14:15:47 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2014-05-19 14:15:47 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2014-05-19 14:15:47 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2014-05-19 14:15:47 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2014-05-19 14:15:48 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2014-05-19 14:15:48 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2014-05-19 14:15:48 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2014-05-19 14:15:48 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2014-05-19 14:15:48 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2014-05-19 14:15:48 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2014-05-19 14:15:48 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2014-05-19 14:15:48 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2014-05-19 14:15:48 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2014-05-19 14:15:49 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2014-05-19 14:15:49 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2014-05-19 14:15:49 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2014-05-19 14:15:49 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2014-05-19 14:15:49 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2014-05-19 14:15:49 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2014-05-19 14:15:49 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2014-05-19 14:15:49 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2014-05-19 14:15:49 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2014-05-19 14:15:49 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2014-05-19 14:15:50 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2014-05-19 14:15:50 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2014-05-19 14:15:50 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2014-05-19 14:15:50 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2014-05-19 14:15:50 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2014-05-19 14:15:50 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2014-05-19 14:15:50 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2014-05-19 14:15:50 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2014-05-19 14:15:50 [INFO    ] Setting Beam 2 xbars
2014-05-19 14:15:50 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2014-05-19 14:15:51 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2014-05-19 14:15:51 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2014-05-19 14:15:51 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2014-05-19 14:15:51 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2014-05-19 14:15:51 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2014-05-19 14:15:51 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2014-05-19 14:15:51 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2014-05-19 14:15:51 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2014-05-19 14:15:51 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2014-05-19 14:15:52 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2014-05-19 14:15:52 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2014-05-19 14:15:52 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2014-05-19 14:15:52 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2014-05-19 14:15:52 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2014-05-19 14:15:52 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2014-05-19 14:15:52 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2014-05-19 14:15:52 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2014-05-19 14:15:52 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2014-05-19 14:15:52 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2014-05-19 14:15:53 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2014-05-19 14:15:53 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2014-05-19 14:15:53 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2014-05-19 14:15:53 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2014-05-19 14:15:53 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2014-05-19 14:15:53 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2014-05-19 14:15:53 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2014-05-19 14:15:53 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2014-05-19 14:15:53 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2014-05-19 14:15:54 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2014-05-19 14:15:54 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2014-05-19 14:15:54 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2014-05-19 14:15:54 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2014-05-19 14:15:54 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2014-05-19 14:15:54 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2014-05-19 14:15:54 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2014-05-19 14:15:54 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2014-05-19 14:15:54 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2014-05-19 14:15:55 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2014-05-19 14:15:55 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2014-05-19 14:15:55 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2014-05-19 14:15:55 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2014-05-19 14:15:55 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2014-05-19 14:15:55 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2014-05-19 14:15:55 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2014-05-19 14:15:55 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2014-05-19 14:15:55 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2014-05-19 14:15:56 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2014-05-19 14:15:56 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2014-05-19 14:15:56 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2014-05-19 14:15:56 [INFO    ] Setting Beam 3 xbars
2014-05-19 14:15:56 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2014-05-19 14:15:56 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2014-05-19 14:15:56 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2014-05-19 14:15:56 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2014-05-19 14:15:56 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2014-05-19 14:15:56 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2014-05-19 14:15:57 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2014-05-19 14:15:57 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2014-05-19 14:15:57 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2014-05-19 14:15:57 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2014-05-19 14:15:57 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2014-05-19 14:15:57 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2014-05-19 14:15:57 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2014-05-19 14:15:57 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2014-05-19 14:15:57 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2014-05-19 14:15:57 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2014-05-19 14:15:58 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2014-05-19 14:15:58 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2014-05-19 14:15:58 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2014-05-19 14:15:58 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2014-05-19 14:15:58 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2014-05-19 14:15:58 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2014-05-19 14:15:58 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2014-05-19 14:15:58 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2014-05-19 14:15:58 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2014-05-19 14:15:59 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2014-05-19 14:15:59 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2014-05-19 14:15:59 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2014-05-19 14:15:59 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2014-05-19 14:15:59 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2014-05-19 14:15:59 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2014-05-19 14:15:59 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2014-05-19 14:15:59 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2014-05-19 14:15:59 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2014-05-19 14:16:00 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2014-05-19 14:16:00 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2014-05-19 14:16:00 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2014-05-19 14:16:00 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2014-05-19 14:16:00 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2014-05-19 14:16:00 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2014-05-19 14:16:00 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2014-05-19 14:16:00 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2014-05-19 14:16:00 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2014-05-19 14:16:01 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2014-05-19 14:16:01 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2014-05-19 14:16:01 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2014-05-19 14:16:01 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2014-05-19 14:16:01 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2014-05-19 14:16:01 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2014-05-19 14:16:01 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2014-05-19 14:16:01 [INFO    ] Setting Beam 4 xbars
2014-05-19 14:16:01 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2014-05-19 14:16:01 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 14:16:02 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 14:16:02 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 14:16:02 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 14:16:02 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2014-05-19 14:16:02 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2014-05-19 14:16:02 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2014-05-19 14:16:02 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2014-05-19 14:16:02 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2014-05-19 14:16:02 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2014-05-19 14:16:02 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2014-05-19 14:16:03 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2014-05-19 14:16:03 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2014-05-19 14:16:03 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2014-05-19 14:16:03 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2014-05-19 14:16:03 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2014-05-19 14:16:03 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2014-05-19 14:16:03 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2014-05-19 14:16:03 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2014-05-19 14:16:03 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2014-05-19 14:16:04 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2014-05-19 14:16:04 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2014-05-19 14:16:04 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2014-05-19 14:16:04 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2014-05-19 14:16:04 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2014-05-19 14:16:04 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2014-05-19 14:16:04 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2014-05-19 14:16:04 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2014-05-19 14:16:04 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2014-05-19 14:16:05 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2014-05-19 14:16:05 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2014-05-19 14:16:05 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2014-05-19 14:16:05 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2014-05-19 14:16:05 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2014-05-19 14:16:05 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2014-05-19 14:16:05 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2014-05-19 14:16:05 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2014-05-19 14:16:05 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2014-05-19 14:16:06 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2014-05-19 14:16:06 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2014-05-19 14:16:06 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2014-05-19 14:16:06 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2014-05-19 14:16:06 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2014-05-19 14:16:06 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2014-05-19 14:16:06 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2014-05-19 14:16:06 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2014-05-19 14:16:06 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2014-05-19 14:16:07 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2014-05-19 14:16:07 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2014-05-19 14:16:07 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2014-05-19 14:16:07 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2014-05-19 14:16:07 [INFO    ] All xbars set
2014-05-19 14:16:07 [INFO    ] Setting adder chain routing for all beams
2014-05-19 14:16:12 [INFO    ] Setting beam ids
2014-05-19 14:16:22 [INFO    ] Starting beam delay calibration
2014-05-19 14:16:34 [INFO    ] Beam 1 latency is 5735
2014-05-19 14:16:34 [INFO    ] Beam 2 latency is 6702
2014-05-19 14:16:34 [INFO    ] Beam 3 latency is 6692
2014-05-19 14:16:34 [INFO    ] Beam 4 latency is 5707
2014-05-19 14:16:34 [INFO    ] Stop calibration
2014-05-19 14:16:42 [ERROR   ] Beam delay calibration failed
2014-05-19 14:16:42 [CRITICAL] Calibration failed
2014-05-19 14:16:42 [DEBUG   ] spawn_process: at 2014-05-19 14:16:42 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2014-05-19 14:16:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2014-05-19 14:16:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2014-05-19 14:16:42 [DEBUG   ] spawn_process: at 2014-05-19 14:16:42 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2014-05-19 14:16:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2014-05-19 14:16:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2014-05-19 14:16:42 [DEBUG   ] spawn_process: at 2014-05-19 14:16:42 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2014-05-19 14:16:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2014-05-19 14:16:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2014-05-19 14:16:42 [DEBUG   ] spawn_process: at 2014-05-19 14:16:42 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2014-05-19 14:16:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2014-05-19 14:16:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2014-05-19 14:16:42 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -5 msec
2014-05-19 14:16:43 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 277 ms (1400509003.283 s) with PID 12394 and time 1400509004
2014-05-19 14:16:43 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -6 msec
2014-05-19 14:16:43 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 367 ms (1400509003.373 s) with PID 12397 and time 1400509004
2014-05-19 14:16:44 [DEBUG   ] Finished updating ant stats in 0.878 seconds
2014-05-19 14:16:44 [INFO    ] Started antennas statistics background thread
2014-05-19 14:16:44 [INFO    ] Started board flags background thread
2014-05-19 14:16:44 [INFO    ] Started BAM background thread
2014-05-19 14:16:44 [INFO    ] Finished the INI process (ref. #193534) in 104.386 s
2014-05-19 14:16:45 [DEBUG   ] Finished updating board flags in 1.086 seconds
2014-05-19 14:17:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 14:17:06 [DEBUG   ] Got command RPT from MCS: ref #193540
2014-05-19 14:17:06 [DEBUG   ] summary = ERROR
2014-05-19 14:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193540  13 56796 51426730 A  ERRORERROR'
2014-05-19 14:17:06 [DEBUG   ] Got command RPT from MCS: ref #193541
2014-05-19 14:17:06 [DEBUG   ] info = T_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x06! Beamformer calibration failed
2014-05-19 14:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193541  72 56796 51426731 A  ERRORT_NOM1 T_NOM2 T_NOM3 T_NOM4! 0x06! Beamformer calibration failed'
2014-05-19 14:17:24 [DEBUG   ] Finished updating ant stats in 0.793 seconds
2014-05-19 14:17:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:17:45 [DEBUG   ] Finished updating board flags in 1.451 seconds
2014-05-19 14:18:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:18:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:18:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:18:45 [DEBUG   ] Finished updating board flags in 1.410 seconds
2014-05-19 14:19:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:19:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:19:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:19:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 14:19:46 [DEBUG   ] Finished updating board flags in 1.955 seconds
2014-05-19 14:20:00 [DEBUG   ] Got command RPT from MCS: ref #193557
2014-05-19 14:20:00 [DEBUG   ] summary = ERROR
2014-05-19 14:20:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193557  13 56796 51600106 A  ERRORERROR'
2014-05-19 14:20:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:20:05 [DEBUG   ] Got command INI from MCS: ref #193558
2014-05-19 14:20:05 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2014-05-19 14:20:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   193558   8 56796 51605119 ABOOTING'
2014-05-19 14:20:25 [INFO    ] -------------------
2014-05-19 14:20:25 [INFO    ]  LWA board status
2014-05-19 14:20:25 [INFO    ] -------------------
2014-05-19 14:20:25 [INFO    ] DP 1: 2 lwa27
2014-05-19 14:20:25 [INFO    ] DP 1: 3 lwa24
2014-05-19 14:20:25 [INFO    ] DP 1: 4 lwa23
2014-05-19 14:20:25 [INFO    ] DP 1: 5 lwa18
2014-05-19 14:20:25 [INFO    ] DP 1: 6 lwa10
2014-05-19 14:20:25 [INFO    ] DP 1: 7 lwa32
2014-05-19 14:20:25 [INFO    ] DP 1: 8 lwa11
2014-05-19 14:20:25 [INFO    ] DP 1: 9 lwa30
2014-05-19 14:20:25 [INFO    ] DP 1: 10 lwa12
2014-05-19 14:20:25 [INFO    ] DP 1: 11 lwa8
2014-05-19 14:20:25 [INFO    ] DP 1: 12 lwa13
2014-05-19 14:20:25 [INFO    ] DP 1: 13 lwa26
2014-05-19 14:20:25 [INFO    ] DP 1: 14 lwa14
2014-05-19 14:20:25 [INFO    ] DP 1: 16 lwa22
2014-05-19 14:20:25 [INFO    ] DP 1: 17 lwa20
2014-05-19 14:20:25 [INFO    ] DP 1: 18 lwa1
2014-05-19 14:20:25 [INFO    ] DP 1: 19 lwa5
2014-05-19 14:20:25 [INFO    ] DP 1: 20 lwa25
2014-05-19 14:20:25 [INFO    ] DP 1: 21 lwa35
2014-05-19 14:20:25 [INFO    ] DP 1: 22 lwa19
2014-05-19 14:20:25 [INFO    ] DP 1: 23 lwa21
2014-05-19 14:20:25 [INFO    ] DP 1: 24 lwa6
2014-05-19 14:20:25 [INFO    ] DP 1: 25 lwa7
2014-05-19 14:20:25 [INFO    ] DP 1: 26 lwa33
2014-05-19 14:20:25 [INFO    ] DP 1: 27 lwa15
2014-05-19 14:20:25 [INFO    ] DP 1: 28 lwa28
2014-05-19 14:20:25 [INFO    ] DP 2: 1 lwa16
2014-05-19 14:20:25 [INFO    ] DP 2: 15 lwa34
2014-05-19 14:20:25 [INFO    ] Total of 28 boards
2014-05-19 14:20:25 [INFO    ] Total of 260 stands
2014-05-19 14:20:25 [INFO    ] -------------------
2014-05-19 14:20:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13183
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13184
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13185
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13186
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13187
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13188
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13189
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13190
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13191
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13192
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13193
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13194
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13195
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13196
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13197
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13198
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13199
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13200
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13201
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13202
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13203
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13204
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13205
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13206
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13207
2014-05-19 14:20:32 [DEBUG   ] 2014-05-19 14:20:32 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 13208
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13183 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13184 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13185 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13186 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13187 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13188 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13189 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13190 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13191 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13192 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13193 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13194 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13195 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13196 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13197 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13198 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13199 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13200 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13201 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13202 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13203 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13204 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13205 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13206 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13207 exited with status 0
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 13208 exited with status 0
2014-05-19 14:20:44 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] spawn_process: at 2014-05-19 14:20:44 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 13235
2014-05-19 14:20:44 [DEBUG   ] 2014-05-19 14:20:44 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 13236
2014-05-19 14:20:50 [DEBUG   ] 2014-05-19 14:20:50 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 13235 exited with status 0
2014-05-19 14:20:50 [DEBUG   ] 2014-05-19 14:20:50 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 13236 exited with status 0
2014-05-19 14:20:50 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2014-05-19 14:20:50 [DEBUG   ] spawn_process: at 2014-05-19 14:20:50 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2014-05-19 14:20:50 [DEBUG   ] spawn_process: at 2014-05-19 14:20:50 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2014-05-19 14:20:50 [INFO    ] Setting Beam 1 xbars
2014-05-19 14:20:50 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2014-05-19 14:20:50 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2014-05-19 14:20:50 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2014-05-19 14:20:50 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2014-05-19 14:20:51 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2014-05-19 14:20:51 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2014-05-19 14:20:51 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2014-05-19 14:20:51 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2014-05-19 14:20:51 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2014-05-19 14:20:51 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2014-05-19 14:20:51 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2014-05-19 14:20:51 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2014-05-19 14:20:51 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2014-05-19 14:20:51 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2014-05-19 14:20:52 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2014-05-19 14:20:52 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2014-05-19 14:20:52 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2014-05-19 14:20:52 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2014-05-19 14:20:52 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2014-05-19 14:20:52 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2014-05-19 14:20:52 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2014-05-19 14:20:52 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2014-05-19 14:20:52 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2014-05-19 14:20:53 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2014-05-19 14:20:53 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2014-05-19 14:20:53 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2014-05-19 14:20:53 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2014-05-19 14:20:53 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2014-05-19 14:20:53 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2014-05-19 14:20:53 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2014-05-19 14:20:53 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2014-05-19 14:20:53 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2014-05-19 14:20:54 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2014-05-19 14:20:54 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2014-05-19 14:20:54 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2014-05-19 14:20:54 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2014-05-19 14:20:54 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2014-05-19 14:20:54 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2014-05-19 14:20:54 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2014-05-19 14:20:54 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2014-05-19 14:20:54 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2014-05-19 14:20:55 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2014-05-19 14:20:55 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2014-05-19 14:20:55 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2014-05-19 14:20:55 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2014-05-19 14:20:55 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2014-05-19 14:20:55 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2014-05-19 14:20:55 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2014-05-19 14:20:55 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2014-05-19 14:20:55 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2014-05-19 14:20:55 [INFO    ] Setting Beam 2 xbars
2014-05-19 14:20:55 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2014-05-19 14:20:56 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2014-05-19 14:20:56 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2014-05-19 14:20:56 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2014-05-19 14:20:56 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2014-05-19 14:20:56 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2014-05-19 14:20:56 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2014-05-19 14:20:56 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2014-05-19 14:20:56 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2014-05-19 14:20:56 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2014-05-19 14:20:57 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2014-05-19 14:20:57 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2014-05-19 14:20:57 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2014-05-19 14:20:57 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2014-05-19 14:20:57 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2014-05-19 14:20:57 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2014-05-19 14:20:57 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2014-05-19 14:20:57 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2014-05-19 14:20:57 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2014-05-19 14:20:58 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2014-05-19 14:20:58 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2014-05-19 14:20:58 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2014-05-19 14:20:58 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2014-05-19 14:20:58 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2014-05-19 14:20:58 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2014-05-19 14:20:58 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2014-05-19 14:20:58 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2014-05-19 14:20:58 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2014-05-19 14:20:59 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2014-05-19 14:20:59 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2014-05-19 14:20:59 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2014-05-19 14:20:59 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2014-05-19 14:20:59 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2014-05-19 14:20:59 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2014-05-19 14:20:59 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2014-05-19 14:20:59 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2014-05-19 14:20:59 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2014-05-19 14:20:59 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2014-05-19 14:21:00 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2014-05-19 14:21:00 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2014-05-19 14:21:00 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2014-05-19 14:21:00 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2014-05-19 14:21:00 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2014-05-19 14:21:00 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2014-05-19 14:21:00 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2014-05-19 14:21:00 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2014-05-19 14:21:00 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2014-05-19 14:21:01 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2014-05-19 14:21:01 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2014-05-19 14:21:01 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2014-05-19 14:21:01 [INFO    ] Setting Beam 3 xbars
2014-05-19 14:21:01 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2014-05-19 14:21:01 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2014-05-19 14:21:01 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2014-05-19 14:21:01 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2014-05-19 14:21:01 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2014-05-19 14:21:01 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2014-05-19 14:21:02 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2014-05-19 14:21:02 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2014-05-19 14:21:02 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2014-05-19 14:21:02 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2014-05-19 14:21:02 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2014-05-19 14:21:02 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2014-05-19 14:21:02 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2014-05-19 14:21:02 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2014-05-19 14:21:02 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2014-05-19 14:21:03 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2014-05-19 14:21:03 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2014-05-19 14:21:03 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2014-05-19 14:21:03 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2014-05-19 14:21:03 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2014-05-19 14:21:03 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2014-05-19 14:21:03 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2014-05-19 14:21:03 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2014-05-19 14:21:03 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2014-05-19 14:21:03 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2014-05-19 14:21:04 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2014-05-19 14:21:04 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2014-05-19 14:21:04 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2014-05-19 14:21:04 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2014-05-19 14:21:04 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2014-05-19 14:21:04 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2014-05-19 14:21:04 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2014-05-19 14:21:04 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2014-05-19 14:21:04 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2014-05-19 14:21:05 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2014-05-19 14:21:05 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2014-05-19 14:21:05 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2014-05-19 14:21:05 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2014-05-19 14:21:05 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2014-05-19 14:21:05 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2014-05-19 14:21:05 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2014-05-19 14:21:05 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2014-05-19 14:21:05 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2014-05-19 14:21:06 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2014-05-19 14:21:06 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2014-05-19 14:21:06 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2014-05-19 14:21:06 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2014-05-19 14:21:06 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2014-05-19 14:21:06 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2014-05-19 14:21:06 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2014-05-19 14:21:06 [INFO    ] Setting Beam 4 xbars
2014-05-19 14:21:06 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2014-05-19 14:21:06 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 14:21:07 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 14:21:07 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 14:21:07 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 14:21:07 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2014-05-19 14:21:07 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2014-05-19 14:21:07 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2014-05-19 14:21:07 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2014-05-19 14:21:07 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2014-05-19 14:21:07 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2014-05-19 14:21:08 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2014-05-19 14:21:08 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2014-05-19 14:21:08 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2014-05-19 14:21:08 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2014-05-19 14:21:08 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2014-05-19 14:21:08 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2014-05-19 14:21:08 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2014-05-19 14:21:08 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2014-05-19 14:21:08 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2014-05-19 14:21:08 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2014-05-19 14:21:09 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2014-05-19 14:21:09 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2014-05-19 14:21:09 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2014-05-19 14:21:09 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2014-05-19 14:21:09 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2014-05-19 14:21:09 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2014-05-19 14:21:09 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2014-05-19 14:21:09 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2014-05-19 14:21:09 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2014-05-19 14:21:10 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2014-05-19 14:21:10 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2014-05-19 14:21:10 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2014-05-19 14:21:10 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2014-05-19 14:21:10 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2014-05-19 14:21:10 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2014-05-19 14:21:10 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2014-05-19 14:21:10 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2014-05-19 14:21:10 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2014-05-19 14:21:11 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2014-05-19 14:21:11 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2014-05-19 14:21:11 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2014-05-19 14:21:11 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2014-05-19 14:21:11 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2014-05-19 14:21:11 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2014-05-19 14:21:11 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2014-05-19 14:21:11 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2014-05-19 14:21:11 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2014-05-19 14:21:12 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2014-05-19 14:21:12 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2014-05-19 14:21:12 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2014-05-19 14:21:12 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2014-05-19 14:21:12 [INFO    ] All xbars set
2014-05-19 14:21:12 [INFO    ] Setting adder chain routing for all beams
2014-05-19 14:21:17 [INFO    ] Setting beam ids
2014-05-19 14:21:27 [INFO    ] Starting beam delay calibration
2014-05-19 14:21:39 [INFO    ] Beam 1 latency is 5755
2014-05-19 14:21:39 [INFO    ] Beam 2 latency is 5745
2014-05-19 14:21:39 [INFO    ] Beam 3 latency is 5746
2014-05-19 14:21:39 [INFO    ] Beam 4 latency is 5724
2014-05-19 14:21:39 [INFO    ] Stop calibration
2014-05-19 14:21:47 [INFO    ] Beam delay calibration successful
2014-05-19 14:21:47 [INFO    ] Calibration passed
2014-05-19 14:21:47 [DEBUG   ] spawn_process: at 2014-05-19 14:21:47 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2014-05-19 14:21:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2014-05-19 14:21:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2014-05-19 14:21:47 [DEBUG   ] spawn_process: at 2014-05-19 14:21:47 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2014-05-19 14:21:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2014-05-19 14:21:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2014-05-19 14:21:47 [DEBUG   ] spawn_process: at 2014-05-19 14:21:47 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2014-05-19 14:21:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2014-05-19 14:21:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2014-05-19 14:21:47 [DEBUG   ] spawn_process: at 2014-05-19 14:21:47 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2014-05-19 14:21:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2014-05-19 14:21:47 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2014-05-19 14:21:47 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -5 msec
2014-05-19 14:21:48 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 282 ms (1400509308.287 s) with PID 14838 and time 1400509309
2014-05-19 14:21:48 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -5 msec
2014-05-19 14:21:48 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 373 ms (1400509308.378 s) with PID 14841 and time 1400509309
2014-05-19 14:21:49 [DEBUG   ] Finished updating ant stats in 0.873 seconds
2014-05-19 14:21:49 [INFO    ] Started antennas statistics background thread
2014-05-19 14:21:49 [INFO    ] Started board flags background thread
2014-05-19 14:21:49 [INFO    ] Started BAM background thread
2014-05-19 14:21:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 14:21:49 [INFO    ] Finished the INI process (ref. #193558) in 104.383 s
2014-05-19 14:21:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 14:21:50 [DEBUG   ] Finished updating board flags in 1.063 seconds
2014-05-19 14:22:06 [DEBUG   ] Got command RPT from MCS: ref #193559
2014-05-19 14:22:06 [DEBUG   ] summary = NORMAL
2014-05-19 14:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193559  14 56796 51726738 A NORMALNORMAL'
2014-05-19 14:22:06 [DEBUG   ] Got command RPT from MCS: ref #193560
2014-05-19 14:22:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 14:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193560  48 56796 51726739 A NORMALSystem calibrated and operating normally'
2014-05-19 14:22:09 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 14:22:29 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 14:22:49 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 14:22:50 [DEBUG   ] Finished updating board flags in 0.994 seconds
2014-05-19 14:23:09 [DEBUG   ] Finished updating ant stats in 0.777 seconds
2014-05-19 14:23:29 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:23:49 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:23:50 [DEBUG   ] Finished updating board flags in 1.272 seconds
2014-05-19 14:24:09 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:24:29 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:24:49 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:24:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 14:24:51 [DEBUG   ] Finished updating board flags in 1.796 seconds
2014-05-19 14:25:05 [DEBUG   ] Got command RPT from MCS: ref #193576
2014-05-19 14:25:05 [DEBUG   ] summary = NORMAL
2014-05-19 14:25:05 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193576  14 56796 51905114 A NORMALNORMAL'
2014-05-19 14:25:09 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:25:29 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:25:49 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:25:51 [DEBUG   ] Finished updating board flags in 1.622 seconds
2014-05-19 14:26:09 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:26:29 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:26:49 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:26:51 [DEBUG   ] Finished updating board flags in 1.518 seconds
2014-05-19 14:27:06 [DEBUG   ] Got command RPT from MCS: ref #193577
2014-05-19 14:27:06 [DEBUG   ] summary = NORMAL
2014-05-19 14:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193577  14 56796 52026734 A NORMALNORMAL'
2014-05-19 14:27:06 [DEBUG   ] Got command RPT from MCS: ref #193578
2014-05-19 14:27:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 14:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193578  48 56796 52026735 A NORMALSystem calibrated and operating normally'
2014-05-19 14:27:09 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:27:29 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:27:49 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:27:51 [DEBUG   ] Finished updating board flags in 1.428 seconds
2014-05-19 14:28:09 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:28:29 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:28:49 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:28:51 [DEBUG   ] Finished updating board flags in 1.388 seconds
2014-05-19 14:29:09 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:29:29 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:29:49 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:29:51 [DEBUG   ] Finished updating board flags in 1.214 seconds
2014-05-19 14:30:00 [DEBUG   ] Got command TBN from MCS: ref #193594
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '1', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '2', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] tbn = exited with status True, code 0x00
2014-05-19 14:30:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBN   193594   8 56796 52200135 A NORMAL'
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '3', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '4', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '5', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '6', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '7', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '8', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '9', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '10', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '11', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '12', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '13', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '14', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '15', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '16', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '17', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '18', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '19', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '20', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '21', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '22', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '23', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '24', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '25', '1400509801']
2014-05-19 14:30:00 [DEBUG   ] spawn_process: at 2014-05-19 14:30:00 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '26', '1400509801']
2014-05-19 14:30:00 [INFO    ] Finished spawning TBN in 0.060 s
2014-05-19 14:30:09 [DEBUG   ] Finished updating ant stats in 0.668 seconds
2014-05-19 14:30:29 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:30:49 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 14:30:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 14:30:51 [DEBUG   ] Finished updating board flags in 1.437 seconds
2014-05-19 14:31:09 [DEBUG   ] Finished updating ant stats in 0.594 seconds
2014-05-19 14:31:30 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-19 14:31:50 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 14:31:51 [DEBUG   ] Finished updating board flags in 1.430 seconds
2014-05-19 14:32:06 [DEBUG   ] Got command RPT from MCS: ref #193595
2014-05-19 14:32:06 [DEBUG   ] summary = NORMAL
2014-05-19 14:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193595  14 56796 52326742 A NORMALNORMAL'
2014-05-19 14:32:06 [DEBUG   ] Got command RPT from MCS: ref #193596
2014-05-19 14:32:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 14:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193596  48 56796 52326743 A NORMALSystem calibrated and operating normally'
2014-05-19 14:32:10 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 14:32:30 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 14:32:50 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 14:32:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 14:32:51 [DEBUG   ] Finished updating board flags in 1.464 seconds
2014-05-19 14:33:10 [DEBUG   ] Finished updating ant stats in 1.195 seconds
2014-05-19 14:33:30 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:33:50 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 14:33:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 14:33:51 [DEBUG   ] Finished updating board flags in 1.461 seconds
2014-05-19 14:34:10 [DEBUG   ] Finished updating ant stats in 0.986 seconds
2014-05-19 14:34:30 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 14:34:50 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 14:34:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 14:34:51 [DEBUG   ] Finished updating board flags in 1.464 seconds
2014-05-19 14:35:10 [DEBUG   ] Finished updating ant stats in 0.788 seconds
2014-05-19 14:35:30 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:35:51 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-19 14:35:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 14:35:52 [DEBUG   ] Finished updating board flags in 2.205 seconds
2014-05-19 14:36:11 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 14:36:31 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 14:36:51 [DEBUG   ] Finished updating ant stats in 1.250 seconds
2014-05-19 14:36:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 14:36:52 [DEBUG   ] Finished updating board flags in 2.088 seconds
2014-05-19 14:37:06 [DEBUG   ] Got command RPT from MCS: ref #193612
2014-05-19 14:37:06 [DEBUG   ] summary = NORMAL
2014-05-19 14:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193612  14 56796 52626738 A NORMALNORMAL'
2014-05-19 14:37:06 [DEBUG   ] Got command RPT from MCS: ref #193613
2014-05-19 14:37:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 14:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193613  48 56796 52626739 A NORMALSystem calibrated and operating normally'
2014-05-19 14:37:11 [DEBUG   ] Finished updating ant stats in 1.153 seconds
2014-05-19 14:37:31 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 14:37:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 14:37:51 [DEBUG   ] Finished updating ant stats in 0.854 seconds
2014-05-19 14:37:52 [DEBUG   ] Finished updating board flags in 2.023 seconds
2014-05-19 14:38:11 [DEBUG   ] Finished updating ant stats in 0.754 seconds
2014-05-19 14:38:31 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:38:51 [DEBUG   ] Finished updating ant stats in 0.648 seconds
2014-05-19 14:38:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 14:38:52 [DEBUG   ] Finished updating board flags in 1.984 seconds
2014-05-19 14:39:11 [DEBUG   ] Finished updating ant stats in 0.558 seconds
2014-05-19 14:39:32 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-19 14:39:52 [DEBUG   ] Finished updating ant stats in 1.247 seconds
2014-05-19 14:39:52 [DEBUG   ] Finished updating board flags in 1.878 seconds
2014-05-19 14:40:12 [DEBUG   ] Finished updating ant stats in 1.160 seconds
2014-05-19 14:40:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:40:52 [DEBUG   ] Finished updating ant stats in 1.044 seconds
2014-05-19 14:40:52 [DEBUG   ] Finished updating board flags in 1.783 seconds
2014-05-19 14:41:12 [DEBUG   ] Finished updating ant stats in 0.960 seconds
2014-05-19 14:41:32 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 14:41:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 14:41:52 [DEBUG   ] Finished updating ant stats in 0.842 seconds
2014-05-19 14:41:52 [DEBUG   ] Finished updating board flags in 1.612 seconds
2014-05-19 14:42:06 [DEBUG   ] Got command RPT from MCS: ref #193629
2014-05-19 14:42:06 [DEBUG   ] summary = NORMAL
2014-05-19 14:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193629  14 56796 52926742 A NORMALNORMAL'
2014-05-19 14:42:06 [DEBUG   ] Got command RPT from MCS: ref #193630
2014-05-19 14:42:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 14:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193630  48 56796 52926743 A NORMALSystem calibrated and operating normally'
2014-05-19 14:42:12 [DEBUG   ] Finished updating ant stats in 0.757 seconds
2014-05-19 14:42:32 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:42:52 [DEBUG   ] Finished updating board flags in 1.613 seconds
2014-05-19 14:42:53 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 14:43:13 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 14:43:33 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 14:43:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 14:43:52 [DEBUG   ] Finished updating board flags in 1.568 seconds
2014-05-19 14:43:53 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 14:44:13 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 14:44:33 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 14:44:52 [DEBUG   ] Finished updating board flags in 1.511 seconds
2014-05-19 14:44:53 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 14:45:13 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:45:33 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 14:45:52 [DEBUG   ] Finished updating board flags in 1.514 seconds
2014-05-19 14:45:53 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 14:46:13 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 14:46:33 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:46:53 [DEBUG   ] Finished updating ant stats in 0.646 seconds
2014-05-19 14:46:53 [DEBUG   ] Finished updating board flags in 2.207 seconds
2014-05-19 14:47:06 [DEBUG   ] Got command RPT from MCS: ref #193646
2014-05-19 14:47:06 [DEBUG   ] summary = NORMAL
2014-05-19 14:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193646  14 56796 53226734 A NORMALNORMAL'
2014-05-19 14:47:06 [DEBUG   ] Got command RPT from MCS: ref #193647
2014-05-19 14:47:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 14:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193647  48 56796 53226735 A NORMALSystem calibrated and operating normally'
2014-05-19 14:47:13 [DEBUG   ] Finished updating ant stats in 0.555 seconds
2014-05-19 14:47:34 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 14:47:53 [DEBUG   ] Finished updating board flags in 2.197 seconds
2014-05-19 14:47:54 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 14:48:14 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 14:48:34 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-19 14:48:53 [DEBUG   ] Finished updating board flags in 2.002 seconds
2014-05-19 14:48:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 14:49:14 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2014-05-19 14:49:34 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 14:49:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 14:49:53 [DEBUG   ] Finished updating board flags in 2.019 seconds
2014-05-19 14:49:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 14:50:14 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2014-05-19 14:50:34 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 14:50:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 14:50:53 [DEBUG   ] Finished updating board flags in 2.036 seconds
2014-05-19 14:50:54 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 14:51:14 [DEBUG   ] Finished updating ant stats in 0.592 seconds
2014-05-19 14:51:35 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 14:51:53 [DEBUG   ] Finished updating board flags in 1.903 seconds
2014-05-19 14:51:55 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 14:52:06 [DEBUG   ] Got command RPT from MCS: ref #193663
2014-05-19 14:52:06 [DEBUG   ] summary = NORMAL
2014-05-19 14:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193663  14 56796 53526746 A NORMALNORMAL'
2014-05-19 14:52:06 [DEBUG   ] Got command RPT from MCS: ref #193664
2014-05-19 14:52:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 14:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193664  48 56796 53526747 A NORMALSystem calibrated and operating normally'
2014-05-19 14:52:15 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-19 14:52:35 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 14:52:53 [DEBUG   ] Finished updating board flags in 1.810 seconds
2014-05-19 14:52:55 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 14:53:15 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 14:53:35 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 14:53:53 [DEBUG   ] Finished updating board flags in 1.778 seconds
2014-05-19 14:53:55 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:54:15 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 14:54:35 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 14:54:53 [DEBUG   ] Finished updating board flags in 1.620 seconds
2014-05-19 14:54:55 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 14:55:15 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 14:55:35 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 14:55:53 [DEBUG   ] Finished updating board flags in 1.580 seconds
2014-05-19 14:55:55 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 14:56:15 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 14:56:35 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 14:56:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 14:56:53 [DEBUG   ] Finished updating board flags in 1.428 seconds
2014-05-19 14:56:55 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:57:06 [DEBUG   ] Got command RPT from MCS: ref #193685
2014-05-19 14:57:06 [DEBUG   ] summary = NORMAL
2014-05-19 14:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193685  14 56796 53826742 A NORMALNORMAL'
2014-05-19 14:57:06 [DEBUG   ] Got command RPT from MCS: ref #193686
2014-05-19 14:57:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 14:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193686  48 56796 53826743 A NORMALSystem calibrated and operating normally'
2014-05-19 14:57:15 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 14:57:36 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-19 14:57:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 14:57:53 [DEBUG   ] Finished updating board flags in 1.382 seconds
2014-05-19 14:57:56 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 14:58:16 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 14:58:36 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2014-05-19 14:58:54 [DEBUG   ] Finished updating board flags in 2.208 seconds
2014-05-19 14:58:56 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 14:59:16 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 14:59:36 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 14:59:54 [DEBUG   ] Finished updating board flags in 2.189 seconds
2014-05-19 14:59:56 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 15:00:16 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 15:00:36 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 15:00:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 15:00:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:00:54 [DEBUG   ] Finished updating board flags in 2.028 seconds
2014-05-19 15:00:56 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 15:01:17 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 15:01:37 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 15:01:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:01:54 [DEBUG   ] Finished updating board flags in 1.975 seconds
2014-05-19 15:01:57 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 15:02:06 [DEBUG   ] Got command RPT from MCS: ref #193702
2014-05-19 15:02:06 [DEBUG   ] summary = NORMAL
2014-05-19 15:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193702  14 56796 54126742 A NORMALNORMAL'
2014-05-19 15:02:06 [DEBUG   ] Got command RPT from MCS: ref #193703
2014-05-19 15:02:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 15:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193703  48 56796 54126743 A NORMALSystem calibrated and operating normally'
2014-05-19 15:02:17 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 15:02:37 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 15:02:54 [DEBUG   ] Finished updating board flags in 1.815 seconds
2014-05-19 15:02:57 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 15:03:17 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 15:03:37 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-19 15:03:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 15:03:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-19 15:03:54 [DEBUG   ] Finished updating board flags in 1.814 seconds
2014-05-19 15:03:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 15:04:17 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 15:04:37 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 15:04:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 15:04:54 [DEBUG   ] Finished updating board flags in 1.798 seconds
2014-05-19 15:04:57 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 15:05:17 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 15:05:37 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 15:05:54 [DEBUG   ] Finished updating board flags in 1.606 seconds
2014-05-19 15:05:57 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:06:17 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 15:06:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:06:54 [DEBUG   ] Finished updating board flags in 1.601 seconds
2014-05-19 15:06:57 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 15:07:06 [DEBUG   ] Got command RPT from MCS: ref #193719
2014-05-19 15:07:06 [DEBUG   ] summary = NORMAL
2014-05-19 15:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193719  14 56796 54426742 A NORMALNORMAL'
2014-05-19 15:07:06 [DEBUG   ] Got command RPT from MCS: ref #193720
2014-05-19 15:07:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 15:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193720  48 56796 54426743 A NORMALSystem calibrated and operating normally'
2014-05-19 15:07:18 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 15:07:38 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 15:07:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 15:07:54 [DEBUG   ] Finished updating board flags in 1.600 seconds
2014-05-19 15:07:58 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 15:08:18 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 15:08:38 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 15:08:54 [DEBUG   ] Finished updating board flags in 1.387 seconds
2014-05-19 15:08:58 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2014-05-19 15:09:18 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 15:09:38 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 15:09:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 15:09:54 [DEBUG   ] Finished updating board flags in 1.365 seconds
2014-05-19 15:09:58 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 15:10:18 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 15:10:38 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 15:10:55 [DEBUG   ] Finished updating board flags in 2.051 seconds
2014-05-19 15:10:58 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 15:11:18 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 15:11:38 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 15:11:55 [DEBUG   ] Finished updating board flags in 1.994 seconds
2014-05-19 15:11:58 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 15:12:06 [DEBUG   ] Got command RPT from MCS: ref #193736
2014-05-19 15:12:06 [DEBUG   ] summary = NORMAL
2014-05-19 15:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193736  14 56796 54726742 A NORMALNORMAL'
2014-05-19 15:12:06 [DEBUG   ] Got command RPT from MCS: ref #193737
2014-05-19 15:12:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 15:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193737  48 56796 54726743 A NORMALSystem calibrated and operating normally'
2014-05-19 15:12:19 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 15:12:39 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 15:12:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 15:12:55 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:12:55 [DEBUG   ] Finished updating board flags in 1.804 seconds
2014-05-19 15:12:59 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 15:13:19 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 15:13:39 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 15:13:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-19 15:13:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-19 15:13:55 [DEBUG   ] Finished updating board flags in 1.792 seconds
2014-05-19 15:13:59 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 15:14:19 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 15:14:39 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 15:14:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 15:14:55 [DEBUG   ] Finished updating board flags in 1.618 seconds
2014-05-19 15:14:59 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-19 15:15:19 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 15:15:39 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 15:15:55 [DEBUG   ] Finished updating board flags in 1.616 seconds
2014-05-19 15:15:59 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:16:19 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 15:16:39 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 15:16:55 [DEBUG   ] Finished updating board flags in 1.576 seconds
2014-05-19 15:16:59 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2014-05-19 15:17:06 [DEBUG   ] Got command RPT from MCS: ref #193753
2014-05-19 15:17:06 [DEBUG   ] summary = NORMAL
2014-05-19 15:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193753  14 56796 55026750 A NORMALNORMAL'
2014-05-19 15:17:06 [DEBUG   ] Got command RPT from MCS: ref #193754
2014-05-19 15:17:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 15:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193754  48 56796 55026751 A NORMALSystem calibrated and operating normally'
2014-05-19 15:17:20 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 15:17:40 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 15:17:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:17:55 [DEBUG   ] Finished updating board flags in 1.412 seconds
2014-05-19 15:18:00 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 15:18:20 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 15:18:40 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 15:18:55 [DEBUG   ] Finished updating board flags in 1.391 seconds
2014-05-19 15:19:00 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 15:19:20 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 15:19:40 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:19:56 [DEBUG   ] Finished updating board flags in 2.220 seconds
2014-05-19 15:20:00 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 15:20:20 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2014-05-19 15:20:40 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:20:56 [DEBUG   ] Finished updating board flags in 2.188 seconds
2014-05-19 15:21:00 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:21:20 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 15:21:40 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 15:21:56 [DEBUG   ] Finished updating board flags in 2.013 seconds
2014-05-19 15:22:01 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2014-05-19 15:22:06 [DEBUG   ] Got command RPT from MCS: ref #193770
2014-05-19 15:22:06 [DEBUG   ] summary = NORMAL
2014-05-19 15:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193770  14 56796 55326746 A NORMALNORMAL'
2014-05-19 15:22:06 [DEBUG   ] Got command RPT from MCS: ref #193771
2014-05-19 15:22:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 15:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193771  48 56796 55326747 A NORMALSystem calibrated and operating normally'
2014-05-19 15:22:21 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 15:22:41 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 15:22:44 [DEBUG   ] Got command RPT from MCS: ref #193787
2014-05-19 15:22:44 [DEBUG   ] summary = NORMAL
2014-05-19 15:22:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193787  14 56796 55364278 A NORMALNORMAL'
2014-05-19 15:22:56 [DEBUG   ] Finished updating board flags in 1.982 seconds
2014-05-19 15:23:01 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 15:23:21 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 15:23:41 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 15:23:56 [DEBUG   ] Finished updating board flags in 1.815 seconds
2014-05-19 15:24:01 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 15:24:21 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:24:41 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 15:24:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 3
2014-05-19 15:24:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:24:56 [DEBUG   ] Finished updating board flags in 1.799 seconds
2014-05-19 15:25:01 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 15:25:21 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 15:25:41 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:25:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:25:56 [DEBUG   ] Finished updating board flags in 1.603 seconds
2014-05-19 15:26:01 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2014-05-19 15:26:21 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 15:26:41 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 15:26:46 [DEBUG   ] Got command INI from MCS: ref #193788
2014-05-19 15:26:46 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2014-05-19 15:26:46 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   193788   8 56796 55606747 ABOOTING'
2014-05-19 15:26:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:26:56 [DEBUG   ] Finished updating board flags in 1.635 seconds
2014-05-19 15:27:06 [DEBUG   ] Got command RPT from MCS: ref #193789
2014-05-19 15:27:06 [DEBUG   ] summary = BOOTING
2014-05-19 15:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193789  15 56796 55626746 ABOOTINGBOOTING'
2014-05-19 15:27:06 [DEBUG   ] Got command RPT from MCS: ref #193790
2014-05-19 15:27:06 [DEBUG   ] info = Running INI sequence
2014-05-19 15:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193790  28 56796 55626747 ABOOTINGRunning INI sequence'
2014-05-19 15:27:10 [INFO    ] -------------------
2014-05-19 15:27:10 [INFO    ]  LWA board status
2014-05-19 15:27:10 [INFO    ] -------------------
2014-05-19 15:27:10 [INFO    ] DP 1: 2 lwa27
2014-05-19 15:27:10 [INFO    ] DP 1: 3 lwa24
2014-05-19 15:27:10 [INFO    ] DP 1: 4 lwa23
2014-05-19 15:27:10 [INFO    ] DP 1: 5 lwa18
2014-05-19 15:27:10 [INFO    ] DP 1: 6 lwa10
2014-05-19 15:27:10 [INFO    ] DP 1: 7 lwa32
2014-05-19 15:27:10 [INFO    ] DP 1: 8 lwa11
2014-05-19 15:27:10 [INFO    ] DP 1: 9 lwa30
2014-05-19 15:27:10 [INFO    ] DP 1: 10 lwa12
2014-05-19 15:27:10 [INFO    ] DP 1: 11 lwa8
2014-05-19 15:27:10 [INFO    ] DP 1: 12 lwa13
2014-05-19 15:27:10 [INFO    ] DP 1: 13 lwa26
2014-05-19 15:27:10 [INFO    ] DP 1: 14 lwa14
2014-05-19 15:27:10 [INFO    ] DP 1: 16 lwa22
2014-05-19 15:27:10 [INFO    ] DP 1: 17 lwa20
2014-05-19 15:27:10 [INFO    ] DP 1: 18 lwa1
2014-05-19 15:27:10 [INFO    ] DP 1: 19 lwa5
2014-05-19 15:27:10 [INFO    ] DP 1: 20 lwa25
2014-05-19 15:27:10 [INFO    ] DP 1: 21 lwa35
2014-05-19 15:27:10 [INFO    ] DP 1: 22 lwa19
2014-05-19 15:27:10 [INFO    ] DP 1: 23 lwa21
2014-05-19 15:27:10 [INFO    ] DP 1: 24 lwa6
2014-05-19 15:27:10 [INFO    ] DP 1: 25 lwa7
2014-05-19 15:27:10 [INFO    ] DP 1: 26 lwa33
2014-05-19 15:27:10 [INFO    ] DP 1: 27 lwa15
2014-05-19 15:27:10 [INFO    ] DP 1: 28 lwa28
2014-05-19 15:27:10 [INFO    ] DP 2: 1 lwa16
2014-05-19 15:27:10 [INFO    ] DP 2: 15 lwa34
2014-05-19 15:27:10 [INFO    ] Total of 28 boards
2014-05-19 15:27:10 [INFO    ] Total of 260 stands
2014-05-19 15:27:10 [INFO    ] -------------------
2014-05-19 15:27:10 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:10 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:10 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:11 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:11 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:11 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:12 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:12 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:12 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:12 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:17 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:17 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:17 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2014-05-19 15:27:17 [DEBUG   ] 2014-05-19 15:27:17 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22447
2014-05-19 15:27:17 [DEBUG   ] 2014-05-19 15:27:17 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22448
2014-05-19 15:27:17 [DEBUG   ] 2014-05-19 15:27:17 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22449
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22450
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22451
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22452
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22453
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22454
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22455
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22456
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22457
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22458
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22459
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22460
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22461
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22462
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22463
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22464
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22465
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22466
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22467
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22468
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22469
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22470
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22471
2014-05-19 15:27:18 [DEBUG   ] 2014-05-19 15:27:18 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 22472
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22447 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22448 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22449 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22450 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22451 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22452 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22453 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22454 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22455 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22456 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22457 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22458 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22459 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22460 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22461 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22462 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22463 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22464 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22465 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22466 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22467 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22468 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22469 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22470 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22471 exited with status 0
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 22472 exited with status 0
2014-05-19 15:27:30 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] spawn_process: at 2014-05-19 15:27:30 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 22499
2014-05-19 15:27:30 [DEBUG   ] 2014-05-19 15:27:30 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 22500
2014-05-19 15:27:35 [DEBUG   ] 2014-05-19 15:27:35 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 22499 exited with status 0
2014-05-19 15:27:35 [DEBUG   ] 2014-05-19 15:27:35 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 22500 exited with status 0
2014-05-19 15:27:35 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2014-05-19 15:27:35 [DEBUG   ] spawn_process: at 2014-05-19 15:27:35 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2014-05-19 15:27:35 [DEBUG   ] spawn_process: at 2014-05-19 15:27:35 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2014-05-19 15:27:35 [INFO    ] Setting Beam 1 xbars
2014-05-19 15:27:35 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2014-05-19 15:27:36 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2014-05-19 15:27:36 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2014-05-19 15:27:36 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2014-05-19 15:27:36 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2014-05-19 15:27:36 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2014-05-19 15:27:36 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2014-05-19 15:27:36 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2014-05-19 15:27:36 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2014-05-19 15:27:36 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2014-05-19 15:27:37 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2014-05-19 15:27:37 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2014-05-19 15:27:37 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2014-05-19 15:27:37 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2014-05-19 15:27:37 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2014-05-19 15:27:37 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2014-05-19 15:27:37 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2014-05-19 15:27:37 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2014-05-19 15:27:37 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2014-05-19 15:27:37 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2014-05-19 15:27:38 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2014-05-19 15:27:38 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2014-05-19 15:27:38 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2014-05-19 15:27:38 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2014-05-19 15:27:38 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2014-05-19 15:27:38 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2014-05-19 15:27:38 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2014-05-19 15:27:38 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2014-05-19 15:27:38 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2014-05-19 15:27:39 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2014-05-19 15:27:39 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2014-05-19 15:27:39 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2014-05-19 15:27:39 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2014-05-19 15:27:39 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2014-05-19 15:27:39 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2014-05-19 15:27:39 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2014-05-19 15:27:39 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2014-05-19 15:27:39 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2014-05-19 15:27:40 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2014-05-19 15:27:40 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2014-05-19 15:27:40 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2014-05-19 15:27:40 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2014-05-19 15:27:40 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2014-05-19 15:27:40 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2014-05-19 15:27:40 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2014-05-19 15:27:40 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2014-05-19 15:27:40 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2014-05-19 15:27:41 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2014-05-19 15:27:41 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2014-05-19 15:27:41 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2014-05-19 15:27:41 [INFO    ] Setting Beam 2 xbars
2014-05-19 15:27:41 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2014-05-19 15:27:41 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2014-05-19 15:27:41 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2014-05-19 15:27:41 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2014-05-19 15:27:41 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2014-05-19 15:27:41 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2014-05-19 15:27:41 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2014-05-19 15:27:42 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2014-05-19 15:27:42 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2014-05-19 15:27:42 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2014-05-19 15:27:42 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2014-05-19 15:27:42 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2014-05-19 15:27:42 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2014-05-19 15:27:42 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2014-05-19 15:27:42 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2014-05-19 15:27:42 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2014-05-19 15:27:43 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2014-05-19 15:27:43 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2014-05-19 15:27:43 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2014-05-19 15:27:43 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2014-05-19 15:27:43 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2014-05-19 15:27:43 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2014-05-19 15:27:43 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2014-05-19 15:27:43 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2014-05-19 15:27:43 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2014-05-19 15:27:44 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2014-05-19 15:27:44 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2014-05-19 15:27:44 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2014-05-19 15:27:44 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2014-05-19 15:27:44 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2014-05-19 15:27:44 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2014-05-19 15:27:44 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2014-05-19 15:27:44 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2014-05-19 15:27:44 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2014-05-19 15:27:45 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2014-05-19 15:27:45 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2014-05-19 15:27:45 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2014-05-19 15:27:45 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2014-05-19 15:27:45 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2014-05-19 15:27:45 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2014-05-19 15:27:45 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2014-05-19 15:27:45 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2014-05-19 15:27:45 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2014-05-19 15:27:45 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2014-05-19 15:27:46 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2014-05-19 15:27:46 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2014-05-19 15:27:46 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2014-05-19 15:27:46 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2014-05-19 15:27:46 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2014-05-19 15:27:46 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2014-05-19 15:27:46 [INFO    ] Setting Beam 3 xbars
2014-05-19 15:27:46 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2014-05-19 15:27:46 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2014-05-19 15:27:46 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2014-05-19 15:27:47 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2014-05-19 15:27:47 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2014-05-19 15:27:47 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2014-05-19 15:27:47 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2014-05-19 15:27:47 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2014-05-19 15:27:47 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2014-05-19 15:27:47 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2014-05-19 15:27:47 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2014-05-19 15:27:47 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2014-05-19 15:27:48 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2014-05-19 15:27:48 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2014-05-19 15:27:48 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2014-05-19 15:27:48 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2014-05-19 15:27:48 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2014-05-19 15:27:48 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2014-05-19 15:27:48 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2014-05-19 15:27:48 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2014-05-19 15:27:48 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2014-05-19 15:27:49 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2014-05-19 15:27:49 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2014-05-19 15:27:49 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2014-05-19 15:27:49 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2014-05-19 15:27:49 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2014-05-19 15:27:49 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2014-05-19 15:27:49 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2014-05-19 15:27:49 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2014-05-19 15:27:49 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2014-05-19 15:27:50 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2014-05-19 15:27:50 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2014-05-19 15:27:50 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2014-05-19 15:27:50 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2014-05-19 15:27:50 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2014-05-19 15:27:50 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2014-05-19 15:27:50 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2014-05-19 15:27:50 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2014-05-19 15:27:50 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2014-05-19 15:27:51 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2014-05-19 15:27:51 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2014-05-19 15:27:51 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2014-05-19 15:27:51 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2014-05-19 15:27:51 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2014-05-19 15:27:51 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2014-05-19 15:27:51 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2014-05-19 15:27:51 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2014-05-19 15:27:51 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2014-05-19 15:27:52 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2014-05-19 15:27:52 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2014-05-19 15:27:52 [INFO    ] Setting Beam 4 xbars
2014-05-19 15:27:52 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2014-05-19 15:27:52 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 15:27:52 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 15:27:52 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 15:27:52 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 15:27:52 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2014-05-19 15:27:52 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2014-05-19 15:27:52 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2014-05-19 15:27:53 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2014-05-19 15:27:53 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2014-05-19 15:27:53 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2014-05-19 15:27:53 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2014-05-19 15:27:53 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2014-05-19 15:27:53 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2014-05-19 15:27:53 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2014-05-19 15:27:53 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2014-05-19 15:27:53 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2014-05-19 15:27:54 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2014-05-19 15:27:54 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2014-05-19 15:27:54 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2014-05-19 15:27:54 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2014-05-19 15:27:54 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2014-05-19 15:27:54 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2014-05-19 15:27:54 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2014-05-19 15:27:54 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2014-05-19 15:27:54 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2014-05-19 15:27:55 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2014-05-19 15:27:55 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2014-05-19 15:27:55 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2014-05-19 15:27:55 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2014-05-19 15:27:55 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2014-05-19 15:27:55 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2014-05-19 15:27:55 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2014-05-19 15:27:55 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2014-05-19 15:27:55 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2014-05-19 15:27:56 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2014-05-19 15:27:56 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2014-05-19 15:27:56 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2014-05-19 15:27:56 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2014-05-19 15:27:56 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2014-05-19 15:27:56 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2014-05-19 15:27:56 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2014-05-19 15:27:56 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2014-05-19 15:27:56 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2014-05-19 15:27:56 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2014-05-19 15:27:57 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2014-05-19 15:27:57 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2014-05-19 15:27:57 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2014-05-19 15:27:57 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2014-05-19 15:27:57 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2014-05-19 15:27:57 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2014-05-19 15:27:57 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2014-05-19 15:27:57 [INFO    ] All xbars set
2014-05-19 15:27:57 [INFO    ] Setting adder chain routing for all beams
2014-05-19 15:28:02 [INFO    ] Setting beam ids
2014-05-19 15:28:12 [INFO    ] Starting beam delay calibration
2014-05-19 15:28:24 [INFO    ] Beam 1 latency is 5730
2014-05-19 15:28:24 [INFO    ] Beam 2 latency is 6722
2014-05-19 15:28:24 [INFO    ] Beam 3 latency is 6710
2014-05-19 15:28:24 [INFO    ] Beam 4 latency is 5728
2014-05-19 15:28:24 [INFO    ] Stop calibration
2014-05-19 15:28:32 [ERROR   ] Beam delay calibration failed
2014-05-19 15:28:32 [CRITICAL] Calibration failed
2014-05-19 15:28:32 [DEBUG   ] spawn_process: at 2014-05-19 15:28:32 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2014-05-19 15:28:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2014-05-19 15:28:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2014-05-19 15:28:33 [DEBUG   ] spawn_process: at 2014-05-19 15:28:33 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2014-05-19 15:28:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2014-05-19 15:28:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2014-05-19 15:28:33 [DEBUG   ] spawn_process: at 2014-05-19 15:28:33 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2014-05-19 15:28:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2014-05-19 15:28:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2014-05-19 15:28:33 [DEBUG   ] spawn_process: at 2014-05-19 15:28:33 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2014-05-19 15:28:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2014-05-19 15:28:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2014-05-19 15:28:33 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -4 msec
2014-05-19 15:28:33 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 343 ms (1400513313.347 s) with PID 24060 and time 1400513314
2014-05-19 15:28:33 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -4 msec
2014-05-19 15:28:33 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 434 ms (1400513313.438 s) with PID 24063 and time 1400513314
2014-05-19 15:28:34 [DEBUG   ] Finished updating ant stats in 0.812 seconds
2014-05-19 15:28:34 [INFO    ] Started antennas statistics background thread
2014-05-19 15:28:34 [INFO    ] Started board flags background thread
2014-05-19 15:28:34 [INFO    ] Started BAM background thread
2014-05-19 15:28:34 [INFO    ] Finished the INI process (ref. #193788) in 107.815 s
2014-05-19 15:28:35 [DEBUG   ] Finished updating board flags in 1.014 seconds
2014-05-19 15:28:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:29:14 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:29:34 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:29:35 [DEBUG   ] Finished updating board flags in 1.451 seconds
2014-05-19 15:29:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:30:08 [DEBUG   ] Got command RPT from MCS: ref #193806
2014-05-19 15:30:08 [DEBUG   ] summary = ERROR
2014-05-19 15:30:08 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193806  13 56796 55808086 A  ERRORERROR'
2014-05-19 15:30:14 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:30:15 [DEBUG   ] Got command INI from MCS: ref #193807
2014-05-19 15:30:15 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2014-05-19 15:30:15 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   193807   8 56796 55815667 ABOOTING'
2014-05-19 15:30:36 [INFO    ] -------------------
2014-05-19 15:30:36 [INFO    ]  LWA board status
2014-05-19 15:30:36 [INFO    ] -------------------
2014-05-19 15:30:36 [INFO    ] DP 1: 2 lwa27
2014-05-19 15:30:36 [INFO    ] DP 1: 3 lwa24
2014-05-19 15:30:36 [INFO    ] DP 1: 4 lwa23
2014-05-19 15:30:36 [INFO    ] DP 1: 5 lwa18
2014-05-19 15:30:36 [INFO    ] DP 1: 6 lwa10
2014-05-19 15:30:36 [INFO    ] DP 1: 7 lwa32
2014-05-19 15:30:36 [INFO    ] DP 1: 8 lwa11
2014-05-19 15:30:36 [INFO    ] DP 1: 9 lwa30
2014-05-19 15:30:36 [INFO    ] DP 1: 10 lwa12
2014-05-19 15:30:36 [INFO    ] DP 1: 11 lwa8
2014-05-19 15:30:36 [INFO    ] DP 1: 12 lwa13
2014-05-19 15:30:36 [INFO    ] DP 1: 13 lwa26
2014-05-19 15:30:36 [INFO    ] DP 1: 14 lwa14
2014-05-19 15:30:36 [INFO    ] DP 1: 16 lwa22
2014-05-19 15:30:36 [INFO    ] DP 1: 17 lwa20
2014-05-19 15:30:36 [INFO    ] DP 1: 18 lwa1
2014-05-19 15:30:36 [INFO    ] DP 1: 19 lwa5
2014-05-19 15:30:36 [INFO    ] DP 1: 20 lwa25
2014-05-19 15:30:36 [INFO    ] DP 1: 21 lwa35
2014-05-19 15:30:36 [INFO    ] DP 1: 22 lwa19
2014-05-19 15:30:36 [INFO    ] DP 1: 23 lwa21
2014-05-19 15:30:36 [INFO    ] DP 1: 24 lwa6
2014-05-19 15:30:36 [INFO    ] DP 1: 25 lwa7
2014-05-19 15:30:36 [INFO    ] DP 1: 26 lwa33
2014-05-19 15:30:36 [INFO    ] DP 1: 27 lwa15
2014-05-19 15:30:36 [INFO    ] DP 1: 28 lwa28
2014-05-19 15:30:36 [INFO    ] DP 2: 1 lwa16
2014-05-19 15:30:36 [INFO    ] DP 2: 15 lwa34
2014-05-19 15:30:36 [INFO    ] Total of 28 boards
2014-05-19 15:30:36 [INFO    ] Total of 260 stands
2014-05-19 15:30:36 [INFO    ] -------------------
2014-05-19 15:30:36 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:37 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:38 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:38 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:38 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:38 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:39 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:39 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:39 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:39 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:40 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:40 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:40 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:40 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:41 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:41 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:41 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:41 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:41 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:42 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:43 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24660
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24661
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24662
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24663
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24664
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24665
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24666
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24667
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24668
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24669
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24670
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24671
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24672
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24673
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24674
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24675
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24676
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24677
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24678
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24679
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24680
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24681
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24682
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24683
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24684
2014-05-19 15:30:43 [DEBUG   ] 2014-05-19 15:30:43 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 24685
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24660 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24661 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24662 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24663 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24664 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24665 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24666 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24667 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24668 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24669 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24670 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24671 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24672 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24673 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24674 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24675 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24676 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24677 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24678 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24679 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24680 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24681 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24682 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24683 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24684 exited with status 0
2014-05-19 15:30:55 [DEBUG   ] 2014-05-19 15:30:55 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 24685 exited with status 0
2014-05-19 15:30:55 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:55 [DEBUG   ] spawn_process: at 2014-05-19 15:30:55 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:56 [DEBUG   ] spawn_process: at 2014-05-19 15:30:56 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:56 [DEBUG   ] spawn_process: at 2014-05-19 15:30:56 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:56 [DEBUG   ] spawn_process: at 2014-05-19 15:30:56 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:56 [DEBUG   ] spawn_process: at 2014-05-19 15:30:56 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:56 [DEBUG   ] spawn_process: at 2014-05-19 15:30:56 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:56 [DEBUG   ] spawn_process: at 2014-05-19 15:30:56 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:56 [DEBUG   ] spawn_process: at 2014-05-19 15:30:56 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:56 [DEBUG   ] spawn_process: at 2014-05-19 15:30:56 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:56 [DEBUG   ] spawn_process: at 2014-05-19 15:30:56 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:56 [DEBUG   ] spawn_process: at 2014-05-19 15:30:56 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:56 [DEBUG   ] spawn_process: at 2014-05-19 15:30:56 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2014-05-19 15:30:56 [DEBUG   ] 2014-05-19 15:30:56 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 24712
2014-05-19 15:30:56 [DEBUG   ] 2014-05-19 15:30:56 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 24713
2014-05-19 15:31:01 [DEBUG   ] 2014-05-19 15:31:01 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 24712 exited with status 0
2014-05-19 15:31:01 [DEBUG   ] 2014-05-19 15:31:01 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 24713 exited with status 0
2014-05-19 15:31:01 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2014-05-19 15:31:01 [DEBUG   ] spawn_process: at 2014-05-19 15:31:01 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2014-05-19 15:31:01 [DEBUG   ] spawn_process: at 2014-05-19 15:31:01 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2014-05-19 15:31:01 [INFO    ] Setting Beam 1 xbars
2014-05-19 15:31:01 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2014-05-19 15:31:01 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2014-05-19 15:31:01 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2014-05-19 15:31:01 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2014-05-19 15:31:02 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2014-05-19 15:31:02 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2014-05-19 15:31:02 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2014-05-19 15:31:02 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2014-05-19 15:31:02 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2014-05-19 15:31:02 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2014-05-19 15:31:02 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2014-05-19 15:31:02 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2014-05-19 15:31:02 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2014-05-19 15:31:03 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2014-05-19 15:31:03 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2014-05-19 15:31:03 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2014-05-19 15:31:03 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2014-05-19 15:31:03 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2014-05-19 15:31:03 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2014-05-19 15:31:03 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2014-05-19 15:31:03 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2014-05-19 15:31:03 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2014-05-19 15:31:04 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2014-05-19 15:31:04 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2014-05-19 15:31:04 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2014-05-19 15:31:04 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2014-05-19 15:31:04 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2014-05-19 15:31:04 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2014-05-19 15:31:04 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2014-05-19 15:31:04 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2014-05-19 15:31:04 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2014-05-19 15:31:04 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2014-05-19 15:31:05 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2014-05-19 15:31:05 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2014-05-19 15:31:05 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2014-05-19 15:31:05 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2014-05-19 15:31:05 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2014-05-19 15:31:05 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2014-05-19 15:31:05 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2014-05-19 15:31:05 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2014-05-19 15:31:05 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2014-05-19 15:31:06 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2014-05-19 15:31:06 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2014-05-19 15:31:06 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2014-05-19 15:31:06 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2014-05-19 15:31:06 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2014-05-19 15:31:06 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2014-05-19 15:31:06 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2014-05-19 15:31:06 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2014-05-19 15:31:06 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2014-05-19 15:31:07 [INFO    ] Setting Beam 2 xbars
2014-05-19 15:31:07 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2014-05-19 15:31:07 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2014-05-19 15:31:07 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2014-05-19 15:31:07 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2014-05-19 15:31:07 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2014-05-19 15:31:07 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2014-05-19 15:31:07 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2014-05-19 15:31:07 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2014-05-19 15:31:07 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2014-05-19 15:31:08 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2014-05-19 15:31:08 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2014-05-19 15:31:08 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2014-05-19 15:31:08 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2014-05-19 15:31:08 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2014-05-19 15:31:08 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2014-05-19 15:31:08 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2014-05-19 15:31:08 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2014-05-19 15:31:08 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2014-05-19 15:31:08 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2014-05-19 15:31:09 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2014-05-19 15:31:09 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2014-05-19 15:31:09 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2014-05-19 15:31:09 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2014-05-19 15:31:09 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2014-05-19 15:31:09 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2014-05-19 15:31:09 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2014-05-19 15:31:09 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2014-05-19 15:31:09 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2014-05-19 15:31:10 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2014-05-19 15:31:10 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2014-05-19 15:31:10 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2014-05-19 15:31:10 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2014-05-19 15:31:10 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2014-05-19 15:31:10 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2014-05-19 15:31:10 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2014-05-19 15:31:10 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2014-05-19 15:31:10 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2014-05-19 15:31:11 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2014-05-19 15:31:11 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2014-05-19 15:31:11 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2014-05-19 15:31:11 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2014-05-19 15:31:11 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2014-05-19 15:31:11 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2014-05-19 15:31:11 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2014-05-19 15:31:11 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2014-05-19 15:31:11 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2014-05-19 15:31:11 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2014-05-19 15:31:12 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2014-05-19 15:31:12 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2014-05-19 15:31:12 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2014-05-19 15:31:12 [INFO    ] Setting Beam 3 xbars
2014-05-19 15:31:12 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2014-05-19 15:31:12 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2014-05-19 15:31:12 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2014-05-19 15:31:12 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2014-05-19 15:31:12 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2014-05-19 15:31:12 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2014-05-19 15:31:13 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2014-05-19 15:31:13 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2014-05-19 15:31:13 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2014-05-19 15:31:13 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2014-05-19 15:31:13 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2014-05-19 15:31:13 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2014-05-19 15:31:13 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2014-05-19 15:31:13 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2014-05-19 15:31:13 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2014-05-19 15:31:14 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2014-05-19 15:31:14 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2014-05-19 15:31:14 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2014-05-19 15:31:14 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2014-05-19 15:31:14 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2014-05-19 15:31:14 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2014-05-19 15:31:14 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2014-05-19 15:31:14 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2014-05-19 15:31:14 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2014-05-19 15:31:15 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2014-05-19 15:31:15 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2014-05-19 15:31:15 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2014-05-19 15:31:15 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2014-05-19 15:31:15 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2014-05-19 15:31:15 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2014-05-19 15:31:15 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2014-05-19 15:31:15 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2014-05-19 15:31:15 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2014-05-19 15:31:15 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2014-05-19 15:31:16 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2014-05-19 15:31:16 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2014-05-19 15:31:16 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2014-05-19 15:31:16 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2014-05-19 15:31:16 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2014-05-19 15:31:16 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2014-05-19 15:31:16 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2014-05-19 15:31:16 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2014-05-19 15:31:16 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2014-05-19 15:31:17 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2014-05-19 15:31:17 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2014-05-19 15:31:17 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2014-05-19 15:31:17 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2014-05-19 15:31:17 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2014-05-19 15:31:17 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2014-05-19 15:31:17 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2014-05-19 15:31:17 [INFO    ] Setting Beam 4 xbars
2014-05-19 15:31:17 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2014-05-19 15:31:17 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 15:31:18 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 15:31:18 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 15:31:18 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 15:31:18 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2014-05-19 15:31:18 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2014-05-19 15:31:18 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2014-05-19 15:31:18 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2014-05-19 15:31:18 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2014-05-19 15:31:18 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2014-05-19 15:31:19 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2014-05-19 15:31:19 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2014-05-19 15:31:19 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2014-05-19 15:31:19 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2014-05-19 15:31:19 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2014-05-19 15:31:19 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2014-05-19 15:31:19 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2014-05-19 15:31:19 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2014-05-19 15:31:19 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2014-05-19 15:31:19 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2014-05-19 15:31:20 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2014-05-19 15:31:20 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2014-05-19 15:31:20 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2014-05-19 15:31:20 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2014-05-19 15:31:20 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2014-05-19 15:31:20 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2014-05-19 15:31:20 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2014-05-19 15:31:20 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2014-05-19 15:31:20 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2014-05-19 15:31:21 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2014-05-19 15:31:21 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2014-05-19 15:31:21 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2014-05-19 15:31:21 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2014-05-19 15:31:21 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2014-05-19 15:31:21 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2014-05-19 15:31:21 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2014-05-19 15:31:21 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2014-05-19 15:31:21 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2014-05-19 15:31:22 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2014-05-19 15:31:22 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2014-05-19 15:31:22 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2014-05-19 15:31:22 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2014-05-19 15:31:22 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2014-05-19 15:31:22 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2014-05-19 15:31:22 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2014-05-19 15:31:22 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2014-05-19 15:31:22 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2014-05-19 15:31:23 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2014-05-19 15:31:23 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2014-05-19 15:31:23 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2014-05-19 15:31:23 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2014-05-19 15:31:23 [INFO    ] All xbars set
2014-05-19 15:31:23 [INFO    ] Setting adder chain routing for all beams
2014-05-19 15:31:28 [INFO    ] Setting beam ids
2014-05-19 15:31:38 [INFO    ] Starting beam delay calibration
2014-05-19 15:31:50 [INFO    ] Beam 1 latency is 5747
2014-05-19 15:31:50 [INFO    ] Beam 2 latency is 5762
2014-05-19 15:31:50 [INFO    ] Beam 3 latency is 5728
2014-05-19 15:31:50 [INFO    ] Beam 4 latency is 5734
2014-05-19 15:31:50 [INFO    ] Stop calibration
2014-05-19 15:31:58 [INFO    ] Beam delay calibration successful
2014-05-19 15:31:58 [INFO    ] Calibration passed
2014-05-19 15:31:58 [DEBUG   ] spawn_process: at 2014-05-19 15:31:58 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2014-05-19 15:31:58 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2014-05-19 15:31:58 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2014-05-19 15:31:58 [DEBUG   ] spawn_process: at 2014-05-19 15:31:58 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2014-05-19 15:31:58 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2014-05-19 15:31:58 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2014-05-19 15:31:58 [DEBUG   ] spawn_process: at 2014-05-19 15:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2014-05-19 15:31:58 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2014-05-19 15:31:58 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2014-05-19 15:31:58 [DEBUG   ] spawn_process: at 2014-05-19 15:31:58 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2014-05-19 15:31:58 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2014-05-19 15:31:58 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2014-05-19 15:31:58 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -3 msec
2014-05-19 15:31:59 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 265 ms (1400513519.269 s) with PID 26282 and time 1400513520
2014-05-19 15:31:59 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -4 msec
2014-05-19 15:31:59 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 355 ms (1400513519.359 s) with PID 26285 and time 1400513520
2014-05-19 15:32:00 [DEBUG   ] Finished updating ant stats in 0.890 seconds
2014-05-19 15:32:00 [INFO    ] Started antennas statistics background thread
2014-05-19 15:32:00 [INFO    ] Started board flags background thread
2014-05-19 15:32:00 [INFO    ] Started BAM background thread
2014-05-19 15:32:00 [INFO    ] Finished the INI process (ref. #193807) in 104.818 s
2014-05-19 15:32:01 [DEBUG   ] Finished updating board flags in 1.072 seconds
2014-05-19 15:32:06 [DEBUG   ] Got command RPT from MCS: ref #193808
2014-05-19 15:32:06 [DEBUG   ] summary = NORMAL
2014-05-19 15:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193808  14 56796 55926750 A NORMALNORMAL'
2014-05-19 15:32:06 [DEBUG   ] Got command RPT from MCS: ref #193809
2014-05-19 15:32:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 15:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193809  48 56796 55926751 A NORMALSystem calibrated and operating normally'
2014-05-19 15:32:20 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:32:40 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:33:00 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:33:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 15:33:01 [DEBUG   ] Finished updating board flags in 1.007 seconds
2014-05-19 15:33:20 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:33:27 [DEBUG   ] Got command RPT from MCS: ref #193825
2014-05-19 15:33:27 [DEBUG   ] summary = NORMAL
2014-05-19 15:33:27 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193825  14 56796 56007526 A NORMALNORMAL'
2014-05-19 15:33:40 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:34:00 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:34:01 [DEBUG   ] Finished updating board flags in 1.057 seconds
2014-05-19 15:34:20 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 15:34:40 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 15:35:00 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:35:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:35:02 [DEBUG   ] Finished updating board flags in 2.007 seconds
2014-05-19 15:35:20 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:35:40 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:36:00 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:36:02 [DEBUG   ] Finished updating board flags in 1.954 seconds
2014-05-19 15:36:20 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:36:40 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:37:00 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:37:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 15:37:02 [DEBUG   ] Finished updating board flags in 1.793 seconds
2014-05-19 15:37:06 [DEBUG   ] Got command RPT from MCS: ref #193826
2014-05-19 15:37:06 [DEBUG   ] summary = NORMAL
2014-05-19 15:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193826  14 56796 56226746 A NORMALNORMAL'
2014-05-19 15:37:06 [DEBUG   ] Got command RPT from MCS: ref #193827
2014-05-19 15:37:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 15:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193827  48 56796 56226747 A NORMALSystem calibrated and operating normally'
2014-05-19 15:37:20 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:37:40 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-19 15:38:00 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2014-05-19 15:38:01 [DEBUG   ] Finished updating board flags in 1.041 seconds
2014-05-19 15:38:20 [DEBUG   ] Finished updating ant stats in 0.403 seconds
2014-05-19 15:38:40 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2014-05-19 15:39:00 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2014-05-19 15:39:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-19 15:39:02 [DEBUG   ] Finished updating board flags in 1.574 seconds
2014-05-19 15:39:20 [DEBUG   ] Finished updating ant stats in 0.402 seconds
2014-05-19 15:39:40 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2014-05-19 15:39:58 [DEBUG   ] Got command DRX from MCS: ref #193844
2014-05-19 15:39:58 [INFO    ] sending drx command to lwa34
2014-05-19 15:39:58 [DEBUG   ] run_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa34', '/lwa/software/drx 5 1 45450000.0 7 9 0']
2014-05-19 15:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-19 15:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   193844   8 56796 56398006 A NORMAL'
2014-05-19 15:39:58 [INFO    ] Finished starting DRX
2014-05-19 15:39:58 [DEBUG   ] Got command DRX from MCS: ref #193845
2014-05-19 15:39:58 [INFO    ] sending drx command to lwa34
2014-05-19 15:39:58 [DEBUG   ] run_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa34', '/lwa/software/drx 5 2 65050000.0 7 9 0']
2014-05-19 15:39:58 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-19 15:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   193845   8 56796 56398202 A NORMAL'
2014-05-19 15:39:58 [DEBUG   ] Got command BAM from MCS: ref #193846
2014-05-19 15:39:58 [INFO    ] Finished processing BAM request
2014-05-19 15:39:58 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 15:39:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193846   8 56796 56398206 A NORMAL'
2014-05-19 15:39:58 [INFO    ] Finished starting DRX
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2014-05-19 15:39:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2014-05-19 15:39:58 [DEBUG   ] spawn_process: at 2014-05-19 15:39:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2014-05-19 15:40:01 [DEBUG   ] Finished updating ant stats in 1.233 seconds
2014-05-19 15:40:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-19 15:40:02 [DEBUG   ] Finished updating board flags in 1.380 seconds
2014-05-19 15:40:21 [DEBUG   ] Finished updating ant stats in 1.171 seconds
2014-05-19 15:40:41 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 15:41:01 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 15:41:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:41:02 [DEBUG   ] Finished updating board flags in 1.210 seconds
2014-05-19 15:41:21 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-19 15:41:23 [DEBUG   ] Got command BAM from MCS: ref #193848
2014-05-19 15:41:23 [INFO    ] Finished processing BAM request
2014-05-19 15:41:23 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 15:41:23 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193848   8 56796 56483026 A NORMAL'
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2014-05-19 15:41:23 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2014-05-19 15:41:23 [DEBUG   ] spawn_process: at 2014-05-19 15:41:23 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2014-05-19 15:41:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 15:42:01 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 15:42:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 15:42:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:42:02 [DEBUG   ] Finished updating board flags in 1.188 seconds
2014-05-19 15:42:06 [DEBUG   ] Got command RPT from MCS: ref #193849
2014-05-19 15:42:06 [DEBUG   ] summary = NORMAL
2014-05-19 15:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193849  14 56796 56526750 A NORMALNORMAL'
2014-05-19 15:42:06 [DEBUG   ] Got command RPT from MCS: ref #193850
2014-05-19 15:42:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 15:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193850  48 56796 56526751 A NORMALSystem calibrated and operating normally'
2014-05-19 15:42:21 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:42:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:42:48 [DEBUG   ] Got command BAM from MCS: ref #193866
2014-05-19 15:42:48 [INFO    ] Finished processing BAM request
2014-05-19 15:42:48 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 15:42:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193866   8 56796 56568034 A NORMAL'
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2014-05-19 15:42:48 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2014-05-19 15:42:48 [DEBUG   ] spawn_process: at 2014-05-19 15:42:48 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2014-05-19 15:43:01 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:43:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:43:02 [DEBUG   ] Finished updating board flags in 1.012 seconds
2014-05-19 15:43:21 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:43:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:44:01 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:44:02 [DEBUG   ] Finished updating board flags in 1.021 seconds
2014-05-19 15:44:13 [DEBUG   ] Got command BAM from MCS: ref #193867
2014-05-19 15:44:13 [INFO    ] Finished processing BAM request
2014-05-19 15:44:13 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 15:44:13 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193867   8 56796 56653034 A NORMAL'
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2014-05-19 15:44:13 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2014-05-19 15:44:13 [DEBUG   ] spawn_process: at 2014-05-19 15:44:13 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2014-05-19 15:44:21 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:44:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:45:01 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:45:03 [DEBUG   ] Finished updating board flags in 2.079 seconds
2014-05-19 15:45:21 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:45:38 [DEBUG   ] Got command BAM from MCS: ref #193868
2014-05-19 15:45:38 [INFO    ] Finished processing BAM request
2014-05-19 15:45:38 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 15:45:38 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193868   8 56796 56738026 A NORMAL'
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1'] to hostname lwa27
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp1 1']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2'] to hostname lwa24
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp1 2']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3'] to hostname lwa23
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp1 3']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4'] to hostname lwa18
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp1 4']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5'] to hostname lwa10
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp1 5']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6'] to hostname lwa32
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp1 6']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7'] to hostname lwa11
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp1 7']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8'] to hostname lwa30
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp1 8']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9'] to hostname lwa12
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp1 9']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10'] to hostname lwa8
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp1 10']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11'] to hostname lwa13
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp1 11']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12'] to hostname lwa26
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp1 12']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13'] to hostname lwa14
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp1 13']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14'] to hostname lwa22
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp1 14']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15'] to hostname lwa20
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp1 15']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16'] to hostname lwa1
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp1 16']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17'] to hostname lwa5
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp1 17']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18'] to hostname lwa25
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp1 18']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19'] to hostname lwa35
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp1 19']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20'] to hostname lwa19
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp1 20']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21'] to hostname lwa21
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp1 21']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22'] to hostname lwa6
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp1 22']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23'] to hostname lwa7
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp1 23']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24'] to hostname lwa33
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp1 24']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25'] to hostname lwa15
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp1 25']
2014-05-19 15:45:38 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26'] to hostname lwa28
2014-05-19 15:45:38 [DEBUG   ] spawn_process: at 2014-05-19 15:45:38 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp1 26']
2014-05-19 15:45:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:46:01 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:46:02 [DEBUG   ] Finished updating board flags in 1.349 seconds
2014-05-19 15:46:21 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:46:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:47:01 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:47:03 [DEBUG   ] Got command BAM from MCS: ref #193869
2014-05-19 15:47:03 [INFO    ] Finished processing BAM request
2014-05-19 15:47:03 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 15:47:03 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193869   8 56796 56823034 A NORMAL'
2014-05-19 15:47:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:47:03 [DEBUG   ] Finished updating board flags in 1.769 seconds
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2014-05-19 15:47:03 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2014-05-19 15:47:03 [DEBUG   ] spawn_process: at 2014-05-19 15:47:03 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2014-05-19 15:47:06 [DEBUG   ] Got command RPT from MCS: ref #193870
2014-05-19 15:47:06 [DEBUG   ] summary = NORMAL
2014-05-19 15:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193870  14 56796 56826746 A NORMALNORMAL'
2014-05-19 15:47:06 [DEBUG   ] Got command RPT from MCS: ref #193871
2014-05-19 15:47:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 15:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193871  48 56796 56826747 A NORMALSystem calibrated and operating normally'
2014-05-19 15:47:21 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:47:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:48:01 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:48:02 [DEBUG   ] Finished updating board flags in 1.039 seconds
2014-05-19 15:48:21 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:48:28 [DEBUG   ] Got command BAM from MCS: ref #193887
2014-05-19 15:48:28 [INFO    ] Finished processing BAM request
2014-05-19 15:48:28 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 15:48:28 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193887   8 56796 56908030 A NORMAL'
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2014-05-19 15:48:28 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2014-05-19 15:48:28 [DEBUG   ] spawn_process: at 2014-05-19 15:48:28 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2014-05-19 15:48:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:49:01 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:49:02 [DEBUG   ] Finished updating board flags in 1.013 seconds
2014-05-19 15:49:21 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:49:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:49:53 [DEBUG   ] Got command BAM from MCS: ref #193888
2014-05-19 15:49:53 [INFO    ] Finished processing BAM request
2014-05-19 15:49:53 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 15:49:53 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193888   8 56796 56993030 A NORMAL'
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1'] to hostname lwa27
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp3 1']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2'] to hostname lwa24
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp3 2']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3'] to hostname lwa23
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp3 3']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4'] to hostname lwa18
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp3 4']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5'] to hostname lwa10
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp3 5']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6'] to hostname lwa32
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp3 6']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7'] to hostname lwa11
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp3 7']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8'] to hostname lwa30
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp3 8']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9'] to hostname lwa12
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp3 9']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10'] to hostname lwa8
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp3 10']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11'] to hostname lwa13
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp3 11']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12'] to hostname lwa26
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp3 12']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13'] to hostname lwa14
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp3 13']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14'] to hostname lwa22
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp3 14']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15'] to hostname lwa20
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp3 15']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16'] to hostname lwa1
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp3 16']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17'] to hostname lwa5
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp3 17']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18'] to hostname lwa25
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp3 18']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19'] to hostname lwa35
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp3 19']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20'] to hostname lwa19
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp3 20']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21'] to hostname lwa21
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp3 21']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22'] to hostname lwa6
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp3 22']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23'] to hostname lwa7
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp3 23']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24'] to hostname lwa33
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp3 24']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25'] to hostname lwa15
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp3 25']
2014-05-19 15:49:53 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26'] to hostname lwa28
2014-05-19 15:49:53 [DEBUG   ] spawn_process: at 2014-05-19 15:49:53 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp3 26']
2014-05-19 15:50:01 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 15:50:03 [DEBUG   ] Finished updating board flags in 1.607 seconds
2014-05-19 15:50:21 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-19 15:50:41 [DEBUG   ] Finished updating ant stats in 0.398 seconds
2014-05-19 15:51:02 [DEBUG   ] Finished updating ant stats in 1.237 seconds
2014-05-19 15:51:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:51:03 [DEBUG   ] Finished updating board flags in 1.546 seconds
2014-05-19 15:51:18 [DEBUG   ] Got command BAM from MCS: ref #193889
2014-05-19 15:51:18 [INFO    ] Finished processing BAM request
2014-05-19 15:51:18 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 15:51:18 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193889   8 56796 57078026 A NORMAL'
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1'] to hostname lwa27
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp5 1']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2'] to hostname lwa24
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp5 2']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3'] to hostname lwa23
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp5 3']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4'] to hostname lwa18
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp5 4']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5'] to hostname lwa10
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp5 5']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6'] to hostname lwa32
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp5 6']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7'] to hostname lwa11
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp5 7']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8'] to hostname lwa30
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp5 8']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9'] to hostname lwa12
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp5 9']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10'] to hostname lwa8
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp5 10']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11'] to hostname lwa13
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp5 11']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12'] to hostname lwa26
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp5 12']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13'] to hostname lwa14
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp5 13']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14'] to hostname lwa22
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp5 14']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15'] to hostname lwa20
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp5 15']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16'] to hostname lwa1
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp5 16']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17'] to hostname lwa5
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp5 17']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18'] to hostname lwa25
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp5 18']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19'] to hostname lwa35
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp5 19']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20'] to hostname lwa19
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp5 20']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21'] to hostname lwa21
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp5 21']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22'] to hostname lwa6
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp5 22']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23'] to hostname lwa7
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp5 23']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24'] to hostname lwa33
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp5 24']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25'] to hostname lwa15
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp5 25']
2014-05-19 15:51:18 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26'] to hostname lwa28
2014-05-19 15:51:18 [DEBUG   ] spawn_process: at 2014-05-19 15:51:18 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp5 26']
2014-05-19 15:51:22 [DEBUG   ] Finished updating ant stats in 1.167 seconds
2014-05-19 15:51:42 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2014-05-19 15:52:02 [DEBUG   ] Finished updating ant stats in 0.832 seconds
2014-05-19 15:52:03 [DEBUG   ] Finished updating board flags in 1.387 seconds
2014-05-19 15:52:06 [DEBUG   ] Got command RPT from MCS: ref #193890
2014-05-19 15:52:06 [DEBUG   ] summary = NORMAL
2014-05-19 15:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193890  14 56796 57126750 A NORMALNORMAL'
2014-05-19 15:52:06 [DEBUG   ] Got command RPT from MCS: ref #193891
2014-05-19 15:52:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 15:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193891  48 56796 57126751 A NORMALSystem calibrated and operating normally'
2014-05-19 15:52:22 [DEBUG   ] Finished updating ant stats in 0.771 seconds
2014-05-19 15:52:42 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:52:44 [DEBUG   ] Got command BAM from MCS: ref #193907
2014-05-19 15:52:44 [INFO    ] Finished processing BAM request
2014-05-19 15:52:44 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 15:52:44 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193907   8 56796 57164026 A NORMAL'
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2014-05-19 15:52:44 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2014-05-19 15:52:44 [DEBUG   ] spawn_process: at 2014-05-19 15:52:44 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2014-05-19 15:53:02 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:53:03 [DEBUG   ] Finished updating board flags in 1.208 seconds
2014-05-19 15:53:22 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:53:42 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:54:02 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:54:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 15:54:03 [DEBUG   ] Finished updating board flags in 1.201 seconds
2014-05-19 15:54:10 [DEBUG   ] Got command BAM from MCS: ref #193910
2014-05-19 15:54:10 [INFO    ] Finished processing BAM request
2014-05-19 15:54:10 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-19 15:54:10 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   193910   8 56796 57250030 A NORMAL'
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1'] to hostname lwa27
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp4 1']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2'] to hostname lwa24
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp4 2']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3'] to hostname lwa23
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp4 3']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4'] to hostname lwa18
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp4 4']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5'] to hostname lwa10
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp4 5']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6'] to hostname lwa32
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp4 6']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7'] to hostname lwa11
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp4 7']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8'] to hostname lwa30
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp4 8']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9'] to hostname lwa12
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp4 9']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10'] to hostname lwa8
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp4 10']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11'] to hostname lwa13
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp4 11']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12'] to hostname lwa26
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp4 12']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13'] to hostname lwa14
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp4 13']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14'] to hostname lwa22
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp4 14']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15'] to hostname lwa20
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp4 15']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16'] to hostname lwa1
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp4 16']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17'] to hostname lwa5
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp4 17']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18'] to hostname lwa25
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp4 18']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19'] to hostname lwa35
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp4 19']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20'] to hostname lwa19
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp4 20']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21'] to hostname lwa21
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp4 21']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22'] to hostname lwa6
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp4 22']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23'] to hostname lwa7
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp4 23']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24'] to hostname lwa33
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp4 24']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25'] to hostname lwa15
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp4 25']
2014-05-19 15:54:10 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26'] to hostname lwa28
2014-05-19 15:54:10 [DEBUG   ] spawn_process: at 2014-05-19 15:54:10 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp4 26']
2014-05-19 15:54:22 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:54:42 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:55:02 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:55:03 [DEBUG   ] Finished updating board flags in 1.207 seconds
2014-05-19 15:55:22 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 15:55:42 [DEBUG   ] Finished updating ant stats in 0.399 seconds
2014-05-19 15:56:03 [DEBUG   ] Finished updating ant stats in 1.230 seconds
2014-05-19 15:56:03 [DEBUG   ] Finished updating board flags in 1.246 seconds
2014-05-19 15:56:23 [DEBUG   ] Finished updating ant stats in 1.175 seconds
2014-05-19 15:56:43 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 15:56:50 [DEBUG   ] Got command INI from MCS: ref #193912
2014-05-19 15:56:50 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2014-05-19 15:56:50 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   193912   8 56796 57410031 ABOOTING'
2014-05-19 15:57:02 [DEBUG   ] Finished updating board flags in 0.275 seconds
2014-05-19 15:57:06 [DEBUG   ] Got command RPT from MCS: ref #193913
2014-05-19 15:57:06 [DEBUG   ] summary = BOOTING
2014-05-19 15:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193913  15 56796 57426750 ABOOTINGBOOTING'
2014-05-19 15:57:06 [DEBUG   ] Got command RPT from MCS: ref #193914
2014-05-19 15:57:06 [DEBUG   ] info = Running INI sequence
2014-05-19 15:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193914  28 56796 57426751 ABOOTINGRunning INI sequence'
2014-05-19 15:57:10 [INFO    ] -------------------
2014-05-19 15:57:10 [INFO    ]  LWA board status
2014-05-19 15:57:10 [INFO    ] -------------------
2014-05-19 15:57:10 [INFO    ] DP 1: 2 lwa27
2014-05-19 15:57:10 [INFO    ] DP 1: 3 lwa24
2014-05-19 15:57:10 [INFO    ] DP 1: 4 lwa23
2014-05-19 15:57:10 [INFO    ] DP 1: 5 lwa18
2014-05-19 15:57:10 [INFO    ] DP 1: 6 lwa10
2014-05-19 15:57:10 [INFO    ] DP 1: 7 lwa32
2014-05-19 15:57:10 [INFO    ] DP 1: 8 lwa11
2014-05-19 15:57:10 [INFO    ] DP 1: 9 lwa30
2014-05-19 15:57:10 [INFO    ] DP 1: 10 lwa12
2014-05-19 15:57:10 [INFO    ] DP 1: 11 lwa8
2014-05-19 15:57:10 [INFO    ] DP 1: 12 lwa13
2014-05-19 15:57:10 [INFO    ] DP 1: 13 lwa26
2014-05-19 15:57:10 [INFO    ] DP 1: 14 lwa14
2014-05-19 15:57:10 [INFO    ] DP 1: 16 lwa22
2014-05-19 15:57:10 [INFO    ] DP 1: 17 lwa20
2014-05-19 15:57:10 [INFO    ] DP 1: 18 lwa1
2014-05-19 15:57:10 [INFO    ] DP 1: 19 lwa5
2014-05-19 15:57:10 [INFO    ] DP 1: 20 lwa25
2014-05-19 15:57:10 [INFO    ] DP 1: 21 lwa35
2014-05-19 15:57:10 [INFO    ] DP 1: 22 lwa19
2014-05-19 15:57:10 [INFO    ] DP 1: 23 lwa21
2014-05-19 15:57:10 [INFO    ] DP 1: 24 lwa6
2014-05-19 15:57:10 [INFO    ] DP 1: 25 lwa7
2014-05-19 15:57:10 [INFO    ] DP 1: 26 lwa33
2014-05-19 15:57:10 [INFO    ] DP 1: 27 lwa15
2014-05-19 15:57:10 [INFO    ] DP 1: 28 lwa28
2014-05-19 15:57:10 [INFO    ] DP 2: 1 lwa16
2014-05-19 15:57:10 [INFO    ] DP 2: 15 lwa34
2014-05-19 15:57:10 [INFO    ] Total of 28 boards
2014-05-19 15:57:10 [INFO    ] Total of 260 stands
2014-05-19 15:57:10 [INFO    ] -------------------
2014-05-19 15:57:10 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:11 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:11 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:11 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:11 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:12 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:12 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:12 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:12 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:12 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:13 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:14 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:15 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:16 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:17 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:17 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29721
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29722
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29723
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29724
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29725
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29726
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29727
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29728
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29729
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29730
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29731
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29732
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29733
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29734
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29735
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29736
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29737
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29738
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29739
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29740
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29741
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29742
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29743
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29744
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29745
2014-05-19 15:57:17 [DEBUG   ] 2014-05-19 15:57:17 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 29746
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29721 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29722 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29723 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29724 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29725 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29726 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29727 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29728 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29729 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29730 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29731 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29732 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29733 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29734 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29735 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29736 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29737 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29738 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29739 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29740 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29741 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29742 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29743 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29744 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29745 exited with status 0
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 29746 exited with status 0
2014-05-19 15:57:29 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] spawn_process: at 2014-05-19 15:57:29 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 29773
2014-05-19 15:57:29 [DEBUG   ] 2014-05-19 15:57:29 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 29774
2014-05-19 15:57:35 [DEBUG   ] 2014-05-19 15:57:35 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 29773 exited with status 0
2014-05-19 15:57:35 [DEBUG   ] 2014-05-19 15:57:35 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 29774 exited with status 0
2014-05-19 15:57:35 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2014-05-19 15:57:35 [DEBUG   ] spawn_process: at 2014-05-19 15:57:35 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2014-05-19 15:57:35 [DEBUG   ] spawn_process: at 2014-05-19 15:57:35 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2014-05-19 15:57:35 [INFO    ] Setting Beam 1 xbars
2014-05-19 15:57:35 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2014-05-19 15:57:35 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2014-05-19 15:57:35 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2014-05-19 15:57:35 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2014-05-19 15:57:35 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2014-05-19 15:57:36 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2014-05-19 15:57:36 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2014-05-19 15:57:36 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2014-05-19 15:57:36 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2014-05-19 15:57:36 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2014-05-19 15:57:36 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2014-05-19 15:57:36 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2014-05-19 15:57:36 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2014-05-19 15:57:36 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2014-05-19 15:57:37 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2014-05-19 15:57:37 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2014-05-19 15:57:37 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2014-05-19 15:57:37 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2014-05-19 15:57:37 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2014-05-19 15:57:37 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2014-05-19 15:57:37 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2014-05-19 15:57:37 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2014-05-19 15:57:37 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2014-05-19 15:57:37 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2014-05-19 15:57:38 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2014-05-19 15:57:38 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2014-05-19 15:57:38 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2014-05-19 15:57:38 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2014-05-19 15:57:38 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2014-05-19 15:57:38 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2014-05-19 15:57:38 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2014-05-19 15:57:38 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2014-05-19 15:57:38 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2014-05-19 15:57:39 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2014-05-19 15:57:39 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2014-05-19 15:57:39 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2014-05-19 15:57:39 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2014-05-19 15:57:39 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2014-05-19 15:57:39 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2014-05-19 15:57:39 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2014-05-19 15:57:39 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2014-05-19 15:57:39 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2014-05-19 15:57:40 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2014-05-19 15:57:40 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2014-05-19 15:57:40 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2014-05-19 15:57:40 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2014-05-19 15:57:40 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2014-05-19 15:57:40 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2014-05-19 15:57:40 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2014-05-19 15:57:40 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2014-05-19 15:57:40 [INFO    ] Setting Beam 2 xbars
2014-05-19 15:57:40 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2014-05-19 15:57:41 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2014-05-19 15:57:41 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2014-05-19 15:57:41 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2014-05-19 15:57:41 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2014-05-19 15:57:41 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2014-05-19 15:57:41 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2014-05-19 15:57:41 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2014-05-19 15:57:41 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2014-05-19 15:57:41 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2014-05-19 15:57:41 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2014-05-19 15:57:42 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2014-05-19 15:57:42 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2014-05-19 15:57:42 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2014-05-19 15:57:42 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2014-05-19 15:57:42 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2014-05-19 15:57:42 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2014-05-19 15:57:42 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2014-05-19 15:57:42 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2014-05-19 15:57:42 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2014-05-19 15:57:43 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2014-05-19 15:57:43 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2014-05-19 15:57:43 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2014-05-19 15:57:43 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2014-05-19 15:57:43 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2014-05-19 15:57:43 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2014-05-19 15:57:43 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2014-05-19 15:57:43 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2014-05-19 15:57:44 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2014-05-19 15:57:44 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2014-05-19 15:57:44 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2014-05-19 15:57:44 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2014-05-19 15:57:44 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2014-05-19 15:57:44 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2014-05-19 15:57:44 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2014-05-19 15:57:44 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2014-05-19 15:57:44 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2014-05-19 15:57:45 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2014-05-19 15:57:45 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2014-05-19 15:57:45 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2014-05-19 15:57:45 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2014-05-19 15:57:45 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2014-05-19 15:57:45 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2014-05-19 15:57:45 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2014-05-19 15:57:45 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2014-05-19 15:57:45 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2014-05-19 15:57:46 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2014-05-19 15:57:46 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2014-05-19 15:57:46 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2014-05-19 15:57:46 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2014-05-19 15:57:46 [INFO    ] Setting Beam 3 xbars
2014-05-19 15:57:46 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2014-05-19 15:57:46 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2014-05-19 15:57:46 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2014-05-19 15:57:46 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2014-05-19 15:57:46 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2014-05-19 15:57:47 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2014-05-19 15:57:47 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2014-05-19 15:57:47 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2014-05-19 15:57:47 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2014-05-19 15:57:47 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2014-05-19 15:57:47 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2014-05-19 15:57:47 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2014-05-19 15:57:47 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2014-05-19 15:57:47 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2014-05-19 15:57:48 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2014-05-19 15:57:48 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2014-05-19 15:57:48 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2014-05-19 15:57:48 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2014-05-19 15:57:48 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2014-05-19 15:57:48 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2014-05-19 15:57:48 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2014-05-19 15:57:48 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2014-05-19 15:57:48 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2014-05-19 15:57:48 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2014-05-19 15:57:49 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2014-05-19 15:57:49 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2014-05-19 15:57:49 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2014-05-19 15:57:49 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2014-05-19 15:57:49 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2014-05-19 15:57:49 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2014-05-19 15:57:49 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2014-05-19 15:57:49 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2014-05-19 15:57:49 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2014-05-19 15:57:50 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2014-05-19 15:57:50 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2014-05-19 15:57:50 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2014-05-19 15:57:50 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2014-05-19 15:57:50 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2014-05-19 15:57:50 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2014-05-19 15:57:50 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2014-05-19 15:57:50 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2014-05-19 15:57:50 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2014-05-19 15:57:51 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2014-05-19 15:57:51 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2014-05-19 15:57:51 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2014-05-19 15:57:51 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2014-05-19 15:57:51 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2014-05-19 15:57:51 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2014-05-19 15:57:51 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2014-05-19 15:57:51 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2014-05-19 15:57:51 [INFO    ] Setting Beam 4 xbars
2014-05-19 15:57:51 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2014-05-19 15:57:52 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 15:57:52 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 15:57:52 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-19 15:57:52 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-19 15:57:52 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2014-05-19 15:57:52 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2014-05-19 15:57:52 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2014-05-19 15:57:52 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2014-05-19 15:57:52 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2014-05-19 15:57:52 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2014-05-19 15:57:53 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2014-05-19 15:57:53 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2014-05-19 15:57:53 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2014-05-19 15:57:53 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2014-05-19 15:57:53 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2014-05-19 15:57:53 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2014-05-19 15:57:53 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2014-05-19 15:57:53 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2014-05-19 15:57:53 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2014-05-19 15:57:54 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2014-05-19 15:57:54 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2014-05-19 15:57:54 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2014-05-19 15:57:54 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2014-05-19 15:57:54 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2014-05-19 15:57:54 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2014-05-19 15:57:54 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2014-05-19 15:57:54 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2014-05-19 15:57:54 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2014-05-19 15:57:55 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2014-05-19 15:57:55 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2014-05-19 15:57:55 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2014-05-19 15:57:55 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2014-05-19 15:57:55 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2014-05-19 15:57:55 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2014-05-19 15:57:55 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2014-05-19 15:57:55 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2014-05-19 15:57:55 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2014-05-19 15:57:55 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2014-05-19 15:57:56 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2014-05-19 15:57:56 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2014-05-19 15:57:56 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2014-05-19 15:57:56 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2014-05-19 15:57:56 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2014-05-19 15:57:56 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2014-05-19 15:57:56 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2014-05-19 15:57:56 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2014-05-19 15:57:56 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2014-05-19 15:57:57 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2014-05-19 15:57:57 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2014-05-19 15:57:57 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2014-05-19 15:57:57 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2014-05-19 15:57:57 [INFO    ] All xbars set
2014-05-19 15:57:57 [INFO    ] Setting adder chain routing for all beams
2014-05-19 15:58:02 [INFO    ] Setting beam ids
2014-05-19 15:58:12 [INFO    ] Starting beam delay calibration
2014-05-19 15:58:24 [INFO    ] Beam 1 latency is 5724
2014-05-19 15:58:24 [INFO    ] Beam 2 latency is 5753
2014-05-19 15:58:24 [INFO    ] Beam 3 latency is 5744
2014-05-19 15:58:24 [INFO    ] Beam 4 latency is 5738
2014-05-19 15:58:24 [INFO    ] Stop calibration
2014-05-19 15:58:32 [INFO    ] Beam delay calibration successful
2014-05-19 15:58:32 [INFO    ] Calibration passed
2014-05-19 15:58:32 [DEBUG   ] spawn_process: at 2014-05-19 15:58:32 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2014-05-19 15:58:32 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2014-05-19 15:58:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2014-05-19 15:58:33 [DEBUG   ] spawn_process: at 2014-05-19 15:58:33 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2014-05-19 15:58:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2014-05-19 15:58:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2014-05-19 15:58:33 [DEBUG   ] spawn_process: at 2014-05-19 15:58:33 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2014-05-19 15:58:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2014-05-19 15:58:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2014-05-19 15:58:33 [DEBUG   ] spawn_process: at 2014-05-19 15:58:33 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2014-05-19 15:58:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2014-05-19 15:58:33 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2014-05-19 15:58:33 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -4 msec
2014-05-19 15:58:33 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 394 ms (1400515113.398 s) with PID 31360 and time 1400515114
2014-05-19 15:58:33 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -4 msec
2014-05-19 15:58:33 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 485 ms (1400515113.489 s) with PID 31363 and time 1400515114
2014-05-19 15:58:34 [DEBUG   ] Finished updating ant stats in 0.761 seconds
2014-05-19 15:58:34 [INFO    ] Started antennas statistics background thread
2014-05-19 15:58:34 [INFO    ] Started board flags background thread
2014-05-19 15:58:34 [INFO    ] Started BAM background thread
2014-05-19 15:58:34 [INFO    ] Finished the INI process (ref. #193912) in 104.582 s
2014-05-19 15:58:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 15:58:36 [DEBUG   ] Finished updating board flags in 1.949 seconds
2014-05-19 15:58:42 [DEBUG   ] Got command RPT from MCS: ref #193930
2014-05-19 15:58:42 [DEBUG   ] summary = NORMAL
2014-05-19 15:58:42 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193930  14 56796 57522046 A NORMALNORMAL'
2014-05-19 15:58:48 [DEBUG   ] Got command TBN from MCS: ref #193931
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '1', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '2', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] tbn = exited with status True, code 0x00
2014-05-19 15:58:48 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBN   193931   8 56796 57528855 A NORMAL'
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '3', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '4', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '5', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '6', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '7', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '8', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '9', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '10', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '11', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '12', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '13', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '14', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '15', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '16', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '17', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '18', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '19', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '20', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '21', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '22', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '23', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '24', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '25', '1400515130']
2014-05-19 15:58:48 [DEBUG   ] spawn_process: at 2014-05-19 15:58:48 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '26', '1400515130']
2014-05-19 15:58:48 [INFO    ] Finished spawning TBN in 0.060 s
2014-05-19 15:58:54 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 15:59:14 [DEBUG   ] Finished updating ant stats in 0.667 seconds
2014-05-19 15:59:34 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 15:59:36 [DEBUG   ] Finished updating board flags in 2.020 seconds
2014-05-19 15:59:54 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 16:00:14 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 16:00:34 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2014-05-19 16:00:36 [DEBUG   ] Finished updating board flags in 2.004 seconds
2014-05-19 16:00:55 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 16:01:15 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 16:01:35 [DEBUG   ] Finished updating ant stats in 1.253 seconds
2014-05-19 16:01:36 [DEBUG   ] Finished updating board flags in 1.988 seconds
2014-05-19 16:01:55 [DEBUG   ] Finished updating ant stats in 1.156 seconds
2014-05-19 16:02:06 [DEBUG   ] Got command RPT from MCS: ref #193932
2014-05-19 16:02:06 [DEBUG   ] summary = NORMAL
2014-05-19 16:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193932  14 56796 57726762 A NORMALNORMAL'
2014-05-19 16:02:06 [DEBUG   ] Got command RPT from MCS: ref #193933
2014-05-19 16:02:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 16:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193933  48 56796 57726763 A NORMALSystem calibrated and operating normally'
2014-05-19 16:02:15 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-19 16:02:35 [DEBUG   ] Finished updating ant stats in 1.045 seconds
2014-05-19 16:02:36 [DEBUG   ] Finished updating board flags in 1.817 seconds
2014-05-19 16:02:55 [DEBUG   ] Finished updating ant stats in 0.955 seconds
2014-05-19 16:03:15 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 16:03:35 [DEBUG   ] Finished updating ant stats in 0.851 seconds
2014-05-19 16:03:36 [DEBUG   ] Finished updating board flags in 1.789 seconds
2014-05-19 16:03:55 [DEBUG   ] Finished updating ant stats in 0.756 seconds
2014-05-19 16:04:15 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 16:04:36 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 16:04:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 16:04:36 [DEBUG   ] Finished updating board flags in 1.615 seconds
2014-05-19 16:04:56 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 16:05:16 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 16:05:36 [DEBUG   ] Finished updating ant stats in 1.252 seconds
2014-05-19 16:05:36 [DEBUG   ] Finished updating board flags in 1.652 seconds
2014-05-19 16:05:56 [DEBUG   ] Finished updating ant stats in 1.157 seconds
2014-05-19 16:06:16 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 16:06:36 [DEBUG   ] Finished updating ant stats in 1.042 seconds
2014-05-19 16:06:36 [DEBUG   ] Finished updating board flags in 1.598 seconds
2014-05-19 16:06:56 [DEBUG   ] Finished updating ant stats in 0.960 seconds
2014-05-19 16:07:06 [DEBUG   ] Got command RPT from MCS: ref #193949
2014-05-19 16:07:06 [DEBUG   ] summary = NORMAL
2014-05-19 16:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193949  14 56796 58026758 A NORMALNORMAL'
2014-05-19 16:07:06 [DEBUG   ] Got command RPT from MCS: ref #193950
2014-05-19 16:07:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 16:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193950  48 56796 58026759 A NORMALSystem calibrated and operating normally'
2014-05-19 16:07:16 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 16:07:36 [DEBUG   ] Finished updating ant stats in 0.844 seconds
2014-05-19 16:07:36 [DEBUG   ] Finished updating board flags in 1.473 seconds
2014-05-19 16:07:56 [DEBUG   ] Finished updating ant stats in 0.755 seconds
2014-05-19 16:08:16 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 16:08:36 [DEBUG   ] Finished updating ant stats in 0.644 seconds
2014-05-19 16:08:36 [DEBUG   ] Finished updating board flags in 1.478 seconds
2014-05-19 16:08:56 [DEBUG   ] Finished updating ant stats in 0.556 seconds
2014-05-19 16:09:17 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 16:09:37 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 16:09:37 [DEBUG   ] Finished updating board flags in 2.203 seconds
2014-05-19 16:09:57 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 16:10:17 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-19 16:10:37 [DEBUG   ] Finished updating ant stats in 1.252 seconds
2014-05-19 16:10:37 [DEBUG   ] Finished updating board flags in 2.242 seconds
2014-05-19 16:10:57 [DEBUG   ] Finished updating ant stats in 1.157 seconds
2014-05-19 16:11:17 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 16:11:37 [DEBUG   ] Finished updating ant stats in 0.846 seconds
2014-05-19 16:11:37 [DEBUG   ] Finished updating board flags in 2.221 seconds
2014-05-19 16:11:57 [DEBUG   ] Finished updating ant stats in 0.759 seconds
2014-05-19 16:12:06 [DEBUG   ] Got command RPT from MCS: ref #193966
2014-05-19 16:12:06 [DEBUG   ] summary = NORMAL
2014-05-19 16:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193966  14 56796 58326754 A NORMALNORMAL'
2014-05-19 16:12:06 [DEBUG   ] Got command RPT from MCS: ref #193967
2014-05-19 16:12:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 16:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193967  48 56796 58326755 A NORMALSystem calibrated and operating normally'
2014-05-19 16:12:17 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 16:12:37 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 16:12:37 [DEBUG   ] Finished updating ant stats in 0.643 seconds
2014-05-19 16:12:37 [DEBUG   ] Finished updating board flags in 2.182 seconds
2014-05-19 16:12:57 [DEBUG   ] Finished updating ant stats in 0.558 seconds
2014-05-19 16:13:18 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 16:13:37 [DEBUG   ] Finished updating board flags in 2.031 seconds
2014-05-19 16:13:38 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 16:13:58 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 16:14:18 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 16:14:37 [DEBUG   ] Finished updating board flags in 1.969 seconds
2014-05-19 16:14:38 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 16:14:58 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 16:15:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 16:15:37 [DEBUG   ] Finished updating board flags in 1.828 seconds
2014-05-19 16:15:38 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 16:15:58 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 16:16:18 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-19 16:16:37 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-19 16:16:37 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 16:16:37 [DEBUG   ] Finished updating board flags in 1.784 seconds
2014-05-19 16:16:38 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 16:16:58 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 16:17:06 [DEBUG   ] Got command RPT from MCS: ref #193983
2014-05-19 16:17:06 [DEBUG   ] summary = NORMAL
2014-05-19 16:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193983  14 56796 58626762 A NORMALNORMAL'
2014-05-19 16:17:06 [DEBUG   ] Got command RPT from MCS: ref #193984
2014-05-19 16:17:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 16:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   193984  48 56796 58626763 A NORMALSystem calibrated and operating normally'
2014-05-19 16:17:18 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 16:17:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 16:17:37 [DEBUG   ] Finished updating board flags in 1.613 seconds
2014-05-19 16:17:38 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 16:17:58 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 16:18:18 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 16:18:37 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 16:18:37 [DEBUG   ] Finished updating board flags in 1.538 seconds
2014-05-19 16:18:38 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 16:18:58 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 16:19:19 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 16:19:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 16:19:37 [DEBUG   ] Finished updating board flags in 1.410 seconds
2014-05-19 16:19:39 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 16:19:59 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 16:20:19 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 16:20:37 [DEBUG   ] Finished updating board flags in 1.407 seconds
2014-05-19 16:20:39 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 16:20:59 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 16:21:19 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 16:21:37 [DEBUG   ] Finished updating board flags in 1.402 seconds
2014-05-19 16:21:39 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 16:21:59 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 16:22:06 [DEBUG   ] Got command RPT from MCS: ref #194000
2014-05-19 16:22:06 [DEBUG   ] summary = NORMAL
2014-05-19 16:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194000  14 56796 58926762 A NORMALNORMAL'
2014-05-19 16:22:06 [DEBUG   ] Got command RPT from MCS: ref #194001
2014-05-19 16:22:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 16:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194001  48 56796 58926763 A NORMALSystem calibrated and operating normally'
2014-05-19 16:22:19 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 16:22:37 [DEBUG   ] Finished updating board flags in 1.389 seconds
2014-05-19 16:22:39 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 16:22:59 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 16:23:19 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 16:23:38 [DEBUG   ] Finished updating board flags in 2.265 seconds
2014-05-19 16:23:39 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 16:23:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 16:24:19 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 16:24:38 [DEBUG   ] Finished updating board flags in 2.142 seconds
2014-05-19 16:24:39 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 16:24:59 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 16:25:20 [DEBUG   ] Finished updating ant stats in 1.388 seconds
2014-05-19 16:25:38 [DEBUG   ] Finished updating board flags in 2.009 seconds
2014-05-19 16:25:40 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 16:26:00 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 16:26:20 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 16:26:38 [DEBUG   ] Finished updating board flags in 1.996 seconds
2014-05-19 16:26:40 [DEBUG   ] Finished updating ant stats in 0.987 seconds
2014-05-19 16:27:00 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 16:27:06 [DEBUG   ] Got command RPT from MCS: ref #194017
2014-05-19 16:27:06 [DEBUG   ] summary = NORMAL
2014-05-19 16:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194017  14 56796 59226766 A NORMALNORMAL'
2014-05-19 16:27:06 [DEBUG   ] Got command RPT from MCS: ref #194018
2014-05-19 16:27:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 16:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194018  48 56796 59226767 A NORMALSystem calibrated and operating normally'
2014-05-19 16:27:20 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 16:27:38 [DEBUG   ] Finished updating board flags in 1.789 seconds
2014-05-19 16:27:40 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-19 16:28:01 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 16:28:21 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 16:28:38 [DEBUG   ] Finished updating board flags in 1.618 seconds
2014-05-19 16:28:41 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 16:29:01 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 16:29:21 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 16:29:38 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-19 16:29:38 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 16:29:38 [DEBUG   ] Finished updating board flags in 1.579 seconds
2014-05-19 16:29:41 [DEBUG   ] Finished updating ant stats in 1.195 seconds
2014-05-19 16:30:01 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 16:30:21 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 16:30:38 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 16:30:38 [DEBUG   ] Finished updating board flags in 1.422 seconds
2014-05-19 16:30:41 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 16:31:01 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 16:31:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 16:31:38 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 16:31:38 [DEBUG   ] Finished updating board flags in 1.390 seconds
2014-05-19 16:31:41 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 16:32:01 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 16:32:06 [DEBUG   ] Got command RPT from MCS: ref #194034
2014-05-19 16:32:06 [DEBUG   ] summary = NORMAL
2014-05-19 16:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194034  14 56796 59526762 A NORMALNORMAL'
2014-05-19 16:32:06 [DEBUG   ] Got command RPT from MCS: ref #194035
2014-05-19 16:32:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 16:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194035  48 56796 59526763 A NORMALSystem calibrated and operating normally'
2014-05-19 16:32:21 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 16:32:38 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 16:32:39 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 16:32:39 [DEBUG   ] Finished updating board flags in 2.238 seconds
2014-05-19 16:32:41 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 16:33:01 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 16:33:21 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 16:33:39 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 16:33:39 [DEBUG   ] Finished updating board flags in 2.163 seconds
2014-05-19 16:33:42 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-19 16:34:02 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 16:34:22 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 16:34:38 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 16:34:39 [DEBUG   ] Finished updating board flags in 2.014 seconds
2014-05-19 16:34:42 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 16:35:02 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 16:35:22 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 16:35:39 [DEBUG   ] Finished updating board flags in 2.013 seconds
2014-05-19 16:35:42 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 16:36:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 16:36:22 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 16:36:39 [DEBUG   ] Finished updating board flags in 1.978 seconds
2014-05-19 16:36:42 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 16:37:02 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 16:37:06 [DEBUG   ] Got command RPT from MCS: ref #194051
2014-05-19 16:37:06 [DEBUG   ] summary = NORMAL
2014-05-19 16:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194051  14 56796 59826762 A NORMALNORMAL'
2014-05-19 16:37:06 [DEBUG   ] Got command RPT from MCS: ref #194052
2014-05-19 16:37:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 16:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194052  48 56796 59826763 A NORMALSystem calibrated and operating normally'
2014-05-19 16:37:22 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-19 16:37:39 [DEBUG   ] Finished updating board flags in 1.824 seconds
2014-05-19 16:37:42 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 16:38:02 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 16:38:22 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 16:38:39 [DEBUG   ] Finished updating board flags in 1.784 seconds
2014-05-19 16:38:42 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 16:39:02 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 16:39:22 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 16:39:39 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 16:39:39 [DEBUG   ] Finished updating board flags in 1.652 seconds
2014-05-19 16:39:42 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 16:40:02 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 16:40:22 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 16:40:39 [DEBUG   ] Finished updating board flags in 1.558 seconds
2014-05-19 16:40:42 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 16:41:02 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 16:41:22 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 16:41:39 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 16:41:39 [DEBUG   ] Finished updating board flags in 1.526 seconds
2014-05-19 16:41:42 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 16:42:02 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 16:42:06 [DEBUG   ] Got command RPT from MCS: ref #194068
2014-05-19 16:42:06 [DEBUG   ] summary = NORMAL
2014-05-19 16:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194068  14 56796 60126766 A NORMALNORMAL'
2014-05-19 16:42:06 [DEBUG   ] Got command RPT from MCS: ref #194069
2014-05-19 16:42:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 16:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194069  48 56796 60126767 A NORMALSystem calibrated and operating normally'
2014-05-19 16:42:22 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 16:42:39 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 16:42:39 [DEBUG   ] Finished updating board flags in 1.567 seconds
2014-05-19 16:42:42 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2014-05-19 16:43:02 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 16:43:22 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 16:43:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 16:43:40 [DEBUG   ] Finished updating board flags in 2.192 seconds
2014-05-19 16:43:42 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 16:44:02 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 16:44:22 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 16:44:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 16:44:40 [DEBUG   ] Finished updating board flags in 1.935 seconds
2014-05-19 16:44:42 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 16:45:02 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 16:45:22 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2014-05-19 16:45:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 16:45:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 16:45:40 [DEBUG   ] Finished updating board flags in 1.804 seconds
2014-05-19 16:45:43 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 16:46:03 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-19 16:46:23 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 16:46:40 [DEBUG   ] Finished updating board flags in 1.810 seconds
2014-05-19 16:46:43 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 16:47:03 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 16:47:06 [DEBUG   ] Got command RPT from MCS: ref #194085
2014-05-19 16:47:06 [DEBUG   ] summary = NORMAL
2014-05-19 16:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194085  14 56796 60426770 A NORMALNORMAL'
2014-05-19 16:47:06 [DEBUG   ] Got command RPT from MCS: ref #194086
2014-05-19 16:47:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-19 16:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194086  48 56796 60426771 A NORMALSystem calibrated and operating normally'
2014-05-19 16:47:23 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 16:47:40 [DEBUG   ] Finished updating board flags in 1.795 seconds
2014-05-19 16:47:43 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 16:48:03 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 16:48:23 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 16:48:39 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 16:48:40 [DEBUG   ] Finished updating board flags in 1.621 seconds
2014-05-19 16:48:43 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 16:49:03 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 16:49:23 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 16:49:40 [DEBUG   ] Finished updating board flags in 1.575 seconds
2014-05-19 16:49:43 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 16:50:03 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 16:50:23 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 16:50:40 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 16:50:40 [DEBUG   ] Finished updating board flags in 1.566 seconds
2014-05-19 16:50:43 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 16:51:03 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 16:51:23 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 16:51:40 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-19 16:51:41 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 16:51:41 [CRITICAL] MGT underflow detected on beam(s) 1
2014-05-19 16:51:41 [DEBUG   ] Finished updating board flags in 1.596 seconds
2014-05-19 16:51:43 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 16:52:03 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 16:52:06 [DEBUG   ] Got command RPT from MCS: ref #194102
2014-05-19 16:52:06 [DEBUG   ] summary = ERROR
2014-05-19 16:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194102  13 56796 60726762 A  ERRORERROR'
2014-05-19 16:52:06 [DEBUG   ] Got command RPT from MCS: ref #194103
2014-05-19 16:52:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 16:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194103  77 56796 60726763 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 16:52:23 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 16:52:41 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 16:52:41 [DEBUG   ] Finished updating board flags in 2.158 seconds
2014-05-19 16:52:43 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 16:53:03 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 16:53:23 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 16:53:41 [DEBUG   ] Finished updating board flags in 1.884 seconds
2014-05-19 16:53:43 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 16:54:03 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 16:54:23 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 16:54:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 16:54:41 [DEBUG   ] Finished updating board flags in 1.837 seconds
2014-05-19 16:54:43 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 16:55:03 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 16:55:23 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 16:55:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 16:55:41 [DEBUG   ] Finished updating board flags in 1.812 seconds
2014-05-19 16:55:43 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-19 16:56:03 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 16:56:23 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 16:56:41 [DEBUG   ] Finished updating board flags in 1.750 seconds
2014-05-19 16:56:43 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-19 16:57:03 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 16:57:06 [DEBUG   ] Got command RPT from MCS: ref #194119
2014-05-19 16:57:06 [DEBUG   ] summary = ERROR
2014-05-19 16:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194119  13 56796 61026774 A  ERRORERROR'
2014-05-19 16:57:06 [DEBUG   ] Got command RPT from MCS: ref #194120
2014-05-19 16:57:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 16:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194120  77 56796 61026775 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 16:57:23 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 16:57:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 16:57:41 [DEBUG   ] Finished updating board flags in 1.611 seconds
2014-05-19 16:57:43 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 16:58:03 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 16:58:23 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 16:58:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 16:58:41 [DEBUG   ] Finished updating board flags in 1.614 seconds
2014-05-19 16:58:43 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 16:59:03 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 16:59:23 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 16:59:41 [DEBUG   ] Finished updating board flags in 1.653 seconds
2014-05-19 16:59:43 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 17:00:03 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 17:00:23 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 17:00:41 [DEBUG   ] Finished updating board flags in 1.632 seconds
2014-05-19 17:00:43 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 17:01:03 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:01:23 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:01:41 [DEBUG   ] Finished updating board flags in 1.499 seconds
2014-05-19 17:01:43 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 17:02:03 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 17:02:06 [DEBUG   ] Got command RPT from MCS: ref #194136
2014-05-19 17:02:06 [DEBUG   ] summary = ERROR
2014-05-19 17:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194136  13 56796 61326774 A  ERRORERROR'
2014-05-19 17:02:06 [DEBUG   ] Got command RPT from MCS: ref #194137
2014-05-19 17:02:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 17:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194137  77 56796 61326775 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 17:02:24 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-19 17:02:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 17:02:41 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 17:02:41 [DEBUG   ] Finished updating board flags in 1.478 seconds
2014-05-19 17:02:44 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-19 17:03:04 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 17:03:24 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 17:03:41 [DEBUG   ] Finished updating board flags in 1.477 seconds
2014-05-19 17:03:44 [DEBUG   ] Finished updating ant stats in 0.964 seconds
2014-05-19 17:04:04 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 17:04:24 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 17:04:41 [DEBUG   ] Finished updating board flags in 1.483 seconds
2014-05-19 17:04:44 [DEBUG   ] Finished updating ant stats in 0.594 seconds
2014-05-19 17:05:05 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 17:05:25 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 17:05:42 [DEBUG   ] Finished updating board flags in 2.183 seconds
2014-05-19 17:05:45 [DEBUG   ] Finished updating ant stats in 1.392 seconds
2014-05-19 17:06:05 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 17:06:25 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 17:06:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 17:06:42 [DEBUG   ] Finished updating board flags in 2.009 seconds
2014-05-19 17:06:45 [DEBUG   ] Finished updating ant stats in 1.191 seconds
2014-05-19 17:07:05 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 17:07:06 [DEBUG   ] Got command RPT from MCS: ref #194153
2014-05-19 17:07:06 [DEBUG   ] summary = ERROR
2014-05-19 17:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194153  13 56796 61626766 A  ERRORERROR'
2014-05-19 17:07:06 [DEBUG   ] Got command RPT from MCS: ref #194154
2014-05-19 17:07:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 17:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194154  77 56796 61626767 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 17:07:25 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 17:07:42 [DEBUG   ] Finished updating board flags in 1.987 seconds
2014-05-19 17:07:45 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-19 17:08:05 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 17:08:25 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 17:08:42 [DEBUG   ] Finished updating board flags in 1.815 seconds
2014-05-19 17:08:45 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 17:09:05 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 17:09:25 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 17:09:42 [DEBUG   ] Finished updating board flags in 1.781 seconds
2014-05-19 17:09:45 [DEBUG   ] Finished updating ant stats in 0.586 seconds
2014-05-19 17:10:06 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 17:10:26 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 17:10:42 [DEBUG   ] Finished updating board flags in 1.629 seconds
2014-05-19 17:10:46 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 17:11:06 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 17:11:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 17:11:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 17:11:42 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 17:11:42 [DEBUG   ] Finished updating board flags in 1.584 seconds
2014-05-19 17:11:46 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 17:12:06 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 17:12:06 [DEBUG   ] Got command RPT from MCS: ref #194170
2014-05-19 17:12:06 [DEBUG   ] summary = ERROR
2014-05-19 17:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194170  13 56796 61926778 A  ERRORERROR'
2014-05-19 17:12:06 [DEBUG   ] Got command RPT from MCS: ref #194171
2014-05-19 17:12:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 17:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194171  77 56796 61926779 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 17:12:26 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 17:12:42 [DEBUG   ] Finished updating board flags in 1.415 seconds
2014-05-19 17:12:46 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 17:13:06 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 17:13:26 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 17:13:42 [DEBUG   ] Finished updating board flags in 1.390 seconds
2014-05-19 17:13:46 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 17:14:06 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 17:14:26 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 17:14:43 [DEBUG   ] Finished updating board flags in 2.243 seconds
2014-05-19 17:14:46 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 17:15:06 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:15:26 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:15:43 [DEBUG   ] Finished updating board flags in 2.167 seconds
2014-05-19 17:15:46 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 17:16:07 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 17:16:27 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 17:16:43 [DEBUG   ] Finished updating board flags in 2.015 seconds
2014-05-19 17:16:47 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 17:17:06 [DEBUG   ] Got command RPT from MCS: ref #194187
2014-05-19 17:17:06 [DEBUG   ] summary = ERROR
2014-05-19 17:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194187  13 56796 62226778 A  ERRORERROR'
2014-05-19 17:17:06 [DEBUG   ] Got command RPT from MCS: ref #194188
2014-05-19 17:17:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 17:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194188  77 56796 62226779 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 17:17:07 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 17:17:27 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 17:17:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 17:17:43 [DEBUG   ] Finished updating board flags in 1.769 seconds
2014-05-19 17:17:47 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 17:18:07 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 17:18:27 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 17:18:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 17:18:43 [DEBUG   ] Finished updating board flags in 1.617 seconds
2014-05-19 17:18:47 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 17:19:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 17:19:27 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 17:19:43 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 17:19:43 [DEBUG   ] Finished updating board flags in 1.596 seconds
2014-05-19 17:19:47 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 17:20:07 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 17:20:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 17:20:43 [DEBUG   ] Finished updating board flags in 1.421 seconds
2014-05-19 17:20:47 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 17:21:07 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 17:21:27 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 17:21:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 17:21:43 [DEBUG   ] Finished updating board flags in 1.409 seconds
2014-05-19 17:21:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 17:22:06 [DEBUG   ] Got command RPT from MCS: ref #194204
2014-05-19 17:22:06 [DEBUG   ] summary = ERROR
2014-05-19 17:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194204  13 56796 62526778 A  ERRORERROR'
2014-05-19 17:22:06 [DEBUG   ] Got command RPT from MCS: ref #194205
2014-05-19 17:22:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 17:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194205  77 56796 62526779 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 17:22:07 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 17:22:27 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 17:22:43 [DEBUG   ] Finished updating board flags in 1.401 seconds
2014-05-19 17:22:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:23:07 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:23:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 17:23:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 17:23:43 [DEBUG   ] Finished updating board flags in 1.405 seconds
2014-05-19 17:23:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 17:24:07 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 17:24:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:24:43 [DEBUG   ] Finished updating board flags in 1.426 seconds
2014-05-19 17:24:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 17:25:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 17:25:27 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 17:25:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 17:25:43 [DEBUG   ] Finished updating board flags in 1.552 seconds
2014-05-19 17:25:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:26:07 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 17:26:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:26:44 [DEBUG   ] Finished updating board flags in 2.305 seconds
2014-05-19 17:26:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:27:06 [DEBUG   ] Got command RPT from MCS: ref #194221
2014-05-19 17:27:06 [DEBUG   ] summary = ERROR
2014-05-19 17:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194221  13 56796 62826774 A  ERRORERROR'
2014-05-19 17:27:06 [DEBUG   ] Got command RPT from MCS: ref #194222
2014-05-19 17:27:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 17:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194222  77 56796 62826775 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 17:27:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:27:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 17:27:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2014-05-19 17:27:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 17:27:44 [DEBUG   ] Finished updating board flags in 2.107 seconds
2014-05-19 17:27:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:28:07 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 17:28:27 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:28:44 [DEBUG   ] Finished updating board flags in 2.019 seconds
2014-05-19 17:28:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:29:07 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 17:29:27 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 17:29:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 17:29:44 [DEBUG   ] Finished updating board flags in 2.014 seconds
2014-05-19 17:29:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:30:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:30:27 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 17:30:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 17:30:44 [DEBUG   ] Finished updating board flags in 1.982 seconds
2014-05-19 17:30:47 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 17:31:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:31:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:31:44 [DEBUG   ] Finished updating board flags in 1.799 seconds
2014-05-19 17:31:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 17:32:06 [DEBUG   ] Got command RPT from MCS: ref #194238
2014-05-19 17:32:06 [DEBUG   ] summary = ERROR
2014-05-19 17:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194238  13 56796 63126778 A  ERRORERROR'
2014-05-19 17:32:06 [DEBUG   ] Got command RPT from MCS: ref #194239
2014-05-19 17:32:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 17:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194239  77 56796 63126779 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 17:32:07 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:32:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:32:44 [DEBUG   ] Finished updating board flags in 1.688 seconds
2014-05-19 17:32:47 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 17:33:07 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 17:33:27 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:33:43 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 17:33:44 [DEBUG   ] Finished updating board flags in 1.613 seconds
2014-05-19 17:33:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:34:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:34:27 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 17:34:44 [DEBUG   ] Finished updating board flags in 1.486 seconds
2014-05-19 17:34:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:35:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:35:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 17:35:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 17:35:44 [DEBUG   ] Finished updating board flags in 1.498 seconds
2014-05-19 17:35:47 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:36:07 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:36:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:36:44 [DEBUG   ] Finished updating board flags in 1.512 seconds
2014-05-19 17:36:47 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2014-05-19 17:37:06 [DEBUG   ] Got command RPT from MCS: ref #194255
2014-05-19 17:37:06 [DEBUG   ] summary = ERROR
2014-05-19 17:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194255  13 56796 63426774 A  ERRORERROR'
2014-05-19 17:37:06 [DEBUG   ] Got command RPT from MCS: ref #194256
2014-05-19 17:37:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 17:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194256  77 56796 63426775 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 17:37:08 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 17:37:28 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 17:37:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 17:37:45 [DEBUG   ] Finished updating board flags in 2.230 seconds
2014-05-19 17:37:48 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 17:38:08 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 17:38:28 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 17:38:45 [DEBUG   ] Finished updating board flags in 2.164 seconds
2014-05-19 17:38:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 17:39:08 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 17:39:28 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 17:39:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 17:39:45 [DEBUG   ] Finished updating board flags in 2.015 seconds
2014-05-19 17:39:48 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 17:40:08 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 17:40:28 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 17:40:45 [DEBUG   ] Finished updating board flags in 2.113 seconds
2014-05-19 17:40:48 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 17:41:08 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 17:41:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 17:41:45 [DEBUG   ] Finished updating board flags in 2.012 seconds
2014-05-19 17:41:48 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 17:42:06 [DEBUG   ] Got command RPT from MCS: ref #194272
2014-05-19 17:42:06 [DEBUG   ] summary = ERROR
2014-05-19 17:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194272  13 56796 63726786 A  ERRORERROR'
2014-05-19 17:42:06 [DEBUG   ] Got command RPT from MCS: ref #194273
2014-05-19 17:42:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 17:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194273  77 56796 63726787 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 17:42:08 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 17:42:28 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 17:42:45 [DEBUG   ] Finished updating board flags in 1.879 seconds
2014-05-19 17:42:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 17:43:08 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 17:43:28 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 17:43:45 [DEBUG   ] Finished updating board flags in 1.821 seconds
2014-05-19 17:43:48 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 17:44:08 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 17:44:28 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 17:44:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 17:44:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 17:44:45 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 17:44:45 [DEBUG   ] Finished updating board flags in 1.780 seconds
2014-05-19 17:44:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 17:45:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 17:45:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 17:45:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 17:45:45 [DEBUG   ] Finished updating board flags in 1.628 seconds
2014-05-19 17:45:48 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 17:46:08 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2014-05-19 17:46:28 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:46:45 [DEBUG   ] Finished updating board flags in 1.565 seconds
2014-05-19 17:46:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 17:47:06 [DEBUG   ] Got command RPT from MCS: ref #194289
2014-05-19 17:47:06 [DEBUG   ] summary = ERROR
2014-05-19 17:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194289  13 56796 64026782 A  ERRORERROR'
2014-05-19 17:47:06 [DEBUG   ] Got command RPT from MCS: ref #194290
2014-05-19 17:47:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 17:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194290  77 56796 64026783 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 17:47:08 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 17:47:29 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 17:47:45 [DEBUG   ] Finished updating board flags in 1.485 seconds
2014-05-19 17:47:49 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 17:48:09 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 17:48:29 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 17:48:44 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 17:48:44 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 4
2014-05-19 17:48:45 [DEBUG   ] Finished updating board flags in 1.482 seconds
2014-05-19 17:48:49 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 17:49:09 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 17:49:29 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 17:49:45 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 17:49:45 [DEBUG   ] Finished updating board flags in 1.576 seconds
2014-05-19 17:49:49 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 17:50:09 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 17:50:29 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 17:50:46 [DEBUG   ] Finished updating board flags in 2.181 seconds
2014-05-19 17:50:49 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 17:51:10 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 17:51:30 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 17:51:46 [DEBUG   ] Finished updating board flags in 1.895 seconds
2014-05-19 17:51:50 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-19 17:52:06 [DEBUG   ] Got command RPT from MCS: ref #194306
2014-05-19 17:52:06 [DEBUG   ] summary = ERROR
2014-05-19 17:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194306  13 56796 64326786 A  ERRORERROR'
2014-05-19 17:52:06 [DEBUG   ] Got command RPT from MCS: ref #194307
2014-05-19 17:52:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 17:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194307  77 56796 64326787 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 17:52:10 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 17:52:30 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 17:52:46 [DEBUG   ] Finished updating board flags in 1.808 seconds
2014-05-19 17:52:50 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 17:53:10 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 17:53:30 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 17:53:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 17:53:46 [DEBUG   ] Finished updating board flags in 1.799 seconds
2014-05-19 17:53:50 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 17:54:10 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 17:54:30 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 17:54:46 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 17:54:46 [DEBUG   ] Finished updating board flags in 1.603 seconds
2014-05-19 17:54:50 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 17:55:10 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 17:55:30 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 17:55:46 [DEBUG   ] Finished updating board flags in 1.595 seconds
2014-05-19 17:55:51 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 17:56:11 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 17:56:31 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 17:56:46 [DEBUG   ] Finished updating board flags in 1.416 seconds
2014-05-19 17:56:51 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 17:57:06 [DEBUG   ] Got command RPT from MCS: ref #194323
2014-05-19 17:57:06 [DEBUG   ] summary = ERROR
2014-05-19 17:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194323  13 56796 64626782 A  ERRORERROR'
2014-05-19 17:57:06 [DEBUG   ] Got command RPT from MCS: ref #194324
2014-05-19 17:57:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 17:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194324  77 56796 64626783 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 17:57:11 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 17:57:31 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 17:57:46 [DEBUG   ] Finished updating board flags in 1.394 seconds
2014-05-19 17:57:51 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 17:58:11 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 17:58:31 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 17:58:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 17:58:47 [DEBUG   ] Finished updating board flags in 2.209 seconds
2014-05-19 17:58:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 17:59:11 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 17:59:31 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 17:59:47 [DEBUG   ] Finished updating board flags in 2.194 seconds
2014-05-19 17:59:51 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 18:00:11 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 18:00:32 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-19 18:00:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 18:00:47 [DEBUG   ] Finished updating board flags in 2.017 seconds
2014-05-19 18:00:52 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 18:01:12 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 18:01:32 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 18:01:47 [DEBUG   ] Finished updating board flags in 1.992 seconds
2014-05-19 18:01:52 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 18:02:06 [DEBUG   ] Got command RPT from MCS: ref #194340
2014-05-19 18:02:06 [DEBUG   ] summary = ERROR
2014-05-19 18:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194340  13 56796 64926782 A  ERRORERROR'
2014-05-19 18:02:06 [DEBUG   ] Got command RPT from MCS: ref #194341
2014-05-19 18:02:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 18:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194341  77 56796 64926783 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 18:02:12 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:02:32 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 18:02:47 [DEBUG   ] Finished updating board flags in 1.813 seconds
2014-05-19 18:02:52 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 18:03:13 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 18:03:33 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 18:03:47 [DEBUG   ] Finished updating board flags in 1.789 seconds
2014-05-19 18:03:53 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 18:04:13 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 18:04:33 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:04:47 [DEBUG   ] Finished updating board flags in 1.611 seconds
2014-05-19 18:04:53 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 18:05:13 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 18:05:33 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 18:05:47 [DEBUG   ] Finished updating board flags in 1.591 seconds
2014-05-19 18:05:53 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:06:13 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:06:33 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 18:06:46 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 18:06:47 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 18:06:47 [DEBUG   ] Finished updating board flags in 1.431 seconds
2014-05-19 18:06:53 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 18:07:06 [DEBUG   ] Got command RPT from MCS: ref #194357
2014-05-19 18:07:06 [DEBUG   ] summary = ERROR
2014-05-19 18:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194357  13 56796 65226790 A  ERRORERROR'
2014-05-19 18:07:06 [DEBUG   ] Got command RPT from MCS: ref #194358
2014-05-19 18:07:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 18:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194358  77 56796 65226791 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 18:07:13 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2014-05-19 18:07:33 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 18:07:47 [DEBUG   ] Finished updating board flags in 1.372 seconds
2014-05-19 18:07:53 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 18:08:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 18:08:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 18:08:47 [DEBUG   ] Finished updating board flags in 1.373 seconds
2014-05-19 18:08:53 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 18:09:13 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 18:09:34 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 18:09:48 [DEBUG   ] Finished updating board flags in 2.042 seconds
2014-05-19 18:09:54 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 18:10:14 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-19 18:10:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:10:48 [DEBUG   ] Finished updating board flags in 2.028 seconds
2014-05-19 18:10:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:11:14 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 18:11:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:11:48 [DEBUG   ] Finished updating board flags in 2.063 seconds
2014-05-19 18:11:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:12:06 [DEBUG   ] Got command RPT from MCS: ref #194374
2014-05-19 18:12:06 [DEBUG   ] summary = ERROR
2014-05-19 18:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194374  13 56796 65526790 A  ERRORERROR'
2014-05-19 18:12:06 [DEBUG   ] Got command RPT from MCS: ref #194375
2014-05-19 18:12:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 18:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194375  77 56796 65526791 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 18:12:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:12:34 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 18:12:48 [DEBUG   ] Finished updating board flags in 2.016 seconds
2014-05-19 18:12:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:13:14 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:13:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:13:48 [DEBUG   ] Finished updating board flags in 1.905 seconds
2014-05-19 18:13:54 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 18:14:14 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:14:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:14:48 [DEBUG   ] Finished updating board flags in 1.831 seconds
2014-05-19 18:14:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:15:14 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:15:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:15:48 [DEBUG   ] Finished updating board flags in 1.777 seconds
2014-05-19 18:15:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:16:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:16:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:16:48 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 18:16:48 [DEBUG   ] Finished updating board flags in 1.617 seconds
2014-05-19 18:16:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:17:06 [DEBUG   ] Got command RPT from MCS: ref #194396
2014-05-19 18:17:06 [DEBUG   ] summary = ERROR
2014-05-19 18:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194396  13 56796 65826790 A  ERRORERROR'
2014-05-19 18:17:06 [DEBUG   ] Got command RPT from MCS: ref #194397
2014-05-19 18:17:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 18:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194397  77 56796 65826791 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 18:17:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:17:34 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 18:17:48 [DEBUG   ] Finished updating board flags in 1.675 seconds
2014-05-19 18:17:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:18:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:18:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:18:48 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 18:18:48 [DEBUG   ] Finished updating board flags in 1.506 seconds
2014-05-19 18:18:54 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 18:19:14 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:19:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:19:48 [DEBUG   ] Finished updating board flags in 1.492 seconds
2014-05-19 18:19:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:20:14 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 18:20:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:20:48 [DEBUG   ] Finished updating board flags in 1.493 seconds
2014-05-19 18:20:54 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:21:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:21:34 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 18:21:48 [DEBUG   ] Finished updating board flags in 1.536 seconds
2014-05-19 18:21:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:22:06 [DEBUG   ] Got command RPT from MCS: ref #194413
2014-05-19 18:22:06 [DEBUG   ] summary = ERROR
2014-05-19 18:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194413  13 56796 66126790 A  ERRORERROR'
2014-05-19 18:22:06 [DEBUG   ] Got command RPT from MCS: ref #194414
2014-05-19 18:22:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 18:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194414  77 56796 66126791 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 18:22:14 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:22:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:22:49 [DEBUG   ] Finished updating board flags in 2.134 seconds
2014-05-19 18:22:54 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 18:23:14 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2014-05-19 18:23:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 18:23:49 [DEBUG   ] Finished updating board flags in 1.982 seconds
2014-05-19 18:23:54 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 18:24:14 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 18:24:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 18:24:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 18:24:49 [DEBUG   ] Finished updating board flags in 1.858 seconds
2014-05-19 18:24:54 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2014-05-19 18:25:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:25:34 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 18:25:49 [DEBUG   ] Finished updating board flags in 1.717 seconds
2014-05-19 18:25:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:26:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:26:34 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:26:49 [DEBUG   ] Finished updating board flags in 1.598 seconds
2014-05-19 18:26:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 18:27:06 [DEBUG   ] Got command RPT from MCS: ref #194430
2014-05-19 18:27:06 [DEBUG   ] summary = ERROR
2014-05-19 18:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194430  13 56796 66426786 A  ERRORERROR'
2014-05-19 18:27:06 [DEBUG   ] Got command RPT from MCS: ref #194431
2014-05-19 18:27:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 18:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194431  77 56796 66426787 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 18:27:14 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:27:34 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:27:49 [DEBUG   ] Finished updating board flags in 1.414 seconds
2014-05-19 18:27:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:28:14 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 18:28:34 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:28:49 [DEBUG   ] Finished updating board flags in 1.462 seconds
2014-05-19 18:28:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:29:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:29:34 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 18:29:49 [DEBUG   ] Finished updating board flags in 1.587 seconds
2014-05-19 18:29:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:30:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:30:34 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:30:50 [DEBUG   ] Finished updating board flags in 2.271 seconds
2014-05-19 18:30:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 18:31:14 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:31:34 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:31:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2014-05-19 18:31:50 [DEBUG   ] Finished updating board flags in 2.070 seconds
2014-05-19 18:31:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 18:32:06 [DEBUG   ] Got command RPT from MCS: ref #194447
2014-05-19 18:32:06 [DEBUG   ] summary = ERROR
2014-05-19 18:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194447  13 56796 66726790 A  ERRORERROR'
2014-05-19 18:32:06 [DEBUG   ] Got command RPT from MCS: ref #194448
2014-05-19 18:32:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 18:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194448  77 56796 66726791 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 18:32:14 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 18:32:34 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:32:50 [DEBUG   ] Finished updating board flags in 2.027 seconds
2014-05-19 18:32:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:33:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:33:34 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 18:33:50 [DEBUG   ] Finished updating board flags in 1.982 seconds
2014-05-19 18:33:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:34:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:34:34 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 18:34:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 18:34:50 [DEBUG   ] Finished updating board flags in 1.819 seconds
2014-05-19 18:34:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 18:35:14 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:35:34 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:35:50 [DEBUG   ] Finished updating board flags in 1.903 seconds
2014-05-19 18:35:54 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 18:36:14 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 18:36:34 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:36:50 [DEBUG   ] Finished updating board flags in 1.699 seconds
2014-05-19 18:36:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:37:06 [DEBUG   ] Got command RPT from MCS: ref #194464
2014-05-19 18:37:06 [DEBUG   ] summary = ERROR
2014-05-19 18:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194464  13 56796 67026790 A  ERRORERROR'
2014-05-19 18:37:06 [DEBUG   ] Got command RPT from MCS: ref #194465
2014-05-19 18:37:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 18:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194465  77 56796 67026791 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 18:37:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:37:34 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 18:37:50 [DEBUG   ] Finished updating board flags in 1.616 seconds
2014-05-19 18:37:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:38:14 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:38:34 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:38:50 [DEBUG   ] Finished updating board flags in 1.606 seconds
2014-05-19 18:38:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 18:39:14 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:39:34 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 18:39:50 [DEBUG   ] Finished updating board flags in 1.577 seconds
2014-05-19 18:39:54 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 18:40:14 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 18:40:34 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 18:40:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 18:40:50 [DEBUG   ] Finished updating board flags in 1.417 seconds
2014-05-19 18:40:54 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 18:41:14 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 18:41:34 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 18:41:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 18:41:50 [DEBUG   ] Finished updating board flags in 1.552 seconds
2014-05-19 18:41:54 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 18:42:06 [DEBUG   ] Got command RPT from MCS: ref #194481
2014-05-19 18:42:06 [DEBUG   ] summary = ERROR
2014-05-19 18:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194481  13 56796 67326798 A  ERRORERROR'
2014-05-19 18:42:06 [DEBUG   ] Got command RPT from MCS: ref #194482
2014-05-19 18:42:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 18:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194482  77 56796 67326799 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 18:42:14 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 18:42:34 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 18:42:50 [DEBUG   ] Finished updating board flags in 1.562 seconds
2014-05-19 18:42:54 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 18:43:14 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 18:43:34 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 18:43:51 [DEBUG   ] Finished updating board flags in 2.180 seconds
2014-05-19 18:43:54 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 18:44:14 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 18:44:34 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 18:44:51 [DEBUG   ] Finished updating board flags in 1.894 seconds
2014-05-19 18:44:54 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 18:45:14 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 18:45:34 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 18:45:51 [DEBUG   ] Finished updating board flags in 1.826 seconds
2014-05-19 18:45:54 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 18:46:14 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 18:46:34 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 18:46:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 18:46:51 [DEBUG   ] Finished updating board flags in 1.810 seconds
2014-05-19 18:46:54 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 18:47:06 [DEBUG   ] Got command RPT from MCS: ref #194498
2014-05-19 18:47:06 [DEBUG   ] summary = ERROR
2014-05-19 18:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194498  13 56796 67626794 A  ERRORERROR'
2014-05-19 18:47:06 [DEBUG   ] Got command RPT from MCS: ref #194499
2014-05-19 18:47:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 18:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194499  77 56796 67626795 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 18:47:14 [DEBUG   ] Finished updating ant stats in 0.593 seconds
2014-05-19 18:47:35 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 18:47:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 18:47:51 [DEBUG   ] Finished updating board flags in 1.763 seconds
2014-05-19 18:47:55 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 18:48:15 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:48:35 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:48:51 [DEBUG   ] Finished updating board flags in 1.624 seconds
2014-05-19 18:48:55 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 18:49:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 18:49:35 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 18:49:51 [DEBUG   ] Finished updating board flags in 1.613 seconds
2014-05-19 18:49:55 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 18:50:15 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 18:50:35 [DEBUG   ] Finished updating ant stats in 1.194 seconds
2014-05-19 18:50:51 [DEBUG   ] Finished updating board flags in 1.569 seconds
2014-05-19 18:50:55 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 18:51:15 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 18:51:35 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 18:51:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 18:51:51 [DEBUG   ] Finished updating board flags in 1.403 seconds
2014-05-19 18:51:55 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 18:52:06 [DEBUG   ] Got command RPT from MCS: ref #194515
2014-05-19 18:52:06 [DEBUG   ] summary = ERROR
2014-05-19 18:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194515  13 56796 67926802 A  ERRORERROR'
2014-05-19 18:52:06 [DEBUG   ] Got command RPT from MCS: ref #194516
2014-05-19 18:52:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 18:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194516  77 56796 67926803 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 18:52:15 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 18:52:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 18:52:51 [DEBUG   ] Finished updating board flags in 1.553 seconds
2014-05-19 18:52:55 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 18:53:15 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 18:53:35 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 18:53:52 [DEBUG   ] Finished updating board flags in 1.594 seconds
2014-05-19 18:53:55 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 18:54:15 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 18:54:35 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 18:54:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 18:54:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 18:54:52 [DEBUG   ] Finished updating board flags in 2.169 seconds
2014-05-19 18:54:55 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 18:55:15 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 18:55:35 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 18:55:52 [DEBUG   ] Finished updating board flags in 2.019 seconds
2014-05-19 18:55:55 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 18:56:15 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 18:56:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 18:56:52 [DEBUG   ] Finished updating board flags in 1.870 seconds
2014-05-19 18:56:55 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 18:57:06 [DEBUG   ] Got command RPT from MCS: ref #194532
2014-05-19 18:57:06 [DEBUG   ] summary = ERROR
2014-05-19 18:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194532  13 56796 68226794 A  ERRORERROR'
2014-05-19 18:57:06 [DEBUG   ] Got command RPT from MCS: ref #194533
2014-05-19 18:57:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 18:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194533  77 56796 68226795 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 18:57:15 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 18:57:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 18:57:52 [DEBUG   ] Finished updating board flags in 1.816 seconds
2014-05-19 18:57:55 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 18:58:15 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 18:58:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 18:58:52 [DEBUG   ] Finished updating board flags in 1.809 seconds
2014-05-19 18:58:55 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 18:59:15 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 18:59:35 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 18:59:52 [DEBUG   ] Finished updating board flags in 1.777 seconds
2014-05-19 18:59:55 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 19:00:15 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 19:00:35 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2014-05-19 19:00:52 [DEBUG   ] Finished updating board flags in 1.642 seconds
2014-05-19 19:00:55 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 19:01:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 19:01:35 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 19:01:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 19:01:52 [DEBUG   ] Finished updating board flags in 1.503 seconds
2014-05-19 19:01:55 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 19:02:06 [DEBUG   ] Got command RPT from MCS: ref #194549
2014-05-19 19:02:06 [DEBUG   ] summary = ERROR
2014-05-19 19:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194549  13 56796 68526798 A  ERRORERROR'
2014-05-19 19:02:06 [DEBUG   ] Got command RPT from MCS: ref #194550
2014-05-19 19:02:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 19:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194550  77 56796 68526799 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 19:02:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 19:02:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 19:02:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 19:02:52 [DEBUG   ] Finished updating board flags in 1.604 seconds
2014-05-19 19:02:55 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2014-05-19 19:03:15 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 19:03:35 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 19:03:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 19:03:52 [DEBUG   ] Finished updating board flags in 1.334 seconds
2014-05-19 19:03:55 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 19:04:15 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 19:04:35 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 19:04:53 [DEBUG   ] Finished updating board flags in 2.172 seconds
2014-05-19 19:04:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 19:05:15 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 19:05:35 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 19:05:53 [DEBUG   ] Finished updating board flags in 2.012 seconds
2014-05-19 19:05:55 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 19:06:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 19:06:35 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 19:06:53 [DEBUG   ] Finished updating board flags in 1.978 seconds
2014-05-19 19:06:55 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 19:07:06 [DEBUG   ] Got command RPT from MCS: ref #194566
2014-05-19 19:07:06 [DEBUG   ] summary = ERROR
2014-05-19 19:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194566  13 56796 68826806 A  ERRORERROR'
2014-05-19 19:07:06 [DEBUG   ] Got command RPT from MCS: ref #194567
2014-05-19 19:07:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 19:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194567  77 56796 68826807 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 19:07:15 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 19:07:35 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 19:07:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 19:07:53 [DEBUG   ] Finished updating board flags in 1.921 seconds
2014-05-19 19:07:55 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 19:08:15 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 19:08:36 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 19:08:53 [DEBUG   ] Finished updating board flags in 1.685 seconds
2014-05-19 19:08:56 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 19:09:16 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 19:09:36 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 19:09:53 [DEBUG   ] Finished updating board flags in 1.608 seconds
2014-05-19 19:09:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 19:10:16 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 19:10:36 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 19:10:53 [DEBUG   ] Finished updating board flags in 1.600 seconds
2014-05-19 19:10:56 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 19:11:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 19:11:36 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 19:11:53 [DEBUG   ] Finished updating board flags in 1.438 seconds
2014-05-19 19:11:56 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2014-05-19 19:12:06 [DEBUG   ] Got command RPT from MCS: ref #194583
2014-05-19 19:12:06 [DEBUG   ] summary = ERROR
2014-05-19 19:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194583  13 56796 69126802 A  ERRORERROR'
2014-05-19 19:12:06 [DEBUG   ] Got command RPT from MCS: ref #194584
2014-05-19 19:12:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 19:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194584  77 56796 69126803 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 19:12:16 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 19:12:36 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 19:12:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2014-05-19 19:12:53 [DEBUG   ] Finished updating board flags in 1.408 seconds
2014-05-19 19:12:56 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 19:13:16 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 19:13:36 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 19:13:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 19:13:53 [DEBUG   ] Finished updating board flags in 1.372 seconds
2014-05-19 19:13:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 19:14:16 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 19:14:36 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 19:14:54 [DEBUG   ] Finished updating board flags in 2.287 seconds
2014-05-19 19:14:56 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 19:15:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 19:15:36 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 19:15:54 [DEBUG   ] Finished updating board flags in 2.130 seconds
2014-05-19 19:15:56 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 19:16:16 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 19:16:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 19:16:54 [DEBUG   ] Finished updating board flags in 1.998 seconds
2014-05-19 19:16:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 19:17:06 [DEBUG   ] Got command RPT from MCS: ref #194600
2014-05-19 19:17:06 [DEBUG   ] summary = ERROR
2014-05-19 19:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194600  13 56796 69426806 A  ERRORERROR'
2014-05-19 19:17:06 [DEBUG   ] Got command RPT from MCS: ref #194601
2014-05-19 19:17:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 19:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194601  77 56796 69426807 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 19:17:16 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 19:17:36 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 19:17:54 [DEBUG   ] Finished updating board flags in 1.783 seconds
2014-05-19 19:17:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 19:18:16 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 19:18:36 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 19:18:54 [DEBUG   ] Finished updating board flags in 1.620 seconds
2014-05-19 19:18:56 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 19:19:16 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 19:19:36 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 19:19:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 19:19:54 [DEBUG   ] Finished updating board flags in 1.585 seconds
2014-05-19 19:19:56 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 19:20:17 [DEBUG   ] Finished updating ant stats in 1.403 seconds
2014-05-19 19:20:37 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 19:20:54 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 19:20:54 [DEBUG   ] Finished updating board flags in 1.405 seconds
2014-05-19 19:20:57 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 19:21:17 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-19 19:21:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 19:21:54 [DEBUG   ] Finished updating board flags in 1.411 seconds
2014-05-19 19:21:57 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 19:22:06 [DEBUG   ] Got command RPT from MCS: ref #194617
2014-05-19 19:22:06 [DEBUG   ] summary = ERROR
2014-05-19 19:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194617  13 56796 69726798 A  ERRORERROR'
2014-05-19 19:22:06 [DEBUG   ] Got command RPT from MCS: ref #194618
2014-05-19 19:22:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 19:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194618  77 56796 69726799 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 19:22:17 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 19:22:37 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-19 19:22:54 [DEBUG   ] Finished updating board flags in 1.390 seconds
2014-05-19 19:22:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 19:23:17 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 19:23:37 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 19:23:55 [DEBUG   ] Finished updating board flags in 2.221 seconds
2014-05-19 19:23:57 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 19:24:17 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 19:24:37 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 19:24:55 [DEBUG   ] Finished updating board flags in 2.177 seconds
2014-05-19 19:24:57 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 19:25:17 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 19:25:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 19:25:55 [DEBUG   ] Finished updating board flags in 2.033 seconds
2014-05-19 19:25:57 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 19:26:17 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 19:26:38 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-19 19:26:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 19:26:55 [DEBUG   ] Finished updating board flags in 1.980 seconds
2014-05-19 19:26:58 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 19:27:06 [DEBUG   ] Got command RPT from MCS: ref #194634
2014-05-19 19:27:06 [DEBUG   ] summary = ERROR
2014-05-19 19:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194634  13 56796 70026810 A  ERRORERROR'
2014-05-19 19:27:06 [DEBUG   ] Got command RPT from MCS: ref #194635
2014-05-19 19:27:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 19:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194635  77 56796 70026811 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 19:27:18 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 19:27:38 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 19:27:55 [DEBUG   ] Finished updating board flags in 1.794 seconds
2014-05-19 19:27:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 19:28:18 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 19:28:38 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 19:28:55 [DEBUG   ] Finished updating board flags in 1.608 seconds
2014-05-19 19:28:58 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 19:29:18 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 19:29:38 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 19:29:55 [DEBUG   ] Finished updating board flags in 1.630 seconds
2014-05-19 19:29:58 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 19:30:18 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 19:30:38 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 19:30:55 [DEBUG   ] Finished updating board flags in 1.562 seconds
2014-05-19 19:30:58 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2014-05-19 19:31:19 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 19:31:39 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 19:31:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 19:31:55 [DEBUG   ] Finished updating board flags in 1.424 seconds
2014-05-19 19:31:59 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 19:32:06 [DEBUG   ] Got command RPT from MCS: ref #194651
2014-05-19 19:32:06 [DEBUG   ] summary = ERROR
2014-05-19 19:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194651  13 56796 70326806 A  ERRORERROR'
2014-05-19 19:32:06 [DEBUG   ] Got command RPT from MCS: ref #194652
2014-05-19 19:32:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 19:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194652  77 56796 70326807 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 19:32:19 [DEBUG   ] Finished updating ant stats in 1.210 seconds
2014-05-19 19:32:39 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 19:32:55 [DEBUG   ] Finished updating board flags in 1.397 seconds
2014-05-19 19:32:59 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 19:33:19 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 19:33:39 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 19:33:56 [DEBUG   ] Finished updating board flags in 2.193 seconds
2014-05-19 19:33:59 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2014-05-19 19:34:19 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 19:34:39 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 19:34:56 [DEBUG   ] Finished updating board flags in 2.056 seconds
2014-05-19 19:34:59 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 19:35:19 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2014-05-19 19:35:39 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 19:35:56 [DEBUG   ] Finished updating board flags in 1.934 seconds
2014-05-19 19:35:59 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 19:36:19 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2014-05-19 19:36:39 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 19:36:56 [DEBUG   ] Finished updating board flags in 1.809 seconds
2014-05-19 19:36:59 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 19:37:06 [DEBUG   ] Got command RPT from MCS: ref #194668
2014-05-19 19:37:06 [DEBUG   ] summary = ERROR
2014-05-19 19:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194668  13 56796 70626806 A  ERRORERROR'
2014-05-19 19:37:06 [DEBUG   ] Got command RPT from MCS: ref #194669
2014-05-19 19:37:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 19:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194669  77 56796 70626807 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 19:37:19 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 19:37:40 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 19:37:56 [DEBUG   ] Finished updating board flags in 1.790 seconds
2014-05-19 19:38:00 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 19:38:20 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 19:38:40 [DEBUG   ] Finished updating ant stats in 1.210 seconds
2014-05-19 19:38:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 19:38:56 [DEBUG   ] Finished updating board flags in 1.619 seconds
2014-05-19 19:39:00 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 19:39:20 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 19:39:40 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 19:39:56 [DEBUG   ] Finished updating board flags in 1.585 seconds
2014-05-19 19:40:00 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 19:40:20 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 19:40:40 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 19:40:56 [DEBUG   ] Finished updating board flags in 1.424 seconds
2014-05-19 19:41:00 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 19:41:20 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2014-05-19 19:41:40 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 19:41:56 [DEBUG   ] Finished updating board flags in 1.382 seconds
2014-05-19 19:42:00 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2014-05-19 19:42:06 [DEBUG   ] Got command RPT from MCS: ref #194685
2014-05-19 19:42:06 [DEBUG   ] summary = ERROR
2014-05-19 19:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194685  13 56796 70926802 A  ERRORERROR'
2014-05-19 19:42:06 [DEBUG   ] Got command RPT from MCS: ref #194686
2014-05-19 19:42:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 19:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194686  77 56796 70926803 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 19:42:20 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 19:42:41 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 19:42:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 19:42:56 [DEBUG   ] Finished updating board flags in 1.367 seconds
2014-05-19 19:43:01 [DEBUG   ] Finished updating ant stats in 1.403 seconds
2014-05-19 19:43:21 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 19:43:41 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 19:43:57 [DEBUG   ] Finished updating board flags in 2.057 seconds
2014-05-19 19:44:01 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 19:44:21 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 19:44:41 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 19:44:57 [DEBUG   ] Finished updating board flags in 1.989 seconds
2014-05-19 19:45:01 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2014-05-19 19:45:21 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 19:45:41 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 19:45:57 [DEBUG   ] Finished updating board flags in 1.803 seconds
2014-05-19 19:46:01 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 19:46:21 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 19:46:41 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2014-05-19 19:46:57 [DEBUG   ] Finished updating board flags in 1.814 seconds
2014-05-19 19:47:01 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 19:47:06 [DEBUG   ] Got command RPT from MCS: ref #194702
2014-05-19 19:47:06 [DEBUG   ] summary = ERROR
2014-05-19 19:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194702  13 56796 71226810 A  ERRORERROR'
2014-05-19 19:47:06 [DEBUG   ] Got command RPT from MCS: ref #194703
2014-05-19 19:47:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 19:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194703  77 56796 71226811 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 19:47:21 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 19:47:42 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 19:47:57 [DEBUG   ] Finished updating board flags in 1.790 seconds
2014-05-19 19:48:02 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 19:48:22 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 19:48:42 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 19:48:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 19:48:57 [DEBUG   ] Finished updating board flags in 1.611 seconds
2014-05-19 19:49:02 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 19:49:22 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 19:49:42 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 19:49:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 19:49:57 [DEBUG   ] Finished updating board flags in 1.589 seconds
2014-05-19 19:50:02 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2014-05-19 19:50:22 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 19:50:42 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 19:50:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 19:50:57 [DEBUG   ] Finished updating board flags in 1.418 seconds
2014-05-19 19:51:02 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 19:51:22 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 19:51:42 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 19:51:57 [DEBUG   ] Finished updating board flags in 1.382 seconds
2014-05-19 19:52:02 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 19:52:06 [DEBUG   ] Got command RPT from MCS: ref #194719
2014-05-19 19:52:06 [DEBUG   ] summary = ERROR
2014-05-19 19:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194719  13 56796 71526814 A  ERRORERROR'
2014-05-19 19:52:06 [DEBUG   ] Got command RPT from MCS: ref #194720
2014-05-19 19:52:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 19:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194720  77 56796 71526815 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 19:52:23 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 19:52:43 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 19:52:57 [DEBUG   ] Finished updating board flags in 1.369 seconds
2014-05-19 19:53:03 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 19:53:23 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 19:53:43 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 19:53:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 19:53:58 [DEBUG   ] Finished updating board flags in 2.053 seconds
2014-05-19 19:54:03 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 19:54:23 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 19:54:43 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 19:54:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 19:54:58 [DEBUG   ] Finished updating board flags in 1.987 seconds
2014-05-19 19:55:03 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 19:55:23 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 19:55:43 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 19:55:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 19:55:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 19:55:58 [DEBUG   ] Finished updating board flags in 1.816 seconds
2014-05-19 19:56:03 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 19:56:23 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 19:56:43 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 19:56:58 [DEBUG   ] Finished updating board flags in 1.788 seconds
2014-05-19 19:57:04 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2014-05-19 19:57:06 [DEBUG   ] Got command RPT from MCS: ref #194736
2014-05-19 19:57:06 [DEBUG   ] summary = ERROR
2014-05-19 19:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194736  13 56796 71826806 A  ERRORERROR'
2014-05-19 19:57:06 [DEBUG   ] Got command RPT from MCS: ref #194737
2014-05-19 19:57:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 19:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194737  77 56796 71826807 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 19:57:24 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2014-05-19 19:57:44 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 19:57:58 [DEBUG   ] Finished updating board flags in 1.625 seconds
2014-05-19 19:58:04 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 19:58:24 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-19 19:58:44 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 19:58:58 [DEBUG   ] Finished updating board flags in 1.578 seconds
2014-05-19 19:59:04 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 19:59:24 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 19:59:44 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2014-05-19 19:59:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 19:59:58 [DEBUG   ] Finished updating board flags in 1.420 seconds
2014-05-19 20:00:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 20:00:24 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2014-05-19 20:00:45 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2014-05-19 20:00:58 [DEBUG   ] Finished updating board flags in 1.388 seconds
2014-05-19 20:01:05 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2014-05-19 20:01:25 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 20:01:45 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:01:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-19 20:01:58 [DEBUG   ] Finished updating board flags in 1.374 seconds
2014-05-19 20:02:05 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 20:02:06 [DEBUG   ] Got command RPT from MCS: ref #194753
2014-05-19 20:02:06 [DEBUG   ] summary = ERROR
2014-05-19 20:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194753  13 56796 72126814 A  ERRORERROR'
2014-05-19 20:02:06 [DEBUG   ] Got command RPT from MCS: ref #194754
2014-05-19 20:02:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 20:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194754  77 56796 72126815 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 20:02:25 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 20:02:45 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 20:02:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 20:02:59 [DEBUG   ] Finished updating board flags in 2.044 seconds
2014-05-19 20:03:05 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 20:03:25 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 20:03:45 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 20:03:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:03:59 [DEBUG   ] Finished updating board flags in 1.982 seconds
2014-05-19 20:04:05 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 20:04:25 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-19 20:04:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 20:04:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 20:04:59 [DEBUG   ] Finished updating board flags in 1.821 seconds
2014-05-19 20:05:05 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 20:05:25 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 20:05:45 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 20:05:59 [DEBUG   ] Finished updating board flags in 1.797 seconds
2014-05-19 20:06:06 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 20:06:26 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 20:06:46 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 20:06:59 [DEBUG   ] Finished updating board flags in 1.593 seconds
2014-05-19 20:07:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 20:07:06 [DEBUG   ] Got command RPT from MCS: ref #194770
2014-05-19 20:07:06 [DEBUG   ] summary = ERROR
2014-05-19 20:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194770  13 56796 72426810 A  ERRORERROR'
2014-05-19 20:07:06 [DEBUG   ] Got command RPT from MCS: ref #194771
2014-05-19 20:07:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 20:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194771  77 56796 72426811 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 20:07:26 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 20:07:46 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 20:07:59 [DEBUG   ] Finished updating board flags in 1.411 seconds
2014-05-19 20:08:06 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:08:26 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-19 20:08:46 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 20:08:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:08:59 [DEBUG   ] Finished updating board flags in 1.391 seconds
2014-05-19 20:09:06 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 20:09:26 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 20:09:46 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 20:09:59 [DEBUG   ] Finished updating board flags in 1.377 seconds
2014-05-19 20:10:06 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 20:10:26 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-19 20:10:46 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 20:10:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 20:11:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 20:11:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:11:00 [DEBUG   ] Finished updating board flags in 2.038 seconds
2014-05-19 20:11:06 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 20:11:26 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 20:11:46 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-19 20:12:00 [DEBUG   ] Finished updating board flags in 2.033 seconds
2014-05-19 20:12:06 [DEBUG   ] Got command RPT from MCS: ref #194787
2014-05-19 20:12:06 [DEBUG   ] summary = ERROR
2014-05-19 20:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194787  13 56796 72726818 A  ERRORERROR'
2014-05-19 20:12:06 [DEBUG   ] Got command RPT from MCS: ref #194788
2014-05-19 20:12:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 20:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194788  77 56796 72726819 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 20:12:07 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-19 20:12:27 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 20:12:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 20:13:00 [DEBUG   ] Finished updating board flags in 1.977 seconds
2014-05-19 20:13:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 20:13:27 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 20:13:47 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 20:13:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 20:14:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:14:00 [DEBUG   ] Finished updating board flags in 1.797 seconds
2014-05-19 20:14:07 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:14:27 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:14:47 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-19 20:15:00 [DEBUG   ] Finished updating board flags in 1.592 seconds
2014-05-19 20:15:07 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 20:15:27 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 20:15:47 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 20:16:00 [DEBUG   ] Finished updating board flags in 1.409 seconds
2014-05-19 20:16:07 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 20:16:27 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 20:16:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 20:17:00 [DEBUG   ] Finished updating board flags in 1.401 seconds
2014-05-19 20:17:06 [DEBUG   ] Got command RPT from MCS: ref #194804
2014-05-19 20:17:06 [DEBUG   ] summary = ERROR
2014-05-19 20:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194804  13 56796 73026810 A  ERRORERROR'
2014-05-19 20:17:06 [DEBUG   ] Got command RPT from MCS: ref #194805
2014-05-19 20:17:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 20:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194805  77 56796 73026811 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 20:17:07 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 20:17:27 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 20:17:47 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 20:18:00 [DEBUG   ] Finished updating board flags in 1.391 seconds
2014-05-19 20:18:08 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2014-05-19 20:18:28 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-19 20:18:48 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 20:19:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 20:19:01 [DEBUG   ] Finished updating board flags in 2.210 seconds
2014-05-19 20:19:08 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:19:28 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:19:48 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 20:20:01 [DEBUG   ] Finished updating board flags in 2.198 seconds
2014-05-19 20:20:08 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 20:20:28 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 20:20:48 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 20:21:01 [DEBUG   ] Finished updating board flags in 2.006 seconds
2014-05-19 20:21:08 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 20:21:28 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 20:21:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 20:22:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 20:22:01 [DEBUG   ] Finished updating board flags in 2.008 seconds
2014-05-19 20:22:06 [DEBUG   ] Got command RPT from MCS: ref #194821
2014-05-19 20:22:06 [DEBUG   ] summary = ERROR
2014-05-19 20:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194821  13 56796 73326818 A  ERRORERROR'
2014-05-19 20:22:06 [DEBUG   ] Got command RPT from MCS: ref #194822
2014-05-19 20:22:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 20:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194822  77 56796 73326819 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 20:22:08 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 20:22:28 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 20:22:48 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 20:23:01 [DEBUG   ] Finished updating board flags in 2.051 seconds
2014-05-19 20:23:08 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-19 20:23:29 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 20:23:49 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 20:24:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-19 20:24:01 [DEBUG   ] Finished updating board flags in 1.933 seconds
2014-05-19 20:24:09 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 20:24:29 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 20:24:49 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 20:25:01 [DEBUG   ] Finished updating board flags in 1.819 seconds
2014-05-19 20:25:09 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 20:25:29 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:25:49 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:26:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 20:26:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:26:01 [DEBUG   ] Finished updating board flags in 1.789 seconds
2014-05-19 20:26:09 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 20:26:29 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:26:49 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:27:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:27:01 [DEBUG   ] Finished updating board flags in 1.614 seconds
2014-05-19 20:27:06 [DEBUG   ] Got command RPT from MCS: ref #194838
2014-05-19 20:27:06 [DEBUG   ] summary = ERROR
2014-05-19 20:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194838  13 56796 73626822 A  ERRORERROR'
2014-05-19 20:27:06 [DEBUG   ] Got command RPT from MCS: ref #194839
2014-05-19 20:27:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 20:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194839  77 56796 73626823 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 20:27:09 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:27:29 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 20:27:49 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:28:01 [DEBUG   ] Finished updating board flags in 1.641 seconds
2014-05-19 20:28:09 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 20:28:29 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:28:49 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 20:29:01 [DEBUG   ] Finished updating board flags in 1.561 seconds
2014-05-19 20:29:09 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:29:29 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:29:49 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:30:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 20:30:01 [DEBUG   ] Finished updating board flags in 1.521 seconds
2014-05-19 20:30:09 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 20:30:29 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 20:30:49 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 20:31:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 20:31:01 [DEBUG   ] Finished updating board flags in 1.529 seconds
2014-05-19 20:31:09 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 20:31:29 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 20:31:49 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-19 20:32:02 [DEBUG   ] Finished updating board flags in 2.207 seconds
2014-05-19 20:32:06 [DEBUG   ] Got command RPT from MCS: ref #194855
2014-05-19 20:32:06 [DEBUG   ] summary = ERROR
2014-05-19 20:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194855  13 56796 73926814 A  ERRORERROR'
2014-05-19 20:32:06 [DEBUG   ] Got command RPT from MCS: ref #194856
2014-05-19 20:32:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 20:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194856  77 56796 73926815 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 20:32:09 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 20:32:29 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 20:32:49 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 20:33:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 20:33:02 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:33:02 [DEBUG   ] Finished updating board flags in 2.172 seconds
2014-05-19 20:33:09 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 20:33:30 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 20:33:50 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 20:34:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 20:34:02 [DEBUG   ] Finished updating board flags in 1.974 seconds
2014-05-19 20:34:10 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 20:34:30 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 20:34:50 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-19 20:35:02 [DEBUG   ] Finished updating board flags in 1.797 seconds
2014-05-19 20:35:10 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 20:35:30 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:35:50 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:36:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 20:36:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:36:02 [DEBUG   ] Finished updating board flags in 1.598 seconds
2014-05-19 20:36:10 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:36:30 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 20:36:50 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:37:02 [DEBUG   ] Finished updating board flags in 1.432 seconds
2014-05-19 20:37:06 [DEBUG   ] Got command RPT from MCS: ref #194872
2014-05-19 20:37:06 [DEBUG   ] summary = ERROR
2014-05-19 20:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194872  13 56796 74226818 A  ERRORERROR'
2014-05-19 20:37:06 [DEBUG   ] Got command RPT from MCS: ref #194873
2014-05-19 20:37:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 20:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194873  77 56796 74226819 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 20:37:10 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:37:30 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:37:50 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 20:38:02 [DEBUG   ] Finished updating board flags in 1.424 seconds
2014-05-19 20:38:10 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-19 20:38:30 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 20:38:50 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 20:39:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 20:39:02 [DEBUG   ] Finished updating board flags in 1.431 seconds
2014-05-19 20:39:10 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 20:39:30 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-19 20:39:51 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 20:40:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:40:02 [DEBUG   ] Finished updating board flags in 1.426 seconds
2014-05-19 20:40:11 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 20:40:31 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 20:40:51 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 20:41:02 [DEBUG   ] Finished updating board flags in 1.425 seconds
2014-05-19 20:41:11 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 20:41:31 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-19 20:41:51 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 20:42:02 [DEBUG   ] Finished updating board flags in 1.437 seconds
2014-05-19 20:42:06 [DEBUG   ] Got command RPT from MCS: ref #194889
2014-05-19 20:42:06 [DEBUG   ] summary = ERROR
2014-05-19 20:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194889  13 56796 74526834 A  ERRORERROR'
2014-05-19 20:42:06 [DEBUG   ] Got command RPT from MCS: ref #194890
2014-05-19 20:42:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 20:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194890  77 56796 74526835 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 20:42:11 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:42:31 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 20:42:51 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:43:02 [DEBUG   ] Finished updating board flags in 1.517 seconds
2014-05-19 20:43:11 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:43:31 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:43:51 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:44:03 [DEBUG   ] Finished updating board flags in 2.236 seconds
2014-05-19 20:44:11 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:44:31 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:44:51 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 20:45:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:45:03 [DEBUG   ] Finished updating board flags in 2.232 seconds
2014-05-19 20:45:11 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:45:31 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:45:51 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 20:46:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 20:46:03 [DEBUG   ] Finished updating board flags in 2.062 seconds
2014-05-19 20:46:11 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 20:46:31 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:46:51 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:47:03 [DEBUG   ] Finished updating board flags in 2.024 seconds
2014-05-19 20:47:06 [DEBUG   ] Got command RPT from MCS: ref #194906
2014-05-19 20:47:06 [DEBUG   ] summary = ERROR
2014-05-19 20:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194906  13 56796 74826834 A  ERRORERROR'
2014-05-19 20:47:06 [DEBUG   ] Got command RPT from MCS: ref #194907
2014-05-19 20:47:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 20:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194907  77 56796 74826835 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 20:47:11 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 20:47:31 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:47:51 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:48:03 [DEBUG   ] Finished updating board flags in 1.979 seconds
2014-05-19 20:48:11 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 20:48:31 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 20:48:51 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 20:49:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 20:49:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:49:03 [DEBUG   ] Finished updating board flags in 1.826 seconds
2014-05-19 20:49:11 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 20:49:31 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 20:49:51 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 20:50:03 [DEBUG   ] Finished updating board flags in 1.785 seconds
2014-05-19 20:50:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 20:50:31 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 20:50:51 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 20:51:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-19 20:51:03 [DEBUG   ] Finished updating board flags in 1.700 seconds
2014-05-19 20:51:11 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2014-05-19 20:51:31 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 20:51:51 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 20:52:03 [DEBUG   ] Finished updating board flags in 1.490 seconds
2014-05-19 20:52:06 [DEBUG   ] Got command RPT from MCS: ref #194923
2014-05-19 20:52:06 [DEBUG   ] summary = ERROR
2014-05-19 20:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194923  13 56796 75126834 A  ERRORERROR'
2014-05-19 20:52:06 [DEBUG   ] Got command RPT from MCS: ref #194924
2014-05-19 20:52:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 20:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194924  77 56796 75126835 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 20:52:11 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 20:52:31 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 20:52:51 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 20:53:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:53:03 [DEBUG   ] Finished updating board flags in 1.490 seconds
2014-05-19 20:53:11 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 20:53:31 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 20:53:51 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 20:54:03 [DEBUG   ] Finished updating board flags in 1.500 seconds
2014-05-19 20:54:11 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 20:54:31 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 20:54:51 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 20:55:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 20:55:03 [DEBUG   ] Finished updating board flags in 1.499 seconds
2014-05-19 20:55:11 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 20:55:31 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 20:55:51 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 20:56:04 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:56:04 [DEBUG   ] Finished updating board flags in 2.224 seconds
2014-05-19 20:56:11 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-19 20:56:32 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-19 20:56:52 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 20:57:04 [DEBUG   ] Finished updating board flags in 2.211 seconds
2014-05-19 20:57:06 [DEBUG   ] Got command RPT from MCS: ref #194940
2014-05-19 20:57:06 [DEBUG   ] summary = ERROR
2014-05-19 20:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194940  13 56796 75426838 A  ERRORERROR'
2014-05-19 20:57:06 [DEBUG   ] Got command RPT from MCS: ref #194941
2014-05-19 20:57:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 20:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194941  77 56796 75426839 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 20:57:12 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 20:57:32 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 20:57:52 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 20:58:04 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 20:58:04 [DEBUG   ] Finished updating board flags in 2.112 seconds
2014-05-19 20:58:12 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 20:58:32 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 20:58:52 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 20:59:04 [DEBUG   ] Finished updating board flags in 1.991 seconds
2014-05-19 20:59:12 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 20:59:32 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 20:59:52 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 21:00:04 [DEBUG   ] Finished updating board flags in 1.787 seconds
2014-05-19 21:00:12 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 21:00:32 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 21:00:52 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 21:01:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 21:01:04 [DEBUG   ] Finished updating board flags in 1.622 seconds
2014-05-19 21:01:12 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 21:01:33 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-19 21:01:53 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-19 21:02:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 21:02:04 [DEBUG   ] Finished updating board flags in 1.595 seconds
2014-05-19 21:02:06 [DEBUG   ] Got command RPT from MCS: ref #194957
2014-05-19 21:02:06 [DEBUG   ] summary = ERROR
2014-05-19 21:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194957  13 56796 75726842 A  ERRORERROR'
2014-05-19 21:02:06 [DEBUG   ] Got command RPT from MCS: ref #194958
2014-05-19 21:02:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 21:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194958  77 56796 75726843 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 21:02:13 [DEBUG   ] Finished updating ant stats in 1.195 seconds
2014-05-19 21:02:33 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 21:02:53 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 21:03:04 [DEBUG   ] Finished updating board flags in 1.397 seconds
2014-05-19 21:03:13 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 21:03:33 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2014-05-19 21:03:53 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 21:04:05 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 21:04:05 [DEBUG   ] Finished updating board flags in 2.216 seconds
2014-05-19 21:04:13 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 21:04:33 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 21:04:53 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 21:05:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 21:05:05 [DEBUG   ] Finished updating board flags in 2.186 seconds
2014-05-19 21:05:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 21:05:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 21:05:53 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 21:06:05 [DEBUG   ] Finished updating board flags in 2.018 seconds
2014-05-19 21:06:13 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2014-05-19 21:06:34 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 21:06:54 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2014-05-19 21:07:05 [DEBUG   ] Finished updating board flags in 2.006 seconds
2014-05-19 21:07:06 [DEBUG   ] Got command RPT from MCS: ref #194974
2014-05-19 21:07:06 [DEBUG   ] summary = ERROR
2014-05-19 21:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194974  13 56796 76026834 A  ERRORERROR'
2014-05-19 21:07:06 [DEBUG   ] Got command RPT from MCS: ref #194975
2014-05-19 21:07:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 21:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194975  77 56796 76026835 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 21:07:14 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-19 21:07:34 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 21:07:54 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 21:08:05 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 21:08:05 [DEBUG   ] Finished updating board flags in 1.980 seconds
2014-05-19 21:08:14 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 21:08:34 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-19 21:08:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 21:09:05 [DEBUG   ] Finished updating board flags in 1.818 seconds
2014-05-19 21:09:14 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 21:09:34 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 21:09:54 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-19 21:10:05 [DEBUG   ] Finished updating board flags in 1.784 seconds
2014-05-19 21:10:14 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 21:10:35 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 21:10:55 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 21:11:05 [DEBUG   ] Finished updating board flags in 1.614 seconds
2014-05-19 21:11:15 [DEBUG   ] Finished updating ant stats in 1.195 seconds
2014-05-19 21:11:35 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-19 21:11:55 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 21:12:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 21:12:05 [DEBUG   ] Finished updating board flags in 1.606 seconds
2014-05-19 21:12:06 [DEBUG   ] Got command RPT from MCS: ref #194991
2014-05-19 21:12:06 [DEBUG   ] summary = ERROR
2014-05-19 21:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194991  13 56796 76326846 A  ERRORERROR'
2014-05-19 21:12:06 [DEBUG   ] Got command RPT from MCS: ref #194992
2014-05-19 21:12:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 21:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   194992  77 56796 76326847 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 21:12:15 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 21:12:35 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 21:12:55 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 21:13:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-19 21:13:05 [DEBUG   ] Finished updating board flags in 1.590 seconds
2014-05-19 21:13:15 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 21:13:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 21:13:55 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 21:14:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 21:14:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 21:14:05 [DEBUG   ] Finished updating board flags in 1.405 seconds
2014-05-19 21:14:15 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2014-05-19 21:14:36 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2014-05-19 21:14:56 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 21:15:05 [DEBUG   ] Finished updating board flags in 1.406 seconds
2014-05-19 21:15:16 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:15:36 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 21:15:56 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 21:16:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 21:16:05 [DEBUG   ] Finished updating board flags in 1.388 seconds
2014-05-19 21:16:16 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-19 21:16:36 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 21:16:56 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 21:17:06 [DEBUG   ] Finished updating board flags in 2.213 seconds
2014-05-19 21:17:06 [DEBUG   ] Got command RPT from MCS: ref #195008
2014-05-19 21:17:06 [DEBUG   ] summary = ERROR
2014-05-19 21:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195008  13 56796 76626838 A  ERRORERROR'
2014-05-19 21:17:06 [DEBUG   ] Got command RPT from MCS: ref #195009
2014-05-19 21:17:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 21:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195009  77 56796 76626839 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 21:17:16 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 21:17:36 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2014-05-19 21:17:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 21:18:06 [DEBUG   ] Finished updating board flags in 2.187 seconds
2014-05-19 21:18:16 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 21:18:36 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 21:18:56 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 21:19:06 [DEBUG   ] Finished updating board flags in 2.013 seconds
2014-05-19 21:19:16 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-19 21:19:36 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 21:19:56 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 21:20:06 [DEBUG   ] Finished updating board flags in 1.992 seconds
2014-05-19 21:20:17 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 21:20:37 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 21:20:57 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 21:21:06 [DEBUG   ] Finished updating board flags in 1.805 seconds
2014-05-19 21:21:17 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 21:21:37 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 21:21:57 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 21:22:06 [DEBUG   ] Finished updating board flags in 1.808 seconds
2014-05-19 21:22:06 [DEBUG   ] Got command RPT from MCS: ref #195025
2014-05-19 21:22:06 [DEBUG   ] summary = ERROR
2014-05-19 21:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195025  13 56796 76926838 A  ERRORERROR'
2014-05-19 21:22:06 [DEBUG   ] Got command RPT from MCS: ref #195026
2014-05-19 21:22:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 21:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195026  77 56796 76926839 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 21:22:17 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 21:22:37 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-19 21:22:57 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 21:23:05 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 21:23:06 [DEBUG   ] Finished updating board flags in 1.786 seconds
2014-05-19 21:23:17 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 21:23:37 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 21:23:57 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 21:24:06 [DEBUG   ] Finished updating board flags in 1.616 seconds
2014-05-19 21:24:17 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 21:24:37 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 21:24:57 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 21:25:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 21:25:06 [DEBUG   ] Finished updating board flags in 1.653 seconds
2014-05-19 21:25:17 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 21:25:37 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 21:25:57 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 21:26:06 [DEBUG   ] Finished updating board flags in 1.535 seconds
2014-05-19 21:26:17 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 21:26:38 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 21:26:58 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 21:27:06 [DEBUG   ] Finished updating board flags in 1.414 seconds
2014-05-19 21:27:06 [DEBUG   ] Got command RPT from MCS: ref #195042
2014-05-19 21:27:06 [DEBUG   ] summary = ERROR
2014-05-19 21:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195042  13 56796 77226846 A  ERRORERROR'
2014-05-19 21:27:06 [DEBUG   ] Got command RPT from MCS: ref #195043
2014-05-19 21:27:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 21:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195043  77 56796 77226847 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 21:27:18 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-19 21:27:38 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 21:27:58 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 21:28:06 [DEBUG   ] Finished updating board flags in 1.392 seconds
2014-05-19 21:28:18 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2014-05-19 21:28:38 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 21:28:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 21:29:07 [DEBUG   ] Finished updating board flags in 2.217 seconds
2014-05-19 21:29:18 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 21:29:38 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 21:29:58 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 21:30:07 [DEBUG   ] Finished updating board flags in 2.183 seconds
2014-05-19 21:30:18 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 21:30:38 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-19 21:30:58 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 21:31:07 [DEBUG   ] Finished updating board flags in 2.023 seconds
2014-05-19 21:31:19 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 21:31:39 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 21:31:59 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 21:32:06 [DEBUG   ] Got command RPT from MCS: ref #195059
2014-05-19 21:32:06 [DEBUG   ] summary = ERROR
2014-05-19 21:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195059  13 56796 77526842 A  ERRORERROR'
2014-05-19 21:32:06 [DEBUG   ] Got command RPT from MCS: ref #195060
2014-05-19 21:32:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 21:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195060  77 56796 77526843 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 21:32:07 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 21:32:07 [DEBUG   ] Finished updating board flags in 1.983 seconds
2014-05-19 21:32:19 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2014-05-19 21:32:39 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 21:32:59 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 21:33:07 [DEBUG   ] Finished updating board flags in 1.818 seconds
2014-05-19 21:33:19 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2014-05-19 21:33:39 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 21:33:59 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 21:34:07 [DEBUG   ] Finished updating board flags in 1.789 seconds
2014-05-19 21:34:19 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 21:34:39 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 21:34:59 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 21:35:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 21:35:07 [DEBUG   ] Finished updating board flags in 1.632 seconds
2014-05-19 21:35:19 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 21:35:39 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 21:35:59 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 21:36:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 21:36:07 [DEBUG   ] Finished updating board flags in 1.639 seconds
2014-05-19 21:36:19 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 21:36:39 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 21:36:59 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 21:37:06 [DEBUG   ] Got command RPT from MCS: ref #195076
2014-05-19 21:37:06 [DEBUG   ] summary = ERROR
2014-05-19 21:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195076  13 56796 77826850 A  ERRORERROR'
2014-05-19 21:37:06 [DEBUG   ] Got command RPT from MCS: ref #195077
2014-05-19 21:37:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 21:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195077  77 56796 77826851 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 21:37:07 [DEBUG   ] Finished updating board flags in 1.541 seconds
2014-05-19 21:37:19 [DEBUG   ] Finished updating ant stats in 0.780 seconds
2014-05-19 21:37:39 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 21:37:59 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 21:38:07 [DEBUG   ] Finished updating board flags in 1.457 seconds
2014-05-19 21:38:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 21:38:39 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 21:38:59 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 21:39:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 21:39:07 [DEBUG   ] Finished updating board flags in 1.523 seconds
2014-05-19 21:39:19 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 21:39:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 21:39:59 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 21:40:07 [DEBUG   ] Finished updating board flags in 1.563 seconds
2014-05-19 21:40:19 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 21:40:39 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 21:40:59 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 21:41:08 [DEBUG   ] Finished updating board flags in 2.147 seconds
2014-05-19 21:41:19 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2014-05-19 21:41:40 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 21:42:00 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 21:42:06 [DEBUG   ] Got command RPT from MCS: ref #195093
2014-05-19 21:42:06 [DEBUG   ] summary = ERROR
2014-05-19 21:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195093  13 56796 78126846 A  ERRORERROR'
2014-05-19 21:42:06 [DEBUG   ] Got command RPT from MCS: ref #195094
2014-05-19 21:42:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 21:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195094  77 56796 78126847 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 21:42:08 [DEBUG   ] Finished updating board flags in 1.924 seconds
2014-05-19 21:42:20 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:42:40 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:43:00 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 21:43:08 [DEBUG   ] Finished updating board flags in 1.820 seconds
2014-05-19 21:43:20 [DEBUG   ] Finished updating ant stats in 1.194 seconds
2014-05-19 21:43:40 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 21:44:00 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 21:44:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 21:44:08 [DEBUG   ] Finished updating board flags in 1.792 seconds
2014-05-19 21:44:20 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 21:44:40 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 21:45:00 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 21:45:08 [DEBUG   ] Finished updating board flags in 1.611 seconds
2014-05-19 21:45:20 [DEBUG   ] Finished updating ant stats in 0.987 seconds
2014-05-19 21:45:40 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 21:46:00 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 21:46:08 [DEBUG   ] Finished updating board flags in 1.586 seconds
2014-05-19 21:46:20 [DEBUG   ] Finished updating ant stats in 0.793 seconds
2014-05-19 21:46:40 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 21:47:00 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 21:47:06 [DEBUG   ] Got command RPT from MCS: ref #195110
2014-05-19 21:47:06 [DEBUG   ] summary = ERROR
2014-05-19 21:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195110  13 56796 78426854 A  ERRORERROR'
2014-05-19 21:47:06 [DEBUG   ] Got command RPT from MCS: ref #195111
2014-05-19 21:47:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 21:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195111  77 56796 78426855 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 21:47:08 [DEBUG   ] Finished updating board flags in 1.415 seconds
2014-05-19 21:47:20 [DEBUG   ] Finished updating ant stats in 0.590 seconds
2014-05-19 21:47:41 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-19 21:48:01 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-19 21:48:08 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 21:48:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 21:48:08 [DEBUG   ] Finished updating board flags in 1.379 seconds
2014-05-19 21:48:21 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 21:48:41 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:49:01 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 21:49:08 [DEBUG   ] Finished updating board flags in 1.366 seconds
2014-05-19 21:49:21 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 21:49:41 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 21:50:01 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 21:50:09 [DEBUG   ] Finished updating board flags in 2.079 seconds
2014-05-19 21:50:21 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:50:41 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 21:51:01 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 21:51:09 [DEBUG   ] Finished updating board flags in 1.988 seconds
2014-05-19 21:51:21 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 21:51:41 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:52:01 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 21:52:06 [DEBUG   ] Got command RPT from MCS: ref #195127
2014-05-19 21:52:06 [DEBUG   ] summary = ERROR
2014-05-19 21:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195127  13 56796 78726850 A  ERRORERROR'
2014-05-19 21:52:06 [DEBUG   ] Got command RPT from MCS: ref #195128
2014-05-19 21:52:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 21:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195128  77 56796 78726851 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 21:52:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 21:52:09 [DEBUG   ] Finished updating board flags in 1.840 seconds
2014-05-19 21:52:21 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 21:52:41 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:53:01 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:53:09 [DEBUG   ] Finished updating board flags in 1.797 seconds
2014-05-19 21:53:21 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 21:53:41 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:54:01 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 21:54:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 21:54:09 [DEBUG   ] Finished updating board flags in 1.625 seconds
2014-05-19 21:54:21 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:54:41 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 21:55:01 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 21:55:09 [DEBUG   ] Finished updating board flags in 1.527 seconds
2014-05-19 21:55:21 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 21:55:41 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:56:01 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 21:56:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 21:56:09 [DEBUG   ] Finished updating board flags in 1.492 seconds
2014-05-19 21:56:21 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:56:41 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 21:57:01 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:57:06 [DEBUG   ] Got command RPT from MCS: ref #195144
2014-05-19 21:57:06 [DEBUG   ] summary = ERROR
2014-05-19 21:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195144  13 56796 79026850 A  ERRORERROR'
2014-05-19 21:57:06 [DEBUG   ] Got command RPT from MCS: ref #195145
2014-05-19 21:57:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 21:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195145  77 56796 79026851 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 21:57:09 [DEBUG   ] Finished updating board flags in 1.567 seconds
2014-05-19 21:57:21 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 21:57:41 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:58:01 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 21:58:10 [DEBUG   ] Finished updating board flags in 2.224 seconds
2014-05-19 21:58:21 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:58:41 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 21:59:01 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 21:59:10 [DEBUG   ] Finished updating board flags in 2.180 seconds
2014-05-19 21:59:21 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 21:59:41 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 22:00:01 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 22:00:10 [DEBUG   ] Finished updating board flags in 2.028 seconds
2014-05-19 22:00:21 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 22:00:41 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 22:01:01 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 22:01:10 [DEBUG   ] Finished updating board flags in 1.928 seconds
2014-05-19 22:01:21 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 22:01:41 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 22:02:01 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 22:02:06 [DEBUG   ] Got command RPT from MCS: ref #195161
2014-05-19 22:02:06 [DEBUG   ] summary = ERROR
2014-05-19 22:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195161  13 56796 79326858 A  ERRORERROR'
2014-05-19 22:02:06 [DEBUG   ] Got command RPT from MCS: ref #195162
2014-05-19 22:02:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 22:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195162  77 56796 79326859 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 22:02:10 [DEBUG   ] Finished updating board flags in 1.802 seconds
2014-05-19 22:02:21 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 22:02:41 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 22:03:01 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 22:03:10 [DEBUG   ] Finished updating board flags in 1.867 seconds
2014-05-19 22:03:21 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 22:03:41 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 22:04:01 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 22:04:10 [DEBUG   ] Finished updating board flags in 1.726 seconds
2014-05-19 22:04:21 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 22:04:41 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 22:05:01 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 22:05:10 [DEBUG   ] Finished updating board flags in 1.627 seconds
2014-05-19 22:05:21 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 22:05:41 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 22:06:01 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 22:06:10 [DEBUG   ] Finished updating board flags in 1.646 seconds
2014-05-19 22:06:21 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 22:06:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 22:07:01 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:07:06 [DEBUG   ] Got command RPT from MCS: ref #195178
2014-05-19 22:07:06 [DEBUG   ] summary = ERROR
2014-05-19 22:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195178  13 56796 79626858 A  ERRORERROR'
2014-05-19 22:07:06 [DEBUG   ] Got command RPT from MCS: ref #195179
2014-05-19 22:07:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 22:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195179  77 56796 79626859 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 22:07:10 [DEBUG   ] Finished updating board flags in 1.541 seconds
2014-05-19 22:07:21 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 22:07:41 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2014-05-19 22:08:01 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 22:08:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 22:08:10 [DEBUG   ] Finished updating board flags in 1.421 seconds
2014-05-19 22:08:21 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 22:08:41 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 22:09:02 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 22:09:10 [DEBUG   ] Finished updating board flags in 1.394 seconds
2014-05-19 22:09:22 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 22:09:42 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2014-05-19 22:10:02 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 22:10:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 22:10:11 [DEBUG   ] Finished updating board flags in 2.193 seconds
2014-05-19 22:10:22 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 22:10:42 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 22:11:02 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 22:11:11 [DEBUG   ] Finished updating board flags in 2.009 seconds
2014-05-19 22:11:22 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2014-05-19 22:11:42 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 22:12:02 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 22:12:06 [DEBUG   ] Got command RPT from MCS: ref #195195
2014-05-19 22:12:06 [DEBUG   ] summary = ERROR
2014-05-19 22:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195195  13 56796 79926854 A  ERRORERROR'
2014-05-19 22:12:06 [DEBUG   ] Got command RPT from MCS: ref #195196
2014-05-19 22:12:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 22:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195196  77 56796 79926855 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 22:12:11 [DEBUG   ] Finished updating board flags in 1.982 seconds
2014-05-19 22:12:22 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 22:12:42 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 22:13:02 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:13:11 [DEBUG   ] Finished updating board flags in 1.865 seconds
2014-05-19 22:13:22 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 22:13:42 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 22:14:02 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 22:14:11 [DEBUG   ] Finished updating board flags in 1.750 seconds
2014-05-19 22:14:22 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 22:14:42 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 22:15:02 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 22:15:11 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 22:15:11 [DEBUG   ] Finished updating board flags in 1.612 seconds
2014-05-19 22:15:22 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 22:15:42 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 22:16:02 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 22:16:11 [DEBUG   ] Finished updating board flags in 1.623 seconds
2014-05-19 22:16:23 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-19 22:16:43 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-19 22:17:03 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 22:17:06 [DEBUG   ] Got command RPT from MCS: ref #195217
2014-05-19 22:17:06 [DEBUG   ] summary = ERROR
2014-05-19 22:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195217  13 56796 80226858 A  ERRORERROR'
2014-05-19 22:17:06 [DEBUG   ] Got command RPT from MCS: ref #195218
2014-05-19 22:17:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 22:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195218  77 56796 80226859 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 22:17:11 [DEBUG   ] Finished updating board flags in 1.565 seconds
2014-05-19 22:17:23 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 22:17:43 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 22:18:03 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 22:18:11 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 22:18:11 [DEBUG   ] Finished updating board flags in 1.455 seconds
2014-05-19 22:18:23 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2014-05-19 22:18:43 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2014-05-19 22:19:03 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 22:19:11 [DEBUG   ] Finished updating board flags in 1.459 seconds
2014-05-19 22:19:23 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 22:19:43 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-19 22:20:03 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 22:20:11 [DEBUG   ] Finished updating board flags in 1.455 seconds
2014-05-19 22:20:23 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 22:20:43 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:21:03 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 22:21:11 [DEBUG   ] Finished updating board flags in 1.482 seconds
2014-05-19 22:21:23 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:21:43 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 22:22:03 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:22:06 [DEBUG   ] Got command RPT from MCS: ref #195234
2014-05-19 22:22:06 [DEBUG   ] summary = ERROR
2014-05-19 22:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195234  13 56796 80526854 A  ERRORERROR'
2014-05-19 22:22:06 [DEBUG   ] Got command RPT from MCS: ref #195235
2014-05-19 22:22:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 22:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195235  77 56796 80526855 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 22:22:12 [DEBUG   ] Finished updating board flags in 2.190 seconds
2014-05-19 22:22:23 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 22:22:43 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 22:23:03 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 22:23:12 [DEBUG   ] Finished updating board flags in 2.067 seconds
2014-05-19 22:23:23 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:23:43 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 22:24:03 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 22:24:12 [DEBUG   ] Finished updating board flags in 2.025 seconds
2014-05-19 22:24:23 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 22:24:43 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:25:03 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:25:12 [DEBUG   ] Finished updating board flags in 1.889 seconds
2014-05-19 22:25:23 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:25:43 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 22:26:03 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 22:26:12 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 22:26:12 [DEBUG   ] Finished updating board flags in 1.813 seconds
2014-05-19 22:26:23 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:26:43 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:27:03 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 22:27:06 [DEBUG   ] Got command RPT from MCS: ref #195251
2014-05-19 22:27:06 [DEBUG   ] summary = ERROR
2014-05-19 22:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195251  13 56796 80826854 A  ERRORERROR'
2014-05-19 22:27:06 [DEBUG   ] Got command RPT from MCS: ref #195252
2014-05-19 22:27:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 22:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195252  77 56796 80826855 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 22:27:12 [DEBUG   ] Finished updating board flags in 1.813 seconds
2014-05-19 22:27:23 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 22:27:43 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:28:03 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:28:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 22:28:12 [DEBUG   ] Finished updating board flags in 1.788 seconds
2014-05-19 22:28:23 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 22:28:43 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 22:29:03 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:29:12 [DEBUG   ] Finished updating board flags in 1.707 seconds
2014-05-19 22:29:23 [DEBUG   ] Finished updating ant stats in 0.793 seconds
2014-05-19 22:29:43 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 22:30:03 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 22:30:12 [DEBUG   ] Finished updating board flags in 1.515 seconds
2014-05-19 22:30:23 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 22:30:43 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 22:31:03 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 22:31:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-19 22:31:12 [DEBUG   ] Finished updating board flags in 1.425 seconds
2014-05-19 22:31:23 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 22:31:43 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 22:32:03 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 22:32:06 [DEBUG   ] Got command RPT from MCS: ref #195268
2014-05-19 22:32:06 [DEBUG   ] summary = ERROR
2014-05-19 22:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195268  13 56796 81126858 A  ERRORERROR'
2014-05-19 22:32:06 [DEBUG   ] Got command RPT from MCS: ref #195269
2014-05-19 22:32:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 22:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195269  77 56796 81126859 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 22:32:12 [DEBUG   ] Finished updating board flags in 1.429 seconds
2014-05-19 22:32:23 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 22:32:43 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 22:33:03 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 22:33:12 [DEBUG   ] Finished updating board flags in 1.435 seconds
2014-05-19 22:33:23 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 22:33:43 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 22:34:03 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 22:34:12 [DEBUG   ] Finished updating board flags in 1.592 seconds
2014-05-19 22:34:23 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 22:34:43 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 22:35:03 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 22:35:13 [DEBUG   ] Finished updating board flags in 2.218 seconds
2014-05-19 22:35:24 [DEBUG   ] Finished updating ant stats in 1.394 seconds
2014-05-19 22:35:44 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-19 22:36:04 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 22:36:13 [DEBUG   ] Finished updating board flags in 2.091 seconds
2014-05-19 22:36:24 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 22:36:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 22:37:04 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-19 22:37:06 [DEBUG   ] Got command RPT from MCS: ref #195285
2014-05-19 22:37:06 [DEBUG   ] summary = ERROR
2014-05-19 22:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195285  13 56796 81426866 A  ERRORERROR'
2014-05-19 22:37:06 [DEBUG   ] Got command RPT from MCS: ref #195286
2014-05-19 22:37:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 22:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195286  77 56796 81426867 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 22:37:13 [DEBUG   ] Finished updating board flags in 2.016 seconds
2014-05-19 22:37:24 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 22:37:44 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 22:38:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 22:38:13 [DEBUG   ] Finished updating board flags in 1.989 seconds
2014-05-19 22:38:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 22:38:44 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 22:39:04 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 22:39:13 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 22:39:13 [DEBUG   ] Finished updating board flags in 1.811 seconds
2014-05-19 22:39:25 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-19 22:39:45 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-19 22:40:05 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 22:40:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 22:40:13 [DEBUG   ] Finished updating board flags in 1.809 seconds
2014-05-19 22:40:25 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 22:40:45 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 22:41:05 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 22:41:13 [DEBUG   ] Finished updating board flags in 1.774 seconds
2014-05-19 22:41:25 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 22:41:45 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 22:42:05 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 22:42:06 [DEBUG   ] Got command RPT from MCS: ref #195302
2014-05-19 22:42:06 [DEBUG   ] summary = ERROR
2014-05-19 22:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195302  13 56796 81726862 A  ERRORERROR'
2014-05-19 22:42:06 [DEBUG   ] Got command RPT from MCS: ref #195303
2014-05-19 22:42:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 22:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195303  77 56796 81726863 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 22:42:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 22:42:13 [DEBUG   ] Finished updating board flags in 1.622 seconds
2014-05-19 22:42:25 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 22:42:45 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 22:43:05 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 22:43:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 22:43:13 [DEBUG   ] Finished updating board flags in 1.583 seconds
2014-05-19 22:43:25 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 22:43:45 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 22:44:05 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 22:44:13 [DEBUG   ] Finished updating board flags in 1.531 seconds
2014-05-19 22:44:25 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2014-05-19 22:44:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 22:45:05 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 22:45:13 [DEBUG   ] Finished updating board flags in 1.531 seconds
2014-05-19 22:45:25 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-19 22:45:45 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 22:46:05 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 22:46:14 [DEBUG   ] Finished updating board flags in 2.251 seconds
2014-05-19 22:46:25 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 22:46:45 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 22:47:05 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 22:47:06 [DEBUG   ] Got command RPT from MCS: ref #195319
2014-05-19 22:47:06 [DEBUG   ] summary = ERROR
2014-05-19 22:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195319  13 56796 82026870 A  ERRORERROR'
2014-05-19 22:47:06 [DEBUG   ] Got command RPT from MCS: ref #195320
2014-05-19 22:47:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 22:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195320  77 56796 82026871 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 22:47:14 [DEBUG   ] Finished updating board flags in 2.117 seconds
2014-05-19 22:47:25 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 22:47:45 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 22:48:05 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 22:48:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 22:48:14 [DEBUG   ] Finished updating board flags in 2.025 seconds
2014-05-19 22:48:26 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-19 22:48:46 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 22:49:06 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 22:49:14 [DEBUG   ] Finished updating board flags in 1.953 seconds
2014-05-19 22:49:26 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 22:49:46 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 22:50:06 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 22:50:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 22:50:14 [DEBUG   ] Finished updating board flags in 1.817 seconds
2014-05-19 22:50:26 [DEBUG   ] Finished updating ant stats in 1.194 seconds
2014-05-19 22:50:46 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 22:51:06 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 22:51:14 [DEBUG   ] Finished updating board flags in 1.787 seconds
2014-05-19 22:51:26 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 22:51:46 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-19 22:52:06 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 22:52:06 [DEBUG   ] Got command RPT from MCS: ref #195336
2014-05-19 22:52:06 [DEBUG   ] summary = ERROR
2014-05-19 22:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195336  13 56796 82326866 A  ERRORERROR'
2014-05-19 22:52:06 [DEBUG   ] Got command RPT from MCS: ref #195337
2014-05-19 22:52:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 22:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195337  77 56796 82326867 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 22:52:14 [DEBUG   ] Finished updating board flags in 1.619 seconds
2014-05-19 22:52:26 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 22:52:46 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-19 22:53:06 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 22:53:14 [DEBUG   ] Finished updating board flags in 1.591 seconds
2014-05-19 22:53:26 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-19 22:53:46 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 22:54:06 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 22:54:14 [DEBUG   ] Finished updating board flags in 1.414 seconds
2014-05-19 22:54:26 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 22:54:46 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 22:55:06 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 22:55:14 [DEBUG   ] Finished updating board flags in 1.401 seconds
2014-05-19 22:55:27 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-19 22:55:47 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 22:56:07 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 22:56:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 22:56:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 22:56:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 22:56:14 [DEBUG   ] Finished updating board flags in 1.398 seconds
2014-05-19 22:56:27 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 22:56:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 22:57:06 [DEBUG   ] Got command RPT from MCS: ref #195353
2014-05-19 22:57:06 [DEBUG   ] summary = ERROR
2014-05-19 22:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195353  13 56796 82626862 A  ERRORERROR'
2014-05-19 22:57:06 [DEBUG   ] Got command RPT from MCS: ref #195354
2014-05-19 22:57:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 22:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195354  77 56796 82626863 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 22:57:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 22:57:15 [DEBUG   ] Finished updating board flags in 2.284 seconds
2014-05-19 22:57:27 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 22:57:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 22:58:07 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 22:58:15 [DEBUG   ] Finished updating board flags in 2.120 seconds
2014-05-19 22:58:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 22:58:47 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 22:59:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 22:59:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 22:59:15 [DEBUG   ] Finished updating board flags in 2.017 seconds
2014-05-19 22:59:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 22:59:47 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:00:07 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-19 23:00:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 23:00:15 [DEBUG   ] Finished updating board flags in 2.017 seconds
2014-05-19 23:00:27 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-19 23:00:47 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 23:01:07 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 23:01:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 23:01:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 23:01:15 [DEBUG   ] Finished updating board flags in 1.960 seconds
2014-05-19 23:01:27 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 23:01:47 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 23:02:06 [DEBUG   ] Got command RPT from MCS: ref #195370
2014-05-19 23:02:06 [DEBUG   ] summary = ERROR
2014-05-19 23:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195370  13 56796 82926866 A  ERRORERROR'
2014-05-19 23:02:06 [DEBUG   ] Got command RPT from MCS: ref #195371
2014-05-19 23:02:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 23:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195371  77 56796 82926867 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 23:02:07 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 23:02:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 23:02:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 23:02:15 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 23:02:15 [DEBUG   ] Finished updating board flags in 1.827 seconds
2014-05-19 23:02:27 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 23:02:47 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 23:03:07 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 23:03:15 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 23:03:15 [DEBUG   ] Finished updating board flags in 1.863 seconds
2014-05-19 23:03:27 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 23:03:47 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 23:04:07 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-19 23:04:15 [DEBUG   ] Finished updating board flags in 1.721 seconds
2014-05-19 23:04:27 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 23:04:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:05:07 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 23:05:15 [DEBUG   ] Finished updating board flags in 1.628 seconds
2014-05-19 23:05:27 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 23:05:47 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 23:06:07 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2014-05-19 23:06:15 [DEBUG   ] Finished updating board flags in 1.569 seconds
2014-05-19 23:06:27 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 23:06:47 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 23:07:06 [DEBUG   ] Got command RPT from MCS: ref #195387
2014-05-19 23:07:06 [DEBUG   ] summary = ERROR
2014-05-19 23:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195387  13 56796 83226874 A  ERRORERROR'
2014-05-19 23:07:06 [DEBUG   ] Got command RPT from MCS: ref #195388
2014-05-19 23:07:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 23:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195388  77 56796 83226875 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 23:07:07 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 23:07:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 23:07:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 23:07:15 [DEBUG   ] Finished updating board flags in 1.406 seconds
2014-05-19 23:07:27 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 23:07:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 23:08:07 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 23:08:15 [DEBUG   ] Finished updating board flags in 1.426 seconds
2014-05-19 23:08:27 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 23:08:47 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 23:09:07 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 23:09:15 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-19 23:09:15 [CRITICAL] MGT underflow detected on beam(s) 1
2014-05-19 23:09:15 [DEBUG   ] Finished updating board flags in 1.435 seconds
2014-05-19 23:09:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 23:09:48 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-19 23:10:08 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 23:10:15 [DEBUG   ] Finished updating board flags in 1.444 seconds
2014-05-19 23:10:28 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 23:10:48 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-19 23:11:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:11:15 [DEBUG   ] Finished updating board flags in 1.442 seconds
2014-05-19 23:11:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:11:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 23:12:06 [DEBUG   ] Got command RPT from MCS: ref #195404
2014-05-19 23:12:06 [DEBUG   ] summary = ERROR
2014-05-19 23:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195404  13 56796 83526866 A  ERRORERROR'
2014-05-19 23:12:06 [DEBUG   ] Got command RPT from MCS: ref #195405
2014-05-19 23:12:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 23:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195405  77 56796 83526867 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 23:12:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 23:12:15 [DEBUG   ] Finished updating board flags in 1.554 seconds
2014-05-19 23:12:28 [DEBUG   ] Finished updating ant stats in 0.789 seconds
2014-05-19 23:12:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 23:13:08 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 23:13:16 [DEBUG   ] Finished updating board flags in 2.200 seconds
2014-05-19 23:13:28 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 23:13:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 23:14:08 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-19 23:14:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 23:14:16 [DEBUG   ] Finished updating board flags in 2.126 seconds
2014-05-19 23:14:28 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-19 23:14:49 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-19 23:15:09 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-19 23:15:16 [DEBUG   ] Finished updating board flags in 2.021 seconds
2014-05-19 23:15:29 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 23:15:49 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 23:16:09 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:16:16 [DEBUG   ] Finished updating board flags in 1.957 seconds
2014-05-19 23:16:29 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 23:16:49 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 23:17:06 [DEBUG   ] Got command RPT from MCS: ref #195421
2014-05-19 23:17:06 [DEBUG   ] summary = ERROR
2014-05-19 23:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195421  13 56796 83826874 A  ERRORERROR'
2014-05-19 23:17:06 [DEBUG   ] Got command RPT from MCS: ref #195422
2014-05-19 23:17:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 23:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195422  77 56796 83826875 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 23:17:09 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 23:17:16 [DEBUG   ] Finished updating board flags in 1.839 seconds
2014-05-19 23:17:29 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:17:49 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 23:18:09 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 23:18:16 [DEBUG   ] Finished updating board flags in 1.761 seconds
2014-05-19 23:18:29 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 23:18:49 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:19:09 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:19:16 [DEBUG   ] Finished updating board flags in 1.719 seconds
2014-05-19 23:19:29 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-19 23:19:49 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 23:20:09 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 23:20:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 23:20:16 [DEBUG   ] Finished updating board flags in 1.502 seconds
2014-05-19 23:20:29 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 23:20:49 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 23:21:09 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 23:21:16 [DEBUG   ] Finished updating board flags in 1.510 seconds
2014-05-19 23:21:29 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 23:21:49 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 23:22:06 [DEBUG   ] Got command RPT from MCS: ref #195438
2014-05-19 23:22:06 [DEBUG   ] summary = ERROR
2014-05-19 23:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195438  13 56796 84126874 A  ERRORERROR'
2014-05-19 23:22:06 [DEBUG   ] Got command RPT from MCS: ref #195439
2014-05-19 23:22:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 23:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195439  77 56796 84126875 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 23:22:09 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 23:22:16 [DEBUG   ] Finished updating board flags in 1.543 seconds
2014-05-19 23:22:29 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-19 23:22:49 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:23:09 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:23:17 [DEBUG   ] Finished updating board flags in 2.178 seconds
2014-05-19 23:23:29 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-19 23:23:49 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 23:24:09 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 23:24:17 [DEBUG   ] Finished updating board flags in 2.013 seconds
2014-05-19 23:24:29 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 23:24:49 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 23:25:09 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 23:25:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 23:25:17 [DEBUG   ] Finished updating board flags in 1.959 seconds
2014-05-19 23:25:29 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 23:25:49 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 23:26:09 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 23:26:17 [DEBUG   ] Finished updating board flags in 1.891 seconds
2014-05-19 23:26:29 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-19 23:26:50 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-19 23:27:06 [DEBUG   ] Got command RPT from MCS: ref #195455
2014-05-19 23:27:06 [DEBUG   ] summary = ERROR
2014-05-19 23:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195455  13 56796 84426871 A  ERRORERROR'
2014-05-19 23:27:06 [DEBUG   ] Got command RPT from MCS: ref #195456
2014-05-19 23:27:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 23:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195456  77 56796 84426871 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 23:27:10 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 23:27:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 23:27:17 [DEBUG   ] Finished updating board flags in 1.732 seconds
2014-05-19 23:27:30 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:27:50 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:28:10 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 23:28:16 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 23:28:17 [DEBUG   ] Finished updating board flags in 1.607 seconds
2014-05-19 23:28:30 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 23:28:50 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:29:10 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:29:17 [DEBUG   ] Finished updating board flags in 1.594 seconds
2014-05-19 23:29:30 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 23:29:50 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 23:30:06 [DEBUG   ] Got command TBW from MCS: ref #195473
2014-05-19 23:30:06 [DEBUG   ] tbw = exited with status True, code 0x00
2014-05-19 23:30:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBW   195473   8 56796 84606083 A  ERROR'
2014-05-19 23:30:07 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 pkill tbn
2014-05-19 23:30:07 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 pkill tbn
2014-05-19 23:30:07 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 pkill tbn
2014-05-19 23:30:08 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 pkill tbn
2014-05-19 23:30:09 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 pkill tbn
2014-05-19 23:30:09 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 pkill tbn
2014-05-19 23:30:09 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 pkill tbn
2014-05-19 23:30:09 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 pkill tbn
2014-05-19 23:30:09 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 pkill tbn
2014-05-19 23:30:09 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 pkill tbn
2014-05-19 23:30:09 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 pkill tbn
2014-05-19 23:30:09 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 pkill tbn
2014-05-19 23:30:09 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 pkill tbn
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa27', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '1', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa24', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '2', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa23', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '3', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa18', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '4', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa10', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '5', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa32', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '6', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa11', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '7', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa30', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '8', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa12', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '9', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa8', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '10', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa13', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '11', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa26', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '12', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa14', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '13', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa22', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '14', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa20', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '15', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa1', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '16', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa5', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '17', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa25', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '18', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa35', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '19', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa19', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '20', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa21', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '21', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa6', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '22', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa7', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '23', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa33', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '24', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa15', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '25', '12000000']
2014-05-19 23:30:09 [DEBUG   ] spawn_process: at 2014-05-19 23:30:09 running ['/usr/local/bin/lwa28', '/lwa/software/tbw', '192.168.0.37', '10000', '12', '0', '26', '12000000']
2014-05-19 23:30:09 [INFO    ] Finished spawning TBW
2014-05-19 23:30:17 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 23:30:17 [DEBUG   ] Finished updating board flags in 1.196 seconds
2014-05-19 23:31:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-19 23:31:18 [DEBUG   ] Finished updating board flags in 2.118 seconds
2014-05-19 23:32:06 [DEBUG   ] Got command RPT from MCS: ref #195479
2014-05-19 23:32:06 [DEBUG   ] summary = ERROR
2014-05-19 23:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195479  13 56796 84726878 A  ERRORERROR'
2014-05-19 23:32:06 [DEBUG   ] Got command RPT from MCS: ref #195480
2014-05-19 23:32:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 23:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195480  77 56796 84726879 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 23:32:17 [DEBUG   ] Finished updating board flags in 1.335 seconds
2014-05-19 23:33:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 23:33:18 [DEBUG   ] Finished updating board flags in 1.860 seconds
2014-05-19 23:33:48 [INFO    ] __waitForTBW - TBW finished in 218.618 s
2014-05-19 23:33:49 [INFO    ] Started antennas statistics background thread
2014-05-19 23:33:49 [INFO    ] Resuming TBN at 37.800 MHz after TBW acquisition
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '1', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '2', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '3', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] Finished updating ant stats in 1.013 seconds
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '4', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '5', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '6', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '7', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '8', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '9', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '10', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '11', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '12', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '13', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '14', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '15', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '16', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '17', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '18', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '19', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '20', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '21', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '22', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '23', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '24', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '25', '1400542430']
2014-05-19 23:33:49 [DEBUG   ] spawn_process: at 2014-05-19 23:33:49 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '26', '1400542430']
2014-05-19 23:33:49 [INFO    ] Finished spawning TBN in 0.066 s
2014-05-19 23:34:09 [DEBUG   ] Finished updating ant stats in 1.066 seconds
2014-05-19 23:34:18 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 23:34:18 [DEBUG   ] Finished updating board flags in 2.024 seconds
2014-05-19 23:34:29 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-19 23:34:49 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 23:35:09 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 23:35:18 [DEBUG   ] Finished updating board flags in 1.861 seconds
2014-05-19 23:35:29 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-19 23:35:49 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-19 23:36:09 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 23:36:18 [DEBUG   ] Finished updating board flags in 1.810 seconds
2014-05-19 23:36:29 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 23:36:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 23:37:06 [DEBUG   ] Got command RPT from MCS: ref #195501
2014-05-19 23:37:06 [DEBUG   ] summary = ERROR
2014-05-19 23:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195501  13 56796 85026878 A  ERRORERROR'
2014-05-19 23:37:06 [DEBUG   ] Got command RPT from MCS: ref #195502
2014-05-19 23:37:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 23:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195502  77 56796 85026879 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 23:37:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 23:37:18 [DEBUG   ] Finished updating board flags in 1.787 seconds
2014-05-19 23:37:29 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:37:49 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:38:09 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:38:18 [DEBUG   ] Finished updating board flags in 1.617 seconds
2014-05-19 23:38:29 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 23:38:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 23:39:09 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 23:39:18 [DEBUG   ] Finished updating board flags in 1.586 seconds
2014-05-19 23:39:29 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 23:39:49 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 23:40:09 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 23:40:18 [DEBUG   ] Finished updating board flags in 1.547 seconds
2014-05-19 23:40:29 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:40:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 23:41:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 23:41:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-19 23:41:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 23:41:18 [DEBUG   ] Finished updating board flags in 1.579 seconds
2014-05-19 23:41:29 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:41:49 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 23:42:06 [DEBUG   ] Got command RPT from MCS: ref #195518
2014-05-19 23:42:06 [DEBUG   ] summary = ERROR
2014-05-19 23:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195518  13 56796 85326874 A  ERRORERROR'
2014-05-19 23:42:06 [DEBUG   ] Got command RPT from MCS: ref #195519
2014-05-19 23:42:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 23:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195519  77 56796 85326875 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 23:42:09 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:42:19 [DEBUG   ] Finished updating board flags in 2.207 seconds
2014-05-19 23:42:29 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 23:42:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 23:43:09 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 23:43:19 [DEBUG   ] Finished updating board flags in 2.267 seconds
2014-05-19 23:43:29 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:43:49 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 23:44:09 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 23:44:19 [DEBUG   ] Finished updating board flags in 2.025 seconds
2014-05-19 23:44:29 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 23:44:49 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:45:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 23:45:19 [DEBUG   ] Finished updating board flags in 2.034 seconds
2014-05-19 23:45:29 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 23:45:49 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 23:46:09 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 23:46:19 [DEBUG   ] Finished updating board flags in 1.960 seconds
2014-05-19 23:46:29 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 23:46:49 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 23:47:06 [DEBUG   ] Got command RPT from MCS: ref #195535
2014-05-19 23:47:06 [DEBUG   ] summary = ERROR
2014-05-19 23:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195535  13 56796 85626882 A  ERRORERROR'
2014-05-19 23:47:06 [DEBUG   ] Got command RPT from MCS: ref #195536
2014-05-19 23:47:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 23:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195536  77 56796 85626883 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 23:47:09 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 23:47:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 23:47:19 [DEBUG   ] Finished updating board flags in 1.829 seconds
2014-05-19 23:47:29 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 23:47:49 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-19 23:48:09 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 23:48:19 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-19 23:48:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 23:48:19 [DEBUG   ] Finished updating board flags in 1.779 seconds
2014-05-19 23:48:29 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-19 23:48:49 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-19 23:49:09 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-19 23:49:19 [DEBUG   ] Finished updating board flags in 1.681 seconds
2014-05-19 23:49:29 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2014-05-19 23:49:50 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-19 23:50:10 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-19 23:50:19 [DEBUG   ] Finished updating board flags in 1.512 seconds
2014-05-19 23:50:30 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 23:50:50 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:51:10 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-19 23:51:19 [DEBUG   ] Finished updating board flags in 1.473 seconds
2014-05-19 23:51:30 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-19 23:51:50 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2014-05-19 23:52:06 [DEBUG   ] Got command RPT from MCS: ref #195552
2014-05-19 23:52:06 [DEBUG   ] summary = ERROR
2014-05-19 23:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195552  13 56796 85926878 A  ERRORERROR'
2014-05-19 23:52:06 [DEBUG   ] Got command RPT from MCS: ref #195553
2014-05-19 23:52:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 23:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195553  77 56796 85926879 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 23:52:10 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:52:19 [DEBUG   ] Finished updating board flags in 1.478 seconds
2014-05-19 23:52:30 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-19 23:52:50 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-19 23:53:10 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-19 23:53:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 23:53:19 [DEBUG   ] Finished updating board flags in 1.523 seconds
2014-05-19 23:53:30 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-19 23:53:50 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-19 23:54:10 [DEBUG   ] Finished updating ant stats in 0.594 seconds
2014-05-19 23:54:20 [DEBUG   ] Finished updating board flags in 2.188 seconds
2014-05-19 23:54:31 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-19 23:54:51 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-19 23:55:11 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:55:20 [DEBUG   ] Finished updating board flags in 1.947 seconds
2014-05-19 23:55:31 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:55:51 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:56:11 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 23:56:20 [DEBUG   ] Finished updating board flags in 1.825 seconds
2014-05-19 23:56:31 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 23:56:51 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:57:06 [DEBUG   ] Got command RPT from MCS: ref #195569
2014-05-19 23:57:06 [DEBUG   ] summary = ERROR
2014-05-19 23:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195569  13 56796 86226878 A  ERRORERROR'
2014-05-19 23:57:06 [DEBUG   ] Got command RPT from MCS: ref #195570
2014-05-19 23:57:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-19 23:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195570  77 56796 86226879 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-19 23:57:11 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:57:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-19 23:57:20 [DEBUG   ] Finished updating board flags in 1.784 seconds
2014-05-19 23:57:31 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-19 23:57:51 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:58:11 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-19 23:58:20 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-19 23:58:20 [DEBUG   ] Finished updating board flags in 1.647 seconds
2014-05-19 23:58:31 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 23:58:51 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-19 23:59:11 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-19 23:59:20 [DEBUG   ] Finished updating board flags in 1.670 seconds
2014-05-19 23:59:31 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2014-05-19 23:59:51 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 00:00:11 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 00:00:20 [DEBUG   ] Finished updating board flags in 1.488 seconds
2014-05-20 00:00:31 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 00:00:51 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 00:01:11 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 00:01:20 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 00:01:20 [DEBUG   ] Finished updating board flags in 1.481 seconds
2014-05-20 00:01:31 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 00:01:51 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 00:02:06 [DEBUG   ] Got command RPT from MCS: ref #195586
2014-05-20 00:02:06 [DEBUG   ] summary = ERROR
2014-05-20 00:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195586  13 56797   126886 A  ERRORERROR'
2014-05-20 00:02:06 [DEBUG   ] Got command RPT from MCS: ref #195587
2014-05-20 00:02:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 00:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195587  77 56797   126887 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 00:02:11 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 00:02:20 [DEBUG   ] Finished updating board flags in 1.503 seconds
2014-05-20 00:02:31 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2014-05-20 00:02:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 00:03:11 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 00:03:20 [DEBUG   ] Finished updating board flags in 1.496 seconds
2014-05-20 00:03:31 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2014-05-20 00:03:51 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 00:04:11 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 00:04:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 00:04:21 [DEBUG   ] Finished updating board flags in 2.148 seconds
2014-05-20 00:04:31 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 00:04:51 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 00:05:11 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 00:05:21 [DEBUG   ] Finished updating board flags in 2.007 seconds
2014-05-20 00:05:31 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 00:05:51 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 00:06:11 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 00:06:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 00:06:21 [DEBUG   ] Finished updating board flags in 2.112 seconds
2014-05-20 00:06:31 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 00:06:51 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 00:07:06 [DEBUG   ] Got command RPT from MCS: ref #195603
2014-05-20 00:07:06 [DEBUG   ] summary = ERROR
2014-05-20 00:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195603  13 56797   426878 A  ERRORERROR'
2014-05-20 00:07:06 [DEBUG   ] Got command RPT from MCS: ref #195604
2014-05-20 00:07:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 00:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195604  77 56797   426879 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 00:07:11 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 00:07:21 [DEBUG   ] Finished updating board flags in 1.872 seconds
2014-05-20 00:07:31 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 00:07:51 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 00:08:11 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 00:08:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 00:08:21 [DEBUG   ] Finished updating board flags in 1.812 seconds
2014-05-20 00:08:31 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 00:08:51 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 00:09:12 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 00:09:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 00:09:21 [DEBUG   ] Finished updating board flags in 1.787 seconds
2014-05-20 00:09:32 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 00:09:52 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 00:10:12 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 00:10:21 [DEBUG   ] Finished updating board flags in 1.625 seconds
2014-05-20 00:10:32 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 00:10:52 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 00:11:12 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 00:11:21 [DEBUG   ] Finished updating board flags in 1.576 seconds
2014-05-20 00:11:32 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 00:11:52 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 00:12:06 [DEBUG   ] Got command RPT from MCS: ref #195620
2014-05-20 00:12:06 [DEBUG   ] summary = ERROR
2014-05-20 00:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195620  13 56797   726878 A  ERRORERROR'
2014-05-20 00:12:06 [DEBUG   ] Got command RPT from MCS: ref #195621
2014-05-20 00:12:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 00:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195621  77 56797   726879 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 00:12:12 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 00:12:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 00:12:21 [DEBUG   ] Finished updating board flags in 1.471 seconds
2014-05-20 00:12:32 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 00:12:52 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 00:13:12 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 00:13:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 00:13:21 [DEBUG   ] Finished updating board flags in 1.571 seconds
2014-05-20 00:13:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 00:13:52 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 00:14:12 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 00:14:22 [DEBUG   ] Finished updating board flags in 2.271 seconds
2014-05-20 00:14:32 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 00:14:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 00:15:12 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 00:15:22 [DEBUG   ] Finished updating board flags in 2.098 seconds
2014-05-20 00:15:33 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2014-05-20 00:15:53 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 00:16:13 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 00:16:22 [DEBUG   ] Finished updating board flags in 2.017 seconds
2014-05-20 00:16:33 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 00:16:53 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 00:17:06 [DEBUG   ] Got command RPT from MCS: ref #195637
2014-05-20 00:17:06 [DEBUG   ] summary = ERROR
2014-05-20 00:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195637  13 56797  1026890 A  ERRORERROR'
2014-05-20 00:17:06 [DEBUG   ] Got command RPT from MCS: ref #195638
2014-05-20 00:17:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 00:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195638  77 56797  1026891 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 00:17:13 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 00:17:22 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 00:17:22 [DEBUG   ] Finished updating board flags in 1.985 seconds
2014-05-20 00:17:33 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 00:17:53 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 00:18:13 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 00:18:22 [DEBUG   ] Finished updating board flags in 1.824 seconds
2014-05-20 00:18:33 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 00:18:53 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 00:19:13 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 00:19:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 00:19:22 [DEBUG   ] Finished updating board flags in 1.783 seconds
2014-05-20 00:19:33 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 00:19:54 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 00:20:14 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 00:20:22 [DEBUG   ] Finished updating board flags in 1.605 seconds
2014-05-20 00:20:34 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 00:20:54 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 00:21:14 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 00:21:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 00:21:22 [DEBUG   ] Finished updating board flags in 1.589 seconds
2014-05-20 00:21:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 00:21:54 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 00:22:06 [DEBUG   ] Got command RPT from MCS: ref #195654
2014-05-20 00:22:06 [DEBUG   ] summary = ERROR
2014-05-20 00:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195654  13 56797  1326882 A  ERRORERROR'
2014-05-20 00:22:06 [DEBUG   ] Got command RPT from MCS: ref #195655
2014-05-20 00:22:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 00:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195655  77 56797  1326883 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 00:22:14 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 00:22:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 00:22:22 [DEBUG   ] Finished updating board flags in 1.415 seconds
2014-05-20 00:22:34 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 00:22:54 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2014-05-20 00:23:14 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 00:23:22 [DEBUG   ] Finished updating board flags in 1.392 seconds
2014-05-20 00:23:34 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 00:23:55 [DEBUG   ] Finished updating ant stats in 1.403 seconds
2014-05-20 00:24:15 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 00:24:23 [DEBUG   ] Finished updating board flags in 2.208 seconds
2014-05-20 00:24:35 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 00:24:55 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 00:25:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 00:25:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 00:25:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 00:25:23 [DEBUG   ] Finished updating board flags in 2.190 seconds
2014-05-20 00:25:35 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2014-05-20 00:25:55 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 00:26:15 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 00:26:23 [DEBUG   ] Finished updating board flags in 2.014 seconds
2014-05-20 00:26:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 00:26:55 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 00:27:06 [DEBUG   ] Got command RPT from MCS: ref #195671
2014-05-20 00:27:06 [DEBUG   ] summary = ERROR
2014-05-20 00:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195671  13 56797  1626886 A  ERRORERROR'
2014-05-20 00:27:06 [DEBUG   ] Got command RPT from MCS: ref #195672
2014-05-20 00:27:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 00:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195672  77 56797  1626887 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 00:27:15 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 00:27:23 [DEBUG   ] Finished updating board flags in 1.996 seconds
2014-05-20 00:27:35 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 00:27:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 00:28:15 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 00:28:23 [DEBUG   ] Finished updating board flags in 1.801 seconds
2014-05-20 00:28:35 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2014-05-20 00:28:56 [DEBUG   ] Finished updating ant stats in 1.403 seconds
2014-05-20 00:29:16 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 00:29:23 [DEBUG   ] Finished updating board flags in 1.812 seconds
2014-05-20 00:29:36 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 00:29:56 [DEBUG   ] Finished updating ant stats in 1.210 seconds
2014-05-20 00:30:16 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 00:30:23 [DEBUG   ] Finished updating board flags in 1.803 seconds
2014-05-20 00:30:36 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 00:30:56 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 00:31:16 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 00:31:23 [DEBUG   ] Finished updating board flags in 1.801 seconds
2014-05-20 00:31:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 00:31:56 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 00:32:06 [DEBUG   ] Got command RPT from MCS: ref #195688
2014-05-20 00:32:06 [DEBUG   ] summary = ERROR
2014-05-20 00:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195688  13 56797  1926894 A  ERRORERROR'
2014-05-20 00:32:06 [DEBUG   ] Got command RPT from MCS: ref #195689
2014-05-20 00:32:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 00:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195689  77 56797  1926895 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 00:32:16 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 00:32:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 00:32:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 00:32:23 [DEBUG   ] Finished updating board flags in 1.810 seconds
2014-05-20 00:32:36 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 00:32:56 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 00:33:16 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 00:33:23 [DEBUG   ] Finished updating board flags in 1.787 seconds
2014-05-20 00:33:36 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2014-05-20 00:33:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 00:34:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 00:34:23 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 00:34:23 [DEBUG   ] Finished updating board flags in 1.609 seconds
2014-05-20 00:34:36 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 00:34:56 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 00:35:16 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 00:35:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 00:35:23 [DEBUG   ] Finished updating board flags in 1.606 seconds
2014-05-20 00:35:37 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-20 00:35:57 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 00:36:17 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 00:36:23 [DEBUG   ] Finished updating board flags in 1.591 seconds
2014-05-20 00:36:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 00:36:57 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 00:37:06 [DEBUG   ] Got command RPT from MCS: ref #195705
2014-05-20 00:37:06 [DEBUG   ] summary = ERROR
2014-05-20 00:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195705  13 56797  2226886 A  ERRORERROR'
2014-05-20 00:37:06 [DEBUG   ] Got command RPT from MCS: ref #195706
2014-05-20 00:37:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 00:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195706  77 56797  2226887 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 00:37:17 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 00:37:23 [DEBUG   ] Finished updating board flags in 1.420 seconds
2014-05-20 00:37:37 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 00:37:57 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 00:38:17 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 00:38:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 00:38:23 [DEBUG   ] Finished updating board flags in 1.416 seconds
2014-05-20 00:38:37 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 00:38:57 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 00:39:17 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 00:39:23 [DEBUG   ] Finished updating board flags in 1.416 seconds
2014-05-20 00:39:37 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 00:39:57 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 00:40:18 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 00:40:23 [DEBUG   ] Finished updating board flags in 1.416 seconds
2014-05-20 00:40:38 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 00:40:58 [DEBUG   ] Finished updating ant stats in 1.179 seconds
2014-05-20 00:41:18 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 00:41:23 [DEBUG   ] Finished updating board flags in 1.423 seconds
2014-05-20 00:41:38 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 00:41:58 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 00:42:06 [DEBUG   ] Got command RPT from MCS: ref #195722
2014-05-20 00:42:06 [DEBUG   ] summary = ERROR
2014-05-20 00:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195722  13 56797  2526894 A  ERRORERROR'
2014-05-20 00:42:06 [DEBUG   ] Got command RPT from MCS: ref #195723
2014-05-20 00:42:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 00:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195723  77 56797  2526895 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 00:42:18 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 00:42:23 [DEBUG   ] Finished updating board flags in 1.417 seconds
2014-05-20 00:42:38 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 00:42:58 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 00:43:18 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 00:43:23 [DEBUG   ] Finished updating board flags in 1.491 seconds
2014-05-20 00:43:38 [DEBUG   ] Finished updating ant stats in 0.592 seconds
2014-05-20 00:43:59 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 00:44:19 [DEBUG   ] Finished updating ant stats in 1.384 seconds
2014-05-20 00:44:23 [DEBUG   ] Finished updating board flags in 1.496 seconds
2014-05-20 00:44:39 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 00:44:59 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2014-05-20 00:45:19 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 00:45:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-20 00:45:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 00:45:24 [DEBUG   ] Finished updating board flags in 2.175 seconds
2014-05-20 00:45:39 [DEBUG   ] Finished updating ant stats in 0.991 seconds
2014-05-20 00:45:59 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 00:46:19 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 00:46:24 [DEBUG   ] Finished updating board flags in 2.047 seconds
2014-05-20 00:46:39 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 00:46:59 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 00:47:06 [DEBUG   ] Got command RPT from MCS: ref #195739
2014-05-20 00:47:06 [DEBUG   ] summary = ERROR
2014-05-20 00:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195739  13 56797  2826894 A  ERRORERROR'
2014-05-20 00:47:06 [DEBUG   ] Got command RPT from MCS: ref #195740
2014-05-20 00:47:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 00:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195740  77 56797  2826895 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 00:47:19 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 00:47:24 [DEBUG   ] Finished updating board flags in 1.902 seconds
2014-05-20 00:47:39 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 00:47:59 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 00:48:20 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 00:48:24 [DEBUG   ] Finished updating board flags in 1.809 seconds
2014-05-20 00:48:40 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 00:49:00 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 00:49:20 [DEBUG   ] Finished updating ant stats in 1.192 seconds
2014-05-20 00:49:24 [DEBUG   ] Finished updating board flags in 1.787 seconds
2014-05-20 00:49:40 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 00:50:00 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 00:50:20 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 00:50:24 [DEBUG   ] Finished updating board flags in 1.621 seconds
2014-05-20 00:50:40 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 00:51:00 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 00:51:20 [DEBUG   ] Finished updating ant stats in 0.793 seconds
2014-05-20 00:51:24 [DEBUG   ] Finished updating board flags in 1.577 seconds
2014-05-20 00:51:40 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 00:52:01 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 00:52:06 [DEBUG   ] Got command RPT from MCS: ref #195756
2014-05-20 00:52:06 [DEBUG   ] summary = ERROR
2014-05-20 00:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195756  13 56797  3126898 A  ERRORERROR'
2014-05-20 00:52:06 [DEBUG   ] Got command RPT from MCS: ref #195757
2014-05-20 00:52:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 00:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195757  77 56797  3126899 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 00:52:21 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 00:52:24 [DEBUG   ] Finished updating board flags in 1.418 seconds
2014-05-20 00:52:41 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 00:53:01 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 00:53:21 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 00:53:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 00:53:24 [DEBUG   ] Finished updating board flags in 1.398 seconds
2014-05-20 00:53:41 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 00:54:01 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 00:54:21 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 00:54:25 [DEBUG   ] Finished updating board flags in 2.210 seconds
2014-05-20 00:54:41 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 00:55:01 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 00:55:21 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 00:55:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 00:55:25 [DEBUG   ] Finished updating board flags in 2.185 seconds
2014-05-20 00:55:41 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 00:56:01 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 00:56:21 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 00:56:25 [DEBUG   ] Finished updating board flags in 2.018 seconds
2014-05-20 00:56:41 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 00:57:01 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 00:57:06 [DEBUG   ] Got command RPT from MCS: ref #195773
2014-05-20 00:57:06 [DEBUG   ] summary = ERROR
2014-05-20 00:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195773  13 56797  3426894 A  ERRORERROR'
2014-05-20 00:57:06 [DEBUG   ] Got command RPT from MCS: ref #195774
2014-05-20 00:57:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 00:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195774  77 56797  3426895 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 00:57:21 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 00:57:25 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 00:57:25 [DEBUG   ] Finished updating board flags in 2.006 seconds
2014-05-20 00:57:42 [DEBUG   ] Finished updating ant stats in 1.267 seconds
2014-05-20 00:58:02 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 00:58:22 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 00:58:25 [DEBUG   ] Finished updating board flags in 1.987 seconds
2014-05-20 00:58:42 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2014-05-20 00:59:02 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 00:59:22 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 00:59:25 [DEBUG   ] Finished updating board flags in 1.816 seconds
2014-05-20 00:59:42 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 01:00:02 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 01:00:22 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 01:00:25 [DEBUG   ] Finished updating board flags in 1.833 seconds
2014-05-20 01:00:42 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 01:01:02 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 01:01:22 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 01:01:25 [DEBUG   ] Finished updating board flags in 1.807 seconds
2014-05-20 01:01:42 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 01:02:03 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 01:02:06 [DEBUG   ] Got command RPT from MCS: ref #195790
2014-05-20 01:02:06 [DEBUG   ] summary = ERROR
2014-05-20 01:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195790  13 56797  3726894 A  ERRORERROR'
2014-05-20 01:02:06 [DEBUG   ] Got command RPT from MCS: ref #195791
2014-05-20 01:02:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 01:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195791  77 56797  3726895 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 01:02:23 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 01:02:25 [DEBUG   ] Finished updating board flags in 1.751 seconds
2014-05-20 01:02:43 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 01:03:03 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 01:03:23 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 01:03:25 [DEBUG   ] Finished updating board flags in 1.487 seconds
2014-05-20 01:03:43 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 01:04:03 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 01:04:23 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 01:04:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-20 01:04:25 [DEBUG   ] Finished updating board flags in 1.413 seconds
2014-05-20 01:04:43 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 01:05:03 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 01:05:23 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 01:05:25 [DEBUG   ] Finished updating board flags in 1.380 seconds
2014-05-20 01:05:43 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 01:06:03 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 01:06:23 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 01:06:25 [DEBUG   ] Finished updating board flags in 1.367 seconds
2014-05-20 01:06:44 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 01:07:04 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 01:07:06 [DEBUG   ] Got command RPT from MCS: ref #195807
2014-05-20 01:07:06 [DEBUG   ] summary = ERROR
2014-05-20 01:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195807  13 56797  4026894 A  ERRORERROR'
2014-05-20 01:07:06 [DEBUG   ] Got command RPT from MCS: ref #195808
2014-05-20 01:07:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 01:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195808  77 56797  4026895 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 01:07:24 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 01:07:26 [DEBUG   ] Finished updating board flags in 2.053 seconds
2014-05-20 01:07:44 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 01:08:04 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 01:08:24 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 01:08:26 [DEBUG   ] Finished updating board flags in 1.998 seconds
2014-05-20 01:08:44 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 01:09:04 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 01:09:24 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 01:09:26 [DEBUG   ] Finished updating board flags in 1.799 seconds
2014-05-20 01:09:44 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 01:10:04 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 01:10:24 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 01:10:26 [DEBUG   ] Finished updating board flags in 1.611 seconds
2014-05-20 01:10:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 01:11:04 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 01:11:24 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 01:11:26 [DEBUG   ] Finished updating board flags in 1.636 seconds
2014-05-20 01:11:44 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 01:12:04 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 01:12:06 [DEBUG   ] Got command RPT from MCS: ref #195824
2014-05-20 01:12:06 [DEBUG   ] summary = ERROR
2014-05-20 01:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195824  13 56797  4326902 A  ERRORERROR'
2014-05-20 01:12:06 [DEBUG   ] Got command RPT from MCS: ref #195825
2014-05-20 01:12:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 01:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195825  77 56797  4326903 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 01:12:25 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 01:12:26 [DEBUG   ] Finished updating board flags in 1.567 seconds
2014-05-20 01:12:45 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 01:13:05 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 01:13:25 [DEBUG   ] Finished updating ant stats in 1.252 seconds
2014-05-20 01:13:26 [DEBUG   ] Finished updating board flags in 1.425 seconds
2014-05-20 01:13:45 [DEBUG   ] Finished updating ant stats in 1.156 seconds
2014-05-20 01:14:05 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 01:14:25 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 01:14:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 01:14:26 [DEBUG   ] Finished updating board flags in 1.370 seconds
2014-05-20 01:14:45 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 01:15:05 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 01:15:25 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 01:15:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 01:15:26 [DEBUG   ] Finished updating board flags in 1.389 seconds
2014-05-20 01:15:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 01:16:05 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 01:16:25 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 01:16:27 [DEBUG   ] Finished updating board flags in 2.042 seconds
2014-05-20 01:16:46 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 01:17:06 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 01:17:06 [DEBUG   ] Got command RPT from MCS: ref #195841
2014-05-20 01:17:06 [DEBUG   ] summary = ERROR
2014-05-20 01:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195841  13 56797  4626894 A  ERRORERROR'
2014-05-20 01:17:06 [DEBUG   ] Got command RPT from MCS: ref #195842
2014-05-20 01:17:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 01:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195842  77 56797  4626895 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 01:17:26 [DEBUG   ] Finished updating ant stats in 1.247 seconds
2014-05-20 01:17:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 01:17:27 [DEBUG   ] Finished updating board flags in 1.978 seconds
2014-05-20 01:17:46 [DEBUG   ] Finished updating ant stats in 1.154 seconds
2014-05-20 01:18:06 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 01:18:26 [DEBUG   ] Finished updating ant stats in 1.050 seconds
2014-05-20 01:18:27 [DEBUG   ] Finished updating board flags in 1.820 seconds
2014-05-20 01:18:46 [DEBUG   ] Finished updating ant stats in 0.954 seconds
2014-05-20 01:19:06 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 01:19:26 [DEBUG   ] Finished updating ant stats in 0.850 seconds
2014-05-20 01:19:27 [DEBUG   ] Finished updating board flags in 1.781 seconds
2014-05-20 01:19:46 [DEBUG   ] Finished updating ant stats in 0.756 seconds
2014-05-20 01:20:06 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 01:20:27 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 01:20:27 [DEBUG   ] Finished updating board flags in 1.617 seconds
2014-05-20 01:20:47 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 01:21:07 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 01:21:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 3
2014-05-20 01:21:27 [DEBUG   ] Finished updating ant stats in 1.246 seconds
2014-05-20 01:21:27 [DEBUG   ] Finished updating board flags in 1.646 seconds
2014-05-20 01:21:47 [DEBUG   ] Finished updating ant stats in 1.157 seconds
2014-05-20 01:22:06 [DEBUG   ] Got command RPT from MCS: ref #195858
2014-05-20 01:22:06 [DEBUG   ] summary = ERROR
2014-05-20 01:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195858  13 56797  4926898 A  ERRORERROR'
2014-05-20 01:22:06 [DEBUG   ] Got command RPT from MCS: ref #195859
2014-05-20 01:22:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 01:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195859  77 56797  4926899 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 01:22:07 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 01:22:27 [DEBUG   ] Finished updating ant stats in 0.847 seconds
2014-05-20 01:22:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-20 01:22:27 [DEBUG   ] Finished updating board flags in 1.613 seconds
2014-05-20 01:22:47 [DEBUG   ] Finished updating ant stats in 0.761 seconds
2014-05-20 01:23:07 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 01:23:27 [DEBUG   ] Finished updating ant stats in 0.643 seconds
2014-05-20 01:23:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 01:23:27 [DEBUG   ] Finished updating board flags in 1.600 seconds
2014-05-20 01:23:47 [DEBUG   ] Finished updating ant stats in 0.558 seconds
2014-05-20 01:24:08 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 01:24:27 [DEBUG   ] Finished updating board flags in 1.592 seconds
2014-05-20 01:24:28 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 01:24:48 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 01:25:08 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 01:25:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 01:25:27 [DEBUG   ] Finished updating board flags in 1.432 seconds
2014-05-20 01:25:28 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 01:25:48 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 01:26:08 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 01:26:27 [DEBUG   ] Finished updating board flags in 1.458 seconds
2014-05-20 01:26:28 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 01:26:48 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 01:27:06 [DEBUG   ] Got command RPT from MCS: ref #195875
2014-05-20 01:27:06 [DEBUG   ] summary = ERROR
2014-05-20 01:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195875  13 56797  5226902 A  ERRORERROR'
2014-05-20 01:27:06 [DEBUG   ] Got command RPT from MCS: ref #195876
2014-05-20 01:27:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 01:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195876  77 56797  5226903 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 01:27:08 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 01:27:27 [DEBUG   ] Finished updating board flags in 1.465 seconds
2014-05-20 01:27:28 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 01:27:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 01:28:08 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 01:28:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 01:28:28 [DEBUG   ] Finished updating ant stats in 0.844 seconds
2014-05-20 01:28:28 [DEBUG   ] Finished updating board flags in 2.329 seconds
2014-05-20 01:28:48 [DEBUG   ] Finished updating ant stats in 0.754 seconds
2014-05-20 01:29:08 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 01:29:28 [DEBUG   ] Finished updating ant stats in 0.647 seconds
2014-05-20 01:29:28 [DEBUG   ] Finished updating board flags in 2.111 seconds
2014-05-20 01:29:48 [DEBUG   ] Finished updating ant stats in 0.559 seconds
2014-05-20 01:30:09 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 01:30:28 [DEBUG   ] Finished updating board flags in 2.013 seconds
2014-05-20 01:30:29 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 01:30:49 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 01:31:09 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 01:31:28 [DEBUG   ] Finished updating board flags in 2.002 seconds
2014-05-20 01:31:29 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 01:31:49 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2014-05-20 01:32:06 [DEBUG   ] Got command RPT from MCS: ref #195892
2014-05-20 01:32:06 [DEBUG   ] summary = ERROR
2014-05-20 01:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195892  13 56797  5526898 A  ERRORERROR'
2014-05-20 01:32:06 [DEBUG   ] Got command RPT from MCS: ref #195893
2014-05-20 01:32:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 01:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195893  77 56797  5526899 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 01:32:09 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 01:32:28 [DEBUG   ] Finished updating board flags in 2.000 seconds
2014-05-20 01:32:29 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 01:32:49 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 01:33:09 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 01:33:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 01:33:28 [DEBUG   ] Finished updating board flags in 1.954 seconds
2014-05-20 01:33:29 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 01:33:50 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 01:34:10 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 01:34:28 [DEBUG   ] Finished updating board flags in 1.785 seconds
2014-05-20 01:34:30 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 01:34:50 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 01:35:10 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 01:35:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-20 01:35:28 [DEBUG   ] Finished updating board flags in 1.618 seconds
2014-05-20 01:35:30 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 01:35:50 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 01:36:10 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 01:36:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-20 01:36:28 [DEBUG   ] Finished updating board flags in 1.591 seconds
2014-05-20 01:36:30 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 01:36:50 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 01:37:06 [DEBUG   ] Got command RPT from MCS: ref #195909
2014-05-20 01:37:06 [DEBUG   ] summary = ERROR
2014-05-20 01:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195909  13 56797  5826898 A  ERRORERROR'
2014-05-20 01:37:06 [DEBUG   ] Got command RPT from MCS: ref #195910
2014-05-20 01:37:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 01:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195910  77 56797  5826899 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 01:37:10 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 01:37:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-20 01:37:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 01:37:28 [DEBUG   ] Finished updating board flags in 1.417 seconds
2014-05-20 01:37:30 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 01:37:50 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 01:38:10 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 01:38:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-20 01:38:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 01:38:28 [DEBUG   ] Finished updating board flags in 1.379 seconds
2014-05-20 01:38:30 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 01:38:51 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 01:39:11 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 01:39:28 [DEBUG   ] Finished updating board flags in 1.369 seconds
2014-05-20 01:39:31 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 01:39:51 [DEBUG   ] Finished updating ant stats in 1.210 seconds
2014-05-20 01:40:11 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 01:40:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 01:40:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 01:40:29 [DEBUG   ] Finished updating board flags in 2.052 seconds
2014-05-20 01:40:31 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 01:40:51 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 01:41:11 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 01:41:29 [DEBUG   ] Finished updating board flags in 1.991 seconds
2014-05-20 01:41:31 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 01:41:51 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 01:42:06 [DEBUG   ] Got command RPT from MCS: ref #195926
2014-05-20 01:42:06 [DEBUG   ] summary = ERROR
2014-05-20 01:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195926  13 56797  6126903 A  ERRORERROR'
2014-05-20 01:42:06 [DEBUG   ] Got command RPT from MCS: ref #195927
2014-05-20 01:42:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 01:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195927  77 56797  6126903 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 01:42:11 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 01:42:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 01:42:29 [DEBUG   ] Finished updating board flags in 1.838 seconds
2014-05-20 01:42:31 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 01:42:51 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2014-05-20 01:43:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 01:43:29 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 01:43:29 [CRITICAL] MGT underflow detected on beam(s) 1
2014-05-20 01:43:29 [DEBUG   ] Finished updating board flags in 1.769 seconds
2014-05-20 01:43:31 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 01:43:51 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 01:44:11 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 01:44:29 [DEBUG   ] Finished updating board flags in 1.612 seconds
2014-05-20 01:44:31 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 01:44:51 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 01:45:12 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 01:45:29 [DEBUG   ] Finished updating board flags in 1.592 seconds
2014-05-20 01:45:32 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 01:45:52 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 01:46:12 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 01:46:29 [DEBUG   ] Finished updating board flags in 1.419 seconds
2014-05-20 01:46:32 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 01:46:52 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 01:47:06 [DEBUG   ] Got command RPT from MCS: ref #195943
2014-05-20 01:47:06 [DEBUG   ] summary = ERROR
2014-05-20 01:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195943  13 56797  6426910 A  ERRORERROR'
2014-05-20 01:47:06 [DEBUG   ] Got command RPT from MCS: ref #195944
2014-05-20 01:47:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 01:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195944  77 56797  6426911 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 01:47:12 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 01:47:29 [DEBUG   ] Finished updating board flags in 1.394 seconds
2014-05-20 01:47:32 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 01:47:52 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 01:48:12 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 01:48:30 [DEBUG   ] Finished updating board flags in 2.224 seconds
2014-05-20 01:48:32 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 01:48:52 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 01:49:12 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 01:49:30 [DEBUG   ] Finished updating board flags in 2.195 seconds
2014-05-20 01:49:32 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 01:49:52 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 01:50:13 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 01:50:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 01:50:30 [DEBUG   ] Finished updating board flags in 1.974 seconds
2014-05-20 01:50:33 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 01:50:53 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 01:51:13 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 01:51:30 [DEBUG   ] Finished updating board flags in 1.814 seconds
2014-05-20 01:51:33 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 01:51:53 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 01:52:06 [DEBUG   ] Got command RPT from MCS: ref #195960
2014-05-20 01:52:06 [DEBUG   ] summary = ERROR
2014-05-20 01:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195960  13 56797  6726906 A  ERRORERROR'
2014-05-20 01:52:06 [DEBUG   ] Got command RPT from MCS: ref #195961
2014-05-20 01:52:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 01:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195961  77 56797  6726907 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 01:52:13 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 01:52:30 [DEBUG   ] Finished updating board flags in 1.790 seconds
2014-05-20 01:52:33 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 01:52:53 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 01:53:13 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 01:53:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-20 01:53:30 [DEBUG   ] Finished updating board flags in 1.616 seconds
2014-05-20 01:53:33 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 01:53:53 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 01:54:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 01:54:30 [DEBUG   ] Finished updating board flags in 1.593 seconds
2014-05-20 01:54:33 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2014-05-20 01:54:54 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 01:55:14 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 01:55:30 [DEBUG   ] Finished updating board flags in 1.416 seconds
2014-05-20 01:55:34 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 01:55:54 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 01:56:14 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 01:56:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 01:56:30 [DEBUG   ] Finished updating board flags in 1.382 seconds
2014-05-20 01:56:34 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 01:56:54 [DEBUG   ] Finished updating ant stats in 0.791 seconds
2014-05-20 01:57:06 [DEBUG   ] Got command RPT from MCS: ref #195977
2014-05-20 01:57:06 [DEBUG   ] summary = ERROR
2014-05-20 01:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195977  13 56797  7026906 A  ERRORERROR'
2014-05-20 01:57:06 [DEBUG   ] Got command RPT from MCS: ref #195978
2014-05-20 01:57:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 01:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195978  77 56797  7026907 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 01:57:14 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 01:57:30 [DEBUG   ] Finished updating board flags in 1.365 seconds
2014-05-20 01:57:34 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 01:57:54 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 01:58:14 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 01:58:31 [DEBUG   ] Finished updating board flags in 2.053 seconds
2014-05-20 01:58:35 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 01:58:55 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 01:59:15 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 01:59:31 [DEBUG   ] Finished updating board flags in 2.010 seconds
2014-05-20 01:59:35 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 01:59:55 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 02:00:15 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 02:00:31 [DEBUG   ] Finished updating board flags in 1.982 seconds
2014-05-20 02:00:35 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 02:00:55 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 02:01:15 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2014-05-20 02:01:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 02:01:31 [DEBUG   ] Finished updating board flags in 1.810 seconds
2014-05-20 02:01:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 02:01:55 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 02:02:06 [DEBUG   ] Got command RPT from MCS: ref #195994
2014-05-20 02:02:06 [DEBUG   ] summary = ERROR
2014-05-20 02:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195994  13 56797  7326906 A  ERRORERROR'
2014-05-20 02:02:06 [DEBUG   ] Got command RPT from MCS: ref #195995
2014-05-20 02:02:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 02:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   195995  77 56797  7326907 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 02:02:15 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 02:02:31 [DEBUG   ] Finished updating board flags in 1.809 seconds
2014-05-20 02:02:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 02:02:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 02:03:15 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 02:03:31 [DEBUG   ] Finished updating board flags in 1.792 seconds
2014-05-20 02:03:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 02:03:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 02:04:15 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 02:04:31 [DEBUG   ] Finished updating board flags in 1.643 seconds
2014-05-20 02:04:35 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 02:04:55 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 02:05:15 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 02:05:31 [DEBUG   ] Finished updating board flags in 1.560 seconds
2014-05-20 02:05:35 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 02:05:56 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 02:06:16 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 02:06:31 [DEBUG   ] Finished updating board flags in 1.429 seconds
2014-05-20 02:06:36 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 02:06:56 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 02:07:06 [DEBUG   ] Got command RPT from MCS: ref #196011
2014-05-20 02:07:06 [DEBUG   ] summary = ERROR
2014-05-20 02:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196011  13 56797  7626906 A  ERRORERROR'
2014-05-20 02:07:06 [DEBUG   ] Got command RPT from MCS: ref #196012
2014-05-20 02:07:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 02:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196012  77 56797  7626907 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 02:07:16 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2014-05-20 02:07:31 [DEBUG   ] Finished updating board flags in 1.427 seconds
2014-05-20 02:07:36 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 02:07:56 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 02:08:16 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 02:08:31 [DEBUG   ] Finished updating board flags in 1.437 seconds
2014-05-20 02:08:36 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2014-05-20 02:08:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 02:09:16 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 02:09:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-20 02:09:31 [DEBUG   ] Finished updating board flags in 1.426 seconds
2014-05-20 02:09:36 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 02:09:56 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 02:10:16 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 02:10:31 [DEBUG   ] Finished updating board flags in 1.430 seconds
2014-05-20 02:10:36 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 02:10:56 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 02:11:17 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 02:11:31 [DEBUG   ] Finished updating board flags in 1.425 seconds
2014-05-20 02:11:37 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 02:11:57 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 02:12:06 [DEBUG   ] Got command RPT from MCS: ref #196028
2014-05-20 02:12:06 [DEBUG   ] summary = ERROR
2014-05-20 02:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196028  13 56797  7926910 A  ERRORERROR'
2014-05-20 02:12:06 [DEBUG   ] Got command RPT from MCS: ref #196029
2014-05-20 02:12:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 02:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196029  77 56797  7926911 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 02:12:17 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 02:12:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2014-05-20 02:12:32 [DEBUG   ] Finished updating board flags in 2.122 seconds
2014-05-20 02:12:37 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 02:12:57 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 02:13:17 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 02:13:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 02:13:32 [DEBUG   ] Finished updating board flags in 1.985 seconds
2014-05-20 02:13:37 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 02:13:57 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 02:14:17 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 02:14:32 [DEBUG   ] Finished updating board flags in 1.814 seconds
2014-05-20 02:14:37 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 02:14:57 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 02:15:17 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 02:15:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 02:15:32 [DEBUG   ] Finished updating board flags in 1.789 seconds
2014-05-20 02:15:37 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 02:15:57 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 02:16:18 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 02:16:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 02:16:32 [DEBUG   ] Finished updating board flags in 1.618 seconds
2014-05-20 02:16:38 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 02:16:58 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 02:17:06 [DEBUG   ] Got command RPT from MCS: ref #196050
2014-05-20 02:17:06 [DEBUG   ] summary = ERROR
2014-05-20 02:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196050  13 56797  8226918 A  ERRORERROR'
2014-05-20 02:17:06 [DEBUG   ] Got command RPT from MCS: ref #196051
2014-05-20 02:17:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 02:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196051  77 56797  8226919 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 02:17:18 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 02:17:32 [DEBUG   ] Finished updating board flags in 1.579 seconds
2014-05-20 02:17:38 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2014-05-20 02:17:58 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 02:18:18 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 02:18:32 [DEBUG   ] Finished updating board flags in 1.422 seconds
2014-05-20 02:18:38 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 02:18:58 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 02:19:18 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 02:19:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 02:19:32 [DEBUG   ] Finished updating board flags in 1.387 seconds
2014-05-20 02:19:39 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 02:19:59 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 02:20:19 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 02:20:32 [DEBUG   ] Finished updating board flags in 1.365 seconds
2014-05-20 02:20:39 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 02:20:59 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 02:21:19 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 02:21:33 [DEBUG   ] Finished updating board flags in 2.055 seconds
2014-05-20 02:21:39 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 02:21:59 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 02:22:06 [DEBUG   ] Got command RPT from MCS: ref #196067
2014-05-20 02:22:06 [DEBUG   ] summary = ERROR
2014-05-20 02:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196067  13 56797  8526918 A  ERRORERROR'
2014-05-20 02:22:06 [DEBUG   ] Got command RPT from MCS: ref #196068
2014-05-20 02:22:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 02:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196068  77 56797  8526919 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 02:22:19 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 02:22:33 [DEBUG   ] Finished updating board flags in 1.988 seconds
2014-05-20 02:22:39 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 02:22:59 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:23:19 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:23:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 02:23:33 [DEBUG   ] Finished updating board flags in 1.810 seconds
2014-05-20 02:23:39 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 02:23:59 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 02:24:19 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 02:24:33 [DEBUG   ] Finished updating board flags in 1.795 seconds
2014-05-20 02:24:40 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 02:25:00 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 02:25:20 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 02:25:33 [DEBUG   ] Finished updating board flags in 1.615 seconds
2014-05-20 02:25:40 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 02:26:00 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 02:26:20 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 02:26:33 [DEBUG   ] Finished updating board flags in 1.589 seconds
2014-05-20 02:26:40 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 02:27:00 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 02:27:06 [DEBUG   ] Got command RPT from MCS: ref #196084
2014-05-20 02:27:06 [DEBUG   ] summary = ERROR
2014-05-20 02:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196084  13 56797  8826914 A  ERRORERROR'
2014-05-20 02:27:06 [DEBUG   ] Got command RPT from MCS: ref #196085
2014-05-20 02:27:06 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 02:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196085  77 56797  8826915 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 02:27:20 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 02:27:33 [DEBUG   ] Finished updating board flags in 1.418 seconds
2014-05-20 02:27:40 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 02:28:00 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 02:28:20 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 02:28:33 [DEBUG   ] Finished updating board flags in 1.379 seconds
2014-05-20 02:28:40 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 02:29:00 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 02:29:20 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 02:29:33 [DEBUG   ] Finished updating board flags in 1.522 seconds
2014-05-20 02:29:40 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 02:30:00 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 02:30:20 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2014-05-20 02:30:34 [DEBUG   ] Finished updating board flags in 2.213 seconds
2014-05-20 02:30:41 [DEBUG   ] Finished updating ant stats in 1.391 seconds
2014-05-20 02:31:01 [DEBUG   ] Finished updating ant stats in 1.210 seconds
2014-05-20 02:31:21 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 02:31:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 02:31:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 02:31:34 [DEBUG   ] Finished updating board flags in 2.091 seconds
2014-05-20 02:31:41 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 02:32:00 [DEBUG   ] Got command INI from MCS: ref #196101
2014-05-20 02:32:00 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2014-05-20 02:32:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   196101   8 56797  9120738 ABOOTING'
2014-05-20 02:32:01 [DEBUG   ] Finished updating ant stats in 1.048 seconds
2014-05-20 02:32:06 [DEBUG   ] Got command RPT from MCS: ref #196102
2014-05-20 02:32:06 [DEBUG   ] summary = BOOTING
2014-05-20 02:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196102  15 56797  9126910 ABOOTINGBOOTING'
2014-05-20 02:32:06 [DEBUG   ] Got command RPT from MCS: ref #196103
2014-05-20 02:32:06 [DEBUG   ] info = Running INI sequence
2014-05-20 02:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196103  28 56797  9126911 ABOOTINGRunning INI sequence'
2014-05-20 02:32:24 [INFO    ] -------------------
2014-05-20 02:32:24 [INFO    ]  LWA board status
2014-05-20 02:32:24 [INFO    ] -------------------
2014-05-20 02:32:24 [INFO    ] DP 1: 2 lwa27
2014-05-20 02:32:24 [INFO    ] DP 1: 3 lwa24
2014-05-20 02:32:24 [INFO    ] DP 1: 4 lwa23
2014-05-20 02:32:24 [INFO    ] DP 1: 5 lwa18
2014-05-20 02:32:24 [INFO    ] DP 1: 6 lwa10
2014-05-20 02:32:24 [INFO    ] DP 1: 7 lwa32
2014-05-20 02:32:24 [INFO    ] DP 1: 8 lwa11
2014-05-20 02:32:24 [INFO    ] DP 1: 9 lwa30
2014-05-20 02:32:24 [INFO    ] DP 1: 10 lwa12
2014-05-20 02:32:24 [INFO    ] DP 1: 11 lwa8
2014-05-20 02:32:24 [INFO    ] DP 1: 12 lwa13
2014-05-20 02:32:24 [INFO    ] DP 1: 13 lwa26
2014-05-20 02:32:24 [INFO    ] DP 1: 14 lwa14
2014-05-20 02:32:24 [INFO    ] DP 1: 16 lwa22
2014-05-20 02:32:24 [INFO    ] DP 1: 17 lwa20
2014-05-20 02:32:24 [INFO    ] DP 1: 18 lwa1
2014-05-20 02:32:24 [INFO    ] DP 1: 19 lwa5
2014-05-20 02:32:24 [INFO    ] DP 1: 20 lwa25
2014-05-20 02:32:24 [INFO    ] DP 1: 21 lwa35
2014-05-20 02:32:24 [INFO    ] DP 1: 22 lwa19
2014-05-20 02:32:24 [INFO    ] DP 1: 23 lwa21
2014-05-20 02:32:24 [INFO    ] DP 1: 24 lwa6
2014-05-20 02:32:24 [INFO    ] DP 1: 25 lwa7
2014-05-20 02:32:24 [INFO    ] DP 1: 26 lwa33
2014-05-20 02:32:24 [INFO    ] DP 1: 27 lwa15
2014-05-20 02:32:24 [INFO    ] DP 1: 28 lwa28
2014-05-20 02:32:24 [INFO    ] DP 2: 1 lwa16
2014-05-20 02:32:24 [INFO    ] DP 2: 15 lwa34
2014-05-20 02:32:24 [INFO    ] Total of 28 boards
2014-05-20 02:32:24 [INFO    ] Total of 260 stands
2014-05-20 02:32:24 [INFO    ] -------------------
2014-05-20 02:32:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4383
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4384
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4385
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4386
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4387
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4388
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4389
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4390
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4391
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4392
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4393
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4394
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4395
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4396
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4397
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4398
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4399
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4400
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4401
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4402
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4403
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4404
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4405
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4406
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4407
2014-05-20 02:32:31 [DEBUG   ] 2014-05-20 02:32:31 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 4408
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4383 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4384 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4385 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4386 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4387 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4388 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4389 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4390 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4391 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4392 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4393 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4394 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4395 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4396 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4397 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4398 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4399 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4400 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4401 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4402 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4403 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4404 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4405 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4406 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4407 exited with status 0
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 4408 exited with status 0
2014-05-20 02:32:44 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] spawn_process: at 2014-05-20 02:32:44 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 4435
2014-05-20 02:32:44 [DEBUG   ] 2014-05-20 02:32:44 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 4436
2014-05-20 02:32:49 [DEBUG   ] 2014-05-20 02:32:49 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 4435 exited with status 0
2014-05-20 02:32:49 [DEBUG   ] 2014-05-20 02:32:49 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 4436 exited with status 0
2014-05-20 02:32:49 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2014-05-20 02:32:49 [DEBUG   ] spawn_process: at 2014-05-20 02:32:49 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2014-05-20 02:32:49 [DEBUG   ] spawn_process: at 2014-05-20 02:32:49 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2014-05-20 02:32:49 [INFO    ] Setting Beam 1 xbars
2014-05-20 02:32:49 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2014-05-20 02:32:49 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2014-05-20 02:32:50 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2014-05-20 02:32:50 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2014-05-20 02:32:50 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2014-05-20 02:32:50 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2014-05-20 02:32:50 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2014-05-20 02:32:50 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2014-05-20 02:32:50 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2014-05-20 02:32:50 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2014-05-20 02:32:50 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2014-05-20 02:32:51 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2014-05-20 02:32:51 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2014-05-20 02:32:51 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2014-05-20 02:32:51 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2014-05-20 02:32:51 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2014-05-20 02:32:51 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2014-05-20 02:32:51 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2014-05-20 02:32:51 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2014-05-20 02:32:51 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2014-05-20 02:32:51 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2014-05-20 02:32:52 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2014-05-20 02:32:52 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2014-05-20 02:32:52 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2014-05-20 02:32:52 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2014-05-20 02:32:52 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2014-05-20 02:32:52 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2014-05-20 02:32:52 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2014-05-20 02:32:52 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2014-05-20 02:32:52 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2014-05-20 02:32:53 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2014-05-20 02:32:53 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2014-05-20 02:32:53 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2014-05-20 02:32:53 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2014-05-20 02:32:53 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2014-05-20 02:32:53 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2014-05-20 02:32:53 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2014-05-20 02:32:53 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2014-05-20 02:32:53 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2014-05-20 02:32:54 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2014-05-20 02:32:54 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2014-05-20 02:32:54 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2014-05-20 02:32:54 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2014-05-20 02:32:54 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2014-05-20 02:32:54 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2014-05-20 02:32:54 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2014-05-20 02:32:54 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2014-05-20 02:32:54 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2014-05-20 02:32:54 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2014-05-20 02:32:55 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2014-05-20 02:32:55 [INFO    ] Setting Beam 2 xbars
2014-05-20 02:32:55 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2014-05-20 02:32:55 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2014-05-20 02:32:55 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2014-05-20 02:32:55 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2014-05-20 02:32:55 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2014-05-20 02:32:55 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2014-05-20 02:32:55 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2014-05-20 02:32:55 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2014-05-20 02:32:56 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2014-05-20 02:32:56 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2014-05-20 02:32:56 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2014-05-20 02:32:56 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2014-05-20 02:32:56 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2014-05-20 02:32:56 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2014-05-20 02:32:56 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2014-05-20 02:32:56 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2014-05-20 02:32:56 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2014-05-20 02:32:57 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2014-05-20 02:32:57 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2014-05-20 02:32:57 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2014-05-20 02:32:57 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2014-05-20 02:32:57 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2014-05-20 02:32:57 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2014-05-20 02:32:57 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2014-05-20 02:32:57 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2014-05-20 02:32:57 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2014-05-20 02:32:58 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2014-05-20 02:32:58 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2014-05-20 02:32:58 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2014-05-20 02:32:58 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2014-05-20 02:32:58 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2014-05-20 02:32:58 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2014-05-20 02:32:58 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2014-05-20 02:32:58 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2014-05-20 02:32:58 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2014-05-20 02:32:58 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2014-05-20 02:32:59 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2014-05-20 02:32:59 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2014-05-20 02:32:59 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2014-05-20 02:32:59 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2014-05-20 02:32:59 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2014-05-20 02:32:59 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2014-05-20 02:32:59 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2014-05-20 02:32:59 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2014-05-20 02:32:59 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2014-05-20 02:33:00 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2014-05-20 02:33:00 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2014-05-20 02:33:00 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2014-05-20 02:33:00 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2014-05-20 02:33:00 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2014-05-20 02:33:00 [INFO    ] Setting Beam 3 xbars
2014-05-20 02:33:00 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2014-05-20 02:33:00 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2014-05-20 02:33:00 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2014-05-20 02:33:00 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2014-05-20 02:33:01 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2014-05-20 02:33:01 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2014-05-20 02:33:01 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2014-05-20 02:33:01 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2014-05-20 02:33:01 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2014-05-20 02:33:01 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2014-05-20 02:33:01 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2014-05-20 02:33:01 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2014-05-20 02:33:01 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2014-05-20 02:33:02 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2014-05-20 02:33:02 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2014-05-20 02:33:02 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2014-05-20 02:33:02 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2014-05-20 02:33:02 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2014-05-20 02:33:02 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2014-05-20 02:33:02 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2014-05-20 02:33:02 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2014-05-20 02:33:02 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2014-05-20 02:33:03 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2014-05-20 02:33:03 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2014-05-20 02:33:03 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2014-05-20 02:33:03 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2014-05-20 02:33:03 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2014-05-20 02:33:03 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2014-05-20 02:33:03 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2014-05-20 02:33:03 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2014-05-20 02:33:03 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2014-05-20 02:33:03 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2014-05-20 02:33:04 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2014-05-20 02:33:04 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2014-05-20 02:33:04 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2014-05-20 02:33:04 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2014-05-20 02:33:04 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2014-05-20 02:33:04 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2014-05-20 02:33:04 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2014-05-20 02:33:04 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2014-05-20 02:33:04 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2014-05-20 02:33:05 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2014-05-20 02:33:05 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2014-05-20 02:33:05 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2014-05-20 02:33:05 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2014-05-20 02:33:05 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2014-05-20 02:33:05 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2014-05-20 02:33:05 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2014-05-20 02:33:05 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2014-05-20 02:33:05 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2014-05-20 02:33:06 [INFO    ] Setting Beam 4 xbars
2014-05-20 02:33:06 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2014-05-20 02:33:06 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-20 02:33:06 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-20 02:33:06 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-20 02:33:06 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-20 02:33:06 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2014-05-20 02:33:06 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2014-05-20 02:33:06 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2014-05-20 02:33:06 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2014-05-20 02:33:07 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2014-05-20 02:33:07 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2014-05-20 02:33:07 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2014-05-20 02:33:07 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2014-05-20 02:33:07 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2014-05-20 02:33:07 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2014-05-20 02:33:07 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2014-05-20 02:33:07 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2014-05-20 02:33:07 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2014-05-20 02:33:07 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2014-05-20 02:33:08 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2014-05-20 02:33:08 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2014-05-20 02:33:08 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2014-05-20 02:33:08 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2014-05-20 02:33:08 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2014-05-20 02:33:08 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2014-05-20 02:33:08 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2014-05-20 02:33:08 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2014-05-20 02:33:08 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2014-05-20 02:33:09 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2014-05-20 02:33:09 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2014-05-20 02:33:09 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2014-05-20 02:33:09 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2014-05-20 02:33:09 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2014-05-20 02:33:09 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2014-05-20 02:33:09 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2014-05-20 02:33:09 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2014-05-20 02:33:09 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2014-05-20 02:33:10 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2014-05-20 02:33:10 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2014-05-20 02:33:10 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2014-05-20 02:33:10 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2014-05-20 02:33:10 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2014-05-20 02:33:10 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2014-05-20 02:33:10 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2014-05-20 02:33:10 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2014-05-20 02:33:10 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2014-05-20 02:33:11 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2014-05-20 02:33:11 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2014-05-20 02:33:11 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2014-05-20 02:33:11 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2014-05-20 02:33:11 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2014-05-20 02:33:11 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2014-05-20 02:33:11 [INFO    ] All xbars set
2014-05-20 02:33:11 [INFO    ] Setting adder chain routing for all beams
2014-05-20 02:33:16 [INFO    ] Setting beam ids
2014-05-20 02:33:26 [INFO    ] Starting beam delay calibration
2014-05-20 02:33:38 [INFO    ] Beam 1 latency is 5739
2014-05-20 02:33:38 [INFO    ] Beam 2 latency is 5753
2014-05-20 02:33:38 [INFO    ] Beam 3 latency is 5751
2014-05-20 02:33:38 [INFO    ] Beam 4 latency is 5734
2014-05-20 02:33:38 [INFO    ] Stop calibration
2014-05-20 02:33:46 [INFO    ] Beam delay calibration successful
2014-05-20 02:33:46 [INFO    ] Calibration passed
2014-05-20 02:33:46 [DEBUG   ] spawn_process: at 2014-05-20 02:33:46 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2014-05-20 02:33:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2014-05-20 02:33:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2014-05-20 02:33:46 [DEBUG   ] spawn_process: at 2014-05-20 02:33:46 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2014-05-20 02:33:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2014-05-20 02:33:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2014-05-20 02:33:46 [DEBUG   ] spawn_process: at 2014-05-20 02:33:46 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2014-05-20 02:33:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2014-05-20 02:33:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2014-05-20 02:33:46 [DEBUG   ] spawn_process: at 2014-05-20 02:33:46 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2014-05-20 02:33:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2014-05-20 02:33:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2014-05-20 02:33:47 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -4 msec
2014-05-20 02:33:47 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 291 ms (1400553227.295 s) with PID 6009 and time 1400553228
2014-05-20 02:33:47 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -4 msec
2014-05-20 02:33:47 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 381 ms (1400553227.386 s) with PID 6012 and time 1400553228
2014-05-20 02:33:48 [DEBUG   ] Finished updating ant stats in 0.864 seconds
2014-05-20 02:33:48 [INFO    ] Started antennas statistics background thread
2014-05-20 02:33:48 [INFO    ] Started board flags background thread
2014-05-20 02:33:48 [INFO    ] Started BAM background thread
2014-05-20 02:33:48 [INFO    ] Finished the INI process (ref. #196101) in 107.770 s
2014-05-20 02:33:49 [DEBUG   ] Finished updating board flags in 1.081 seconds
2014-05-20 02:34:08 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 02:34:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:34:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:34:49 [DEBUG   ] Finished updating board flags in 1.440 seconds
2014-05-20 02:35:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:35:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:35:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:35:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 02:35:50 [DEBUG   ] Finished updating board flags in 2.010 seconds
2014-05-20 02:36:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:36:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:36:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:36:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 02:36:50 [DEBUG   ] Finished updating board flags in 1.925 seconds
2014-05-20 02:37:00 [DEBUG   ] Got command RPT from MCS: ref #196119
2014-05-20 02:37:00 [DEBUG   ] summary = NORMAL
2014-05-20 02:37:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196119  14 56797  9420734 A NORMALNORMAL'
2014-05-20 02:37:06 [DEBUG   ] Got command RPT from MCS: ref #196120
2014-05-20 02:37:06 [DEBUG   ] summary = NORMAL
2014-05-20 02:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196120  14 56797  9426914 A NORMALNORMAL'
2014-05-20 02:37:06 [DEBUG   ] Got command RPT from MCS: ref #196121
2014-05-20 02:37:06 [DEBUG   ] info = System calibrated and operating normally
2014-05-20 02:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196121  48 56797  9426914 A NORMALSystem calibrated and operating normally'
2014-05-20 02:37:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:37:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:37:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:37:49 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 02:37:49 [DEBUG   ] Finished updating board flags in 1.258 seconds
2014-05-20 02:38:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:38:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:38:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:38:49 [WARNING ] MonitorFlags: MGT underflow on hostname lwa8, chip 5, top port
2014-05-20 02:38:50 [CRITICAL] MGT underflow detected on beam(s) 2
2014-05-20 02:38:50 [DEBUG   ] Finished updating board flags in 1.746 seconds
2014-05-20 02:39:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:39:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:39:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:39:49 [DEBUG   ] Finished updating board flags in 1.045 seconds
2014-05-20 02:40:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:40:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:40:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:40:49 [DEBUG   ] Finished updating board flags in 1.075 seconds
2014-05-20 02:41:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:41:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:41:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:41:50 [DEBUG   ] Finished updating board flags in 1.567 seconds
2014-05-20 02:42:06 [DEBUG   ] Got command RPT from MCS: ref #196137
2014-05-20 02:42:06 [DEBUG   ] summary = ERROR
2014-05-20 02:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196137  13 56797  9726922 A  ERRORERROR'
2014-05-20 02:42:06 [DEBUG   ] Got command RPT from MCS: ref #196138
2014-05-20 02:42:06 [DEBUG   ] info = T_NOM2! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 02:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196138  77 56797  9726923 A  ERRORT_NOM2! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 02:42:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:42:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:42:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:42:50 [DEBUG   ] Finished updating board flags in 1.320 seconds
2014-05-20 02:43:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:43:28 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 02:43:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:43:50 [DEBUG   ] Finished updating board flags in 1.211 seconds
2014-05-20 02:44:08 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 02:44:28 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 02:44:48 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 02:44:50 [DEBUG   ] Finished updating board flags in 1.187 seconds
2014-05-20 02:44:58 [DEBUG   ] Got command DRX from MCS: ref #196158
2014-05-20 02:44:58 [INFO    ] sending drx command to lwa16
2014-05-20 02:44:58 [DEBUG   ] run_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 1 45450000.0 7 9 0']
2014-05-20 02:44:58 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-20 02:44:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   196158   8 56797  9898018 A  ERROR'
2014-05-20 02:44:58 [DEBUG   ] Got command DRX from MCS: ref #196159
2014-05-20 02:44:58 [DEBUG   ] drx = exited with status False, code 0x0d
2014-05-20 02:44:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   196159  75 56797  9898019 R  ERROR0x0D! DRX: Beamformer sub-subsystem for requested beam is not ready'
2014-05-20 02:44:58 [DEBUG   ] Got command DRX from MCS: ref #196157
2014-05-20 02:44:58 [INFO    ] sending drx command to lwa16
2014-05-20 02:44:58 [DEBUG   ] run_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 1 45450000.0 7 9 0']
2014-05-20 02:44:58 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-20 02:44:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   196157   8 56797  9898026 A  ERROR'
2014-05-20 02:44:58 [DEBUG   ] Got command DRX from MCS: ref #196160
2014-05-20 02:44:58 [INFO    ] sending drx command to lwa16
2014-05-20 02:44:58 [DEBUG   ] run_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 5 2 65050000.0 7 9 0']
2014-05-20 02:44:58 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-20 02:44:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   196160   8 56797  9898034 A  ERROR'
2014-05-20 02:44:58 [DEBUG   ] Got command DRX from MCS: ref #196161
2014-05-20 02:44:58 [INFO    ] sending drx command to lwa16
2014-05-20 02:44:58 [DEBUG   ] run_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa16', '/lwa/software/drx 1 2 65050000.0 7 9 0']
2014-05-20 02:44:58 [DEBUG   ] drx = exited with status True, code 0x00
2014-05-20 02:44:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   196161   8 56797  9898042 A  ERROR'
2014-05-20 02:44:58 [DEBUG   ] Got command DRX from MCS: ref #196162
2014-05-20 02:44:58 [DEBUG   ] drx = exited with status False, code 0x0d
2014-05-20 02:44:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_DRX   196162  75 56797  9898043 R  ERROR0x0D! DRX: Beamformer sub-subsystem for requested beam is not ready'
2014-05-20 02:44:58 [INFO    ] Finished starting DRX
2014-05-20 02:44:58 [INFO    ] Finished starting DRX
2014-05-20 02:44:58 [INFO    ] Finished starting DRX
2014-05-20 02:44:58 [INFO    ] Finished starting DRX
2014-05-20 02:44:58 [DEBUG   ] Got command BAM from MCS: ref #196165
2014-05-20 02:44:58 [DEBUG   ] bam = exited with status False, code 0x0d
2014-05-20 02:44:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   196165  75 56797  9898210 R  ERROR0x0D! BAM: Beamformer sub-subsystem for requested beam is not ready'
2014-05-20 02:44:58 [DEBUG   ] Got command BAM from MCS: ref #196163
2014-05-20 02:44:58 [INFO    ] Finished processing BAM request
2014-05-20 02:44:58 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-20 02:44:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   196163   8 56797  9898310 A  ERROR'
2014-05-20 02:44:58 [DEBUG   ] Got command BAM from MCS: ref #196164
2014-05-20 02:44:58 [INFO    ] Finished processing BAM request
2014-05-20 02:44:58 [DEBUG   ] bam = exited with status True, code 0x00
2014-05-20 02:44:58 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_BAM   196164   8 56797  9898422 A  ERROR'
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1'] to hostname lwa27
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa27', '/lwa/software/bamx /lwa/test/bamx.tmp2 1']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2'] to hostname lwa24
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa24', '/lwa/software/bamx /lwa/test/bamx.tmp2 2']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3'] to hostname lwa23
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa23', '/lwa/software/bamx /lwa/test/bamx.tmp2 3']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4'] to hostname lwa18
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa18', '/lwa/software/bamx /lwa/test/bamx.tmp2 4']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5'] to hostname lwa10
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa10', '/lwa/software/bamx /lwa/test/bamx.tmp2 5']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6'] to hostname lwa32
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa32', '/lwa/software/bamx /lwa/test/bamx.tmp2 6']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7'] to hostname lwa11
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa11', '/lwa/software/bamx /lwa/test/bamx.tmp2 7']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8'] to hostname lwa30
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa30', '/lwa/software/bamx /lwa/test/bamx.tmp2 8']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9'] to hostname lwa12
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa12', '/lwa/software/bamx /lwa/test/bamx.tmp2 9']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10'] to hostname lwa8
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa8', '/lwa/software/bamx /lwa/test/bamx.tmp2 10']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11'] to hostname lwa13
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa13', '/lwa/software/bamx /lwa/test/bamx.tmp2 11']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12'] to hostname lwa26
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa26', '/lwa/software/bamx /lwa/test/bamx.tmp2 12']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13'] to hostname lwa14
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa14', '/lwa/software/bamx /lwa/test/bamx.tmp2 13']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14'] to hostname lwa22
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa22', '/lwa/software/bamx /lwa/test/bamx.tmp2 14']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15'] to hostname lwa20
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa20', '/lwa/software/bamx /lwa/test/bamx.tmp2 15']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16'] to hostname lwa1
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa1', '/lwa/software/bamx /lwa/test/bamx.tmp2 16']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17'] to hostname lwa5
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa5', '/lwa/software/bamx /lwa/test/bamx.tmp2 17']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18'] to hostname lwa25
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa25', '/lwa/software/bamx /lwa/test/bamx.tmp2 18']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19'] to hostname lwa35
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa35', '/lwa/software/bamx /lwa/test/bamx.tmp2 19']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20'] to hostname lwa19
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa19', '/lwa/software/bamx /lwa/test/bamx.tmp2 20']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21'] to hostname lwa21
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa21', '/lwa/software/bamx /lwa/test/bamx.tmp2 21']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22'] to hostname lwa6
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa6', '/lwa/software/bamx /lwa/test/bamx.tmp2 22']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23'] to hostname lwa7
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa7', '/lwa/software/bamx /lwa/test/bamx.tmp2 23']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24'] to hostname lwa33
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa33', '/lwa/software/bamx /lwa/test/bamx.tmp2 24']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25'] to hostname lwa15
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa15', '/lwa/software/bamx /lwa/test/bamx.tmp2 25']
2014-05-20 02:44:58 [INFO    ] SendBAM: Sent ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26'] to hostname lwa28
2014-05-20 02:44:58 [DEBUG   ] spawn_process: at 2014-05-20 02:44:58 running ['/usr/local/bin/lwa28', '/lwa/software/bamx /lwa/test/bamx.tmp2 26']
2014-05-20 02:45:08 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 02:45:28 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 02:45:48 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 02:45:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 02:45:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 02:45:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 02:45:50 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 02:45:50 [DEBUG   ] Finished updating board flags in 1.007 seconds
2014-05-20 02:46:08 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 02:46:28 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 02:46:48 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 02:46:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 02:46:50 [DEBUG   ] Finished updating board flags in 1.033 seconds
2014-05-20 02:47:00 [DEBUG   ] Got command TBN from MCS: ref #196169
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '1', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '2', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] tbn = exited with status True, code 0x00
2014-05-20 02:47:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBN   196169   8 56797 10020751 A  ERROR'
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '3', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '4', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '5', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '6', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '7', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '8', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '9', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '10', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '11', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '12', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '13', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '14', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '15', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '16', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '17', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '18', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '19', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '20', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '21', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '22', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '23', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '24', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '25', '1400554022']
2014-05-20 02:47:00 [DEBUG   ] spawn_process: at 2014-05-20 02:47:00 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '26', '1400554022']
2014-05-20 02:47:00 [INFO    ] Finished spawning TBN in 0.060 s
2014-05-20 02:47:06 [DEBUG   ] Got command RPT from MCS: ref #196170
2014-05-20 02:47:06 [DEBUG   ] summary = ERROR
2014-05-20 02:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196170  13 56797 10026914 A  ERRORERROR'
2014-05-20 02:47:06 [DEBUG   ] Got command RPT from MCS: ref #196171
2014-05-20 02:47:06 [DEBUG   ] info = T_NOM2! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 02:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196171  77 56797 10026915 A  ERRORT_NOM2! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 02:47:08 [DEBUG   ] Finished updating ant stats in 0.671 seconds
2014-05-20 02:47:28 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 02:47:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 02:47:51 [DEBUG   ] Finished updating board flags in 2.248 seconds
2014-05-20 02:48:09 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2014-05-20 02:48:29 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2014-05-20 02:48:49 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 02:48:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 02:48:51 [DEBUG   ] Finished updating board flags in 2.138 seconds
2014-05-20 02:49:09 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 02:49:29 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 02:49:49 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 02:49:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 02:49:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 02:49:51 [CRITICAL] MGT underflow detected on beam(s) 2, 1
2014-05-20 02:49:51 [DEBUG   ] Finished updating board flags in 2.010 seconds
2014-05-20 02:50:09 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 02:50:29 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 02:50:50 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 02:50:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 02:50:51 [DEBUG   ] Finished updating board flags in 1.991 seconds
2014-05-20 02:51:10 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 02:51:30 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 02:51:50 [DEBUG   ] Finished updating ant stats in 1.243 seconds
2014-05-20 02:51:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 02:51:51 [DEBUG   ] Finished updating board flags in 1.817 seconds
2014-05-20 02:52:06 [DEBUG   ] Got command RPT from MCS: ref #196187
2014-05-20 02:52:06 [DEBUG   ] summary = ERROR
2014-05-20 02:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196187  13 56797 10326922 A  ERRORERROR'
2014-05-20 02:52:06 [DEBUG   ] Got command RPT from MCS: ref #196188
2014-05-20 02:52:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 02:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196188  84 56797 10326923 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 02:52:10 [DEBUG   ] Finished updating ant stats in 1.156 seconds
2014-05-20 02:52:30 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 02:52:50 [DEBUG   ] Finished updating ant stats in 0.849 seconds
2014-05-20 02:52:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-20 02:52:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 02:52:51 [DEBUG   ] Finished updating board flags in 1.787 seconds
2014-05-20 02:53:10 [DEBUG   ] Finished updating ant stats in 0.756 seconds
2014-05-20 02:53:30 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 02:53:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 02:53:50 [DEBUG   ] Finished updating ant stats in 0.645 seconds
2014-05-20 02:53:51 [DEBUG   ] Finished updating board flags in 1.611 seconds
2014-05-20 02:54:10 [DEBUG   ] Finished updating ant stats in 0.555 seconds
2014-05-20 02:54:31 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 02:54:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2014-05-20 02:54:51 [DEBUG   ] Finished updating ant stats in 1.251 seconds
2014-05-20 02:54:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 02:54:51 [DEBUG   ] Finished updating board flags in 1.670 seconds
2014-05-20 02:55:11 [DEBUG   ] Finished updating ant stats in 1.160 seconds
2014-05-20 02:55:31 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 02:55:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 02:55:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 02:55:51 [DEBUG   ] Finished updating ant stats in 1.043 seconds
2014-05-20 02:55:51 [DEBUG   ] Finished updating board flags in 1.587 seconds
2014-05-20 02:56:11 [DEBUG   ] Finished updating ant stats in 0.959 seconds
2014-05-20 02:56:31 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 02:56:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 02:56:51 [DEBUG   ] Finished updating ant stats in 0.846 seconds
2014-05-20 02:56:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 02:56:51 [DEBUG   ] Finished updating board flags in 1.473 seconds
2014-05-20 02:57:06 [DEBUG   ] Got command RPT from MCS: ref #196204
2014-05-20 02:57:06 [DEBUG   ] summary = ERROR
2014-05-20 02:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196204  13 56797 10626922 A  ERRORERROR'
2014-05-20 02:57:06 [DEBUG   ] Got command RPT from MCS: ref #196205
2014-05-20 02:57:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 02:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196205  84 56797 10626923 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 02:57:11 [DEBUG   ] Finished updating ant stats in 0.756 seconds
2014-05-20 02:57:31 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 02:57:51 [DEBUG   ] Finished updating board flags in 1.445 seconds
2014-05-20 02:57:52 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 02:58:12 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 02:58:32 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 02:58:51 [DEBUG   ] Finished updating board flags in 1.428 seconds
2014-05-20 02:58:52 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 02:59:12 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 02:59:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 02:59:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 02:59:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-20 02:59:51 [DEBUG   ] Finished updating board flags in 1.428 seconds
2014-05-20 02:59:52 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 03:00:12 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 03:00:32 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:00:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 03:00:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 03:00:52 [DEBUG   ] Finished updating ant stats in 0.844 seconds
2014-05-20 03:00:52 [DEBUG   ] Finished updating board flags in 2.256 seconds
2014-05-20 03:01:12 [DEBUG   ] Finished updating ant stats in 0.756 seconds
2014-05-20 03:01:32 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 03:01:52 [DEBUG   ] Finished updating ant stats in 0.645 seconds
2014-05-20 03:01:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:01:52 [DEBUG   ] Finished updating board flags in 2.170 seconds
2014-05-20 03:02:06 [DEBUG   ] Got command RPT from MCS: ref #196221
2014-05-20 03:02:06 [DEBUG   ] summary = ERROR
2014-05-20 03:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196221  13 56797 10926926 A  ERRORERROR'
2014-05-20 03:02:06 [DEBUG   ] Got command RPT from MCS: ref #196222
2014-05-20 03:02:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 03:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196222  84 56797 10926927 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 03:02:12 [DEBUG   ] Finished updating ant stats in 0.557 seconds
2014-05-20 03:02:33 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 03:02:51 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 03:02:52 [DEBUG   ] Finished updating board flags in 2.008 seconds
2014-05-20 03:02:53 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 03:03:13 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 03:03:33 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 03:03:52 [DEBUG   ] Finished updating board flags in 2.007 seconds
2014-05-20 03:03:53 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 03:04:13 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:04:33 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 03:04:52 [DEBUG   ] Finished updating board flags in 2.022 seconds
2014-05-20 03:04:53 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 03:05:13 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 03:05:33 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:05:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 03:05:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:05:52 [DEBUG   ] Finished updating board flags in 2.034 seconds
2014-05-20 03:05:53 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 03:06:13 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 03:06:33 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 03:06:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 03:06:52 [DEBUG   ] Finished updating board flags in 1.976 seconds
2014-05-20 03:06:53 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 03:07:06 [DEBUG   ] Got command RPT from MCS: ref #196238
2014-05-20 03:07:06 [DEBUG   ] summary = ERROR
2014-05-20 03:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196238  13 56797 11226922 A  ERRORERROR'
2014-05-20 03:07:06 [DEBUG   ] Got command RPT from MCS: ref #196239
2014-05-20 03:07:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 03:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196239  84 56797 11226923 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 03:07:13 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 03:07:33 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 03:07:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 03:07:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 03:07:52 [DEBUG   ] Finished updating board flags in 1.780 seconds
2014-05-20 03:07:53 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2014-05-20 03:08:13 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 03:08:33 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:08:52 [DEBUG   ] Finished updating board flags in 1.601 seconds
2014-05-20 03:08:53 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 03:09:13 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2014-05-20 03:09:33 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 03:09:52 [DEBUG   ] Finished updating board flags in 1.557 seconds
2014-05-20 03:09:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 03:10:13 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 03:10:33 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 03:10:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 03:10:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:10:52 [DEBUG   ] Finished updating board flags in 1.422 seconds
2014-05-20 03:10:53 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 03:11:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 03:11:33 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 03:11:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 03:11:52 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:11:52 [DEBUG   ] Finished updating board flags in 1.442 seconds
2014-05-20 03:11:53 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 03:12:06 [DEBUG   ] Got command RPT from MCS: ref #196255
2014-05-20 03:12:06 [DEBUG   ] summary = ERROR
2014-05-20 03:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196255  13 56797 11526930 A  ERRORERROR'
2014-05-20 03:12:06 [DEBUG   ] Got command RPT from MCS: ref #196256
2014-05-20 03:12:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 03:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196256  84 56797 11526931 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 03:12:13 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 03:12:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 03:12:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:12:52 [DEBUG   ] Finished updating board flags in 1.490 seconds
2014-05-20 03:12:53 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 03:13:13 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 03:13:33 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 03:13:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 03:13:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:13:53 [DEBUG   ] Finished updating board flags in 1.611 seconds
2014-05-20 03:13:53 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 03:14:13 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 03:14:33 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 03:14:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 03:14:53 [DEBUG   ] Finished updating board flags in 1.647 seconds
2014-05-20 03:14:53 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 03:15:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 03:15:33 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 03:15:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 03:15:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 03:15:53 [DEBUG   ] Finished updating ant stats in 0.646 seconds
2014-05-20 03:15:53 [DEBUG   ] Finished updating board flags in 2.286 seconds
2014-05-20 03:16:13 [DEBUG   ] Finished updating ant stats in 0.559 seconds
2014-05-20 03:16:34 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-20 03:16:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 03:16:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-20 03:16:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:16:53 [DEBUG   ] Finished updating board flags in 2.226 seconds
2014-05-20 03:16:54 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 03:17:06 [DEBUG   ] Got command RPT from MCS: ref #196272
2014-05-20 03:17:06 [DEBUG   ] summary = ERROR
2014-05-20 03:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196272  13 56797 11826930 A  ERRORERROR'
2014-05-20 03:17:06 [DEBUG   ] Got command RPT from MCS: ref #196273
2014-05-20 03:17:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 03:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196273  84 56797 11826931 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 03:17:14 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 03:17:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 03:17:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:17:53 [DEBUG   ] Finished updating board flags in 2.154 seconds
2014-05-20 03:17:54 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2014-05-20 03:18:14 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 03:18:34 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 03:18:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 03:18:53 [DEBUG   ] Finished updating board flags in 1.973 seconds
2014-05-20 03:18:54 [DEBUG   ] Finished updating ant stats in 0.791 seconds
2014-05-20 03:19:14 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 03:19:34 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 03:19:53 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:19:53 [DEBUG   ] Finished updating board flags in 1.784 seconds
2014-05-20 03:19:54 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 03:20:15 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-20 03:20:35 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 03:20:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 03:20:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 03:20:53 [DEBUG   ] Finished updating board flags in 1.608 seconds
2014-05-20 03:20:55 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 03:21:15 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 03:21:35 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 03:21:53 [DEBUG   ] Finished updating board flags in 1.563 seconds
2014-05-20 03:21:55 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2014-05-20 03:22:06 [DEBUG   ] Got command RPT from MCS: ref #196289
2014-05-20 03:22:06 [DEBUG   ] summary = ERROR
2014-05-20 03:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196289  13 56797 12126926 A  ERRORERROR'
2014-05-20 03:22:06 [DEBUG   ] Got command RPT from MCS: ref #196290
2014-05-20 03:22:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 03:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196290  84 56797 12126927 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 03:22:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:22:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 03:22:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 03:22:53 [DEBUG   ] Finished updating board flags in 1.421 seconds
2014-05-20 03:22:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:23:15 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 03:23:35 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 03:23:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 03:23:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:23:53 [DEBUG   ] Finished updating board flags in 1.386 seconds
2014-05-20 03:23:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 03:24:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:24:35 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 03:24:54 [DEBUG   ] Finished updating board flags in 2.269 seconds
2014-05-20 03:24:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:25:15 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 03:25:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:25:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:25:54 [DEBUG   ] Finished updating board flags in 2.163 seconds
2014-05-20 03:25:55 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 03:26:15 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 03:26:35 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 03:26:54 [DEBUG   ] Finished updating board flags in 1.806 seconds
2014-05-20 03:26:55 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2014-05-20 03:27:06 [DEBUG   ] Got command RPT from MCS: ref #196306
2014-05-20 03:27:06 [DEBUG   ] summary = ERROR
2014-05-20 03:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196306  13 56797 12426934 A  ERRORERROR'
2014-05-20 03:27:06 [DEBUG   ] Got command RPT from MCS: ref #196307
2014-05-20 03:27:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 03:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196307  84 56797 12426935 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 03:27:16 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 03:27:36 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 03:27:54 [DEBUG   ] Finished updating board flags in 1.814 seconds
2014-05-20 03:27:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:28:16 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 03:28:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:28:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 03:28:54 [DEBUG   ] Finished updating board flags in 1.813 seconds
2014-05-20 03:28:56 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 03:29:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:29:36 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 03:29:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 03:29:54 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 03:29:54 [CRITICAL] MGT underflow detected on beam(s) 2, 1
2014-05-20 03:29:54 [DEBUG   ] Finished updating board flags in 1.780 seconds
2014-05-20 03:29:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:30:16 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 03:30:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:30:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 03:30:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:30:54 [DEBUG   ] Finished updating board flags in 1.642 seconds
2014-05-20 03:30:56 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 03:31:16 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 03:31:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:31:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 03:31:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 03:31:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:31:54 [DEBUG   ] Finished updating board flags in 1.593 seconds
2014-05-20 03:31:56 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 03:32:06 [DEBUG   ] Got command RPT from MCS: ref #196323
2014-05-20 03:32:06 [DEBUG   ] summary = ERROR
2014-05-20 03:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196323  13 56797 12726934 A  ERRORERROR'
2014-05-20 03:32:06 [DEBUG   ] Got command RPT from MCS: ref #196324
2014-05-20 03:32:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 03:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196324  84 56797 12726935 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 03:32:16 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 03:32:36 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 03:32:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 03:32:54 [DEBUG   ] Finished updating board flags in 1.503 seconds
2014-05-20 03:32:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:33:16 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:33:36 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 03:33:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 03:33:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 03:33:54 [DEBUG   ] Finished updating board flags in 1.572 seconds
2014-05-20 03:33:56 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 03:34:16 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 03:34:36 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 03:34:55 [DEBUG   ] Finished updating board flags in 2.164 seconds
2014-05-20 03:34:56 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 03:35:16 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 03:35:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:35:55 [DEBUG   ] Finished updating board flags in 2.012 seconds
2014-05-20 03:35:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:36:16 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 03:36:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:36:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:36:55 [DEBUG   ] Finished updating board flags in 1.958 seconds
2014-05-20 03:36:56 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:37:06 [DEBUG   ] Got command RPT from MCS: ref #196340
2014-05-20 03:37:06 [DEBUG   ] summary = ERROR
2014-05-20 03:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196340  13 56797 13026930 A  ERRORERROR'
2014-05-20 03:37:06 [DEBUG   ] Got command RPT from MCS: ref #196341
2014-05-20 03:37:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 03:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196341  84 56797 13026931 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 03:37:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:37:36 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 03:37:54 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 03:37:55 [DEBUG   ] Finished updating board flags in 1.838 seconds
2014-05-20 03:37:56 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 03:38:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:38:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:38:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:38:55 [DEBUG   ] Finished updating board flags in 1.814 seconds
2014-05-20 03:38:56 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 03:39:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:39:36 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 03:39:55 [DEBUG   ] Finished updating board flags in 1.725 seconds
2014-05-20 03:39:56 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 03:40:16 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 03:40:36 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 03:40:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 03:40:55 [DEBUG   ] Finished updating board flags in 1.598 seconds
2014-05-20 03:40:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 03:41:16 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 03:41:36 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 03:41:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:41:55 [DEBUG   ] Finished updating board flags in 1.547 seconds
2014-05-20 03:41:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 03:42:06 [DEBUG   ] Got command RPT from MCS: ref #196357
2014-05-20 03:42:06 [DEBUG   ] summary = ERROR
2014-05-20 03:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196357  13 56797 13326934 A  ERRORERROR'
2014-05-20 03:42:06 [DEBUG   ] Got command RPT from MCS: ref #196358
2014-05-20 03:42:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 03:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196358  84 56797 13326935 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 03:42:16 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 03:42:36 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 03:42:55 [DEBUG   ] Finished updating board flags in 1.568 seconds
2014-05-20 03:42:56 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 03:43:17 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 03:43:37 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 03:43:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:43:56 [DEBUG   ] Finished updating board flags in 2.133 seconds
2014-05-20 03:43:57 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 03:44:17 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 03:44:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:44:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 03:44:56 [DEBUG   ] Finished updating board flags in 2.037 seconds
2014-05-20 03:44:57 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:45:17 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 03:45:37 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 03:45:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-20 03:45:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 03:45:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:45:56 [DEBUG   ] Finished updating board flags in 2.023 seconds
2014-05-20 03:45:57 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 03:46:17 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 03:46:37 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 03:46:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 03:46:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 03:46:56 [DEBUG   ] Finished updating board flags in 2.016 seconds
2014-05-20 03:46:57 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 03:47:06 [DEBUG   ] Got command RPT from MCS: ref #196374
2014-05-20 03:47:06 [DEBUG   ] summary = ERROR
2014-05-20 03:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196374  13 56797 13626938 A  ERRORERROR'
2014-05-20 03:47:06 [DEBUG   ] Got command RPT from MCS: ref #196375
2014-05-20 03:47:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 03:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196375  84 56797 13626939 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 03:47:17 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 03:47:37 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:47:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-20 03:47:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:47:56 [DEBUG   ] Finished updating board flags in 1.923 seconds
2014-05-20 03:47:57 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 03:48:17 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 03:48:37 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 03:48:56 [DEBUG   ] Finished updating board flags in 1.800 seconds
2014-05-20 03:48:57 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 03:49:17 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 03:49:37 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 03:49:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 03:49:56 [DEBUG   ] Finished updating board flags in 1.771 seconds
2014-05-20 03:49:57 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 03:50:17 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 03:50:37 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 03:50:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:50:56 [DEBUG   ] Finished updating board flags in 1.611 seconds
2014-05-20 03:50:57 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 03:51:18 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 03:51:38 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 03:51:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 03:51:56 [DEBUG   ] Finished updating board flags in 1.577 seconds
2014-05-20 03:51:58 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 03:52:06 [DEBUG   ] Got command RPT from MCS: ref #196391
2014-05-20 03:52:06 [DEBUG   ] summary = ERROR
2014-05-20 03:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196391  13 56797 13926938 A  ERRORERROR'
2014-05-20 03:52:06 [DEBUG   ] Got command RPT from MCS: ref #196392
2014-05-20 03:52:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 03:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196392  84 56797 13926939 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 03:52:18 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 03:52:38 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 03:52:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:52:56 [DEBUG   ] Finished updating board flags in 1.423 seconds
2014-05-20 03:52:58 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 03:53:18 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:53:38 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 03:53:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 03:53:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2014-05-20 03:53:56 [DEBUG   ] Finished updating board flags in 1.470 seconds
2014-05-20 03:53:58 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 03:54:18 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 03:54:38 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 03:54:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 03:54:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:54:56 [DEBUG   ] Finished updating board flags in 1.472 seconds
2014-05-20 03:54:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 03:55:18 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 03:55:38 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 03:55:55 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 03:55:56 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 03:55:56 [CRITICAL] MGT underflow detected on beam(s) 2, 1
2014-05-20 03:55:56 [DEBUG   ] Finished updating board flags in 1.470 seconds
2014-05-20 03:55:58 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2014-05-20 03:56:18 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 03:56:38 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 03:56:57 [DEBUG   ] Finished updating board flags in 2.197 seconds
2014-05-20 03:56:58 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 03:57:06 [DEBUG   ] Got command RPT from MCS: ref #196408
2014-05-20 03:57:06 [DEBUG   ] summary = ERROR
2014-05-20 03:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196408  13 56797 14226934 A  ERRORERROR'
2014-05-20 03:57:06 [DEBUG   ] Got command RPT from MCS: ref #196409
2014-05-20 03:57:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 03:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196409  84 56797 14226935 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 03:57:18 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 03:57:39 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 03:57:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 03:57:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 03:57:57 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 03:57:57 [DEBUG   ] Finished updating board flags in 1.973 seconds
2014-05-20 03:57:59 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 03:58:19 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 03:58:39 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 03:58:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-20 03:58:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 03:58:57 [DEBUG   ] Finished updating board flags in 1.820 seconds
2014-05-20 03:58:59 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 03:59:19 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 03:59:39 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 03:59:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 03:59:57 [DEBUG   ] Finished updating board flags in 1.815 seconds
2014-05-20 03:59:59 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:00:19 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 04:00:39 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 04:00:57 [DEBUG   ] Finished updating board flags in 1.806 seconds
2014-05-20 04:00:59 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 04:01:19 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 04:01:39 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 04:01:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 04:01:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-20 04:01:57 [DEBUG   ] Finished updating board flags in 1.776 seconds
2014-05-20 04:01:59 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 04:02:06 [DEBUG   ] Got command RPT from MCS: ref #196425
2014-05-20 04:02:06 [DEBUG   ] summary = ERROR
2014-05-20 04:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196425  13 56797 14526939 A  ERRORERROR'
2014-05-20 04:02:06 [DEBUG   ] Got command RPT from MCS: ref #196426
2014-05-20 04:02:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 04:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196426  84 56797 14526939 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 04:02:19 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 04:02:39 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 04:02:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 04:02:57 [DEBUG   ] Finished updating board flags in 1.618 seconds
2014-05-20 04:02:59 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 04:03:19 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 04:03:40 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 04:03:57 [DEBUG   ] Finished updating board flags in 1.588 seconds
2014-05-20 04:04:00 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 04:04:20 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 04:04:40 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:04:56 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 04:04:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 3
2014-05-20 04:04:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:04:57 [DEBUG   ] Finished updating board flags in 1.405 seconds
2014-05-20 04:05:00 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:05:20 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 04:05:40 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 04:05:57 [DEBUG   ] Finished updating board flags in 1.402 seconds
2014-05-20 04:06:00 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 04:06:20 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 04:06:40 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 04:06:56 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 04:06:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:06:57 [DEBUG   ] Finished updating board flags in 1.409 seconds
2014-05-20 04:07:00 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 04:07:06 [DEBUG   ] Got command RPT from MCS: ref #196442
2014-05-20 04:07:06 [DEBUG   ] summary = ERROR
2014-05-20 04:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196442  13 56797 14826946 A  ERRORERROR'
2014-05-20 04:07:06 [DEBUG   ] Got command RPT from MCS: ref #196443
2014-05-20 04:07:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 04:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196443  84 56797 14826947 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 04:07:20 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 04:07:40 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 04:07:57 [DEBUG   ] Finished updating board flags in 1.380 seconds
2014-05-20 04:08:00 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 04:08:20 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 04:08:41 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 04:08:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 04:08:57 [DEBUG   ] Finished updating board flags in 1.367 seconds
2014-05-20 04:09:01 [DEBUG   ] Finished updating ant stats in 1.210 seconds
2014-05-20 04:09:21 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 04:09:41 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:09:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 04:09:58 [DEBUG   ] Finished updating board flags in 2.059 seconds
2014-05-20 04:10:01 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2014-05-20 04:10:21 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 04:10:41 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 04:10:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 04:10:58 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:10:58 [DEBUG   ] Finished updating board flags in 2.081 seconds
2014-05-20 04:11:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 04:11:21 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 04:11:41 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 04:11:58 [DEBUG   ] Finished updating board flags in 2.013 seconds
2014-05-20 04:12:01 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 04:12:06 [DEBUG   ] Got command RPT from MCS: ref #196459
2014-05-20 04:12:06 [DEBUG   ] summary = ERROR
2014-05-20 04:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196459  13 56797 15126946 A  ERRORERROR'
2014-05-20 04:12:06 [DEBUG   ] Got command RPT from MCS: ref #196460
2014-05-20 04:12:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 04:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196460  84 56797 15126947 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 04:12:21 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 04:12:41 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 04:12:58 [DEBUG   ] Finished updating board flags in 1.904 seconds
2014-05-20 04:13:01 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 04:13:21 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 04:13:42 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 04:13:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 04:13:58 [DEBUG   ] Finished updating board flags in 1.805 seconds
2014-05-20 04:14:02 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 04:14:22 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 04:14:42 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:14:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 04:14:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 04:14:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-20 04:14:58 [DEBUG   ] Finished updating board flags in 1.791 seconds
2014-05-20 04:15:02 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 04:15:22 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 04:15:42 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 04:15:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 04:15:58 [DEBUG   ] Finished updating board flags in 1.616 seconds
2014-05-20 04:16:03 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 04:16:23 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 04:16:43 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 04:16:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 04:16:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 04:16:58 [DEBUG   ] Finished updating board flags in 1.590 seconds
2014-05-20 04:17:03 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 04:17:06 [DEBUG   ] Got command RPT from MCS: ref #196476
2014-05-20 04:17:06 [DEBUG   ] summary = ERROR
2014-05-20 04:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196476  13 56797 15426946 A  ERRORERROR'
2014-05-20 04:17:06 [DEBUG   ] Got command RPT from MCS: ref #196477
2014-05-20 04:17:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 04:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196477  84 56797 15426947 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 04:17:23 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 04:17:43 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 04:17:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:17:58 [DEBUG   ] Finished updating board flags in 1.414 seconds
2014-05-20 04:18:03 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:18:23 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:18:43 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 04:18:57 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 04:18:58 [DEBUG   ] Finished updating board flags in 1.396 seconds
2014-05-20 04:19:03 [DEBUG   ] Finished updating ant stats in 0.995 seconds
2014-05-20 04:19:23 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 04:19:43 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 04:19:59 [DEBUG   ] Finished updating board flags in 2.205 seconds
2014-05-20 04:20:03 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 04:20:23 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 04:20:43 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 04:20:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 04:20:59 [DEBUG   ] Finished updating board flags in 2.189 seconds
2014-05-20 04:21:03 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 04:21:23 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 04:21:43 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 04:21:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:21:59 [DEBUG   ] Finished updating board flags in 2.018 seconds
2014-05-20 04:22:04 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 04:22:06 [DEBUG   ] Got command RPT from MCS: ref #196493
2014-05-20 04:22:06 [DEBUG   ] summary = ERROR
2014-05-20 04:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196493  13 56797 15726946 A  ERRORERROR'
2014-05-20 04:22:06 [DEBUG   ] Got command RPT from MCS: ref #196494
2014-05-20 04:22:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 04:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196494  84 56797 15726947 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 04:22:24 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 04:22:44 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 04:22:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-20 04:22:59 [DEBUG   ] Finished updating board flags in 1.985 seconds
2014-05-20 04:23:04 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 04:23:24 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 04:23:44 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 04:23:59 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:23:59 [DEBUG   ] Finished updating board flags in 1.820 seconds
2014-05-20 04:24:04 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 04:24:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 04:24:44 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 04:24:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 04:24:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 04:24:59 [DEBUG   ] Finished updating board flags in 1.786 seconds
2014-05-20 04:25:04 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 04:25:24 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 04:25:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 04:25:58 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 04:25:59 [DEBUG   ] Finished updating board flags in 1.609 seconds
2014-05-20 04:26:04 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 04:26:24 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 04:26:44 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 04:26:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:26:59 [DEBUG   ] Finished updating board flags in 1.607 seconds
2014-05-20 04:27:04 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 04:27:06 [DEBUG   ] Got command RPT from MCS: ref #196510
2014-05-20 04:27:06 [DEBUG   ] summary = ERROR
2014-05-20 04:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196510  13 56797 16026942 A  ERRORERROR'
2014-05-20 04:27:06 [DEBUG   ] Got command RPT from MCS: ref #196511
2014-05-20 04:27:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 04:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196511  84 56797 16026943 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 04:27:25 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 04:27:45 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 04:27:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 04:27:59 [DEBUG   ] Finished updating board flags in 1.587 seconds
2014-05-20 04:28:05 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 04:28:25 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 04:28:45 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 04:28:59 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:28:59 [DEBUG   ] Finished updating board flags in 1.415 seconds
2014-05-20 04:29:05 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 04:29:25 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:29:45 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:29:58 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 04:29:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:29:59 [DEBUG   ] Finished updating board flags in 1.390 seconds
2014-05-20 04:30:05 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 04:30:25 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2014-05-20 04:30:45 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 04:30:59 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:30:59 [DEBUG   ] Finished updating board flags in 1.367 seconds
2014-05-20 04:31:05 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 04:31:25 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 04:31:45 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 04:32:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 04:32:00 [DEBUG   ] Finished updating board flags in 2.053 seconds
2014-05-20 04:32:06 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 04:32:06 [DEBUG   ] Got command RPT from MCS: ref #196527
2014-05-20 04:32:06 [DEBUG   ] summary = ERROR
2014-05-20 04:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196527  13 56797 16326950 A  ERRORERROR'
2014-05-20 04:32:06 [DEBUG   ] Got command RPT from MCS: ref #196528
2014-05-20 04:32:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 04:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196528  84 56797 16326951 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 04:32:26 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 04:32:46 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 04:33:00 [DEBUG   ] Finished updating board flags in 1.986 seconds
2014-05-20 04:33:06 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 04:33:26 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 04:33:46 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 04:33:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 04:34:00 [DEBUG   ] Finished updating board flags in 1.811 seconds
2014-05-20 04:34:06 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 04:34:26 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 04:34:46 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 04:35:00 [DEBUG   ] Finished updating board flags in 1.802 seconds
2014-05-20 04:35:06 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 04:35:26 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 04:35:46 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 04:36:00 [DEBUG   ] Finished updating board flags in 1.799 seconds
2014-05-20 04:36:06 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 04:36:26 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 04:36:46 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 04:37:00 [DEBUG   ] Finished updating board flags in 1.640 seconds
2014-05-20 04:37:06 [DEBUG   ] Got command RPT from MCS: ref #196544
2014-05-20 04:37:06 [DEBUG   ] summary = ERROR
2014-05-20 04:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196544  13 56797 16626946 A  ERRORERROR'
2014-05-20 04:37:06 [DEBUG   ] Got command RPT from MCS: ref #196545
2014-05-20 04:37:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 04:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196545  84 56797 16626947 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 04:37:07 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 04:37:27 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 04:37:47 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 04:38:00 [DEBUG   ] Finished updating board flags in 1.554 seconds
2014-05-20 04:38:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 04:38:27 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 04:38:47 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 04:38:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 04:39:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 04:39:00 [DEBUG   ] Finished updating board flags in 1.417 seconds
2014-05-20 04:39:07 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 04:39:27 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:39:47 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 04:40:00 [DEBUG   ] Finished updating board flags in 1.401 seconds
2014-05-20 04:40:07 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 04:40:27 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 04:40:47 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 04:40:59 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 04:41:00 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 04:41:00 [CRITICAL] MGT underflow detected on beam(s) 2, 1
2014-05-20 04:41:00 [DEBUG   ] Finished updating board flags in 1.397 seconds
2014-05-20 04:41:07 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 04:41:27 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 04:41:47 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 04:42:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 04:42:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 04:42:01 [DEBUG   ] Finished updating board flags in 2.207 seconds
2014-05-20 04:42:06 [DEBUG   ] Got command RPT from MCS: ref #196561
2014-05-20 04:42:06 [DEBUG   ] summary = ERROR
2014-05-20 04:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196561  13 56797 16926954 A  ERRORERROR'
2014-05-20 04:42:06 [DEBUG   ] Got command RPT from MCS: ref #196562
2014-05-20 04:42:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 04:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196562  84 56797 16926955 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 04:42:07 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 04:42:27 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 04:42:47 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 04:43:01 [DEBUG   ] Finished updating board flags in 2.206 seconds
2014-05-20 04:43:08 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 04:43:28 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 04:43:48 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 04:44:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 04:44:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 04:44:01 [DEBUG   ] Finished updating board flags in 2.185 seconds
2014-05-20 04:44:08 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:44:28 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:44:48 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 04:45:00 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 04:45:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 04:45:01 [DEBUG   ] Finished updating board flags in 2.013 seconds
2014-05-20 04:45:08 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 04:45:28 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 04:45:48 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 04:46:01 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 04:46:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:46:01 [DEBUG   ] Finished updating board flags in 2.005 seconds
2014-05-20 04:46:08 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 04:46:28 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 04:46:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 04:47:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 04:47:01 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 04:47:01 [DEBUG   ] Finished updating board flags in 1.994 seconds
2014-05-20 04:47:06 [DEBUG   ] Got command RPT from MCS: ref #196578
2014-05-20 04:47:06 [DEBUG   ] summary = ERROR
2014-05-20 04:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196578  13 56797 17226954 A  ERRORERROR'
2014-05-20 04:47:06 [DEBUG   ] Got command RPT from MCS: ref #196579
2014-05-20 04:47:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 04:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196579  84 56797 17226955 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 04:47:08 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 04:47:28 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 04:47:48 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 04:48:01 [DEBUG   ] Finished updating board flags in 1.856 seconds
2014-05-20 04:48:09 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 04:48:29 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 04:48:49 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 04:49:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 04:49:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 04:49:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:49:01 [DEBUG   ] Finished updating board flags in 1.748 seconds
2014-05-20 04:49:09 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:49:29 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:49:49 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 04:50:01 [DEBUG   ] Finished updating board flags in 1.623 seconds
2014-05-20 04:50:09 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 04:50:29 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 04:50:49 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 04:51:00 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-20 04:51:01 [DEBUG   ] Finished updating board flags in 1.584 seconds
2014-05-20 04:51:09 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 04:51:29 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 04:51:49 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 04:52:01 [DEBUG   ] Finished updating board flags in 1.416 seconds
2014-05-20 04:52:06 [DEBUG   ] Got command RPT from MCS: ref #196595
2014-05-20 04:52:06 [DEBUG   ] summary = ERROR
2014-05-20 04:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196595  13 56797 17526958 A  ERRORERROR'
2014-05-20 04:52:06 [DEBUG   ] Got command RPT from MCS: ref #196596
2014-05-20 04:52:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 04:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196596  84 56797 17526959 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 04:52:09 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 04:52:29 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 04:52:49 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 04:53:01 [DEBUG   ] Finished updating board flags in 1.501 seconds
2014-05-20 04:53:09 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 04:53:30 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 04:53:50 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 04:54:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:54:01 [DEBUG   ] Finished updating board flags in 1.499 seconds
2014-05-20 04:54:10 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 04:54:30 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 04:54:50 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 04:55:02 [DEBUG   ] Finished updating board flags in 2.268 seconds
2014-05-20 04:55:10 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 04:55:30 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 04:55:50 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 04:56:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 04:56:02 [DEBUG   ] Finished updating board flags in 2.124 seconds
2014-05-20 04:56:10 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 04:56:30 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 04:56:50 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 04:57:02 [WARNING ] MonitorFlags: VCCINT 0.976 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:57:02 [DEBUG   ] Finished updating board flags in 2.011 seconds
2014-05-20 04:57:06 [DEBUG   ] Got command RPT from MCS: ref #196612
2014-05-20 04:57:06 [DEBUG   ] summary = ERROR
2014-05-20 04:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196612  13 56797 17826958 A  ERRORERROR'
2014-05-20 04:57:06 [DEBUG   ] Got command RPT from MCS: ref #196613
2014-05-20 04:57:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 04:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196613  84 56797 17826959 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 04:57:10 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 04:57:30 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 04:57:50 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 04:58:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 04:58:02 [DEBUG   ] Finished updating board flags in 1.989 seconds
2014-05-20 04:58:10 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 04:58:30 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 04:58:50 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 04:59:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 04:59:02 [DEBUG   ] Finished updating board flags in 1.822 seconds
2014-05-20 04:59:10 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 04:59:30 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 04:59:50 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 05:00:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 05:00:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:00:02 [DEBUG   ] Finished updating board flags in 1.780 seconds
2014-05-20 05:00:10 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 05:00:30 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 05:00:51 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 05:01:01 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 05:01:02 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 05:01:02 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:01:02 [CRITICAL] MGT underflow detected on beam(s) 2, 1
2014-05-20 05:01:02 [DEBUG   ] Finished updating board flags in 1.620 seconds
2014-05-20 05:01:11 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 05:01:31 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 05:01:51 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 05:02:02 [DEBUG   ] Finished updating board flags in 1.584 seconds
2014-05-20 05:02:06 [DEBUG   ] Got command RPT from MCS: ref #196629
2014-05-20 05:02:06 [DEBUG   ] summary = ERROR
2014-05-20 05:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196629  13 56797 18126950 A  ERRORERROR'
2014-05-20 05:02:06 [DEBUG   ] Got command RPT from MCS: ref #196630
2014-05-20 05:02:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 05:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196630  84 56797 18126951 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 05:02:11 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 05:02:31 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 05:02:51 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 05:03:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 05:03:02 [DEBUG   ] Finished updating board flags in 1.422 seconds
2014-05-20 05:03:11 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 05:03:31 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 05:03:51 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 05:04:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:04:02 [DEBUG   ] Finished updating board flags in 1.397 seconds
2014-05-20 05:04:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:04:31 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 05:04:51 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 05:05:03 [DEBUG   ] Finished updating board flags in 2.204 seconds
2014-05-20 05:05:11 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 05:05:31 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:05:51 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:06:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 05:06:03 [DEBUG   ] Finished updating board flags in 2.186 seconds
2014-05-20 05:06:11 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 05:06:31 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 05:06:52 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 05:07:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:07:03 [DEBUG   ] Finished updating board flags in 2.021 seconds
2014-05-20 05:07:06 [DEBUG   ] Got command RPT from MCS: ref #196646
2014-05-20 05:07:06 [DEBUG   ] summary = ERROR
2014-05-20 05:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196646  13 56797 18426958 A  ERRORERROR'
2014-05-20 05:07:06 [DEBUG   ] Got command RPT from MCS: ref #196647
2014-05-20 05:07:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 05:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196647  84 56797 18426959 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 05:07:12 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 05:07:32 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 05:07:52 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 05:08:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 1
2014-05-20 05:08:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:08:03 [DEBUG   ] Finished updating board flags in 1.985 seconds
2014-05-20 05:08:12 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 05:08:32 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 05:08:52 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 05:09:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 05:09:03 [DEBUG   ] Finished updating board flags in 1.811 seconds
2014-05-20 05:09:12 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:09:32 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 05:09:52 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 05:10:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 05:10:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:10:03 [DEBUG   ] Finished updating board flags in 1.814 seconds
2014-05-20 05:10:12 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 05:10:33 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 05:10:53 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 05:11:03 [DEBUG   ] Finished updating board flags in 1.784 seconds
2014-05-20 05:11:13 [DEBUG   ] Finished updating ant stats in 1.123 seconds
2014-05-20 05:11:33 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 05:11:53 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 05:12:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 05:12:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 05:12:03 [DEBUG   ] Finished updating board flags in 1.607 seconds
2014-05-20 05:12:06 [DEBUG   ] Got command RPT from MCS: ref #196663
2014-05-20 05:12:06 [DEBUG   ] summary = ERROR
2014-05-20 05:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196663  13 56797 18726954 A  ERRORERROR'
2014-05-20 05:12:06 [DEBUG   ] Got command RPT from MCS: ref #196664
2014-05-20 05:12:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 05:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196664  84 56797 18726955 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 05:12:13 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 05:12:33 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 05:12:53 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:13:02 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 05:13:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 05:13:03 [DEBUG   ] Finished updating board flags in 1.611 seconds
2014-05-20 05:13:13 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:13:33 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 05:13:53 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 05:14:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 05:14:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 05:14:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:14:03 [DEBUG   ] Finished updating board flags in 1.692 seconds
2014-05-20 05:14:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:14:33 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:14:53 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 05:15:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 05:15:03 [DEBUG   ] Finished updating board flags in 1.501 seconds
2014-05-20 05:15:13 [DEBUG   ] Finished updating ant stats in 0.594 seconds
2014-05-20 05:15:34 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 05:15:54 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 05:16:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:16:03 [DEBUG   ] Finished updating board flags in 1.408 seconds
2014-05-20 05:16:14 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 05:16:34 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 05:16:54 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 05:17:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:17:03 [DEBUG   ] Finished updating board flags in 1.408 seconds
2014-05-20 05:17:06 [DEBUG   ] Got command RPT from MCS: ref #196680
2014-05-20 05:17:06 [DEBUG   ] summary = ERROR
2014-05-20 05:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196680  13 56797 19026954 A  ERRORERROR'
2014-05-20 05:17:06 [DEBUG   ] Got command RPT from MCS: ref #196681
2014-05-20 05:17:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 05:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196681  84 56797 19026955 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 05:17:14 [DEBUG   ] Finished updating ant stats in 1.195 seconds
2014-05-20 05:17:34 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 05:17:54 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 05:18:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 05:18:03 [DEBUG   ] Finished updating board flags in 1.461 seconds
2014-05-20 05:18:14 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 05:18:34 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 05:18:54 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 05:19:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-20 05:19:03 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 05:19:03 [CRITICAL] MGT underflow detected on beam(s) 2, 1
2014-05-20 05:19:03 [DEBUG   ] Finished updating board flags in 1.459 seconds
2014-05-20 05:19:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:19:34 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 05:19:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 05:20:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 05:20:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-20 05:20:03 [DEBUG   ] Finished updating board flags in 1.461 seconds
2014-05-20 05:20:14 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:20:34 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:20:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 05:21:03 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:21:03 [DEBUG   ] Finished updating board flags in 1.503 seconds
2014-05-20 05:21:14 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2014-05-20 05:21:34 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:21:54 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:22:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 05:22:04 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:22:04 [DEBUG   ] Finished updating board flags in 2.142 seconds
2014-05-20 05:22:06 [DEBUG   ] Got command RPT from MCS: ref #196697
2014-05-20 05:22:06 [DEBUG   ] summary = ERROR
2014-05-20 05:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196697  13 56797 19326958 A  ERRORERROR'
2014-05-20 05:22:06 [DEBUG   ] Got command RPT from MCS: ref #196698
2014-05-20 05:22:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 05:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196698  84 56797 19326959 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 05:22:14 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 05:22:34 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 05:22:54 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 05:23:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-20 05:23:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 05:23:04 [DEBUG   ] Finished updating board flags in 1.985 seconds
2014-05-20 05:23:14 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 05:23:34 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 05:23:54 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 05:24:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 05:24:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 05:24:04 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:24:04 [DEBUG   ] Finished updating board flags in 1.896 seconds
2014-05-20 05:24:14 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2014-05-20 05:24:35 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 05:24:55 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 05:25:04 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:25:04 [DEBUG   ] Finished updating board flags in 1.717 seconds
2014-05-20 05:25:15 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 05:25:35 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 05:25:55 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 05:26:04 [DEBUG   ] Finished updating board flags in 1.585 seconds
2014-05-20 05:26:15 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 05:26:35 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 05:26:55 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 05:27:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 05:27:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 05:27:03 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 05:27:04 [DEBUG   ] Finished updating board flags in 1.397 seconds
2014-05-20 05:27:06 [DEBUG   ] Got command RPT from MCS: ref #196714
2014-05-20 05:27:06 [DEBUG   ] summary = ERROR
2014-05-20 05:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196714  13 56797 19626966 A  ERRORERROR'
2014-05-20 05:27:06 [DEBUG   ] Got command RPT from MCS: ref #196715
2014-05-20 05:27:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 05:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196715  84 56797 19626967 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 05:27:15 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 05:27:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 05:27:55 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 05:28:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 05:28:05 [DEBUG   ] Finished updating board flags in 2.234 seconds
2014-05-20 05:28:15 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 05:28:35 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 05:28:55 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 05:29:05 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:29:05 [DEBUG   ] Finished updating board flags in 2.269 seconds
2014-05-20 05:29:15 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 05:29:35 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 05:29:55 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 05:30:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:30:05 [DEBUG   ] Finished updating board flags in 2.228 seconds
2014-05-20 05:30:15 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 05:30:35 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 05:30:55 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 05:31:05 [DEBUG   ] Finished updating board flags in 2.072 seconds
2014-05-20 05:31:15 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 05:31:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:31:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:32:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 05:32:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-20 05:32:05 [DEBUG   ] Finished updating board flags in 2.017 seconds
2014-05-20 05:32:06 [DEBUG   ] Got command RPT from MCS: ref #196731
2014-05-20 05:32:06 [DEBUG   ] summary = ERROR
2014-05-20 05:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196731  13 56797 19926966 A  ERRORERROR'
2014-05-20 05:32:06 [DEBUG   ] Got command RPT from MCS: ref #196732
2014-05-20 05:32:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 05:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196732  84 56797 19926967 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 05:32:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:32:35 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 05:32:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:33:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 05:33:05 [DEBUG   ] Finished updating board flags in 1.993 seconds
2014-05-20 05:33:15 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:33:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:33:55 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 05:34:05 [DEBUG   ] Finished updating board flags in 1.806 seconds
2014-05-20 05:34:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:34:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:34:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:35:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 05:35:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 05:35:05 [DEBUG   ] Finished updating board flags in 1.813 seconds
2014-05-20 05:35:15 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 05:35:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:35:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:36:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:36:05 [DEBUG   ] Finished updating board flags in 1.872 seconds
2014-05-20 05:36:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:36:35 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 05:36:55 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:37:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 05:37:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 05:37:05 [DEBUG   ] Finished updating board flags in 1.820 seconds
2014-05-20 05:37:06 [DEBUG   ] Got command RPT from MCS: ref #196748
2014-05-20 05:37:06 [DEBUG   ] summary = ERROR
2014-05-20 05:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196748  13 56797 20226962 A  ERRORERROR'
2014-05-20 05:37:06 [DEBUG   ] Got command RPT from MCS: ref #196749
2014-05-20 05:37:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 05:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196749  84 56797 20226963 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 05:37:15 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:37:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:37:55 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 05:38:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 05:38:05 [DEBUG   ] Finished updating board flags in 1.707 seconds
2014-05-20 05:38:15 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:38:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:38:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:39:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 05:39:05 [DEBUG   ] Finished updating board flags in 1.602 seconds
2014-05-20 05:39:15 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 05:39:35 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:39:55 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:40:04 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 05:40:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 05:40:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:40:05 [DEBUG   ] Finished updating board flags in 1.598 seconds
2014-05-20 05:40:15 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:40:35 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 05:40:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:41:05 [DEBUG   ] Finished updating board flags in 1.389 seconds
2014-05-20 05:41:15 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 05:41:35 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 05:41:55 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 05:42:06 [DEBUG   ] Finished updating board flags in 2.207 seconds
2014-05-20 05:42:06 [DEBUG   ] Got command RPT from MCS: ref #196765
2014-05-20 05:42:06 [DEBUG   ] summary = ERROR
2014-05-20 05:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196765  13 56797 20526966 A  ERRORERROR'
2014-05-20 05:42:06 [DEBUG   ] Got command RPT from MCS: ref #196766
2014-05-20 05:42:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 05:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196766  84 56797 20526967 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 05:42:15 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 05:42:35 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 05:42:55 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:43:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:43:06 [DEBUG   ] Finished updating board flags in 2.189 seconds
2014-05-20 05:43:15 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 05:43:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:43:55 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 05:44:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 1
2014-05-20 05:44:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 05:44:06 [DEBUG   ] Finished updating board flags in 2.104 seconds
2014-05-20 05:44:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:44:35 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 05:44:55 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 05:45:06 [DEBUG   ] Finished updating board flags in 1.911 seconds
2014-05-20 05:45:15 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 05:45:35 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 05:45:55 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 05:46:05 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 05:46:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 05:46:06 [DEBUG   ] Finished updating board flags in 1.833 seconds
2014-05-20 05:46:15 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 05:46:35 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 05:46:55 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 05:47:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 05:47:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:47:06 [DEBUG   ] Finished updating board flags in 1.802 seconds
2014-05-20 05:47:06 [DEBUG   ] Got command RPT from MCS: ref #196782
2014-05-20 05:47:06 [DEBUG   ] summary = ERROR
2014-05-20 05:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196782  13 56797 20826962 A  ERRORERROR'
2014-05-20 05:47:06 [DEBUG   ] Got command RPT from MCS: ref #196783
2014-05-20 05:47:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 05:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196783  84 56797 20826963 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 05:47:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:47:35 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 05:47:55 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 05:48:06 [DEBUG   ] Finished updating board flags in 1.773 seconds
2014-05-20 05:48:15 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:48:35 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:48:55 [DEBUG   ] Finished updating ant stats in 0.573 seconds
2014-05-20 05:49:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 05:49:05 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 05:49:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:49:06 [DEBUG   ] Finished updating board flags in 1.617 seconds
2014-05-20 05:49:16 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 05:49:36 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 05:49:56 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 05:50:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 05:50:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 05:50:06 [DEBUG   ] Finished updating board flags in 1.569 seconds
2014-05-20 05:50:16 [DEBUG   ] Finished updating ant stats in 1.193 seconds
2014-05-20 05:50:36 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 05:50:56 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 05:51:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 05:51:06 [DEBUG   ] Finished updating board flags in 1.460 seconds
2014-05-20 05:51:16 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 05:51:36 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 05:51:56 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 05:52:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 05:52:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:52:06 [DEBUG   ] Finished updating board flags in 1.462 seconds
2014-05-20 05:52:06 [DEBUG   ] Got command RPT from MCS: ref #196799
2014-05-20 05:52:06 [DEBUG   ] summary = ERROR
2014-05-20 05:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196799  13 56797 21126962 A  ERRORERROR'
2014-05-20 05:52:06 [DEBUG   ] Got command RPT from MCS: ref #196800
2014-05-20 05:52:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 05:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196800  84 56797 21126963 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 05:52:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:52:36 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:52:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:53:06 [DEBUG   ] Finished updating board flags in 1.474 seconds
2014-05-20 05:53:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 05:53:36 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:53:56 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 05:54:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 05:54:07 [DEBUG   ] Finished updating board flags in 1.580 seconds
2014-05-20 05:54:16 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 05:54:36 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 05:54:56 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 05:55:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 05:55:07 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:55:07 [DEBUG   ] Finished updating board flags in 2.127 seconds
2014-05-20 05:55:16 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 05:55:36 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 05:55:57 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 05:56:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 05:56:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 05:56:07 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 05:56:07 [DEBUG   ] Finished updating board flags in 1.910 seconds
2014-05-20 05:56:17 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 05:56:37 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 05:56:57 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 05:57:06 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 05:57:06 [DEBUG   ] Got command RPT from MCS: ref #196816
2014-05-20 05:57:06 [DEBUG   ] summary = ERROR
2014-05-20 05:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196816  13 56797 21426962 A  ERRORERROR'
2014-05-20 05:57:06 [DEBUG   ] Got command RPT from MCS: ref #196817
2014-05-20 05:57:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 05:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196817  84 56797 21426963 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 05:57:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 05:57:07 [DEBUG   ] Finished updating board flags in 1.805 seconds
2014-05-20 05:57:17 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 05:57:37 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 05:57:57 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 05:58:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 05:58:07 [DEBUG   ] Finished updating board flags in 1.790 seconds
2014-05-20 05:58:17 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 05:58:37 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2014-05-20 05:58:57 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 05:59:06 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 05:59:07 [DEBUG   ] Finished updating board flags in 1.620 seconds
2014-05-20 05:59:17 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 05:59:37 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 05:59:57 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 06:00:07 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:00:07 [DEBUG   ] Finished updating board flags in 1.586 seconds
2014-05-20 06:00:17 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2014-05-20 06:00:37 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 06:00:58 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 06:01:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 06:01:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa6, chip 4
2014-05-20 06:01:07 [DEBUG   ] Finished updating board flags in 1.414 seconds
2014-05-20 06:01:18 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 06:01:38 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 06:01:58 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 06:02:06 [DEBUG   ] Got command RPT from MCS: ref #196833
2014-05-20 06:02:06 [DEBUG   ] summary = ERROR
2014-05-20 06:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196833  13 56797 21726966 A  ERRORERROR'
2014-05-20 06:02:06 [DEBUG   ] Got command RPT from MCS: ref #196834
2014-05-20 06:02:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 06:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196834  84 56797 21726967 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 06:02:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 06:02:07 [DEBUG   ] Finished updating board flags in 1.392 seconds
2014-05-20 06:02:18 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 06:02:38 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 06:02:58 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 06:03:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 06:03:07 [DEBUG   ] Finished updating board flags in 1.372 seconds
2014-05-20 06:03:18 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 06:03:38 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 06:03:58 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 06:04:08 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:04:08 [DEBUG   ] Finished updating board flags in 2.054 seconds
2014-05-20 06:04:18 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 06:04:38 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 06:04:58 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 06:05:08 [DEBUG   ] Finished updating board flags in 1.991 seconds
2014-05-20 06:05:18 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 06:05:38 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 06:05:58 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 06:06:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 06:06:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 06:06:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:06:08 [DEBUG   ] Finished updating board flags in 1.796 seconds
2014-05-20 06:06:18 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2014-05-20 06:06:39 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2014-05-20 06:06:59 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 06:07:06 [DEBUG   ] Got command RPT from MCS: ref #196850
2014-05-20 06:07:06 [DEBUG   ] summary = ERROR
2014-05-20 06:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196850  13 56797 22026970 A  ERRORERROR'
2014-05-20 06:07:06 [DEBUG   ] Got command RPT from MCS: ref #196851
2014-05-20 06:07:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 06:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196851  84 56797 22026971 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 06:07:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:07:08 [DEBUG   ] Finished updating board flags in 1.610 seconds
2014-05-20 06:07:19 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 06:07:39 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 06:07:59 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 06:08:08 [DEBUG   ] Finished updating board flags in 1.609 seconds
2014-05-20 06:08:19 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 06:08:39 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 06:08:59 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 06:09:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 06:09:07 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 06:09:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:09:08 [DEBUG   ] Finished updating board flags in 1.581 seconds
2014-05-20 06:09:20 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 06:09:40 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 06:10:00 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 06:10:08 [DEBUG   ] Finished updating board flags in 1.423 seconds
2014-05-20 06:10:20 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 06:10:40 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 06:11:00 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 06:11:07 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 06:11:08 [DEBUG   ] Finished updating board flags in 1.379 seconds
2014-05-20 06:11:20 [DEBUG   ] Finished updating ant stats in 1.195 seconds
2014-05-20 06:11:40 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 06:12:00 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 06:12:06 [DEBUG   ] Got command RPT from MCS: ref #196867
2014-05-20 06:12:06 [DEBUG   ] summary = ERROR
2014-05-20 06:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196867  13 56797 22326966 A  ERRORERROR'
2014-05-20 06:12:06 [DEBUG   ] Got command RPT from MCS: ref #196868
2014-05-20 06:12:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 06:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196868  84 56797 22326967 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 06:12:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 06:12:08 [DEBUG   ] Finished updating board flags in 1.367 seconds
2014-05-20 06:12:20 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 06:12:40 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 06:13:00 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 06:13:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 06:13:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 06:13:09 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:13:09 [DEBUG   ] Finished updating board flags in 2.050 seconds
2014-05-20 06:13:20 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 06:13:40 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 06:14:01 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 06:14:08 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 06:14:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 06:14:09 [DEBUG   ] Finished updating board flags in 1.990 seconds
2014-05-20 06:14:21 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 06:14:41 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 06:15:01 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 06:15:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 06:15:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 06:15:09 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:15:09 [DEBUG   ] Finished updating board flags in 1.842 seconds
2014-05-20 06:15:21 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 06:15:41 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 06:16:01 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 06:16:09 [DEBUG   ] Finished updating board flags in 1.767 seconds
2014-05-20 06:16:21 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 06:16:41 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 06:17:01 [DEBUG   ] Finished updating ant stats in 1.077 seconds
2014-05-20 06:17:06 [DEBUG   ] Got command RPT from MCS: ref #196884
2014-05-20 06:17:06 [DEBUG   ] summary = ERROR
2014-05-20 06:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196884  13 56797 22626966 A  ERRORERROR'
2014-05-20 06:17:06 [DEBUG   ] Got command RPT from MCS: ref #196885
2014-05-20 06:17:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 06:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196885  84 56797 22626967 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 06:17:09 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:17:09 [DEBUG   ] Finished updating board flags in 1.613 seconds
2014-05-20 06:17:21 [DEBUG   ] Finished updating ant stats in 0.927 seconds
2014-05-20 06:17:41 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 06:18:01 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 06:18:09 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 06:18:09 [CRITICAL] MGT underflow detected on beam(s) 2, 1
2014-05-20 06:18:09 [DEBUG   ] Finished updating board flags in 1.607 seconds
2014-05-20 06:18:21 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 06:18:41 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 06:19:01 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 06:19:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 06:19:09 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:19:09 [DEBUG   ] Finished updating board flags in 1.583 seconds
2014-05-20 06:19:21 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2014-05-20 06:19:42 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 06:20:02 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 06:20:10 [DEBUG   ] Finished updating board flags in 2.419 seconds
2014-05-20 06:20:22 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 06:20:42 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 06:21:02 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 06:21:08 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 06:21:09 [DEBUG   ] Finished updating board flags in 1.400 seconds
2014-05-20 06:21:22 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 06:21:42 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 06:22:02 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 06:22:06 [DEBUG   ] Got command RPT from MCS: ref #196901
2014-05-20 06:22:06 [DEBUG   ] summary = ERROR
2014-05-20 06:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196901  13 56797 22926974 A  ERRORERROR'
2014-05-20 06:22:06 [DEBUG   ] Got command RPT from MCS: ref #196902
2014-05-20 06:22:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 06:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196902  84 56797 22926975 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 06:22:10 [DEBUG   ] Finished updating board flags in 2.202 seconds
2014-05-20 06:22:22 [DEBUG   ] Finished updating ant stats in 0.994 seconds
2014-05-20 06:22:42 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 06:23:02 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 06:23:09 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 06:23:10 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 06:23:10 [DEBUG   ] Finished updating board flags in 2.208 seconds
2014-05-20 06:23:22 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 06:23:42 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 06:24:02 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 06:24:10 [DEBUG   ] Finished updating board flags in 2.185 seconds
2014-05-20 06:24:22 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 06:24:43 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2014-05-20 06:25:03 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 06:25:10 [DEBUG   ] Finished updating board flags in 2.017 seconds
2014-05-20 06:25:23 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 06:25:43 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 06:26:03 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 06:26:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-20 06:26:10 [DEBUG   ] Finished updating board flags in 1.985 seconds
2014-05-20 06:26:23 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 06:26:43 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 06:27:03 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 06:27:06 [DEBUG   ] Got command RPT from MCS: ref #196918
2014-05-20 06:27:06 [DEBUG   ] summary = ERROR
2014-05-20 06:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196918  13 56797 23226974 A  ERRORERROR'
2014-05-20 06:27:06 [DEBUG   ] Got command RPT from MCS: ref #196919
2014-05-20 06:27:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 06:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196919  84 56797 23226975 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 06:27:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 06:27:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 06:27:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 06:27:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 06:27:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:27:10 [DEBUG   ] Finished updating board flags in 1.874 seconds
2014-05-20 06:27:23 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 06:27:43 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 06:28:03 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 06:28:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 06:28:10 [DEBUG   ] Finished updating board flags in 1.729 seconds
2014-05-20 06:28:23 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 06:28:44 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 06:29:04 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 06:29:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 06:29:10 [DEBUG   ] Finished updating board flags in 1.614 seconds
2014-05-20 06:29:24 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 06:29:44 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 06:30:04 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 06:30:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 06:30:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-20 06:30:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 06:30:10 [DEBUG   ] Finished updating board flags in 1.595 seconds
2014-05-20 06:30:24 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 06:30:44 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 06:31:04 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 06:31:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa10, chip 1
2014-05-20 06:31:09 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 06:31:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:31:10 [DEBUG   ] Finished updating board flags in 1.435 seconds
2014-05-20 06:31:24 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 06:31:44 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 06:32:04 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 06:32:06 [DEBUG   ] Got command RPT from MCS: ref #196935
2014-05-20 06:32:06 [DEBUG   ] summary = ERROR
2014-05-20 06:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196935  13 56797 23526974 A  ERRORERROR'
2014-05-20 06:32:06 [DEBUG   ] Got command RPT from MCS: ref #196936
2014-05-20 06:32:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 06:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196936  84 56797 23526975 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 06:32:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 06:32:10 [DEBUG   ] Finished updating board flags in 1.425 seconds
2014-05-20 06:32:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 06:32:44 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 06:33:04 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 06:33:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 06:33:10 [DEBUG   ] Finished updating board flags in 1.427 seconds
2014-05-20 06:33:24 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 06:33:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 06:34:04 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 06:34:10 [DEBUG   ] Finished updating board flags in 1.431 seconds
2014-05-20 06:34:24 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 06:34:44 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 06:35:05 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 06:35:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 06:35:10 [DEBUG   ] Finished updating board flags in 1.476 seconds
2014-05-20 06:35:25 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 06:35:45 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 06:36:05 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 06:36:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 06:36:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:36:10 [DEBUG   ] Finished updating board flags in 1.437 seconds
2014-05-20 06:36:25 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 06:36:45 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 06:37:05 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 06:37:06 [DEBUG   ] Got command RPT from MCS: ref #196952
2014-05-20 06:37:06 [DEBUG   ] summary = ERROR
2014-05-20 06:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196952  13 56797 23826974 A  ERRORERROR'
2014-05-20 06:37:06 [DEBUG   ] Got command RPT from MCS: ref #196953
2014-05-20 06:37:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 06:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196953  84 56797 23826975 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 06:37:11 [DEBUG   ] Finished updating board flags in 2.214 seconds
2014-05-20 06:37:25 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 06:37:45 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 06:38:05 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 06:38:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 06:38:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2014-05-20 06:38:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:38:11 [DEBUG   ] Finished updating board flags in 2.182 seconds
2014-05-20 06:38:25 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 06:38:46 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 06:39:06 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 06:39:11 [DEBUG   ] Finished updating board flags in 1.997 seconds
2014-05-20 06:39:26 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 06:39:46 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 06:40:06 [DEBUG   ] Finished updating ant stats in 1.210 seconds
2014-05-20 06:40:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 06:40:11 [DEBUG   ] Finished updating board flags in 1.788 seconds
2014-05-20 06:40:26 [DEBUG   ] Finished updating ant stats in 1.195 seconds
2014-05-20 06:40:46 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 06:41:06 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 06:41:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 06:41:11 [DEBUG   ] Finished updating board flags in 1.617 seconds
2014-05-20 06:41:26 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 06:41:46 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 06:42:06 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 06:42:06 [DEBUG   ] Got command RPT from MCS: ref #196969
2014-05-20 06:42:06 [DEBUG   ] summary = ERROR
2014-05-20 06:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196969  13 56797 24126970 A  ERRORERROR'
2014-05-20 06:42:06 [DEBUG   ] Got command RPT from MCS: ref #196970
2014-05-20 06:42:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 06:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196970  84 56797 24126971 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 06:42:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 06:42:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 06:42:11 [DEBUG   ] Finished updating board flags in 1.515 seconds
2014-05-20 06:42:26 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 06:42:46 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 06:43:06 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 06:43:11 [DEBUG   ] Finished updating board flags in 1.466 seconds
2014-05-20 06:43:26 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 06:43:46 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 06:44:06 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 06:44:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 06:44:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:44:11 [DEBUG   ] Finished updating board flags in 1.457 seconds
2014-05-20 06:44:26 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 06:44:46 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2014-05-20 06:45:06 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2014-05-20 06:45:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:45:11 [DEBUG   ] Finished updating board flags in 1.474 seconds
2014-05-20 06:45:26 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 06:45:46 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 06:46:06 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 06:46:10 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 06:46:11 [DEBUG   ] Finished updating board flags in 1.470 seconds
2014-05-20 06:46:26 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 06:46:46 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 06:47:06 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 06:47:06 [DEBUG   ] Got command RPT from MCS: ref #196986
2014-05-20 06:47:06 [DEBUG   ] summary = ERROR
2014-05-20 06:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196986  13 56797 24426978 A  ERRORERROR'
2014-05-20 06:47:06 [DEBUG   ] Got command RPT from MCS: ref #196987
2014-05-20 06:47:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 06:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   196987  84 56797 24426979 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 06:47:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:47:12 [DEBUG   ] Finished updating board flags in 2.238 seconds
2014-05-20 06:47:26 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 06:47:46 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 06:48:06 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 06:48:11 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 06:48:12 [DEBUG   ] Finished updating board flags in 2.224 seconds
2014-05-20 06:48:26 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 06:48:46 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 06:49:06 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 06:49:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2014-05-20 06:49:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:49:12 [DEBUG   ] Finished updating board flags in 2.090 seconds
2014-05-20 06:49:26 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 06:49:46 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 06:50:07 [DEBUG   ] Finished updating ant stats in 1.602 seconds
2014-05-20 06:50:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 06:50:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2014-05-20 06:50:12 [DEBUG   ] Finished updating board flags in 2.052 seconds
2014-05-20 06:50:26 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 06:50:47 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 06:51:07 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 06:51:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-20 06:51:12 [DEBUG   ] Finished updating board flags in 1.969 seconds
2014-05-20 06:51:27 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 06:51:47 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 06:52:06 [DEBUG   ] Got command RPT from MCS: ref #197003
2014-05-20 06:52:06 [DEBUG   ] summary = ERROR
2014-05-20 06:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197003  13 56797 24726970 A  ERRORERROR'
2014-05-20 06:52:06 [DEBUG   ] Got command RPT from MCS: ref #197004
2014-05-20 06:52:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 06:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197004  84 56797 24726971 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 06:52:07 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 06:52:12 [DEBUG   ] Finished updating board flags in 1.788 seconds
2014-05-20 06:52:27 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2014-05-20 06:52:47 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 06:53:07 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 06:53:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 06:53:12 [DEBUG   ] Finished updating board flags in 1.602 seconds
2014-05-20 06:53:27 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 06:53:47 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 06:54:07 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 06:54:12 [DEBUG   ] Finished updating board flags in 1.607 seconds
2014-05-20 06:54:27 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 06:54:47 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 06:55:07 [DEBUG   ] Finished updating ant stats in 0.615 seconds
2014-05-20 06:55:12 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:55:12 [DEBUG   ] Finished updating board flags in 1.632 seconds
2014-05-20 06:55:27 [DEBUG   ] Finished updating ant stats in 0.589 seconds
2014-05-20 06:55:48 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 06:56:08 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 06:56:11 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 06:56:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa30, chip 5
2014-05-20 06:56:12 [DEBUG   ] Finished updating board flags in 1.560 seconds
2014-05-20 06:56:28 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 06:56:48 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 06:57:06 [DEBUG   ] Got command RPT from MCS: ref #197020
2014-05-20 06:57:06 [DEBUG   ] summary = ERROR
2014-05-20 06:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197020  13 56797 25026974 A  ERRORERROR'
2014-05-20 06:57:06 [DEBUG   ] Got command RPT from MCS: ref #197021
2014-05-20 06:57:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 06:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197021  84 56797 25026975 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 06:57:08 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 06:57:11 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 06:57:12 [DEBUG   ] Finished updating board flags in 1.424 seconds
2014-05-20 06:57:28 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 06:57:48 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 06:58:08 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 06:58:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 06:58:12 [DEBUG   ] Finished updating board flags in 1.445 seconds
2014-05-20 06:58:28 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 06:58:48 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 06:59:08 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 06:59:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 06:59:12 [DEBUG   ] Finished updating board flags in 1.528 seconds
2014-05-20 06:59:28 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 06:59:48 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 07:00:08 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 07:00:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 07:00:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 07:00:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:00:12 [DEBUG   ] Finished updating board flags in 1.562 seconds
2014-05-20 07:00:28 [DEBUG   ] Finished updating ant stats in 0.993 seconds
2014-05-20 07:00:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:01:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:01:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 07:01:13 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 07:01:13 [CRITICAL] MGT underflow detected on beam(s) 2, 1
2014-05-20 07:01:13 [DEBUG   ] Finished updating board flags in 2.119 seconds
2014-05-20 07:01:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 07:01:48 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 07:02:06 [DEBUG   ] Got command RPT from MCS: ref #197037
2014-05-20 07:02:06 [DEBUG   ] summary = ERROR
2014-05-20 07:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197037  13 56797 25326982 A  ERRORERROR'
2014-05-20 07:02:06 [DEBUG   ] Got command RPT from MCS: ref #197038
2014-05-20 07:02:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 07:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197038  84 56797 25326983 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 07:02:08 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:02:13 [DEBUG   ] Finished updating board flags in 2.061 seconds
2014-05-20 07:02:28 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 07:02:48 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 07:03:08 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 07:03:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 07:03:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:03:13 [DEBUG   ] Finished updating board flags in 2.017 seconds
2014-05-20 07:03:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:03:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:04:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 07:04:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 07:04:13 [DEBUG   ] Finished updating board flags in 1.861 seconds
2014-05-20 07:04:28 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:04:48 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:05:08 [DEBUG   ] Finished updating ant stats in 0.817 seconds
2014-05-20 07:05:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 07:05:13 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:05:13 [DEBUG   ] Finished updating board flags in 1.845 seconds
2014-05-20 07:05:28 [DEBUG   ] Finished updating ant stats in 0.785 seconds
2014-05-20 07:05:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 07:06:08 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 07:06:13 [DEBUG   ] Finished updating board flags in 1.767 seconds
2014-05-20 07:06:28 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 07:06:48 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 07:07:06 [DEBUG   ] Got command RPT from MCS: ref #197054
2014-05-20 07:07:06 [DEBUG   ] summary = ERROR
2014-05-20 07:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197054  13 56797 25626982 A  ERRORERROR'
2014-05-20 07:07:06 [DEBUG   ] Got command RPT from MCS: ref #197055
2014-05-20 07:07:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 07:07:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197055  84 56797 25626983 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 07:07:08 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 07:07:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 07:07:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:07:13 [DEBUG   ] Finished updating board flags in 1.613 seconds
2014-05-20 07:07:28 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 07:07:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 07:08:08 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 07:08:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 07:08:13 [DEBUG   ] Finished updating board flags in 1.580 seconds
2014-05-20 07:08:28 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2014-05-20 07:08:49 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 07:09:09 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 07:09:13 [DEBUG   ] Finished updating board flags in 1.476 seconds
2014-05-20 07:09:29 [DEBUG   ] Finished updating ant stats in 1.193 seconds
2014-05-20 07:09:49 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 07:10:09 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 07:10:12 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 07:10:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 07:10:13 [DEBUG   ] Finished updating board flags in 1.490 seconds
2014-05-20 07:10:29 [DEBUG   ] Finished updating ant stats in 0.993 seconds
2014-05-20 07:10:49 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 07:11:09 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:11:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 07:11:13 [DEBUG   ] Finished updating board flags in 1.479 seconds
2014-05-20 07:11:29 [DEBUG   ] Finished updating ant stats in 0.786 seconds
2014-05-20 07:11:49 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 07:12:06 [DEBUG   ] Got command RPT from MCS: ref #197071
2014-05-20 07:12:06 [DEBUG   ] summary = ERROR
2014-05-20 07:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197071  13 56797 25926978 A  ERRORERROR'
2014-05-20 07:12:06 [DEBUG   ] Got command RPT from MCS: ref #197072
2014-05-20 07:12:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 07:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197072  84 56797 25926979 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 07:12:09 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 07:12:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 07:12:14 [DEBUG   ] Finished updating board flags in 2.182 seconds
2014-05-20 07:12:29 [DEBUG   ] Finished updating ant stats in 0.595 seconds
2014-05-20 07:12:50 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 07:13:10 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 07:13:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2014-05-20 07:13:14 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:13:14 [DEBUG   ] Finished updating board flags in 1.987 seconds
2014-05-20 07:13:30 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 07:13:50 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 07:14:10 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 07:14:14 [DEBUG   ] Finished updating board flags in 1.809 seconds
2014-05-20 07:14:30 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 07:14:50 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 07:15:10 [DEBUG   ] Finished updating ant stats in 1.014 seconds
2014-05-20 07:15:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:15:14 [DEBUG   ] Finished updating board flags in 1.833 seconds
2014-05-20 07:15:30 [DEBUG   ] Finished updating ant stats in 0.990 seconds
2014-05-20 07:15:50 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 07:16:10 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 07:16:14 [DEBUG   ] Finished updating board flags in 1.757 seconds
2014-05-20 07:16:30 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 07:16:50 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 07:17:06 [DEBUG   ] Got command RPT from MCS: ref #197088
2014-05-20 07:17:06 [DEBUG   ] summary = ERROR
2014-05-20 07:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197088  13 56797 26226978 A  ERRORERROR'
2014-05-20 07:17:06 [DEBUG   ] Got command RPT from MCS: ref #197089
2014-05-20 07:17:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 07:17:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197089  84 56797 26226979 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 07:17:10 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 07:17:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 07:17:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 07:17:14 [DEBUG   ] Finished updating board flags in 1.617 seconds
2014-05-20 07:17:30 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:17:50 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 07:18:10 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 07:18:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 07:18:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 07:18:14 [DEBUG   ] Finished updating board flags in 1.644 seconds
2014-05-20 07:18:30 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 07:18:50 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:19:10 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 07:19:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-20 07:19:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 07:19:14 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:19:14 [DEBUG   ] Finished updating board flags in 1.621 seconds
2014-05-20 07:19:30 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 07:19:50 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 07:20:10 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:20:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 07:20:14 [DEBUG   ] Finished updating board flags in 1.684 seconds
2014-05-20 07:20:30 [DEBUG   ] Finished updating ant stats in 0.789 seconds
2014-05-20 07:20:50 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 07:21:10 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 07:21:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 07:21:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:21:14 [DEBUG   ] Finished updating board flags in 1.452 seconds
2014-05-20 07:21:30 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 07:21:50 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 07:22:06 [DEBUG   ] Got command RPT from MCS: ref #197105
2014-05-20 07:22:06 [DEBUG   ] summary = ERROR
2014-05-20 07:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197105  13 56797 26526978 A  ERRORERROR'
2014-05-20 07:22:06 [DEBUG   ] Got command RPT from MCS: ref #197106
2014-05-20 07:22:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 07:22:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197106  84 56797 26526979 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 07:22:10 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 07:22:14 [DEBUG   ] Finished updating board flags in 1.500 seconds
2014-05-20 07:22:30 [DEBUG   ] Finished updating ant stats in 0.597 seconds
2014-05-20 07:22:51 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-20 07:23:11 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 07:23:13 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 07:23:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 07:23:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 07:23:14 [DEBUG   ] Finished updating board flags in 1.501 seconds
2014-05-20 07:23:31 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 07:23:51 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 07:24:11 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 07:24:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-20 07:24:14 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 07:24:15 [DEBUG   ] Finished updating board flags in 2.224 seconds
2014-05-20 07:24:31 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 07:24:51 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 07:25:11 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 07:25:15 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 07:25:15 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:25:15 [DEBUG   ] Finished updating board flags in 2.213 seconds
2014-05-20 07:25:31 [DEBUG   ] Finished updating ant stats in 1.193 seconds
2014-05-20 07:25:51 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 07:26:11 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 07:26:15 [DEBUG   ] Finished updating board flags in 2.163 seconds
2014-05-20 07:26:31 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 07:26:51 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 07:27:06 [DEBUG   ] Got command RPT from MCS: ref #197122
2014-05-20 07:27:06 [DEBUG   ] summary = ERROR
2014-05-20 07:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197122  13 56797 26826990 A  ERRORERROR'
2014-05-20 07:27:06 [DEBUG   ] Got command RPT from MCS: ref #197123
2014-05-20 07:27:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 07:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197123  84 56797 26826991 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 07:27:11 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 07:27:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 07:27:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 07:27:15 [DEBUG   ] Finished updating board flags in 2.065 seconds
2014-05-20 07:27:31 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 07:27:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:28:11 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 07:28:15 [DEBUG   ] Finished updating board flags in 1.942 seconds
2014-05-20 07:28:31 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 07:28:51 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:29:11 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 07:29:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:29:15 [DEBUG   ] Finished updating board flags in 1.842 seconds
2014-05-20 07:29:31 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2014-05-20 07:29:51 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 07:30:11 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2014-05-20 07:30:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:30:15 [DEBUG   ] Finished updating board flags in 1.835 seconds
2014-05-20 07:30:31 [DEBUG   ] Finished updating ant stats in 0.590 seconds
2014-05-20 07:30:52 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-20 07:31:12 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 07:31:15 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:31:15 [DEBUG   ] Finished updating board flags in 1.734 seconds
2014-05-20 07:31:32 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 07:31:52 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 07:32:06 [DEBUG   ] Got command RPT from MCS: ref #197139
2014-05-20 07:32:06 [DEBUG   ] summary = ERROR
2014-05-20 07:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197139  13 56797 27126990 A  ERRORERROR'
2014-05-20 07:32:06 [DEBUG   ] Got command RPT from MCS: ref #197140
2014-05-20 07:32:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 07:32:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197140  84 56797 27126991 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 07:32:12 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 07:32:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 07:32:15 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:32:15 [DEBUG   ] Finished updating board flags in 1.622 seconds
2014-05-20 07:32:32 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 07:32:52 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 07:33:12 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 07:33:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 07:33:15 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:33:15 [DEBUG   ] Finished updating board flags in 1.575 seconds
2014-05-20 07:33:32 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 07:33:52 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 07:34:12 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 07:34:14 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 07:34:15 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:34:15 [DEBUG   ] Finished updating board flags in 1.416 seconds
2014-05-20 07:34:32 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 07:34:52 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:35:12 [DEBUG   ] Finished updating ant stats in 0.811 seconds
2014-05-20 07:35:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 07:35:15 [DEBUG   ] Finished updating board flags in 1.453 seconds
2014-05-20 07:35:32 [DEBUG   ] Finished updating ant stats in 0.795 seconds
2014-05-20 07:35:52 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 07:36:12 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 07:36:15 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 07:36:15 [DEBUG   ] Finished updating board flags in 1.387 seconds
2014-05-20 07:36:32 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 07:36:52 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 07:37:06 [DEBUG   ] Got command RPT from MCS: ref #197156
2014-05-20 07:37:06 [DEBUG   ] summary = ERROR
2014-05-20 07:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197156  13 56797 27426994 A  ERRORERROR'
2014-05-20 07:37:06 [DEBUG   ] Got command RPT from MCS: ref #197157
2014-05-20 07:37:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 07:37:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197157  84 56797 27426995 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 07:37:13 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 07:37:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 07:37:16 [DEBUG   ] Finished updating board flags in 2.165 seconds
2014-05-20 07:37:33 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 07:37:53 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 07:38:13 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 07:38:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 07:38:16 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 07:38:16 [DEBUG   ] Finished updating board flags in 2.007 seconds
2014-05-20 07:38:33 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 07:38:53 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 07:39:13 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 07:39:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 07:39:16 [DEBUG   ] Finished updating board flags in 1.996 seconds
2014-05-20 07:39:33 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 07:39:53 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 07:40:13 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 07:40:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 07:40:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-20 07:40:16 [DEBUG   ] Finished updating board flags in 1.803 seconds
2014-05-20 07:40:33 [DEBUG   ] Finished updating ant stats in 0.592 seconds
2014-05-20 07:40:54 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 07:41:14 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 07:41:16 [DEBUG   ] Finished updating board flags in 1.804 seconds
2014-05-20 07:41:34 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 07:41:54 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 07:42:06 [DEBUG   ] Got command RPT from MCS: ref #197173
2014-05-20 07:42:06 [DEBUG   ] summary = ERROR
2014-05-20 07:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197173  13 56797 27726990 A  ERRORERROR'
2014-05-20 07:42:06 [DEBUG   ] Got command RPT from MCS: ref #197174
2014-05-20 07:42:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 07:42:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197174  84 56797 27726991 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 07:42:14 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 07:42:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 07:42:16 [DEBUG   ] Finished updating board flags in 1.788 seconds
2014-05-20 07:42:34 [DEBUG   ] Finished updating ant stats in 1.193 seconds
2014-05-20 07:42:54 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 07:43:14 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:43:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 07:43:16 [DEBUG   ] Finished updating board flags in 1.617 seconds
2014-05-20 07:43:34 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 07:43:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 07:44:14 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 07:44:16 [DEBUG   ] Finished updating board flags in 1.597 seconds
2014-05-20 07:44:34 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 07:44:54 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 07:45:15 [DEBUG   ] Finished updating ant stats in 1.392 seconds
2014-05-20 07:45:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 07:45:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 07:45:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 07:45:16 [DEBUG   ] Finished updating board flags in 1.401 seconds
2014-05-20 07:45:35 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 07:45:55 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 07:46:15 [DEBUG   ] Finished updating ant stats in 1.246 seconds
2014-05-20 07:46:15 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 07:46:16 [DEBUG   ] Finished updating board flags in 1.406 seconds
2014-05-20 07:46:35 [DEBUG   ] Finished updating ant stats in 1.155 seconds
2014-05-20 07:46:55 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 07:47:06 [DEBUG   ] Got command RPT from MCS: ref #197190
2014-05-20 07:47:06 [DEBUG   ] summary = ERROR
2014-05-20 07:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197190  13 56797 28026986 A  ERRORERROR'
2014-05-20 07:47:06 [DEBUG   ] Got command RPT from MCS: ref #197191
2014-05-20 07:47:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 07:47:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197191  84 56797 28026987 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 07:47:15 [DEBUG   ] Finished updating ant stats in 1.043 seconds
2014-05-20 07:47:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 07:47:16 [DEBUG   ] Finished updating board flags in 1.383 seconds
2014-05-20 07:47:35 [DEBUG   ] Finished updating ant stats in 0.960 seconds
2014-05-20 07:47:55 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 07:48:15 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 07:48:16 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:48:16 [DEBUG   ] Finished updating board flags in 1.369 seconds
2014-05-20 07:48:35 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2014-05-20 07:48:55 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 07:49:15 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 07:49:17 [DEBUG   ] Finished updating board flags in 2.054 seconds
2014-05-20 07:49:36 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 07:49:56 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 07:50:16 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 07:50:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:50:17 [DEBUG   ] Finished updating board flags in 1.988 seconds
2014-05-20 07:50:36 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2014-05-20 07:50:56 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 07:51:16 [DEBUG   ] Finished updating ant stats in 1.248 seconds
2014-05-20 07:51:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:51:17 [DEBUG   ] Finished updating board flags in 1.807 seconds
2014-05-20 07:51:36 [DEBUG   ] Finished updating ant stats in 1.152 seconds
2014-05-20 07:51:56 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 07:52:06 [DEBUG   ] Got command RPT from MCS: ref #197207
2014-05-20 07:52:06 [DEBUG   ] summary = ERROR
2014-05-20 07:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197207  13 56797 28326998 A  ERRORERROR'
2014-05-20 07:52:06 [DEBUG   ] Got command RPT from MCS: ref #197208
2014-05-20 07:52:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 07:52:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197208  84 56797 28326999 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 07:52:16 [DEBUG   ] Finished updating ant stats in 1.048 seconds
2014-05-20 07:52:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 07:52:17 [DEBUG   ] Finished updating board flags in 1.804 seconds
2014-05-20 07:52:36 [DEBUG   ] Finished updating ant stats in 0.953 seconds
2014-05-20 07:52:56 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 07:53:16 [DEBUG   ] Finished updating ant stats in 0.646 seconds
2014-05-20 07:53:17 [DEBUG   ] Finished updating board flags in 1.800 seconds
2014-05-20 07:53:36 [DEBUG   ] Finished updating ant stats in 0.558 seconds
2014-05-20 07:53:57 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-20 07:54:17 [DEBUG   ] Finished updating ant stats in 1.250 seconds
2014-05-20 07:54:17 [DEBUG   ] Finished updating board flags in 1.856 seconds
2014-05-20 07:54:37 [DEBUG   ] Finished updating ant stats in 1.158 seconds
2014-05-20 07:54:57 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 07:55:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 07:55:16 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa24, chip 4
2014-05-20 07:55:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 07:55:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 07:55:17 [DEBUG   ] Finished updating ant stats in 1.042 seconds
2014-05-20 07:55:17 [DEBUG   ] Finished updating board flags in 1.797 seconds
2014-05-20 07:55:37 [DEBUG   ] Finished updating ant stats in 0.960 seconds
2014-05-20 07:55:57 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 07:56:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 07:56:17 [DEBUG   ] Finished updating ant stats in 0.844 seconds
2014-05-20 07:56:17 [DEBUG   ] Finished updating board flags in 1.611 seconds
2014-05-20 07:56:37 [DEBUG   ] Finished updating ant stats in 0.756 seconds
2014-05-20 07:56:57 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 07:57:06 [DEBUG   ] Got command RPT from MCS: ref #197224
2014-05-20 07:57:06 [DEBUG   ] summary = ERROR
2014-05-20 07:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197224  13 56797 28626998 A  ERRORERROR'
2014-05-20 07:57:06 [DEBUG   ] Got command RPT from MCS: ref #197225
2014-05-20 07:57:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 07:57:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197225  84 56797 28626999 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 07:57:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 07:57:17 [DEBUG   ] Finished updating ant stats in 0.647 seconds
2014-05-20 07:57:17 [DEBUG   ] Finished updating board flags in 1.605 seconds
2014-05-20 07:57:37 [DEBUG   ] Finished updating ant stats in 0.557 seconds
2014-05-20 07:57:58 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 07:58:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 07:58:17 [DEBUG   ] Finished updating board flags in 1.577 seconds
2014-05-20 07:58:18 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 07:58:38 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 07:58:58 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 07:59:17 [DEBUG   ] Finished updating board flags in 1.427 seconds
2014-05-20 07:59:18 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 07:59:38 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 07:59:58 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 08:00:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 08:00:17 [DEBUG   ] Finished updating board flags in 1.502 seconds
2014-05-20 08:00:18 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2014-05-20 08:00:38 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 08:00:58 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 08:01:18 [DEBUG   ] Finished updating ant stats in 0.845 seconds
2014-05-20 08:01:18 [DEBUG   ] Finished updating board flags in 2.236 seconds
2014-05-20 08:01:38 [DEBUG   ] Finished updating ant stats in 0.757 seconds
2014-05-20 08:01:58 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 08:02:06 [DEBUG   ] Got command RPT from MCS: ref #197241
2014-05-20 08:02:06 [DEBUG   ] summary = ERROR
2014-05-20 08:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197241  13 56797 28926998 A  ERRORERROR'
2014-05-20 08:02:06 [DEBUG   ] Got command RPT from MCS: ref #197242
2014-05-20 08:02:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 08:02:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197242  84 56797 28926999 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 08:02:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 1
2014-05-20 08:02:18 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 08:02:18 [DEBUG   ] Finished updating ant stats in 0.647 seconds
2014-05-20 08:02:18 [DEBUG   ] Finished updating board flags in 2.182 seconds
2014-05-20 08:02:38 [DEBUG   ] Finished updating ant stats in 0.556 seconds
2014-05-20 08:02:59 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 08:03:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 08:03:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2014-05-20 08:03:18 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:03:18 [DEBUG   ] Finished updating board flags in 2.024 seconds
2014-05-20 08:03:19 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 08:03:39 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 08:03:59 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 08:04:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 08:04:18 [DEBUG   ] Finished updating board flags in 1.951 seconds
2014-05-20 08:04:19 [DEBUG   ] Finished updating ant stats in 1.194 seconds
2014-05-20 08:04:39 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:04:59 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:05:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:05:18 [DEBUG   ] Finished updating board flags in 1.844 seconds
2014-05-20 08:05:19 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 08:05:39 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 08:05:59 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:06:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 08:06:18 [DEBUG   ] Finished updating board flags in 1.828 seconds
2014-05-20 08:06:19 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 08:06:39 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:06:59 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:07:07 [DEBUG   ] Got command RPT from MCS: ref #197258
2014-05-20 08:07:07 [DEBUG   ] summary = ERROR
2014-05-20 08:07:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197258  13 56797 29227002 A  ERRORERROR'
2014-05-20 08:07:07 [DEBUG   ] Got command RPT from MCS: ref #197259
2014-05-20 08:07:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 08:07:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197259  84 56797 29227003 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 08:07:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 08:07:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 08:07:18 [DEBUG   ] Finished updating board flags in 1.840 seconds
2014-05-20 08:07:19 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:07:39 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 08:07:59 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:08:17 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 08:08:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 08:08:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 08:08:18 [DEBUG   ] Finished updating board flags in 1.696 seconds
2014-05-20 08:08:19 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:08:39 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:08:59 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 08:09:17 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 08:09:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:09:18 [DEBUG   ] Finished updating board flags in 1.647 seconds
2014-05-20 08:09:19 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:09:39 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 08:09:59 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:10:18 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:10:18 [DEBUG   ] Finished updating board flags in 1.558 seconds
2014-05-20 08:10:19 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:10:39 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:10:59 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:11:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 08:11:18 [DEBUG   ] Finished updating board flags in 1.557 seconds
2014-05-20 08:11:19 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:11:39 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 08:11:59 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:12:06 [DEBUG   ] Got command RPT from MCS: ref #197275
2014-05-20 08:12:06 [DEBUG   ] summary = ERROR
2014-05-20 08:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197275  13 56797 29526994 A  ERRORERROR'
2014-05-20 08:12:06 [DEBUG   ] Got command RPT from MCS: ref #197276
2014-05-20 08:12:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 08:12:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197276  84 56797 29526995 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 08:12:19 [DEBUG   ] Finished updating board flags in 1.640 seconds
2014-05-20 08:12:19 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 08:12:39 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 08:12:59 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 08:13:19 [DEBUG   ] Finished updating ant stats in 0.842 seconds
2014-05-20 08:13:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 08:13:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:13:19 [DEBUG   ] Finished updating board flags in 2.301 seconds
2014-05-20 08:13:39 [DEBUG   ] Finished updating ant stats in 0.758 seconds
2014-05-20 08:13:59 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 08:14:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 08:14:19 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 08:14:19 [DEBUG   ] Finished updating ant stats in 0.644 seconds
2014-05-20 08:14:19 [DEBUG   ] Finished updating board flags in 2.208 seconds
2014-05-20 08:14:39 [DEBUG   ] Finished updating ant stats in 0.556 seconds
2014-05-20 08:15:00 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-20 08:15:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:15:19 [DEBUG   ] Finished updating board flags in 2.120 seconds
2014-05-20 08:15:20 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 08:15:40 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 08:16:00 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 08:16:19 [DEBUG   ] Finished updating board flags in 2.022 seconds
2014-05-20 08:16:20 [DEBUG   ] Finished updating ant stats in 1.193 seconds
2014-05-20 08:16:40 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2014-05-20 08:17:00 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 08:17:07 [DEBUG   ] Got command RPT from MCS: ref #197292
2014-05-20 08:17:07 [DEBUG   ] summary = ERROR
2014-05-20 08:17:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197292  13 56797 29827002 A  ERRORERROR'
2014-05-20 08:17:07 [DEBUG   ] Got command RPT from MCS: ref #197293
2014-05-20 08:17:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 08:17:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197293  84 56797 29827003 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 08:17:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:17:19 [DEBUG   ] Finished updating board flags in 1.974 seconds
2014-05-20 08:17:20 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 08:17:40 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 08:18:00 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 08:18:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:18:19 [DEBUG   ] Finished updating board flags in 1.834 seconds
2014-05-20 08:18:20 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 08:18:40 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 08:19:00 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 08:19:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 08:19:19 [DEBUG   ] Finished updating board flags in 1.766 seconds
2014-05-20 08:19:20 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2014-05-20 08:19:40 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 08:20:00 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 08:20:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 08:20:19 [DEBUG   ] Finished updating board flags in 1.687 seconds
2014-05-20 08:20:20 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 08:20:40 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 08:21:00 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 08:21:18 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 08:21:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 08:21:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 08:21:19 [DEBUG   ] Finished updating board flags in 1.520 seconds
2014-05-20 08:21:20 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 08:21:40 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 08:22:00 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 08:22:07 [DEBUG   ] Got command RPT from MCS: ref #197309
2014-05-20 08:22:07 [DEBUG   ] summary = ERROR
2014-05-20 08:22:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197309  13 56797 30127002 A  ERRORERROR'
2014-05-20 08:22:07 [DEBUG   ] Got command RPT from MCS: ref #197310
2014-05-20 08:22:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 08:22:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197310  84 56797 30127003 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 08:22:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:22:19 [DEBUG   ] Finished updating board flags in 1.586 seconds
2014-05-20 08:22:20 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 08:22:40 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 08:23:00 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 08:23:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 08:23:20 [DEBUG   ] Finished updating ant stats in 0.650 seconds
2014-05-20 08:23:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa21, chip 1
2014-05-20 08:23:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:23:20 [DEBUG   ] Finished updating board flags in 2.264 seconds
2014-05-20 08:23:40 [DEBUG   ] Finished updating ant stats in 0.553 seconds
2014-05-20 08:24:01 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 08:24:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 08:24:20 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:24:20 [DEBUG   ] Finished updating board flags in 2.225 seconds
2014-05-20 08:24:21 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 08:24:41 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 08:25:01 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 08:25:19 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 08:25:20 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:25:20 [DEBUG   ] Finished updating board flags in 2.150 seconds
2014-05-20 08:25:21 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 08:25:41 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2014-05-20 08:26:01 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:26:20 [DEBUG   ] Finished updating board flags in 2.019 seconds
2014-05-20 08:26:21 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:26:41 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:27:01 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:27:06 [DEBUG   ] Got command RPT from MCS: ref #197326
2014-05-20 08:27:06 [DEBUG   ] summary = ERROR
2014-05-20 08:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197326  13 56797 30426998 A  ERRORERROR'
2014-05-20 08:27:06 [DEBUG   ] Got command RPT from MCS: ref #197327
2014-05-20 08:27:06 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 08:27:06 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197327  84 56797 30426999 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 08:27:20 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 08:27:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:27:20 [CRITICAL] MGT underflow detected on beam(s) 2, 1
2014-05-20 08:27:20 [DEBUG   ] Finished updating board flags in 2.031 seconds
2014-05-20 08:27:21 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:27:41 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:28:01 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:28:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 08:28:20 [DEBUG   ] Finished updating board flags in 2.015 seconds
2014-05-20 08:28:21 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 08:28:41 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 08:29:01 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:29:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 08:29:20 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:29:20 [DEBUG   ] Finished updating board flags in 1.926 seconds
2014-05-20 08:29:21 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:29:41 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 08:30:01 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:30:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 08:30:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 08:30:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:30:20 [DEBUG   ] Finished updating board flags in 1.871 seconds
2014-05-20 08:30:21 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:30:41 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:31:01 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 08:31:19 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 08:31:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 08:31:20 [DEBUG   ] Finished updating board flags in 1.726 seconds
2014-05-20 08:31:21 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 08:31:41 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 08:32:01 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 08:32:07 [DEBUG   ] Got command RPT from MCS: ref #197343
2014-05-20 08:32:07 [DEBUG   ] summary = ERROR
2014-05-20 08:32:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197343  13 56797 30727006 A  ERRORERROR'
2014-05-20 08:32:07 [DEBUG   ] Got command RPT from MCS: ref #197344
2014-05-20 08:32:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 08:32:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197344  84 56797 30727007 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 08:32:20 [DEBUG   ] Finished updating board flags in 1.624 seconds
2014-05-20 08:32:21 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2014-05-20 08:32:41 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 08:33:01 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 08:33:20 [DEBUG   ] Finished updating board flags in 1.566 seconds
2014-05-20 08:33:21 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 08:33:41 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 08:34:01 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 08:34:20 [DEBUG   ] Finished updating board flags in 1.479 seconds
2014-05-20 08:34:21 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 08:34:41 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 08:35:01 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 08:35:20 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 08:35:20 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 08:35:20 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:35:20 [DEBUG   ] Finished updating board flags in 1.504 seconds
2014-05-20 08:35:22 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 08:35:42 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 08:36:02 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 08:36:20 [DEBUG   ] Finished updating board flags in 1.500 seconds
2014-05-20 08:36:22 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 08:36:42 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 08:37:02 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 08:37:07 [DEBUG   ] Got command RPT from MCS: ref #197360
2014-05-20 08:37:07 [DEBUG   ] summary = ERROR
2014-05-20 08:37:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197360  13 56797 31027010 A  ERRORERROR'
2014-05-20 08:37:07 [DEBUG   ] Got command RPT from MCS: ref #197361
2014-05-20 08:37:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 08:37:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197361  84 56797 31027011 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 08:37:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 08:37:21 [DEBUG   ] Finished updating board flags in 2.229 seconds
2014-05-20 08:37:22 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 08:37:42 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:38:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:38:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2014-05-20 08:38:21 [DEBUG   ] Finished updating board flags in 2.127 seconds
2014-05-20 08:38:22 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 08:38:42 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 08:39:02 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 08:39:20 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 08:39:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:39:21 [DEBUG   ] Finished updating board flags in 2.065 seconds
2014-05-20 08:39:22 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 08:39:42 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 08:40:02 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 08:40:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 08:40:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:40:21 [DEBUG   ] Finished updating board flags in 1.911 seconds
2014-05-20 08:40:22 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 08:40:42 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 08:41:02 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 08:41:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 08:41:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2014-05-20 08:41:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:41:21 [DEBUG   ] Finished updating board flags in 1.815 seconds
2014-05-20 08:41:23 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 08:41:43 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 08:42:03 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 08:42:07 [DEBUG   ] Got command RPT from MCS: ref #197377
2014-05-20 08:42:07 [DEBUG   ] summary = ERROR
2014-05-20 08:42:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197377  13 56797 31327006 A  ERRORERROR'
2014-05-20 08:42:07 [DEBUG   ] Got command RPT from MCS: ref #197378
2014-05-20 08:42:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 08:42:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197378  84 56797 31327007 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 08:42:21 [DEBUG   ] Finished updating board flags in 1.778 seconds
2014-05-20 08:42:23 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 08:42:43 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 08:43:03 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 08:43:21 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 08:43:21 [CRITICAL] MGT underflow detected on beam(s) 2, 1
2014-05-20 08:43:21 [DEBUG   ] Finished updating board flags in 1.624 seconds
2014-05-20 08:43:23 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 08:43:43 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 08:44:03 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 08:44:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 08:44:21 [DEBUG   ] Finished updating board flags in 1.589 seconds
2014-05-20 08:44:23 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 08:44:43 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 08:45:03 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 08:45:21 [DEBUG   ] Finished updating board flags in 1.412 seconds
2014-05-20 08:45:23 [DEBUG   ] Finished updating ant stats in 0.606 seconds
2014-05-20 08:45:43 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 08:46:03 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 08:46:20 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 08:46:21 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:46:21 [DEBUG   ] Finished updating board flags in 1.385 seconds
2014-05-20 08:46:24 [DEBUG   ] Finished updating ant stats in 1.403 seconds
2014-05-20 08:46:44 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 08:47:04 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 08:47:07 [DEBUG   ] Got command RPT from MCS: ref #197394
2014-05-20 08:47:07 [DEBUG   ] summary = ERROR
2014-05-20 08:47:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197394  13 56797 31627002 A  ERRORERROR'
2014-05-20 08:47:07 [DEBUG   ] Got command RPT from MCS: ref #197395
2014-05-20 08:47:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 08:47:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197395  84 56797 31627003 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 08:47:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:47:21 [DEBUG   ] Finished updating board flags in 1.394 seconds
2014-05-20 08:47:24 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 08:47:44 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2014-05-20 08:48:04 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 08:48:22 [DEBUG   ] Finished updating board flags in 2.023 seconds
2014-05-20 08:48:24 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 08:48:44 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 08:49:04 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:49:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:49:22 [DEBUG   ] Finished updating board flags in 2.007 seconds
2014-05-20 08:49:24 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 08:49:44 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 08:50:04 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 08:50:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:50:22 [DEBUG   ] Finished updating board flags in 1.988 seconds
2014-05-20 08:50:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 08:50:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 08:51:04 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 08:51:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 08:51:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:51:22 [DEBUG   ] Finished updating board flags in 1.816 seconds
2014-05-20 08:51:24 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 08:51:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 08:52:04 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 08:52:07 [DEBUG   ] Got command RPT from MCS: ref #197411
2014-05-20 08:52:07 [DEBUG   ] summary = ERROR
2014-05-20 08:52:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197411  13 56797 31927010 A  ERRORERROR'
2014-05-20 08:52:07 [DEBUG   ] Got command RPT from MCS: ref #197412
2014-05-20 08:52:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 08:52:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197412  84 56797 31927011 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 08:52:22 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 08:52:22 [DEBUG   ] Finished updating board flags in 1.782 seconds
2014-05-20 08:52:24 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 08:52:45 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 08:53:05 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 08:53:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 08:53:22 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:53:22 [DEBUG   ] Finished updating board flags in 1.623 seconds
2014-05-20 08:53:25 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 08:53:45 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 08:54:05 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 08:54:22 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:54:22 [DEBUG   ] Finished updating board flags in 1.590 seconds
2014-05-20 08:54:25 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 08:54:45 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 08:55:05 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 08:55:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 08:55:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 08:55:22 [DEBUG   ] Finished updating board flags in 1.411 seconds
2014-05-20 08:55:25 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 08:55:45 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 08:56:05 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 08:56:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:56:22 [DEBUG   ] Finished updating board flags in 1.389 seconds
2014-05-20 08:56:25 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 08:56:45 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 08:57:05 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 08:57:07 [DEBUG   ] Got command RPT from MCS: ref #197428
2014-05-20 08:57:07 [DEBUG   ] summary = ERROR
2014-05-20 08:57:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197428  13 56797 32227006 A  ERRORERROR'
2014-05-20 08:57:07 [DEBUG   ] Got command RPT from MCS: ref #197429
2014-05-20 08:57:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 08:57:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197429  84 56797 32227007 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 08:57:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 08:57:21 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 08:57:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 08:57:22 [DEBUG   ] Finished updating board flags in 1.374 seconds
2014-05-20 08:57:26 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2014-05-20 08:57:46 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 08:58:06 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 08:58:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 08:58:23 [DEBUG   ] Finished updating board flags in 2.046 seconds
2014-05-20 08:58:26 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 08:58:46 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 08:59:06 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 08:59:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 08:59:23 [DEBUG   ] Finished updating board flags in 1.983 seconds
2014-05-20 08:59:26 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 08:59:46 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 09:00:06 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 09:00:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 09:00:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:00:23 [DEBUG   ] Finished updating board flags in 1.815 seconds
2014-05-20 09:00:26 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2014-05-20 09:00:46 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 09:01:07 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 09:01:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 09:01:23 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:01:23 [DEBUG   ] Finished updating board flags in 1.799 seconds
2014-05-20 09:01:27 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 09:01:47 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 09:02:07 [DEBUG   ] Got command RPT from MCS: ref #197445
2014-05-20 09:02:07 [DEBUG   ] summary = ERROR
2014-05-20 09:02:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197445  13 56797 32527014 A  ERRORERROR'
2014-05-20 09:02:07 [DEBUG   ] Got command RPT from MCS: ref #197446
2014-05-20 09:02:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 09:02:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197446  84 56797 32527015 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 09:02:07 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 09:02:23 [DEBUG   ] Finished updating board flags in 1.599 seconds
2014-05-20 09:02:27 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 09:02:47 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 09:03:07 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 09:03:23 [DEBUG   ] Finished updating board flags in 1.415 seconds
2014-05-20 09:03:27 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 09:03:47 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 09:04:07 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 09:04:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 09:04:23 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:04:23 [DEBUG   ] Finished updating board flags in 1.465 seconds
2014-05-20 09:04:27 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 09:04:47 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 09:05:07 [DEBUG   ] Finished updating ant stats in 0.805 seconds
2014-05-20 09:05:22 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 09:05:23 [DEBUG   ] Finished updating board flags in 1.418 seconds
2014-05-20 09:05:27 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 09:05:47 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 09:06:07 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 09:06:23 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:06:23 [DEBUG   ] Finished updating board flags in 1.557 seconds
2014-05-20 09:06:27 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 09:06:47 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 09:07:07 [DEBUG   ] Got command RPT from MCS: ref #197462
2014-05-20 09:07:07 [DEBUG   ] summary = ERROR
2014-05-20 09:07:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197462  13 56797 32827010 A  ERRORERROR'
2014-05-20 09:07:07 [DEBUG   ] Got command RPT from MCS: ref #197463
2014-05-20 09:07:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 09:07:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197463  84 56797 32827011 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 09:07:08 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 09:07:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-20 09:07:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:07:24 [DEBUG   ] Finished updating board flags in 2.206 seconds
2014-05-20 09:07:28 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 09:07:48 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 09:08:08 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 09:08:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:08:24 [DEBUG   ] Finished updating board flags in 2.145 seconds
2014-05-20 09:08:28 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 09:08:48 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 09:09:08 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 09:09:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 09:09:24 [DEBUG   ] Finished updating board flags in 2.021 seconds
2014-05-20 09:09:28 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 09:09:48 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 09:10:08 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2014-05-20 09:10:24 [DEBUG   ] Finished updating board flags in 1.986 seconds
2014-05-20 09:10:28 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 09:10:48 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 09:11:08 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 09:11:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 09:11:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:11:24 [DEBUG   ] Finished updating board flags in 1.818 seconds
2014-05-20 09:11:28 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 09:11:48 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 09:12:07 [DEBUG   ] Got command RPT from MCS: ref #197479
2014-05-20 09:12:07 [DEBUG   ] summary = ERROR
2014-05-20 09:12:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197479  13 56797 33127010 A  ERRORERROR'
2014-05-20 09:12:07 [DEBUG   ] Got command RPT from MCS: ref #197480
2014-05-20 09:12:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 09:12:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197480  84 56797 33127011 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 09:12:09 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 09:12:24 [DEBUG   ] Finished updating board flags in 1.786 seconds
2014-05-20 09:12:29 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 09:12:49 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 09:13:09 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 09:13:24 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:13:24 [DEBUG   ] Finished updating board flags in 1.617 seconds
2014-05-20 09:13:29 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 09:13:49 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 09:14:09 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 09:14:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:14:24 [DEBUG   ] Finished updating board flags in 1.662 seconds
2014-05-20 09:14:29 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 09:14:49 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 09:15:09 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 09:15:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:15:24 [DEBUG   ] Finished updating board flags in 1.523 seconds
2014-05-20 09:15:29 [DEBUG   ] Finished updating ant stats in 0.787 seconds
2014-05-20 09:15:49 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 09:16:09 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 09:16:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 09:16:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:16:24 [DEBUG   ] Finished updating board flags in 1.430 seconds
2014-05-20 09:16:29 [DEBUG   ] Finished updating ant stats in 0.588 seconds
2014-05-20 09:16:50 [DEBUG   ] Finished updating ant stats in 1.395 seconds
2014-05-20 09:17:07 [DEBUG   ] Got command RPT from MCS: ref #197496
2014-05-20 09:17:07 [DEBUG   ] summary = ERROR
2014-05-20 09:17:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197496  13 56797 33427018 A  ERRORERROR'
2014-05-20 09:17:07 [DEBUG   ] Got command RPT from MCS: ref #197497
2014-05-20 09:17:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 09:17:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197497  84 56797 33427019 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 09:17:10 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 09:17:24 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 09:17:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:17:24 [DEBUG   ] Finished updating board flags in 1.429 seconds
2014-05-20 09:17:30 [DEBUG   ] Finished updating ant stats in 1.196 seconds
2014-05-20 09:17:50 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 09:18:10 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 09:18:23 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 09:18:24 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 09:18:24 [DEBUG   ] Finished updating board flags in 1.432 seconds
2014-05-20 09:18:30 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 09:18:50 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 09:19:10 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 09:19:24 [DEBUG   ] Finished updating board flags in 1.432 seconds
2014-05-20 09:19:31 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 09:19:51 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 09:20:11 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 09:20:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:20:24 [DEBUG   ] Finished updating board flags in 1.432 seconds
2014-05-20 09:20:31 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 09:20:51 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 09:21:11 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 09:21:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 09:21:24 [DEBUG   ] Finished updating board flags in 1.432 seconds
2014-05-20 09:21:31 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 09:21:51 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 09:22:07 [DEBUG   ] Got command RPT from MCS: ref #197513
2014-05-20 09:22:07 [DEBUG   ] summary = ERROR
2014-05-20 09:22:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197513  13 56797 33727010 A  ERRORERROR'
2014-05-20 09:22:07 [DEBUG   ] Got command RPT from MCS: ref #197514
2014-05-20 09:22:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 09:22:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197514  84 56797 33727011 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 09:22:11 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 09:22:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:22:24 [DEBUG   ] Finished updating board flags in 1.432 seconds
2014-05-20 09:22:31 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 09:22:51 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 09:23:11 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 09:23:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:23:25 [DEBUG   ] Finished updating board flags in 2.196 seconds
2014-05-20 09:23:31 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 09:23:51 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 09:24:11 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 09:24:25 [DEBUG   ] Finished updating board flags in 1.990 seconds
2014-05-20 09:24:31 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 09:24:51 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 09:25:11 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 09:25:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 09:25:25 [DEBUG   ] Finished updating board flags in 1.868 seconds
2014-05-20 09:25:31 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 09:25:52 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 09:26:12 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 09:26:25 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 09:26:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 5
2014-05-20 09:26:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:26:25 [DEBUG   ] Finished updating board flags in 1.740 seconds
2014-05-20 09:26:32 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 09:26:52 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 09:27:07 [DEBUG   ] Got command RPT from MCS: ref #197530
2014-05-20 09:27:07 [DEBUG   ] summary = ERROR
2014-05-20 09:27:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197530  13 56797 34027018 A  ERRORERROR'
2014-05-20 09:27:07 [DEBUG   ] Got command RPT from MCS: ref #197531
2014-05-20 09:27:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 09:27:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197531  84 56797 34027019 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 09:27:12 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 09:27:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 09:27:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:27:25 [DEBUG   ] Finished updating board flags in 1.603 seconds
2014-05-20 09:27:32 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 09:27:52 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 09:28:12 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 09:28:24 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 09:28:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-20 09:28:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 09:28:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:28:25 [DEBUG   ] Finished updating board flags in 1.593 seconds
2014-05-20 09:28:32 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 09:28:52 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 09:29:12 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 09:29:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 09:29:25 [DEBUG   ] Finished updating board flags in 1.426 seconds
2014-05-20 09:29:32 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 09:29:52 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 09:30:12 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 09:30:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 09:30:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 09:30:25 [DEBUG   ] Finished updating board flags in 1.470 seconds
2014-05-20 09:30:32 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2014-05-20 09:30:52 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 09:31:12 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 09:31:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 4
2014-05-20 09:31:25 [DEBUG   ] Finished updating board flags in 1.471 seconds
2014-05-20 09:31:32 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 09:31:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 09:32:07 [DEBUG   ] Got command RPT from MCS: ref #197547
2014-05-20 09:32:07 [DEBUG   ] summary = ERROR
2014-05-20 09:32:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197547  13 56797 34327010 A  ERRORERROR'
2014-05-20 09:32:07 [DEBUG   ] Got command RPT from MCS: ref #197548
2014-05-20 09:32:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 09:32:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197548  84 56797 34327011 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 09:32:12 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 09:32:24 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 09:32:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 09:32:25 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:32:25 [DEBUG   ] Finished updating board flags in 1.471 seconds
2014-05-20 09:32:32 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 09:32:52 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 09:33:12 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 09:33:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 09:33:26 [DEBUG   ] Finished updating board flags in 2.244 seconds
2014-05-20 09:33:32 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 09:33:52 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 09:34:12 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 09:34:26 [DEBUG   ] Finished updating board flags in 2.251 seconds
2014-05-20 09:34:33 [DEBUG   ] Finished updating ant stats in 1.398 seconds
2014-05-20 09:34:53 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 09:35:13 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 09:35:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 09:35:26 [DEBUG   ] Finished updating board flags in 2.087 seconds
2014-05-20 09:35:33 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 09:35:53 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 09:36:13 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 09:36:25 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 09:36:26 [DEBUG   ] Finished updating board flags in 2.033 seconds
2014-05-20 09:36:33 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 09:36:53 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 09:37:07 [DEBUG   ] Got command RPT from MCS: ref #197564
2014-05-20 09:37:07 [DEBUG   ] summary = ERROR
2014-05-20 09:37:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197564  13 56797 34627022 A  ERRORERROR'
2014-05-20 09:37:07 [DEBUG   ] Got command RPT from MCS: ref #197565
2014-05-20 09:37:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 09:37:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197565  84 56797 34627023 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 09:37:13 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 09:37:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 09:37:26 [DEBUG   ] Finished updating board flags in 1.981 seconds
2014-05-20 09:37:33 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 09:37:53 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 09:38:13 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 09:38:26 [DEBUG   ] Finished updating board flags in 1.814 seconds
2014-05-20 09:38:33 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 09:38:53 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 09:39:13 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 09:39:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2014-05-20 09:39:26 [DEBUG   ] Finished updating board flags in 1.820 seconds
2014-05-20 09:39:33 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 09:39:53 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 09:40:13 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 09:40:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 09:40:26 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:40:26 [DEBUG   ] Finished updating board flags in 1.765 seconds
2014-05-20 09:40:33 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 09:40:53 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 09:41:13 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 09:41:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:41:26 [DEBUG   ] Finished updating board flags in 1.609 seconds
2014-05-20 09:41:33 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 09:41:53 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 09:42:07 [DEBUG   ] Got command RPT from MCS: ref #197581
2014-05-20 09:42:07 [DEBUG   ] summary = ERROR
2014-05-20 09:42:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197581  13 56797 34927022 A  ERRORERROR'
2014-05-20 09:42:07 [DEBUG   ] Got command RPT from MCS: ref #197582
2014-05-20 09:42:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 09:42:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197582  84 56797 34927023 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 09:42:13 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 09:42:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 09:42:26 [DEBUG   ] Finished updating board flags in 1.698 seconds
2014-05-20 09:42:33 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 09:42:53 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 09:43:13 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 09:43:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 09:43:26 [DEBUG   ] Finished updating board flags in 1.496 seconds
2014-05-20 09:43:33 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 09:43:53 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 09:44:13 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 09:44:26 [DEBUG   ] Finished updating board flags in 1.488 seconds
2014-05-20 09:44:33 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 09:44:53 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 09:45:13 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 09:45:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 09:45:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 09:45:26 [DEBUG   ] Finished updating board flags in 1.492 seconds
2014-05-20 09:45:33 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 09:45:53 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 09:46:14 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 09:46:25 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 09:46:26 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:46:26 [DEBUG   ] Finished updating board flags in 1.493 seconds
2014-05-20 09:46:34 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 09:46:54 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 09:47:07 [DEBUG   ] Got command RPT from MCS: ref #197598
2014-05-20 09:47:07 [DEBUG   ] summary = ERROR
2014-05-20 09:47:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197598  13 56797 35227022 A  ERRORERROR'
2014-05-20 09:47:07 [DEBUG   ] Got command RPT from MCS: ref #197599
2014-05-20 09:47:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 09:47:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197599  84 56797 35227023 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 09:47:14 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 09:47:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 09:47:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 09:47:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 09:47:27 [DEBUG   ] Finished updating board flags in 2.175 seconds
2014-05-20 09:47:34 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 09:47:54 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 09:48:14 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 09:48:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:48:27 [DEBUG   ] Finished updating board flags in 1.962 seconds
2014-05-20 09:48:34 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 09:48:54 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 09:49:14 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 09:49:27 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:49:27 [DEBUG   ] Finished updating board flags in 1.907 seconds
2014-05-20 09:49:34 [DEBUG   ] Finished updating ant stats in 0.798 seconds
2014-05-20 09:49:54 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 09:50:14 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 09:50:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 09:50:27 [DEBUG   ] Finished updating board flags in 1.811 seconds
2014-05-20 09:50:34 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 09:50:54 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 09:51:15 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 09:51:27 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:51:27 [DEBUG   ] Finished updating board flags in 1.695 seconds
2014-05-20 09:51:35 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 09:51:55 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 09:52:07 [DEBUG   ] Got command RPT from MCS: ref #197615
2014-05-20 09:52:07 [DEBUG   ] summary = ERROR
2014-05-20 09:52:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197615  13 56797 35527018 A  ERRORERROR'
2014-05-20 09:52:07 [DEBUG   ] Got command RPT from MCS: ref #197616
2014-05-20 09:52:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 09:52:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197616  84 56797 35527019 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 09:52:15 [DEBUG   ] Finished updating ant stats in 1.207 seconds
2014-05-20 09:52:27 [DEBUG   ] Finished updating board flags in 1.611 seconds
2014-05-20 09:52:35 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 09:52:55 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 09:53:15 [DEBUG   ] Finished updating ant stats in 1.003 seconds
2014-05-20 09:53:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 09:53:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 09:53:27 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:53:27 [DEBUG   ] Finished updating board flags in 1.614 seconds
2014-05-20 09:53:35 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 09:53:55 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 09:54:15 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 09:54:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 09:54:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 3
2014-05-20 09:54:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:54:27 [DEBUG   ] Finished updating board flags in 1.587 seconds
2014-05-20 09:54:35 [DEBUG   ] Finished updating ant stats in 0.795 seconds
2014-05-20 09:54:55 [DEBUG   ] Finished updating ant stats in 0.605 seconds
2014-05-20 09:55:15 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 09:55:26 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 09:55:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:55:27 [DEBUG   ] Finished updating board flags in 1.411 seconds
2014-05-20 09:55:35 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 09:55:55 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 09:56:16 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 09:56:27 [DEBUG   ] Finished updating board flags in 1.387 seconds
2014-05-20 09:56:36 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 09:56:56 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 09:57:07 [DEBUG   ] Got command RPT from MCS: ref #197632
2014-05-20 09:57:07 [DEBUG   ] summary = ERROR
2014-05-20 09:57:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197632  13 56797 35827018 A  ERRORERROR'
2014-05-20 09:57:07 [DEBUG   ] Got command RPT from MCS: ref #197633
2014-05-20 09:57:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 09:57:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197633  84 56797 35827019 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 09:57:16 [DEBUG   ] Finished updating ant stats in 1.209 seconds
2014-05-20 09:57:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa26, chip 3
2014-05-20 09:57:27 [DEBUG   ] Finished updating board flags in 1.387 seconds
2014-05-20 09:57:36 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 09:57:56 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 09:58:16 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 09:58:28 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 09:58:28 [DEBUG   ] Finished updating board flags in 2.037 seconds
2014-05-20 09:58:36 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2014-05-20 09:58:56 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 09:59:16 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 09:59:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 09:59:28 [DEBUG   ] Finished updating board flags in 1.996 seconds
2014-05-20 09:59:36 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 09:59:56 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2014-05-20 10:00:16 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 10:00:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 10:00:28 [DEBUG   ] Finished updating board flags in 1.799 seconds
2014-05-20 10:00:36 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 10:00:56 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:01:16 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 10:01:28 [DEBUG   ] Finished updating board flags in 1.601 seconds
2014-05-20 10:01:36 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 10:01:56 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 10:02:07 [DEBUG   ] Got command RPT from MCS: ref #197649
2014-05-20 10:02:07 [DEBUG   ] summary = ERROR
2014-05-20 10:02:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197649  13 56797 36127030 A  ERRORERROR'
2014-05-20 10:02:07 [DEBUG   ] Got command RPT from MCS: ref #197650
2014-05-20 10:02:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 10:02:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197650  84 56797 36127031 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 10:02:16 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 10:02:28 [DEBUG   ] Finished updating board flags in 1.662 seconds
2014-05-20 10:02:37 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 10:02:57 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 10:03:17 [DEBUG   ] Finished updating ant stats in 1.396 seconds
2014-05-20 10:03:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 10:03:28 [DEBUG   ] Finished updating board flags in 1.541 seconds
2014-05-20 10:03:37 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 10:03:57 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:04:17 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:04:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 10:04:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 10:04:28 [DEBUG   ] Finished updating board flags in 1.403 seconds
2014-05-20 10:04:37 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 10:04:57 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 10:05:17 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 10:05:27 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 10:05:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:05:28 [DEBUG   ] Finished updating board flags in 1.401 seconds
2014-05-20 10:05:37 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 10:05:57 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 10:06:17 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 10:06:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 10:06:28 [DEBUG   ] Finished updating board flags in 1.406 seconds
2014-05-20 10:06:37 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2014-05-20 10:06:57 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:07:07 [DEBUG   ] Got command RPT from MCS: ref #197666
2014-05-20 10:07:07 [DEBUG   ] summary = ERROR
2014-05-20 10:07:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197666  13 56797 36427022 A  ERRORERROR'
2014-05-20 10:07:07 [DEBUG   ] Got command RPT from MCS: ref #197667
2014-05-20 10:07:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 10:07:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197667  84 56797 36427023 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 10:07:17 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 10:07:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa24, chip 4
2014-05-20 10:07:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 10:07:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 10:07:28 [DEBUG   ] Finished updating board flags in 1.392 seconds
2014-05-20 10:07:37 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 10:07:58 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-20 10:08:18 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 10:08:27 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 10:08:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:08:28 [DEBUG   ] Finished updating board flags in 1.374 seconds
2014-05-20 10:08:38 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:08:58 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:09:18 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 10:09:29 [DEBUG   ] Finished updating board flags in 2.194 seconds
2014-05-20 10:09:38 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 10:09:58 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 10:10:18 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 10:10:28 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 10:10:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 10:10:29 [DEBUG   ] Finished updating board flags in 1.833 seconds
2014-05-20 10:10:38 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2014-05-20 10:10:58 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:11:18 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 10:11:28 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 10:11:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 10:11:29 [DEBUG   ] Finished updating board flags in 1.819 seconds
2014-05-20 10:11:38 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 10:11:58 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 10:12:07 [DEBUG   ] Got command RPT from MCS: ref #197683
2014-05-20 10:12:07 [DEBUG   ] summary = ERROR
2014-05-20 10:12:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197683  13 56797 36727030 A  ERRORERROR'
2014-05-20 10:12:07 [DEBUG   ] Got command RPT from MCS: ref #197684
2014-05-20 10:12:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 10:12:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197684  84 56797 36727031 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 10:12:19 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 10:12:29 [DEBUG   ] Finished updating board flags in 1.787 seconds
2014-05-20 10:12:39 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 10:12:59 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 10:13:19 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 10:13:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 10:13:29 [DEBUG   ] Finished updating board flags in 1.616 seconds
2014-05-20 10:13:39 [DEBUG   ] Finished updating ant stats in 1.210 seconds
2014-05-20 10:13:59 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:14:19 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 10:14:28 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-20 10:14:29 [DEBUG   ] Finished updating board flags in 1.588 seconds
2014-05-20 10:14:39 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 10:14:59 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 10:15:19 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 10:15:30 [DEBUG   ] Finished updating board flags in 2.407 seconds
2014-05-20 10:15:39 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 10:15:59 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:16:19 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 10:16:29 [DEBUG   ] Finished updating board flags in 1.390 seconds
2014-05-20 10:16:39 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 10:16:59 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 10:17:07 [DEBUG   ] Got command RPT from MCS: ref #197700
2014-05-20 10:17:07 [DEBUG   ] summary = ERROR
2014-05-20 10:17:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197700  13 56797 37027026 A  ERRORERROR'
2014-05-20 10:17:07 [DEBUG   ] Got command RPT from MCS: ref #197701
2014-05-20 10:17:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 10:17:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197701  84 56797 37027027 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 10:17:20 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 10:17:30 [DEBUG   ] Finished updating board flags in 2.216 seconds
2014-05-20 10:17:40 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 10:18:00 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 10:18:20 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 10:18:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 10:18:29 [DEBUG   ] Finished updating board flags in 1.334 seconds
2014-05-20 10:18:40 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 10:19:00 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 10:19:20 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 10:19:30 [DEBUG   ] Finished updating board flags in 2.077 seconds
2014-05-20 10:19:40 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 10:20:00 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:20:20 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:20:30 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 10:20:30 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:20:30 [CRITICAL] MGT underflow detected on beam(s) 2, 1
2014-05-20 10:20:30 [DEBUG   ] Finished updating board flags in 1.990 seconds
2014-05-20 10:20:40 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 10:21:00 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2014-05-20 10:21:20 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 10:21:30 [DEBUG   ] Finished updating board flags in 1.816 seconds
2014-05-20 10:21:40 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 10:22:00 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 10:22:07 [DEBUG   ] Got command RPT from MCS: ref #197717
2014-05-20 10:22:07 [DEBUG   ] summary = ERROR
2014-05-20 10:22:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197717  13 56797 37327034 A  ERRORERROR'
2014-05-20 10:22:07 [DEBUG   ] Got command RPT from MCS: ref #197718
2014-05-20 10:22:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 10:22:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197718  84 56797 37327035 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 10:22:20 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 10:22:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 10:22:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 10:22:30 [DEBUG   ] Finished updating board flags in 1.787 seconds
2014-05-20 10:22:40 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 10:23:00 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 10:23:20 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:23:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 10:23:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:23:30 [DEBUG   ] Finished updating board flags in 1.694 seconds
2014-05-20 10:23:40 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 10:24:00 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 10:24:20 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 10:24:30 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:24:30 [DEBUG   ] Finished updating board flags in 1.608 seconds
2014-05-20 10:24:40 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 10:25:00 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 10:25:20 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 10:25:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 10:25:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:25:30 [DEBUG   ] Finished updating board flags in 1.505 seconds
2014-05-20 10:25:40 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 10:26:00 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 10:26:20 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 10:26:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 10:26:30 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:26:30 [DEBUG   ] Finished updating board flags in 1.498 seconds
2014-05-20 10:26:41 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-20 10:27:01 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 10:27:07 [DEBUG   ] Got command RPT from MCS: ref #197734
2014-05-20 10:27:07 [DEBUG   ] summary = ERROR
2014-05-20 10:27:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197734  13 56797 37627034 A  ERRORERROR'
2014-05-20 10:27:07 [DEBUG   ] Got command RPT from MCS: ref #197735
2014-05-20 10:27:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 10:27:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197735  84 56797 37627035 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 10:27:21 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 10:27:29 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 10:27:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:27:30 [DEBUG   ] Finished updating board flags in 1.495 seconds
2014-05-20 10:27:41 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:28:01 [DEBUG   ] Finished updating ant stats in 1.197 seconds
2014-05-20 10:28:21 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 10:28:30 [DEBUG   ] Finished updating board flags in 1.502 seconds
2014-05-20 10:28:41 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 10:29:01 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 10:29:21 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 10:29:31 [DEBUG   ] Finished updating board flags in 2.145 seconds
2014-05-20 10:29:41 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:30:01 [DEBUG   ] Finished updating ant stats in 0.608 seconds
2014-05-20 10:30:21 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 10:30:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 10:30:31 [DEBUG   ] Finished updating board flags in 1.961 seconds
2014-05-20 10:30:42 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-20 10:31:02 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 10:31:22 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:31:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 10:31:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 10:31:31 [DEBUG   ] Finished updating board flags in 1.809 seconds
2014-05-20 10:31:42 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 10:32:02 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 10:32:07 [DEBUG   ] Got command RPT from MCS: ref #197751
2014-05-20 10:32:07 [DEBUG   ] summary = ERROR
2014-05-20 10:32:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197751  13 56797 37927038 A  ERRORERROR'
2014-05-20 10:32:07 [DEBUG   ] Got command RPT from MCS: ref #197752
2014-05-20 10:32:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 10:32:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197752  84 56797 37927039 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 10:32:22 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:32:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-20 10:32:31 [DEBUG   ] Finished updating board flags in 1.804 seconds
2014-05-20 10:32:42 [DEBUG   ] Finished updating ant stats in 1.198 seconds
2014-05-20 10:33:02 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 10:33:22 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 10:33:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 10:33:31 [DEBUG   ] Finished updating board flags in 1.798 seconds
2014-05-20 10:33:42 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 10:34:02 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 10:34:22 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 10:34:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 10:34:30 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 10:34:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa15, chip 1
2014-05-20 10:34:31 [DEBUG   ] Finished updating board flags in 1.606 seconds
2014-05-20 10:34:42 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 10:35:02 [DEBUG   ] Finished updating ant stats in 0.998 seconds
2014-05-20 10:35:22 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 10:35:31 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:35:31 [DEBUG   ] Finished updating board flags in 1.596 seconds
2014-05-20 10:35:42 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:36:02 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:36:22 [DEBUG   ] Finished updating ant stats in 0.797 seconds
2014-05-20 10:36:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 10:36:31 [DEBUG   ] Finished updating board flags in 1.401 seconds
2014-05-20 10:36:42 [DEBUG   ] Finished updating ant stats in 0.598 seconds
2014-05-20 10:37:03 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-20 10:37:07 [DEBUG   ] Got command RPT from MCS: ref #197768
2014-05-20 10:37:07 [DEBUG   ] summary = ERROR
2014-05-20 10:37:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197768  13 56797 38227030 A  ERRORERROR'
2014-05-20 10:37:07 [DEBUG   ] Got command RPT from MCS: ref #197769
2014-05-20 10:37:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 10:37:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197769  84 56797 38227031 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 10:37:23 [DEBUG   ] Finished updating ant stats in 1.206 seconds
2014-05-20 10:37:30 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 10:37:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 10:37:31 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:37:31 [DEBUG   ] Finished updating board flags in 1.412 seconds
2014-05-20 10:37:43 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:38:03 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:38:23 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:38:31 [DEBUG   ] Finished updating board flags in 1.398 seconds
2014-05-20 10:38:43 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:39:03 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:39:23 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 10:39:32 [DEBUG   ] Finished updating board flags in 2.277 seconds
2014-05-20 10:39:43 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 10:40:03 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 10:40:23 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 10:40:32 [DEBUG   ] Finished updating board flags in 2.208 seconds
2014-05-20 10:40:43 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2014-05-20 10:41:03 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:41:23 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 10:41:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 10:41:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 10:41:32 [DEBUG   ] Finished updating board flags in 2.112 seconds
2014-05-20 10:41:43 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 10:42:03 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:42:07 [DEBUG   ] Got command RPT from MCS: ref #197785
2014-05-20 10:42:07 [DEBUG   ] summary = ERROR
2014-05-20 10:42:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197785  13 56797 38527034 A  ERRORERROR'
2014-05-20 10:42:07 [DEBUG   ] Got command RPT from MCS: ref #197786
2014-05-20 10:42:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 10:42:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197786  84 56797 38527035 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 10:42:23 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:42:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 10:42:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 10:42:32 [DEBUG   ] Finished updating board flags in 2.039 seconds
2014-05-20 10:42:43 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 10:43:03 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 10:43:23 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:43:31 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 10:43:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 10:43:32 [DEBUG   ] Finished updating board flags in 1.973 seconds
2014-05-20 10:43:43 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 10:44:03 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 10:44:23 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 10:44:32 [DEBUG   ] Finished updating board flags in 1.825 seconds
2014-05-20 10:44:43 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 10:45:04 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 10:45:24 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 10:45:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 10:45:32 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:45:32 [DEBUG   ] Finished updating board flags in 1.772 seconds
2014-05-20 10:45:44 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 10:46:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:46:24 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:46:32 [DEBUG   ] Finished updating board flags in 1.622 seconds
2014-05-20 10:46:44 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:47:04 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 10:47:07 [DEBUG   ] Got command RPT from MCS: ref #197802
2014-05-20 10:47:07 [DEBUG   ] summary = ERROR
2014-05-20 10:47:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197802  13 56797 38827034 A  ERRORERROR'
2014-05-20 10:47:07 [DEBUG   ] Got command RPT from MCS: ref #197803
2014-05-20 10:47:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 10:47:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197803  84 56797 38827035 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 10:47:24 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 10:47:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 10:47:32 [DEBUG   ] Finished updating board flags in 1.578 seconds
2014-05-20 10:47:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:48:04 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:48:24 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 10:48:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:48:32 [DEBUG   ] Finished updating board flags in 1.465 seconds
2014-05-20 10:48:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:49:04 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:49:24 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:49:32 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 10:49:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:49:32 [DEBUG   ] Finished updating board flags in 1.597 seconds
2014-05-20 10:49:44 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 10:50:04 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:50:24 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:50:33 [DEBUG   ] Finished updating board flags in 2.272 seconds
2014-05-20 10:50:44 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:51:04 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 10:51:24 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:51:32 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 10:51:33 [DEBUG   ] Finished updating board flags in 2.096 seconds
2014-05-20 10:51:44 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 10:52:04 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 10:52:07 [DEBUG   ] Got command RPT from MCS: ref #197819
2014-05-20 10:52:07 [DEBUG   ] summary = ERROR
2014-05-20 10:52:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197819  13 56797 39127042 A  ERRORERROR'
2014-05-20 10:52:07 [DEBUG   ] Got command RPT from MCS: ref #197820
2014-05-20 10:52:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 10:52:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197820  84 56797 39127043 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 10:52:24 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 10:52:33 [DEBUG   ] Finished updating board flags in 2.003 seconds
2014-05-20 10:52:44 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 10:53:04 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 10:53:24 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 10:53:33 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:53:33 [DEBUG   ] Finished updating board flags in 2.005 seconds
2014-05-20 10:53:44 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 10:54:04 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2014-05-20 10:54:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 10:54:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 10:54:33 [DEBUG   ] Finished updating board flags in 1.983 seconds
2014-05-20 10:54:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 10:55:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 10:55:24 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 10:55:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-20 10:55:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:55:33 [DEBUG   ] Finished updating board flags in 1.818 seconds
2014-05-20 10:55:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 10:56:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 10:56:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 10:56:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 10:56:33 [DEBUG   ] Finished updating board flags in 1.891 seconds
2014-05-20 10:56:44 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 10:57:04 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 10:57:07 [DEBUG   ] Got command RPT from MCS: ref #197836
2014-05-20 10:57:07 [DEBUG   ] summary = ERROR
2014-05-20 10:57:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197836  13 56797 39427042 A  ERRORERROR'
2014-05-20 10:57:07 [DEBUG   ] Got command RPT from MCS: ref #197837
2014-05-20 10:57:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 10:57:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197837  84 56797 39427043 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 10:57:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 10:57:32 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 10:57:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 10:57:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:57:33 [DEBUG   ] Finished updating board flags in 1.710 seconds
2014-05-20 10:57:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:58:04 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 10:58:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 10:58:32 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 10:58:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 10:58:33 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 10:58:33 [DEBUG   ] Finished updating board flags in 1.608 seconds
2014-05-20 10:58:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 10:59:04 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 10:59:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 10:59:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 10:59:33 [DEBUG   ] Finished updating board flags in 1.608 seconds
2014-05-20 10:59:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:00:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:00:24 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:00:33 [DEBUG   ] Finished updating board flags in 1.574 seconds
2014-05-20 11:00:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:01:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:01:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:01:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:01:33 [DEBUG   ] Finished updating board flags in 1.552 seconds
2014-05-20 11:01:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:02:04 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 11:02:07 [DEBUG   ] Got command RPT from MCS: ref #197853
2014-05-20 11:02:07 [DEBUG   ] summary = ERROR
2014-05-20 11:02:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197853  13 56797 39727042 A  ERRORERROR'
2014-05-20 11:02:07 [DEBUG   ] Got command RPT from MCS: ref #197854
2014-05-20 11:02:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 11:02:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197854  84 56797 39727043 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 11:02:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:02:33 [DEBUG   ] Finished updating board flags in 1.592 seconds
2014-05-20 11:02:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:03:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:03:24 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 11:03:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:03:34 [DEBUG   ] Finished updating board flags in 2.180 seconds
2014-05-20 11:03:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:04:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:04:24 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:04:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 11:04:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 11:04:34 [DEBUG   ] Finished updating board flags in 1.894 seconds
2014-05-20 11:04:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:05:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:05:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:05:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 11:05:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 2
2014-05-20 11:05:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:05:34 [DEBUG   ] Finished updating board flags in 1.804 seconds
2014-05-20 11:05:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:06:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:06:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:06:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:06:34 [DEBUG   ] Finished updating board flags in 1.803 seconds
2014-05-20 11:06:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:07:04 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:07:07 [DEBUG   ] Got command RPT from MCS: ref #197870
2014-05-20 11:07:07 [DEBUG   ] summary = ERROR
2014-05-20 11:07:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197870  13 56797 40027042 A  ERRORERROR'
2014-05-20 11:07:07 [DEBUG   ] Got command RPT from MCS: ref #197871
2014-05-20 11:07:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 11:07:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197871  84 56797 40027043 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 11:07:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:07:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:07:34 [DEBUG   ] Finished updating board flags in 1.790 seconds
2014-05-20 11:07:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:08:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:08:24 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:08:34 [DEBUG   ] Finished updating board flags in 1.659 seconds
2014-05-20 11:08:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:09:04 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 11:09:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:09:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 11:09:34 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:09:34 [DEBUG   ] Finished updating board flags in 1.671 seconds
2014-05-20 11:09:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:10:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:10:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:10:33 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 11:10:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:10:34 [DEBUG   ] Finished updating board flags in 1.467 seconds
2014-05-20 11:10:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:11:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:11:24 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 11:11:34 [DEBUG   ] Finished updating board flags in 1.436 seconds
2014-05-20 11:11:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:12:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:12:07 [DEBUG   ] Got command RPT from MCS: ref #197887
2014-05-20 11:12:07 [DEBUG   ] summary = ERROR
2014-05-20 11:12:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197887  13 56797 40327038 A  ERRORERROR'
2014-05-20 11:12:07 [DEBUG   ] Got command RPT from MCS: ref #197888
2014-05-20 11:12:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 11:12:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197888  84 56797 40327039 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 11:12:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:12:34 [WARNING ] MonitorFlags: VCCINT 0.979 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:12:34 [DEBUG   ] Finished updating board flags in 1.421 seconds
2014-05-20 11:12:44 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 11:13:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:13:24 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:13:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa14, chip 5
2014-05-20 11:13:34 [DEBUG   ] Finished updating board flags in 1.474 seconds
2014-05-20 11:13:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:14:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:14:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:14:34 [DEBUG   ] Finished updating board flags in 1.595 seconds
2014-05-20 11:14:44 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:15:04 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:15:24 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 11:15:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 11:15:35 [DEBUG   ] Finished updating board flags in 2.230 seconds
2014-05-20 11:15:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:16:04 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:16:24 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:16:35 [DEBUG   ] Finished updating board flags in 2.065 seconds
2014-05-20 11:16:44 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:17:04 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:17:07 [DEBUG   ] Got command RPT from MCS: ref #197904
2014-05-20 11:17:07 [DEBUG   ] summary = ERROR
2014-05-20 11:17:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197904  13 56797 40627046 A  ERRORERROR'
2014-05-20 11:17:07 [DEBUG   ] Got command RPT from MCS: ref #197905
2014-05-20 11:17:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 11:17:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197905  84 56797 40627047 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 11:17:24 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:17:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 11:17:35 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:17:35 [DEBUG   ] Finished updating board flags in 2.027 seconds
2014-05-20 11:17:44 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:18:04 [DEBUG   ] Finished updating ant stats in 0.794 seconds
2014-05-20 11:18:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:18:35 [DEBUG   ] Finished updating board flags in 1.975 seconds
2014-05-20 11:18:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:19:04 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:19:24 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 11:19:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 11:19:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 11:19:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 11:19:35 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:19:35 [DEBUG   ] Finished updating board flags in 1.823 seconds
2014-05-20 11:19:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:20:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:20:24 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:20:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 11:20:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:20:35 [DEBUG   ] Finished updating board flags in 1.783 seconds
2014-05-20 11:20:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:21:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:21:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:21:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 11:21:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-20 11:21:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:21:35 [DEBUG   ] Finished updating board flags in 1.698 seconds
2014-05-20 11:21:44 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:22:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:22:07 [DEBUG   ] Got command RPT from MCS: ref #197921
2014-05-20 11:22:07 [DEBUG   ] summary = ERROR
2014-05-20 11:22:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197921  13 56797 40927050 A  ERRORERROR'
2014-05-20 11:22:07 [DEBUG   ] Got command RPT from MCS: ref #197922
2014-05-20 11:22:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 11:22:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197922  84 56797 40927051 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 11:22:24 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 11:22:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 11:22:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 11:22:35 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 11:22:35 [CRITICAL] MGT underflow detected on beam(s) 2, 1
2014-05-20 11:22:35 [DEBUG   ] Finished updating board flags in 1.630 seconds
2014-05-20 11:22:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:23:04 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:23:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:23:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 11:23:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 11:23:35 [DEBUG   ] Finished updating board flags in 1.480 seconds
2014-05-20 11:23:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:24:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:24:24 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:24:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 11:24:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa19, chip 5
2014-05-20 11:24:35 [DEBUG   ] Finished updating board flags in 1.477 seconds
2014-05-20 11:24:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:25:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:25:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:25:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 11:25:35 [DEBUG   ] Finished updating board flags in 1.486 seconds
2014-05-20 11:25:44 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:26:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:26:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:26:34 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 11:26:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 11:26:35 [DEBUG   ] Finished updating board flags in 1.501 seconds
2014-05-20 11:26:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:27:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:27:07 [DEBUG   ] Got command RPT from MCS: ref #197938
2014-05-20 11:27:07 [DEBUG   ] summary = ERROR
2014-05-20 11:27:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197938  13 56797 41227050 A  ERRORERROR'
2014-05-20 11:27:07 [DEBUG   ] Got command RPT from MCS: ref #197939
2014-05-20 11:27:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 11:27:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197939  84 56797 41227051 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 11:27:24 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 11:27:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-20 11:27:36 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:27:36 [DEBUG   ] Finished updating board flags in 2.193 seconds
2014-05-20 11:27:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:28:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:28:24 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:28:36 [DEBUG   ] Finished updating board flags in 2.017 seconds
2014-05-20 11:28:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:29:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:29:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:29:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 11:29:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 1
2014-05-20 11:29:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 11:29:36 [DEBUG   ] Finished updating board flags in 2.040 seconds
2014-05-20 11:29:44 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:30:04 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:30:24 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 11:30:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 11:30:36 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:30:36 [DEBUG   ] Finished updating board flags in 1.894 seconds
2014-05-20 11:30:44 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:31:04 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:31:24 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 11:31:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 11:31:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:31:36 [DEBUG   ] Finished updating board flags in 1.810 seconds
2014-05-20 11:31:44 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 11:32:04 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:32:07 [DEBUG   ] Got command RPT from MCS: ref #197955
2014-05-20 11:32:07 [DEBUG   ] summary = ERROR
2014-05-20 11:32:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197955  13 56797 41527042 A  ERRORERROR'
2014-05-20 11:32:07 [DEBUG   ] Got command RPT from MCS: ref #197956
2014-05-20 11:32:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 11:32:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197956  84 56797 41527043 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 11:32:24 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:32:36 [DEBUG   ] Finished updating board flags in 1.841 seconds
2014-05-20 11:32:44 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:33:04 [DEBUG   ] Finished updating ant stats in 0.603 seconds
2014-05-20 11:33:24 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:33:36 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:33:36 [DEBUG   ] Finished updating board flags in 1.758 seconds
2014-05-20 11:33:44 [DEBUG   ] Finished updating ant stats in 0.592 seconds
2014-05-20 11:34:05 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 11:34:25 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 11:34:36 [DEBUG   ] Finished updating board flags in 1.609 seconds
2014-05-20 11:34:45 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 11:35:05 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 11:35:25 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 11:35:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 11:35:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 11:35:36 [DEBUG   ] Finished updating board flags in 1.581 seconds
2014-05-20 11:35:45 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 11:36:05 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 11:36:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:36:35 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 11:36:36 [DEBUG   ] Finished updating board flags in 1.479 seconds
2014-05-20 11:36:45 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:37:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:37:07 [DEBUG   ] Got command RPT from MCS: ref #197972
2014-05-20 11:37:07 [DEBUG   ] summary = ERROR
2014-05-20 11:37:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197972  13 56797 41827054 A  ERRORERROR'
2014-05-20 11:37:07 [DEBUG   ] Got command RPT from MCS: ref #197973
2014-05-20 11:37:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 11:37:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197973  84 56797 41827055 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 11:37:25 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:37:35 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 11:37:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 11:37:36 [DEBUG   ] Finished updating board flags in 1.557 seconds
2014-05-20 11:37:45 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:38:05 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 11:38:25 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:38:37 [DEBUG   ] Finished updating board flags in 2.267 seconds
2014-05-20 11:38:45 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:39:05 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:39:25 [DEBUG   ] Finished updating ant stats in 0.790 seconds
2014-05-20 11:39:37 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 11:39:37 [DEBUG   ] Finished updating board flags in 2.117 seconds
2014-05-20 11:39:45 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:40:05 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:40:26 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 11:40:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 11:40:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 11:40:37 [DEBUG   ] Finished updating board flags in 2.017 seconds
2014-05-20 11:40:46 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 11:41:06 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 11:41:26 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 11:41:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa11, chip 5
2014-05-20 11:41:37 [DEBUG   ] Finished updating board flags in 1.998 seconds
2014-05-20 11:41:46 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 11:42:06 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 11:42:07 [DEBUG   ] Got command RPT from MCS: ref #197989
2014-05-20 11:42:07 [DEBUG   ] summary = ERROR
2014-05-20 11:42:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197989  13 56797 42127050 A  ERRORERROR'
2014-05-20 11:42:07 [DEBUG   ] Got command RPT from MCS: ref #197990
2014-05-20 11:42:07 [DEBUG   ] info = T_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 11:42:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   197990  84 56797 42127051 A  ERRORT_NOM2 T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 11:42:26 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 11:42:37 [DEBUG   ] Finished updating board flags in 1.835 seconds
2014-05-20 11:42:46 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 11:43:06 [DEBUG   ] Finished updating ant stats in 1.403 seconds
2014-05-20 11:43:26 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 11:43:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa32, chip 5
2014-05-20 11:43:37 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 11:43:37 [DEBUG   ] Finished updating board flags in 1.754 seconds
2014-05-20 11:43:46 [DEBUG   ] Finished updating ant stats in 1.204 seconds
2014-05-20 11:44:06 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 11:44:26 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 11:44:37 [DEBUG   ] Finished updating board flags in 1.624 seconds
2014-05-20 11:44:46 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 11:45:06 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 11:45:26 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 11:45:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 11:45:36 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 11:45:37 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 11:45:37 [DEBUG   ] Finished updating board flags in 1.605 seconds
2014-05-20 11:45:46 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 11:46:06 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 11:46:26 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 11:46:37 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 5
2014-05-20 11:46:37 [DEBUG   ] Finished updating board flags in 1.567 seconds
2014-05-20 11:46:46 [DEBUG   ] Finished updating ant stats in 0.997 seconds
2014-05-20 11:47:00 [DEBUG   ] Got command INI from MCS: ref #198006
2014-05-20 11:47:00 [INFO    ] Parsing config file '/lwa/software/defaults.cfg'
2014-05-20 11:47:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_INI   198006   8 56797 42420755 ABOOTING'
2014-05-20 11:47:07 [DEBUG   ] Got command RPT from MCS: ref #198007
2014-05-20 11:47:07 [DEBUG   ] summary = BOOTING
2014-05-20 11:47:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198007  15 56797 42427050 ABOOTINGBOOTING'
2014-05-20 11:47:07 [DEBUG   ] Got command RPT from MCS: ref #198008
2014-05-20 11:47:07 [DEBUG   ] info = Running INI sequence
2014-05-20 11:47:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198008  28 56797 42427051 ABOOTINGRunning INI sequence'
2014-05-20 11:47:24 [INFO    ] -------------------
2014-05-20 11:47:24 [INFO    ]  LWA board status
2014-05-20 11:47:24 [INFO    ] -------------------
2014-05-20 11:47:24 [INFO    ] DP 1: 2 lwa27
2014-05-20 11:47:24 [INFO    ] DP 1: 3 lwa24
2014-05-20 11:47:24 [INFO    ] DP 1: 4 lwa23
2014-05-20 11:47:24 [INFO    ] DP 1: 5 lwa18
2014-05-20 11:47:24 [INFO    ] DP 1: 6 lwa10
2014-05-20 11:47:24 [INFO    ] DP 1: 7 lwa32
2014-05-20 11:47:24 [INFO    ] DP 1: 8 lwa11
2014-05-20 11:47:24 [INFO    ] DP 1: 9 lwa30
2014-05-20 11:47:24 [INFO    ] DP 1: 10 lwa12
2014-05-20 11:47:24 [INFO    ] DP 1: 11 lwa8
2014-05-20 11:47:24 [INFO    ] DP 1: 12 lwa13
2014-05-20 11:47:24 [INFO    ] DP 1: 13 lwa26
2014-05-20 11:47:24 [INFO    ] DP 1: 14 lwa14
2014-05-20 11:47:24 [INFO    ] DP 1: 16 lwa22
2014-05-20 11:47:24 [INFO    ] DP 1: 17 lwa20
2014-05-20 11:47:24 [INFO    ] DP 1: 18 lwa1
2014-05-20 11:47:24 [INFO    ] DP 1: 19 lwa5
2014-05-20 11:47:24 [INFO    ] DP 1: 20 lwa25
2014-05-20 11:47:24 [INFO    ] DP 1: 21 lwa35
2014-05-20 11:47:24 [INFO    ] DP 1: 22 lwa19
2014-05-20 11:47:24 [INFO    ] DP 1: 23 lwa21
2014-05-20 11:47:24 [INFO    ] DP 1: 24 lwa6
2014-05-20 11:47:24 [INFO    ] DP 1: 25 lwa7
2014-05-20 11:47:24 [INFO    ] DP 1: 26 lwa33
2014-05-20 11:47:24 [INFO    ] DP 1: 27 lwa15
2014-05-20 11:47:24 [INFO    ] DP 1: 28 lwa28
2014-05-20 11:47:24 [INFO    ] DP 2: 1 lwa16
2014-05-20 11:47:24 [INFO    ] DP 2: 15 lwa34
2014-05-20 11:47:24 [INFO    ] Total of 28 boards
2014-05-20 11:47:24 [INFO    ] Total of 260 stands
2014-05-20 11:47:24 [INFO    ] -------------------
2014-05-20 11:47:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa27 command /usr/local/bin/lwa27 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:24 [DEBUG   ] sendHostnameCommand - Sent hostname lwa24 command /usr/local/bin/lwa24 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa23 command /usr/local/bin/lwa23 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa18 command /usr/local/bin/lwa18 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa10 command /usr/local/bin/lwa10 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:25 [DEBUG   ] sendHostnameCommand - Sent hostname lwa32 command /usr/local/bin/lwa32 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa11 command /usr/local/bin/lwa11 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa30 command /usr/local/bin/lwa30 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa12 command /usr/local/bin/lwa12 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:26 [DEBUG   ] sendHostnameCommand - Sent hostname lwa8 command /usr/local/bin/lwa8 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa13 command /usr/local/bin/lwa13 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa26 command /usr/local/bin/lwa26 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:27 [DEBUG   ] sendHostnameCommand - Sent hostname lwa14 command /usr/local/bin/lwa14 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa22 command /usr/local/bin/lwa22 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa20 command /usr/local/bin/lwa20 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:28 [DEBUG   ] sendHostnameCommand - Sent hostname lwa1 command /usr/local/bin/lwa1 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa5 command /usr/local/bin/lwa5 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa25 command /usr/local/bin/lwa25 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa35 command /usr/local/bin/lwa35 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:29 [DEBUG   ] sendHostnameCommand - Sent hostname lwa19 command /usr/local/bin/lwa19 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa21 command /usr/local/bin/lwa21 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa6 command /usr/local/bin/lwa6 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa7 command /usr/local/bin/lwa7 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:30 [DEBUG   ] sendHostnameCommand - Sent hostname lwa33 command /usr/local/bin/lwa33 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa15 command /usr/local/bin/lwa15 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:31 [DEBUG   ] sendHostnameCommand - Sent hostname lwa28 command /usr/local/bin/lwa28 /lwa/software/dp_shutdown.sh
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2744
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2745
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2746
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2747
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2748
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2749
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2750
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2751
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2752
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2753
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2754
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2755
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2756
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2757
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2758
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2759
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2760
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2761
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2762
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2763
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2764
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2765
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2766
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2767
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2768
2014-05-20 11:47:31 [DEBUG   ] 2014-05-20 11:47:31 Starting ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] pid 2769
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa27', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2744 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa24', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2745 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa23', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2746 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa18', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2747 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa10', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2748 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa32', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2749 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa11', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2750 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa30', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2751 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa12', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2752 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa8', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2753 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa13', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2754 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa26', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2755 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa14', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2756 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa22', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2757 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa20', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2758 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa1', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2759 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa5', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2760 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa25', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2761 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa35', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2762 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa19', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2763 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa21', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2764 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa6', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2765 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa7', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2766 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa33', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2767 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa15', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2768 exited with status 0
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 ['/usr/local/bin/lwa28', '/lwa/software/Start.sh', '/lwa/hardware/lwa_dp_top_2_1_1_49.bin', '/lwa/software'] Process 2769 exited with status 0
2014-05-20 11:47:44 [INFO    ] Finish loading DP1 firmware'/lwa/hardware/lwa_dp_top_2_1_1_49.bin'
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa27', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa24', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa23', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa18', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa10', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa32', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa11', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa30', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa12', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa8', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa13', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa26', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa14', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa22', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa20', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa1', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa5', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa25', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa35', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa19', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa21', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa6', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa7', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa33', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa15', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] spawn_process: at 2014-05-20 11:47:44 running ['/usr/local/bin/lwa28', '/lwa/software/dp1_init.sh']
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 Starting ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 2796
2014-05-20 11:47:44 [DEBUG   ] 2014-05-20 11:47:44 Starting ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] pid 2797
2014-05-20 11:47:49 [DEBUG   ] 2014-05-20 11:47:49 ['/usr/local/bin/lwa16', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 2796 exited with status 0
2014-05-20 11:47:49 [DEBUG   ] 2014-05-20 11:47:49 ['/usr/local/bin/lwa34', '/lwa/software/Start2.sh', '/lwa/hardware/lwa_drx_top_3_2_0_6.bin', '/lwa/software'] Process 2797 exited with status 0
2014-05-20 11:47:49 [INFO    ] Finish loading DP2 firmware '/lwa/hardware/lwa_drx_top_3_2_0_6.bin'
2014-05-20 11:47:49 [DEBUG   ] spawn_process: at 2014-05-20 11:47:49 running ['/usr/local/bin/lwa16', '/lwa/software/dp2_init.sh']
2014-05-20 11:47:49 [DEBUG   ] spawn_process: at 2014-05-20 11:47:49 running ['/usr/local/bin/lwa34', '/lwa/software/dp2_init.sh']
2014-05-20 11:47:49 [INFO    ] Setting Beam 1 xbars
2014-05-20 11:47:49 [DEBUG   ] board 11 xbars (bd11 to bd5 through bot5) - beam 1
2014-05-20 11:47:49 [DEBUG   ] board 5 xbars (bd5 from bd11 through bot1) - beam 1
2014-05-20 11:47:49 [DEBUG   ] board 5 xbars (bd5 to bd13 through bot5) - beam 1
2014-05-20 11:47:50 [DEBUG   ] board 13 xbars (bd13 from bd5 through bot1) - beam 1
2014-05-20 11:47:50 [DEBUG   ] board 13 xbars (bd13 to bd7 through bot5) - beam 1
2014-05-20 11:47:50 [DEBUG   ] board 7 xbars (bd7 from bd13 through bot1) - beam 1
2014-05-20 11:47:50 [DEBUG   ] board 7 xbars (bd7 to bd9 through bot5) - beam 1
2014-05-20 11:47:50 [DEBUG   ] board 9 xbars (bd9 from bd7 through bot1) - beam 1
2014-05-20 11:47:50 [DEBUG   ] board 9 xbars (bd9 to bd3 through bot5) - beam 1
2014-05-20 11:47:50 [DEBUG   ] board 3 xbars (bd3 from bd9 through bot1) - beam 1
2014-05-20 11:47:50 [DEBUG   ] board 3 xbars (bd3 to bd2 through bot5) - beam 1
2014-05-20 11:47:50 [DEBUG   ] board 2 xbars (bd2 from bd3 through bot1) - beam 1
2014-05-20 11:47:51 [DEBUG   ] board 2 xbars (bd2 to bd6 through bot5) - beam 1
2014-05-20 11:47:51 [DEBUG   ] board 6 xbars (bd6 from bd2 through bot1) - beam 1
2014-05-20 11:47:51 [DEBUG   ] board 6 xbars (bd6 to bd12 through bot5) - beam 1
2014-05-20 11:47:51 [DEBUG   ] board 12 xbars (bd12 from bd6 through bot1) - beam 1
2014-05-20 11:47:51 [DEBUG   ] board 12 xbars (bd12 to bd8 through bot5) - beam 1
2014-05-20 11:47:51 [DEBUG   ] board 8 xbars (bd8 from bd12 through bot1) - beam 1
2014-05-20 11:47:51 [DEBUG   ] board 8 xbars (bd8 to bd4 through bot5) - beam 1
2014-05-20 11:47:51 [DEBUG   ] board 4 xbars (bd4 from bd8 through bot1) - beam 1
2014-05-20 11:47:51 [DEBUG   ] board 4 xbars (bd4 to bd10 through bot5) - beam 1
2014-05-20 11:47:52 [DEBUG   ] board 10 xbars (bd10 from bd4 through bot1) - beam 1
2014-05-20 11:47:52 [DEBUG   ] board 10 xbars (bd10 to bd14 through bot5) - beam 1
2014-05-20 11:47:52 [DEBUG   ] board 14 xbars (bd14 from bd10 through top1) - beam 1
2014-05-20 11:47:52 [DEBUG   ] board 28 xbars  (bd 28 to bd24 through top1)- beam 1
2014-05-20 11:47:52 [DEBUG   ] board 24 xbars (bd24 from bd28 through bot5) - beam 1
2014-05-20 11:47:52 [DEBUG   ] board 24 xbars (bd24 to bd18 through bot1) - beam 1
2014-05-20 11:47:52 [DEBUG   ] board 18 xbars (bd18 from bd24 through bot5) - beam 1
2014-05-20 11:47:52 [DEBUG   ] board 18 xbars (bd18 to bd22 through bot1) - beam 1
2014-05-20 11:47:52 [DEBUG   ] board 22 xbars (bd22 from bd18 through bot5) - beam 1
2014-05-20 11:47:53 [DEBUG   ] board 22 xbars (bd22 to bd26 through bot1) - beam 1
2014-05-20 11:47:53 [DEBUG   ] board 26 xbars (bd26 from bd22 through bot5) - beam 1
2014-05-20 11:47:53 [DEBUG   ] board 26 xbars (bd26 to bd20 through bot1) - beam 1
2014-05-20 11:47:53 [DEBUG   ] board 20 xbars (bd20 from bd26 through bot5) - beam 1
2014-05-20 11:47:53 [DEBUG   ] board 20 xbars (bd20 to bd16 through bot1) - beam 1
2014-05-20 11:47:53 [DEBUG   ] board 16 xbars (bd16 from bd20 through bot5) - beam 1
2014-05-20 11:47:53 [DEBUG   ] board 16 xbars (bd16 to bd17 through bot1) - beam 1
2014-05-20 11:47:53 [DEBUG   ] board 17 xbars (bd17 from bd16 through bot5) - beam 1
2014-05-20 11:47:53 [DEBUG   ] board 17 xbars (bd17 to bd23 through bot1) - beam 1
2014-05-20 11:47:54 [DEBUG   ] board 23 xbars (bd23 from bd17 through bot5) - beam 1
2014-05-20 11:47:54 [DEBUG   ] board 23 xbars (bd23 to bd21 through bot1) - beam 1
2014-05-20 11:47:54 [DEBUG   ] board 21 xbars (bd21 from bd23 through bot5) - beam 1
2014-05-20 11:47:54 [DEBUG   ] board 21 xbars (bd21 to bd27 through bot1) - beam 1
2014-05-20 11:47:54 [DEBUG   ] board 27 xbars (bd27 from bd21 through bot5) - beam 1
2014-05-20 11:47:54 [DEBUG   ] board 27 xbars (bd27 to bd19 through bot1) - beam 1
2014-05-20 11:47:54 [DEBUG   ] board 19 xbars (bd19 from bd27 through bot5) - beam 1
2014-05-20 11:47:54 [DEBUG   ] board 19 xbars (bd19 to bd25 through bot1) - beam 1
2014-05-20 11:47:54 [DEBUG   ] board 25 xbars (bd25 from bd19 through bot5) - beam 1
2014-05-20 11:47:54 [DEBUG   ] board 25 xbars (bd25 to bd15 through bot1) - beam 1
2014-05-20 11:47:55 [DEBUG   ] board 15 xbars (board 15 from board 25 through bot5) - beam 1
2014-05-20 11:47:55 [INFO    ] Setting Beam 2 xbars
2014-05-20 11:47:55 [DEBUG   ] board 3 xbars (bd3 to bd7 through top1) - beam 2
2014-05-20 11:47:55 [DEBUG   ] board 7 xbars (bd7 from bd3 through top5) - beam 2
2014-05-20 11:47:55 [DEBUG   ] board 7 xbars (bd7 to bd11 through top1) - beam 2
2014-05-20 11:47:55 [DEBUG   ] board 11 xbars (bd11 from bd7 through top5) - beam 2
2014-05-20 11:47:55 [DEBUG   ] board 11 xbars (bd11 to bd13 through top1) - beam 2
2014-05-20 11:47:55 [DEBUG   ] board 13 xbars (bd13 from bd11 through top5) - beam 2
2014-05-20 11:47:55 [DEBUG   ] board 13 xbars (bd13 to bd9 through top1) - beam 2
2014-05-20 11:47:55 [DEBUG   ] board 9 xbars (bd9 from bd13 through top5) - beam 2
2014-05-20 11:47:56 [DEBUG   ] board 9 xbars (bd9 to bd5 through top1) - beam 2
2014-05-20 11:47:56 [DEBUG   ] board 5 xbars (bd5 from bd9 through top5) - beam 2
2014-05-20 11:47:56 [DEBUG   ] board 5 xbars (bd5 to bd2 through top1) - beam 2
2014-05-20 11:47:56 [DEBUG   ] board 2 xbars (bd2 from bd5 through top5) - beam 2
2014-05-20 11:47:56 [DEBUG   ] board 2 xbars (bd2 to bd4 through top1) - beam 2
2014-05-20 11:47:56 [DEBUG   ] board 4 xbars (bd4 from bd2 through top5) - beam 2
2014-05-20 11:47:56 [DEBUG   ] board 4 xbars (bd4 to bd6 through top1) - beam 2
2014-05-20 11:47:56 [DEBUG   ] board 6 xbars (bd6 from bd4 through top5) - beam 2
2014-05-20 11:47:56 [DEBUG   ] board 6 xbars (bd6 to bd8 through top1) - beam 2
2014-05-20 11:47:57 [DEBUG   ] board 8 xbars (bd8 from bd6 through top5) - beam 2
2014-05-20 11:47:57 [DEBUG   ] board 8 xbars (bd8 to bd10 through top1) - beam 2
2014-05-20 11:47:57 [DEBUG   ] board 10 xbars (bd10 from bd8 through top5) - beam 2
2014-05-20 11:47:57 [DEBUG   ] board 10 xbars (bd10 to bd12 through top1) - beam 2
2014-05-20 11:47:57 [DEBUG   ] board 12 xbars (bd12 from bd10 through top5) - beam 2
2014-05-20 11:47:57 [DEBUG   ] board 12 xbars (bd12 to bd14 through top1) - beam 2
2014-05-20 11:47:57 [DEBUG   ] board 14 xbars (bd14 from bd12 through bot5) - beam 2
2014-05-20 11:47:57 [DEBUG   ] board 28 xbars (bd28 to bd26 through bot5) - beam 2
2014-05-20 11:47:57 [DEBUG   ] board 26 xbars (bd26 from bd28 through top1) - beam 2
2014-05-20 11:47:58 [DEBUG   ] board 26 xbars (bd26 to bd24 through top5) - beam 2
2014-05-20 11:47:58 [DEBUG   ] board 24 xbars (bd24 from bd26 through top1) - beam 2
2014-05-20 11:47:58 [DEBUG   ] board 24 xbars (bd24 to bd22 through top5) - beam 2
2014-05-20 11:47:58 [DEBUG   ] board 22 xbars (bd22 from bd24 through top1) - beam 2
2014-05-20 11:47:58 [DEBUG   ] board 22 xbars (bd22 to bd20 through top5) - beam 2
2014-05-20 11:47:58 [DEBUG   ] board 20 xbars (bd20 from bd22 through top1) - beam 2
2014-05-20 11:47:58 [DEBUG   ] board 20 xbars (bd20 to bd18 through top5) - beam 2
2014-05-20 11:47:58 [DEBUG   ] board 18 xbars (bd18 from bd20 through top1) - beam 2
2014-05-20 11:47:58 [DEBUG   ] board 18 xbars (bd18 to bd16 through top5) - beam 2
2014-05-20 11:47:58 [DEBUG   ] board 16 xbars (bd16 from bd18 through top1) - beam 2
2014-05-20 11:47:59 [DEBUG   ] board 16 xbars (bd16 to bd19 through top5) - beam 2
2014-05-20 11:47:59 [DEBUG   ] board 19 xbars (bd19 from bd16 through top1) - beam 2
2014-05-20 11:47:59 [DEBUG   ] board 19 xbars (bd19 to bd23 through top5) - beam 2
2014-05-20 11:47:59 [DEBUG   ] board 23 xbars (bd23 from bd19 through top1) - beam 2
2014-05-20 11:47:59 [DEBUG   ] board 23 xbars (bd23 to bd27 through top5) - beam 2
2014-05-20 11:47:59 [DEBUG   ] board 27 xbars (bd27 from bd23 through top1) - beam 2
2014-05-20 11:47:59 [DEBUG   ] board 27 xbars (bd27 to bd25 through top5) - beam 2
2014-05-20 11:47:59 [DEBUG   ] board 25 xbars (bd25 from bd27 through top1) - beam 2
2014-05-20 11:47:59 [DEBUG   ] board 25 xbars (bd25 to bd21 through top5) - beam 2
2014-05-20 11:48:00 [DEBUG   ] board 21 xbars (bd21 from bd25 through top1) - beam 2
2014-05-20 11:48:00 [DEBUG   ] board 21 xbars (bd21 to bd17 through top5) - beam 2
2014-05-20 11:48:00 [DEBUG   ] board 17 xbars (bd17 from bd21 through top1) - beam 2
2014-05-20 11:48:00 [DEBUG   ] board 17 xbars (bd17 to bd15 through top5) - beam 2
2014-05-20 11:48:00 [DEBUG   ] board 15 xbars (bd15 from bd17 through top1) - beam 2
2014-05-20 11:48:00 [INFO    ] Setting Beam 3 xbars
2014-05-20 11:48:00 [DEBUG   ] board 25 xbars (bd25 to bd19 through bot5) - beam 3
2014-05-20 11:48:00 [DEBUG   ] board 19 xbars (bd19 from bd25 through bot1) - beam 3
2014-05-20 11:48:00 [DEBUG   ] board 19 xbars (bd19 to bd27 through bot5) - beam 3
2014-05-20 11:48:00 [DEBUG   ] board 27 xbars (bd27 from bd19 through bot1) - beam 3
2014-05-20 11:48:01 [DEBUG   ] board 27 xbars (bd27 to bd21 through bot5) - beam 3
2014-05-20 11:48:01 [DEBUG   ] board 21 xbars (bd21 from bd27 through bot1) - beam 3
2014-05-20 11:48:01 [DEBUG   ] board 21 xbars (bd21 to bd23 through bot5) - beam 3
2014-05-20 11:48:01 [DEBUG   ] board 23 xbars (bd23 from bd21 through bot1) - beam 3
2014-05-20 11:48:01 [DEBUG   ] board 23 xbars (bd23 to bd17 through bot5) - beam 3
2014-05-20 11:48:01 [DEBUG   ] board 17 xbars (bd17 from bd23 through bot1) - beam 3
2014-05-20 11:48:01 [DEBUG   ] board 17 xbars (bd17 to bd16 through bot5) - beam 3
2014-05-20 11:48:01 [DEBUG   ] board 16 xbars (bd16 from bd17 through bot1) - beam 3
2014-05-20 11:48:01 [DEBUG   ] board 16 xbars (bd16 to bd20 through bot5) - beam 3
2014-05-20 11:48:02 [DEBUG   ] board 20 xbars (bd20 from bd16 through bot1) - beam 3
2014-05-20 11:48:02 [DEBUG   ] board 20 xbars (bd20 to bd26 through bot5) - beam 3
2014-05-20 11:48:02 [DEBUG   ] board 26 xbars (bd26 from bd20 through bot1) - beam 3
2014-05-20 11:48:02 [DEBUG   ] board 26 xbars (bd26 to bd22 through bot5) - beam 3
2014-05-20 11:48:02 [DEBUG   ] board 22 xbars (bd22 from bd26 through bot1) - beam 3
2014-05-20 11:48:02 [DEBUG   ] board 22 xbars (bd22 to bd18 through bot5) - beam 3
2014-05-20 11:48:02 [DEBUG   ] board 18 xbars (bd18 from bd22 through bot1) - beam 3
2014-05-20 11:48:02 [DEBUG   ] board 18 xbars (bd18 to bd24 through bot5) - beam 3
2014-05-20 11:48:02 [DEBUG   ] board 24 xbars (bd24 from bd18 through bot1) - beam 3
2014-05-20 11:48:02 [DEBUG   ] board 24 xbars (bd24 to bd28 through bot5) - beam 3
2014-05-20 11:48:03 [DEBUG   ] board 28 xbars (bd28 from bd24 through top1) - beam 3
2014-05-20 11:48:03 [DEBUG   ] board 14 xbars (bd14 to bd10 through top1) - beam 3
2014-05-20 11:48:03 [DEBUG   ] board 10 xbars (bd10 from bd14 through bot5) - beam 3
2014-05-20 11:48:03 [DEBUG   ] board 10 xbars (bd10 to bd4 through bot1) - beam 3
2014-05-20 11:48:03 [DEBUG   ] board 4 xbars (bd4 from bd10 through bot5) - beam 3
2014-05-20 11:48:03 [DEBUG   ] board 4 xbars (bd4 to bd8 through bot1) - beam 3
2014-05-20 11:48:03 [DEBUG   ] board 8 xbars (bd8 from bd4 through bot5) - beam 3
2014-05-20 11:48:03 [DEBUG   ] board 8 xbars (bd8 to bd12 through bot1) - beam 3
2014-05-20 11:48:03 [DEBUG   ] board 12 xbars (bd12 from bd8 through bot5) - beam 3
2014-05-20 11:48:04 [DEBUG   ] board 12 xbars (bd12 to bd6 through bot1) - beam 3
2014-05-20 11:48:04 [DEBUG   ] board 6 xbars (bd6 from bd12 through bot5) - beam 3
2014-05-20 11:48:04 [DEBUG   ] board 6 xbars (bd6 to bd2 through bot1) - beam 3
2014-05-20 11:48:04 [DEBUG   ] board 2 xbars (bd2 from bd6 through bot5) - beam 3
2014-05-20 11:48:04 [DEBUG   ] board 2 xbars (bd2 to bd3 through bot1) - beam 3
2014-05-20 11:48:04 [DEBUG   ] board 3 xbars (bd3 from bd2 through bot5) - beam 3
2014-05-20 11:48:04 [DEBUG   ] board 3 xbars (bd3 to bd9 through bot1) - beam 3
2014-05-20 11:48:04 [DEBUG   ] board 9 xbars (bd9 from bd3 through bot5) - beam 3
2014-05-20 11:48:04 [DEBUG   ] board 9 xbars (bd9 to bd7 through bot1) - beam 3
2014-05-20 11:48:05 [DEBUG   ] board 7 xbars (bd7 from bd9 through bot5) - beam 3
2014-05-20 11:48:05 [DEBUG   ] board 7 xbars (bd7 to bd13 through bot1) - beam 3
2014-05-20 11:48:05 [DEBUG   ] board 13 xbars (bd13 from bd7 through bot5) - beam 3
2014-05-20 11:48:05 [DEBUG   ] board 13 xbars (bd13 to bd5 through bot1) - beam 3
2014-05-20 11:48:05 [DEBUG   ] board 5 xbars (bd5 from bd13 through bot5) - beam 3
2014-05-20 11:48:05 [DEBUG   ] board 5 xbars (bd5 to bd11 through bot1) - beam 3
2014-05-20 11:48:05 [DEBUG   ] board 11 xbars (bd11 from bd5 through bot5) - beam 3
2014-05-20 11:48:05 [DEBUG   ] board 11 xbars (bd11 to bd1 through bot1) - beam 3
2014-05-20 11:48:05 [DEBUG   ] board 1 xbars (bd1 from bd11 through bot5) - beam 3
2014-05-20 11:48:06 [INFO    ] Setting Beam 4 xbars
2014-05-20 11:48:06 [DEBUG   ] board 17 xbars (bd17 to bd21 through top1) - beam 4
2014-05-20 11:48:06 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-20 11:48:06 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-20 11:48:06 [DEBUG   ] board 21 xbars (bd21 from bd17 through top5) - beam 4
2014-05-20 11:48:06 [DEBUG   ] board 21 xbars (bd21 to bd25 through top1) - beam 4
2014-05-20 11:48:06 [DEBUG   ] board 25 xbars (bd25 from bd21 through top5) - beam 4
2014-05-20 11:48:06 [DEBUG   ] board 25 xbars (bd25 to bd27 through top1) - beam 4
2014-05-20 11:48:06 [DEBUG   ] board 27 xbars (bd27 from bd25 through top5) - beam 4
2014-05-20 11:48:06 [DEBUG   ] board 27 xbars (bd27 to bd23 through top1) - beam 4
2014-05-20 11:48:06 [DEBUG   ] board 23 xbars (bd23 from bd27 through top5) - beam 4
2014-05-20 11:48:07 [DEBUG   ] board 23 xbars (bd23 to bd19 through top1) - beam 4
2014-05-20 11:48:07 [DEBUG   ] board 19 xbars (bd19 from bd23 through top5) - beam 4
2014-05-20 11:48:07 [DEBUG   ] board 19 xbars (bd19 to bd16 through top1) - beam 4
2014-05-20 11:48:07 [DEBUG   ] board 16 xbars (bd16 from bd19 through top5) - beam 4
2014-05-20 11:48:07 [DEBUG   ] board 16 xbars (bd16 to bd18 through top1) - beam 4
2014-05-20 11:48:07 [DEBUG   ] board 18 xbars (bd18 from bd16 through top5) - beam 4
2014-05-20 11:48:07 [DEBUG   ] board 18 xbars (bd18 to bd20 through top1) - beam 4
2014-05-20 11:48:07 [DEBUG   ] board 20 xbars (bd20 from bd18 through top5) - beam 4
2014-05-20 11:48:07 [DEBUG   ] board 20 xbars (bd20 to bd22 through top1) - beam 4
2014-05-20 11:48:08 [DEBUG   ] board 22 xbars (bd22 from bd20 through top5) - beam 4
2014-05-20 11:48:08 [DEBUG   ] board 22 xbars (bd22 to bd24 through top1) - beam 4
2014-05-20 11:48:08 [DEBUG   ] board 24 xbars (bd24 from bd22 through top5) - beam 4
2014-05-20 11:48:08 [DEBUG   ] board 24 xbars (bd24 to bd26 through top1) - beam 4
2014-05-20 11:48:08 [DEBUG   ] board 26 xbars (bd26 from bd24 through top5) - beam 4
2014-05-20 11:48:08 [DEBUG   ] board 26 xbars (bd26 to bd28 through top1) - beam 4
2014-05-20 11:48:08 [DEBUG   ] board 28 xbars (bd28 from bd26 through bot5) - beam 4
2014-05-20 11:48:08 [DEBUG   ] board 14 xbars (bd14 to bd12 through bot5) - beam 4
2014-05-20 11:48:08 [DEBUG   ] board 12 xbars (bd12 from bd14 through top1) - beam 4
2014-05-20 11:48:09 [DEBUG   ] board 12 xbars (bd12 to bd10 through top5) - beam 4
2014-05-20 11:48:09 [DEBUG   ] board 10 xbars (bd10 from bd12 through top1) - beam 4
2014-05-20 11:48:09 [DEBUG   ] board 10 xbars (bd10 to bd8 through top5) - beam 4
2014-05-20 11:48:09 [DEBUG   ] board 8 xbars (bd8 from bd10 through top1) - beam 4
2014-05-20 11:48:09 [DEBUG   ] board 8 xbars (bd8 to bd6 through top5) - beam 4
2014-05-20 11:48:09 [DEBUG   ] board 6 xbars (bd6 from bd8 through top1) - beam 4
2014-05-20 11:48:09 [DEBUG   ] board 6 xbars (bd6 to bd4 through top5) - beam 4
2014-05-20 11:48:09 [DEBUG   ] board 4 xbars (bd4 from bd6 through top1) - beam 4
2014-05-20 11:48:09 [DEBUG   ] board 4 xbars (bd4 to bd2 through top5) - beam 4
2014-05-20 11:48:10 [DEBUG   ] board 2 xbars (bd2 from bd4 through top1) - beam 4
2014-05-20 11:48:10 [DEBUG   ] board 2 xbars (bd2 to bd5 through top5) - beam 4
2014-05-20 11:48:10 [DEBUG   ] board 5 xbars (bd5 from bd2 through top1) - beam 4
2014-05-20 11:48:10 [DEBUG   ] board 5 xbars (bd5 to bd9 through top5) - beam 4
2014-05-20 11:48:10 [DEBUG   ] board 9 xbars (bd9 from bd5 through top1) - beam 4
2014-05-20 11:48:10 [DEBUG   ] board 9 xbars (bd9 to bd13 through top5) - beam 4
2014-05-20 11:48:10 [DEBUG   ] board 13 xbars (bd13 from bd9 through top1) - beam 4
2014-05-20 11:48:10 [DEBUG   ] board 13 xbars (bd13 to bd11 through top5) - beam 4
2014-05-20 11:48:10 [DEBUG   ] board 11 xbars (bd11 from bd13 through top1) - beam 4
2014-05-20 11:48:10 [DEBUG   ] board 11 xbars (bd11 to bd7 through top5) - beam 4
2014-05-20 11:48:11 [DEBUG   ] board 7 xbars (bd7 from bd11 through top1) - beam 4
2014-05-20 11:48:11 [DEBUG   ] board 7 xbars (bd7 to bd3 through top5) - beam 4
2014-05-20 11:48:11 [DEBUG   ] board 3 xbars (bd3 from bd7 through top1) - beam 4
2014-05-20 11:48:11 [DEBUG   ] board 3 xbars (bd3 to bd1 through top5) - beam 4
2014-05-20 11:48:11 [DEBUG   ] board 1 xbars (bd1 from bd3 through top1) - beam 4
2014-05-20 11:48:11 [INFO    ] All xbars set
2014-05-20 11:48:11 [INFO    ] Setting adder chain routing for all beams
2014-05-20 11:48:16 [INFO    ] Setting beam ids
2014-05-20 11:48:26 [INFO    ] Starting beam delay calibration
2014-05-20 11:48:38 [INFO    ] Beam 1 latency is 5726
2014-05-20 11:48:38 [INFO    ] Beam 2 latency is 5759
2014-05-20 11:48:38 [INFO    ] Beam 3 latency is 5748
2014-05-20 11:48:38 [INFO    ] Beam 4 latency is 5715
2014-05-20 11:48:38 [INFO    ] Stop calibration
2014-05-20 11:48:46 [INFO    ] Beam delay calibration successful
2014-05-20 11:48:46 [INFO    ] Calibration passed
2014-05-20 11:48:46 [DEBUG   ] spawn_process: at 2014-05-20 11:48:46 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 4 1 192.168.0.1 192.168.0.33 10000']
2014-05-20 11:48:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x138 8909
2014-05-20 11:48:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 5 0x13c 9111
2014-05-20 11:48:46 [DEBUG   ] spawn_process: at 2014-05-20 11:48:46 running ['/usr/local/bin/lwa34', '/lwa/software/cfg_eth 2 1 192.168.0.2 192.168.0.34 10000']
2014-05-20 11:48:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x138 8a0a
2014-05-20 11:48:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa34 command /usr/local/bin/lwa34 /ape/core/xwl 1 0x13c 9212
2014-05-20 11:48:46 [DEBUG   ] spawn_process: at 2014-05-20 11:48:46 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 4 1 192.168.0.3 192.168.0.35 10000']
2014-05-20 11:48:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x138 8b0b
2014-05-20 11:48:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 5 0x13c 9313
2014-05-20 11:48:46 [DEBUG   ] spawn_process: at 2014-05-20 11:48:46 running ['/usr/local/bin/lwa16', '/lwa/software/cfg_eth 2 1 192.168.0.4 192.168.0.36 10000']
2014-05-20 11:48:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x138 8c0c
2014-05-20 11:48:46 [DEBUG   ] sendHostnameCommand - Sent hostname lwa16 command /usr/local/bin/lwa16 /ape/core/xwl 1 0x13c 9414
2014-05-20 11:48:47 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -5 msec
2014-05-20 11:48:47 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa16 at 291 ms (1400586527.296 s) with PID 4401 and time 1400586528
2014-05-20 11:48:47 [DEBUG   ] SetDRXTime: DP2 - DP/MCS NTP skew is -5 msec
2014-05-20 11:48:47 [DEBUG   ] SetDRXTime: started set_drx_time on hostname lwa34 at 381 ms (1400586527.386 s) with PID 4404 and time 1400586528
2014-05-20 11:48:48 [DEBUG   ] Finished updating ant stats in 0.864 seconds
2014-05-20 11:48:48 [INFO    ] Started antennas statistics background thread
2014-05-20 11:48:48 [INFO    ] Started board flags background thread
2014-05-20 11:48:48 [INFO    ] Started BAM background thread
2014-05-20 11:48:48 [INFO    ] Finished the INI process (ref. #198006) in 107.758 s
2014-05-20 11:48:49 [DEBUG   ] Finished updating board flags in 1.049 seconds
2014-05-20 11:49:08 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:49:28 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 11:49:48 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:49:49 [DEBUG   ] Finished updating board flags in 0.985 seconds
2014-05-20 11:50:08 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:50:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:50:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:50:49 [DEBUG   ] Finished updating board flags in 1.283 seconds
2014-05-20 11:51:08 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:51:28 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:51:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:51:50 [DEBUG   ] Finished updating board flags in 1.789 seconds
2014-05-20 11:52:00 [DEBUG   ] Got command RPT from MCS: ref #198024
2014-05-20 11:52:00 [DEBUG   ] summary = NORMAL
2014-05-20 11:52:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198024  14 56797 42720758 A NORMALNORMAL'
2014-05-20 11:52:07 [DEBUG   ] Got command RPT from MCS: ref #198025
2014-05-20 11:52:07 [DEBUG   ] summary = NORMAL
2014-05-20 11:52:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198025  14 56797 42727058 A NORMALNORMAL'
2014-05-20 11:52:07 [DEBUG   ] Got command RPT from MCS: ref #198026
2014-05-20 11:52:07 [DEBUG   ] info = System calibrated and operating normally
2014-05-20 11:52:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198026  48 56797 42727058 A NORMALSystem calibrated and operating normally'
2014-05-20 11:52:08 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:52:28 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:52:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:52:50 [DEBUG   ] Finished updating board flags in 1.547 seconds
2014-05-20 11:53:08 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:53:28 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 11:53:48 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:53:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 11:53:50 [DEBUG   ] Finished updating board flags in 1.426 seconds
2014-05-20 11:54:08 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:54:28 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:54:48 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2014-05-20 11:54:50 [DEBUG   ] Finished updating board flags in 1.367 seconds
2014-05-20 11:55:09 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 11:55:29 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 11:55:49 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 11:55:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 11:55:50 [DEBUG   ] Finished updating board flags in 1.213 seconds
2014-05-20 11:56:09 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 11:56:29 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 11:56:49 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 11:56:50 [DEBUG   ] Finished updating board flags in 1.188 seconds
2014-05-20 11:57:07 [DEBUG   ] Got command RPT from MCS: ref #198042
2014-05-20 11:57:07 [DEBUG   ] summary = NORMAL
2014-05-20 11:57:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198042  14 56797 43027050 A NORMALNORMAL'
2014-05-20 11:57:07 [DEBUG   ] Got command RPT from MCS: ref #198043
2014-05-20 11:57:07 [DEBUG   ] info = System calibrated and operating normally
2014-05-20 11:57:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198043  48 56797 43027051 A NORMALSystem calibrated and operating normally'
2014-05-20 11:57:09 [DEBUG   ] Finished updating ant stats in 0.996 seconds
2014-05-20 11:57:29 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:57:49 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 11:57:49 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 11:57:50 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 11:57:50 [DEBUG   ] Finished updating board flags in 1.021 seconds
2014-05-20 11:58:09 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:58:29 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:58:49 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 11:58:50 [WARNING ] MonitorFlags: VCCINT 0.981 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 11:58:50 [DEBUG   ] Finished updating board flags in 1.007 seconds
2014-05-20 11:59:09 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2014-05-20 11:59:29 [DEBUG   ] Finished updating ant stats in 0.400 seconds
2014-05-20 11:59:49 [DEBUG   ] Finished updating ant stats in 0.401 seconds
2014-05-20 11:59:50 [DEBUG   ] Finished updating board flags in 0.985 seconds
2014-05-20 12:00:00 [DEBUG   ] Got command TBN from MCS: ref #198059
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa27', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '1', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa24', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '2', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] tbn = exited with status True, code 0x00
2014-05-20 12:00:00 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_TBN   198059   8 56797 43200775 A NORMAL'
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa23', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '3', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa18', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '4', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa10', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '5', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa32', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '6', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa11', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '7', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa30', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '8', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa12', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '9', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa8', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '10', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa13', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '11', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa26', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '12', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa14', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '13', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa22', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '14', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa20', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '15', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa1', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '16', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa5', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '17', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa25', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '18', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa35', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '19', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa19', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '20', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa21', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '21', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa6', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '22', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa7', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '23', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa33', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '24', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa15', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '25', '1400587202']
2014-05-20 12:00:00 [DEBUG   ] spawn_process: at 2014-05-20 12:00:00 running ['/usr/local/bin/lwa28', '/lwa/software/tbn', '192.168.0.37', '10000', '37800000.0', '7', '20', '0', '26', '1400587202']
2014-05-20 12:00:00 [INFO    ] Finished spawning TBN in 0.063 s
2014-05-20 12:00:09 [DEBUG   ] Finished updating ant stats in 0.470 seconds
2014-05-20 12:00:30 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-20 12:00:50 [DEBUG   ] Finished updating ant stats in 1.253 seconds
2014-05-20 12:00:51 [DEBUG   ] Finished updating board flags in 2.017 seconds
2014-05-20 12:01:10 [DEBUG   ] Finished updating ant stats in 1.153 seconds
2014-05-20 12:01:30 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 12:01:50 [DEBUG   ] Finished updating ant stats in 0.850 seconds
2014-05-20 12:01:51 [DEBUG   ] Finished updating board flags in 2.013 seconds
2014-05-20 12:02:07 [DEBUG   ] Got command RPT from MCS: ref #198060
2014-05-20 12:02:07 [DEBUG   ] summary = NORMAL
2014-05-20 12:02:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198060  14 56797 43327050 A NORMALNORMAL'
2014-05-20 12:02:07 [DEBUG   ] Got command RPT from MCS: ref #198061
2014-05-20 12:02:07 [DEBUG   ] info = System calibrated and operating normally
2014-05-20 12:02:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198061  48 56797 43327051 A NORMALSystem calibrated and operating normally'
2014-05-20 12:02:10 [DEBUG   ] Finished updating ant stats in 0.758 seconds
2014-05-20 12:02:30 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 12:02:50 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa23, chip 5
2014-05-20 12:02:50 [DEBUG   ] Finished updating ant stats in 0.646 seconds
2014-05-20 12:02:51 [WARNING ] MonitorFlags: MGT underflow on hostname lwa7, chip 5, bottom port
2014-05-20 12:02:51 [CRITICAL] MGT underflow detected on beam(s) 1
2014-05-20 12:02:51 [DEBUG   ] Finished updating board flags in 2.007 seconds
2014-05-20 12:03:10 [DEBUG   ] Finished updating ant stats in 0.556 seconds
2014-05-20 12:03:31 [DEBUG   ] Finished updating ant stats in 1.397 seconds
2014-05-20 12:03:51 [DEBUG   ] Finished updating ant stats in 1.248 seconds
2014-05-20 12:03:51 [DEBUG   ] Finished updating board flags in 2.054 seconds
2014-05-20 12:04:11 [DEBUG   ] Finished updating ant stats in 1.156 seconds
2014-05-20 12:04:31 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 12:04:51 [DEBUG   ] Finished updating ant stats in 1.044 seconds
2014-05-20 12:04:51 [DEBUG   ] Finished updating board flags in 1.985 seconds
2014-05-20 12:05:11 [DEBUG   ] Finished updating ant stats in 0.959 seconds
2014-05-20 12:05:31 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 12:05:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 12:05:51 [DEBUG   ] Finished updating ant stats in 0.845 seconds
2014-05-20 12:05:51 [DEBUG   ] Finished updating board flags in 1.828 seconds
2014-05-20 12:06:11 [DEBUG   ] Finished updating ant stats in 0.757 seconds
2014-05-20 12:06:31 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 12:06:51 [DEBUG   ] Finished updating board flags in 1.780 seconds
2014-05-20 12:06:52 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 12:07:07 [DEBUG   ] Got command RPT from MCS: ref #198077
2014-05-20 12:07:07 [DEBUG   ] summary = ERROR
2014-05-20 12:07:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198077  13 56797 43627058 A  ERRORERROR'
2014-05-20 12:07:07 [DEBUG   ] Got command RPT from MCS: ref #198078
2014-05-20 12:07:07 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 12:07:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198078  77 56797 43627059 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 12:07:12 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 12:07:32 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 12:07:51 [DEBUG   ] Finished updating board flags in 1.613 seconds
2014-05-20 12:07:52 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 12:08:12 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 12:08:32 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 12:08:51 [DEBUG   ] Finished updating board flags in 1.577 seconds
2014-05-20 12:08:52 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 12:09:12 [DEBUG   ] Finished updating ant stats in 1.004 seconds
2014-05-20 12:09:32 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 12:09:51 [DEBUG   ] Finished updating board flags in 1.422 seconds
2014-05-20 12:09:52 [DEBUG   ] Finished updating ant stats in 1.001 seconds
2014-05-20 12:10:12 [DEBUG   ] Finished updating ant stats in 0.999 seconds
2014-05-20 12:10:32 [DEBUG   ] Finished updating ant stats in 0.796 seconds
2014-05-20 12:10:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 3
2014-05-20 12:10:51 [DEBUG   ] Finished updating board flags in 1.380 seconds
2014-05-20 12:10:52 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 12:11:12 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 12:11:32 [DEBUG   ] Finished updating ant stats in 0.599 seconds
2014-05-20 12:11:52 [DEBUG   ] Finished updating board flags in 2.232 seconds
2014-05-20 12:11:53 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 12:12:07 [DEBUG   ] Got command RPT from MCS: ref #198094
2014-05-20 12:12:07 [DEBUG   ] summary = ERROR
2014-05-20 12:12:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198094  13 56797 43927062 A  ERRORERROR'
2014-05-20 12:12:07 [DEBUG   ] Got command RPT from MCS: ref #198095
2014-05-20 12:12:07 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 12:12:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198095  77 56797 43927063 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 12:12:13 [DEBUG   ] Finished updating ant stats in 1.400 seconds
2014-05-20 12:12:33 [DEBUG   ] Finished updating ant stats in 1.203 seconds
2014-05-20 12:12:52 [DEBUG   ] Finished updating board flags in 2.168 seconds
2014-05-20 12:12:53 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 12:13:13 [DEBUG   ] Finished updating ant stats in 1.005 seconds
2014-05-20 12:13:33 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 12:13:52 [DEBUG   ] Finished updating board flags in 2.009 seconds
2014-05-20 12:13:53 [DEBUG   ] Finished updating ant stats in 0.994 seconds
2014-05-20 12:14:13 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 12:14:33 [DEBUG   ] Finished updating ant stats in 0.806 seconds
2014-05-20 12:14:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 12:14:52 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa5, chip 3
2014-05-20 12:14:52 [DEBUG   ] Finished updating board flags in 1.984 seconds
2014-05-20 12:14:53 [DEBUG   ] Finished updating ant stats in 0.800 seconds
2014-05-20 12:15:13 [DEBUG   ] Finished updating ant stats in 0.596 seconds
2014-05-20 12:15:34 [DEBUG   ] Finished updating ant stats in 1.402 seconds
2014-05-20 12:15:51 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa27, chip 5
2014-05-20 12:15:52 [DEBUG   ] Finished updating board flags in 1.790 seconds
2014-05-20 12:15:54 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 12:16:14 [DEBUG   ] Finished updating ant stats in 1.208 seconds
2014-05-20 12:16:34 [DEBUG   ] Finished updating ant stats in 1.205 seconds
2014-05-20 12:16:52 [DEBUG   ] Finished updating board flags in 1.611 seconds
2014-05-20 12:16:54 [DEBUG   ] Finished updating ant stats in 1.201 seconds
2014-05-20 12:17:07 [DEBUG   ] Got command RPT from MCS: ref #198111
2014-05-20 12:17:07 [DEBUG   ] summary = ERROR
2014-05-20 12:17:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198111  13 56797 44227062 A  ERRORERROR'
2014-05-20 12:17:07 [DEBUG   ] Got command RPT from MCS: ref #198112
2014-05-20 12:17:07 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 12:17:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198112  77 56797 44227063 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 12:17:14 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 12:17:34 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 12:17:52 [DEBUG   ] Finished updating board flags in 1.586 seconds
2014-05-20 12:17:54 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 12:18:14 [DEBUG   ] Finished updating ant stats in 0.802 seconds
2014-05-20 12:18:34 [DEBUG   ] Finished updating ant stats in 0.799 seconds
2014-05-20 12:18:52 [DEBUG   ] Finished updating board flags in 1.409 seconds
2014-05-20 12:18:54 [DEBUG   ] Finished updating ant stats in 0.604 seconds
2014-05-20 12:19:14 [DEBUG   ] Finished updating ant stats in 0.601 seconds
2014-05-20 12:19:34 [DEBUG   ] Finished updating ant stats in 0.602 seconds
2014-05-20 12:19:52 [DEBUG   ] Finished updating board flags in 1.383 seconds
2014-05-20 12:19:54 [DEBUG   ] Finished updating ant stats in 0.600 seconds
2014-05-20 12:20:15 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 12:20:35 [DEBUG   ] Finished updating ant stats in 1.401 seconds
2014-05-20 12:20:52 [DEBUG   ] Finished updating board flags in 1.368 seconds
2014-05-20 12:20:55 [DEBUG   ] Finished updating ant stats in 1.399 seconds
2014-05-20 12:21:15 [DEBUG   ] Finished updating ant stats in 1.210 seconds
2014-05-20 12:21:35 [DEBUG   ] Finished updating ant stats in 1.202 seconds
2014-05-20 12:21:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa22, chip 1
2014-05-20 12:21:53 [DEBUG   ] Finished updating board flags in 2.050 seconds
2014-05-20 12:21:55 [DEBUG   ] Finished updating ant stats in 1.200 seconds
2014-05-20 12:22:07 [DEBUG   ] Got command RPT from MCS: ref #198128
2014-05-20 12:22:07 [DEBUG   ] summary = ERROR
2014-05-20 12:22:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198128  13 56797 44527058 A  ERRORERROR'
2014-05-20 12:22:07 [DEBUG   ] Got command RPT from MCS: ref #198129
2014-05-20 12:22:07 [DEBUG   ] info = T_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected
2014-05-20 12:22:07 [DEBUG   ] mcsSend - Sent to MCS 'MCSDP_RPT   198129  77 56797 44527059 A  ERRORT_NOM1! 0x07! Beamformer calibration lost - MGT underflow(s) detected'
2014-05-20 12:22:15 [DEBUG   ] Finished updating ant stats in 1.199 seconds
2014-05-20 12:22:35 [DEBUG   ] Finished updating ant stats in 1.006 seconds
2014-05-20 12:22:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 12:22:53 [DEBUG   ] Finished updating board flags in 2.011 seconds
2014-05-20 12:22:55 [DEBUG   ] Finished updating ant stats in 1.002 seconds
2014-05-20 12:23:15 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 12:23:35 [DEBUG   ] Finished updating ant stats in 1.000 seconds
2014-05-20 12:23:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa20, chip 4
2014-05-20 12:23:53 [WARNING ] MonitorFlags: VCCINT 0.984 out of range 0.987 to 1.04 on hostname lwa28, chip 5
2014-05-20 12:23:53 [DEBUG   ] Finished updating board flags in 2.090 seconds
2014-05-20 12:23:55 [DEBUG   ] Finished updating ant stats in 0.804 seconds
2014-05-20 12:24:15 [DEBUG   ] Finished updating ant stats in 0.803 seconds
2014-05-20 12:24:35 [DEBUG   ] Finished updating ant stats in 0.801 seconds
2014-05-20 12:24:53 [DEBUG   ] Finished updating board flags in 1.901 seconds
2014-05-20 12:24:55 [DEBUG   ] Finished updating ant stats in 0.800 seconds
